Modeling Improvisation as a Dynamical System

Improvisation is often described as spontaneous creation, yet experienced improvisers consistently report that the process feels more like navigation: moving through a well-internalized space of musical possibilities under real-time constraints of tempo, harmony, and interaction. This tension between spontaneity and structure motivates a dynamical systems perspective. If improvisation is navigation through a learned musical space, then the tools developed to characterize the geometry of dynamical system trajectories should be applicable to musical time series.

Recurrence Quantification Analysis (RQA) was developed by Eckmann, Kamphorst, and Ruelle (1987) as a method to characterize the recurrence structure of dynamical systems from time-series data alone, without requiring a known model. The central idea is that a deterministic dynamical system will repeatedly revisit nearby regions of its state space, producing characteristic diagonal line structures in a recurrence plot (a binary matrix whose entry (i,j) is 1 if the states at times i and j are similar). Applied to musical pitch sequences, RQA quantifies the degree to which an improvisation revisits similar states, corresponding to musical self-reference, motivic development, and phrase-level coherence.

The hypothesis motivating this analysis is that expert improvisers, who have internalized a musical grammar that produces coherent phrase structures, should show higher RQA determinism than less experienced players, and that this quantitative measure should track qualitative musical structure assessable by expert annotation. This would constitute evidence that dynamical systems analysis is capturing something musically meaningful, not just statistical regularity.

We apply RQA to MIDI-encoded jazz piano solos from the Weimar Jazz Database (Jazzomat Research Project), which contains expert-transcribed solos from canonical recordings with precise timing, pitch, and onset information verified by hand. The state vector at time t is constructed using delay embedding: the pitch value at t and its three preceding values (embedding dimension 4, delay 1 beat), following the embedding theorem of Takens (1981), which guarantees that the reconstructed state space preserves the topological structure of the original dynamical system under mild conditions.

The distance threshold for the recurrence matrix is chosen as 10% of the maximum phase-space distance across all pairs of state vectors, which is a common heuristic that ensures the recurrence rate remains in a computationally informative range. From the recurrence plot, we compute four standard RQA statistics: recurrence rate (RR, the overall fraction of recurrent states), determinism (DET, the fraction of recurrent points forming diagonal lines, indicating predictable sequential structure), average diagonal line length (L, the average length of diagonal structures), and entropy of the diagonal line length distribution (ENT, capturing the complexity of the recurrence structure).

These statistics are computed both globally across entire solos and in sliding windows of 32 beats to capture temporal variation. We compare across 50 solos spanning different experience levels and musical styles (bebop, modal, post-bop) from the Weimar database, and align windowed RQA statistics with expert-annotated structural boundaries.