Extracting musically-relevant rhythmic information from dance movement by applying pitch tracking techniques to a video signal

In this paper, I discuss in detail the approach taken in the implementation of two external objects in Max/MSP [11] that can extract musically relevant rhythmic information from dance movement as captured by a video camera. These objects perform certain types of analysis on the digitized video stream and can enable dancers to generate musical rhythmic structures and/or to control the musical tempo of an electronicallygenerated sequence in real time. One of the objects, m.bandit,¹ implements an algorithm that does the spectral representation of the frame-differencing video analysis signal and calculates its fundamental frequency in a fashion akin to pitch tracking. The fundamental frequency of the signal as calculated by this object, can be treated as a beat candidate and sent to another object, m.clock, an adaptive clock that can adjust the tempo of a musical sequence being played, thereby enabling the dancer the control of musical tempo in real time.