Instrument Designer: David Birnbaum
Composer: Joseph Malloch
Performers: Kristie Ibrahim, Julieanne Klein
Commotion and Contingence are two musical instruments based on gloves and a head-mounted camera that take advantage of the extraordinary muscular precision of the mouth and hands. A piece composed for the them was performed at several concerts in Montreal, including the Jusqu’aux Oreilles contemporary music festival.
Here’s an interview I did about this project on the CBC One radio show “Home Run.”
Download (mp3, 4.4 MB)
The following overview is derived from the project’s documentation.
Commotion is a drum sequencing instrument. Three drum tracks are layered atop one another—kicks, mid-range drums (such as snares and toms) and hi-hats. Flex sensors along the fingers of the left glove determine the speed of a linear signal that plays the samples back, either at 0 Hz (equivalent to turning off the drum track), 1 Hz, 3 Hz, 6 Hz, 9 Hz, 12 Hz, or 15 Hz. An accelerometer on the right hand determines the starting and ending points in a 64 step pattern according to hand tilt. Values for both the starting and ending points of the pattern are snapped to a multiple of four in order to preserve a groove.
In addition to the gloves, a small head-mounted video camera captures an image of the performer’s mouth to control a delay effect on the drums. Height of the mouth is mapped to feedback amount, and area of the mouth is mapped to delay resolution. The combination of multiples of three and four in the drum pattern and the delay effect makes for quite a racket.
Contingence is a five-operator FM synth. The harmonic ratios are preprogrammed, but remain constant as the frequency of the carrier changes. Pressure sensors on the tips of the fingers determine the amplitudes of each of the operators; the thumb affects the carrier signal, and the other fingers control the amount of modulation of that signal. Sound is thus produced when the player touches. Flex sensors along the thumb and forefinger of the right hand determine the frequency of the carrier—the thumb controls a range of six octaves, and the index finger controls pitch along a pentatonic scale. The decision to use a pentatonic scale was necessary because it was found that twelve semitones per octave was too fine a pitch resolution to be controlled with finger bend.
Alignment, composed by Joseph Malloch, was the only musical piece written for these instruments. The notation of the part for Contingence was mostly conventional, with some graphic elements. Pitch and rhythm were notated precisely, and timbre more generally. The addition of modulation to the sound was usually indicated with written words (e.g., “pure sound,” “full modulation”), much as timbral change is indicated with more standard instruments (e.g., “pesante,” “sul tasto”). Standard notation practices were used for pitch, amplitude, and rhythm. Graphic notation was used at rehearsal marks D and H in the second section, and it was found (perhaps not surprisingly for an instrument of this type) that the performer’s interpretation of the drawn gestures was much more physical than in the precisely notated sections.
For the part of Commotion, graphic notation was used. The first section added fairly common graphic shapes to what was essentially extended conventional notation. Rhythms were notated precisely in order to present strong performance goals but nevertheless imply that playback is approximate. Elements of conventional notation were used whenever possible (e.g., staves, fermatas, rests).
The piece is divided into two sections, each using a different approach. The first section attempts to attain maximum compositional control of the sound by changing focus rather than requiring high precision. The movement is short, but slow in pace, consisting of only four large sound-shapes with focus on slow timbral shifts in the sound. The second section is faster, and requires gestures of much finer detail. The section is divided in turn into eight shorter subsections, or phrases, and a coda, allowing the performers to align the two parts correctly even though one part is written on a graph and cannot be read linearly.
With Contingence, the pentatonic scale made pitch accuracy and consistency achievable. The relationship between sound and hand position turned out to be a very natural one. The mapping for the right hand also worked well. The performer had to learn to cope with sensor drift each time she played the piece, but the more she played, the easier it became to adjust to the new pitch positions. Developing finite muscle control took a lot of practice, particularly for the thumb, where an octave leap, which is a large gesture on many instruments such as piano and marimba, was only a few millimeters wide. On the left hand, the thumb control of amplitude worked well, as it could be interpreted as an anchor for the modulators. With practice, modulation was manipulated with precision. One of the most important decisions was how to activate the sensors—what to touch in order to play. The decision to play on the body was made in order to avoid the need for any external components such as a music stand or table. The type of pressure sensors installed in the glove can fail when bent, necessitating pressing the finger tips without bending the fingers. Some muscle strain at the base of the thumb resulted from this.
For Commotion, two primary challenges were found: the initial individual hand coordination and the muscle strength required to maintain a stable rhythmic loop. A tremendous amount of arm tension and concentration were required in order to keep the synthesizer in a constant state. Initially it was difficult to find musically-pleasing settings for the mouth delay effect, but this improved with practice. As more skill was developed, it was possible to control the delay with tiny gestures, allowing for a wide range of musical expression.