Understanding musical interaction

Part of my graduate work was concerned with the characterization and comparison of digital music interfaces. This is a difficult problem because many people no longer agree on the definition of “musical instrument,” or even “music”!

A new conceptual framework for digital musical instruments

Human-machine interaction theorist Jens Rasmussen based his theory of interaction partly on semiotics. When applied to digital musical instruments, this approach can help us classify interfaces and propose design criteria without relying on measures of expressivity (how musically expressive the interface allows you to be), or potential for virtuosity (the ability of an interface to be played by a virtuoso).  Instead, “levels of control” of human actions are categorized as knowledge-based, rule-based, or skill-based direct object manipulation.

This framework attempts to clarify some of the issues surrounding music interface design. It can be used to analyze and compare existing instruments, guide the design of new ones, or see how instruments could be used in new ways. For details, please read Towards a new conceptual framework for digital musical instruments.

 

Dimension spaces for interface comparison

This paper addresses the question, how can we distinguish a musical instrument from a musical toy from an interactive sound installation? A multi-axis plot called a dimension space can be used to visualize these kinds of problems. We developed our own specialized dimension space, based on a radar plot, for music interfaces.

When music interfaces are visualized in this way, trends become apparent in the shapes of the resulting plots. Instrument-like, toy-like, and installation-like interfaces can now be easily distinguished at a glance.

Example plots using the seven-axis dimension space. Clockwise from top left: Hyper-flute, The Hands, Maybe…1910, Block Jam. Note that Maybe…1910, which is an interactive museum installation, looks radically different from The Hands, which is a stage performance instrument.

For more information, you can read Towards a dimension space for musical devices.

← Back to Projects