Physical Interactions With Sound 

by: Edmund Oetgen and Akhil Nair

Introduction 

With our project we are aiming to provide users with an intuitive, fun way to experiment with audio samples and granular synthesis. As these audio concepts can often seem abstract and intimidating to a beginner, our goal is to provide a way for them to physically interact with the the basic principles and allow them to build their own parallels to the tangible world.

In reflection of these goals we chose to focus our research on products and companies who have a similar focus on playful experimentation in sound, as well as artists who have explored some more avant garde forms of musical interaction.

Ototo

Ototo is a synthesizer that allows users to connect notes and parameters to conductive materials. This makes for the rapid implementation of a wide variety of unconventional interfaces for people to interact with sound that can be used anywhere from performance to sound installation to product design.

In this video, Ototo has been configured to trigger patterns through cups of water. Furthermore, parameters from sounds are being modified using various sensors connected to Ototo. In this context, Ototo has been used to create a non-standard midi controller that could be used as part of a performance. The most common styles of midi controllers are keyboards or pads, with the former geared to recording, and the latter for performance. Ototo provides the functionality to implement both of these workflows, but also allows for the opposite, where an interface can be created with the sound as the starting point.

in this video, Ototo is used as a capacitive sensor to detect the level of liquid in a glass using capacitive sensing. In this use, Ototo has been implemented in an interface usually unrelated to sound. This emphasises that Ototo can be easily implemented in virtually any object to trigger and manipulate sound.

Mogees

Mogees are vibration sensors that can be attached to solid objects . The frequencies created when the object is struck in different ways can be mapped to discrete midi data, and can then be used to trigger samples or notes. This allows the spontaneous nature of drumming on an object to be translated to playing synthesizers or samples, and allows for users to take advantage of the unique sound properties of any object.

This video demonstrates the mogees keys app, where mogees is used to play notes on a synth. here, various vibrations created by striking the object have been mapped to discrete notes. Allowing any surface to become a musical instrument adds a sense of playfulness that is less intimidating to beginners, and requires less investment to people who are interested in exploring music and synthesis.

This video shows mogees being used as part of a performance, where dancers interact with objects that trigger sounds. The inherent rhythm in dance is often created to complement a preconceived piece of music, however the use of dance as the starting point for the music provides interesting improvisational possibilities and turns the act of playing an instrument into a performance in itself.

Reactable

Reactable is physical audio workstation that represents modules as blocks that can be placed on a multi touch enabled table. Each brick is assigned either a synth, sample or effect and its placement on the screen adjusts the assigned module’s parameters and playback. For example, a brick of drum loops is placed on a board and it will begin to play, a line is drawn from the brick to the centre of the screen, indicating it is playing. The user could then place a reverb brick in this line and the effect would be added, rotating the brick would adjust the dry/wet mix. Below is a video showing how Reactable can be used to produce a trance track.

Adachi Tomomi

Adachi Tomomi is a multimedia artist and musician known for his unconventional music performances using self made instruments. As a sound poet, his performances often center around an instrument that transforms his voice using various sensors that track data such as movement. His use of his own body’s movements and unconventional everyday objects as inputs for control data to manipulate sound.

in this performance, Tomomi distorts his voice through motion sensors in his hands, arms and on his torso. The use of his body as a tool to control his vocals complements the improvisational nature of his work.

Laetiti Sonami

Laetitia Sonami is an artist whose  work explores new and unusual ways of controlling and generating sound with various homemade instruments. With her ‘Lady’s Glove’, a dress glove fitted with a variety of distance sensors, magnets and switches, Sonami created an intuitive new way to manipulate audio through their movements. Initially a joke project, the glove has evolved into a fully fledged instrument in which it’s play style is as much performance as it is musicianship. Below is a video of Sonami discussing and demoing the glove.

Conclusion

With projects such as Ototo and Mogees, interfaces can be created that allow users inexperienced in music to create sound and music without the overwhelming and intimidating input of a traditional musical instrument. Reactable is a more technical workflow, but still allows an element of fun that can be appealing to non musicians.

Laetitia Sonami’s glove links music to movements and gestures that are natural to most people, and Adachi Tomomi’s self-made instruments either use the body, or common objects that are familiar to people, such as cutlery. Both of these artists present some interesting ideas in how to remove the technical skill  and harsh learning curve of playing an instrument, and also incorporating an element of performance.

These projects and artists each have an unconventional approach to interacting with sound and music, replacing the skill or knowledge required to play an instrument or program synthesizers with methods more natural to a wider range of people. As well as this, many of the projects mentioned have an element of play to make the prospect of creating sound or music either less intimidating, or more spontaneous. We want to incorporate these ideas of working with sound into our project, and create a way for users to create and effect sound in ways that are easy to pick up and use for anyone.

References

• Ototo http://www.ototo.fm
• Mogees http://www.mogees.co.uk
• Reactable http://reactable.com
• Adachi Tomomi http://www.adachitomomi.com/n/biography.html
• Laetitia Sonami http://sonami.net/ladys-glove/