4th International Sound and Music Computing Summer School
SMC08
I attended the SMC Summer School in Genoa, Italy from the 9th-11th June 2008.
It was held at Casa Paganini in the heart of the old town where the InfoMus Lab is based.
All students gave a speed talk about their work and research.
The days were split into seminars and lectures for the first half of the day, and then a workshop in the second half. I took part in workshop 3: 'Developing an EyesWeb module for analysis of expressive content in human full-body movement and gesture.' This workshop was lead by Alexander Jensenius (University of Oslo). I thought he looked familiar and then it clicked that I'd performed at the same concert as him in Oslo a few years ago.
I found there was not enough time for the workshops in the afternoon to make anything significant. The workshop failed to cover 'mapping the extracted measures' from movements and gestures 'onto real-time generation of sound', as was stated in the handout. The object being developed is called a Motiongram which functions on the same idea as a Sonogram, just that it is used for analysing motion of a body instead of spectral content of a sound. I began to look at what values were being measured and then sending them via OSC to PD to begin to map them to sounds and music as I wanted to see if I could do it because the leaders of the workshop weren't covering it. This is when the ammount of time given to the workshops became apparent as it soon ran out and I had not really made much. However, I did make some nice pictures with it!
The morning sessions of seminars and lectures were really informative and interesting. These are my notes (if you can make any sense out of them!):
Notes from SMC Summer School, Genova, 2008.
09|06|2008
Expression and Emotion in Non-verbal Full-body Gesture
Antonio Camurri and Gualitiero Volpe (University of Genova)
Casa Paganini – InfoMus Lab
DIST – University of Genova, Genova, Italy
Sound Cage, 1994 – dancer moves around in a ‘cage’. Sensors detect movement and trigger sounds.
Laban theories – bio mechanics, etc.
Disappearing Dancer-
Research on non-verbal expressive gesture:
Music performance – Schaeffers’ Morphology
Dance – Labans’ Effort Theory
Non-verbal multimodal interaction:
Emotion and Music
Importance of body movement (Woldort, Scherbart???) : emotion expression and communication.
Verify whether expressions can be distinguished based solely on motor behaviour.
http://emotionresearch.net
“Automated Analysis of Body Movement ….” 2008.
From recognising emotions to understanding the empathic relations, the entrainment, the synchronisation between humans in emotional tasks.
Experiments on motoric response of subjects exposed to music performances.
Synchronisation – “Adjustment of rhythms of oscillating objects due to their weak interaction”. [Pikovsky et al., 2001]
motions and gestures – video tracking
physiological data – emg, accel, plantar pressure
///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
10|06|2008
Embodied Music Cognition
Mark Leman
IPEM
Ghent University, Ghent, Belgium
The human body is the natural mediator between mind and environment
Extend the body so the mind can enter into a new environment – enhanced reality
Instrument becomes part of the ‘body image’
Embodiment = understanding the human body in relation to music
Mediation technology = make technology which extends body
The body in music is not a new idea
*Aristotle – mimesis theory, Politics, Book VIII, Ch5.
*Descartes – 2 substances (material and spiritual)
connecting mind and body
*Baruch de Spinoza – there is only one substance (God : infinite)
don’t have to assume two different substances
mind is embodied
*Hume (sensitivity)
*Kant (sensus communis)
*Lipps and Merleau-Ponty – embodiment in phenomenology
*Manfred Clynes – 1977, ‘Sentics, the touch of emotions’. NY: Doubleday.
What is new?
Subject is an ‘active explorer’ of the world, rather than passive receptor. Aiming as much as possible to keep them active and involved.
Subject has action-oriented ontology (body schema/image)
Research topics
Typology of musical gestures
Body mirrors metrical layers in the music (dance)
Group synchronisation – optimal experience – social interaction
Resonance with people surrounding you – synchronisation
Sonic and visual cues exchanged between musicians and audience
In social condition individuals synchronise, can form coalitions and groups within the main group
Project EmcoMetecca
Embodiment is a major issue:
///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
11|06|08
Sonic Interaction Design and Multimodal Perception
Stefania Serafin
Medialogy
Aalborg University, Copenhagen, Denmark
Multimodal – many senses
Interaction amongst the senses. Vision and audition
Why multimodal perception? – our perceptual systems have evolved to encompass everything
Humans by nature are multi-sensorial
Audio-visual interactions – a way to understand and explain what we perceive – intersensory biases
Intersensory biases:
Illusionary flashes:
Two beeps and one flash – you perceive two flashes
Visual is predominant in spatial processing.
Audition is predominant in temporal processing.
Audio-tactile interaction:
Zampini and Spence, Journal of sensory studies, 2004. – same as Parchment skin illusion but with eating crisps.
Pseudo-haptics
Artificial sensibiltiy
Audio-haptic simulation
---------------------
The teachers and scientific committee were:
Nicola Bernardini (Conservatory of Padova), Antonio Camurri (University of Genoa), Roberto Doati (Conservatory of Genoa), Federico Avanzini (University of Padova), Giovanni De Poli (University of Padove), Goffredo Haus (University of Milan), Alexander Jensenius (University of Oslo), Ben Knapp (University of Belfast), Marc Leman (Ghent University), Barbara Mazzarino (University of Genoa), Augusto Sarti (Politecnico di Milano), Uwe Seifert (University of Koln), Stefania Serafin (Aalborg University Copenhagen), Xavier Serra (Pompeu Fabra University), and Gualtiero Volpe (University Genoa).