Marco GilliesDepartment of Computing, Goldsmiths, University of London

A new video from Bruno Zamborlin of the EAVI group

The MIT press have now published online the audio-visual materials from the Performing Presence project:

http://www.mitpressjournals.org/doi/suppl/10.1162/DRAM_a_00124/suppl_file/dram_a_00124.suppl.html

Performing Presence: from the live to the simulated was an AHRC project run by Nick Kaye at the University of Exeter that explores the concept of presence across multiple disciplines ranging from theatre and performance to virtual reality.

I supplied character animation software and was involved in the development of the virtual reality elements of the project.

If you are interested in reading more there is an article by Nick Kaye and Gabriella Giannachi about the virtual reality elements

http://www.mitpressjournals.org/toc/dram/55/4

There is also a book published documenting the project as a whole:

http://www.amazon.co.uk/Performing-Presence-Simulated-Practice-Performance/dp/0719080045

This is new work from Bruno Zamborlin of the EAVI group:

Mogees is an interactive gestural-based surface for realtime audio mosaicing.

When the performer touches the surface, Mogees analyses the incoming audio signal and continuously looks for its closest segment within the sound database. These segments are played one after the other over time: this technique is called concatenative synthesis. For instance, loaded a series of voice samples, a graze in the surface could corresponds to a whispering while a scratch would trigger more shouted sounds.

The wooden surface can be “played” with any tool such as hands and Mogees will always try to find a correspondent sound to it. It can also be applied to other sound sources such as voice or acoustic/electric instruments.

Mooges has been developed in collaboration with Norbert Schnell and takes full advantage of the MuBu environment for MaxMSP. It is currently used in the Airplay project by the IRCAM composer Lorenzo Pagliei.

Mogees has been exposed at the Beam festival at Brunel University in London on the 24/25/26 of June 2011.

Participate in an experiment with

MOTION CAPTURE!

EARN MONEY and PLAY and DESIGN motion capture GAMES!!

We are looking for participants for a FUN and EXCITING experiment to examine how people interact with a user interface to design aspects of a motion capture based pong game.

The experiment involves tasks that get you to play a game, have your body motions recorded and test out a user interface.

Earn £25 for 2.5 hours

Participants needed:

3 August and 15 August – 2 September

Times flexible – please contact either
a.kleinsmith@gold.ac.uk or m.gillies@gold.ac.uk to arrange

Today I will be giving a masterclass at Haberdashers’ Aske Hatcham College about computing.

I want to stress how computers are no longer just a business tool:

They are now a medium, and the dominant medium of the 21st century.

I will explore a bunch of themes via links.

Interactivity

Exemplified by the kinect:

http://www.youtube.com/watch?v=m2TB5YOKDyI&feature=relmfu

http://www.youtube.com/watch?v=ho8KVOe_y08

Democratization

For example Kite Mapping:

http://vimeo.com/11734964

or simply youtube:

http://www.youtube.com/watch?v=_OBlgSz8sSM

Many to many communication

Of course, Facebook:

http://www.facebook.com/

But also more specialist sites like SoundCloud

http://soundcloud.com/meltusriddler/james-blake-i-never-learnt-to

Unlimited Manipulation

Most obviously in film post-production

http://www.youtube.com/user/HarryPotter?blend=1&ob=5

http://www.youtube.com/watch?v=eB2gyXWqNZc

Universal Machine

From the theory of Turing:

http://en.wikipedia.org/wiki/Alan_Turing

To the practicality of devices that increasingly seem to do everything:

http://www.apple.com/uk/iphone/features/

A few weeks ago we ran a motion capture workshop for 13-18 year olds at the British Museum Samsung Digital Discovery Centre.

Participants could animate characters from the Museum’s collection using our MoCap suite.

Here are some photos of the event

http://www.flickr.com/photos/britishmuseum_samsungcentre/sets/72157626091895909/

One of the participants has written a post about it here:

http://www.gunsandgrapple.com/2011/04/article-motion-capture-session-at.html

I have just finished reading The Cybernetic Brain an interesting history and philosophy of a small group of British cybernetics researchers.

The history is very interesting but in many ways the key point is the philosophy. Pickering claims that cybernetics is distinctive because it does not accept what he calls (follow Bruno Latour) a modern ontology, which is characterised by:

• Dualism between people and things. This is subtly different from the Cartesian dualism of mind and matter, though closely related. The dualism does not necessarily imply an acceptance of Cartesian dualism at a metaphysical level, but does imply a practical division of what and how we study: things in physics, chemistry, etc and people in social science and humanities. This seems to be an internalisation of cartesian dualism, even by those who do not accept it in theory.
• Representation and deliberation The purpose of science and study in general is to create representations of the world and to think about them
• Knowability In principle the world can be known in all its detail, we just need to put in more work to get more knowledge.

I’m a bit unconvinced by the use of the term “modern” in this case. As Pickering acknowledges it has too many resonances. In many cases I found myself naturally reading the term in ways that was not what was implied in its given use. Firstly the “modern” ontology has been challenged in many areas of modernism, primarily the arts (Pollock, Boulez and Joyce spring to mind) but also to some degree in science (some philosophies of quantum mechanics). So a reading equating “modern” ontology with modernism feels a bit wrong. Another natural reading when comparing work done 50 years ago with “modern” ways of thinking is to read “modern” as meaning “now”. I did that a lot without thinking, but as I will note below, that also seems very wrong. To save coining another term (and incompatibility with Pickering’s text) I will put it in quotes.

Pickering contrasts this “modern” ontology with a cybernetic one characterised by:

• Monism. people and things are not distinct. Its less clear from the book what type of monism this is. To me the implication is of a materialist ontology in which mind is an emergent property of matter. I think this covers a lot of the work, but there are also some spiritual dimensions to this (e.g. in Stafford Beer’s ideas) that I’m not sure how to place
• Non-representational and Performative. We do not represent and think about the world but figure out ways of acting in the world.
• Unknowability The world is fundamentally unknowable, primarily due to its create complexity.

I think that Pickering has identified one of the most interesting and enduring innovations of this early work in cybernetics. He himself sees this ontology as highly marginal, but I in fact I see it as a very common one in the areas of current AI, neuroscience and psychology that I interact with. In fact, my colleague Mark Bishop teaches a whole masters programme around this very philosophy (though he was trained as a cybernetician so it does all make sense).

The revival of this “cybernetic ontology” stems in large part from Rodney Brooks’ critique of the AI of the time (though it has many other streams, in psychology and neuroscience for instance). Good old fashioned AI was in many ways dualist (representation and thing itself were different); certainly was representation and deliberative and did assume that the world was knowable. Brooks contrasted this with an AI in which the world was unknowable but it was possible to construct material artefacts that could perform in the world without representation.

However, while Brooks’ technique was highly influential, his own suggested techniques never quite scaled up or became ubiquitous. So can we say that modern AI still uses a cybernetic ontology? I will take statistical and probabilistic and statistical methods as an example. One of the great recent successes in AI has been the development of statistical and probabilistic machine learning methods, what ontology do they represent. Probability is very clearly a principled mathematical method for dealing with an ultimately unknowable world, we can know it up to a certain degree of probability but no further. So these methods seem, at least, compatible with an onotology of unknowability. What about representation and performance. Many techniques are aimed at directly performing a task (e.g. classification) without a clearly understandable internal representation (e.g. Support Vector Machines). Many other methods are more hybrid, for example, Bayesian Networks do include clear representations and their purpose is to infer probability representations rather than act per se. Also feature extraction can be viewed as a way of creating representations from data (though automated feature extraction is often not humanly understandable. So the case isn’t quite clear, representation is mixed with non-representation and some deliberation is mixed with a lot of performance. What about dualism vs materialism? A lot of work in machine learning is closely linked with contemporary neuroscience (e.g. the Gatsby Unit), which does have a fundamentally materialist outlook in which mind emerges as a property of the interaction of matter. So we can say that the cybernetic ontology is alive an well, and in some ways dominant in certain domains, though still interacting with other philosophies and Pickering suggests it should in his last chapter.

Today I will be presenting my work to applicants to our undergraduate programmes in computing and music computing. For those who are interested here is a link to my slides:

Applicant Talk

And here is some student work that I will be showing:

Student Work

This has been what has getting us excited this month. The kinect promises to provide embodied interaction at a low cost without requiring you to wear anything. The story of how quickly the protocol got hacked was pretty amazing (3 hours after the launch) and resulted of a rapid release of drivers:

http://openkinect.org/

PrimeSense, who developed their original technology, have also release an opensource frameworks, and a gratis library that extracts a skeleton (very exciting), though it doesn’t work on a mac (boo!).

http://www.openni.org/

Hopefully I will have a bit of time to play with it all in the new year and get it working for research.

I’m very please to announce our new research group at Goldsmiths that I helped found together with Mick Grierson, the Embodied Audio-Visual Interaction research group:

http://www.doc.gold.ac.uk/eavi/EAVI/