More selected projects

React Glow - An Interactive Dance Performance

Gitlab Repo

by: Terry Clark

Project Description

React Glow is an Interactive dance performance which utilises the xBox Kinect, tracking Digital Multiplex (DMX) lighting, Open Sound Control (OSC) communicating with Ableton Live and audience participation through Twitter.

The aim for this project was to experiment with three mediums namely, projection, interaction and performance. To achieve this, I created a toolset for a full 5-6 minute dance performance, whereby the dancer interacts with the audio and visual elements of the piece.

Gustaf Svenungsson, creative partner on my last year’s project, wrote and helped develop the main soundtrack of the performance and the dancer is still to be concluded. To further clarify, this project is demonstrating the components of the performance and will be performed and recorded after the project deadline of 12th May 2017, an explanation of this outcome is given in the evaluation below.

Target Audience

This piece is an experience that the audience can have through a  performance they can both enjoy and interact with. The intended audience would be both participator and observer, which anyone with a twitter account can take part in by tweeting with a specific hashtag ‘#ReactGlow’. More than one tweet can be made and the piece will store which orbs are the individual audience members along with their username in order to display the end data visualisation. The more the audience interacts with the piece the more the interactions are accentuated by the flocking and the more brighter the outcome of the final performance scene.

People not participating would be able to enjoy the different interactions with both audio and visual elements along with the choreography created by the dancer. Once the performance is complete the audience will see their username and collection of glowing orbs displayed, creating a kind of data visualisation which I want to use to further connect the audience member with the piece.

Creative and Technical Process

The main aim of this project was to explore more deeply technology within performance. I began this project with a processing sketch over last Summer of some glowing orbs with sinusoidal movement where the points eased between the different predefine shape paths, for example circles, an infinity symbol, and some experimental shapes by mixing sines and cosines together. However, this sketch used a simple particle system to add new orbs and lacked the dynamic flowing elements I needed to emphasise the dancer’s movements.

I decided to look into other particle system structures and integrated a vector field based with further boid style behaviours such as flocking to create a more organic and realistic effect whilst the boids were not being interacted with I believe this makes them seem like floating light critters in digital space and personifies an online society being affected by the physical being.

The piece takes shape by the increasing number of orbs, caused by the audience tweets, that appear as they flow around the projected space through a vector field. This is also how I am able to capture the dancer’s flowed motion by seeing the particles sweep in their kinect path. As just briely mentioned, the narrative follows a series of projected orbs created by the audience through Twitter responses whereby the colour of the dot is the average colour of their twitter avatar. As the dancer enters the stage there is an interplay between them of which revolves around three themes, repelling, control and attraction, reminiscent of general boid and flocking behaviour.

Whilst working on Kinectica’s 10 year anniversary exhibit I was able to work along-side Paul Firedlander, light wizard and Kimatica, performance and body projection mapping artists creating a performance using flashing lights and holographic, non-polarising scrim, a type of netting material used to project onto. The interesting effect here was that the dancer was only visible to the audience when there was a lot of light on them, as if to merge the physical with the digital. I found this blend to be visually stunning and incorporated this within my project.

As mentioned, the interaction for the piece is mainly affected by the dancer’s movements, I used the Kinect 3D camera along with ofxKinectForWindows2, an openFramework’s library written by Elliott Woods, to capture the skeleton and pointcloud information. By using the skeleton tracking the dancer could then control parameters inside Ableton and manipulate certain instrument effects along with affecting the vector field system. The pointcloud is used towards the end where the orbs would pick a random point of the pointcloud and attach themselves to it, creating the sense of attraction.

The relationship between the dancer and the orbs is important to note as  after the critters have been hypnotised by the dancers movements and interactions the visual effect would be of a big glowing entity.

As the dancer will constantly be behind the scrim I believe that this will feel as though they are within the same space as the orbs. The three different interactions and repelling, control and attraction further solidifies this link to the audience. As described above, at the end of the piece, the attraction scene felt to be a really good place to work with blending the dancer with a big glowing entity. This would make it difficult to see the dancer however I wanted to use this creatively but fading up and down lights which point and track them. I therefore needed to bring lighting into this piece and decided to try creating a tracking system with two lights and the middle point of the dancer. By using DMX I am able to access tilt and pan of the lighting system and with some simple trigonometry was able to implement this.

I found that once the orbs crossed paths the shader merged them together similar to how light works in reality. I thought that this would really work well as a data visualisation where by the orbs are all lined up on top of each other depending on the number of tweets per person and the audience twitter name is underneath thanking the audience for participating and further creating the bond between the audience and the performance.

Evaluation

Unfortunately, I was unable to fully complete this project in time for the deadline. However,  I will finish and record this at a later date. The reason for this were mainly due to some lack in project management and how long some components took to create. For instance having a better understanding of the graphic pipeline, 3D and different spaces.

This project demonstrates the toolset of this performance and I beleieve I managed to incorporate features that were coherent with my visual ideas and themes of repelling from the dancer, following the dancers path and swarming over the dancer.

The complexitiy of the project was also a factor as there are lots of classes being used together and passing from one another. I needed to create a map of this to better understand the flow of data.

I had a dancer lined up to practice with the piece but it was difficult, by the time I have a functioning program, to aquire the space I needed to setup the system and practice.

Also I had very specific Ideas about what I wanted and I believe this hindered my exploration of the project allowing for flexibility in the ideas.