SoundLab – a pioneering project to help people with learning disabilities express themselves musically – has been named Best Special Educational Needs Resource at the annual Music Teacher Awards for Excellence.
SoundLab is a collaboration between the EAVI group in the Goldsmiths Computing, award-winning creative arts company Heart n Soul, and Public Domain Corporation, a company providing interactive experiences and technology for the games and digital arts sectors.
The Music Teacher Awards for Excellence 2016 took place on Thursday 25 February, attended by some 280 industry guests including teachers, hub leaders, musicians and VIPs – representing the best and brightest in performing arts education.
Shortlisted alongside Soundlab in one of thirteen award categories were projects such as Moog Theremini, the Skoog, and the Alphasphere – chosen as outstanding resources for the education or music therapy sectors that combine current research with practical application to allow students with special educational needs and disabilities to engage with music.
From music apps that let you compose, DJ or play countless instruments with a fingertip, to those that make a voice sound amazing even if it’s not quite in tune, the SoundLab researchers have rigorously tested iPhone/Pad, Android and web programmes that can help people with disabilities make the music that they want to make.
Dr Mick Grierson, Reader in the Department of Computing at Goldsmiths, is lead Soundlab researcher, working with Mark Williams from Heart n Soul (project owner), Justin Spooner from Heart n Soul (project lead), Casper Sawyer from Public Domain Corp (technical director) and colleagues, including Goldsmiths’ Dr Simon Katan and Dr Rebecca Fiebrink.
In November last year the group hosted a sold-out event at Nesta, where participants could experiment with top musical technology, talk to the developers who make it, play in a digital pop-up band, and watch live-performances.
SoundLab has been funded by the Digital R&D Fund for the Arts, an initiative created by Nesta, Arts Council England, AHRC and the National Lottery.
On Monday 7 March, Goldsmiths Computing academics discuss the challenges of creating emotionally-engaging games.
Creating emotionally engaging games in VR is full of hidden pitfalls and challenges. We draw upon our own experiences as VR researchers to explain what they are, and what you can do about them. We also present a Unity plugin we are developing to create virtual characters that behave realistically.
When: 6pm – 8pm Monday 7 March 2016
Where: Playhubs, Room M48, New Wing, Somerset House, London WC2R 1LA
6:00pm – Welcome
6.15pm – Introduction: key challenges.
6.25pm – Lessons learnt from 15 years of research.
6.40pm – The BlipC framework – work in progress.
6.50pm – Panel Q&A
7:55pm onwards – Networking, beers and demos including “A Chair In A Room”
In February 2016, femhype.com interviewed Phoenix Perry, lecturer in physical computing and games at Goldsmiths. We reprint the conversation here.
“Blanket Fort Chats” is a weekly column featuring women and nonbinary game makers talking about the craft of making games. In this week’s post, we featurePhoenix Perry, an experienced developer, accidental public figure, and general rabble-rouser. She’s currently a lecturer at Goldsmiths, University of London where she teaches physical computing and games.
Miss N: Can you tell us a little bit about your background and how you got into making games?
Phoenix: I got into games from making experimental movies. From there, I had a huge desire to work on projects that created empathy and emotion in a very physical way. My first project was an emergent system/interactive story that focused on bee colonies and collective ecologies. The underpinning idea behind it could be best summed up by Spock in The Wrath of Khan: “Logic clearly dictates that the needs of the many outweigh the needs of the few.”
Players worked in teams to achieve their team’s goal in a system where every choice impacted the entire game ecology. Your choice to take more or less of a resource impacted all other players who could, in turn, impact you. This game was called Honey and I made it back in 2006. It also yielded a pervasive version calledPicky Sticky Pollen that I showed at Come Out and Play in 2008.
Honey by Phoenix Perry
Miss N: What’s your earliest memory of playing games?
Phoenix: That’s definitely Pac-Man. It was the ’80s and I was wearing a rainbow swimsuit. I remember laying on brown carpet with my joystick for hours after that. It was awesome. My other favorite games were ET and Missile Command. Beyond that, there was a game about coming to NYC as an Italian immigrant on the Apple II—I was really fond of being Italian. I wish I knew the name of it. It was a choose your own adventure-style game. I would imagine being my grandmother.
Miss N: Can you tell us a little bit about your creative process?
Phoenix: I get my ideas from science and my own body. Nature is by far the most inspiring thing to me. Usually, I’ll be looking at some scientific principle and use play as a way to explore it in a group context. My most recent game is inspired by a book called Sync. I also read academic papers and follow a few key HCI and neuroscience journals. Science fiction also has wellspring of ideas I love exploring.
Early prototype of Nightgames by Phoenix Perry & Adelle Lin
Miss N: Previously, you’ve mentioned that you’re interested in exploring “the ways we could suggest new interaction through the bodies.” What drew you to this research area?
Phoenix: Interfaces destroying my own body drew me here. Regular interfaces hurt me. Right now, answering your questions causes me pain. After 34 years with this stuff, everything hurts. I have Scoliosis, Costochondritis, Carpal Tunnel and RSI, and pretty extreme scar tissue I just deal with. My daily pain levels sometimes are so extreme I get too nauseous to do anything else. I want to live in a world where people who lack the same abilities as others can interact with technology that adapts to them, not vice versa. The human body should never need to conform to the affordances of an interface. The interface should conform to me. It should empower all, not just a privileged set of people.
Miss N: How do you think this area of research can change games?
Phoenix: Games are great testing grounds. People are very open to trying new things out and play gives them a context that allows them to break social convention and not feel strange. It has also given me a way to get people to reflect on larger ideas. My games aren’t for the internet or online mainly, they are for sparking ideas and reflective experiences in small groups. Games with interfaces that use the whole body just have a much larger possibility space for interaction. Also, you can augment the senses, and for me, that’s really exciting. How do we hack into our own hardware?
Nightmare Kitty by Phoenix Perry & Nick Fox-Gieg
Miss N: When you’re developing projectsexploring this idea (such as Game Over or Yamove), what’s the process like?
Phoenix: Nightmare Kitty was my first game in this set of games and it came out first in 2011. I worked with Nick Fox-Gieg on it. It pre-dated all of the others and was my first attempt at creating emotions with physical game mechanics. It was an attempt to get children to encounter fear and overcome it in a controlled environment using their entire body. It grew out of my experiences in yoga.
Yoga is something I ended up doing for around 15 years now because my body was so fractured. It allowed me to see physicality in an entirely new way. I wanted to use a game to give that experience to a broader audience. In certain positions, I would find myself upset or scared, and in others, I found myself relieved. I built a game on those poses to re-create those emotions in a game context. Much of the science that came out that justified my own experiences started popping up between 2011 to 2012. That gave me a nice, theoretical way to justify both [Nightmare Kitty] andGame Over. In hindsight, I’d love to go back to the core of Nightmare Kitty and re-create it to work with children who deal with depression.
Katherine Isbister and Syed Salahuddin made Yamove after I made Nightmare Kitty and while I was making Game Over. Since I joined her lab in 2012, there was some obvious cross-pollination, and I’ve been friends with Syed for years. She and I have largely similar aims.
When their game, Yamove, got ripped to shreds by Eric Zimmerman at an Eyebeam playtest for lacking any visual appeal and having a bad user experience, I offered to come in and help them clean it up. It was a great idea, but they had a whole pack of grad students working on it part-time, none with the professional design background I had. I jumped onboard really at the ninth hour to help clean it up and give it a more professional polish.
It took three weeks of nearly no sleep right before No Quarter, but I did all the art direction. In fact, it’s pretty safe to say it had no design before. It was all largely clip art. Susan Kirkpatrick did the user experience and the grad students on the project had largely ignored her thinking during their development, which up until then was largely focused on game mechanics and functionality. I went in and tried to re-integrate her ideas. I based the entire feel of 1970’s disco and Delight’s Groove Is In The Heart. That said, it was a game with aims I really support. It was a nice way to contribute to a group effort in a contained way.
Miss N: Were there any challenges you encountered or things that took longer to figure out when you’re making these types of projects?
Phoenix: It’s always the hardware with me. I build/hack/write all of my stuff and, given my interests, I’m often working with materials that have undocumented SDKs or are not supposed to do what I am asking them to do at all.
In the case of Nightmare Kitty, I ended up writing the documentation for how to use the machine learning platform I worked with to pull off that game as early as we did. We were the only group at Maker Faire that year with a working, stable game using the Kinect. The reason for that was I just didn’t touch any of the techniques everyone else was so hot for and went straight to machine learning, which is the only real way to deeply control a sensor like that with gestures. I am now smiling as, five years on, everyone else finally has started to realize this.
Nightgames by Phoenix Perry & Adelle Lin
Nightgames has been my most recent project and it’s with Adelle Lin. She and I have so much tech in that project. I actually panic when I stop to think about re-building it. There are five separate micro controllers all from different companies with different SDKs and around 30 sensors.
Miss N: How do you get through these challenges?
Phoenix: Frankly, I just will not take no from my computer or my materials. That’s how I get through it. Hours of Google, research, coding, testing, laser cutting, Slack chatting with other devs, 3D printing prototypes, and cursing while blaring banging techno and slamming club matte at like 2 AM when I’d much rather sleep. I’m still not making things as stable or as smooth as I’d like. Stability is my goal for 2016.
Miss N: What’s been the most challenging thing you’ve encountered in making games?
Phoenix: Staying focused. Continuing when your projects take so long to make and sometimes don’t work out. Sometimes I get it really right. Sometimes I fail and get it wrong and spectacularly fail.
Last year at A MAZE., I got it wrong. We didn’t test the sensors outside and it turned out the density of the humidity caused them to not work; they were based on capacitive paint and we failed to think about how humidity would impact them. We also horribly underestimated our build time and tried to do something way too ambitious for a festival. It was really crushing and like a crucible for us. We learned so much and got our egos spanked in a really public way.
Then I had to come back to my studio and think—okay, what worked? How do we fix it? How do we make it not like that next time? That’s the hard part. Sometimes you walk with awards. Sometimes you walk away crushed. That’s just how it goes. It’s how you get up the next day and keep working that defines you.
Picky Sticky Pollen by Phoenix Perry & Marie Evelyn
Miss N: On the flip side, what’s been the most fulfilling?
Phoenix: There are two. After Picky Sticky Pollen was over and had done really well at Come Out and Play in 2008, a little girl grabbed my hand. She told me she just turned eight and her mom read all the game descriptions to her and she’d chosen mine. She said playing it was amazing and this was the best birthday she’d ever had.
Experience two happened at Maker Faire when I showed Nightmare Kitty to a few thousand children. It was popular, and after two days of non-stop lines out the door, I was at my end. I had spent two whole days on my knees explaining the game to children. So much so, I’d rubbed the leather off the toes of my Mary Janes and you could see the material. I had lost my voice and I was sitting there exhausted. Then the Maker team walked over and handed us two blue ribbons. I was bowled over. It felt like winning at adult science fair. It was really gratifying.
Miss N: In addition to being a game maker, you’re also one of the founders ofCode Liberation. Can you tell us a little bit of how it got started and what drew you to it?
Phoenix: Code Liberation came out of my desire to change the gender ratio in games after going to the GDC and seeing just how unbalanced the games space was. I’d always assumed the reason I wasn’t more important in games was because I made weird art games. After the GDC, I realized it might also have something to do with the fact I was a woman.
I came up with the idea on International Women’s Day and, in classic designer fashion, immediately designed the logo and blogged it. From there, I asked Luke Dubois and Frank Lanz to help out and support the idea with space and resources. When they agreed, I then invited Nina Freeman, Jane Friedhoff, Gavin Chan, and Catt Small to my house to discuss if they’d like to work with me on it. Really, at that table over pizza, the CLF as it exists today was born.
Miss N: Do you think there are things that are inherently unique to games (as a medium) compared to other creative mediums?
Phoenix: Yes—I think what makes games different is that they involve far more aspects of human experience than other genres. They can encompass a huge range of technical and creative spaces that other medium struggle with. Also, they can take hundreds of hours to experience and can generatively change with each new interaction. They regularly involve thousands of strangers you’ve never met from all over the world. This is just not the general case with art, movies, or music. That said, games can also be art or music or narrative stories. They are like the Katamari Damacy of creativity where they can absorb whatever is evolving in the culture at the time. Also, they are living interactive systems in a way a flat work of art is not.
Crystallon by Dozen Eyes (Phoenix Perry & Ben Johnson)
Miss N: Are there any games that you’ve felt have pushed the boundaries of the medium?
Phoenix: I have been doing this with nearly every physical game I’ve ever made. I’ve also made more conventional stuff like Crystallon and the work I do with my game studio, Dozen Eyes, is less on the experimental side and more formalist.
When I work with or make sensors, I feel like I’m pushing the space outward. Picky Sticky Pollen, Nightmare Kitty, Game Over, Emotional Growth, and Nightgameshave all pushed at the edges of what you can do in a game context in their own way. Right now, Nightgames is turning into an interactive, distributed, reactive sonic forest that takes over a city. I’m really excited to keep working on it.
Nightgames by Phoenix Perry & Adelle Lin
Miss N: Are there any women or nonbinary game makers who you really admire?
Phoenix: Yes! I love so many women in games it’s hard to know where to start. I love the work of Heather Kelley. She sees play as possible in an expanded context and I love her for that. Also, I really love Sophie Holden. She makes really fun mechanical games and has a deeply experimental expressive approach. Also, her publication rate is staggering. I’d be remiss if I didn’t mention Auriea Harvey. Her worlds are so hauntingly beautiful, I dream about them. Finally, Liz Ryerson has such insightful things to say about play and games. I love just reading her musing from time to time. Also, her music rocks.
Miss N: If you could go back and give yourself advice when you were just starting out as a game maker, what would it be?
At the end of January, Goldsmiths and Hacksmiths joined the marathon 48-hour Global Game Jam. Zoe O’Shea reports on what happened, and Jeremy Gow picks four standout games.
What can a group of people achieve in 48 hours? This was a question many of the new ‘Game Jammers’ were probably asking themselves as we approached the delightfully ominous doors of the St. James Hatcham building – a converted church – on Friday 29 January 2016. The Jam was set to begin at 5pm and finish at the same time that Sunday.
The space St. James afforded us was put to good use as the ever increasing buzz of Jammers had started to pour in from before four. White walls echoed the sounds of excited chatting, exclamations of friends catching sight of one another, murmurs of busy hosts trying to assemble and prep the site for the onslaught about to take place, and of course, the steady tapping of many fingers across many keyboards.
Members of the congregation included Goldsmiths alumni, current students, IGGI-letts, friends and a number of dedicated staff.
Our collective anticipation was put to rest when the theme was finally announced as ‘ritual’. And since this was a global event across multiple timezones, we were reminded: “Don’t post ‘til Hawaii knows.”
With a combined sigh of relief (“Last year’s theme was terrible!”), individuals started to make their way around tables introducing themselves. A look at the whiteboard at the top of the room indicated that “speed dating” had begun and the Global Game Jam had kicked off with earnest.
While it would be impossible to cover absolutely everything of interest that took place behind those glass doors in the following two days, honourable mentions must include:
a sound designer having to create game sounds (using only their mouth) in a secluded backroom
the use of the word “duress” quite repeatedly by an unnamed Jammer
a minor Wi-Fi meltdown on the Saturday (addressed with admirable haste)
a Jammer scaring themselves with their own game in the middle of the afternoon
a near constant stream of bodies in-and-out of the refreshments space
and a showing of great dedication (sleeping bags were involved) and camaraderie between all those present this year.
It’s not an easy thing to make “crunch” bearable, but the creativity and positivity shown by the Goldsmiths Jammers at St James reminds us all what the heart of this industry is about.
Jeremy Gow’s top games from Goldsmiths’ Global Game Jam
Ace Exorcist
A card game following the story of an exorcist trying to save the world from evil summoners attempting to bring evil into the world. Jeremy says: “It’s a well-designed card game with beautiful artwork. Our only non-digital game.”)
Habitual
Jeremy says: “Two first year students made a game about maintaining habits to deal with depression.” You play as an individual suffering from depression who begins to deal with their problems through maintaining and repeating habits; as simple as getting up everyday and leaving the house to sharing time with friends and just eating some good food. This game attempts to allow players to look into the ways of dealing with their own problems or helping them on their way.
Curse of Macbeth
A virtual reality horror game which follows the viewpoint of Moira – a young, committed British actress – who is totally consumed by her work after she gets the role of Lady Macbeth. She accidentally unleashes the Curse of Macbeth and the only way to escape is a cleansing ritual.
Mass Effect
You’re a priest who needs to collect all the flaming torches to complete his ritual. Unfortunately the crypt has started flooding. Collect all the torches before they’re covered by the water and make sure you don’t drown yourself. Jeremy says: “There’s a popular [and completely unrelated] series of ‘Mass Effect’ games… so the title is a terrible pun!”
Zoe is currently doing a PhD in Intelligent Games & Game Intelligence at Goldsmiths. She is researching the development of “self-theory” in digital games in order to create adaptive game-play experiences that respond to the player and their current psychological state.
In the first of our Music Computing Mixtape series, we present tracks from first year students’ portfolios – all made before they started at Goldsmiths.
“Welcome to the first week of the rest of your lives” is how Dr Freida Abtan welcomed a new cohort of students onto Goldsmiths’ BMus/BSc Music Computing programme.
Music Computing Mixtape #1 (aka Welcome to the first week of the rest of your lives) features tracks that students made during their first week at Goldsmiths. All pieces were made from the same set of recordings of a sound walk taken across the Millennium Bridge in London.
Over the next three years, these students will be immersed in performance, composition, musicology, design, psychoacoustics, digital signal processing and computer science – and we’ll be listening to the results. Subscribe to this blog (enter your email into the ‘subscribe’ widget on our homepage) for regular updates on how our students progress.
On 22 January 2016, Digital Arts Computing recently had the pleasure of meeting Belgian visual artist, self-taught software programmer and data activist Nicolas Malevé. Undergraduate student Isabella Maund reports on what happened.
Based in Barcelona, internationally active Malevé is a core member of Constant for Arts and Media, an artist run non-profit organization that explores an interdisciplinary approach to art, media and technology. Technical curiosity is what led to Malevé’s interest in data and the creative analysis of data.
Active Archives, a research project led by Malevé and Michael Murtaugh, investigates the mutation of an archive in a digital context. With his peers he is redefining our understanding of archives and focusing on original ways to classify, annotate and disseminate archives of images, with a strong focus on photography.
This masterclass was particularly relevant and fascinating, as data has become a large and ever growing part of our everyday lives. We are constantly creating and interacting with data through everyday searches, selfies, public transport, etc. Huge amounts of data are constantly being collected, becoming a resource that is transforming society.
Data collection has become a growing topic of conversation in the art world. Current exhibition Big Bang Data at the Somerset House has brought together artists, journalists and designers to further discuss data and its huge presence in our everyday lives, and what this may mean for our future.
“It was interesting to see how Nicolas’s own passion for archiving lead him to discover new and interesting ways to computationally categorise images. Analysing data in a way that is so notoriously difficult to do. His additional insight – that redacted documents reveal almost a core template – was also inspiring, leading me to think of several ideas for my own upcoming projects.” Joe McAlister, BSc Digital Arts Computing
“I enjoyed hearing Nicolas talk to us about his work, and the way that he approaches the subject matter as a computational artist. Getting feedback from him was really valuable in helping me develop my ideas about the place of computers in my work, and the discussions about our subject area were extremely interesting.” Rebecca Dunn, BSc Digital Arts Computing
PhD student Rain Ashford recently travelled to Osaka, Japan to exhibit and present her ThinkerBelle EEG Amplifying Dress. In this blogpost she describes her experience.
In September 2015, I was very excited to exhibit and present my paper on part of my PhD practice, the ThinkerBelle EEG Amplifying Dress, at the annual design exhibition of the 19th International Symposium on Wearable Computers.
The symposium (part of the 2015 ACM joint international conference of ISWC and Ubicomp) is devoted to discussing and sharing pioneering research, knowledge and issues in wearable technologies. It attracts international attendees including academics, manufacturers, fashion and textile designers, users, and related professionals working with wearables. The conference includes workshops, a gadget show, various gatherings and great opportunities to meet peers researching and working in the field, as well as research presentations.
About the ThinkerBelle EEG Amplifying Dress
I created the dress in response to a subsection of feedback data from field trials and focus groups investigating the functionality, aesthetics and user experience of wearables and in particular wearer and observer feedback on conducted in the course of my research on Goldsmiths’ Art & Computational Technology PhD programme.
The user experience case for creating the dress was to facilitate engagement in social situations in which a wearer finds themselves in a noisy or crowded area, where it is not easy to hear others, communicate and where forms of non-verbal communication may prove useful.
I constructed the dress using a satin fabric and fibre optic filament which is woven into organza. Using a NeuroSky MindWave Mobile EEG headset, data in the form of two separate streams, ‘attention’ and meditation’, are sent via Bluetooth to the dress, which amplifies and visualises the data via the fibre optic filament.
Attention data is shown as red light, and meditation signal data as green light. The dress is constructed so the two streams of data light overlap and interweave. The fibre optic filament is repositionable allowing the wearer to make their own lighting arrangements and dress design. The red and green light fades in an out as the levels of attention and meditation data of wearer heighten or decline, allowing observers to make their own interpretations of the data. The choice is left up to the wearer whether they want to divulge information regarding the physiological source of the data being visualised.
The design exhibition invited submissions of new and original examples of wearable technology and textile research. I exhibited my ThinkerBelle dress alongside the other selected submissions of garments, accessories, textiles and devices. The exhibition was divided into the categories of Functional, Aesthetic and Fibre Arts and the jury panel consisted of renowned designers and academics including:
writer, artist, designer and technologist Maggie Orth
founding Co-Director of CoLab, co-director of the Textile and Design Laboratory, and an associate researcher at the Knowledge Engineering & Discovery Research Institute, Dr. Frances Joseph.
I also participated in two workshops. The first, Wear and Tear: Constructing Wearable Technology for The Real World, was organised by colleagues at Georgia Tech Wearable Computing Centre and was a really useful and enjoyable day of reportage on building devices and systems. Georgia Tech’s Professor Thad Starner gave a keynote on wearable technology and was followed by various speakers who discussed challenges during the process of building their devices.
In the second workshop, Broadening Participation, I presented a poster on my PhD research on responsive and emotive wearables. The event was aimed at increasing “the involvement of women, all students from developing countries, as well as underrepresented minorities, including persons with disabilities, in the field of ubiquitous and wearable computing”. The day included keynote speakers and a careers panel in which speakers discussed issues, such as, career paths employment and work-life balance.
I am very grateful to the Goldsmiths Postgrad Research Committee for the bursary that enabled my travel to the conference. It was fantastic to exhibit and discuss the dress with a new audience in Japan and I enjoyed immensely seeing new examples of wearable technology and developments in the field, plus meeting local students and academics from Japan, as well as from China, Singapore, USA, Canada and Europe.
Mass Effect
I constructed the dress using a satin fabric and fibre optic filament which is woven into organza. Using a