World’s first computer-generated musical set for West End premiere

beyondthefence1

The first musical theatre production to be conceived and crafted by computers makes its debut in February 2016, with a premise and plot created by Goldsmiths, University of London’s ‘What-If Machine’.

2016. The computer revolution. In a world becoming ever more technologically advanced and reliant upon computers, machine learning and artificial intelligence are rapidly and fundamentally changing every aspect of human experience.

But how does technology affect art and the creative process? Can a computer ever create an entertaining and emotionally powerful show?

Presented by Wingspan Theatricals and Sky Arts, Beyond the Fence runs at the Arts Theatre in London’s West End from 22 February – 5 March 2016, directed by Luke Sheppard, choreographed by Cressida Carre and produced by Neil Laidlaw.

Beyond the Fence is both conceived and substantially crafted by computer, modeled on a statistical study of the ‘recipe for success’ in hit musicals.

In collaboration with leading experts in music, computation and the science of human creativity, composer Benjamin Till and his husband, writer and actor Nathan Taylor (the award-winning team behind Channel 4’s ‘Our Gay Wedding: The Musical‘), will bring a range of computer-generated material to life.

Designed and co-ordinated by Dr Catherine Gale, the whole process is also being filmed for a Sky Arts TV series titled ‘Computer Says Show’, to be broadcast in spring 2016.

Beyond the Fence started as an experiment, with researchers delving into what makes a good musical, from production and story to music and lyrics.

The process began with a predictive, big data analysis of success in musical theatre conducted at the University of Cambridge. Researchers interrogated everything from cast size to backdrop, emotional structure to the importance of someone falling in love, dying (or both!) – in more and less successful shows – to create a set of constraints to which the musical had to conform, to theoretically optimise chances of success.

Dr Teresa Llano from Goldsmiths explains to Benjamin Till and Nathan Taylor the workings behind the What-If Machine, the system that has provided the premise for ‘Beyond the Fence’
Dr Teresa Llano from Goldsmiths explains to Benjamin Till and Nathan Taylor the workings behind the What-If Machine, the system that has provided the premise for ‘Beyond the Fence’

Next, the team visited what’s known as the What-If Machine at Goldsmiths.

The Machine was created under a three year initiative, starting in 2013, to answer the question of whether creative software can move to the next level by generating, assessing, and presenting interesting ideas – whether it’s stories, jokes, films or paintings – that are really valued by the people who are exposed to them.

Funded by the European Community’s Seventh Framework Programme and with Goldsmiths Computing’s Professor Simon Colton, Dr Teresa Llano and Dr Rose Hepworth at the helm, the machine generated multiple central premises, featuring key characters, for a show.

From those options, the team selected this as the starting point and the original idea for the musical:

What if a wounded soldier had to learn how to understand a child in order to find true love?

September 1982. Mary and her daughter George are celebrating one year of living at the Greenham Common peace camp. The group of women they have joined are all committed to stopping the arrival of US cruise missiles through non-violent protest. When Mary is faced with losing her child to the authorities, an unlikely ally is found in US Airman Jim Meadow. How can she continue to do what is best for her daughter while staying true to her ideals?

A plot structure for the musical was also generated computationally, thanks to work led by Dr Pablo Gervás (Complutense University of Madrid), with a team then writing lyrics with the assistance of some other computational tools, that fitted all these constraints.

Finally, the music has been provided by Dr Nick Collins (Durham University) through his computer composition system Android Lloyd Webber, based on a machine listening analysis of musical theatre music at Queen Mary and City University, with additional material generated using the FlowComposer system created at Sony Computer Science Laboratory in Paris.


This blog post is an adaptation of a news story written by Sarah Cox, first published on Goldsmiths News.

← Subscribe here to win BIG BANG DATA tickets

COMPETITION NOW CLOSED.

We’re giving away two free tickets to the new Somerset House exhibition BIG BANG DATA.

BIG BANG DATA is a major new exhibition at Somerset House featuring artists, designers and innovators exploring how the data explosion is transforming our world.

To enter the competition, subscribe to Goldsmiths Computing’s blog using the ‘GET POSTS BY EMAIL’ widget on the left of this blogpost. This will sign you up to receive new blogposts by email. (If you can’t see the sign-up widget, go to the homepage)

Closing date: 11pm Sunday 3 January 2016

bigbangdata

We will email the two winners on Monday 4 January with info on how to claim the free ticket for a time slot of your choice.

And if you’re not a ticket-winner, you still win. You get lots of wonderful blogposts about Goldsmiths’ fascinating, extraordinary department of Computing. And if you book a group of 10 or more people, BIG BANG DATA will give you 10% off – just quote the discount code GROUP10BBD. Merry Christmas!

Small print: This offer is valid until 28 February 2016 (subject to availability). Open daily 10.00 – 18.00 (last admission 17.15). Late night Thursdays & Fridays until 21.00 (last admission 20.15). Under 12s are free (no booking required). Visitors with disabilities can bring an escort / carer free of charge (no booking required).

Update!
Creative Data Club is running a BIG BANG DATA special event on Thursday 28 January 2016. Speakers include Kate Hayes & Mathieu Barthet, who present their Open Symphony project (which invites audience to influence the musical direction of the performers through a web based app manifested in live data visualisations). Register for your free ticket here

‘Mathematics & Art’ book review by Prof William Latham

Professor of Computing William Latham recently reviewed the book Mathematics and Art for New Scientist magazine. We reprint his review here, with added hyperlinks and images.


From Renaissance painters’ first use of perspective to artistic algorithms shaping 21st-century works, mathematics and art have a long, rich history. “Cells and tissues, shell and bone, leaf and flower, are so many portions of matter, and it is their obedience to the laws of physics that their particles have been moved, molded and conformed. Their problems of form are in the first instance mathematical problems,” wrote the Scottish polymath D’Arcy Wentworth Thompson in his influential 1917 book, On Growth and Form.

This is a text that the author of the excellent new book, Mathematics and Art, has taken to heart and built on. In 500-plus, sumptuously illustrated pages, Lynn Gamwell has interleaved mathematics and culture (art, in particular) from 3000 BC to the present day, as she works to show how artists have harnessed maths for their own creative goals and how the arts, albeit to a lesser extent, have influenced maths.

flagellation700

There are many telling examples. Take Piero della Francesca’s 1455 painting The Flagellation of Christ, in which he positioned Jesus in a three-dimensional, naturalistic scene rather than an out-of-scale figure on a flat, 2D plane as his early Renaissance predecessors such as Giotto had done. This was a radical and daring innovation. What made it possible was the painter’s use of a set of new mathematical rules, which we now call linear perspective, that had been invented by mathematician and architect Filippo Brunelleschi.

Brunelleschi had himself been influenced by an 11th-century Islamic treatise on optics and visual distortion that had helped shape his ideas on perspective. This single mathematical step was to influence the whole of Western art, as exemplified in works by Leonardo da Vinci, Hans Holbein, Albrecht Dürer, Salvador Dali and, of course, M. C. Escher.

“Early Renaissance artists no longer painted saints floating in a golden mist in a faraway place; linear perspective gave them the tool to depict Jesus and the apostles existing right here, right now before their eyes in the natural world,” writes Gamwell.

Mandel_zoom_08_satellite_antenna

There have been many examples of these mathematical cross-overs: think of Mandelbrot’s fractal maths translated into psychedelic-style computer art in the 1980s, or the influence of quantum mechanics on post-modernist painting and sculpture. They may not all be of the same magnitude as Francesca’s use of perspective but they are significant, and it’s illuminating to discover the background to these innovations.

It’s also important to recognise how many mathematical fields inform art. Crystallography, celestial geometry, phyllotaxis, differential calculus – all helped to shape Renaissance art and movements such as surrealism, constructivism, pop art and minimalism.

Mathematics and Art is split in two, with the first section bringing us up to about 1900, and serving as a handbook for readers who want to choose specific topics. Among the mathematical gems and anecdotes, Gamwell cites conversations between da Vinci and Franciscan friar and mathematician Luca Pacioli discussing what would become Pacioli’s book, On The Divine Proportion. There are also reproductions of John Dalton’s rough but extraordinary diagrams of atomic elements from 1806.

The second half, post-1900, has fewer diagrams and works less well as a mathematical handbook. Instead, its strong suit is the presentation of the philosophical relationship between the arts and maths – as when Gamwell discusses the detail of quantum mechanics, taking Antony Gormley’s Quantum Cloud V sculpture as her hook.

Gamwell also dives into the compelling area of how we measure aesthetic value, citing George D. Birkhoff’s attempts in the 1930s to reduce aesthetics to a mathematical formula, M=O:C, or the amount of aesthetic pleasure produced by an object (M) equals the ratio of the object’s order (O) to its complexity (C).

“George D. Birkhoff attempted to reduce aesthetics to a single mathematical formula”
This is particularly relevant to the emerging field of creative robotics, where the goal is, apparently, to create a robot that will create art for its own aesthetic enjoyment, emulating the human creative process.

Gamwell must have had her work cut out deciding what to include and exclude in what aims to be a comprehensive tome. There are casualties. In the computation section, for example, it was right to make much of fractal mathematics, Alan Turing, John Conway’s Game of Life and computer artworks by Roman Verostko, Manfred Mohr and Yoichiro Kawaguchi. But some classic computer graphic algorithms are missing, such as Ken Perlin’s noise texture algorithm or the Blinn-Phong reflection model, which have had a major impact across the arts and in film.

And we really do need more than a brief reference to artist Robert Rauschenberg, composer John Cage and the Experiments in Art and Technology group’s show in 1966 at The Armory in New York. The group was set up to foster collaborations between artists and engineers through direct personal contact rather than through any kind of formal process. The creative talents that came together then helped define the work of a generation – and generations to come.

Overall this is a comprehensive, valuable and detailed book. It is written in an accessible style, with enough mathematics to interest the technical reader without overwhelming one with an arts background. It doesn’t quite rival Douglas Hofstadter’s hugely influential Gödel, Escher, Bach from 1979, but its rich anthology is particularly relevant today, given the explosion of interest in the digital arts and the need for digital artists to use maths creatively. I will definitely be keeping it close at hand.


 

Postgraduate degrees at Goldsmiths Computing include:

Ergonomics, autism & audience participation at the Science Museum

sarah2

Post doc teaching & research fellow Sarah Wiseman reports on a talk she recently gave at at the Science Museum’s Night Owls event.


The Science Museum Lates are a staple of any London geek’s calendar – once a month the museum opens its doors for an adults-only evening of events. These range from talks and quizzes, to workshops and printmaking, all of which can be as wine-fuelled as you desire. Each month they are packed with eager adults hoping to get a look around the museum without any children hogging all the fun toys.

The museum is now trying out a new idea for an evening event, and this one has a slightly different aim. A sister event to the morning Early Birds scheme, the Science Museum’s Night Owls events run on a Saturday evening after the museum so that families with children with autism can experience the museum in quieter, calmer settings. In the words of the museum, “Night Owls enables young adults who have Autism to come along and enjoy the museum free from the hustle and bustle of the general public.”

This November I was asked to give an in-depth talk on a topic relating to engineering, physics or maths. I wanted to talk about something I’m excited and passionate about, so naturally I chose to talk about the design of the telephone keypad. This is not only a topic that I myself find fascinating, but it’s a great introduction to the concept of Human Factors Engineering.

Arriving on Saturday evening before the event opened was eerie – I’d never seen the museum so quiet. I had to resist the urge to pretend I was the only one there and go exploring.

I gave two talks through the evening to small groups of people. The small groups meant I felt like I was having a chat with everyone there rather than giving a presentation. It also made the audience participation parts a bit easier for me – it’s far easier to pick on people when you’ve been chatting to them just a moment earlier.

The audiences were enthusiastic and chatty, with plenty of questions for me both during and after the talks. I have to say that this was refreshing, as the room can sometimes be a bit quiet when talking to students this age. But this audience had plenty to ask to me; some even went so far as to suggest new interfaces that could be designed in the future. There was also some post-talk career advice to one eager student, as I explained my experience of moving from my undergraduate to my PhD to my current post doc.

sarah3

The evening was really fun for me. I love talking about things I’m excited by – and to get an enthusiastic reception makes it all the better. I had a great time talking to the young adults who came along, as well as their friends and family. This was the first of the Night Owls events, and I could see that everyone who came along really appreciated the evening. I only hope this isn’t the last!


Sarah Wiseman is a post doc teaching and research fellow working on Human Computer Interaction at Goldsmiths Computing. Her interests lie in understanding more about number entry and errors, and investigating the use of haptic technologies to improve cultural experiences for people with visual impairments. She has also begun exploring citizen science, and how that might be a useful tool when recruiting for HCI experiments.

She really enjoys talking about research, and has done stand-up comedy based on it. She likes engaging people with science and other interesting ideas. Her other passion is making things – working with arduinos, sifteo cubes and general crafts.

West End debut for world’s first computer-generated musical

Goldsmiths' Maria Teresa Llano with Benjamin Till and Nathan Taylor
Goldsmiths’ Maria Teresa Llano with Benjamin Till and Nathan Taylor

The first musical conceived and crafted by computer, makes its debut in February with a premise and plot created by the Goldsmiths, University of London’s ‘What-If Machine’ program.

In a world becoming ever more technologically advanced, and reliant on computers, machine learning and artificial intelligence are rapidly and fundamentally changing every aspect of human experience. Now we’re about to see how technology can affect art and the creative process.

The show is both conceived and substantially crafted by computer, modeled on a statistical study of the ‘recipe for success’ in hit musicals.

In collaboration with leading experts in music, computation and the science of human creativity, composer Benjamin Till and his husband, writer and actor Nathan Taylor, will bring a range of computer-generated material to life. They have aimed to create an emotionally powerful and exciting West End show, which is, at the same time, the grandest of experiments, designed and co-ordinated by Dr Catherine Gale.

Beyond the Fence started as an experiment, with researchers delving into what makes a good musical, from production and story to music and lyrics. The process began with a predictive, big data analysis of success in musical theatre, conducted at the University of Cambridge. Researchers examined everything from cast size to backdrop, emotional structure to the importance of someone falling in love, dying (or both) – in more and less successful shows – to create a set of constraints to which the musical had to conform, to theoretically optimise chances of success.

Next, the team visited what’s known as the What-If Machine at Goldsmiths.

The Machine was created under a three-year initiative, starting in 2013, to answer the question of whether creative software can generate, assess, and present interesting ideas – whether it’s stories, jokes, films or paintings – that will be appreciated by people who are exposed to them. The team eventually settled on one original idea for the musical – what if a wounded soldier had to learn how to understand a child in order to find true love?

And so Beyond The Fence was born. Set in 1982, Mary and her daughter George are celebrating a year of living at the Greenham Common peace camp. The group of women they have joined are all committed to stopping the arrival of US cruise missiles through non-violent protest. When Mary is faced with losing her child to the authorities, an unlikely ally is found in US Airman Jim Meadow. How can she continue to do what is best for her daughter while staying true to her ideals?

A plot structure for the musical was also generated computationally, thanks to work led by Dr Pablo Gervás of Complutense University of Madrid. Finally, the music has been provided by Dr Nick Collins, of Durham University, through his computer composition system, the wittily-named Android Lloyd Webber.

Beyond The Fence plays at the Arts Theatre from 22 February – 5 March 2015.


This post is an edited abstract of an article published in Stage Review

Computational Arts graduate featured in WIRED magazine

Let-It-Brain

In a piece entitled “This Face-Melding Art Project Is Made to Teach You Empathy”, WIRED magazine has profiled Goldsmiths MFA Computational Arts graduate Antonio Daniele.

The article focuses on the This Is Not Private, Daniele’s final MFA artwork, which showed at the EXCEPT/0N Computational Arts exhibition in September 2015.

http://www.wired.com/2015/11/this-face-melding-art-project-is-made-to-teach-you-empathy/

“While you watch, an algorithm uses the characteristics of your face—the distance between your eyes, the shape of your jaw, the size of your nose—to create a ratio between the values. It does the same for the person on screen. Daniele wrote the software so the ratio would correlate to one of six basic facial expressions (anger, fear, sadness, joy, disgust, and surprise.) When you exhibit empathy—which in this case is determined by how closely your expression mirrors that of the person on screen—the image takes on elements of your face. The more empathy you show, the more the two of you become one.”


 

 

UK premiere: Laetitia Sonami performs Radigue’s “OCCAM IX for Electronic Spring Spyre”

sonami_headshot

We are pleased to announce a rare and intimate concert by Laetitia Sonami for the UK premiere of Eliane Radigue’s “OCCAM IX for Electronic Spring Spyre”.

WHEN: Wednesday 2 December 2015. 7.30pm for 8pm start
WHERE: SIML Space (G05), St James Hatcham, New Cross. View map
BOOK TICKETS: Free online registration

There will be an open Q&A session with Laetitia after the performance. Laetitia will also hold a free workshop on 3 December 2015 at Goldsmiths (separate registration required).

ABOUT OCCAM IX
OCCAM IX inscribes itself in a larger series of compositions entitled OCCAM OCEAN created by composer Eliane Radigue for, and with instrumentalists and composers. Compositions for harp, violin, viola, bass clarinet, and cello have been created with Rhodri Davis, Carol Robinson, Charles Curtis, amongst others and received critical acclaim.

This ninth composition was created with Laetitia Sonami on electronics with an instrument designed for the occasion by Sonami, named the Electronic Spring Spyre. Sonami originally studied with Radigue in France in 1976. While her music took on a very different expression through her design of unique controllers and live performance, they both remained vey close. In 2011 Sonami requested Eliane to create a piece for her new instrument. This instrument is made of springs which are being analyzed by neural networks. These “impress” the activity of the mechanical springs onto the sound synthesis in real-time.

ABOUT LAETITIA SONAMI
Composer, performer, and sound installation artist Laetitia Sonami was born in France and settled in the United States in 1975 to pursue her interest in the emerging field of electronic music. She studied with Eliane Radigue, Joel Chadabe, Robert Ashley and David Behrman.

Laetitia Sonami’s sound installations combine audio and kinetic elements embedded in ubiquitous objects such as light bulbs, rubber gloves, bags and more recently toilet plungers. She collects electrical wire and embroids them in walls.

Her work combines text, music and “found sound” from the world, in compositions which have been described as “performance novels. Her signature instrument, the Lady’s Glove, is fitted with a vast array of sensors which track the slightest motion of her enigmatic dance: with it Sonami can create performances where her movements can shape the music and in some instances visual environments. The lady’s glove has become a fine instrument which challenges notions of technology and virtuosity.

Please note that there will be no admission to the concert once the performance has started. Seating is limited, if you require a seat or have any accessibility requirements please email sh@goldsmithsdigital.com in advance of the concert.

The concert is organised by the Embodied Audio Visual Interaction group (EAVI) and Goldsmiths’ Music department.


 

Creativity, independence and learning by doing.