Category Archives: Staff profiles and activity

World’s first computer-generated musical set for West End premiere

beyondthefence1

The first musical theatre production to be conceived and crafted by computers makes its debut in February 2016, with a premise and plot created by Goldsmiths, University of London’s ‘What-If Machine’.

2016. The computer revolution. In a world becoming ever more technologically advanced and reliant upon computers, machine learning and artificial intelligence are rapidly and fundamentally changing every aspect of human experience.

But how does technology affect art and the creative process? Can a computer ever create an entertaining and emotionally powerful show?

Presented by Wingspan Theatricals and Sky Arts, Beyond the Fence runs at the Arts Theatre in London’s West End from 22 February – 5 March 2016, directed by Luke Sheppard, choreographed by Cressida Carre and produced by Neil Laidlaw.

Beyond the Fence is both conceived and substantially crafted by computer, modeled on a statistical study of the ‘recipe for success’ in hit musicals.

In collaboration with leading experts in music, computation and the science of human creativity, composer Benjamin Till and his husband, writer and actor Nathan Taylor (the award-winning team behind Channel 4’s ‘Our Gay Wedding: The Musical‘), will bring a range of computer-generated material to life.

Designed and co-ordinated by Dr Catherine Gale, the whole process is also being filmed for a Sky Arts TV series titled ‘Computer Says Show’, to be broadcast in spring 2016.

Beyond the Fence started as an experiment, with researchers delving into what makes a good musical, from production and story to music and lyrics.

The process began with a predictive, big data analysis of success in musical theatre conducted at the University of Cambridge. Researchers interrogated everything from cast size to backdrop, emotional structure to the importance of someone falling in love, dying (or both!) – in more and less successful shows – to create a set of constraints to which the musical had to conform, to theoretically optimise chances of success.

Dr Teresa Llano from Goldsmiths explains to Benjamin Till and Nathan Taylor the workings behind the What-If Machine, the system that has provided the premise for ‘Beyond the Fence’
Dr Teresa Llano from Goldsmiths explains to Benjamin Till and Nathan Taylor the workings behind the What-If Machine, the system that has provided the premise for ‘Beyond the Fence’

Next, the team visited what’s known as the What-If Machine at Goldsmiths.

The Machine was created under a three year initiative, starting in 2013, to answer the question of whether creative software can move to the next level by generating, assessing, and presenting interesting ideas – whether it’s stories, jokes, films or paintings – that are really valued by the people who are exposed to them.

Funded by the European Community’s Seventh Framework Programme and with Goldsmiths Computing’s Professor Simon Colton, Dr Teresa Llano and Dr Rose Hepworth at the helm, the machine generated multiple central premises, featuring key characters, for a show.

From those options, the team selected this as the starting point and the original idea for the musical:

What if a wounded soldier had to learn how to understand a child in order to find true love?

September 1982. Mary and her daughter George are celebrating one year of living at the Greenham Common peace camp. The group of women they have joined are all committed to stopping the arrival of US cruise missiles through non-violent protest. When Mary is faced with losing her child to the authorities, an unlikely ally is found in US Airman Jim Meadow. How can she continue to do what is best for her daughter while staying true to her ideals?

A plot structure for the musical was also generated computationally, thanks to work led by Dr Pablo Gervás (Complutense University of Madrid), with a team then writing lyrics with the assistance of some other computational tools, that fitted all these constraints.

Finally, the music has been provided by Dr Nick Collins (Durham University) through his computer composition system Android Lloyd Webber, based on a machine listening analysis of musical theatre music at Queen Mary and City University, with additional material generated using the FlowComposer system created at Sony Computer Science Laboratory in Paris.


This blog post is an adaptation of a news story written by Sarah Cox, first published on Goldsmiths News.

‘Mathematics & Art’ book review by Prof William Latham

Professor of Computing William Latham recently reviewed the book Mathematics and Art for New Scientist magazine. We reprint his review here, with added hyperlinks and images.


From Renaissance painters’ first use of perspective to artistic algorithms shaping 21st-century works, mathematics and art have a long, rich history. “Cells and tissues, shell and bone, leaf and flower, are so many portions of matter, and it is their obedience to the laws of physics that their particles have been moved, molded and conformed. Their problems of form are in the first instance mathematical problems,” wrote the Scottish polymath D’Arcy Wentworth Thompson in his influential 1917 book, On Growth and Form.

This is a text that the author of the excellent new book, Mathematics and Art, has taken to heart and built on. In 500-plus, sumptuously illustrated pages, Lynn Gamwell has interleaved mathematics and culture (art, in particular) from 3000 BC to the present day, as she works to show how artists have harnessed maths for their own creative goals and how the arts, albeit to a lesser extent, have influenced maths.

flagellation700

There are many telling examples. Take Piero della Francesca’s 1455 painting The Flagellation of Christ, in which he positioned Jesus in a three-dimensional, naturalistic scene rather than an out-of-scale figure on a flat, 2D plane as his early Renaissance predecessors such as Giotto had done. This was a radical and daring innovation. What made it possible was the painter’s use of a set of new mathematical rules, which we now call linear perspective, that had been invented by mathematician and architect Filippo Brunelleschi.

Brunelleschi had himself been influenced by an 11th-century Islamic treatise on optics and visual distortion that had helped shape his ideas on perspective. This single mathematical step was to influence the whole of Western art, as exemplified in works by Leonardo da Vinci, Hans Holbein, Albrecht Dürer, Salvador Dali and, of course, M. C. Escher.

“Early Renaissance artists no longer painted saints floating in a golden mist in a faraway place; linear perspective gave them the tool to depict Jesus and the apostles existing right here, right now before their eyes in the natural world,” writes Gamwell.

Mandel_zoom_08_satellite_antenna

There have been many examples of these mathematical cross-overs: think of Mandelbrot’s fractal maths translated into psychedelic-style computer art in the 1980s, or the influence of quantum mechanics on post-modernist painting and sculpture. They may not all be of the same magnitude as Francesca’s use of perspective but they are significant, and it’s illuminating to discover the background to these innovations.

It’s also important to recognise how many mathematical fields inform art. Crystallography, celestial geometry, phyllotaxis, differential calculus – all helped to shape Renaissance art and movements such as surrealism, constructivism, pop art and minimalism.

Mathematics and Art is split in two, with the first section bringing us up to about 1900, and serving as a handbook for readers who want to choose specific topics. Among the mathematical gems and anecdotes, Gamwell cites conversations between da Vinci and Franciscan friar and mathematician Luca Pacioli discussing what would become Pacioli’s book, On The Divine Proportion. There are also reproductions of John Dalton’s rough but extraordinary diagrams of atomic elements from 1806.

The second half, post-1900, has fewer diagrams and works less well as a mathematical handbook. Instead, its strong suit is the presentation of the philosophical relationship between the arts and maths – as when Gamwell discusses the detail of quantum mechanics, taking Antony Gormley’s Quantum Cloud V sculpture as her hook.

Gamwell also dives into the compelling area of how we measure aesthetic value, citing George D. Birkhoff’s attempts in the 1930s to reduce aesthetics to a mathematical formula, M=O:C, or the amount of aesthetic pleasure produced by an object (M) equals the ratio of the object’s order (O) to its complexity (C).

“George D. Birkhoff attempted to reduce aesthetics to a single mathematical formula”
This is particularly relevant to the emerging field of creative robotics, where the goal is, apparently, to create a robot that will create art for its own aesthetic enjoyment, emulating the human creative process.

Gamwell must have had her work cut out deciding what to include and exclude in what aims to be a comprehensive tome. There are casualties. In the computation section, for example, it was right to make much of fractal mathematics, Alan Turing, John Conway’s Game of Life and computer artworks by Roman Verostko, Manfred Mohr and Yoichiro Kawaguchi. But some classic computer graphic algorithms are missing, such as Ken Perlin’s noise texture algorithm or the Blinn-Phong reflection model, which have had a major impact across the arts and in film.

And we really do need more than a brief reference to artist Robert Rauschenberg, composer John Cage and the Experiments in Art and Technology group’s show in 1966 at The Armory in New York. The group was set up to foster collaborations between artists and engineers through direct personal contact rather than through any kind of formal process. The creative talents that came together then helped define the work of a generation – and generations to come.

Overall this is a comprehensive, valuable and detailed book. It is written in an accessible style, with enough mathematics to interest the technical reader without overwhelming one with an arts background. It doesn’t quite rival Douglas Hofstadter’s hugely influential Gödel, Escher, Bach from 1979, but its rich anthology is particularly relevant today, given the explosion of interest in the digital arts and the need for digital artists to use maths creatively. I will definitely be keeping it close at hand.


 

Postgraduate degrees at Goldsmiths Computing include:

Ergonomics, autism & audience participation at the Science Museum

sarah2

Post doc teaching & research fellow Sarah Wiseman reports on a talk she recently gave at at the Science Museum’s Night Owls event.


The Science Museum Lates are a staple of any London geek’s calendar – once a month the museum opens its doors for an adults-only evening of events. These range from talks and quizzes, to workshops and printmaking, all of which can be as wine-fuelled as you desire. Each month they are packed with eager adults hoping to get a look around the museum without any children hogging all the fun toys.

The museum is now trying out a new idea for an evening event, and this one has a slightly different aim. A sister event to the morning Early Birds scheme, the Science Museum’s Night Owls events run on a Saturday evening after the museum so that families with children with autism can experience the museum in quieter, calmer settings. In the words of the museum, “Night Owls enables young adults who have Autism to come along and enjoy the museum free from the hustle and bustle of the general public.”

This November I was asked to give an in-depth talk on a topic relating to engineering, physics or maths. I wanted to talk about something I’m excited and passionate about, so naturally I chose to talk about the design of the telephone keypad. This is not only a topic that I myself find fascinating, but it’s a great introduction to the concept of Human Factors Engineering.

Arriving on Saturday evening before the event opened was eerie – I’d never seen the museum so quiet. I had to resist the urge to pretend I was the only one there and go exploring.

I gave two talks through the evening to small groups of people. The small groups meant I felt like I was having a chat with everyone there rather than giving a presentation. It also made the audience participation parts a bit easier for me – it’s far easier to pick on people when you’ve been chatting to them just a moment earlier.

The audiences were enthusiastic and chatty, with plenty of questions for me both during and after the talks. I have to say that this was refreshing, as the room can sometimes be a bit quiet when talking to students this age. But this audience had plenty to ask to me; some even went so far as to suggest new interfaces that could be designed in the future. There was also some post-talk career advice to one eager student, as I explained my experience of moving from my undergraduate to my PhD to my current post doc.

sarah3

The evening was really fun for me. I love talking about things I’m excited by – and to get an enthusiastic reception makes it all the better. I had a great time talking to the young adults who came along, as well as their friends and family. This was the first of the Night Owls events, and I could see that everyone who came along really appreciated the evening. I only hope this isn’t the last!


Sarah Wiseman is a post doc teaching and research fellow working on Human Computer Interaction at Goldsmiths Computing. Her interests lie in understanding more about number entry and errors, and investigating the use of haptic technologies to improve cultural experiences for people with visual impairments. She has also begun exploring citizen science, and how that might be a useful tool when recruiting for HCI experiments.

She really enjoys talking about research, and has done stand-up comedy based on it. She likes engaging people with science and other interesting ideas. Her other passion is making things – working with arduinos, sifteo cubes and general crafts.

West End debut for world’s first computer-generated musical

Goldsmiths' Maria Teresa Llano with Benjamin Till and Nathan Taylor
Goldsmiths’ Maria Teresa Llano with Benjamin Till and Nathan Taylor

The first musical conceived and crafted by computer, makes its debut in February with a premise and plot created by the Goldsmiths, University of London’s ‘What-If Machine’ program.

In a world becoming ever more technologically advanced, and reliant on computers, machine learning and artificial intelligence are rapidly and fundamentally changing every aspect of human experience. Now we’re about to see how technology can affect art and the creative process.

The show is both conceived and substantially crafted by computer, modeled on a statistical study of the ‘recipe for success’ in hit musicals.

In collaboration with leading experts in music, computation and the science of human creativity, composer Benjamin Till and his husband, writer and actor Nathan Taylor, will bring a range of computer-generated material to life. They have aimed to create an emotionally powerful and exciting West End show, which is, at the same time, the grandest of experiments, designed and co-ordinated by Dr Catherine Gale.

Beyond the Fence started as an experiment, with researchers delving into what makes a good musical, from production and story to music and lyrics. The process began with a predictive, big data analysis of success in musical theatre, conducted at the University of Cambridge. Researchers examined everything from cast size to backdrop, emotional structure to the importance of someone falling in love, dying (or both) – in more and less successful shows – to create a set of constraints to which the musical had to conform, to theoretically optimise chances of success.

Next, the team visited what’s known as the What-If Machine at Goldsmiths.

The Machine was created under a three-year initiative, starting in 2013, to answer the question of whether creative software can generate, assess, and present interesting ideas – whether it’s stories, jokes, films or paintings – that will be appreciated by people who are exposed to them. The team eventually settled on one original idea for the musical – what if a wounded soldier had to learn how to understand a child in order to find true love?

And so Beyond The Fence was born. Set in 1982, Mary and her daughter George are celebrating a year of living at the Greenham Common peace camp. The group of women they have joined are all committed to stopping the arrival of US cruise missiles through non-violent protest. When Mary is faced with losing her child to the authorities, an unlikely ally is found in US Airman Jim Meadow. How can she continue to do what is best for her daughter while staying true to her ideals?

A plot structure for the musical was also generated computationally, thanks to work led by Dr Pablo Gervás of Complutense University of Madrid. Finally, the music has been provided by Dr Nick Collins, of Durham University, through his computer composition system, the wittily-named Android Lloyd Webber.

Beyond The Fence plays at the Arts Theatre from 22 February – 5 March 2015.


This post is an edited abstract of an article published in Stage Review

New audiovisual performance app developed by Goldsmiths researcher

Goldsmiths researcher Nuno Correia has developed AudioVisual Zones, a new iPad app for audiovisual performance which will be available at the App Store soon.

AVZones is open source and work in progress, built with openFrameworks and Maximilian, and part of Goldsmiths’ Enabling Audiovisual User Interfaces research project.

It’s composed of an audio sequencer/looper with a visualizer. By default, three audiovisual columns or “zones” allow for the manipulation of three audio loops. Each zone had three XY pads for audio manipulation: pitch shift, delay and filter. There are nine sounds available per zone. The application is scalable: the number of zones, XY pads and sounds can be modified in the code.

Nuno Correia has performed using AVZones in Berlin and at London’s EAVI XIII. In a performance, only the iPad is used for audiovisuals; the visuals from the iPad are projected behind the performer, and the sound comes from the iPad as well.

The project is supported by a Marie Curie EU fellowship, and hosted by the EAVI research group in Goldsmiths Computing. The code for AVZones is available on GitHub, and the app can be sideloaded manually on an iPad using Xcode 7.


 

 

EVENT: Goldsmiths’ SoundLab Play Space breaks down barriers to digital music making

Goldsmiths Computing experts have been testing musical technologies to work out which are the easiest to use, sound the best – and can be used by people with learning disabilities to make the music they want to make.

On Wednesday 25 November they’re hosting a day of free, fun and interactive performance, debate and play sessions at Nesta.

Where: Nesta, 1 Plough Place, London EC4A 1DE
When: 4-6pm and 6-8pm Wednesday 25 November
Tickets: SOLD OUT Get free tickets for the SoundLab Play Space

Participants will be able to experiment with top musical technology and talk to the developers who make it. There’ll be a chance to play in a digital pop-up band, watch live-performances, try out fantastic music-making apps, and take some home for free.

SOUNDLAB2[1]

From music apps that let you compose, DJ or play countless instruments with a fingertip, to those that make a voice sound amazing even if it’s not quite in tune, the SoundLab researchers have spent the last year rigorously testing iPhone/Pad, Android and web programmes.

Whether you’re a technologist, artist, musician, teacher, health expert or policy maker, SoundLab Play Space is designed to bring different people together to get a new perspective on, and debate, the future of music making.

SOUNDLAB1[1]

The SoundLab project is led by Heart n Soul, with Goldsmiths’ Dr Mick Grierson, Dr Rebecca Fiebrink and Dr Simon Katan working on research. SoundLab is a collaboration between our EAVI group, award-winning creative arts company Heart n Soul, and Public Domain Corporation, a company providing interactive experiences and technology for the games and digital arts sectors.

SoundLab has been funded by the Digital R&D Fund for the Arts, an initiative created by Nesta, Arts Council England, AHRC and the National Lottery.


Adapted from an article originally published in Goldsmiths News

FREE TRAINING: Computing & Entrepreneurship Business Club

digital-entrepreneurship

This autumn, Goldsmiths Computing are running free drop-in workshops on the first Tuesday of 
the month.

Come to all, or come to just one. Participants will work in groups to complete a series of business computing & entrepreneurial challenges. All welcome, but especially suitable for people considering studying business or computing at university.

5pm-7pm, Tuesday 6 October 2015
How much am I worth? Discover how businesses use and track your data to create a picture of who you are, to sell you products and services. Book your place

5pm-7pm, Tuesday 3 November 2015
Why are we all brand junkies? Learn how businesses use brands and digital identities to sell products & services – and why we love them. Book your place

5pm-7pm, Tuesday 1 December 2015
How do social media sites actually make money? You use social media like Instagram, Facebook and Twitter, but rarely pay for it. Find out how they make money and why these businesses are worth so much. Book your place

ABOUT THE TUTOR

Dr James Ohene-Djan is a senior lecturer in Business Computing at Goldsmiths, University of London. He is an internationally recognized innovator, designer and creator of digital products, services and businesses and has a strong strategic and technical understanding of creative approaches to implementing business solutions.

As co-founder and inventor of WinkBall Video Media, James pioneered key aspects of the use of video in social networking, reporting and mobile technologies. He is currently managing director of the digital businesses Bizwinks and RecoverySpaces.
He has appeared numerous times in the press and on BBC1, Sky News and CNBC.