Goldsmiths’ MIMIC project: ‘Cyborg’ musicians could be the future of music

Massive-Attack-WEB

Massive Attack at Weekendance 2007 in Barcelona. Photo: Alterna2

Musicians will be able to use Artificial Intelligence to create new music and sound to share or sell, thanks to a project led by Goldsmiths.

At a time when many in the music industry worry their livelihoods are under threat from new technology, the MIMIC (Musically Intelligent Machines Interacting Creatively) project puts humans back in control of making music.

MIMIC will develop free, user-friendly web tools that harness the power of AI to listen to existing recordings and come up with new sounds and instruments interactively. Artists will own the sounds they create and can incorporate them into their music or sell them to others. The tools will meld the latest ‘deep learning’ AI methods with people’s creativity to empower a new generation of ‘cyborg’ musicians.

The £1m project is a collaboration between Goldsmiths, the University of Durham, the University of Sussex, and Google Magenta and has been funded by the UK’s Arts and Humanities Research Council (AHRC).

Mick Grierson, Professor of Computing at Goldsmiths and MIMIC project leader said: “In the past, to use these powerful Artificial Intelligence technologies you had to be an expert in programming: we want to make these technologies free and easy for anyone to use – from amateur music-makers and sonic experimenters to professional musicians.

“Rather than simply creating autonomous musical ‘robots’, we are harnessing Artificial Intelligence systems to augment human creativity. We’re inviting people to meld their musical talents and sonic curiosity to the very latest deep learning systems. Our interfaces will mean you don’t have to already know how to code to benefit from AI, you just have to want to make some noise. However, if you do want to code, you’ll be able to do so using a new language we will be creating specifically for making AI music systems.”

The AI technology powering MIMIC has already proved a hit with professional musicians: Massive Attack are currently using it to create new musical instruments for the upcoming tour celebrating the 20th anniversary of their Mezzanine album. Back in 2016, Sigur Ros used a similar system built by Grierson to create an evolving version of a single song for a 24-hour televised journey around the Icelandic coast. Now the Goldsmiths-led team wants to make these technologies accessible to anyone with an interest in musical creation and exploration.

The MIMIC team aim to upload the first prototype web tools for people to experiment with within the next year. The tools will use a browser-based simplified live coding language written on top of JavaScript specially designed for musicians and artists. As well as working with the music industry, the team plan to produce learning materials for university, secondary school, and professional learners introducing them to how they can enhance their creativity with AI systems.


 

This post, written by Pete Wilton, was originally published on Goldsmiths News

New model reveals forgotten influencers and ’sleeping beauties’ of science

books-1655783_1280

The influence of ‘forgotten’ scientific papers has been demonstrated in a new study led by a researcher from Goldsmiths, University of London.

A team from Goldsmiths, the University of Chicago, Google, the University of Maryland, and Columbia University, developed a model that tracks ‘discursive influence’, or recurring words and phrases in historical texts that measure how scholars actually talk about a field, instead of just their attributions. To determine a particular scientific paper’s influence, the researchers can statistically remove it from history and see how scientific discourse would have unfolded without it.

aaron-gerow380Aaron Gerow, Lecturer in Computing at Goldsmiths, who led t
he study said: “Citations are one kind of impact, and discursive influence is a different kind. Neither one is the complete story, but they work together to give a better picture of what’s influencing science.”

The researchers report in the journal PNAS how they trained the model on massive text collections from computational linguistics, physics, and across science and scholarship (JSTOR) and then traced distinct patterns of influence. They found that scientists who persistently published in a single field were more likely to be ‘canonised’ in a way that compelled others to cite them disproportionate to their papers’ discursive contributions. On the other hand, discoveries that crossed disciplinary boundaries were more likely to have outsized discursive impact but fewer citations, likely because the ‘owner’ of the idea and her allies remain socially and institutionally distant from the citing author.

The model also sheds light on so-called ‘sleeping beauties’: papers that went relatively unacknowledged for years or even decades before experiencing a late burst of citations. For example, a 1947 paper on graphene remained obscure and forgotten until the 1990s with a resurgence of research interest in the material and an eventual Nobel Prize.

Study co-author James Evans, director of Knowledge Lab and professor of sociology at the University of Chicago, said: “Papers have a news cycle, when lots of people chat about them and cite them, and then they’re no longer new news. Our model shows that some papers have much more influence than citations will typically demonstrate, such as these ‘sleeping beauties,’ which didn’t have much influence early but come to be appreciated and important later.”

The study used a computational method known as ‘topic modeling’ that was invented by co-author David Blei of Columbia University. The authors said the same model can also be used to trace influence in other areas, such as literature and music. Text from poems or song lyrics, and even extra-textual characteristics such as stanza structure or chord progressions, could feed into the model to find under-credited influencers and map the spread of new concepts and innovations.

A report of the research, ‘Measuring discursive influence across scholarship’ by Aaron Gerow, Yuening Hu, Jordan Boyd-Graber, David M. Blei and James A. Evans, is published in Proceedings of the National Academy of Sciences.


This article is based on an original story written by Rob Mitchum for University of Chicago News, which was then adapted by Peter Wilton for Goldsmiths News.


🍕 ‘Secret language’ of emoji revealed

emoji

People create their own ‘secret languages’ by attaching lasting alternative meanings to emoji unrelated to what they are designed to represent, according to a study from Goldsmiths Computing.

In people’s secret languages emoji of pizza or wedges of cheese mean ‘I love you’ (because these were foods people love), a bathtub emoji means a coffin (because it was the closest to a coffin shape), and a thinking face means ‘lesbian’ (because the position of the thumb and forefinger on the chin means ‘lesbian’ in American Sign Language).

These alternative meanings can be assigned randomly but become permanent and are used consistently over time between partners, friends, or family members, the research found.

The study, by researchers from Goldsmiths and the University of Birmingham, is due to be presented at the Computer Human Interaction 2018 conference in Montreal, Canada (21-26 April 2018).

In 2016 there was a furious customer backlash against Apple for changing the rendering of its peach emoji to look smoother. Researchers found that most Apple users were using this emoji to refer to buttocks, with only 7% referring to the foodstuff, and were angry the redrawn emoji did not fit this alternative meaning.

The Goldsmiths-led team launched an online survey to investigate how individuals personalise emoji to create ‘secret’ meanings. Those responding reported repurposing 69 different emoji for secret communication with the most common emoji chosen being an octopus, the most common emoji for an affectionate name being a penguin, and the most common category of emoji used ‘Animals & Nature’.

Dr Sarah Wiseman, lecturer in Computer Science at Goldsmiths and co-author of the study, said: “While we know some fruit and vegetable emoji have been repurposed by many people to mean something else, we were intrigued to find out about personal instances of this – examples of emoji that have a special meaning for just two people. Often this was about more than just typing something more quickly: people found that by using emoji they could convey very complex meanings and thoughts with them that could not be described in words.”

Of the survey’s 72 respondents (134 participants in total) who reported repurposing emoji:

  • 47% exchanged them with partners and 28% exchanged with friends
  • 21% used the emoji to express some form of affection
  • 19% used them to symbolise a particular person or pet
  • 7% used them to refer to sex
  • 6% used them to be covert while referring to sex or illegal activity

Dr Sarah Wiseman said: “Our study shows that people use emoji in a similar way to nicknames or slang, as a handy shortcut to what they mean, which through consistent use creates an intimate ‘secret language’ others don’t understand. Creators of emoji need to bear in mind the subtle way that people repurpose them and the impact even small visual changes to them could have on these alternative meanings.”

A report of the research, entitled Repurposing emoji for personalised communication: Why  🍕means “I love you” will be presented at the Computer Human Interaction 2018 conference in Montreal, Canada (21-26 April 2018).


Adapted from an article first published on Goldsmiths News.

Fifteen reasons to be cheerful

smile

January is over and the days are getting longer, so we’re in the mood to post a list of fifteen things from the past 12 months that we’re proud of. 

  1. We set up a playable library of computer games in the ground floor of the library.
  2. We ran our best ever Digital Arts Computing exhibition in April
  3. Our academics shared their research with students on Wednesday afternoons. More to come soon…
  4. Our Music Computing students set up Resolution, a programme of live music and visual performances
  5. Hacksmiths ran a ton of amazing events, including Non Binary in TechGlobal Game Jam, Anvil Hack, and our Welcome Week student get-together DoC.Hack.
  6. We hired Eilidh Macdonald to be our Industry Employability Champion who develops opportunities for placements, internships and other collaborations with employers.
  7. We ran an international art & tech symposium ‘Future Mind’ with our new friends at Kyoto University
  8. Post Doc Researcher Perla Maiolino exhibited at the Science Museum’s ROBOTS show
  9. Our EAVI (Embodied AudioVisual Interaction) research group ran a brilliant day of audiovisual workshops and performances at the ICA
  10. We launched five new online courses in Virtual Reality
  11. Our students were on primetime BBC1 telly
  12. We took part in an acid house revival
  13. Sex Tech Hack 2!
  14. Our students won a prize at the Living Data City Challenge hackathon in Eindhoven
  15. London Evening Standard named senior lecturer Dr Kate Devlin ‘one of London’s most influential people of 2017’

If you think of any more, send your suggestions to p.fry@gold.ac.uk.

Beyond sex robots: the real sex tech revolution

sextechchat

Goldsmiths press office Pete Wilton interviewed Dr Kate Devlin ahead of Sex Tech Hack II, where experts gather at Goldsmiths to discuss and make new kinds of intimate technology.


Sex robots are all over the news but is the technology as advanced as some suggest or could the real sex tech revolution look very different? This Friday (24 November 2017) Sex Tech Hack II will see experts gather at Goldsmiths, University of London to discuss and make new kinds of intimate technology.

Ahead of the event I talked to Dr Kate Devlin, Senior Lecturer in Computing at Goldsmiths, who is researching a new book about sex robots and was recently named on the Evening Standard’s Progress 1000 list, about how to separate scientific reality from electric wet dreams…

Pete Wilton: What myths about ‘sex robots’ need debunking?

Kate Devlin: That they exist. They don’t really, despite the flurry of media stories. There are mechanised sex dolls with some chatbot AI, but that’s about it. But they are being developed, and they currently extend the sex doll market, rather than looking at new or innovative forms. Like much technology, it’s very hetereonormative: these tend to be dolls made by men, for men. The hypersexualised female form is presented as the default.

PW: Which sex tech developments should we be most concerned or hopeful about?

KD: Sex tech has great potential to bring people happiness, whether it’s by enhancing pleasure and fun or providing a sex life for someone who – for psychological or physiological reasons – might face difficulties otherwise. It’s an industry estimated at around $30 billion worldwide and climbing.

New technologies can forge new forms of intimacy. Smart toys can be connected via the Internet, helping long distance relationships, for example, or changing the landscape of sex work, such as the cam industry. That said, there are areas that need close attention: security and privacy issues are key. The past year has seen at least three security and privacy vulnerabilities in smart sex toys.

PW: What are the major challenges to advancing this area of technology?

KD: Funding is problematic: in industry, venture capitalists don’t tend to fund sex tech as they have vice clauses that prohibit them from investing in adult ventures. Start-ups are reliant on angel investors.

Academia doesn’t fare much better: it seems that the best way of funding research into sex is to spin it from a health point of view. There’s also an attitude that research into sex and intimacy is trivial, which seems odd as for many people it’s such an intrinsic part of being human.

PW: How can initiatives like ADA-AI help to change the Artificial Intelligence agenda?

KD: ADA-AI is a new international non-profit organisation focused on evaluating, developing and lobbying around AI policy and regulation. I am one of 25 advisors and we look at how to ensure AI can contribute positively to society, especially for marginalised and underrepresented groups.

The current threat of AI is not superintelligence and a robot takeover; instead, it’s the unconscious bias in datasets and the lack of diversity being perpetuated and reinforced by systems that are now integrated into our lives.

PW: What do you hope will result from Sex Tech Hack II?

KD: Last year’s Sex Tech Hack was a great success and 50 people made 14 wonderful new examples of intimate technology. This year we have more people attending, plus a discussion day on Friday 24 November, with industry and academic experts giving talks and leading break-out sessions.

Hacksmiths, the SU tech society, have done an amazing job bringing it all together. We’ve ended up with an incredibly diverse group of attendees all set to make accessible, fair, fun prototypes. This year’s challenges are: “intimacy”, “accessibility”, and “personalisation”.

PW: How is work on your upcoming book going? What topics will it cover?

KD: The book (Turned On: The Science of the Sex Robot) continues and the deadline approaches – I wish I could say I’m as close to finishing as I should be! It’s a popular science book about sex robots – the origins of the narratives, which go right back to Greek myth, through to the sci-fi portrayals in films today. I’m writing about artificial intelligence, robotics, attachment, love, ethics and law. Send me your bad puns.


First published in Goldsmiths News, 21 November 2017

Mick Grierson becomes a professor

mick

Goldsmiths academic Mick Grierson has recently been honoured with a professorship in the Department of Computing. Here’s what he said…


Goldsmiths’ commitment to interdisciplinarity has made it the perfect place for me to develop as an academic. In the late 1990s, my obsession with experimental film and video led me to undertake a PhD across two departments, analysing and producing work through computation, in an attempt to progress the sci-art fusion that inspired the earliest computer art.

The idea of programming computers to analyse and create art was considered unfashionable by many at the time. But thanks to academics such as Janis Jefferies, Robert Zimmer, Michael Casey and later, Mark d’Inverno, this wasn’t the case at Goldsmiths. So I came.

I joined Goldsmiths from Kent University’s Film Studies Department in 2006, bringing with me an interdisciplinary AHRC Creative Fellowship in interactive filmmaking, computation and cognitive science. Although I was originally in Music, I soon found myself taking a permanent job in the relatively embryonic Computing department, as the director of the newly launched Creative Computing BSc programme.

Over the past decade, it’s been immensely satisfying to see the department develop into one that is as successful as it is interdisciplinary. I’ve been fortunate to be able to continue my funded research in Creative Technologies throughout this period, whilst helping to launch and maintain a number of academic programmes, including Creative Computing, MA/MFA in Computational Arts, Music Computing (with the Music Department), Digital Arts Computing (with the Art Department), and the PhD in Arts and Computational Technology.

Goldsmiths is now one of the UK’s leading institutions for creative code, attracting the finest academics in the field, whilst producing world-renowned research in interaction, creativity and machine learning. I’d like to think I’ve played a part in that transformation, and hope I can continue to do so.​

Mick Grierson and Matthew Yee-King ask how computer science can benefit from art school approaches to learning.

Mick Grierson’s website


This article was first published on the Goldsmiths staff intranet.