DIGITAL MUSIC MAKING & THE RISE OF VR

“What is real? How do you define 'real'? If you're talking about what you can feel, what you can smell, what you can taste and see, then 'real' is simply electrical signals interpreted by your brain”  - Morpheus (The Matrix, 1999)

By: Haider Masroor and Francesco Perticarari

Introduction

From the early 20th Century until today, music has enjoyed a period of rapid expansion and innovation. With technologies developing faster than ever in history, an increasingly growing number of people reshaped or multiplied the ways music can be made, performed and listened to thanks to ingenious new uses of the latest available technologies. Today, one of the most promising avenues of technological development is Virtual Reality (VR), which is, therefore, a totally valid instrument to use and bring about the next breakthrough in music making

Innovative ways of making music through software

1951 - The first computer music recording
The oldest known digital music recording is from a Ferranti Mark 1 computer at the University of Manchester – it played God Save The Queen, Baa Baa Black Sheep and In The Mood. The computational revolution of music had begun…

1982 – Commodore 64 launched
When Commodore launched the C64 home computer music really started to get going. It was notable for the inclusion of its SID sound chip, which enabled users to create music using three channels of synthesis. As a matter of fact, the distinctive SID sound is still popular today, which is one of the reasons why you can download C64 emulators for your Mac or PC.

1983 – The birth of MIDI
Standing for Musical Instrument Digital Interface, the introduction of MIDI was of crucial importance to high-tech music makers. This new protocol enabled computers, synths and other hardware to talk to each other, and is still in widespread use today. It particularly proved invaluable in allowing music-making controllers and interfaces to develop separately from music generators and players.

1990 - Max (soon to be Max/MSP) is first released commercially
Max is a visual programming language for music and multimedia developed and maintained by San Francisco-based software company Cycling '74. During its history, it has been used by composers, performers, software designers, researchers, and artists to create recordings, performances, and installations. 
(image below: Letitia Sonami playing a self-made instrument/controller built with springs, contact microphones, and a digital core coded in Max/MSP)

1989 – Steinberg releases Cubase
Steinberg had previously achieved recognition with the widely used Pro-24 sequencer, but Cubase is the software that made its name. The first version introduced the concept of the 'arrange page' with its vertical list of tracks and horizontal timeline – a design that quickly became the standard interface for all commercially developed sequencers. Cubase Audio for Mac also allowed sound recording in 1991.

2000s – Ableton releases Live, Traktor and other software start laptop DJ-ing
Native Instruments Traktor software was one of the first to recognise the potential in software-based DJ-ing. Exploiting the huge libraries afforded by using computers, along with the huge display, it made digital DJ technology more powerful, more flexible and more intuitive.
In 2001 Live broke the mould when it was released, and quickly made both users and developers realise that there isn't just one way of making computer music. It looks and feels like no other software on the market, and is now used by musicians of all sorts around the world.
Though it wasn’t originally designed for the purpose, Ableton Live has, perhaps, done more to change the way DJs work than anything else.
Conceived as a live performance tool for electronic musicians, Live was quickly adopted by DJs due to its rock solid performance, and ability to auto tempo match audio files. Inadvertently, it also provided those using it with a new set of tools that made reconstructing, re-editing tracks on the fly an absolute breeze. 
The protocol OSC was also developed and is often used as an alternative to the 1983 MIDI standard, where higher resolution and a richer parameter space is desired

2012 - AudioGL (Beta)
AudioGL is a virtual modular synthesiser and sequencer, which never made it beyond its beta.
Its key point as an innovative digital-audio-workstation was enhanced workflow, and it chose a 3D space as an innovative way to visualise, automate and interact with sound processing and sound generating elements.
AudioGL - 3D DAW project (currently suspended)

2012 - Music Gloves Ltd develop The Gloves (featuring Imogen Heap)
The Gloves are a cutting edge experimental gestural music hardware being developed today for the purpose of Imogen Heap's studio and stage work. 
Using a unique gestural vocabulary, motion data-capture systems, and user interfaces to parameter functions developed by Imogen Heap and her team, artists and other users will be able to use their motion to guide computer-based digital creations. The Musical Gloves are both an instrument and a controller in effect, designed to connect the user fluidly with gear performers usually use, such as Ableton - think minority report for musicians brought to you by the DIY/maker revolution.
(Image Below: Imogen Heap wearing The Gloves)

2012? - Simon Katan presents Cube with Magic Ribbons
Cube with Magic Ribbons is a computer visual and synthesised sound composition for live performance developed by computer artist Simon Katan. The piece was created using a custom visual sequencer: SoundCircuit, which rather than employing a conventional DAW layout, allows multiple virtual tapeheads to travel through a two-dimensional wrapped space along tracks that can be freely interconnected. SoundCircuit has never been released to the public.
Other software developers have also created innovative sequencers which are either nonlinear or polyrhythmic (or both). A few beta versions of circular drum machines and other non-standard approaches have been released as add-ons for Ableton Live coded in Max (Max 4 Live). None has so far been released commercially.

VR: from a sci-fi dream to a reality for user interfaces

The earliest mentions of what we consider VR today trace back to 1935. Stanley G Weinbaum’s short story “Pygmallion’s Spectacles” described a goggle-based virtual reality system with holographic recordings of fictional experiences, including smell and touch. Since then VR or augmented reality have been the focal point of many movies and books eventually leading to the creation of the ever so popular Oculus Rift in 2014. Although there had been numerous attempts at creating a commercial VR device prior to the Oculus, the Oculus was made popular due to its application for gaming and was championed by YouTube gamers playing horror games, thus leading to some fairly entertaining viral videos.

Given its application for gaming, the Oculus also developed the Touch interface which consists of a pair of tracked controllers designed to deliver a "hand presence", which is something that Oculus VR has described as "the sensation of feeling as though your virtual hands are actually your own". Now with a sophisticated headset and an intuitive input system, one could now use the Oculus system for a multitude of uses, including music creation.

Existing examples of music experiences in VR

The reason we use the term experiences here is because when it comes to actual music creation, there’s only really one that’s available to the public, and that’s Soundstage for the HTC Vive.
Soundstage allows you to create and build your own studio virtually as you would in real life and go on to create music however you would like. But whereas this could be described as a VR DAW, for which you would require some sort of base knowledge of music production, when it comes to more accessible and game-like programmes, there’s a handful worth noting.

The problem we have here is that there’s nothing that meets the middle ground. You’ve either got games that don’t allow you to create your own original pieces of music, and then we have a sophisticated DAW which requires you to patch together your own studio. Wouldn’t it be great if there was a programme that allowed a beginner to understand the basics of music creation and to be able to pick up the headset and controllers and create music intuitively?

How is all this relevant for our own project?

Wouldn’t it be awesome if people could wear a headset, like Anklepants, Deadmau5, or Daft Punk, and enter a virtual world where they can make and mix music without being constrained by 2D screens? We think so! And we think we will be able to use the 3D space that VR provides to craft an amazing musical interface. One possible route this could take is in using screens and projectors to bring the audience into this world too - allowing all sorts of DJ-sets, live sets, and streamlined music making environments to be easily crafted and showed to the audience without relying on loads of physical controllers or stage machinery and effects.

References

[1] CSound Community - Computer Music History

[2] Paul Doornbusch - A Chronology / History of Electronic and Computer Music and Related Events 1900 - 2015

[3] MBryonic - 10 Best VR Music Experiences

[4]  Wikipedia - Virtual Reality (Retrieved October 2016)

[5] Byron Mallet - Controlling Live in Virtual Reality