Teaching VR in a MOOC

I’m really excited that we have recently completed the launch of a MOOC Specialisation about Virtual Reality that Xueni Pan and I have been developing with the University of London International Programmes (thank you to their amazing production team). The course will be on Coursera starting on the 25th September.

Developing this course has been an opportunity for me to reflect on my work for the last 15 or more years and bring Xueni and my experience to a new generation of VR developers.

I’ve been very privileged to work with some of the worlds leading VR researchers, mostly during my time at Mel Slater and Anthony Steed’s lab at University College London. It was a very exciting place to work and one that combined the technological development of VR (more my area) with experimental work that established our understanding of the psychological experience of VR leading to Mel’s theory of the three illusions: Place Illusion, Plausibility Illusion and Embodiment Illusion (a key underpinning of our MOOC).

When I left UCL I joined Goldsmiths, University of London and there I was lucky to participate in the development of a new approach to teaching computing, centred on interdisciplinarity and creativity. It was there that I really started thinking about pedagogy: how creative and individual practice can drive people to improve their coding skills.

The final piece of the puzzle was when, in 2013, we worked with Coursera and the University of London International Programmes to develop “Creative Programming” one of the first batch of MOOCs to be released from an English University. Making this MOOC really opened my eyes to the possibility of technology for education, using automatic grading and peer assessment to provide fast, constant feedback to students to improve their learning. This experience has informed both my online and on campus teaching since. The online technologies that I have been able to bring to my students at Goldsmiths have supported their learning and enabled personalised experiences even in very large classes.

The Virtual Reality Specialisation brings all of this learning together. VR feels like such a new medium, but it is one that is founded on decades of research on the technology and the psychology of the VR experience. This specialisation allows Xueni and I to share our experience and knowledge of this research with a new generation of VR creators. People are struggling to understand VR, particularly as it is so different from existing media. There is still a lot we don’t know, but actually, if you look at past research, there are a lot more answers than people think. We hope that our MOOC will help people find those answers and start their career as VR creators. VR right now is crying out for good content, which means good content creators. People making VR now are pioneers in the way film makers were in the early 20th century and web developers were in the 1990s: they are not only creating work, they are creating the basic grammar of the medium itself. We feel very privileged and excited to be able to help people get a start in developing what could be the most important medium of the 21st century.

Designing natural gesture interaction for archaeological data in immersive environments

We’ve published a new paper with colleagues in Pisa:

Archaeological data are heterogeneous and it is difficult to correlate between the different types. Data-sheets and pictures, stratigraphic data and 3D models, time and space mixed together: are only few of the categories a researcher has to deal with. New technologies may be able to help in this process, filling the gap between history and future, and trying to solve research needs with innovative solutions. In this paper, we describe the whole process for the design and development of a prototype application that uses an Immersive Virtual Reality system to acces archaeological excavation 3D data through the Gesture Variation Follower (GVF) algorithm, that allows to recognise which gesture is being performed and how it is performed. Archaeologists participated actively to the design of the interface and the set of gestures used for triggering the different tasks. Interactive machine learning techniques have been used for the real time detection of the gestures. As a case study the agora of Segesta (Sicily, Italy) has been selected. Indeed, due to the complex architectural features and the still on-going fieldwork activities, Segesta represents an ideal context where to test and develop a research approach integrating both traditional and more innovative tools and methods.



Albertini, Niccolò; Brogni, Andrea; Olivito, Riccardo; Taccola, Emanuele; Caramiaux, Baptiste and Gillies, Marco. 22 May 2017.Designing natural gesture interaction for archaeological data in immersive environments. Virtual Archaeology Review, 8(16), pp. 12-21.

We’re hiring

Edit: just added a lecturer in Physical Computing, below. 

I’m very excited to say that Goldsmiths’ Computing is now hiring, with a number of lectureships now available, plus our innovative Post-Doctoral Teaching and Research fellowships.

We have general lectureships in Computer Science and Computational Arts, and specialist games lectureships.

We have one full time lectureship in any area of either Computer Science or Computational arts and a 0.5 lectureship in either field. We particularly welcome applications that relate to some of our key teaching and research specialisms like Physical Computing, Data Science/Machine Learning/Interactive Machine Learning or Audio-Visual Interaction (this list is slightly biased to my interests, see the advert for a full list of specialism).




A lectureship in Physical Computing for some one who can teach arduino and related:


We have two Video Games/Graphics lectureships, one in programming (with a focus on C++ and graphics) and the other (0.5) in  Art and Design:



Finally, our Postdoctoral Teaching and Research Fellowships provide a fantastic opportunity for those of you who have just finished or are writing up a PhD and want to take a first step into an academic career. You would have a split between research and teaching time, just like a full lecturer. In your research time you would be free to pursue your own research interests as you please (including writing up and publishing your PhD) but would do so within one of our research groups that would provide mentorship from eperienced academics, particularly with applying for funding. In terms of teaching, you would be responsible for planning and delivering sessions within major undergraduate courses, but you will always be team teaching with an experienced academic, who will mentor you in your transition into teaching.


Please consider applying, we are really excited to be joined by new, dynamic academics.

Pedro Kirk’s “Can Specialised Electronic Musical Instruments Aid Stroke Rehabilitation?”

Everyone here at Goldsmiths is extremely proud of Pedro Kirk’s for winning the ACM Chi student research award for his undergraduate project in Music Computing. This is a fantastic achievement,  winning such a great prize at a conference that is so important to us.

Pedro developed custom tangible  musical interfaces to aide strike patients in their rehabilitation therapy.  A largish study with patients suggested that they could be promising for increasing patients motivation. You can read the paper here at the ACM digital library:


Pedro is currently doing the MSc Music Mind and Brain here at goldsmiths and I hope he gets the opportunity to carry on this work after he finishes. Congratulations to Pedro and beat of luck in the future.


Jonathan Hook: Participatory Design with Artists

I’ve recently been talking to Jonathan Hook at the University of York about his fascinating work on participatory design with creative teams.

I thought I’d share one of his papers A VJ Centered Exploration of Expressive Interaction. It’s a really interesting piece of work in the way it engages with participants creative skills by asking them to produce a creative response to the research (in this case a video). Artists generally deal with concepts that are hard to express explicitly in words, but are also very good at creating (implicit) artistic expressions of these concepts, and Jon’s method makes good use of this.

This engagement with creative skills also raises the question of the relation between researcher and participant. In participatory design this can very much blur the traditional scientist/subject distinction, and particularly when working with artists, it makes a lot more sense to frame ourselves as collaborators, both of whom are learning from each other. The question then becomes how to frame it when you publish the work in a way that is comfortable and acknowledges this relationship without making too much of an issue of it.

Mocap, Oscars and Apes

Last week I went to the Human Interactive where Rich Holleworth of the imaginarium  gave a great talk, which was an animated history of mocap, and we had a really good chat afterwards.

This was very much in my mind when Mark Bishop forwarded me an article: Should Oscar go to Andy Serkis or the computer that turned him into an ape?

It addresses some interesting issues, but there are also some real problems with the article that I wanted to respond to.

Firstly the title. It plays on a typical trope of the human/computer divide, but this kind of performance is really not just about Andy Serkis and a Computer. It Andy Serkis and a bit team of highly talented and creative animators, mocap technicians and programmers (not least of whom, Rich).

The article quotes Jeff Bridges as saying “Actors will kind of be a thing of the past, We’ll be turned into combinations. A director will be able to say: ‘I want 60 per cent Clooney; give me 10 per cent Bridges and throw some Charles Bronson in there’.” I also think that this quote completely misses the point. What Andy Serkis has done is really prove that acting is still central to good film making in the era of mocap and CG. Also, the purpose of CG is really not about getting 60% Clooney and 10% Bridges. It was about getting 100% Cesar, with Serkis as a person being (literally) hidden. Andy Serkis may be a bit of a star, but one that no body can recognise, and no director hires him to “be Andy Serkis” as they might for other movie stars. It may be pointing to a new role of acting which is much more focused on character, rather than on stars. That is by no means a bad thing.

Maybe what this is really about is that calling into question the cult of the individual that pervades hollywood (and is implicit in the Oscars). Maybe we can’t pinpoint the single “genius” behind Cesar in planet of the apes, as it was the creation of a large team, but that doesn’t mean it isn’t a good performance or a good film (I haven’t seen it so I won’t comment on that). In fact, more traditional movie acting isn’t too far from that either. Jeff Bridges brings a lot to a performance, but it is also made by makeup, lighting, camera work, editing. etc. etc. The medium of film is inherently a collaborative enterprise, the work of The Imaginarium simply highlights this more.

Creative Programming for Digital Media & Mobile Apps

Or MOOCpocalypse Now.


After months of hard work and tight deadlines our Massively Open Online Course, Creative Programming for Digital Media & Mobile Apps, will be launching on Coursera on Monday with over 70 000 students already enrolled. The course will:

teach you how to develop and apply programming skills to creative work. This is an important skill within the development of creative mobile applications, digital music and video games. It will teach technical skills needed to write software that make use of images, audio and graphics, and will concentrate on the application of these skills to creative projects.  Additional resources will be provided for students with no programming background.

The course is being taught be Mick Grierson, Matthew Yee-King and me.

Thanks to everyone who has supported us, including Niklaas van Poortvliet of UCL Publications and Marketing Services (PAMS) for the epic amount of work he and his team have put in to produce the fantastic videos. Barney Grainger and Michael Kerrison of the University of London International Academy for all the help and support they have given through the creation of this course. And of course all the current and former Goldsmiths students who will be helping support you on the forums: Vlad Voina, Tom Rushmore, Will Gallia and Joe Boston.




Goldsmiths Masterclass

Welcome to everyone attending our masters classes at Goldsmiths this week.

You can see our schedule here:


Today and tomorrow I’ll be running the computing masterclasses and will be looking at some of the exciting new web technologies that have been developed in recent years. The main focus will be on HTML5. We will be using some of the HTML5 example developed at Goldsmiths that you can access here:


We will also introduce Processing, a great programming environment for rich media interactive web sites, and the main teaching language we use in first year at Goldsmiths. You can download Processing here:


and the documentation is here:


If you are interested you can look at some examples.  I will be showing this today:



and this is a great resource of processing examples:



Bruno Latour’s challenges for CHI

Bruno Latour just gave the closing keynote for CHI 2013 and he issued four challenges for HCI research. I thought I would get down some thoughts about them before I forget.

Before I talk about the challenges, I should try to describe his central theme. He was arguing against the division of sociology into two scales the unconnected individual and the unindividualised collective. He instead argues that we should think in terms of overlapping and interconnecting “monads” (I won’t try to explain the term). He thinks that digital technology can help to analyse data without going to the two poles of individual qualitative datum or collective, aggregate statistics.

This kind of aligns with my thoughts that interactive machine learning could help to bridge this divide by having human interaction that focuses on the detail of different aspects and items of data within the statistical analysis of machine learning (this is still very vague on my part, but I think there is something there and maybe Bruno Latour does too).

Overall, Latour wants the CHI community to help break down the individual/collective polarity, but in particular he issued four challenges:

Getting rid of data

His first challenge was to help get rid of data from large data sets (presumably so you are only left with “interesting” data in some sense). Given the rest of this talk I interpret this not as wanting to focus on individual items but to pick up connected elements that are important without either aggregating all of the data or removing them from their connections with the rest of the dataset. I can imagine that there could be powerful tool that allows researchers to investigating small snippets of data while a statistical engine runs in the background, clustering or otherwise picking out connections between those snippets and the rest of the dataset.

Capturing the inner narrativity of overlapping monads

I will have to think about this one, but it is something about bridging the gap between human narrative and statistical analysis. He referred to data journalism and how data is used in both interactive and narrative contexts in things like the guardian coverage of the london riots.

Visualizing heritage, process and genealogy

How to visualise these temporal qualities without relying in static structure or loosing connectedness. While answering questions he stressed the importance of not falling back on unchanging structures but acknowledging the changing nature of monads and their connections. This seems to me to relate to another theme that came up quite a lot in CHI (from Bill Buxton to the NIME SIG), the need to have time as a first class concept. This would make it possible to model the evolution of data without relying on static structures (maybe).

Replacing model building and emergent structure by highlighting differently overlapping monads. 

I guess that this would require a very dynamic analysis that made it possible to apply many different and changing models to data. I think that interactive machine learning could help a lot here by using the human element to navigate different interpretations, learnt models and views of the data.


Nate Matias has a much more accurate write up of the challenges. I’ll quote them below, but I’ll just warn that I think he’s not quite right about collective phenomena. He says that “Collective phenomena grow out of these collecting sites”, but this isn’t quite what Latour is saying, after all collective phenomena don’t exist so they can’t emerge. Latour is from a Science Studies background so when he talks about collecting sites he is thinking about scientific data collection instruments (or their aggregations) like microscopes, telescopes, mass spectrometers, semi-structured interviews or surveys. While we might think of a survey as collective and an interview as individual they are both just methods of collecting data which both observe and transform (“perform”) the data thus creating a particular view on the world. There is no innate distinction between individual and collective phenomena in the world, just phenomena that are created by methods of data collection (or more precisely by their interaction with the world). This means that there isn’t a division into two (or more scales) collective and individual but a mass of different views of the world, each specific to it’s data collection method.

Anyway, that small criticism aside, well done to Nate, for the otherwise excellent explanation, here is his summary of the challenges:

Visual complexity produces opacity. Massive individualizing data produces beautiful, playful hairballs which show us nothing. How do we get filter and focus data while still appreciating monads?

How can we capture the inner narrativity of overlapping monads? Latour shows us the “512 paths to the White House” visualization by Mike Bostock and Shan Carter. The other, the Guardian’s Rumour tracker, following the 2011 London riots. The idea that quantitative is different from qualitative is an artifact of the history of social science and a fallacy arising from the distinction between the individual and the collective, he tells us.

How can we visualize heritage, process, and genealogies? Latour shows us a paper he worked with on “complex systems science” (I couldn’t find it). To be a monad is to establish connections, but timeseries visualizations can focus on structure rather than connectedness (like the paper on Phylomemetic Patterns in Science Evolution by Chavalarias, Cointet et al)

How can we replace models about emergent structures with models that highlight differentially overlapping monads? He shows us a hairball network diagram and talks about the difficulty of moving beyond the hairball to understand the overlapping monads