More selected projects

                                                                         Synaesthesia

by Marlon Youngerman

Synesthesia is a phenomenon where stimulation of one sense will be experienced as stimulation of different sense. There are numerous forms of synesthesia, the one we will interest ourselves with is chromesthesia, when hearing sound produces a phenomenological experience of color. In the art world, this phenomenon but also the idea of it has led to diverse and sometimes fruitful paths, notably we can mention the collaboration between the composer Schoenberg the painter Kandinsky. Some artists (who are chromesthet) have attempted to give an accurate account of it, drawing what they see, others (and they are more interesting) aim at translating the experience of music to a visual form.   Synaesthesia is one such attempt using digital tools and techniques of precdural generation. 

Visual art has traditionally been fixed in time, as opposed to music which is a time-based art form. Furthermore, if an art would be a visual equivalent of music it would have to be abstract because music (if we exclude lyrics) is a purely formal art form. Let’s see those who have gone in this direction throughout history.

Oskar Fischinger, An Optical Poem (1938), [1] an early attempt at transposing a sonic experience to an optic one, with the limited technical possibilities offered by his epoch (he uses stop motion technique), the visuals he produced are synchronized with the music it accompanies (Franz Liszt’s 2nd Hungarian Rhapsody).

At the beginning of his film he put this panel:

This is his clearly stated goal taken up by him and others later on, and which I partly take up myself. Even though my project is not as directly linked with music as his (Fischinger created his video based on a music and thought as a complement to it), I still see it as close to music in that it is time based contrarily to most visual art.

Some of Norman McLaren’s works [2] (his sounds and visuals produced by directly drawing on the film pellicle), exhibiting a playful interaction between sound and visuals, note that whereas in Fishinger the music had clear predominance, here both are on equal position.

The John and James Whitney brothers [3], pioneers of digital motion graphics, John Whitney also talks about his work as a sort ‘visual music’. But music or sound itself has completely disappeared from his work, his visuals are by themselve evocative of music.

 

Outline of the functioning of the program

There are 4 templates of images: Grid, Triangle, Square, Circle (see below and example of each).

Each base image “template” has different parameters which are randomly set for each individual one, those could be for example the distance between the bars of the grid, their thickness etc…

All colors are obtained using common methods of procedural colors generation (triad, complementary…), however all images which are displayed at the same time share the same palette.

This ‘base’ image is passed through an effect (kaleidoscope) and drawn to the screen. Each effect can take itself or another effect as it’s input, when that happens more complex geometrical structures appear as a result. the way the program generates visuals is loosely inspired from the way sound can be generated with modular synthesis.

Below the program runing (this is a slightly earlier version than the one presented at the gallery)

Gitlab repository: http://myoun013@gitlab.doc.gold.ac.uk/myoun13/Synesthesia.git

Reference:

[1] https://www.youtube.com/watch?v=they7m6YePo          

While he is probably the best others at the same period others were doing similar things : Walter Ruttmann, Hans Richter, Viking Eggeling, Mary Ellen Bute

 

[2] Examples: https://www.youtube.com/watch?v=qJwfeG3Mntk

 https://www.youtube.com/watch?v=XiBiO66pOqg

 

[3] Some of their work: https://www.youtube.com/watch?v=ryKT1uIPIaI,

https://archive.org/details/experimentsinmotiongraphics  where John Whitney makes some interesting remarks on his motion graphics.