Practical Applications of Fourier analysis in Sound and Music
Ryan Jordan
Computing and the Arts
Tutor: F.F. Leymarie
Goldsmiths Digital Studios
March 2008.
Introduction
Composing and performing electronic and computer music is a thoroughly rich area today as it has been since the first electro acoustic compositions of the 1950’s and 60’s. When artists began using machines and most notably computers, the language and subject of music shifted away from ‘classical’ notation and instrument playing to a greater ability of controlling and creating ‘new’ sounds and timbre. The computer enabled composers and practitioners not only to play sounds but to critically and mathematically analyse the very nature of the sound itself.
There are various methods of analysis but this paper looks at Fourier analysis and the FFT algorithm as it is a core component of many musical tools: spectrum analyzers and resynthersizers, frequency-domain pitch detectors, spectrum-based filters, and convolvers (Roads 1996). We shall explore the basics of Fourier analysis and the maths behind it and look at composers and music which employ methods of spectrum analysis.
Introducing Fourier analysis
A theory of Fourier analysis was published in 1822 by French engineer and aristocrat Jean Baptiste Joseph, Baron de Fourier (1768 – 1830). The theory he stated was attacked by established scientists fifteen years previously, commenting that it was mathematically impossible.
Fourier analysis represents signals as sums of sinusoidal waveforms, each at a different frequency, amplitude, and initial pulse. A good example of this is from Kevin Cowtan’s Book of Fourier , where he explains Fourier Transforms in Crystallography but it works the same when analysing a sound wave.
For example, if the initial sound wave looks like this:
To begin building this wave from individual sine waves, the first sine wave will have a frequency of 2 (there are two oscillations):
The second sine wave has frequency of 3 but also will have different amplitude (the height of the wave) and the phase will be offset (a different place on the wave):
Finally we introduce a third sine wave with a frequency of 5 and again the amplitude and phase is different:
The sum of all three of the sine waves is a close approximation of the original:
Sine waves are used because they have a small number of parameters and when added together peaks and troughs are emphasised as explained in the above diagram. When dealing with sound, the sine waves used are harmonics of a fundamental frequency, with the fundamental frequency in this case of frequency 2. This is not an exact representation but by adding more and more sine waves with varying frequencies, amplitudes and phase then we begin to get closer and closer to the original until they are perceptibly indistinguishable from each other. The process of decomposition of the original signal is specifically referred to as Fourier analysis. The recombination of sine tones to create the original signal is called Fourier synthesis.
This is the basic idea behind Fourier analysis/synthesis. Without delving too much into the mathematical functions of Fourier analysis we shall explore it in a bit more detail and how its application is used for music, sound art and audio visual applications.
Overview of the Maths
We have seen how the decomposition and composition of sine waves of varying frequency demonstrate how the Fourier theory works. Each frequency can be represented by a complex number with polar or rectangular coordinates. A complex number can be plotted on a two dimensional plane who’s horizontal dimension is labelled the real axis, and vertical dimension the imaginary axis. The rectangular coordinate is the point where a, b intersect, with a and b being values on the real and imaginary axis.
Another way to represent a complex number is as a vector, defined by its distance from the origin and its direction or angle from the origin. The distance and the angle give the point’s polar coordinates. The point of origin is also called the amplitude, magnitude, or modulus (Roads, 1996).
Because we are working with sound and music we must add a third dimension; time. The angle of the vector rotates around the circle. One revolution around a circle is equal to an angle of 2Pi radians. Here we enter into the Radian system and Circular functions. When a regular rotating point on a circle is plotted along an axis of time we see a sine wave. Frequency can be measured by radian velocity; the higher the velocity, the higher the frequency. The magnitude of a sine wave can be measured by the size of the radius of the circle; phase as the starting point for rotation; signals as complex exponentials. These all relate to a theory by the Swiss mathematician Leonard Euler, called Euler’s relation. Euler’s relation states that the quantity e raised to the power of an imaginary exponential is equivalent to a complex sum of sinusoidal functions (Roads 1996). For a more detailed explanation of these functions I refer you to the Appendix of The Computer Music Tutorial, Roads, 1996.
Fourier analysis has various forms depending upon what type of signal is being processed, from analogue signals to digital. There is Fourier Transform (FT), Discrete-time Fourier Transform (DTFT), Discrete Fourier Transform (DFT), Short-time Fourier Transform (STFT), Inverse Discrete Fourier Transform (IDFT), and Fast Fourier Transform (FFT).
There was a need in the 1960’s for a quicker version of DFT for real-time sound analysis and synthesis. Cooley and Turkey (1965) demonstrated a way to speed up this function making FFT between 10 and 1100 times faster than DFT. FFT is basically a compact algorithm of DFT and is now used in real-time applications thanks to the advancements and demands of the concert stage and live performance.
FFT is useful for techniques such as Additive analysis and resynthesis where they replace filter banks. Granular synthesis, where the sound wave is chopped up into tiny grains also uses techniques from Fourier analysis. The granular representation is implicit to the windowing technique applied in STFT. STFT can be computed by using FFT. The grain in this case is a set of overlapping analysis windows and we can view them in a grid. The window is merely segmenting the incoming signal into smaller bits which can be computed. Many types of window functions can be made. There is usually a selection available which includes Rectangular, Hamming, Hanning, Gaussian, Blackman, Blackman-Harris and Kaiser Window types. Each affects the sound in a slightly different way as a window acts like an envelope with different attack and decay.
For a detailed explanation of these I again refer you to the Computer Music Tutorial.
FFT in Music
The application of FFT in music and later with audio visual works was hinted at in the late 1800’s and early 1900’s by various people such as Helmholtz, Busoni, Russolo and Varese. They weren’t specifically intending to use Fourier series in composition but to break away from the constraints of the traditional octave and to explore a new musical space and freedom. In the 1960’s and 1970’s there was a large growth of the use of computers in composition and performance. One genre which originated from France, and mainly IRCAM, Paris, in the 1970’s is Spectral music. Although the term can be applied to more musical genres and is in some ways constraining we shall look at it briefly because of its relevance to FFT. Spectral music is a technique and approach which prefers emphasis on timbre rather than traditional pitch and rhythm. As the term implies, spectral music is concerned with the frequency spectrum of a sound; its internal nature. Through employing FFTs the composer/performer is able to see, select, synthesis the original sound. The composer is able to change the timbre of a sound and create complex hybrid timbres using cross-synthesis. The idea of spectral morphology, where you can analyse the morphing of a sound spectrum over time is made possible via this method.
Composer Tristan Murail analysed traditional instrument tones and created synthetic components to these tones that blend seamlessly with the instruments but weave out dramatically when the instruments stop in his 1983 piece, Desintegrations (Roads, 1996).
In recent years many musicians and artists have used FFT and variations of Fourier analysis as the application is useful and rich for synthesis and analysis so it has become an integral part of computer music tools. Computer musician Mokira has even released and album entitled FFT Pop. As the computer became quicker and more efficient at processing signals it is also possible to visualise sound through FFT. Go to nearly any commercial music software and there is usually an analysis page with FFT graphs and choices for windowing, sample rate etc. FFT is especially useful for VJ’s using a live audio input as they can set specific images, animations etc to trigger when specific frequencies are present. Programmes such as Processing using the Sonia library take an FFT analysis of real time audio input and allow the used to map the data to an image or animation. Very basically it can display a wave form but then you add to that and create a new ‘living’ picture which responds to live audio input. Below are two screen shots from a sketch of mine written in Processing using live audio input.
Figure 1. Stills from sketch in Processing using Sonia library and FFT.
Limitations of FFT
FFT analysis is slightly constrained in real-time as it changes the representation of information from the time to the frequency domain. Analyses of short durations will show poor frequency resolution and analyses of long durations will show finer grained spectral information. FFT gives poor frequency resolution in lower octaves and too much in the upper (Rowe 2001).
Conclusion
This paper has looked at Fourier analysis and how it has lead up to FFT – a central procedure in sound analysis (Roads, 1996). We have seen that the decomposition and composition of individual sine waves represent and can make complex wave forms and how these wave forms are then studied and used in composition and real-time performance. We explored how FFTs are used in synthesis methods such as Additive and Granular synthesis and how these techniques are being employed by musicians and artists exploring the internal natures of sound and developing other languages for musical expression and understanding different to that of traditional notation. We briefly looked at audio visual representations of a sound via FFT and can see that this leads to aesthetically pleasing results.
The use of FFT in music and the arts has come a long way from mathematical theory and laboratory experiments to being deeply embedded into music and visual software, allowing artists to use these powerful tools without having to deal with the mathematics; and we have seen how the arts and music can influence mathematical and scientific development by its demands for quicker real time tools for use in live performance situations.
Bibliography
Books:
Cook, P. R. 2001. Music, Cognition and Computerized Sound: An Introduction to Psychoacoustics. Cambridge: MIT Press.
Roads, C. 1996. The Computer Music Tutorial. Massachusetts: The MIT Press.
Rowe, R. 2001. Machine Musicianship. Massachusetts: MIT Press.
Wishart, T. (S. Emmerson, ed.) 1996. On Sonic Art. Reading: Harwood Academic Publishers.
Web:
http://processing.org
http://en.wikipedia.org
http://www.ysbl.york.ac.uk/~cowtan/fourier/ftheory.html