Audio is a window into our world

Audio is a window into our world. It can also be a window into other worlds created by a filmmaker's imagination. For documentaries, there's no better way to set the scene; to make it seem real or feel like you're actually there.

We talk about our senses in a way that seems burnt-in to our languages. We tend to separate them as if they're all independent of each other. In reality, taste is closely interconnected with smell. And hearing is woven into the same cognitive fabric as vision, as it paints a credible picture of the world. Sound and vision are so closely associated with each other that it's hard to imagine a life with only one or the other of these profoundly important senses. Inside our heads, all of our senses contribute to a unified and compelling sense of "what's out there".

No one would want to give up their vision or their hearing. But imagine, for the sake of argument, that you have to rely on audio for a while. Don't worry about bumping into things or not knowing where you are - this is just an experiment. Now picture yourself in a meadow. What you hear is incredibly rich, even though there's nothing much going on. Insects buzzing, the wind rustling gently through the grass. Maybe sounds of a distant road, the occasional aircraft, and, all the time, audible clues about the environment. If you're in a field, you won't have much in the way of reflections - although if there's a stone wall, houses in the background or even fir trees with exposed trunks, you might hear muffled and dispersed reflections. You might not even listen to them distinctly, but the sophisticated processing in your brain will be painting an aural picture for you.

Sound fields

While it's true that there are point sources of sound, most of the time, we experience a sound field: the sum total of sounds from all directions. Record it with a single microphone, and most of the field will be lost. Without a spacial element, it's just a sequence of sounds. Record it in stereo, and it jumps into a space. A stereo soundstage goes a very long way towards establishing an audio painting of the recording environment. An excellent stereo recording can be incredibly real. One of the best ways to test a HiFi system or a studio monitoring set-up is to listen to the stereo soundstage. A good recording that's reproduced well should create a soundstage where you can "picture" the layout of instruments in an orchestra.

Add more channels, and you can have depth and height as well. Recording with a "dummy head" can give you stunning binaural reproduction on headphones with incredible spacial information. Ambisonic recording and reproduction can almost have a laser-like precision. Newer formats like Dolby Atmos allow you to place sounds anywhere - up, down, front, back, right, left: with almost complete freedom. So you now have a choice between authentic and made-up sound fields. Both are valid in the proper context.

Our inclination to talk about our senses as if they exist independently of each other is firmly embedded, perhaps because sound and vision require totally different "sensors" or "transducers" to capture the sensory data. But inside our heads, we re-integrate these streams of sense data. We are powerfully aware of our environment, even if it's a complete fabrication while we watch a fantasy or science fiction film.

When CDs first arrived, some thought they sounded harsh and 'edgy'.

Sound is remarkable

Sound has some remarkable and quite surprising abilities. For example, how can we hear the sound of an entire orchestra when a loudspeaker cone can only be in one place at one time? It's baffling until you realise that sound is about pressure and time. To be precise, it's about amplitude and time. It's different from images, which are spacial and not temporal (although, of course, temporally changing images are what we call film or video). So when you listen to an orchestra (or a crowd at a football match or the dawn chorus), all the pressure waves in the air add and subtract. Eventually, what hits your ear is the sum of the pressure waves at an instant, in a single place, and the way these pressure waves change over time will determine what we hear.

Typically what we hear - or what we perceive - is a soundscape. It can be as detailed as any image if your hearing is good. Dynamic range in audio can be even more dramatic than in video. Imagine sitting in a field listening to the grasshoppers when a jet fighter flies past at tree-level. Think of the chirp of a finch followed by the rumble of distant thunder. Sound can fill a gigantic or a tiny space and everything in between.

Digital sound

The most impressive thing when digital audio burst onto the consumer scene forty years ago was the silky silence between the tracks. The audio sounded a bit "edgy" because the analogue side of the digital audio chain wasn't very subtle. That didn't stop it from being impressive, and CDs from back then played through modern equipment can sound very good indeed.

Making a digital recording is every bit as critical as shooting a film. A single sample error can ruin the entire piece. You can't really "hear" a missing sample: you almost "feel" it as something slightly wrong. If there are many errors, it feels like an unpleasant pressure in your ears.

Amazingly, an audio pulse the length of a single sample can paint the most detailed pictures of a physical space. A single "impulse" of sound reflected off the walls, floor and ceiling of, say, a cathedral can be captured as an "impulse response". Feed this through a DSP process called convolution, and you can apply these wonderfully authentic acoustic "signatures" to any sound. Ironically, completely artificial sounds like those from a synthesiser can sound amazing, fed through a convolved reverb.

Sound is very giving - and forgiving. It can fill a space with crystal clarity or lend itself to extreme distortion or glitchiness for artistic effect. Sound can act like human timecode: dancing to music is one of the oldest social activities. Recent research shows that it can even help restore lost limb function in stroke victims, who are given music to listen to on headphones while they re-learn to walk.

We often hear that sound is the poor relation of video and film. Sometimes that's the unfortunate truth when budgets are tight and creative, knowledgable sound professionals are left out of the crew. But it needn't be like that. It's never been easier or cheaper to buy outstanding professional audio gear. But we need to ensure that audio is taught with the same emphasis - and excitement - as camera craft.

Most of all, we should never underestimate the importance of audio in creative works, and we should never forget that sound can paint some of the best pictures.