Welcome back! If you missed the first part going over the brief history of the tech being used to deliver "spatial audio" to headphones, please check it out below.
So I'm not going to bury the lede - Immersion is in the EARS of the listener.
That's it.
We can use fancy tools to put sounds around our listener - or use the simplest method of talking into a microphone to create an immersive audio story. But, in the end, it's up to the listener to decide if they are immersed.
Think about it - is the person sitting in the driveway with the car running until they hear the rest of a story on NPR any less "immersed" than a person listening to a Dolby Atmos mixed audio drama?
So with that in mind, let's return to the question I raised in the previous article - WHY use spatial audio tools to tell your story? Will using these tools create that driveway moment for your listener? It might! But not only because those tools are being used.
People get immersed in STORY and CHARACTER - the audio treatment of those components (like spatial audio) can enhance, detract or make NO difference to the kind of immersion the listener feels. But story and characters are the main tools to build immersion - and then the treatment.
For me - the reason for nearly everything I do in design and direction has to be in the story. And better still - on the page. Tools and techniques come later when we ask and answer, “what’s our intent”?
It doesn't matter if we're using Ambisonic recording techniques (Wolverine), Neumann Head recording (The Darkest Night), Dolby Atmos mixes (QCODE), or the many, many hybrid approaches (like I have used since 2017) if the story, the writing, and presentation don't lend itself to immersive USE of spatial audio techniques - simply putting sounds 360 degrees around our listener won't create more immersion for them.
Technical aside: Headphones are the great equalizer of spatial audio as all the tools output a similar-sounding Binaural headphone mix to deliver the 3D effect.
In-person, however, like a theater with multi-speaker arrays, the 3D audio effect can be truly remarkable with several tools operating in that space.
Dolby Atmos has the most buzz presently. But even Atmos delivers a Binaural mix to headphones. At that point, the spatial results between the tools are minimal.
Craft and intent still rule the day regardless of the tools used.
What kinds of shows lend themselves to the immersive use of spatial audio?
On the surface, Fiction seems obvious, more than Documentary maybe. More than celebrity chat shows, maybe. But it's not always an all-or-nothing approach.
I used spatialized audio in all my work (documentary and fiction) since Inside The Exorcist in 2017. But, even then, it was not the main dish - it was spice done for effect in an otherwise “normal” stereo production.
The reason this has been my preferred approach is two-fold. First and most importantly - I haven’t worked on many shows whose stories called for full spatial audio treatment. Second, writing for audio is hard - writing for Immersive use of Spatial Audio is still harder.
The second is more personal. I often haven’t liked the sound of spatial audio as the main dish in shows I’ve heard. Over time, I've realized this has more to do with the design approach and craft employed than the tech itself. I've heard some great spatial examples where story and design came together to create a rich sonic experience. But more often hear clumsy uses of the tools and techniques that don't seem narratively necessary.
So it’s a good idea to listen to as much spatial audio as you can. See what you like, what you don't like. What works and what doesn't. You'll find many design decisions are head-scratchers - like panning characters to the extreme sides for no narrative reason other than (Hey, look, I’m over here!)
You'll also get a sense of what it sounds like. Spatial audio definitely has a SOUND. The various tools all have different ways of recording or computing the binaural output you hear. You probably won't tell what tools are being used just by listening - but they all impart a certain quality. You may like it - you may not. Listen and be the judge.
Quick sidebar: Unfortunately, we all HEAR differently. It's true. It's based on physics and beyond the scope of this article, but the shape of your ears, their distance from each other, the size of your head, the distance from your shoulders all conspire to create the unique way you HEAR the world. It’s subtle, but it’s there. Presently mass-market spatializing software only delivers you a few models of what the world sounds like to an average-sized person. So unless you're close to average, some of this spatial audio stuff is just going to sound weird for you. For me, for example, binaural audio sounds like it’s coming from just behind my ears, leaving the center in front sounding empty.
Anyway - examine your story for how spatial audio might help tell the story and enrich the story world for the listener. A few starter questions:
Is there a lot of activity, movement, and action? Spatial audio is perfect for that.
Is there a lot of sonically interesting locations where you spend a fair amount of time? Spatial audio can deepen those scenes and put the listener there.
Is it narratively important for sound and voices to come from anywhere AROUND or above the listener? Spatial audio can do that!
Does it need to sound REAL? Spatial audio can help with that. But so can properly be recorded and designed productions. Listen to “The Truth” by Jonathan Mitchell, which has long set the standard for vérité style audio drama without the “3D” effect. I seriously doubt his listeners feel “less immersed” for that fact.
So the bottom line - examine if you have reasons built into your story to use Spatial audio. Is it narratively driven - or as an effect? Effects can be cool - I use them ALL THE TIME for moments that need special treatment!
I’m simply encouraging you to make a conscious and deliberate creative decision about spatial audio rather than succumbing to the hype, encouraging a few too many people to slap "3D Audio" onto their productions without deeper thought or intent.
Next time I'll go through examples from my work where Binaural audio was hiding in plain… uhh hearing? And if it’s “hiding,” - can it still be immersive?
Till then!
