Building a platform for the best immersive audio experience
Content
What is an immersive audio experience? Following on from my article Why immersive storytelling works best with 3D sound I was reached out to by one of the co-founders of Ammersive . They are using 3D sound design to tell stories and immerse their listeners. They shared more on why they started their company and their approach to content creation. Since I already produced a lot of immersive audio dramas in 3d audio, I thought it’d be interesting to learn from the experience they made.
Inspiration?
Their journey began by being inspired by one of the earliest examples of binaural audio that can be found. The Virtual Barber shop was created by QSound Labs in 1996 for a QSound client. The video with over 38m views can be found on YouTube here
Close your Eyes
The video title includes ‘Audio… use headphones, close ur eyes’. It is this aspect of closing your eyes and being immersed in the audio unfolding around you that stimulates your senses and entices your imagination to kick into life.
A reason why some augmented reality experiences don’t work is that your eyes still tell you where you are at the moment. While the ears could have the best immersive audio experience this is not how immersion works. So if we take away the visual sense, this is where magical audio experiences can happen.
Focus on your Ears
Your sense of hearing can really help orientate you to your surroundings. 3D audio opens up a dimension around and also above and below the listener This approach to spatiality when used in creative ways can transport listeners to a variety of situations.
The virtual barber shop is a great example of how audio can play with your perception. In my opinion it meets the sweet spot of technology and content.
3d audio platform?
The audio quality of the virtual barbershop is not great compared to today’s standards. It got delivered on YouTube and probably ripped from somewhere on the internet. Making it lose some of the high fidelity. The spatial element still achieves its desired effect, most likely through a binaural microphone set-up. But nowadays there are more possibilities to make immersive audio experiences. As you can have new technical recommendations
what’s the best platform for 3d audio?
Ammersive’s co-founders searched for a platform that offered a variety of 3D audio sounds that could be experienced similar to this, yet did not come across anything that satisfied this desire so decided to create this platform themselves. It’s great that all of the big social media platforms at least support stereo audio. This can be used for binaural audio, but this is just the beginning.
After exploring what is available in 3D audio, a search on YouTube will show you videos boasting ‘8D’ or ‘12D’ audio as if there was a superior to a full 360 soundspace. All of these of course being uses of 3D audio, most examples being music that is simply moving the whole track around the 360 soundspace. Yet the popularity is evident with several million views. This is most likely due to the novelty factor of experiencing audio in such a way where it moves around you as a listener. A novelty that will quickly fade as the prevalence of 3D audio increases.
What hardware is used for 3d audio?
An important aspect of 3D audio that can not be ignored is that its effects are best achieved via earphones / headphones or a surround sound system. This opens up the plane of speakers for the sounds to move across.
Technology of this type of hardware is improving all the time. The quality that can be achieved on even low priced headphones is generally superior to what was available only several years ago. This trend will continue with the emergence of spatial audio.
Dolby Atmos and more immersive audio content creates a need for more people consuming content that is 3D audio. Next step is the implementation of dynamic head-tracking into the premium headphones and earbuds. With this technology we can have an even more immersive audio experience. Sounds like a marketing term, but it does help our brain to localice virtual audio objects and is based on our natural listening behaviour.
What business opportunities are there with 3d audio?
Ammersive have chosen to generate revenue via a freemium business model where all users can enjoy a selection of immersive audio experiences for free. But if you would like to become a premium member to unlock the full library of sounds and features – you can enjoy a free 3 day trial then there is a £4.99 per month or £39.99 annual charge should you wish to continue.
The creators receive 40% of all in-app subscription revenue based on the performance of their sounds. If you would like to chat to the founders or creators, there is also a growing community of 3D audio enthusiasts you can join on their discord channel too: https://discord.gg/NrqmWmGrCx
Content first!
Ammersive utilise sound design in the 360 space to stimulate both real life and fictional scenarios, they then encourage listeners to close their eyes, listen intently and see where their imagination takes them.
Most sounds are typically between 5 to 15 minutes long, intentionally short to suit the modern busy lifestyles of today. The focus here is on audio ‘experiences’. To generate content ideas there are several approaches taken to build an immersive audio library that ‘has something for everyone’.
Seasonal holidays and current trends are followed in an attempt to remain relevant and provide content that resonates with people. Immersive audio documentaries have also been explored to bring key historical moments to life that educate and raise awareness such as the Apollo 11 mission to the moon or Andrée’s Arctic Balloon Expedition. The narrated stories on the platform are mostly taken from the public domain and chosen to avoid copyright issues; this does however allow famous classics such as The Jungle Book or Wind In the Willows to be given a fresh new twist in a 3D soundscape.
Music, binaural beats, sound therapy and meditations are all also offered in the hope that they may support their listeners’ mental wellbeing. Ultimately, they feel it is important to push the boundaries of 3D audio and experiment with a wide variety of content. All sounds are created with the intent to offer a brief, enjoyable aural journey that ignites the imagination, instills a sense of nostalgia or simply nurtures the mind.
It may look that such an immersive audio experience is just a nice-to-have. Something that you’d listen to or create for entertainment. But guess what, such spatial audio productions can also be part of B2B applications.
I was approached by a Swiss event agency looking for a 3D audio experience that they could play during their virtual event. There is so much potential with spatial sound for virtual events that I explained in my previous article. The industry client decided for a livestream for I created an immersive audio experience.
It created an entertaining, but relaxing binaural audio drama that was based on the companie’s values. Team spirit, innovation, sustainability, engineering and DACH (the German, Austrian, Swiss area). So obviously it’s German with some dialect from the neighboring countries.
Like that the audience of nearly 3000 people could follow the livestream at home. But for 7 minutes they closed their eyes and immersed themselves into rich sounding worlds. As you know, industry presentations of two hours can be exhausting. So this was a perfect use-case of an immersive audio experience. It kept the audience engaged with interesting sounds. Created coherence with the storytelling. And brightened up the whole event. Here is what it sounded like:
Why use 3d audio for audio experiences?
3D audio can have a really powerful effect as it replicates a listeners real life hearing. The sound designers challenge is in the natural sound placement to the listener to replicate this. Or to accompany narratives there is use of both diegetic and nondiegetic sounds. Voice artists are also utilised for both purposes and this can really help to guide the listener or take them through a story.
There are also a series of short stories where non-diagetic sound such as the narration of the story is accompanied by the use of diegetic sound to bring the characters and their actions to life. For the co-founders, 3D audio is an enabler that can really help give the story some life and provide a much more immersive experience for the listener. The Petting Zoo is a particular favourite and has recently been entered into the BBC Audio Drama Awards.
https://on.soundcloud.com/XAoGT
The possibilities are endless with all real life and fictional opportunities available. The desire is to always utilise the 360 sound space to its fullest potential. This means the listeners can close their eyes and truly believe as if they are placed in the situation or that the experience is so immersive they briefly forget their own surroundings, as if to escape reality for a short while.
One could think you could just produce those stories in stereo and add some sound effects and music. But this is not how it works. If the storytelling and context is right, it can make sense to have actors walk around in 3d space. Unfortunately most of the time this is used as gimmick, as described here with 3d podcasts.
How can 3d audio be used successfully?
A key element of their creative process is usually starting with a storyboard. This ensures all of the sound design aspects are well thought through and purposeful. Creators are often encouraged to consider the listener as a protagonist of the story, putting them in the heart of the action.
Crafting the proper sound design or achieving the desired sound effect can require a lot of effort. Typically the hardest aspect to get right is the distance from the listener without the sound becoming too unnatural. The combination of sounds together is also incredibly important to get right and then finally the movement. Moving the sound across the 360 plane at a realistic pace can provide some of the most immersive effects.
“It is thin line between making movement with 3d audio – or becoming an annoying gimmick like 8d audio
We are all still learning
All creators are constantly learning how to improve their 3D audio creation skills. A plug-in that is extremely popular with creators is Dear Reality dearVR Pro plug-in. It provides a highly intuitive interface to move sounds around the 360 plane. Most people just throw their mono sounds in and call it “immersive audio experience” afterwards. But as I can tell you there are many more factors you need to consider to make it work: You can’t just take a random sound from your library and magically make distance work. It’s crucial what frequency curve is burnt into the file. the plugin can’t take into account the size of the object. The ocean for instance is not just a single audio object. It is rather a plane-wave. if you recorded in a studio, the performance will still sound like a studio. No matter how much fancy 3d audio design you add.
Binaural microphones and other field recordings are utilised to layer onto the productions. Here it is important to note that it needn’t always be binaural audio.The use of stereo is also used to support the experience if it is the right choice to make. The right choice of course being dependent on the desired effect the creator wants to have on its listener.
Conclusion on immersive audio experience
The main aim for Ammersive is to provide the listener with an escape from the everyday, a chance to put your headphones on, take some time to yourself and explore a realm of endless possibilities through sound
Utilising 3D audio in creative ways can be a lot of fun and opens up new possibilities in sound design. However, it is important to tread carefully and be mindful of the execution with consideration of your intended purpose.
Ammersive are on a mission to create a wide variety of immersive audio experiences that will entertain listeners and offer a brief escape from the daily stresses of life. Ideas for new sounds are actively encouraged and users can submit their own via the app.
When the use-case allows it, you can even go beyond entertainment. Like I created an immersive audio experience for the German bloodcancer aid. With immersive storytelling, listeners could feel empathy and understand why it is important to become a blood donor. Since the experience was in a container I could use spatial sound to create an “audio avatar”. So it really sounded like an actual blood cancer patient opened the door to tell his or her story. Read more about this immersive sound experience vs bloodcancer.
There are already some rule of thumbs out there. Like the point I made when criticizing the approach of New York times, linked here. But since the medium is so new I also had to break a lot of conventional rules that applied to stereo for years.
You want to know how to create immersive audio experiences, that amaze your listeners? Just write me a mail
Write me a mail