Film Sound Guide of the Professional Association of Film Sound (BVFT) shows immersive 3D audio with Dolby Atmos, VR, 360 Video and Storytelling<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_yoast_wpseo_opengraph-image":"https:\/\/www.vrtonung.de\/wp-content\/uploads\/vrtestung_A_silhouette_of_a_person_1200x630-aimg.jpg","_yoast_wpseo_opengraph-image-id":"18720","_yoast_wpseo_twitter-image":"https:\/\/www.vrtonung.de\/wp-content\/uploads\/vrtestung_A_silhouette_of_a_person_1024x512.jpg","_yoast_wpseo_twitter-image-id":"8748","footnotes":""},"categories":[1],"tags":[23,25],"class_list":["post-2882","post","type-post","status-publish","format-standard","hentry","category-unkategorisiert","tag-overview-en","tag-storytelling-en"],"acf":{"remove_from_blog_overview":false,"alternative_title":"","previewimg":18718,"text":"# 3D Audio \u2013 the immersive spatial soundtrack from all directions\r\n\r\nThis article was written for the [Filmton Guide](https:\/\/www.bvft.de\/service\/filmton-guide\/) of the Berufsvereinigung Filmton (BVFT). Of course, it should also be available to non-members. My aim here was to highlight the topic of immersive 3D audio in the context of cinema in film sound and TV, especially with advancements like head tracking .\r\n\r\nBut beyond that, there are many possibilities for the soundtrack of the moving image. What these are and how the good, spatial sound can be used to its full advantage are described below. Have fun reading!\r\n\r\nFor the technical basics behind all of this, see [how spatial audio works](https:\/\/www.vrtonung.de\/en\/spatial-audio-how-does-the-3d-sound-work\/).\r\n\r\n## Immersion, what even is that?\r\n\r\nAdmittedly, hardly any other word is as hyped in the audio scene as \u201cimmersive\u201d. Yet \u201cimmersive audio\u201d is often used as a synonym for \u201c3D audio\u201d. This underestimates the fact that even mono playback on headphones or speakers can be immersive when combined with a stirring story. And music recorded in good old stereo also invites you to close your eyes and immerse yourself in a world of sound.\r\n\r\n### So nothing new? Yes, it is!\r\n\r\nDummy head stereophony has been a popular recording method for playback on headphones for decades. Currently, however, there is a lot of momentum in the subject again. As sound is now also becoming more accessible as a three-dimensional event for a larger audience with loudspeakers.\r\n\r\nFurthermore, additional technologies such as head trackers, head-mounted displays, and real-time renderings are popping up in a wide variety of areas, giving us an increasingly realistic impression of hearing as we are used to in our natural environment.\r\n\r\nPut simply, they can trigger emotions, feelings of presence, and perceptions that are deeply connected to our brain and own experiences \u2013 more directly than mono could. For example, because a level of abstraction is omitted and is easier for our brain to process. Therefore, the keyword immersion, i.e. diving in a virtual world environment, is quite apt.\r\n\r\n### Advanced value of immersive sound aka 3D audio\r\n\r\nOf course, this changes everything \u2013 doesn\u2019t it?! The euphoria among film sound engineers is great and the topic is a perennial topic at the relevant conferences, especially as it relates to the rise of streaming services . But where and when can 3D audio actually offer added value? In recent years, I have had the opportunity to deal with immersive audio in a variety of contexts more intensively than almost anyone else. But precisely because of my enthusiasm, I like to question whether spatial audio is always the first choice of things.\r\n\r\nBecause I would like to warn against mentally storing \u201c3D audio\u201d in the mental drawer \u201cbetter than stereo\u201d. So it\u2019s a matter of filtering out where the new technology works best and what might be promising areas of application. Since it is difficult enough to do justice to the topic in its entirety, we still want to scratch the surface as best we can.\r\n\r\n\r\n\r\n## Content is King\r\n\r\nThe possibilities for working with 3D audio are almost unlimited. Most people first think of Dolby Atmos for binaural audio or music productions. But this is actually only a small part of the conceivable applications and \u201cdaring spoiler alert\u201d: I think the real potential lies hidden elsewhere. But more on that later.\r\n\r\n### We have the technology, but the knowledge?\r\n\r\nFortunately, there are now a number of manufacturers who have jumped on the immersive bandwagon: Microphones, plug-ins, distribution channels and platforms. Nevertheless, there is still a lot of research to be done in the field of [Next Generation Audio](https:\/\/www.vrtonung.de\/en\/next-generation-audio\/), personalized listening, binauralisation, etc. And it seems that every discussion at relevant conferences ends with the keyword HRTF (Head-Related Transfer Function). But I would argue that what we currently lack is not so much the tools and theories, but the knowledge of what we can do with them in creative practice.\r\n\r\n### The jobs will come \u2026 for sure.\r\n\r\nI know at least a handful of colleagues who snapped up the Sennheiser Ambeo VR mic in combination with a Zoom F8 recorder as soon as the product was released. Wisely anticipating that all the 360\u00b0 requests would come soon. A few years later, it is now clear: that never happened. But how can it be that in other areas there is more production with 3D audio than ever before? So let\u2019s start with the best-known example.\r\n\r\n\r\n\r\n## Film and television productions with 3D audio\r\n\r\nThe typical approach is to use 3D audio which it has already worked well with surround sound. It is no longer a secret that more than a thousand moving pictures have already been mixed immersively in Dolby Atmos. This fact alone is remarkable for the sound. While the listening experience can currently be enjoyed almost exclusively in cinemas or friendly studios. In the future, the three-dimensional mix will soon find its way increasingly into the living room at home via [mehr erfahren](https:\/\/www.vrtonung.de\/en\/3d-soundbar-surround-sound-dolby-atmos-thx-audio\/). Clever algorithms with virtual loudspeakers make it possible.\r\n\r\nOr even simpler: in almost every household there are earbuds and headphones that make three-dimensional audio reproduction in the form of binaural stereo accessible to consumers. As a listener of binaural audio, you increasingly get the feeling that you are not just watching a film, but that you are part of the action. Mixing engineers can also be happy about the better transparency because, with the spatial distribution of signals across the different sound levels in the room, fewer compromises have to be made than with two-channel speaker layout.\r\n\r\n### Next Generation Audio (object-based NGA).\r\n\r\nThe technology behind this system is called object-based audio. Put simply, the so-called \u201cbed\u201d serves as the basis for diffuse sounds such as atmospheres and reverbs. This is supplemented by mono objects, which give the system the whole thing its name. These objects are not, as in the classic workflow setup, mixed inseparably into one sum, but are still present in the master file as separate files that are additionally described with metadata.\r\n\r\nBut object-based audio not only offers the possibility of three-dimensionality, but personalized interactivity is also becoming more and more exciting. The fact that the files are also available separately in the codec for end-users opens up unimagined possibilities.\r\n\r\nHow would it be, to name one example, to be able to mute the audio object \u201ccommentator\u201d during a football broadcast? In order to enjoy the enveloping stadium atmosphere and better be able to make your own comments with your mates on the couch in real life? The development of immersive sound then offers the same possibility for entertainment formats that the cinema hall cannot provide.\r\n\r\n### Are you even allowed doing that without annoying somebody?\r\n\r\nThis means that the actually sacred audio master of the mixing sound master can be manipulated by the viewer \u2013 at least as far as it is defined in the mix. An idea that the makers will have to get used to.\r\n\r\nBut the exciting question is: How could interactive TV formats work in the future, for example, in which the sound can be personalized? This is a field in which interactivity could perhaps be even more exciting than three-dimensionality and is already being implemented at BR, for example, as \u201cDialog+\u201d, within the framework of a pilot project.\r\n\r\nSpeaking of personalized listening \u2013 let\u2019s now look at an area that literally enables immersive listener immersion in the film and hear its own quality of sound.\r\n\r\nSubscribe to the latest 3D audio news\r\n\r\n## 360\u00b0 videos and VR (Virtual Reality) with binaural audio\r\n\r\nFirst of all, a big \u201cAttention\u201d: Most sound engineers talk about VR, but actually mean 360\u00b0 videos. However, such productions are only a very small niche of the whole virtual reality and should rather be considered as a special case. There are two major limitations: 360\u00b0 videos are linear in time, just like conventional films, and secondly, they only allow rotation of the viewing direction around three axes: X, Y and Z. This is known as the three degrees of freedom.\r\n\r\nAlso known as three degrees of freedom (3DoF). The hype around such productions has already died down and will probably only play a minor role in the future \u2013 even if the flood of tools, software and plug-ins suggests a need that hardly exists on the market.\r\n\r\n### The true power of virtual reality\r\n\r\nVR applications in the true sense should be considered an interactive experience, and have more in common with computer games than film productions. Here you can\u2019t get around game engines like Unity or Unreal and middlewares like Fmod or Wwise. This requires programming skills and a certain rethinking of audio production.\r\n\r\nHowever, the true potential of VR lies in this interactivity, which does not have to be linear in time. Furthermore, 3D audio gets another component, because in addition to the axis rotation, the transformation based on the three spatial axes is added. This means that users can not only rotate from a fixed position, i.e. look around, but also move freely in space. This adds three more degrees of freedom and is thus also referred to as experiences with 6DoF (six degrees of freedom).\r\n\r\nThus, there is no longer one optimal listening position, as it is taken as a basis for audio mixes. It is much more important to take into account that the playback location can be anywhere in the virtual space. This opens up an incredible number of possible applications for rethinking surround sound. Since this article cannot do justice to this, I will now concentrate on the area of 360VR in terms of surround sound.\r\n\r\n[newsletteroptin text=\"Experience 3D audio spatial sound \u2013 subscribe now!\"]\r\n\r\n### 360\u00b0 videos are and remain a niche\r\n\r\nBack to the world of spherical videos. How could it be otherwise, not only the image is a round thing here, but also the sound. The [Ambisonics](https:\/\/www.vrtonung.de\/en\/ambisonics\/) format has established itself as a quasi-standard for 360\u00b0 videos and has been officially supported by YouTube and Facebook, among others, since 2016. In addition, there is a head-locked stereo track that, in contrast to the Ambisonics sound field, does not change when the head is moved.\r\n\r\nThis additional, optional audio stream is preferably used for voice-overs or motion picture soundtrack recordings. This is to narrate audio content that is heard but not in virtual space. This blurs the boundaries between diegetic and non-diegetic content. There is thus still a lot of need for discussion about not confusing users with speech and music that is not visually reflected in a 360\u00b0 scene, but rather heard to support storytelling with a targeted soundtrack.\r\n\r\n### Ambisonics \u2013 curse and blessing for surround sound\r\n\r\nAmbisonics is often badmouthed in the sound scene because it certainly has disadvantages, such as localisation, which even higher orders can only solve to a limited extent. Nevertheless, due to its practicability and in combination with object-based spatial audio, in the form of mono sources, it absolutely has its raison d\u2019\u00eatre here. But since this is mainly about the content and less about the constantly developing technology behind it, let\u2019s continue with the text:\r\n\r\nFor sound engineers, mixing 360\u00b0 videos means saying goodbye to the \u201ccentre channel\u201d. There is no longer a fixed viewing direction and the user now decides where the interest falls. This requires a big change for filmmakers in almost all departments. That\u2019s why this area is currently mostly staffed with people who understand cross-media work and also have experience with video game productions.\r\n\r\nAnd yet the soundtrack is not only used here to set the picture to music. I like to go so far as to say that sound is even more than 50% of the immersive experience in VR. Because we can miss things in a VR experience but not overhear such spatial cues.\r\n\r\n### Advantages for storytelling\r\n\r\nFor example: if someone comes through the door from behind on the left, we can add crotch sounds about a few seconds earlier. This subtly directs the viewer\u2019s gaze and they don\u2019t miss the moment when the new protagonist enters the room. But if you hear footsteps at the back left in the cinema, it influences the narrative, but only in extreme cases motivates the audience to turn their heads away from the screen, which is usually not desired. In VR, however, it is precisely this physical turning of the head that we want to achieve in order to influence the gaze and thus the image detail by intuitive means.\r\n\r\n\r\n\r\n## When the eyes help to listen\u2026\r\n\r\nMost applications with 3D audio like to aim at making the quality of the reproduction as realistic as possible. This may be the maxim point for some applications, but we as film sound designers know: reality usually sounds pretty disappointing. However, the term \u201clarger than life\u201d also applies in three dimensional space: the aim is not to reproduce reality, but to create an illusion of a credible 3d sound-scape that above all supports the storytelling.\r\n\r\n### Dolby Atmos and Sony 360 Reality\r\n\r\nNow for the more complicated keyword: added value. However, it is not at all easy to make a 3D mix really sound better than a stereo mix in comparison, as the following example will show: [Dolby Atmos Music](https:\/\/www.vrtonung.de\/en\/dolby-atmos-music\/) and [Sony 360 Reality](https:\/\/www.vrtonung.de\/en\/sony-360-reality-audio\/). Here, over 1000 well-known songs were remixed in 3D and offered on streaming providers such as Amazon Music or Tidal. Produced in major studios, this is a really big deal and a huge opportunity to get consumers excited about 3D audio.\r\n\r\nSeveral listening tests and hours of 5.1.4 listening sessions, however, left a handful of sound engineers and me quite disenchanted with the r\u00e9sum\u00e9: most 3D mixes do not necessarily sound better than the respective original stereo mix.\r\n\r\nI don\u2019t want to generalize here, as there are also very beautifully made mixes, but most of the time I had the feeling that all the pressure had been lost in favor of spatiality, which is not necessarily good for most productions. Who wants to do the self-experiment: [Here are all the streaming providers](https:\/\/www.vrtonung.de\/en\/music-streaming-spatial-audio\/) and formats listed that currently allow 3D audio playback.\r\n\r\n### 3D music can do even more\r\n\r\nThe possible causes are beyond the scope of this article but are briefly touched on with time pressure, unfamiliar new tools, stereo stems instead of individual tracks, and object-based mastering.\r\n\r\nDoes this mean that 3D music will not be of great importance in the future? Of course not! Therefore, I would like to mention another extreme example that we like to ridicule but still gives us hope: [8D Audio](https:\/\/www.vrtonung.de\/en\/8d-audio\/). For more details, it\u2019s best to ask the search engine you trust, but the short version is: someone came up with the idea of running a song through a spatialiser and letting it circle endlessly around your head. Sounds absurd, and to our trained ears it is.\r\n\r\n### 8D is not the holy grail, but it is an interesting compass\r\n\r\nAnd yet: millions of people have been reached here with 3D audio content and the click numbers are in the nine-digit range. This shows how good it can be to think unconventionally and do things that one might resist.\r\n\r\nOne must not forget for whom and why we are actually mixing something. Do we want a full 3D sound mix that can be shown to colleagues with a clear conscience (Dolby Atmos would probably be the choice here), or do we want to reach the consumer (8D Audio shows that it can be done). The truth is probably somewhere in between, so it\u2019s time to harness the potential of both worlds and find the added value of 3D audio.\r\n\r\n\r\n\r\n## Rethink Sound from scratch\r\n\r\nBy the way, the left ear and right ear have two slightly unexpected allies here: our eyes. As humans listeners, it is much easier for us to interpret what we hear if we also see it. In the 3D audio context, too, I advocate not subordinating the listener from sound to the image or, conversely, the image to the sound, but creating a symbiosis that must be approached differently for 3D audio than we are used to from classic film sound.\r\n\r\n### Film sound is great \u2013 even in stereo\r\n\r\nIt\u2019s nice to have an immersive sound mix for a feature film. But if it were stereo only, the narrative would probably work just as well and only the physical immersion experience would suffer.\r\n\r\nThis is not a bad thing, but it makes it difficult to get viewers excited about 3D audio. Nevertheless, [Netflix](https:\/\/www.vrtonung.de\/en\/netflix-ambeo-spatial-audio-dolby-atmos-movies-streaming\/) ventures a step forward with its Spatial Audio and has developed an algorithm with Sennheiser that works seamlessly on any device . This enables selected movies to sound more spatial via headphones (binaural) and speakers (transaural).\r\n\r\nBut it\u2019s even better than artificially making movies sound more like surround. This is where the next hype word comes into play: immersive storytelling. It\u2019s a complex topic, but the principle is simple: find an application where spatial sound has an added value that goes beyond the mere sound waves and soundtrack, and that can support spatial audio .\r\n\r\n### Sound waves in our hands \u2026 or ears\r\n\r\nThis means that we as sound engineers are now in demand. And this is the case long before we go on set or into post-production. So the sound production has to be extended with the mere recording and mixing. Also it needs to act as a director\u2019s department and project management in a way, in order to be able to influence the production at all.\r\n\r\nPerhaps linear narrated films and movies are also not the best medium to experiment with 3D audio dubbing. However, it is worth turning the tables and looking beyond gaming and the film sector to see in which industries and applications there is undreamed-of potential. True to the motto \u201cSound First\u201d.\r\n\r\n\r\n\r\n## Outlook\r\n\r\nAmidst all the immersive hype, however, one must not lose sight of the big topic of immersive sound. Already, the next hype topics are hitting: Artificial intelligence, blockchain, voice assistants, [smart speakers](https:\/\/www.vrtonung.de\/en\/soundbar-smartspeaker-3d-audio-speakers\/), etc. What at first feels like buzzwords that are far removed from our film sound world are actually very hot topics that sound engineers should take a close look at.\r\n\r\nSounds a bit abstract, but the wonderful thing about sound is that it is a cross-sectional technology with which you can virtually dock onto any other topic. This goes far beyond immersive audio. So now it\u2019s time to become a doer yourself, to leave your comfort zone and venture into unimagined audio territory. For it is in our hands to shape the music of the future and this has more than just three dimensions.\r\n\r\n### Acknowledgements\r\n\r\nThanks to the editors just before Christmas: Philipp Eibl, Regina B\u00e4ck, Alexander Rubin, Felix Andriessens, J\u00f6rg Elsner, Mathis Nitschke\r\n\r\n[Get in contact](https:\/\/www.vrtonung.de\/en\/contact\/)\r\n\r\n###Let's talk about your project\r\n\r\nWant to mix or master your music in 3D and need support along the way?\r\n\r\n[Get in touch \u2192](https:\/\/www.vrtonung.de\/en\/contact\/)\r\n","related_articles":[2281,2589,2425,6343,4610]},"yoast_head":"\n