Wie kann man eine head-locked stereo-Spur wie etwa Musik in den Ambisonic-Raum bringen, ohne, dass er sich beim head-tracking mitbewegt? Wie das geht und was die Social Media Riesen Facebook und YouTube damit zu tun haben.<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_yoast_wpseo_opengraph-image":"https:\/\/www.vrtonung.de\/wp-content\/uploads\/og-kopfverfolgung-raumklang-vs-fixiert-3d-sound-headtracking-360-video-audio_1200x630-aimg.jpg","_yoast_wpseo_opengraph-image-id":"","_yoast_wpseo_twitter-image":"https:\/\/www.vrtonung.de\/wp-content\/uploads\/og-kopfverfolgung-raumklang-vs-fixiert-3d-sound-headtracking-360-video-audio_1200x630.jpg","_yoast_wpseo_twitter-image-id":"","footnotes":""},"categories":[1],"tags":[134,144],"class_list":["post-10673","post","type-post","status-publish","format-standard","hentry","category-unkategorisiert","tag-3dof-en","tag-basics-en"],"acf":{"remove_from_blog_overview":false,"alternative_title":"How to mix the head-locked stereo track with Ambisonic","previewimg":18780,"text":"##What does head-locked mean?\r\n\r\nHead-locked means two different things, and confusing them leads to the wrong conclusion. On headphones it is an operating state: the sound sticks to your head and turns with it, because an ordinary headphone cannot do anything else. In production it is a separate audio track that runs alongside the spatial scene.\r\n\r\nOn the playback side it is an either-or, because you pick a mode. On AirPods Pro both sit right next to each other, Spatialize Stereo with or without head tracking. If that is what you are looking for, I have covered it in [link id=\"8237\" text=\"spatialized audio with Apple AirPods\"].\r\n\r\nIn production it is not an either-or but a side-by-side. The scene is rendered head-tracked and turns when you turn your head. Everything that does not belong to the scene, so music, narration, effects sitting on top of the image, stays fixed to the head. That is what this article is about.\r\n\r\n##Why this is a problem at all\r\n\r\nThe reason is the material. Music comes in stereo ninety-nine percent of the time, and stereo does not sit well with ambisonics or other multichannel layouts. So you need a way to get stereo into the scene without it turning along.\r\n\r\nFor 360\u00b0 video ambisonics has become the de facto standard, officially supported since 2016. On top of that comes a head-locked stereo track which, unlike the ambisonics bed, does not rotate. A mix out of the FB360 workstation therefore consists of an eight-channel ambisonics file in TBE format plus an optional stereo track.\r\n\r\nThe catch: YouTube does not accept this hybrid format. Four channels of ambiX, nothing more. The head-locked track drops out, and that is exactly where most practical questions start. The workarounds below are what remains.\r\n\r\n##What you need it for, with one example\r\n\r\nFor the fiftieth anniversary of the Munich Olympics, a 360\u00b0 video was meant to make the intervening decades tangible without pulling viewers out through hard cuts between eras. The solution ran through sound: your viewing direction picks the decade. Look forward and you hear one musical era, turn around and another one comes in.\r\n\r\nThat only works if you can place music inside the scene rather than locking it to the head. It is the opposite case to everything below, and it shows that choosing between head-locked and head-tracked is a creative decision, not a technical one. In [link id=\"21159\" text=\"UNvisible\"], a VR short film built in Unreal Engine, I split it: background music stays locked to the head, and in one scene the music sits in the space.\r\n\r\n##State of play\r\n\r\nOne of the problems that is probably most frequently asked about in the Facebook Spatial Audio group is the following: Mixes created with the FB360 workstation have an obligatory 8-channel Ambisonic wav file in TBE format, as well as an optional stereo track for music, i.e. so-called head-locked audio, which, unlike the spatial audio file, does not rotate when the head moves. \r\n\r\nHead-tracking therefore means that the sound changes with the direction of gaze, while head-locked remains independent of the direction of gaze during playback. How a dubbed voice still has to go into the ambisonic field because YouTube offers no second track is shown in the [link id=\"20778\" text=\"Humboldt 360\u00b0\"] case.\r\n\r\nIf you are new to Ambisonics, I recommend that you first read my short article as an introduction: [link id=\"263\" text=\"Ambisonics for Virtual Reality Video\"]. \r\n\r\nSo far so good, working with your own TBE format on Facebook is no problem. In addition to the 360\u00b0 video, its own Audio 360 encoder can select the multi-channel spatial sound file as well as the static head-locked as input and it outputs a final video file with the 8 + 2 audio channels.\r\n\r\nHowever, YouTube has supported 360\u00b0 sound for some time, but as 4-channel audio, Ambisonics first order in ambiX format. If you now try to select \"YouTube Video (with 1st order ambiX)\" instead of \"Facebook 360 Video\" as the format in the Facebook encoder, the \"Head-Locked Stereo\" option is greyed out. \r\n\r\nBecause this fact is very unsatisfactory and is causing a lot of trouble on said social media channels, I wanted to remedy the situation with this article.\r\n\r\n[image id=\"16749\"] \r\n\r\nBack to the topic, the fact that the encoder refuses to work here is not even its fault: YouTube simply does not support Facebook's hybrid format. You can now only have your 8-channel tbe file automatically transcoded into 4-channel ambiX, the static stereo is left out.\r\n\r\nBut what do you do with music? Before I start a fundamental discussion about whether music should be spatial or static, I have already discussed this a little [link id=\"277\" text=\"here\"]. \r\nBut now let's get down to the solutions:\r\n\r\n##Workarounds for head-locked stereo\r\n\r\n###Make it mono\r\n\r\nThe simplest solution is to convert your head-locked stereo file into a summed mono file. This is then routed to the first channel of the Ambisonics file. This represents a mono-compatible, omnidirectional stream, regardless of whether TBE or ambiX is used. \r\n\r\nThis solution is therefore best suited for voice-overs, which are usually mono and desired as in-head localisation.\r\n\r\nStereo music, however, will sound significantly worse because all side components are lost.\r\n\r\nIn general, you should make sure that the first channel is not too loud, as even if the level is not 0dBFS, this can lead to distortion when decoding the binaural playback later.\r\n\r\n###MS - Mid\/Side Stereo\r\n\r\nA similar approach that can retain the side components is as follows. The head-locked stereo track, which normally has two tracks, i.e. left and right channel, is now MS-encoded. \r\n\r\nHere, the first channel selects all mono components (left+right)x0.5 and the second channel selects all side components (left-right)x0.5. Now the first channel is routed to the first channel of the spatial audio source as described in the mono workaround.\r\n\r\nThe second channel can now be routed to any of the other axes. The height axis is recommended, as this is the axis around which most of the rotation will occur when viewed later on the VR glasses. \r\n\r\nThis means that the side signal would only change if the viewer moves their head up or down, or rolls their head to the left or right, for whatever reason.\r\n[newsletteroptin text=\"Subscribe to the latest 3D audio news\"]\r\n\r\n[image id=\"16755\"] \r\n\r\n###Quad layout\r\n\r\nAnother approach is as follows. Here, the two channels of the head-locked stereo are actually placed in the room, e.g. at +45 and -45\u00b0 azimuth. \r\n\r\nNow you would have achieved exactly what you don't actually want - the music turning noticeably - but we're not finished yet. That's why you simply duplicate the music and place channels 3 and 4 at -135\u00b0 and +135\u00b0 azimuth. \r\n\r\nNow you have virtually created a quadrophony, comparable to a 5.1 set-up without centre + LFE and symmetrical angle distribution of 90\u00b0. It is advisable to tinker with the \"Spread\" parameters in order to avoid the 4 objects in the room becoming really localisable during head-tracking, i.e. becoming more diffuse and merging somewhat with a higher spread.\r\n\r\nYou can fine-tune even more and take a total of 6 objects instead of 4 and select angles at a distance of 60\u00b0. This makes the head-tracking, i.e. the movement of the music, less obvious, but brings it closer to the sound of the mono workaround.\r\n\r\nIt is similar with the approach of not routing the 4 objects 100% left, right, left, right, but rather mixing in, for example, 50% of the other channel in each case.\r\n\r\n[image id=\"16751\"] \r\n\r\n##Conclusion\r\n\r\nThere are many possibilities, but none of them is perfect. Nevertheless, with a little experience it is possible to bend the whole thing so that the layman does not notice the difference. \r\n\r\nThe content on the head-locked track is highly dependent, e.g. classical music differs so much from electronic music that a different workaround makes sense in each case.\r\n\r\nIn combination with the 3D audio stream, it may be that the difference is barely audible, for example when it comes to typical film sound. With sensitive music applications, however, it can quickly become a problem.\r\n\r\nIt would be desirable for the Audio 360 encoder of the FB360 workstation to implement one of the workarounds automatically, because otherwise a new mix has to be created each time with spatial audio containing the music. \r\n\r\nThis can quickly become annoying for mere testing.\r\nIt would be even better if a head-locked stereo track was supported as standard on all [link id=\"267\" text=\"Virtual Reality platforms\"] or by MPEG-H, for example.\r\n[ytvideo id=\"Wfszsh01DEU\"]\r\n\r\nI hope I could be helpful with my article. If you have any suggestions, please send me an e-mail.\r\n\r\n###Let's talk about your project\r\n\r\nWant to mix or master your music in 3D and need support along the way?\r\n\r\n[link id=\"155\" text=\"Get in touch \u2192\"]\r\n","related_articles":""},"yoast_head":"\n