Spatial 3D Audio Media from Google for Android, Chrome, SDKs etc.
Content
Will spatial audio by Google soon be available natively on Android? After all, Apple users have already had the opportunity to experience Spatial Audio on their iPhones and other Apple devices since 2020. And looking at the latest release notes for Android 13, Android users could soon be able to enjoy 3D audio natively as well.
To me, it has a bit of a Star Wars charm. The dark side versus the light side of the force. Whether Apple is good or bad with Dolby now, readers can make up their own minds about that on this blog. In this article I share my opinion about this topic. Although in fact, eyes and ears are now slowly turning to Google. Because there is a competitor to AC-4 of Dolby Atmos, MPEG-H from Fraunhofer. So, will Google choose the Dolby Atmos path or will it choose another side of the force? It’s time to put the speculation under the microscope.
Update 22/09:
- Sony Xperia 10 IV smartphone comes with big support for 360 reality audio and 360 reality audio upmixer.
- Google Resonance also supports Audio middlewares such as FMOD and Wwise. Added in chapter SDK
- Google Pixel Buds Pro and Samsung Galaxy Buds Pro 2 are supposed to become a software update to enable spatial audio and headtracking
- Project Caviar could seriously attack Dolby, other than MPEG-H
- added Google Glass for fun since they have bone conducting headphones
Update 23/01:
- Google finally brings 3D audio to the Pixel 6 (Pro), 7 (Pro) and Pixel Buds Pro. The Samsung Buds Pro also support multi-channel audio.
What is Google doing in the area of spatial audio?
As several media outlets have reported in the past, Google has been in the process of buying up companies in the audio space for some time, investing millions in the process. For example, in December 2020, Google spent over $35 million to acquire the audio division of Synaptics, and with it, very important patents.
The 3D audio startup Dysonics was also acquired by Google in the same month. This company had specialized in motion and audio adaptations for headphones. Google then also added in-ear monitoring and wireless headphone experts RevX and French startup Tempow to its roster.
New audio features with Android 13
Google Spatial Audio has already confirmed some new features for Android 13. For example, there are new API interfaces in the audio framework that could support head tracking over Bluetooth in the future. Head tracking includes the listener’s head position in the calculation of the audio signal during sound playback. This means that the virtual position of sound sources can be adjusted to the user’s head movement. Apple promises a cinema-like surround experience with these features.
It can be assumed that many devices running Android 13 will eventually support Spatial Audio. If you compare Apple’s approach from the time of the switch to Spatial Audio, you will notice that even old devices from the iPhone 7 series still support Dolby Atmos by now. This means that Android 13 will be released in the fall, and most Android devices will probably support Spatial Audio by the beginning of next year. For example, the Spatializer function has already been discovered in an update of the Google Pixel 6 Pro.
ro.audio.spatializer_enabled=true
These facts and Google’s job ads for an experimental acoustics lab suggested that Google Spatial Audio will soon launch new software or hardware. And so it was, the 3D-capable Pixel Buds Pro are finally coming to market in July already! So, after the 3D-capable Pixel Buds Pro, is MPEG-H support coming soon?
Sony Xperia 10 IV with 360 Reality Audio
The Sony Xperia 10 IV is designed with Sony 360 Reality Audio to optimize sound quality. So Sony makes use of the Android smartphone infrastructure. This work best when streaming music on Amazon Music, Deezer, nugs.net, and TIDAL through headphones. So listening binaural audio that has been mixed object-based is still a big thing. Dolby Atmos remains the biggest competitor.
With 360 Reality Audio Upmix, the Xperia 10 IV can optimize 2-channel stereo tracks in real time. Similar to Apple’s spatialize stereo feature, this means the music fights in-head-localisation. You have rather the feeling of the music coming from speakers inside your romm – despite wearing headhpones.
Spatial Audio like Apple for Samsung Galaxy and Pixel.
The 360° audio sound with Direct Multi-Channel (5.1ch / 7.1ch / Dolby Atmos) almost creates a cinema atmosphere and concert feeling. Instruments, voices and sounds are impressively placed on the sound stage. If you move your head, the sound still comes from the appropriate direction for a realistic experience. A fast-responding head tracking makes it possible.
Samsung: Only available on Samsung Galaxy smartphones and tablets running Android One UI version 4.1 or higher. 360 Audio and Head Tracking™ support may vary by application and content. Head Tracking and Direct Multi-Channel is supported with Android One UI version 4.1. or higher.
Google: Spatial Audio is one of the many features that Google had initially announced for the Pixel Buds Pro, but did not want to deliver via update until later. It has to be supported from both sides. On the one hand from the headphones (Pixel Buds Pro), on the other hand from the smartphone (Pixel 6, Pixel 7).
So with the January 2023 update, it finally gets going. According to Google, Spatial Audio will come to the Pixel 6 series and also the Pixel 7 series with the first update of the new year. The Pixel 6a remains unmentioned despite the Tensor chipset.
What does MPEG-H actually do with Google Spatial Audio ?
Of course, it would be good to know what MPEG-H actually entails. MPEG-H is the Next Generation Audio format from the Fraunhofer Institute. The listener is enveloped in sound all around, similar to Dolby Atmos. Nevertheless, one also has the option to strongly personalize the sound. For example, Sony’s 360 Reality Audio format is based on MPEG-H and is already used diligently by the music streaming services TIDAL and Amazon Music HD.
It can also be used, for example, to adjust dialog volumes or the volume levels of audio descriptions. MPEG-H was developed specifically for integration into the streaming and broadcasting world. The 3D components can therefore be played back through all possible speaker, soundbar or headphone configurations. So, with the addition of height speakers and Higher Order Ambisonics technology, a cinema-like sound can easily be created at home. This is also similar to the previous approach of Dolby, but the applications are probably more in the live than cinema area.
What else is coming? In 2019, for example, a prototype was also presented by Fraunhofer IIS, Sony and Audi for immersive music streaming in the car. So it remains exciting to see what new 3D audio technology will be coming our way in the near future.
Project Caviar
My speculation about MPEG-H is not far off. In fact, Google is working in the background on a new open source media format. This is intended to attack Dolby Atmos and Dolby Vision. The name suggests premium features here. HDR (High Dynamic Range) and 3D audio are to be supported. Unlike Dolby, there are no license fees.
I don’t think Dolby can be competed with for theatrical movies. They have already gained too much ground in Hollywood and streaming portals for that. But that’s not a bad thing at all, because Google’s focus with Project CAviar seems to be on YouTube. After all, what would it be like if influencers, with billions of followers, suddenly got access to high-tech audio-visual formats? So it remains exciting.
What applications for 3D audio are already available on Android?
For one, there’s already the Galaxy Buds Pro, which Samsung launched shortly after Apple’s Airpods Pro. These use Dolby Head Tracking Technology™. In general, however, usage is rather mixed. I’ve already shed some light on what the problems are with these in this article. In short, the headphones don’t give an immersive, enveloping sound, but rather reproduce sound directed from the phone, which obviously doesn’t do justice to the potential of 3D audio.
Otherwise, in this article I’ve already pointed out some apps that already run on Android and use 3D audio without head tracking.
SDKs (Software Development Kits)
ARCore
Google’s SDK for augmented reality applications already includes a 3D audio feature.
Google Resonance
Google Resonance Audio was released in 2017 and open-sourced in 2018. It aims to create more realistic and compelling surround experiences for AR and VR apps. Even though it’s not being developed further, the VR scene is still using it.
Resonance Audio is made to work on mobile and on desktop and is compatible across many platforms, from Unity to the web. There are also plugins for DAWs to create sound for 360° videos, for example. Resonance works with Ambisonics and Head-Related Transfer Functions to trick the auditory system into believing distance and positioning of sounds around us.
Darüber hinaus werden auch die Audio Middlwares FMOD und Wwise unterstützt. Das Paket enthält:
- Resonance Audio Listener: Ermöglicht die binaurale Räumlichkeit von Klangquellen, die mit den Plugins Resonance Audio Source oder Resonance Audio Soundfield erstellt wurden. Ermöglicht außerdem die Simulation von Raumeffekten.
- Resonanz-Audioquelle: Verräumlicht Mono-Klangquellen an beliebiger Stelle um einen Zuhörer herum. Wenden Sie Effekte wie Abstandsdämpfung und Richtcharakteristiken an. Erfordert Resonance Audio Listener
- Resonanz-Audio-Klangfeld: Spielen Sie Ambisonic-Klangfelder erster Ordnung ab, die auf die Kopfdrehung des Benutzers reagieren. Erfordert Resonance Audio Listener
Google VR
Now here comes a unique selling point that Google has over Apple. Namely, Google Spatial Audio is also willing to be its own platform. For example, in the social media area. This failed a bit with Google+. However, the purchase of YouTube was quite successful.
YouTubeVR
Since 2015, it has been possible to upload 360° videos on YouTube. In 2016, the functionality was expanded with Ambisonics in the first order. So with the right metadata, four-channel audio can be tacked onto a video and allows spatial sound in the web browser.
There was another adjustment made a few years later because content creators missed the feature of head-locked audio. So, in addition to the Ambisonics bed that dynamically adjusts to the direction of the view, a stereo track can now also be implemented that always plays through constantly. This can be handy for using typical music or speech that comes from off-screen. For voice-overs that are not part of the scene, for example.
Google Jump Inspector
Higher-Order Ambisonics has been missing on YouTube ever since. However, there was the Google Jump Inspector for a while. This allowed playing Quicktime files, which allow uncompressed audio. With this, you could not only attach 4 audio channels, but even 16. With this, you could use Android smartphones to watch third-order Ambisonics in combination with a Cardboard, for example.
The APK is unfortunately no longer officially available, but can be obtained on the Internet via third-party providers. Otherwise, I’m sure I still have the installation file on some hard drive 😉
Google Daydream
If and how the Google Daydream also had 3D audio functionality is difficult to research retrospectively. However, it can be assumed that this was possible via the YouTubeVR app. Similar to how it was solved with the Oculus Go or Samsung GearVR back then. Conventional Google VR apps – like all other VR apps – had to have 3D audio implemented in Unity. Which means that Google’s VR glasses have less to do with it, but the content is the deciding factor whether 3D audio is available for Google at all.
Google Smartspeaker
Unlike Apple, Google’s in-house smart speaker Google Nest and Google Nest Mini unfortunately lack 3D audio features. Other Google-enabled devices, such as Bang&Olufsen’s “Beosound Balance,” Sonos’ “Move” speaker, and Bose’s “Portable Smart Speaker” also unfortunately lack 3D audio features.
Headphones
Can the Galaxy Buds Pro do spatial audio?
The Galaxy Buds Pro could do head tracking (see my AR headphones article), but the stereo was turned into mono, which doesn’t really add any value.
Do the Pixel Buds 3 have head tracking?
So the new generation of Pixel Buds 3 will follow in the footsteps of the Air Pods Pro. This means that not only mono and stereo will be artificially mixed up to 3D. Multi-channel audio formats can finally be spatialized. Apple is already doing this with Dolby, but Fraunhofer technology could be used here.
It seems that a similar spatial audio functionality as in the Apple universe with Dolby Atmos will be possible in the fall of 2022 via an upcoming software update for Android.
This could also affect the Galaxy Bud Pro 2.
Google Glass
How could I have forgotten about these? Way ahead of their time, they were hyped as one of the first major products in augmented reality. I couldn’t find any information on whether spatial audio was actually used back then. Still, the audio technology is exciting:
Google Glass’ audio comes from the Bone Conduction Transducer (BCT). Also known as the speaker, it’s located inside the device next to the battery. It’s the oval part with the Glass logo on it that sits just behind the ear. But you could also plug in headphones.
Summary Google Spatial Audio
As we can see, there are already a few ways to consume 3D audio on Android, but unfortunately mostly without head tracking. However, it seems that this will soon be expanded to include the ability to consume 3D audio natively on Android with head tracking. Whether and when Google will come up with MPEG-H is not yet known. It would be a welcome alternative, at least in view of the Apple-Dolby cooperation, should this speculation prove to be true.
Explore more