Hearables: AR for Your Ears – from Earbuds to Audio Glasses

Everybody is talking about AR glasses. Meta has sold millions of them, Google and Apple are lining up behind, and every keynote promises information floating in front of your eyes.

AR for your ears is already here.

The earbuds people wear on the train have motion sensors, microphones that can steer their pickup pattern, and enough processing to place a voice where you happen to be looking. The hardware question is largely answered. What is missing is something worth listening to, and that is the part nobody wants to pay for.

I consult on immersive headphone products, and the same pattern keeps repeating: a manufacturer ships a clever transducer, the press calls it a breakthrough, and a year later the feature sits unused because nothing was ever produced for it. Technology gets adopted when there is a reason to use it, not when the spec sheet allows it.

That is what this article is about. What hearables actually are, which form factor changes what you can do with sound, and which use cases are real rather than announced.

It’s a play on words from “hear” and “wearables”. The latter is also a new term for wearable smart devices. Such as the Apple Watch for instance. And smart is also the buzzword for hearables: The devices, which are therefore also called smart headphones, are equipped with increasingly sophisticated functions. The applications are versatile, including active noise cancellation, and range from entertainment to healthcare. They are also often referred to as True Wireless Stereo (TWS) wearables.

Diverse Use-cases and Technologies

Basically, these Bluetooth headphones are characterized by their sound, their practical use and the predominantly intuitive operation. Therefore, earbuds and co. can also be used in a variety of ways. James Hayward, a researcher in the field of wearables, categorizes Hearables into headphones, i.e. consumer audio, hearing aids, and biometric tracking.

Thus, one could break down and divide the devices into the cross-cutting themes of entertainment and health. However, hearables are also useful in terms of logistics. For example in warehouses, but we’ll get to that in a moment.

Form factors: in-ear, open-ear, bone conduction and audio glasses

Most writing about hearables assumes one shape: something wedged into your ear canal. That stopped being true a while ago, and the shape matters more than the brand, because it decides what the device can and cannot do with sound.

Sealed in-ear. The classic earbud. A tip seals the canal, which gives you isolation, noise cancellation that actually works, and real bass. It also gives you the occlusion effect: your own voice booms inside your head, and you are cut off from the room unless the device pipes it back in through a transparency mode.

Open-ear. Drivers sit next to the ear rather than inside it, firing sound across the opening. Shokz OpenFit and Bose Ultra Open work this way. Your ear canal stays free, so you hear the street and the earbud at the same time with no processing in between. You pay for that with thin bass, audible leakage, and no privacy on a quiet train.

Bone conduction. A transducer presses against the cheekbone and vibrates the skull directly, skipping the outer ear entirely. Speech comes through well. Music does not, because the useful bandwidth is narrow and low frequencies need so much amplitude that you feel a tickle long before you hear bass.

Audio glasses. Bose Frames, Razer Anzu, Ray-Ban Meta, Xiaomi. Small drivers in the temples, firing down towards the ear. Acoustically this is open-ear with one extra problem: the frames move.

That last point tends to get skipped in reviews, and it is the reason I would not build a spatial audio product on glasses today. Convincing spatial audio depends on a stable, known geometry between the driver and your eardrum, because HRTF-based rendering assumes exactly that. In-ear gives you it. Glasses do not. They sit differently on every face, they slide down your nose, and a few millimetres change the frequency response arriving at the ear.

Leakage makes it worse. Once sound radiates into open air, the real room adds its own reflections on top of the reverberation the renderer just generated. You end up with a simulated space layered over a physical one, and no firmware update removes that.

None of this makes glasses a bad product. It makes them a good product for a specific job: speech, navigation prompts, notifications, a podcast while you cycle. Anything where localisation only has to be roughly right. For music, for cinematic mixes, for anything where a sound has to sit convincingly behind you, sealed in-ear is still the only form factor that delivers. I compared how the leading spatial audio earbuds actually sound if you want specifics.

Entertainment

At present, the entertainment sector is still the main field of application for hearables. Earbuds like the Apple AirPods Pro or the Samsung Galaxy Buds Pro are strong representatives here. You can read more about these in-ears here. The most important areas of application in the entertainment sector of hearables are presented below. However, this is no guarantee of completeness. Especially since these areas cannot be strictly divided into entertainment or health, and there are many overlaps in this topic.

What is the added value for streaming?

Music streaming is probably the most widely used application of the hearables. The newer generations of earbuds, such as the Apple Airpods, offer the possibility of playing spatial audio and lossless audio. So high quality audio, also as 3D sound. With additional head tracking, an unrestricted immersive audio experience can be offered. If you want to get more information about streaming, click here. After all, movie watching is also becoming more and more interesting. The cinema feeling is even possible via the small earbuds.

How can Hearbles be used for Augmented Reality?

AR is one of these examples that can be used in healthcare, entertainment, and logistics. Many hearables have built-in motion sensors and microphones. This enables head tracking and noise cancelling – two important features for augmented reality. Which is also labeled AAR (Augmented Audio Reality).

As mentioned, there are many possible applications here: games such as Pokemon Go which is not so interesting regarding sound. But navigation assistance, like logistics in warehouses is for sure. In addition, it really makes sense to focus more on auditory stimuli than on visual stimuli in certain areas:

One important benefit of placing the entire interaction unit in-ear and addressing users purely through acoustic signals is the lower grade of overall distraction compared to vision based augmented reality tools or wearables with tactile signal. – Wikipedia

The interesting part is not the sensor, it is what you do with the degrees of freedom it gives you. Head tracking on a hearable tells you where the listener is facing, and that alone is enough to keep a voice anchored to a place in the world while the person turns around. A navigation prompt or a museum guide needs nothing more than that, and it costs a fraction of what the equivalent visual layer costs to produce. I have collected five AR audio use cases that show what this looks like in practice.

It is also where most projects fail. Teams treat the audio layer as a playlist with coordinates attached, then wonder why people take the headphones off after two minutes. A sound that sits perfectly fixed in space while you walk past it draws attention to how synthetic it is. Real environments have reflections, occlusion and a floor underneath you. Leaving those out is what makes AR audio feel like a demo instead of a place.

That is the gap between a working prototype and something anyone wants to use twice, and it is a content and mixing problem, not a hardware one.

How can Spatial Audio be used here?

Spatial 3D audio can also be beneficial in so many different ways that the entertainment sector can only reflect a part of the possibilities. In terms of hearables, the devices with motion sensors already mentioned are particularly interesting. Only through head tracking can spatial audio applications really come into their own. You can find out what is already on the market in this regard here.

So you can think of these applications as a new generation of navigation devices. Not only do you hear the announcement “please go left” but you also hear the announcer from this direction. So that I don’t have to wonder any longer: “Err, so which left is it?!”.

We humans hear the sound from all directions. Each sound source, such as a conversation partner, is always perceived as equally loud. Even if we turn away from the sound source. However, microphones can use the so-called directional characterization. Like that the sound is not picked up equally loud from all directions in a spherical shape. This could be used, for example, to make sound louder from the front, i.e. where you are looking. A bit like having binoculars – for the ears.

Health and Medical Sector

There’s a lot going on in the health, fitness, and wellness sector right now in terms of hearables. As it turns out, placement in the ear is advantageous for collecting various biometric data:

… the measurement of biometric data such as temperature, heart rate, or oxygen saturation can be monitored through via PPG with significantly higher reliability and better response times through in-ear monitoring than contact devices placed on the wrist or torso. – Wikipedia

But that’s not all, you can probably already imagine that the hearing aid market will really start to sweat in the future. So here are the most important applications of hearables in the health sector.

Stay up to date with bearable and more.

What can the new generation of hearing aids do?

As just mentioned, earbuds and the like have a good chance of taking over the hearing aid market: Starting with active noise cancelling, which suppresses or reduces background noise. With conventional hearing aids, the louder the volume, the louder the ambient noise.\ With active noise canceling, on the other hand, the ambient noise is cleverly reduced. This changes signal to noise ratio, which significantly improves speech intelligibility. In the future, artificial intelligence and speech recognition will be increasingly used to further improve the listening experience. This will also allow it to be better adjusted depending on the situation.

Furthermore, unlike classic hearing aids, hearables allow you to connect to your smartphone or smart TV. This means that the volume can be adjusted directly in the ear, so the loudspeaker system does not have to be turned up loud for hearing impaired people. The user can personalize this for him/herself. The other viewers in front of the telly are not affected.

I remember my grandma struggling with hearing aids. Before she could even fail with the handling that’s not intuitive with seniors: The tiny acoustic hearing apparatus needed even tinier batteries to be replaced like every day. Mission impossible.

Also, a sound treatment for the therapy of Tinnitus is easier with Bluetooth headphones. Thus, a specifically played sound can cover the disturbing beeps and ringing in the ears. The Bose Sleepbuds also encourage wearing headphones while sleeping. Not only for on the road a tempting application. You can hear, for example, the sound of the sea, which prepares our bodies for slumber.

The visual aspect should also not be ignored. Hearables are trendy and fashionable, at least much more so than old-fashioned hearing aids. As a hearing-impaired person, you are no longer branded with a “hearing crutch”. They are even stylish and fully in line with the trend! One of many added values on the subject of inclusion.

Sound therapy and biometric measurements

Sound therapy will probably play a greater role in the future and is another area of application for hearables. The treatment of tinnitus has already been mentioned, but that’s not all: personal sound amplification products and apps such as “Headspace” or “Calm” – meditation apps or apps for relaxation and stress reduction – are becoming increasingly popular.

With built-in sensors to monitor biometric processes such as pulse, movement patterns, or calories burned, adaptive sound therapy solutions could be offered in combination with these or similar apps. Imagine the music or sound scene changing depending on how you feel to create the best possible effect. Treatment of anxiety and sleep disorders is also conceivable. You see and hear, there is lots of potentials there.

But who says that sound therapy always has to be relaxing? How about the perfect music playlist for the next workout? Because with the biometric data, you could get exactly the right beat on your ears that your body needs at the moment. Once again, content is king.

Hardware – what is already on the market?

Not all Hearables are suitable for all the applications mentioned. Of course, where can you find a perfect solution? You can divide these devices into three categories according to their strengths: Earbuds, Assistive Listening Devices, and Hearing Instruments. Here is a brief overview of the differences and available devices.

Earbuds

With Earbuds, the focus is on good sound. You could also say the main field of application is the entertainment sector, to stay in our categories. Often there is the option of active noise cancelling or a transparency mode that only lets essential sounds through, such as the voice. This ensures a high-quality sound experience in pretty much any environment.

The new generations, above all Apple and Samsung, increasingly rely on spatial audio. If you want to look into this further, you’ll find quite a bit in my blog by now: This would be a good place to start, or here whoever wants to look into Apple’s Spatial Audio, and you’ll find a critical point of view here.

So well-known representatives would be:

• Apple AirPods Pro

• Samsung Galaxy Buds Pro

• Bose Frames

• Sony WF-1000XM4

Assistive Listening Devices

In addition to qualitative audio enjoyment, these devices also aim to improve hearing. They form the link, so to speak, between entertainment and health-specialized devices. Often these Hearables are equipped with the ability to create an individual audio profile – to compensate for hearing loss. There is basically more focus on personalization options and speech enhancement than with earbuds.

Examples are:

• Nuheara IQbuds MAx

• Brag The Dash Pro

• Hy by Zoku (the processing of the headtracking is done in the device. Unlike the earbuds mentioned above who have to be connected to a smartphone)

Hearing Instruments

As the name suggests, these Hearables are primarily intended as hearing aids. Here, a professional fitting to one’s hearing is ensured. By amplifying certain ambient frequencies, speech enhancement and noise reduction. As well as direction-dependent noise detection, a better hearing experience is made possible for people with hearing loss. These devices also have the longest average battery life among hearables.

Examples are:

• Signia Active

• Starkey Livio Edge AI

Conclusion

The topic of hearables is really diverse, as you can probably tell from this article. It’s definitely exciting to see what else is in store for this potential. I know 3D audio has been mentioned comparatively little here, but don’t be fooled: Spatial audio already plays a big role in most application areas, such as AR, streaming, or even apps like Headspace. And I’m sure that the possibilities of Spatial Audio in terms of Earbuds and Co. are far from exhausted!

So as you can see, or rather hear, the application precedes the hardware. But in the end, it is the question of what the user actually hears? That brings us back to the topic of audio content. So what is the optimal listening experience, the right device, and appropriate application? I am happy to help you solve these challenges

Get in contact

Let’s talk about your project

Want to use 3D audio in a real project, not just understand it in theory?

Get in touch →