You are standing in the middle of the Colosseum, headset on, and somewhere Mario is speaking. Mario is the narrator you see in the video, and that is the whole point: before you spot him, you should hear where he is. Turn towards his voice and you are looking in the right direction.
In 2019 I did the 360° sound for three films in the “Follow Me” series by Faber Courtial, a studio for digital worlds in Darmstadt: the Colosseum, the Roman Forum and the Circus Maximus, each in German and in a version with an English voice-over.
All three films follow the same structure. At the start Mario gives an introduction in a hall, then each film shows three places. At every place Mario appears once in a different spot, otherwise he hardly moves, a little more in the Forum than in the Colosseum.
In a 360° scene that is a challenge for guiding the viewer’s eye. The viewer can look anywhere and easily misses the moment Mario changes position. If Mario then speaks like a classic voice-over from the middle of the head, there is no hint of where he is now. That makes the “Follow Me” concept ideal for spatial sound: the voice itself becomes the signpost.
The specification was first-order Ambisonics with four channels, and testing was done on the Oculus Go. The audio tracks came from the studio’s previous sound engineer, who had already roughly positioned them in space. I had music and effects delivered separately so I could place individual accents in the space.
Placing Mario. His voice sits where he stands in the picture and jumps with him when he changes position. I mixed in third-order Ambisonics with 16 channels. Every smaller version can be derived from that, and the master can later be decoded for loudspeaker formats too.
How much room can a voice take? This led to the most interesting discussion. I had given Mario’s voice reverb so that it sounds like the same room as the picture. Without reverb a speaker “sticks” to the viewer’s face, because there is no distance information at all. For the studio it was too much. Compared with the old stereo version, Mario seemed further away and less clear. Both sides had a point, so we looked for a compromise between space and clarity: much less reverb, but enough that Mario does not sound like a voice-over.
A hiss in the wrong ear. In the next round the client heard something that is easy to miss: when Mario stood on the left, a faint hissing echo of his S sounds appeared in the right ear. These were artefacts of the low Ambisonics order. The fix was a format with more channels. I switched to Facebook’s TBE format with eight channels, which the Oculus Go plays just as well but with finer resolution.
The English voice-over. In the bilingual version the English voice sits slightly offset over the German one, because it is not a synchronous translation. I placed it close to Mario and slightly above him. That keeps it clear who is speaking, and the translation does not compete with the original.
I delivered six clips, three films in two language versions each, along with instructions on how the studio could combine picture and sound themselves. After the last round Faber Courtial wrote back that everything now sounded wonderful and much clearer than before.
What I take from the project: in 360° films with a narrator, the sound decides whether viewers follow the story or look into the wrong corner. And how much room a voice can take is best decided together with the client, who knows their narrator.
Why shouldn’t a narrator in 360° speak from the middle of the head? Because the viewer then doesn’t know where to look. If the voice comes from where the person stands, you turn there by yourself.
Is first-order Ambisonics enough for VR films? For many projects, yes. With clear voices and sibilants, though, the low resolution can create artefacts in the opposite ear. A format with more channels solves this.
Where does a voice-over in a second language go? Close to the original voice, but not exactly on top of it. That keeps the link to the person in the picture, and both languages stay easy to understand.
Sound that guides the eye before anyone gets lost.
Get in touch