Humboldt 360° – Bilingual Spatial Sound for the Goethe-Institut

Alexander von Humboldt travelled with instruments. Barometer, sextant, thermometer, everything that could turn a foreign world into numbers. Two hundred years later a film crew travels the same route with a 360° camera, which at its core is the same idea: recording a place so that others can follow it without having been there.

For the Goethe-Institut this became “Humboldt 360°”, four episodes along Humboldt’s route through Colombia and Ecuador. I built the spatial sound version for it, in German and in Spanish.

Why a dubbed voice in 360° must not sit in the middle

The films were already edited when I joined. There was a Spanish original and a German dubbed version, both for the institute’s YouTube channel. And that is exactly where the problem lay that the team was stuck on.

360° video on YouTube supports first-order Ambisonics, four channels that describe the sound field around the head. What it does not have is a second, head-locked track. Sound that stays in place when you turn your head (head-tracked) and sound that is fixed to your head (head-locked) cannot be stored separately on YouTube. Facebook could do this at the time, YouTube could not. In practice this means: everything you want to hear has to go into the sound field.

For a dubbed version that is awkward. In a normal film you put the German voice where the original sound was, and that is usually the centre. In 360° there is no centre, because the viewer turns their head. If you still fix the dubbed voice somewhere, the viewer hears someone talking, turns towards where the voice comes from, and finds nobody there.

There is a second layer that has to be separated cleanly. Some voices belong to the scene, because the person is in the picture and speaking. Others are narration, a comment from outside that has nothing to do with the place. In a normal film this distinction hardly matters. In 360° it decides whether a scene holds together.

How four finished films got a space

The work had three layers, all with the same goal: solving head-tracked and head-locked within a single Ambisonics track so that you can hear the difference.

The scene voices. They sit in the sound field where the speaking person actually stands, and in the same place in both language versions. When Ernesto Mamanche, leader of the Muisca community in Sesquilé, sits to the right in front of his hut, his voice comes from front right, whether you listen to the Spanish or the German version. That turns an interview into a conversation in which you turn towards the person.

Because the camera moves in several scenes, in the boat along the coast or on the way up the mountain, a fixed position was not enough. The voices travel through the space via automation so they match the person in the picture. That is the part of the work nobody notices when it is right.

The music. It came as a stereo file. Simply placing it in front would have been the easy way, but it would have set the same trap as a misplaced dubbed voice. So I converted it into the Ambisonics field and spread it so that it carries the space instead of sticking to one spot.

The narration. I deliberately treated it differently. It is not part of the place and should not sound like it, otherwise the audience looks for someone who isn’t there. It stays wide and unlocalised, and because everything else has a clear direction, you hear the difference immediately. I described how this workaround works technically in a separate article: mixing head-locked stereo with Ambisonics.

What was delivered

In the end every episode went out in two languages and two forms: a first-order Ambisonics master for the 360° version on YouTube and a stereo version for everything else. That makes eight masters and eight stereo files, delivered in three rounds in June 2019.

The four episodes lead from the arrival on the Colombian coast via Bogotá and the Muisca community in Sesquilé to Quito and finally up Chimborazo, the mountain where Humboldt had to turn back at over five thousand metres. The films were released in 2019, the year of his 250th birthday, in the Goethe-Institut’s digital culture magazine “Humboldt für Südamerika”, funded by the German Federal Foreign Office.

According to the credits, direction, camera and editing were by Tim Hamelberg, production management by Christian Stein, concept by Thomas Lilge, produced by Playersjourney UG. The voices were Bettina Conradi and Christian Conradi, translation by Lina-Marie Büttner, stitching by Matias Bertoia. I was responsible for the spatial sound version in both languages.

What I take from this project applies to every bilingual 360° production: the dubbed version has to serve the same positions in space as the original. Otherwise the two versions fall apart into two different films, and only one of them works. How a 360° documentary sounds when the sound is planned from the start is shown by Crossing Borders. The German version of Emperor was a similar case, where every language file had to fit an existing timeline exactly.

Frequently asked questions about dubbing 360° videos

What is the difference between head-tracked and head-locked? Head-tracked sound stays in place when you turn your head, like a real sound source. Head-locked sound turns with your head and always sounds the same, for example a narrator or music without a location.

Can YouTube do both in one 360° video? No. For 360° video YouTube only accepts first-order Ambisonics, there is no second head-locked track. Everything has to be solved inside the sound field.

Where does the dubbed voice go in 360°? In the same place as the original voice, where the person in the picture is speaking. Only the narration stays unlocalised, because it does not belong to the place.

Planning a multilingual 360° production?

If your film is going to run in several languages, how much extra work the second version costs is decided early. Talk to me before the first mix is done.