Blended Isolation – Sound Design for a VR Documentary from Seven Isolated Regions

Julia Leeb travels to places no film crew follows. The photojournalist has worked in North Korea, in Congo, in South Sudan, in Libya. For “Blended Isolation” she took a 360° camera to five places cut off in very different ways:

• Democratic Republic of Congo — the jungle territory of a warlord, where she interviews a female rebel about a life between revenge and motherhood
• Transnistria — a state with its own currency and president that no country recognises
• Sudan, Nuba Mountains — an official no-go area, reachable only by cargo plane
• Chile, Colonia Dignidad — the former German sect colony where abuse and torture were hidden for 40 years
• Uganda — a refugee camp for people who fled South Sudan

For the later VR experience, Belarus and Croatia were added, which I scored as well. Seven places, seven distinct sound worlds.

Leeb is among the first German journalists to use 360° for reporting from crisis regions. Her work has screened at VR festivals worldwide and drawn interest from newsrooms ranging from Al Jazeera to the Doha Film Institute. She has described her own aim to deutschland.de like this: with virtual reality she can make the blind spots of our world something you can experience.

In post-production at INVR.SPACE in Berlin I handled the VR sound design and the mix. Sound design is literal here, because nobody was on location for sound: the entire audio layer of this VR documentary was created in the studio.

Why 360° in particular makes sound indispensable

There is one argument for this format that matters more to me than any technical enthusiasm: in 360° you cannot arrange the picture. The usual footage from crisis regions shows a frame chosen for emotion, which often makes it feel overstated. Once the whole environment around the viewer stays visible, that stops working. You see what is really to your left and right.

That promise breaks the moment the sound stays flat. Anyone who can look around while hearing the same stereo track knows immediately that something is off, even without being able to name it. The head turns, the world does not. And then you are a spectator again instead of someone present.

Then there is the practical side. Julia Leeb travelled alone, with a GoPro Omni rig of six cameras and a Gear 360 for situations where a large rig would have been too conspicuous or too slow. Working that way, you cannot bring a sound engineer, let alone set up an ambisonic microphone while life carries on around you. Production audio from those cameras works as a memory of the situation, but not as the basis for a spatial mix.

How seven countries got their own sound worlds

My work started where the picture was already finished. I salvaged what I could from the camera audio and placed every distinct element into the scene as an object: a voice, a tool, an animal. On top of that sits a designed atmosphere that closes the room in the first place, because the rest was too noisy or too compressed to leave standing.

The appeal of this project lies in the contrasts. The inside of a hut in Congo sounds nothing like the ministry of a state that officially does not exist, and neither sounds like a workshop in Colonia Dignidad where every movement rings. Add places only a cargo plane reaches, and a tent camp holding thousands. Five stops mean five acoustic signatures, and each had to be credible although I was never there. The reference was Julia Leeb’s own recordings and her descriptions.

This surfaces a point I often have to explain: the camera position is not automatically the best microphone position. The camera stands where the image happens. Sound would want its place closer to the source so a voice stays intelligible. On a solo shoot in a refugee camp that trade-off disappears anyway, because there is only one position and it belongs to the camera. All the more work is left for the studio.

With voices, direction mattered more than level. In a 360° scene the sound decides where someone looks. If a person speaks behind the viewer, they turn around by themselves, without a cut and without an arrow. That is the real directing work in VR sound design, and it only works when the position is right.

We deliberately pulled the music back. In reportage from regions like these, the temptation to manufacture meaning with music is strong. Instead we decided again and again to turn it down or take it out entirely, so the environment could speak for itself.

What came out of it

The film was completed at INVR.SPACE and screened at VR festivals, among them the programme of the Cinematic VR Festival. The project later grew into the VR app “Brave New Realities”, where you can move freely between all seven places. For the app I also produced the intro and the menu layer, including the static and switching sounds that carry the move between places.

I delivered in several formats: ambisonics for playback in VR headsets, TBE for the app, and a binaural downmix for everything running without one. In 2018 that variety was not a comfort but a necessity, because every platform had its own idea of how spatial audio should sit inside a file.

One limitation belongs in the picture: the public trailer runs in stereo. Watch it and you see the images, but you do not hear the thing this case is about. That is exactly why I am so stubborn about the delivery chain on projects like this. A spatial mix helps nobody if it gets flattened again on the way to the audience.

A second lesson concerns monitoring. A spatial mix cannot be judged on a phone speaker, and yet that is what happens constantly, because it is the fastest route. What arrives there is a downmix without the layer you worked on for weeks. Since then I state at approval which playback is required, and I supply the binaural downmix straight away so nobody ends up reviewing the wrong version.

It went the same way on Crossing Borders, another VR documentary where the sound was what gave the material its direction. What remains is a lesson I have found in every documentary 360° project since: the picture brings people to the place. Whether they stay there is decided by the sound.

Planning a 360° reportage or a documentary VR production?

Even when nobody can cover sound on location, a credible space can still be built. The earlier we talk about it, the less has to be rescued later.