BMW and Samsung: Car POV Drive with 3D Audio and VR

A smartphone connecting to a car is pretty boring as a picture. You see a phone in a charging tray and a display where something appears. What makes it special only becomes clear when you sit in the car yourself, drive off and it simply happens.

For Samsung and BMW this became a POV drive in 360°: a BMW 7 Series and a Samsung Galaxy S7 that charges inductively, connects to the car automatically and brings its data onto the display. I was responsible for the sound and built the post-production.

Why a product demo needs the driver’s perspective

Connected car features have a presentation problem. They are invisible as long as they work. The phone charges, the connection is there, the navigation runs, and nobody thinks about it. That is exactly their promise, and exactly what makes them hard to show.

A car POV drive solves this because it puts viewers in the place where the experience happens. You sit behind the wheel, look over the bonnet onto the road and can look around freely in 360°, into the interior, to the passenger seat, at the display. The technology isn’t explained, you experience it along the way, just as a driver would.

For that to work, the sound needs the same perspective as the picture. If I turn around in a 360-degree video and the sound stays stuck at the front, I’m not sitting in the car but in front of a screen. The car has to sound all around me: the road underneath, the surroundings outside, the details in the cabin.

How to record a car you can’t follow

The biggest problem wasn’t the technology in the car but the camera. A 360° camera sees everything. A sound recordist sitting next to it with a microphone, or a chase vehicle following behind, would have been in shot. Following the car was out of the question.

So the approach was different: the recording is set up in the car, started once, and then the people drive off. Only when they come back is the material reviewed. For the sound that means everything has to be right beforehand. Levels that hold for an entire drive without anyone being able to adjust them, and a position in the car where the microphone hides nothing and doesn’t get in the way. Whether it worked, you only find out afterwards. The recording itself was not spatial. The spatial sound image was only created in post-production, as an upmix to Ambisonics.

It worked well.

Where the real work was

It got harder in post-production, which I built. 360° cameras of this kind had no timecode. Picture and sound ran on separate devices with no shared reference, and every drive had to be synced by hand. In a normal film a few frames of offset hardly show. In 360° you hear it immediately, because sound and picture sit at the same point in space and viewers look exactly where something is happening.

After that it was about building a space out of ordinary recordings. In an upmix you distribute the material in the Ambisonics field so that it matches the viewing direction in the 360° picture: the road in front, the surroundings around you, the cabin close by. Every movement of the car and every change in the surroundings has to stay traceable in the sound image, wherever viewers happen to be looking. It plays back as binaural sound over headphones so that the spatial effect works without special monitoring.

What came of it

The result is a 360° demo that makes an invisible feature tangible: you get in, the phone connects, and you drive off. The video runs on my channel and is embedded above.

For me the project shows two things that apply to every 360° production in a car. First, the sound is decided before the drive, because you can’t intervene during it. Second, a 360° shoot needs a plan for synchronisation before the first camera rolls. How a sound system becomes tangible in a virtual car is shown in the Harman/Kardon x BMW virtual showroom case.

Frequently asked questions about 360° car POV drives

What is a car POV drive? A drive from the driver’s perspective. Viewers sit behind the wheel, look over the bonnet and can look around the car freely in a 360° video.

How do you record sound for a 360° car drive? With a recorder in the car that is started once and runs for the whole drive. Following the car or sitting next to the camera doesn’t work, because the 360° camera sees everything. If spatial recording isn’t possible, the space is created in post-production as an upmix to Ambisonics.

Why is synchronisation so important in 360° video? Because sound and picture sit at the same point in space. Many 360° cameras have no timecode, so syncing has to be done by hand, and any offset is noticeable immediately.

What is a car POV video good for in marketing? For features you have to experience to understand, such as connected car functions, driver assistance or a sound system. The technology isn’t explained, it happens during the drive.

Planning a 360° production in a vehicle?

Recording concept, location sound and post-production from one source.