How to Record Spatial Audio for a VR Boxing Experience (ranFIGHTING Pro7 Sat1)
In this article, we focus on the unique challenges and insights of capturing spatial audio for a 360° video in the world of boxing. The production followed WBC world champion Tina Rupprecht on her journey from intensive training preparation to the pivotal fight, all captured in immersive 360° perspective.
Capturing high quality spatial audio is crucial for producing an immersive 360° video experience, allowing listeners to experience sound from all directions.
In addition to recording spatial audio on-site, I handled the complete post production process—from stitching to sound mixing. Music production played a central role in creating an immersive audio experience. Originally, the video was available on Facebook 360 but has since been taken offline due to licensing restrictions.
Introduction to Immersive Audio
Immersive audio is revolutionizing the way we experience sound by creating a three-dimensional sound field that envelops the listener. Unlike traditional audio, which is often limited to left and right channels, immersive audio places sounds all around you, making you feel as if you are right in the middle of the action. This technology is particularly popular in virtual reality, video production, and music, where it enhances the sense of presence and realism.
There are several techniques to achieve immersive audio, including binaural recording, ambisonic recording, and object-based audio. Binaural recording uses two microphones to mimic the way human ears perceive sound, creating a natural and lifelike listening experience. Ambisonic recording captures sound from all directions using a multi-capsule microphone, allowing for flexible post-production manipulation. Object-based audio, on the other hand, treats each sound as an individual object that can be placed and moved within a 3D space.
As surround sound continues to grow in popularity, its applications are expanding. From creating more engaging virtual reality experiences to enhancing the depth and richness of music, immersive audio is transforming the way we listen and interact with sound.
Capturing the Essentials: 3D Audio from Training to the Ring
From the initial drops of sweat in training to the ring walk and fight itself, Tina Rupprecht’s journey to the match in her hometown of Augsburg was brought to life in a comprehensive VR boxing experience. The goal was to draw viewers directly into the action, giving them a front-row seat to an intense and captivating journey.
Each detail—the creak of the ring floor, the breathing, and the sound of punches connecting—was authentically and vividly captured, creating a nearly tangible experience. An immersive spatial audio workflow was employed to capture the sounds of the boxing match, utilizing advanced equipment and post-production methods to enhance the spatial audio and complement the visual storytelling. Much of today’s music is available in Dolby Atmos, which fully realizes the potential of 3D audio technologies.
Portable equipment setup under Tight Deadlines
The entire production, from concept through recording and final mixing, had to be completed within a week. This required the audio engineering team to ensure high-quality audio under tight deadlines by coordinating all equipment and crew quickly, executing all audio recordings as efficiently as possible.
The noise level in the hall—from cheering fans and speaker announcements to the rhythmic, heavy beats of the music—pushed the recording equipment to its limits.
Fundamentals of 3D Audio and Spatial Sound
What is 3D Audio?
3D audio is an advanced audio recording technique that allows sound to be perceived spatially. Unlike traditional stereo audio, which only conveys left-right audio channels, 3D audio introduces depth, height, and spatial accuracy, allowing listeners to experience sound as if it were coming from specific directions. This is especially useful in immersive formats like VR and 360° videos. Spatial audio is becoming a standard audio format due to its advantages over traditional formats, such as ease of use and distribution, making it more accessible to music creators and consumers.
The Role of ambisonic recordings in Immersive Experiences
Spatial sound takes immersive experiences to a new level by placing the viewer at the center of the action. Immersive video formats enhance the viewing experience by providing sound in all directions. In the VR Boxing Experience, spatialized sound design allowed viewers to feel the visual and auditory dynamics of the boxing match up close. From the crowd’s cheers to the sound of punches and ring announcements, spatial sound created a comprehensive atmosphere that went beyond visual impact alone.
Technical Implementation in a Limited Timeframe: Immersive Spatial Audio Workflow
Due to tight deadlines and platform limitations, the project was finalized in stereo recording format. Creating a spatial audio mix is crucial for achieving high-quality spatial audio, but it requires specific mixing expertise in post-production. Creating a complete 3D audio mix with various microphones and setups to capture the X, Y, and Z axes would have been unfeasible in the available timeframe. Since the video would primarily be accessed on mobile devices, stereo recording was the most pragmatic solution. This approach still provided high-quality sound that delivered an immersive experience, even without headphones.
Choosing the Right Equipment for Recording Spatial Audio
Recording spatial audio requires specialized equipment that can capture the intricate details of a three-dimensional soundscape. The choice of equipment depends on the type of recording you plan to make, the environment in which you’ll be recording, and your budget.
An Ambisonic microphone, such as the Sennheiser AMBEO VR Mic, are a popular choice for capturing spatial audio. These microphones use multiple capsules to record sound from all directions, providing a comprehensive and immersive audio experience. Binaural microphones, like the Neumann KU 100, are another excellent option, especially for creating recordings that mimic human hearing.
Portable recorders with built-in multi-mic arrays, such as the Zoom H3-VR, offer a convenient and versatile solution for on-the-go recordings. These devices are particularly useful for capturing spatial audio in dynamic environments, where quick setup and ease of use are essential.
In addition to the hardware, the software and plugins you use for processing and mixing your recordings are crucial. Tools like Pro Tools, Reaper, and specialized plugins for spatial audio can help you refine and enhance your recordings, ensuring they deliver the desired immersive experience.
By carefully selecting the right equipment and software, you can effectively capture and reproduce the nuances of spatial audio, creating a truly immersive listening experience.
Microphone Setup and Recording Techniques for 3D Audio in Boxing
Different Microphone Setups
Various microphone setups were used in the VR Boxing Experience to achieve the best sound quality for an immersive listening experience. Comparing the compact Zoom H3-VR to the Sennheiser AMBEO® VR Mic highlighted the benefits of both ambisonic microphones:
The Zoom H3-VR was ideal for fast deployments and a simple setup, making it great for spontaneous recording spatial audio. Meanwhile, the Sennheiser AMBEO VR Mic provided greater detail and spatial accuracy, thanks to its ambisonic audio capabilities.
Additionally, the pseudo binaural recording method offers a cost-effective approach to capturing binaural audio using standard equipment, such as lavalier microphones attached to a baseball cap. This method is effective for achieving a wide sound field for ambience recording.
These ambisonic microphones captured the sound of the boxing ring so that viewers felt as if they were standing ringside, hearing the action from all directions.
Direct Microphones and 3D Recorder setup
Direct microphones play a critical role in 3D recording by focusing on the sound source from specific directions. Techniques and technologies to record binaural sound, such as using binaural dummy heads and Ambisonic microphones, are essential for capturing immersive audio. In the boxing ring, these microphones captured the sounds of punches and movements directly, while other microphones recorded crowd noise and environmental sounds. This combination provided a dynamic and immersive audio landscape, positioning the viewer at the heart of the action.
Advantages of Ambisonic Technology for capturing audio in the Boxing Ring
The advantage of ambisonic technology is its ability to record sound in all directions, creating a realistic soundscape that immerses viewers in the action. By using ambisonic audio formats, a 3D sound space is generated, dynamically adjusting sound volumes and positions based on the listener’s location. This creates a truly authentic spatial perception for the viewer, ideal for the intense soundscape of a boxing match.
Overcoming Common Challenges in Spatial Audio Recording
Recording spatial audio can be a complex task, especially when dealing with intricate environments or multiple sound sources. Some common challenges include ensuring accurate microphone placement, managing noise and interference, and achieving a balanced mix.
Accurate microphone placement is critical for capturing a realistic and immersive soundscape. Experimenting with different positions and configurations can help you find the optimal setup for your recording environment. Using high-quality ambisonic or binaural microphones can also enhance the spatial accuracy of your recordings.
Noise and interference are other significant challenges in spatial audio recording. To manage these issues, it’s essential to use denoisers and equalizers during post-production to minimize unwanted background noise without compromising the overall audio quality. Additionally, selecting recording locations with minimal ambient noise can help reduce interference.
Achieving a balanced mix is crucial for creating an immersive medium experience. Understanding audio engineering fundamentals, such as gain staging, EQ, and compression, can help you achieve a well-balanced mix. Using high-quality equipment and software plugins can also make a significant difference in the final sound quality.
By addressing these common challenges and employing best practices in audio engineering, you can overcome the hurdles of recording spatial audio and create high-quality, immersive audio experiences.
Post Production and Sound Mixing in VR
Challenges of Post Production for a virtual space
The post-production and audio engineering for the VR Boxing Experience required targeted audio optimization to immerse the viewer in the action. Denoisers and equalizers were crucial in minimizing unwanted background noise, like the rumble of the tractor during the boxer’s entrance, without compromising the overall audio quality.
A specially encoded stereo file was used to deliver a full 360-degree soundscape that must be experienced through headphones. This format captures sound as it reflects around the head and mimics human hearing, creating audio files with an ambisonic mic.
Balancing noise reduction with the enhancement of essential sounds was key to achieving clean, dynamic audio quality.
The Mixing Process in Pro Tools
The mixing process was executed in Pro Tools, allowing for precise editing and optimization of the final product for social media platforms, particularly Facebook 360. Since the video was intended mainly for Facebook, sound design had to be adjusted to platform requirements. This included balancing sound to ensure it performed well on both mobile devices and various audio quality levels. The final mix guaranteed that the immersive audio experience would impress both headphone users and mobile speakers, immersing viewers fully in the action.
Application of 3D Audio in Immersive Sports Experiences
Importance of 360° Sound in Boxing
Boxing offers an ideal setting for 360° sound and VR technology, as the direct proximity to the action, combined with the sounds of the ring and audience, heightens the excitement. For a sport that combines power and precision, 3D audio perfectly complements the sport’s dynamic nature.
The Potential of VR Boxing for Sports
With VR boxing, the potential for immersing the audience in the boxing world is significantly enhanced. This technology allows viewers to feel a stronger emotional connection, as if they’re in the ring. The blend of 360° sound and VR visuals creates an impressive atmosphere that appeals to a wide audience. This format represents a promising avenue for sports broadcasting, offering a new level of audience engagement that elevates sports to a more emotional plane.
Conclusion: Collaboration and Coordination in Production
The successful completion of this project was largely due to close collaboration and coordination with Concept360 and the TV team on-site. With the support and expertise of Matthias Taugwalder from Concept360, a pioneer in 360° video and panoramic photography, visual and audio elements were seamlessly integrated into the VR Boxing experience.
Despite the tight deadlines and challenging conditions in the ring, audio quality and production efficiency were maintained at a high level. The collaboration of all team members was crucial in creating an immersive VR experience in record time, transporting viewers into the world of boxing and conveying the energy and intensity of a boxing match in a completely new way.
