Games and Films seem like very different media formats. While game audio is interactive, film sound is linear. But that doesn’t mean that they cannot benefit from one other. Get inspired by both worlds.<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_yoast_wpseo_opengraph-image":"https:\/\/www.vrtonung.de\/wp-content\/uploads\/sean-do-EHLd2utEf68-unsplash_tint_1200x630-aimg.jpg","_yoast_wpseo_opengraph-image-id":"18728","_yoast_wpseo_twitter-image":"","_yoast_wpseo_twitter-image-id":"","footnotes":""},"categories":[1],"tags":[22,25],"class_list":["post-2748","post","type-post","status-publish","format-standard","hentry","category-unkategorisiert","tag-news-en","tag-storytelling-en"],"acf":{"remove_from_blog_overview":false,"alternative_title":"","previewimg":18726,"text":"## Written by: Luca Spadavecchia\r\n\r\nThe PlayStation press conference that occurred in June 2020 showed a preview of gameplay and cinematic scenes of some announced game titles. And the impact was massive.\r\n\r\n[ytvideo id=\"tjji8NEW9lo\"]\r\n\r\nWhy am I bringing this up? For the answer, we need to take a step back. Earlier that year, Mark Cerny presented the technological progress of the Playstation 5. On this occasion, the PS5 lead system architect revealed that the next console will have a custom hardware unit dedicated to audio: TEMPEST 3D AudioTech at 38:30 in the video.\r\n\r\n[ytvideo id=\"ph8LyNIT9sg?t=2309\"]\r\n\r\nAs an audio enthusiast and sound designer, I had to process this information. That is why I decided to use this argument as the main topic of my Master of Arts Dissertation. Let me explain how this is relevant to you.\r\n\r\n### Fundamentals of Sound Design\r\n\r\nSound design is a crucial element in filmmaking that enhances the emotional connection to the story. It involves the creation, curation, and manipulation of audio elements to complement and elevate the visual aspects of a film. \r\n\r\nSound design includes dialogue, foley sound effects, original sounds, ambient sounds, music, and more. It can inform the listener of offscreen information, build anticipation, or create surprise. Sound design can transport us through time and space by recreating the ambiance of different eras and environments.\r\n\r\n### The Role of the Sound Designer\r\n\r\nA sound designer is responsible for crafting a film\u2019s sound design. On a small production, the sound designer may be the only person responsible for the film\u2019s audio component. \r\n\r\nOn larger productions, the sound designer leads an audio team consisting of various sound design jobs, including foley artists, audio engineers, re-recording mixers, and more. The sound designer\u2019s work is done during post-production, but they may be involved in the film as early as the pre-production stage. \r\n\r\nA professional sound designer must have a deep understanding of the film\u2019s narrative and tone, as well as the ability to create a balance between different audio elements.\r\n\r\n### Sound Design Techniques\r\n\r\nSound design involves using various techniques, including layering, processing, and sculpting sounds. Sound designers can use these techniques to create a wide range of effects, from subtle ambiance to dramatic sound effects. \r\n\r\nFor example, a sound designer might use layering to create a complex sound effect, such as a spaceship taking off, by combining multiple individual sounds. They might use processing to manipulate the sound of a voice, making it sound like it\u2019s coming from a different environment. \r\n\r\nAnd they might use sculpting to create a unique sound effect, such as the sound of a monster roaring.\r\n\r\n### What can a Sound Designer add to the New Consoles\u2019 Game Audio?\r\n\r\nThe creators of the new generation consoles, PS5 and Xbox series X, announced that the new built-in audio card will allow them to develop more detailed sonic worlds. The consoles engineers managed to solve most of the problems related to sound memory storage, which frees space for sound designers\u2019 creativity. \r\n\r\nThis information made me think about the possibilities of analyzing crucial audio workflows in both film and game productions.\r\n\r\nA video game audio engineer plays a crucial role in leveraging the new audio capabilities of the PS5 and Xbox Series X, creating immersive soundscapes that enhance the gaming experience.\r\n\r\nThe audio description of a space has a pivotal role in the viewer\u2019s immersion. The more accurate the sounds will appear in the space the easier the audience can pinpoint the position of the sources. But how a sound designer can achieve this?\r\n\r\n[image id=\"16401\"]\r\n\r\n### Object-based Mixing\r\n\r\nA sound designer aims to create soundscapes that allow the viewer to extend the screen boundaries. There are many techniques to achieve this goal, but the one I want to highlight is the object-based mixing technique. \r\n\r\nIt consists of encoding all the sound sources that describe the landscape. These sources have parameters that are registered as metadata. They indicate the specific coordinates to allow locating the objects accurately.\r\n\r\nThe audio format delivered in the aforementioned PS5 press conference was binaural. This format allows you to perceive the height of a soundscape. Therefore, the object-based mixing technique, which includes crucial steps of sound mixing, suits perfectly this format since it allows to accurately position the sound sources within the landscape. \r\n\r\nFor the first-ever time, the PlayStation users could experience a taste of the TEMPEST 3D AudioTech.\r\n\r\nThis made me wonder something: if the gamers are experiencing gameplay in binaural format, won\u2019t they notice the differences when they will listen to different audio formats delivered on most of the streaming websites in the market? \r\n\r\nI do, and that\u2019s why I started comparing game scenarios with film sequences to understand why some of the software and workflows used in-game houses are not yet integrated into the film industry. Can binaural audio become a standard in film production?\r\n\r\n[image id=\"16395\"]\r\n\r\n[newsletteroptin text=\"Subscribe to the latest 3D audio news\"]\r\n\r\n## Film and game audio listening scenarios\r\n\r\nObject-based is the technique used in video games to create real-time changes within the sonic world. The game industry avails itself of game engines (i.e. Unity and Unreal) that are able to communicate with middleware (Wwise, Fmod) in order to assign a sound to every moving or stable source in the landscape. \r\n\r\nWhilst this is a standard for game production workflows it is not for films where even if some directors are starting to work with game engines the middleware is still not used. A sound design team plays a crucial role in creating immersive audio experiences in both films and games.\r\n\r\nI want to show you some similar scenarios in which, in my opinion, films got influenced by games mixing techniques. So, grab your headphones, and let\u2019s listen to some examples. Creative sound design is essential in enhancing the storytelling and immersive experience in both mediums.\r\n\r\n### Ratchet and Clank: Rift Apart\u2019\r\n\r\nRatchet and Clank: Rift Apart\u2019s Marcus Smith, during the \u201cSony PlayStation 5 Reveal Event\u201d affirms that the binaural audio characteristics supported by the new generation console represent a fundamental difference between listening to a television set and playing. \r\n\r\nHe says that listening to this new gameplay it\u2019s like going out in the middle of the forest, it brings you into these worlds in a way that you have never been able to do before.\r\n\r\nIn Ratchet and Clank: Rift Apart gameplay the use of the object-based mix is evident. The voices in the mix are perceived according to the rotation angle of the main character\u2019s head. This doesn\u2019t just enhance the gamer immersion, thanks to this audio format we can perceive the height of the spaceship passing from above Ratchet. \r\n\r\nThe sonic world is described as a three-dimensional space. How cool is that? The sound designer can let us hear exactly what the main character it\u2019s hearing.\r\n\r\n[image id=\"16399\"]\r\n\r\n[ytvideo id=\"GffelVJeGws\"]\r\n\r\n### Gravity Film Sound Design compared to Star Wars Battlefront II Gameplay\r\n\r\nCuaron\u2019s Gravity presents similar mixing techniques in the context of the film's sound design. At minute 3:50 the viewer sees the astronaut coming from the right of the scene to the left, following his path towards the back. \r\n\r\nAudio-wise, this scene is described as the audience is listening from where the camera is positioned: the astronaut dialogue is tracked with his body throughout his movement in the space. The listener point-of-audition is located in the POV, which theoretically means this is the same mixing technique used in the aforementioned game.\r\n\r\nSound editing plays a crucial role in achieving these audio effects, distinguishing itself from sound mixing by focusing on the creation and arrangement of specific audio elements during post-production.\r\n\r\n[ytvideo id=\"Ou_04SOqmdU?t=179\"]\r\n\r\nIn Star Wars Battlefront II gameplay the perception of the auditory space drastically changes between the first-person shooter mode and the third-person shooter mode, which occurs by moving the camera behind the player to have a wider view. \r\n\r\nWhen the player chooses the first mode the sound is muffled as the character is in the spaceship. In the second mode, the spectator is always immersed in the sonic world but they are looking at the action from a different POV, hence, they listen to the vastness of the space from outside the spaceship.\r\n\r\n[ytvideo id=\"CBNF6lJTEwE\"]\r\n\r\nThe video below represents the interior of the astronaut helmet; the spectator is, therefore, hearing\/seeing what the character is experiencing. We are not only perceiving the auditory space. The emotional status is rendered by the use of a low-pass filter that conveys confusion. \r\n\r\nGradually, the filter disappears as the camera moves the frame outside of the helmet: here we perceive the scene as a second astronaut, immersed in the three-dimensional space but further from the main character.\r\n\r\n[ytvideo id=\"ZV-UEca2W9U?t=22\"]\r\n\r\n### Shadow of Colossus Sound Design\r\n\r\nIn Shadow of Colossus, the player apprehends the action from a third-person camera angle. When the character goes underwater the player understands it thanks to the splash sound effects. But if the camera doesn\u2019t go underwater, the gamer will still hear the scene from outside of the water. \r\n\r\nOnce the camera reaches the character underwater, the player perceives the ambiance as they were immersed in the water. Hence, The Point-of-Audition is connected with the Point-of-view.\r\n\r\n### Blade Runner 2049 Film's Sound Design\r\n\r\nBlade Runner 2049 has similar action scenes. The one displayed below describes a fighting scene where a character is trying to drown his opponent. How the sound mixer designed the audio follows the same concepts of Shadow of Colossus.\r\n\r\n[ytvideo id=\"pDTGvPwTBDI?t=125\"]\r\n\r\n### The Sound Design Process\r\n\r\nThe sound design process involves five steps: preproduction, recording, editing, mixing, and mastering. During preproduction, the sound designer works with the director and other crew members to develop a plan for the film\u2019s sound design. \r\n\r\nDuring recording, the sound designer captures the necessary audio elements, such as dialogue, foley sounds, and ambient noise. During editing, the sound designer assembles the audio elements into a cohesive soundtrack. During mixing, the sound designer balances the levels of the different audio elements to create a balanced audio mix. \r\n\r\nAnd during mastering, the sound designer prepares the final soundtrack for distribution. Throughout the process, the sound designer works closely with the film\u2019s editor, composer, and other crew members to ensure that the sound design complements the visual elements of the film.\r\n\r\n## Conclusion\r\n\r\nThe film industry is already taking advantage of game software, mostly for reaching outstanding visual results. The Mandalorian, Spielberg\u2019s Ready Player One, and Villeneuve\u2019s Blade Runner 2049 used Unreal Engine to visually look at the VFX results in real-time through the virtual camera plug-in directly on-location. \r\n\r\nWhy the sound isn\u2019t considered yet in this process?\r\n\r\nWe have seen how the game experience is not so far from the film one. Now that the game audio is taking a step further, I\u2019m expecting the film audio will react reaching another level.\r\n\r\n###Let's talk about your project\r\n\r\nLooking for spatial sound that gives your film real depth?\r\n\r\n[Get in touch \u2192](https:\/\/www.vrtonung.de\/en\/contact\/)\r\n","related_articles":[1555,5087,6845,3115]},"yoast_head":"\n