This story appears in the October 2020 issue of Television Asia Plus.

The pandemic has pushed producers and filmmakers to be more creative in their storytelling. Despite the lockdowns in place, many stellar examples of remote production came to the forefront, showcasing the industry’s determination to create. This has opened up the pathway for the industry to deliberate on its future and how virtual and remote production can be woven into the filmmaking process. 

Filming is still inefficient

Over the past 40 years, technology in filming has progressed greatly. Animation and special effects are continuously becoming more realistic, making possible what was never imagined just two decades ago. 

However, the production process remains riddled with both inefficiencies and high costs, and real solutions have been hard to come by. 

In the pre-production phase, planning and visualising complex scenes takes a significant amount of time, particularly when alignment between the creative vision and technical realities is required. Complex movie sets can sometimes take months to build, and unfavourable weather and light conditions often throw the filming schedule into disarray. Even after all the footage is captured, post-production can take another year or more. 

In a post-pandemic world, producers with tightening budgets can’t afford to allow inefficiencies to seep in. Virtual production and leveraging real-time video game technology might be the answer.

Reimagining filmmaking

Virtual production, powered by video game technology, can be a gamechanger in the creation of content, ranging from DIY digital productions to future blockbusters.

With its ability to save on production time and costs, virtual production can be a lifeline for filmmakers of all sizes. For instance, with the use of video game technology, filmmakers can build digital environments that mirror the quality of final frame renders. This gives everyone in the production team the same vision of the final product. Changes can also be incorporated through an iterative and collaborative process, essentially transforming the rigid, linear pipeline of traditional filmmaking into parallel processes that cut down the overall timeline of a production. Creative decisions about shots and sequences can be resolved much earlier in the process and not left to post-production. 

Another core benefit of using a game engine is that the technology is real-time, providing an incredible advantage for virtual production. When applied to filmmaking, live rendering alleviates the uncertainty of traditionally siloed pre-production and visual effects production, as they are replaced with working imagery that is far closer to the final cut.

Extended post-production cycles are par for the course in the making of visual effects-intensive movies. As a result, it is often difficult for filmmakers to visualize final screen output while directing on the live-action set. With in-camera visual effects captured on LED stages, final frames can be previewed through the camera lens, and creative teams can manipulate lighting, virtual environments, and effects collaboratively on set.  This more intuitive process also means the crew can make adjustments during shooting, avoiding the need to compromise the final cut after shooting has wrapped. 

Pushing the boundaries with photorealism 

Will the sense of realism and excitement be lost in virtual production? On the contrary, by replacing traditional green screens with scene projections, actors can get a better sense of the environment they are acting in and the shared vision of the production crew. 

With real-time rendering, the background can adapt according to the camera’s perspective, making the entire scene more interactive. The real-time nature of an LED stage also allows for movement of objects in a scene, such as the position of the sun or other light source, or trees moving in a breeze. Physical set pieces in the foreground can be merged seamlessly with digital environments and objects, heightening the sense of realism. 

Overall, the ability to view shots that are very close to final helps to ensure the continuity, fluidity, and agency of each scene, which potentially shaves off days of unnecessary shooting and months off post-production. 

A natural evolution for the entertainment industry

Virtual production is not new. The award-winning Game of Thrones series utilised virtual production techniques to exceed the audience’s expectations of the series. In the beginning, virtual production was used in previsualisation to ensure the alignment of creative and technical vision across film departments. As the storylines got more complex, a virtual scouting toolset—powered by Unreal Engine—was also deployed to stage elaborate and highly imaginative sets. 

In The Mandalorian Season 1, Epic’s Unreal Engine was used to project lifelike virtual environments onto massive LED screens that immersed the actors and crew in the Star Wars universe, giving directors full visibility and creative control of digitally enhanced sequences while shooting on set. 

Virtual production allows film departments to work more closely in real-time, to reach the shared vision of the final product in the most time and cost-efficient manner. At a time when the industry is grappling with the fallout of the global pandemic, virtual production may be the natural evolution for the entertainment industry. 

Quentin serves as Epic Games’ General Manager of Southeast Asia and India and is leading the company’s expansion into the region. A data-driven business leader, passionate about creative content, social media, and disruptive technologies, Quentin holds over 25 years of experience at the intersection of media and technology. He is also a game engine guru, having held previous regional and C-suite roles at Unity Technologies, Improbable, and Take-Two Interactive Software.