Last month, when Apple released “Submerged,” its first original scripted film captured in immersive video exclusively to Vision Pro, director Edward Berger (Oscar nominated for 2022’s “All Quiet on the Western Front”) declared it would change the future of filmmaking.
And it’s true. Immersive technologies such as advanced spatial tech, LED screens, dome projections and a new generation of headsets — think Vision Pro, Meta Quest and HTC Vive — are indeed creating previously unimaginable storytelling possibilities, despite the hardware still not achieving mass adoption.
In the case of “Submerged,” rather than staring at a screen watching actors trapped in a submarine, Vision Pro wearers can viscerally submerge themselves in the underwater claustrophobia.
Unlike the first wave of XR a decade ago, today’s consumers are primed for the opportunities created — and deliverable — by XR, with COSM, Sphere and Netflix Houses just a few of the uniquely revolutionary entertainment spots driving the push.
Beyond the emergence of dome screen venues, though, success in this realm will require new skills and novel ways of thinking on the part of director and screenwriter. Consider these examples of how immersive tech will evolve both these roles and the industry at large.
Full-Body Motion: A maker of immersive content needs to understand not only plot and character but space, because the audience must literally move through the film to interact with characters as the director maps out the scenes. Successful delivery opens a huge creative opportunity because it palpably heightens the intensity of emotions and connection to the story.
The director can give viewers the ability to move quickly if they’re scared or move slowly if they’re hesitant, to duck under large objects being hurled at them, to peek around corners. To be clear, this is not like watching a 3D movie; rather, this is navigating 3D space. Storytellers with strong backgrounds in production design, theater and improv will excel.
Multisensory Experience: Immersive storytelling leverages spatial technology to put the viewers in an environment as close to reality as possible. With its “magical realism,” directors have the opportunity to target to the senses they want viewers to experience. Think about a scene where audiences are following the action by walking across slippery paving stones. In XR, that sense of wetness is so real the viewer will actually walk gingerly across the rocks, slowing the movement of the story and thus building anticipation.
VIP+ Special Report: A Data-Filled Look at the State and Future of Apple Vision Pro & Entertainment
A director can leverage these technological capabilities to enhance a moment and add pulse-pounding effect. One early and outstanding example of a filmmaker exploring this advance is Alejandro G. Iñárritu’s “Carne Y Arena,” a six-and-a-half-minute virtual reality installation conveying the harrowing experience of refugees crossing the southern U.S. border, traversing a barren sandy space while barefoot and afraid — viewers can literally feel the travelers’ thumping hearts. The groundbreaking project won Iñárritu a special Oscar in 2018.
Social Cinematic Storytelling: Immersive entertainment is also making possible the kind of deeply engaging repeat experiences the gaming industry has long delivered — and Hollywood has envied. People are clearly looking to enjoy and document their experiences with friends. It’s why the ABBA Voyage experience in London, Vegas’ Sphere and even the traveling Van Gogh and Monet immersive exhibits, each notching its own content investment and audience price but all placing the viewer into the experience, have seen tens of millions of exhilarated visitors.
Tomorrow’s Movie Theaters: After two decades mourning the loss of U.S. movie theaters with the advent of streaming, immersive technologies are revitalizing local theatergoing with the interactive, social and shareable moments audiences are craving. Netflix is out in front with its planned Netflix Houses, aka experiential entertainment venues, aimed at taking fans beyond the screen in Netflix titles that have gone viral.
COSM has raised $250 million to build “shared reality” venues around the country for an industry that’s predicted to generate $426 billion by 2030. The experiential hub allows fans to watch sports together on a massive dome screen, but future events will move past screens to let fans co-play with their favorite game characters in an interactive space, such as played out at augmented reality Pokemon Go events.
From Viewers to Visitors: One trick for filmmakers venturing into immersive content is to think about the audience as more than mere observers and instead participants of sorts who will become lost in time and space, spending hours engaging with characters in the world you have created. This is what today’s cinephiles want, just as die-hard rock fans want to feel the vibes at Coachella. Filmmaking in XR is about experiencing the fans, the moment, the soundtrack and the full-body engagement. With the same budget as, say, a “Joker 2,” directors can create such a place.
While the tools are here, the challenge for screen-based content creators is to push through existing forms and wholly reimagine how great tales can be delivered. Good stories don’t get old, but formats do. Immersive technology can revitalize the movie industry and allow new life to be injected into old stories so fans can truly re-experience them.
Anthony Batt is co-founder of Wevr, a creative development and production studio specializing in interactive and spatial experiences. He previously served as head of digital for Ashton Kutcher at Katalyst.