VR: You Haven’t Seen This Movie Before

XR Lab

If you were alive in the year 1890, how would you imagine entertainment in 2020?

Chances are you couldn’t conceive of a world that tells stories using sophisticated special effects, modern editing techniques, and tiny screens you can hold in your hand.

Much as a vaudeville audience in 1890 couldn’t imagine Jurassic Park II, at Pixvana we believe we’re on the precipice of a similar shift in storytelling - one that will be heralded by new virtual reality (VR) technologies. As the cost of mobile headsets decreases, we’re witnessing the first experiments in VR cinema. In fact, VR is a new wave of technology that’s giving birth to a new art form and medium. It will have a unique syntax, tropes of its own, and a new set of best practices. It will be much more than a simple mapping of other types of media, such as TV and movies, applied to a new technology.

Our best hope for understanding what lies ahead for VR may be to look back at how early technologies shaped the history of filmmaking, and look for parallels in the technology shifts of today.

 

History of Storytelling through Film

In the 1890s, before the moving picture camera was invented by the Lumière brothers in France, people were using still photography. The most popular forms of entertainment at the time were overconsumption of alcohol (there were a tremendous number of bars and saloons in the US) and vaudeville.

 

 

Turn of the century pastimes: drinking and vaudeville Turn of the century pastimes: drinking and vaudeville

 

 

When the moving picture camera came along, the earliest films (in the first five or six years) fell into a few categories. One was outdoor scenes: a camera pointed at a real event - people leaving the factory, people leaving the train station, people having dinner on a park bench. The famous train-arriving-at-the-station film literally was just that: one take of a train arriving and people getting off at the station. By some accounts, when audiences saw the film in theaters, they got up and ran in terror, believing they were in danger.

 

 

The second type of early film was, not surprisingly, vaudeville. You can see the aesthetics of vaudeville in early cinema - a lot of staging and performance, and two cameras from a fixed point of view in the audience as if observing the stage. These early films simply duplicated the experience of sitting in the audience to watch a performance.

Uniquely cinematic techniques didn’t start to emerge until the early 1900s. George Méliès’ films marked the beginning of image composition and visual effects that were more like paintings. It was the beginning of a unique visual language. Edwin Porter’s The Great Train Robbery, one of the only surviving films from that time, is almost universally cited as an inflection point of modern cinematic language. Complex techniques such as cross-cutting, close ups, wide shots, and basic tracking and panning were used for the first time. It was the beginning of modern cinema.

 

How a New Technology Evolves

The development phase repeats itself with each new technology. An innovation is introduced and then matures over time; it takes time to standardize. Each innovation requires a period of adaptation and incorporation. Talkies and musicals phased in and overcame silent films quickly, but not overnight. Then the art form matured over a period of fifteen or twenty years.

The notion that a new technology arrives and then shifts everything immediately is false. For instance, people didn’t watch silent movies in silence. There was always music playing, even though it was played live rather than recorded.

For any new media technology, the development phase - which includes a period of experimentation and trial and error - is usually followed by a rapid-adoption phase. With sound, for instance, twenty-five years of experimentation were followed by a five-year period in the late twenties and early thirties of shifting one hundred percent to synchronized sound.

 

 

Development phase vs. rapid-adoption phase Development phase vs. rapid-adoption phase

 

 

Once a new media technology is standardized, it becomes ubiquitous and is a baseline for all future media. No matter the innovation - frame rate (24 fps), color, synchronized sound, stereo sound, surround sound, widescreen aspect ratios, or digital effects—each one goes through a cycle of development followed by rapid adoption and then standardization. A development period of decades precedes rapid adoption, which happens very quickly.

 

 

Standardization – All innovations tend to go through the same three phases: development, rapid adoption, and standardization. Standardization – All innovations tend to go through the same three phases: development, rapid adoption, and standardization.

 

 

What about VR?

Virtual Reality is no longer in its initial development phase. In fact, it’s been around since the 1960s, with a surge in the 1990s. Developers quickly realized that it wasn’t yet ready for prime time back then because the VR experiences tended to make viewers sick. The hardware wasn’t able to deliver the sense of presence that it does now, and didn’t have the lower cost and higher performance level that we have today. After decades of development, VR is now entering the rapid adoption phase. It will be a massive disruption to television, movies, gaming, and other forms of entertainment.

 

 

Early examples of immersive technologies (1950s) Early examples of immersive technologies (1950s) 

KaleidoVR-Film-Festival.jpg

Technology Changes Storytelling

When a new technology arrives, it transforms the stories we tell. It changes not just the way content is made, the economics, and the way it’s distributed, but also the content itself.

This is what happened with cinema when it disrupted the vaudeville stage. It’s also what happened when television encroached on cinema, and again when digital tools became available for production in television and movies. Film editing with a Movieola was slow, but the new digital tools pioneered by AVID allowed editors to make dozens more cuts in the same amount of time.

Die Hard 2, released in 1990, has relatively slow action and fewer cuts, compared to a movie like The Bourne Identity in 2002. It had tighter, almost frenetic cuts. Digital editing led to new kinds of stories with new pacing.

Digital cameras were the next innovation, which fed the enormous growth of reality television in the late nineties. These shows, like Cops and Big Brother, were inexpensively produced with new aesthetics and were easy to scale. They became an important part of television.

The ability to stream shows and have on-demand video through channels like Netflix has changed the content itself. Being able to binge-watch several episodes or entire seasons meant that users could consume content like a novel. Shows could have a more cohesive story. Look at House of Cards or Daredevil to see how content adapts to a new delivery system. When the technology for distribution changed, it changed the content.

 

How Will Stories Be Told with VR?

It’s easy enough to imagine horrific images from cinema, like the monsters from Alien or Nosferatu, in virtual reality environments. What will be different is that they’ll feel even more horrific because they’ll feel real. We’ll have empathy and a feeling of presence, and that will terrify us in new ways. Creature design will be an interesting area of innovation.

Hitchcock inventions became tropes that have been used ever since. In Dial M for Murder, Grace Kelly’s character Margo is on the phone in a pivotal scene, and the murderer (played by Robert Cummings) appears from behind the curtain. The horror to the viewer is seeing the imminent danger to the heroine that the heroine can’t see herself. James Cameron used this same device to great effect in Aliens. The audience can see the horrific creature behind Newt (Carrie Henn) in the water, but she can’t.

In a key scene from the original Alien (Ridley Scott), Ripley packs up on the ship, and we think she’s safe from the alien. But, in fact, the alien’s head is looming right there in the frame, blended in by the art direction to look like it’s part of the ship. This is followed by a jump scare—a hand reaches out, startling the audience. This technique that will carry over well in some sense to VR. As we explore a VR scene and are present, the scene can reveal itself to have more tension than we realized.

 

 

Alien (1979) Alien (1979)

 

 

Psycho demonstrates a hallmark moment in editing. It was meticulously edited to construct terror. Viewers were absolutely sure they saw things, such as blood on the heroine’s body and nudity that weren’t actually shown. The illusion was constructed through editing. It used a difficult technique that won’t map directly to VR because the effect would make people sick, but the point is that new technology inspires new techniques. Filmmakers and storytellers will learn to achieve the same kind of narrative control or projection of their vision using different techniques.

 

 

Psycho (1960) Psycho (1960)

 

 

In the found-footage concept pioneered by The Blair Witch Project, the camera represents the viewer. When the camera points to something horrific, you experience horror more viscerally as a viewer because you feel like you are the victim. The sense of immersion in VR has the potential to generate even more fear in the viewer. There is potential danger in causing trauma for viewers by overusing effects. Being able to dial in the amount of horror is going to be a critical skill.

As mature as cinema is, it’s thrilling to see how much innovation continues. It Follows (2014) was widely admired for its new approach to psychological terror.

 

VR Primetime

Consider a hundred years of the kind of innovation we’ve seen from creative professionals and geniuses in film, but applied to the new medium of VR. Imagine the new kinds of experiences that are going to be told with the unique VR experience: the unique sense of presence.

Read more: Virtual Reality's Fundamental Question, by Jessica Brillhart (Principal VR Filmmaker, Google)

As the technology matures in the next three to five years, you’ll start to see VR innovation, just as we did with The Great Train Robbery, which emerged about five years after the invention of the movie camera. The golden age of VR is ahead of us, and it will be thrilling to watch. Storytellers have always used new tools in new ways. Those who see the future of VR as no more than a spherical, round video - a 360-degree video - based on video from TV or film completely underestimate what creative professionals will do.

Developing VR is a tremendous opportunity that’s going to require an entire industry. Pixvana’s mission is to help realize the potential for mixed reality, virtual reality, and augmented reality storytelling. We’re working on developing cutting edge tools for VR storytellers to innovate with this new medium.

Check out our ongoing technology series about the advantages and challenges of VR video on our YouTube channel.

XR Lab