Step Within a VR Video Stream

XR Lab

abstract visual of virtual reality VR video streaming

There are many complexities involved with presenting spheres on planes, and it's a key challenge for delivering content that feels immersive. 

 When you put on VR goggles and watch a 360-degree video, you feel that you are in the center of a sphere. Actually, you are inside of a geometric box that only appears to be a sphere.

 

Gif from Stanley Kubrick's 2001: A Space Odyssey

 

At Pixvana, we call this shape a "viewbox". Our XR Lab is testing a variety of viewbox projection layouts to understand what method best optimizes the quality and speed of VR video streaming. The goal is to create a video stream that maximizes resolution while minimizing bandwidth.

 

The Pixvana SPIN Technical Preview (download on Steam for Oculus Rift or HTC Vive) has a special capability where you can experience the effect of projecting a sphere onto a flat surface within a VR video stream. Our field-of-view adaptive streaming (FOVAS) technology optimizes pixel resolution based on where you are looking. In the Technical Preview, you will see the projection of a frustum when you hit pause and expose the viewbox. We have discovered that this shape reduces bandwidth requirements to under 4mbits while delivering ultra high resolution.

 

Frame-by-Frame Tour of the Pixvana SPIN Technical Preview

 

With the headset on, press pause and the still frame will show "foveated pixel density". The highest resolution will exist at the "front" of the video stream. In this case, the fashion models are at full resolution while the photographer toward the "side" appears at lower resolution:

 

Pixvana Technical Preview Tour 1

 

Directly behind the models is the photographer. He appears blurry because he is viewed at the out-of-field area before the switching occurs. However, when you turn your head to look at him then FOVAS will "switch" and quickly increase the resolution. The latency is almost imperceivable at 100 - 200 milliseconds. Here is the photographer when he is viewed at the out-of-field area before the switching occurs:

 

Pixvana Technical Preview Tour 2

 

After the switching, the photographer comes into the current optimized field-of-view, which delivers full resolution from the source file (in this case a 10k master resolution film):

 

Pixvana Technical Preview Tour 3

 

This is what the actual video stream would look like:

 

Pixvana Technical Preview Tour 4

 

Here is the full 10k frame as a latlong - a file that is way too large for a headset to process or to stream!

 

Pixvana Technical Preview Tour 4

 

XR Lab