There is a magic; an experience of presence when you’re in a virtual world that is like nothing else. Suspended disbelief is a beautiful thing.
But what makes that world immersive and allows you to feel like you’re really there?
VR Environments
Essentially there are two types of VR environments:
3D CGI
This is a graphics-based world. It creates a physiological illusion similar to our real world, but is made up of simulated graphics.
The opportunities in this CGI environment are limited only by one’s imagination and the project budget, the latter being critical. The cost of creating immersive worlds through 3D graphics can be considerable, and so, not as easily accessible to most content producers.
Video
Video is a strongly immersive world. With an environment familiar to our natural surroundings, video triggers an emotional orientation that we’re accustomed to. It beckons us in and allows us to forget the goggles on our heads.
Unlike 3D CGI, video is accessible to a large audience of content producers, because it can be created at a significantly lower cost. It is for that reason that I’m pretty excited about VR. With improved technology coupled with continued accessibility, this medium of immersive storytelling will be revolutionary.
Video Technology
Spherical Video
This method utilizes video images stitched together.
A rig of cameras shoots for full 360 degree coverage. These rigs can be used with any level of camera from GoPro to Red.
Once shot, the footage is stitched together to be accessible in 360 degrees. As you can imagine this involves a lot of data that needs to be processed through various tools, organized, and stitched together. But at the end of the day, I believe these videos will be the low-cost approach to incredibly immersive storytelling.
Computational Video
Microsoft has done some really exciting work in this arena. Rather than shooting from a rig of cameras filming outward, this approach utilizes a series of cameras shooting inward.
Footage from each of the cameras are re-applied to the scene as textures. The result is a world that the user can walk around and immerse themselves in.
Plenoptic Systems
This works with light fields shooting from thousands and millions of small lenses. The end result is an image that you have far more control over than RAW photos.
Challenges
So what are some of the technological obstacles in the way? A good virtual world simulates reality both physically and visually. Here-in lie two challenges:
Parallax
Your head can move in 6 degrees. During this movement, your brain receives visual cues from the scene before you, helping you to determine proximity. Objects at a farther distance in view appear to move more slowly than objects that are closer to you.
Simulating this movement is fairly simple to modify with CGI, but poses more of a challenge with video when it cannot simulate intricate movement with the same precision and subtlety as our heads. The parallax of the scene is limited to the location of each lens that recorded it.
Projections
When the footage from the rig is stitched together, it creates a sphere. This allows you to look around in every direction, but the process requires image distortion in order to fit the sphere.
Think of a globe. Locations in the high and low hemispheres are distorted to fit the spherical shape.
Greenland, for example, often appears to be an extremely large land mass when in reality it is much smaller.
In the virtual world, these problems of parallax and projections, although seemingly subtle can be major roadblocks preventing the brain from suspending its disbelief and immersing in the world.
Melanie Moore 