Video for Virtual Reality

Christian Richardt, James Tompkin, Jordan Halsey, Aaron Hertzmann, Jonathan Starck, Oliver Wang

Research output: Contribution to conferenceOtherpeer-review

2 Citations (SciVal)
138 Downloads (Pure)


Video can capture the dynamic appearance of the real world in a way no other technology does; virtual reality technology, on the other hand, enables the display of dynamic visual content with unparalleled realism and immersion. The fusion of these two technologies—video for virtual reality (VR)—promises to enable many exciting photo-realistic experiences.

Over half a day, this course will provide an overview of three aspects of this exciting medium: the technical foundations, current systems in practice, and the potential for future systems of VR video.

In the first section, we will explore the geometric and optical problems underpinning VR video. Then, we will introduce both 360 degree video and stereoscopic video, including how 360 video is captured, analyzed, and stitched, including the mathematics behind how stereo 360 video can be captured.

This background material provides the prerequisites for understanding current systems in use. In the middle hour of the course, we explain how state-of-the-art stereo 360 video is produced from camera systems and computational processing. Then, we will consider the art of storytelling in VR, and how new tools for editing VR video can aid in the craft of this art production. Finally, this section provides an industry perspective covering current production and post-production choices and practice, including CG integration.

The final part of our course focuses on the next generation of video for VR, where we move to 6 degrees-of-freedom (6DoF) experiences. We introduce the basics and challenges behind light field cameras, processing and displays, and see how they can enable 6DoF experiences. This will be followed by another industry perspective on how light field camera arrays have been used to create cutting-edge experiences integrating volumetric live-action elements. To conclude the course, we will see how far we still must go toward the ideal system, in hopes of inspiring the attendees to push the boundary farther to reach it.

We hope this course is useful to a broad audience—at SIGGRAPH and beyond—as we cover the academic, artistic, and production sides of VR video.
Original languageEnglish
Publication statusPublished - 30 Jul 2017
EventSIGGRAPH 2017 - Los Angeles, USA United States
Duration: 30 Jul 20173 Aug 2017


ConferenceSIGGRAPH 2017
Abbreviated titleSIGGRAPH
Country/TerritoryUSA United States
CityLos Angeles
Internet address


Dive into the research topics of 'Video for Virtual Reality'. Together they form a unique fingerprint.

Cite this