Real-time Virtual Object Insertion for Moving 360° Videos

Research output: Contribution to conferencePaper

1 Citation (Scopus)
81 Downloads (Pure)

Abstract

We propose an approach for real-time insertion of virtual objects into pre-recorded moving-camera 360° video. First, we reconstruct camera motion and sparse scene content via structure from motion on stitched equirectangular video. Then, to plausibly reproduce real-world lighting conditions for virtual objects, we use inverse tone mapping to recover high dynamic range environment maps which vary spatially along the camera path. We implement our approach into the Unity rendering engine for real-time virtual object insertion via differential rendering, with dynamic lighting, image-based shadowing, and user interaction. This expands the use and flexibility of 360° video for interactive computer graphics and visual effects applications.
Original languageEnglish
Pages1-9
Number of pages9
DOIs
Publication statusPublished - 14 Nov 2019
EventInternational Conference on Virtual-Reality Continuum and its Applications in Industry - Brisbane, Australia
Duration: 14 Nov 201916 Nov 2019
Conference number: 17
https://vrcai.siggraph.org/

Conference

ConferenceInternational Conference on Virtual-Reality Continuum and its Applications in Industry
Abbreviated titleVRCAI
CountryAustralia
CityBrisbane
Period14/11/1916/11/19
Internet address

Projects

Cite this

Tarko, J., Tompkin, J., & Richardt, C. (2019). Real-time Virtual Object Insertion for Moving 360° Videos. 1-9. Paper presented at International Conference on Virtual-Reality Continuum and its Applications in Industry, Brisbane, Australia. https://doi.org/10.1145/3359997.3365708