Projects per year
Abstract
We propose the first approach for the decomposition of a monocular color video into direct and indirect illumination components in real time. We retrieve, in separate layers, the contribution made to the scene appearance by the scene reflectance, the light sources and the reflections from various coherent scene regions to one another. Existing techniques that invert global light transport require image capture under multiplexed controlled lighting, or only enable the decomposition of a single image at slow off-line frame rates. In contrast, our approach works for regular videos and produces temporally coherent decomposition layers at real-time frame rates. At the core of our approach are several sparsity priors that enable the estimation of the per-pixel direct and indirect illumination layers based on a small set of jointly estimated base reflectance colors. The resulting variational decomposition problem uses a new formulation based on sparse and dense sets of non-linear equations that we solve efficiently using a novel alternating data-parallel optimization strategy. We evaluate our approach qualitatively and quantitatively, and show improvements over the state of the art in this field, in both quality and runtime. In addition, we demonstrate various realtime appearance editing applications for videos with consistent illumination.
Original language | English |
---|---|
Article number | 22 |
Pages (from-to) | 1-16 |
Number of pages | 16 |
Journal | ACM Transactions on Graphics |
Volume | 40 |
Issue number | 3 |
DOIs | |
Publication status | Published - 30 Jun 2021 |
Keywords
- Illumination decomposition
- direct and indirect illumination
- real-time sparse-dense optimization
ASJC Scopus subject areas
- Computer Graphics and Computer-Aided Design
Fingerprint
Dive into the research topics of 'Real-time Global Illumination Decomposition of Videos'. Together they form a unique fingerprint.Projects
- 2 Finished
-
Fellowship - Towards Immersive 360° VR Video with Motion Parallax
Richardt, C.
Engineering and Physical Sciences Research Council
25/06/18 → 24/12/21
Project: Research council
-
Centre for the Analysis of Motion, Entertainment Research and Applications (CAMERA)
Cosker, D., Bilzon, J., Campbell, N., Cazzola, D., Colyer, S., Fincham Haines, T., Hall, P., Kim, K. I., Lutteroth, C., McGuigan, P., O'Neill, E., Richardt, C., Salo, A., Seminati, E., Tabor, A. & Yang, Y.
Engineering and Physical Sciences Research Council
1/09/15 → 28/02/21
Project: Research council