Projects per year
Abstract
We present a novel real-time approach for user-guided intrinsic decomposition of static scenes captured by an RGB-D sensor. In the first step, we acquire a three-dimensional representation of the scene using a dense volumetric reconstruction framework. The obtained reconstruction serves as a proxy to densely fuse reflectance estimates and to store user-provided constraints in three-dimensional space. User constraints, in the form of constant shading and reflectance strokes, can be placed directly on the real-world geometry using an intuitive touch-based interaction metaphor, or using interactive mouse strokes. Fusing the decomposition results and constraints in three-dimensional space allows for robust propagation of this information to novel views by re-projection. We leverage this information to improve on the decomposition quality of existing intrinsic video decomposition techniques by further constraining the ill-posed decomposition problem. In addition to improved decomposition quality, we show a variety of live augmented reality applications such as recoloring of objects, relighting of scenes and editing of material appearance.
Original language | English |
---|---|
Journal | IEEE Transactions on Visualization and Computer Graphics |
DOIs | |
Publication status | Published - 11 Aug 2017 |
Event | IEEE International Symposium on Mixed and Augmented Reality - Nantes, France Duration: 9 Oct 2017 → 13 Oct 2017 Conference number: 16 https://ismar2017.sciencesconf.org/ |
Keywords
- intrinsic video decomposition
- reflectance fusion
- user-guided shading refinement
Fingerprint Dive into the research topics of 'Live User-Guided Intrinsic Video for Static Scenes'. Together they form a unique fingerprint.
Projects
- 1 Finished
-
Centre for the Analysis of Motion, Entertainment Research and Applications (CAMERA)
Cosker, D., Campbell, N., Fincham Haines, T., Hall, P., Kim, K. I., Lutteroth, C., O'Neill, E., Richardt, C. & Yang, Y.
Engineering and Physical Sciences Research Council
1/09/15 → 28/02/21
Project: Research council