InverseFaceNet: Deep Monocular Inverse Face Rendering

Hyeongwoo Kim, Michael Zollhöfer, Ayush Tewari, Justus Thies, Christian Richardt, Christian Theobalt

Research output: Chapter or section in a book/report/conference proceedingChapter in a published conference proceeding

47 Citations (SciVal)
109 Downloads (Pure)


We introduce InverseFaceNet, a deep convolutional inverse rendering framework for faces that jointly estimates facial pose, shape, expression, reflectance and illumination from a single input image. By estimating all parameters from just a single image, advanced editing possibilities on a single face image, such as appearance editing and relighting, become feasible in real time. Most previous learning-based face reconstruction approaches do not jointly recover all dimensions, or are severely limited in terms of visual quality. In contrast, we propose to recover high-quality facial pose, shape, expression, reflectance and illumination using a deep neural network that is trained using a large, synthetically created training corpus. Our approach builds on a novel loss function that measures model-space similarity directly in parameter space and significantly improves reconstruction accuracy. We further propose a self-supervised bootstrapping process in the network training loop, which iteratively updates the synthetic training corpus to better reflect the distribution of real-world imagery. We demonstrate that this strategy outperforms completely synthetically trained networks. Finally, we show high-quality reconstructions and compare our approach to several state-of-the-art approaches.
Original languageEnglish
Title of host publication2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
Number of pages10
ISBN (Electronic)978-1-5386-6421-6
Publication statusPublished - 18 Jun 2018
EventInternational Conference on Computer Vision and Pattern Recognition - Salt Lake City, USA United States
Duration: 18 Jun 201822 Jun 2018

Publication series

NameProceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Online)
ISSN (Print)2575-7075


ConferenceInternational Conference on Computer Vision and Pattern Recognition
Abbreviated titleCVPR
Country/TerritoryUSA United States
CitySalt Lake City
Internet address


Dive into the research topics of 'InverseFaceNet: Deep Monocular Inverse Face Rendering'. Together they form a unique fingerprint.

Cite this