Projects per year
We introduce InverseFaceNet, a deep convolutional inverse rendering framework for faces that jointly estimates facial pose, shape, expression, reflectance and illumination from a single input image. By estimating all parameters from just a single image, advanced editing possibilities on a single face image, such as appearance editing and relighting, become feasible in real time. Most previous learning-based face reconstruction approaches do not jointly recover all dimensions, or are severely limited in terms of visual quality. In contrast, we propose to recover high-quality facial pose, shape, expression, reflectance and illumination using a deep neural network that is trained using a large, synthetically created training corpus. Our approach builds on a novel loss function that measures model-space similarity directly in parameter space and significantly improves reconstruction accuracy. We further propose a self-supervised bootstrapping process in the network training loop, which iteratively updates the synthetic training corpus to better reflect the distribution of real-world imagery. We demonstrate that this strategy outperforms completely synthetically trained networks. Finally, we show high-quality reconstructions and compare our approach to several state-of-the-art approaches.
|Title of host publication||2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition|
|Number of pages||10|
|Publication status||Published - 18 Jun 2018|
|Event||International Conference on Computer Vision and Pattern Recognition - Salt Lake City, USA United States|
Duration: 18 Jun 2018 → 22 Jun 2018
|Name||Proceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Online)|
|Conference||International Conference on Computer Vision and Pattern Recognition|
|Country||USA United States|
|City||Salt Lake City|
|Period||18/06/18 → 22/06/18|
FingerprintDive into the research topics of 'InverseFaceNet: Deep Monocular Inverse Face Rendering'. Together they form a unique fingerprint.
- 1 Finished
1/09/15 → 28/02/21
Project: Research council