Coarse-to-Fine: Facial Structure Editing of Portrait Images via Latent Space Classifications

Yiqian Wu, Yongliang Yang, Qinjie Xiao, Xiaogang Jin

Research output: Contribution to journalArticlepeer-review

6 Citations (SciVal)
265 Downloads (Pure)

Abstract

Facial structure editing of portrait images is challenging given the facial variety, the lack of ground-truth, the necessity of jointly adjusting color and shape, and the requirement of no visual artifacts. In this paper, we investigate how to perform chin editing as a case study of editing facial structures. We present a novel method that can automatically remove the double chin effect in portrait images. Our core idea is to train a fine classification boundary in the latent space of the portrait images. This can be used to edit the chin appearance by manipulating the latent code of the input portrait image while preserving the original portrait features. To achieve such a fine separation boundary, we employ a carefully designed training stage based on latent codes of paired synthetic images with and without a double chin. In the testing stage, our method can automatically handle portrait images with only a refinement to subtle misalignment before and after double chin editing. Our model enables alteration to the neck region of the input portrait image while keeping other regions unchanged, and guarantees the rationality of neck structure and the consistency of facial characteristics. To the best of our knowledge, this presents the first effort towards an effective application for editing double chins. We validate the efficacy and efficiency of our approach through extensive experiments and user studies.
Original languageEnglish
Article number46
Pages (from-to)1-13
JournalACM Transactions on Graphics
Volume40
Issue number4
Early online date19 Jul 2021
DOIs
Publication statusPublished - 31 Aug 2021

Bibliographical note

Funding Information:
Xiaogang Jin was supported by the National Key R&D Program of China (Grant No. 2017YFB1002600), the National Natural Science Foundation of China (Grant Nos. 61972344, 61732015). Yong-Liang Yang was supported by RCUK grant CAMERA (EP/M023281/1, EP/T014865/1), and a gift from Adobe. We thank the anonymous reviewers for their detailed comments, NVIDIA Research for making the Flickr-Faces-HQ (FFHQ) dataset, and all Flickr users for sharing their portrait photos under the Creative Commons License.

Publisher Copyright:
© 2021 ACM.

Keywords

  • StyleGAN
  • double chin
  • face editing
  • latent code manipulation

ASJC Scopus subject areas

  • Computer Graphics and Computer-Aided Design

Fingerprint

Dive into the research topics of 'Coarse-to-Fine: Facial Structure Editing of Portrait Images via Latent Space Classifications'. Together they form a unique fingerprint.

Cite this