Identity-consistent transfer learning of portraits for digital apparel sample display

Luyuan Wang, Yiqian Wu, Yongliang Yang, Chen Liu, Xiaogang Jin

Research output: Contribution to journalArticlepeer-review

Abstract

The rapid development of the online apparel shopping industry demands innovative solutions for high-quality digital apparel sample displays with virtual avatars. However, developing such displays is prohibitively expensive and prone to the well-known “uncanny valley” effect, where a nearly human-looking artifact arouses eeriness and repulsiveness, thus affecting the user experience. To effectively mitigate the “uncanny valley” effect and improve the overall authenticity of digital apparel sample displays, we present a novel photo-realistic portrait generation framework. Our key idea is to employ transfer learning to learn an identity-consistent mapping from the latent space of rendered portraits to that of real portraits. During the inference stage, the input portrait of an avatar can be directly transferred to a realistic portrait by changing its appearance style while maintaining the facial identity. To this end, we collect a new dataset, Daz-Rendered-Faces-HQ (DRFHQ), specifically designed for rendering-style portraits. We leverage this dataset to fine-tune the StyleGAN2-FFHQ generator, using our carefully crafted framework, which helps to preserve the geometric and color features relevant to facial identity. We evaluate our framework using portraits with diverse gender, age, and race variations. Qualitative and quantitative evaluations, along with ablation studies, highlight our method’s advantages over state-of-the-art approaches
Original languageEnglish
Article numbere2278
JournalComputer Animation and Virtual Worlds
Volume35
Issue number3
Early online date10 Jun 2024
DOIs
Publication statusPublished - 30 Jun 2024

Keywords

  • StyleGAN
  • digital apparel
  • portrait authenticity
  • transfer learning
  • uncanny valley effect

ASJC Scopus subject areas

  • Software
  • Computer Graphics and Computer-Aided Design

Cite this