Beauty3DFaceNet: Deep geometry and texture fusion for 3D facial attractiveness prediction

Qinjie Xiao, You Wu, Dinghong Wang, Yong Liang Yang, Xiaogang Jin

Research output: Contribution to journalArticlepeer-review

Abstract

We present Beauty3DFaceNet, the first deep convolutional neural network to predict attractiveness on 3D faces with both geometry and texture information. The proposed network can learn discriminative and complementary 2D and 3D facial features, allowing accurate attractiveness prediction for 3D faces. The main component of our network is a fusion module that fuses geometric features and texture features. We further employ a novel sampling strategy for our network based on a prior of facial landmarks, which improves the performance of learning aesthetic features from a face point cloud. Comparing to previous work, our approach takes full advantage of 3D geometry and 2D texture and does not rely on handcrafted features based on highly accurate facial characteristics such as feature points. To facilitate 3D facial attractiveness research, we also construct the first 3D face dataset ShadowFace3D, which contains 6,000 high-quality 3D faces with attractiveness labeled by human annotators. Extensive quantitative and qualitative evaluations show that Beauty3DFaceNet achieves a significant correlation with the average human ratings. This validates that a deep learning network can effectively learn and predict 3D facial attractiveness.

Original languageEnglish
Pages (from-to)11-18
JournalComputers and Graphics (Pergamon)
Volume98
Early online date23 Apr 2021
DOIs
Publication statusE-pub ahead of print - 23 Apr 2021

Keywords

  • 3D facial attractiveness prediction
  • Datasets
  • Deep learning
  • Feature fusion

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Engineering(all)
  • Human-Computer Interaction
  • Computer Vision and Pattern Recognition
  • Computer Graphics and Computer-Aided Design

Cite this