Reading Between The Dots: Combining 3D Markers And FACS Classification For High-Quality Blendshape Facial Animation

S. Ravikumar, C. Davidson, D. Kit, N. D. F. Campbell, L. Benedetti, Darren Cosker

Research output: Chapter or section in a book/report/conference proceedingChapter in a published conference proceeding

6 Citations (SciVal)


Marker based performance capture is one of the most widely used approaches for facial tracking owing to its robustness. In practice, marker based systems do not capture the performance with complete fidelity and often require subsequent manual adjustment to incorporate missing visual details. This problem persists even when using larger number of markers. Tracking a large number of markers can also quickly become intractable due to issues such as occlusion, swapping and merging of markers. We present a new approach for fitting blendshape models to motion-capture data that improves quality, by exploiting information from sparse make-up patches in the video between the markers, while using fewer markers. Our method uses a classification based approach that detects FACS Action Units and their intensities to assist the solver in predicting optimal blendshape weights while taking perceptual quality into consideration. Our classifier is independent of the performer; once trained, it can be applied to multiple performers. Given performances captured using a Head Mounted Camera (HMC), which provides 3D facial marker based tracking and corresponding video, we fit accurate, production quality blendshape models to this data resulting in high-quality animations.
Original languageEnglish
Title of host publication Proceedings of Graphics Interface 2016: Victoria, British Columbia, Canada, 1- 3 June 2016
EditorsK. Moffatt, T. Popa
Place of PublicationOntario, Canada
PublisherCanadian Human-Computer Communications Society
Number of pages9
ISBN (Print)9780994786814
Publication statusPublished - 3 Jun 2016


Dive into the research topics of 'Reading Between The Dots: Combining 3D Markers And FACS Classification For High-Quality Blendshape Facial Animation'. Together they form a unique fingerprint.

Cite this