Reading Between The Dots: Combining 3D Markers And Facs Classification For High-Quality Blendshape Facial Animation

S. Ravikumar, C. Davidson, D. Kit, N. D. F. Campbell, L. Benedetti, Darren Cosker

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

Marker based performance capture is one of the most widely used approaches for facial tracking owing to its robustness. In practice, marker based systems do not capture the performance with complete fidelity and often require subsequent manual adjustment to incorporate missing visual details. This problem persists even when using larger number of markers. Tracking a large number of markers can also quickly become intractable due to issues such as occlusion, swapping and merging of markers. We present a new approach for fitting blendshape models to motion-capture data that improves quality, by exploiting information from sparse make-up patches in the video between the markers, while using fewer markers. Our method uses a classification based approach that detects FACS Action Units and their intensities to assist the solver in predicting optimal blendshape weights while taking perceptual quality into consideration. Our classifier is independent of the performer; once trained, it can be applied to multiple performers. Given performances captured using a Head Mounted Camera (HMC), which provides 3D facial marker based tracking and corresponding video, we fit accurate, production quality blendshape models to this data resulting in high-quality animations.
Original languageEnglish
Title of host publicationGI '16 Proceedings of the 42nd Graphics Interface Conference
EditorsK. Moffatt, T. Popa
Place of PublicationOntario, Canada
PublisherCanadian Human-Computer Communications Society
Pages143-151
Number of pages9
ISBN (Print)9780994786814
DOIs
Publication statusPublished - 3 Jun 2016

Cite this

Ravikumar, S., Davidson, C., Kit, D., Campbell, N. D. F., Benedetti, L., & Cosker, D. (2016). Reading Between The Dots: Combining 3D Markers And Facs Classification For High-Quality Blendshape Facial Animation. In K. Moffatt, & T. Popa (Eds.), GI '16 Proceedings of the 42nd Graphics Interface Conference (pp. 143-151). Ontario, Canada: Canadian Human-Computer Communications Society. https://doi.org/10.20380/GI2016.18