Facial Capture and Animation in Visual Effects

In recent years, there has been increasing interest in facial animation re- search from both academia and the entertainment industry. Visual effects and video game companies both want to deliver new audience experiences – whether that is a hyper-realistic human character [Duncan 09] or a fantasy creature driven by a human performer [Duncan 10]. Having more efficient ways of delivering high quality animation, as well as increasing the visual realism of performances, has motivated a surge of innovative academic and industrial developments.
Central to many of these developments are key technical advances in computer vision and graphics. Of particular note are advances in multi- view stereo reconstruction, facial tracking and motion capture, dense non- rigid registration of meshes, measurement of skin rendering attributes (e.g. BRDFs for skin and skin subsurface scattering models) and sensing tech- nology.
This chapter builds on concepts already described earlier in this book – such as 3D capture, rigging, and non-rigid registration – and takes a more practical look at how they might typically be applied in visual effects. First, methods and applications for facial static capture and rendering are considered, before dynamic capture is addressed. Finally, a case study is examined called The Gathering involving the creation of an animated face – from animation to final composite.