Improving Facial Animation using Non-linear Motion

: (Alternative Format Thesis)

Abstract

In this thesis, we look to improve the realism and robustness of facial animation through the use of data-driven facial retargeting methods that incorporate non-linear motion. The industrial standard for facial animation comprises of formation of a (largely linear) facial rig which is driven through sparse tracking of landmarks to produce smooth animation curves. However, such an approach is an approximation of true facial motion, where the resulting lack of organic transient detail might have a detrimental effect on the perception of facial animation. Consequently, there is often a heavy reliance on manual artistry to refine performance capture to compensate for a lack of non-linearity. To this end, the purpose of this work is to extend the traditional linear framework by capturing non-linearity and incorporating it into the facial animation pipeline through non-linear data-driven learning of non-linear motion − in doing so also alleviating manual overhead. This commences with presenting a semi-automated multi-view stereo based 4D Capture pipeline for the acquisition of non-linear performance capture, utilising off-the-shelf products available in industry. Next, a non-linear detail augmentation framework is devised through supervised learning of 4D Capture training data. This training scheme is subsequently applied to enhancing two key facial animation tasks with non-linear detail: for marker-based performance capture (in a process coined as Sparse-to-Dense); and personalisation of auto-generated character rigs (4D Rigging). Lastly, we explore the traditional pipeline’s dependence on artistic refinement by proposing a framework for user-guided animation and rig design through non-linear evolutionary algorithms.

Date of Award	14 Sept 2022
Original language	English
Awarding Institution	University of Bath
Supervisor	Darren Cosker (Supervisor) & Wenbin Li (Supervisor)