Addressing Animal Body Analysis through Synthetic Data and Labels

  • Jake Deane

Student thesis: Doctoral ThesisPhD


Animal body analysis is an under-researched area compared to human body analysis in the computer vision community mostly as a result of a lack of sufficient data. Tasks such as pose estimation and part-segmentation require large amounts of data to train models but this data is expensive and time consuming to create particularly if we want dense labels (a large number of labels for a task).

In this thesis we present a 3D parametric canine body model, RGBT-Dog, to alleviate this problem allowing us to produce synthetic dense multi-task labels for canine bodies. This 3D canine parametric model improves upon existing parametric animal models with respect to canines, providing a more complete canine skeleton which can be posed via the largest set of canine motion capture data. This motion capture data also allowed us to produce the first variational pose prior for canines and indeed animals is general. We also introduce a novel texture Principal Component Analysis space and texture parameter that is used to generate a texture for the 3D mesh produced by our parametric model. With this parametric model we can generate synthetic canine images with paired, dense multi-task data labels. In addition we can also fit RGBT-Dog to real images creating paired synthetic data labels for multiple tasks; producing the first large scale, multi-domain, multi-task dataset for canine body analysis comprising of detailed synthetic labels on both real images and fully synthetic images in a range of realistic poses. This data includes dense 2D and 3D keypoint labels and dense part-segmentation labels.

We show in a series of experiments how we can use this data to tackle body analysis tasks for canines using dense multi-task labels. We demonstrate the effectiveness of our labels for training computer vision models on tasks such as parts-based segmentation and dense pose estimation and show such models can generalise to other animal species, where no previous examples have been used in training. We also highlight the advances made by our 3D parametric model including the largest set of canine motion capture data and a trained canine pose prior.
Date of Award28 Jun 2023
Original languageEnglish
Awarding Institution
  • University of Bath
SupervisorDarren Cosker (Supervisor), Wenbin Li (Supervisor) & Kwang In Kim (Supervisor)

Cite this