RGBD-Dog: Predicting Canine Pose from RGBD Sensors

Sinead Kearney, Wenbin Li, Martin Parsons, Kwang In Kim, Darren Cosker

Research output: Chapter or section in a book/report/conference proceedingChapter in a published conference proceeding

36 Citations (SciVal)
180 Downloads (Pure)


The automatic extraction of animal 3D pose from images without markers is of interest in a range of scientific fields. Most work to date predicts animal pose from RGB images, based on 2D labelling of joint positions. However, due to the difficult nature of obtaining training data, no ground truth dataset of 3D animal motion is available to quantitatively evaluate these approaches. In addition, a lack of 3D animal pose data also makes it difficult to train 3D pose-prediction methods in a similar manner to the popular field of body-pose prediction. In our work, we focus on the problem of 3D canine pose estimation from RGBD images, recording a diverse range of dog breeds with several Microsoft Kinect v2s, simultaneously obtaining the 3D ground truth skeleton via a motion capture system. We generate a dataset of synthetic RGBD images from this data. A stacked hourglass network is trained to predict 3D joint locations, which is then constrained using prior models of shape and pose. We evaluate our model on both synthetic and real RGBD images and compare our results to previously published work fitting canine models to images. Finally, despite our training set consisting only of dog data, visual inspection implies that our network can produce good predictions for images of other quadrupeds – e.g. horses or cats – when their pose is similar to that contained in our training set.
Original languageEnglish
Title of host publication2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Number of pages10
ISBN (Electronic)9781728171685
ISBN (Print)978728171692
Publication statusPublished - 31 Dec 2020
EventIEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2020 - The Washington State Convention Center, Seattle, USA United States
Duration: 16 Jun 202018 Jun 2020

Publication series

NameIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
ISSN (Print)1063-6919
ISSN (Electronic)2575-7075


ConferenceIEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2020
Abbreviated titleCVPR 2020
Country/TerritoryUSA United States
Internet address


  • motion capture
  • Shape Models
  • pose estimation


Dive into the research topics of 'RGBD-Dog: Predicting Canine Pose from RGBD Sensors'. Together they form a unique fingerprint.

Cite this