General Automatic Human Shape and Motion Capture Using Volumetric Contour Cues

Helge Rhodin, Nadia Robertini, Dan Casas, Christian Richardt, H.-P. Seidel, Christian Theobalt

Research output: Chapter or section in a book/report/conference proceedingChapter or section

57 Citations (SciVal)
266 Downloads (Pure)


Markerless motion capture algorithms require a 3D body with properly personalized skeleton dimension and/or body shape and appearance to successfully track a person. Unfortunately, many tracking methods consider model personalization a different problem and use manual or semi-automatic model initialization, which greatly reduces applicability. In this paper, we propose a fully automatic algorithm that jointly creates a rigged actor model commonly used for animation - skeleton, volumetric shape, appearance, and optionally a body surface - and estimates the actor's motion from multi-view video input only. The approach is rigorously designed to work on footage of general outdoor scenes recorded with very few cameras and without background subtraction. Our method uses a new image formation model with analytic visibility and analytically differentiable alignment energy. For reconstruction, 3D body shape is approximated as Gaussian density field. For pose and shape estimation, we minimize a new edge-based alignment energy inspired by volume raycasting in an absorbing medium. We further propose a new statistical human body model that represents the body surface, volumetric Gaussian density, as well as variability in skeleton shape. Given any multi-view sequence, our method jointly optimizes the pose and shape parameters of this model fully automatically in a spatiotemporal way.
Original languageEnglish
Title of host publicationComputer Vision - ECCV 2016
Subtitle of host publicationProceedings of the 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016
EditorsBastian Leibe, Jiri Matas, Nicu Sebe, Max Welling
Place of PublicationCham, Switzerland
Number of pages18
VolumePart V
ISBN (Electronic)978-3-319-46454-1
ISBN (Print)978-3-319-46453-4
Publication statusPublished - 16 Sept 2016
EventEuropean Conference on Computer Vision 2016 - Amsterdam, Netherlands
Duration: 8 Oct 201616 Oct 2016

Publication series

NameLecture Notes in Computer Science
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


ConferenceEuropean Conference on Computer Vision 2016
Abbreviated titleECCV
Internet address


Dive into the research topics of 'General Automatic Human Shape and Motion Capture Using Volumetric Contour Cues'. Together they form a unique fingerprint.

Cite this