EgoCap: egocentric marker-less motion capture with two fisheye cameras

Helge Rhodin, Christian Richardt, Dan Casas, Eldar Insafutinov, Mohammad Shafiei, H.-P. Seidel, Bernt Schiele, Christian Theobalt

Research output: Contribution to journalArticle

34 Citations (Scopus)
108 Downloads (Pure)

Abstract

Marker-based and marker-less optical skeletal motion-capture methods use an outside-in arrangement of cameras placed around a scene, with viewpoints converging on the center. They often create discomfort by possibly needed marker suits, and their recording volume is severely restricted and often constrained to indoor scenes with controlled backgrounds. Alternative suit-based systems use several inertial measurement units or an exoskeleton to capture motion. This makes capturing independent of a confined volume, but requires substantial, often constraining, and hard to set up body instrumentation. We therefore propose a new method for real-time, marker-less and egocentric motion capture which estimates the full-body skeleton pose from a lightweight stereo pair of fisheye cameras that are attached to a helmet or virtual reality headset. It combines the strength of a new generative pose estimation framework for fisheye views with a ConvNet-based body-part detector trained on a large new dataset. Our inside-in method captures full-body motion in general indoor and outdoor scenes, and also crowded scenes with many people in close vicinity. The setup time and effort is minimal and the captured user can freely move around, which enables reconstruction of larger-scale activities and is particularly useful in virtual reality to freely roam and interact, while seeing the fully motion-captured virtual body.
Original languageEnglish
Article number162
Pages (from-to)1-11
Number of pages11
JournalACM Transactions on Graphics
Volume35
Issue number6
Early online date30 Nov 2016
DOIs
Publication statusPublished - 30 Nov 2016
EventSIGGRAPH Asia 2016: The 9th ACM SIGGRAPH conference and exhibition on computer graphics and interactive techniques in Asia - Macao, Macao
Duration: 5 Dec 20168 Dec 2016
https://sa2016.siggraph.org/en/

Keywords

  • motion capture
  • first-person vision
  • markerless
  • optical
  • inside-in
  • crowded scenes
  • large-scale

Fingerprint Dive into the research topics of 'EgoCap: egocentric marker-less motion capture with two fisheye cameras'. Together they form a unique fingerprint.

  • Projects

    Cite this

    Rhodin, H., Richardt, C., Casas, D., Insafutinov, E., Shafiei, M., Seidel, H-P., Schiele, B., & Theobalt, C. (2016). EgoCap: egocentric marker-less motion capture with two fisheye cameras. ACM Transactions on Graphics, 35(6), 1-11. [162]. https://doi.org/10.1145/2980179.2980235