Real-time Halfway Domain Reconstruction of Motion and Geometry

Lucas Thies, Michael Zollhöfer, Christian Richardt, Christian Theobalt, Günther Greiner

Research output: Contribution to conferencePaper

1 Citation (Scopus)
56 Downloads (Pure)

Abstract

We present a novel approach for real-time joint reconstruction of 3D scene motion and geometry from binocular stereo videos. Our approach is based on a novel variational halfway-domain scene flow formulation, which allows us to obtain highly accurate spatiotemporal reconstructions of shape and motion. We solve the underlying optimization problem at real-time frame rates using a novel data-parallel robust non-linear optimization strategy. Fast convergence and large displacement flows are achieved by employing a novel hierarchy that stores delta flows between hierarchy levels. High performance is obtained by the introduction of a coarser warp grid that decouples the number of unknowns from the input resolution of the images. We demonstrate our approach in a live setup that is based on two commodity webcams, as well as on publicly available video data. Our extensive experiments and evaluations show that our approach produces high-quality dense reconstructions of 3D geometry and scene flow at real-time frame rates, and compares favorably to the state of the art.
Original languageEnglish
Pages450-459
Number of pages10
DOIs
Publication statusPublished - 25 Oct 2016
EventInternational Conference on 3D Vision - Stanford University, Palo Alto, USA United States
Duration: 25 Oct 201628 Oct 2016
http://3dv.stanford.edu/

Conference

ConferenceInternational Conference on 3D Vision
Abbreviated title3DV
CountryUSA United States
CityPalo Alto
Period25/10/1628/10/16
Internet address

Fingerprint

Binoculars
Geometry
Experiments

Cite this

Thies, L., Zollhöfer, M., Richardt, C., Theobalt, C., & Greiner, G. (2016). Real-time Halfway Domain Reconstruction of Motion and Geometry. 450-459. Paper presented at International Conference on 3D Vision, Palo Alto, USA United States. https://doi.org/10.1109/3DV.2016.55

Real-time Halfway Domain Reconstruction of Motion and Geometry. / Thies, Lucas; Zollhöfer, Michael; Richardt, Christian; Theobalt, Christian; Greiner, Günther.

2016. 450-459 Paper presented at International Conference on 3D Vision, Palo Alto, USA United States.

Research output: Contribution to conferencePaper

Thies, L, Zollhöfer, M, Richardt, C, Theobalt, C & Greiner, G 2016, 'Real-time Halfway Domain Reconstruction of Motion and Geometry', Paper presented at International Conference on 3D Vision, Palo Alto, USA United States, 25/10/16 - 28/10/16 pp. 450-459. https://doi.org/10.1109/3DV.2016.55
Thies L, Zollhöfer M, Richardt C, Theobalt C, Greiner G. Real-time Halfway Domain Reconstruction of Motion and Geometry. 2016. Paper presented at International Conference on 3D Vision, Palo Alto, USA United States. https://doi.org/10.1109/3DV.2016.55
Thies, Lucas ; Zollhöfer, Michael ; Richardt, Christian ; Theobalt, Christian ; Greiner, Günther. / Real-time Halfway Domain Reconstruction of Motion and Geometry. Paper presented at International Conference on 3D Vision, Palo Alto, USA United States.10 p.
@conference{cd94baa852dd40ab9b7ba74bf76b16fe,
title = "Real-time Halfway Domain Reconstruction of Motion and Geometry",
abstract = "We present a novel approach for real-time joint reconstruction of 3D scene motion and geometry from binocular stereo videos. Our approach is based on a novel variational halfway-domain scene flow formulation, which allows us to obtain highly accurate spatiotemporal reconstructions of shape and motion. We solve the underlying optimization problem at real-time frame rates using a novel data-parallel robust non-linear optimization strategy. Fast convergence and large displacement flows are achieved by employing a novel hierarchy that stores delta flows between hierarchy levels. High performance is obtained by the introduction of a coarser warp grid that decouples the number of unknowns from the input resolution of the images. We demonstrate our approach in a live setup that is based on two commodity webcams, as well as on publicly available video data. Our extensive experiments and evaluations show that our approach produces high-quality dense reconstructions of 3D geometry and scene flow at real-time frame rates, and compares favorably to the state of the art.",
author = "Lucas Thies and Michael Zollh{\"o}fer and Christian Richardt and Christian Theobalt and G{\"u}nther Greiner",
year = "2016",
month = "10",
day = "25",
doi = "10.1109/3DV.2016.55",
language = "English",
pages = "450--459",
note = "International Conference on 3D Vision, 3DV ; Conference date: 25-10-2016 Through 28-10-2016",
url = "http://3dv.stanford.edu/",

}

TY - CONF

T1 - Real-time Halfway Domain Reconstruction of Motion and Geometry

AU - Thies, Lucas

AU - Zollhöfer, Michael

AU - Richardt, Christian

AU - Theobalt, Christian

AU - Greiner, Günther

PY - 2016/10/25

Y1 - 2016/10/25

N2 - We present a novel approach for real-time joint reconstruction of 3D scene motion and geometry from binocular stereo videos. Our approach is based on a novel variational halfway-domain scene flow formulation, which allows us to obtain highly accurate spatiotemporal reconstructions of shape and motion. We solve the underlying optimization problem at real-time frame rates using a novel data-parallel robust non-linear optimization strategy. Fast convergence and large displacement flows are achieved by employing a novel hierarchy that stores delta flows between hierarchy levels. High performance is obtained by the introduction of a coarser warp grid that decouples the number of unknowns from the input resolution of the images. We demonstrate our approach in a live setup that is based on two commodity webcams, as well as on publicly available video data. Our extensive experiments and evaluations show that our approach produces high-quality dense reconstructions of 3D geometry and scene flow at real-time frame rates, and compares favorably to the state of the art.

AB - We present a novel approach for real-time joint reconstruction of 3D scene motion and geometry from binocular stereo videos. Our approach is based on a novel variational halfway-domain scene flow formulation, which allows us to obtain highly accurate spatiotemporal reconstructions of shape and motion. We solve the underlying optimization problem at real-time frame rates using a novel data-parallel robust non-linear optimization strategy. Fast convergence and large displacement flows are achieved by employing a novel hierarchy that stores delta flows between hierarchy levels. High performance is obtained by the introduction of a coarser warp grid that decouples the number of unknowns from the input resolution of the images. We demonstrate our approach in a live setup that is based on two commodity webcams, as well as on publicly available video data. Our extensive experiments and evaluations show that our approach produces high-quality dense reconstructions of 3D geometry and scene flow at real-time frame rates, and compares favorably to the state of the art.

UR - http://richardt.name/publications/halfway-domain-scene-flow/

U2 - 10.1109/3DV.2016.55

DO - 10.1109/3DV.2016.55

M3 - Paper

SP - 450

EP - 459

ER -