A Versatile Scene Model with Differentiable Visibility Applied to Generative Pose Estimation

Helge Rhodin, Nadia Robertini, Christian Richardt, H.-P. Seidel, Christian Theobalt

Research output: Chapter in Book/Report/Conference proceedingChapter

17 Citations (Scopus)
43 Downloads (Pure)

Abstract

Generative reconstruction methods compute the 3D configuration (such as pose and/or geometry) of a shape by optimizing the overlap of the projected 3D shape model with images. Proper handling of occlusions is a big challenge, since the visibility function that indicates if a surface point is seen from a camera can often not be formulated in closed form, and is in general discrete and non-differentiable at occlusion boundaries. We present a new scene representation that enables an analytically differentiable closed-form formulation of surface visibility. In contrast to previous methods, this yields smooth, analytically differentiable, and efficient to optimize pose similarity energies with rigorous occlusion handling, fewer local minima, and experimentally verified improved convergence of numerical optimization. The underlying idea is a new image formation model that represents opaque objects by a translucent medium with a smooth Gaussian density distribution which turns visibility into a smooth phenomenon. We demonstrate the advantages of our versatile scene model in several generative pose estimation problems, namely marker-less multi-object pose estimation, marker-less human motion capture with few cameras, and image-based 3D geometry estimation.
Original languageEnglish
Title of host publicationProceedings of the 2015 IEEE International Conference on Computer Vision, 11-18 December 2015, Santiago, Chile
Place of PublicationLos Alamitos, CA, USA
PublisherIEEE
Pages765-773
Number of pages9
ISBN (Print)978-1-4673-8390-5
DOIs
Publication statusPublished - 13 Dec 2015
EventIEEE International Conference on Computer Vision (ICCV 2015) - , UK United Kingdom
Duration: 13 Dec 2015 → …

Conference

ConferenceIEEE International Conference on Computer Vision (ICCV 2015)
CountryUK United Kingdom
Period13/12/15 → …

Fingerprint

Visibility
Cameras
Geometry
Image processing

Cite this

Rhodin, H., Robertini, N., Richardt, C., Seidel, H-P., & Theobalt, C. (2015). A Versatile Scene Model with Differentiable Visibility Applied to Generative Pose Estimation. In Proceedings of the 2015 IEEE International Conference on Computer Vision, 11-18 December 2015, Santiago, Chile (pp. 765-773). Los Alamitos, CA, USA: IEEE. https://doi.org/10.1109/ICCV.2015.94

A Versatile Scene Model with Differentiable Visibility Applied to Generative Pose Estimation. / Rhodin, Helge; Robertini, Nadia; Richardt, Christian; Seidel, H.-P.; Theobalt, Christian.

Proceedings of the 2015 IEEE International Conference on Computer Vision, 11-18 December 2015, Santiago, Chile. Los Alamitos, CA, USA : IEEE, 2015. p. 765-773.

Research output: Chapter in Book/Report/Conference proceedingChapter

Rhodin, H, Robertini, N, Richardt, C, Seidel, H-P & Theobalt, C 2015, A Versatile Scene Model with Differentiable Visibility Applied to Generative Pose Estimation. in Proceedings of the 2015 IEEE International Conference on Computer Vision, 11-18 December 2015, Santiago, Chile. IEEE, Los Alamitos, CA, USA, pp. 765-773, IEEE International Conference on Computer Vision (ICCV 2015), UK United Kingdom, 13/12/15. https://doi.org/10.1109/ICCV.2015.94
Rhodin H, Robertini N, Richardt C, Seidel H-P, Theobalt C. A Versatile Scene Model with Differentiable Visibility Applied to Generative Pose Estimation. In Proceedings of the 2015 IEEE International Conference on Computer Vision, 11-18 December 2015, Santiago, Chile. Los Alamitos, CA, USA: IEEE. 2015. p. 765-773 https://doi.org/10.1109/ICCV.2015.94
Rhodin, Helge ; Robertini, Nadia ; Richardt, Christian ; Seidel, H.-P. ; Theobalt, Christian. / A Versatile Scene Model with Differentiable Visibility Applied to Generative Pose Estimation. Proceedings of the 2015 IEEE International Conference on Computer Vision, 11-18 December 2015, Santiago, Chile. Los Alamitos, CA, USA : IEEE, 2015. pp. 765-773
@inbook{f1aa6775962f4203ad2698be2b81bd49,
title = "A Versatile Scene Model with Differentiable Visibility Applied to Generative Pose Estimation",
abstract = "Generative reconstruction methods compute the 3D configuration (such as pose and/or geometry) of a shape by optimizing the overlap of the projected 3D shape model with images. Proper handling of occlusions is a big challenge, since the visibility function that indicates if a surface point is seen from a camera can often not be formulated in closed form, and is in general discrete and non-differentiable at occlusion boundaries. We present a new scene representation that enables an analytically differentiable closed-form formulation of surface visibility. In contrast to previous methods, this yields smooth, analytically differentiable, and efficient to optimize pose similarity energies with rigorous occlusion handling, fewer local minima, and experimentally verified improved convergence of numerical optimization. The underlying idea is a new image formation model that represents opaque objects by a translucent medium with a smooth Gaussian density distribution which turns visibility into a smooth phenomenon. We demonstrate the advantages of our versatile scene model in several generative pose estimation problems, namely marker-less multi-object pose estimation, marker-less human motion capture with few cameras, and image-based 3D geometry estimation.",
author = "Helge Rhodin and Nadia Robertini and Christian Richardt and H.-P. Seidel and Christian Theobalt",
year = "2015",
month = "12",
day = "13",
doi = "10.1109/ICCV.2015.94",
language = "English",
isbn = "978-1-4673-8390-5",
pages = "765--773",
booktitle = "Proceedings of the 2015 IEEE International Conference on Computer Vision, 11-18 December 2015, Santiago, Chile",
publisher = "IEEE",
address = "USA United States",

}

TY - CHAP

T1 - A Versatile Scene Model with Differentiable Visibility Applied to Generative Pose Estimation

AU - Rhodin, Helge

AU - Robertini, Nadia

AU - Richardt, Christian

AU - Seidel, H.-P.

AU - Theobalt, Christian

PY - 2015/12/13

Y1 - 2015/12/13

N2 - Generative reconstruction methods compute the 3D configuration (such as pose and/or geometry) of a shape by optimizing the overlap of the projected 3D shape model with images. Proper handling of occlusions is a big challenge, since the visibility function that indicates if a surface point is seen from a camera can often not be formulated in closed form, and is in general discrete and non-differentiable at occlusion boundaries. We present a new scene representation that enables an analytically differentiable closed-form formulation of surface visibility. In contrast to previous methods, this yields smooth, analytically differentiable, and efficient to optimize pose similarity energies with rigorous occlusion handling, fewer local minima, and experimentally verified improved convergence of numerical optimization. The underlying idea is a new image formation model that represents opaque objects by a translucent medium with a smooth Gaussian density distribution which turns visibility into a smooth phenomenon. We demonstrate the advantages of our versatile scene model in several generative pose estimation problems, namely marker-less multi-object pose estimation, marker-less human motion capture with few cameras, and image-based 3D geometry estimation.

AB - Generative reconstruction methods compute the 3D configuration (such as pose and/or geometry) of a shape by optimizing the overlap of the projected 3D shape model with images. Proper handling of occlusions is a big challenge, since the visibility function that indicates if a surface point is seen from a camera can often not be formulated in closed form, and is in general discrete and non-differentiable at occlusion boundaries. We present a new scene representation that enables an analytically differentiable closed-form formulation of surface visibility. In contrast to previous methods, this yields smooth, analytically differentiable, and efficient to optimize pose similarity energies with rigorous occlusion handling, fewer local minima, and experimentally verified improved convergence of numerical optimization. The underlying idea is a new image formation model that represents opaque objects by a translucent medium with a smooth Gaussian density distribution which turns visibility into a smooth phenomenon. We demonstrate the advantages of our versatile scene model in several generative pose estimation problems, namely marker-less multi-object pose estimation, marker-less human motion capture with few cameras, and image-based 3D geometry estimation.

UR - https://gvv.mpi-inf.mpg.de/projects/DiffVis/

U2 - 10.1109/ICCV.2015.94

DO - 10.1109/ICCV.2015.94

M3 - Chapter

SN - 978-1-4673-8390-5

SP - 765

EP - 773

BT - Proceedings of the 2015 IEEE International Conference on Computer Vision, 11-18 December 2015, Santiago, Chile

PB - IEEE

CY - Los Alamitos, CA, USA

ER -