Patch based synthesis for single depth image super-resolution

Oisin Mac Aodha, Neill D. F. Campbell, Arun Nair, Gabriel J. Brostow

Research output: Chapter in Book/Report/Conference proceedingChapter

102 Citations (Scopus)

Abstract

We present an algorithm to synthetically increase the resolution of a solitary depth image using only a generic database of local patches. Modern range sensors measure depths with non-Gaussian noise and at lower starting resolutions than typical visible-light cameras. While patch based approaches for upsampling intensity images continue to improve, this is the first exploration of patching for depth images. We match against the height field of each low resolution input depth patch, and search our database for a list of appropriate high resolution candidate patches. Selecting the right candidate at each location in the depth image is then posed as a Markov random field labeling problem. Our experiments also show how important further depth-specific processing, such as noise removal and correct patch normalization, dramatically improves our results. Perhaps surprisingly, even better results are achieved on a variety of real test scenes by providing our algorithm with only synthetic training depth data.
Original languageEnglish
Title of host publicationComputer Vision – ECCV 2012
Subtitle of host publicationProceedings of 12th European Conference on Computer Vision, Part III
EditorsA. Fitzgibbon, S. Lazebnik, P. Perona, Y. Sato, C. Schmid
Place of PublicationBerlin, Germany
PublisherSpringer
Pages71-84
Number of pages14
ISBN (Print)9783642337116
DOIs
Publication statusPublished - 2012
Event12th European Conference on Computer Vision,2012 - Florence, Italy
Duration: 7 Oct 201213 Oct 2012

Publication series

NameLecture Notes in Computer Science
Volume7574

Conference

Conference12th European Conference on Computer Vision,2012
CountryItaly
CityFlorence
Period7/10/1213/10/12

Fingerprint

Labeling
Cameras
Sensors
Processing
Experiments

Cite this

Mac Aodha, O., Campbell, N. D. F., Nair, A., & Brostow, G. J. (2012). Patch based synthesis for single depth image super-resolution. In A. Fitzgibbon, S. Lazebnik, P. Perona, Y. Sato, & C. Schmid (Eds.), Computer Vision – ECCV 2012: Proceedings of 12th European Conference on Computer Vision, Part III (pp. 71-84). (Lecture Notes in Computer Science; Vol. 7574). Berlin, Germany: Springer. https://doi.org/10.1007/978-3-642-33712-3_6

Patch based synthesis for single depth image super-resolution. / Mac Aodha, Oisin; Campbell, Neill D. F.; Nair, Arun; Brostow, Gabriel J.

Computer Vision – ECCV 2012: Proceedings of 12th European Conference on Computer Vision, Part III. ed. / A. Fitzgibbon; S. Lazebnik; P. Perona; Y. Sato; C. Schmid. Berlin, Germany : Springer, 2012. p. 71-84 (Lecture Notes in Computer Science; Vol. 7574).

Research output: Chapter in Book/Report/Conference proceedingChapter

Mac Aodha, O, Campbell, NDF, Nair, A & Brostow, GJ 2012, Patch based synthesis for single depth image super-resolution. in A Fitzgibbon, S Lazebnik, P Perona, Y Sato & C Schmid (eds), Computer Vision – ECCV 2012: Proceedings of 12th European Conference on Computer Vision, Part III. Lecture Notes in Computer Science, vol. 7574, Springer, Berlin, Germany, pp. 71-84, 12th European Conference on Computer Vision,2012, Florence, Italy, 7/10/12. https://doi.org/10.1007/978-3-642-33712-3_6
Mac Aodha O, Campbell NDF, Nair A, Brostow GJ. Patch based synthesis for single depth image super-resolution. In Fitzgibbon A, Lazebnik S, Perona P, Sato Y, Schmid C, editors, Computer Vision – ECCV 2012: Proceedings of 12th European Conference on Computer Vision, Part III. Berlin, Germany: Springer. 2012. p. 71-84. (Lecture Notes in Computer Science). https://doi.org/10.1007/978-3-642-33712-3_6
Mac Aodha, Oisin ; Campbell, Neill D. F. ; Nair, Arun ; Brostow, Gabriel J. / Patch based synthesis for single depth image super-resolution. Computer Vision – ECCV 2012: Proceedings of 12th European Conference on Computer Vision, Part III. editor / A. Fitzgibbon ; S. Lazebnik ; P. Perona ; Y. Sato ; C. Schmid. Berlin, Germany : Springer, 2012. pp. 71-84 (Lecture Notes in Computer Science).
@inbook{268741bdc2d149969ef628780e94cce4,
title = "Patch based synthesis for single depth image super-resolution",
abstract = "We present an algorithm to synthetically increase the resolution of a solitary depth image using only a generic database of local patches. Modern range sensors measure depths with non-Gaussian noise and at lower starting resolutions than typical visible-light cameras. While patch based approaches for upsampling intensity images continue to improve, this is the first exploration of patching for depth images. We match against the height field of each low resolution input depth patch, and search our database for a list of appropriate high resolution candidate patches. Selecting the right candidate at each location in the depth image is then posed as a Markov random field labeling problem. Our experiments also show how important further depth-specific processing, such as noise removal and correct patch normalization, dramatically improves our results. Perhaps surprisingly, even better results are achieved on a variety of real test scenes by providing our algorithm with only synthetic training depth data.",
author = "{Mac Aodha}, Oisin and Campbell, {Neill D. F.} and Arun Nair and Brostow, {Gabriel J.}",
year = "2012",
doi = "10.1007/978-3-642-33712-3_6",
language = "English",
isbn = "9783642337116",
series = "Lecture Notes in Computer Science",
publisher = "Springer",
pages = "71--84",
editor = "A. Fitzgibbon and S. Lazebnik and P. Perona and Y. Sato and C. Schmid",
booktitle = "Computer Vision – ECCV 2012",

}

TY - CHAP

T1 - Patch based synthesis for single depth image super-resolution

AU - Mac Aodha, Oisin

AU - Campbell, Neill D. F.

AU - Nair, Arun

AU - Brostow, Gabriel J.

PY - 2012

Y1 - 2012

N2 - We present an algorithm to synthetically increase the resolution of a solitary depth image using only a generic database of local patches. Modern range sensors measure depths with non-Gaussian noise and at lower starting resolutions than typical visible-light cameras. While patch based approaches for upsampling intensity images continue to improve, this is the first exploration of patching for depth images. We match against the height field of each low resolution input depth patch, and search our database for a list of appropriate high resolution candidate patches. Selecting the right candidate at each location in the depth image is then posed as a Markov random field labeling problem. Our experiments also show how important further depth-specific processing, such as noise removal and correct patch normalization, dramatically improves our results. Perhaps surprisingly, even better results are achieved on a variety of real test scenes by providing our algorithm with only synthetic training depth data.

AB - We present an algorithm to synthetically increase the resolution of a solitary depth image using only a generic database of local patches. Modern range sensors measure depths with non-Gaussian noise and at lower starting resolutions than typical visible-light cameras. While patch based approaches for upsampling intensity images continue to improve, this is the first exploration of patching for depth images. We match against the height field of each low resolution input depth patch, and search our database for a list of appropriate high resolution candidate patches. Selecting the right candidate at each location in the depth image is then posed as a Markov random field labeling problem. Our experiments also show how important further depth-specific processing, such as noise removal and correct patch normalization, dramatically improves our results. Perhaps surprisingly, even better results are achieved on a variety of real test scenes by providing our algorithm with only synthetic training depth data.

UR - http://www.scopus.com/inward/record.url?scp=84867882703&partnerID=8YFLogxK

UR - http://dx.doi.org/10.1007/978-3-642-33712-3_6

U2 - 10.1007/978-3-642-33712-3_6

DO - 10.1007/978-3-642-33712-3_6

M3 - Chapter

SN - 9783642337116

T3 - Lecture Notes in Computer Science

SP - 71

EP - 84

BT - Computer Vision – ECCV 2012

A2 - Fitzgibbon, A.

A2 - Lazebnik, S.

A2 - Perona, P.

A2 - Sato, Y.

A2 - Schmid, C.

PB - Springer

CY - Berlin, Germany

ER -