Deferred Neural Rendering for View Extrapolation

Tobias Bertel; Yusuke Tomoto; Srinivas Rao; Rodrigo Ortiz-Cayon; Stefan Holzer; Christian Richardt

doi:10.1145/3415264.3425441

Deferred Neural Rendering for View Extrapolation

Tobias Bertel, Yusuke Tomoto, Srinivas Rao, Rodrigo Ortiz-Cayon, Stefan Holzer, Christian Richardt

Fyusion Inc.

Research output: Chapter or section in a book/report/conference proceeding › Chapter in a published conference proceeding

195 Downloads (Pure)

Abstract

Image-based rendering methods that support visually pleasing specular surface reflections require accurate surface geometry and a large number of input images. Recent advances in neural scene representations show excellent visual quality while requiring only imperfect mesh proxies or no surface-based proxies at all. While providing state-of-the-art visual quality, the inference time of learned models is usually too slow for interactive applications. While using a casually captured circular video sweep as input, we extend Deferred Neural Rendering to extrapolate smooth viewpoints around specular objects like a car.

This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 665992

Original language	English
Title of host publication	SA '20 Posters: SIGGRAPH Asia 2020 Posters
Publisher	Association for Computing Machinery
Pages	1-2
Number of pages	2
DOIs	https://doi.org/10.1145/3415264.3425441
Publication status	Published - 31 Dec 2020
Event	ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia - Online, Online, Korea, Republic of Duration: 4 Dec 2020 → 13 Dec 2020 Conference number: 13 https://sa2020.siggraph.org/en/

Conference

Conference	ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia
Abbreviated title	SIGGRAPH Asia
Country/Territory	Korea, Republic of
City	Online
Period	4/12/20 → 13/12/20
Internet address	https://sa2020.siggraph.org/en/

Keywords

novel-view synthesis
surface light field
extrapolation

Access to Document

10.1145/3415264.3425441

DNR4VE-BertelEtAl-SA2020-Poster-Abstract.PDF
© ACM, 2020. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in SA'20 Posters: SIGGRAPH Asia 2020 Posters. http://doi.acm.org/10.1145/3415264.3425441
Accepted author manuscript, 1.12 MB

1 Active
2 Finished

Centre for the Analysis of Motion, Entertainment Research and Applications (CAMERA) - 2.0
Campbell, N. (PI), Cosker, D. (PI), Bilzon, J. (CoI), Campbell, N. (CoI), Cazzola, D. (CoI), Colyer, S. (CoI), Cosker, D. (CoI), Lutteroth, C. (CoI), McGuigan, P. (CoI), O'Neill, E. (CoI), Petrini, K. (CoI), Proulx, M. (CoI) & Yang, Y. (CoI)
Engineering and Physical Sciences Research Council
1/11/20 → 31/10/25
Project: Research council
Fellowship - Towards Immersive 360° VR Video with Motion Parallax
Richardt, C. (PI)
Engineering and Physical Sciences Research Council
25/06/18 → 24/12/21
Project: Research council
Centre for the Analysis of Motion, Entertainment Research and Applications (CAMERA)
Cosker, D. (PI), Bilzon, J. (CoI), Campbell, N. (CoI), Cazzola, D. (CoI), Colyer, S. (CoI), Fincham Haines, T. (CoI), Hall, P. (CoI), Kim, K. I. (CoI), Lutteroth, C. (CoI), McGuigan, P. (CoI), O'Neill, E. (CoI), Richardt, C. (CoI), Salo, A. (CoI), Seminati, E. (CoI), Tabor, A. (CoI) & Yang, Y. (CoI)
Engineering and Physical Sciences Research Council
1/09/15 → 28/02/21
Project: Research council

Cite this

Bertel, T, Tomoto, Y, Rao, S, Ortiz-Cayon, R, Holzer, S & Richardt, C 2020, Deferred Neural Rendering for View Extrapolation. in SA '20 Posters: SIGGRAPH Asia 2020 Posters., 39, Association for Computing Machinery, pp. 1-2, ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia, Online, Korea, Republic of, 4/12/20. https://doi.org/10.1145/3415264.3425441

@inproceedings{81061d774fc64262a2cd2755595fc268,

title = "Deferred Neural Rendering for View Extrapolation",

abstract = "Image-based rendering methods that support visually pleasing specular surface reflections require accurate surface geometry and a large number of input images. Recent advances in neural scene representations show excellent visual quality while requiring only imperfect mesh proxies or no surface-based proxies at all. While providing state-of-the-art visual quality, the inference time of learned models is usually too slow for interactive applications. While using a casually captured circular video sweep as input, we extend Deferred Neural Rendering to extrapolate smooth viewpoints around specular objects like a car.This project has received funding from the European Union{\textquoteright}s Horizon 2020 research and innovation programme under the Marie Sk{\l}odowska-Curie grant agreement No 665992 ",

keywords = "novel-view synthesis, surface light field, extrapolation",

author = "Tobias Bertel and Yusuke Tomoto and Srinivas Rao and Rodrigo Ortiz-Cayon and Stefan Holzer and Christian Richardt",

year = "2020",

month = dec,

day = "31",

doi = "10.1145/3415264.3425441",

language = "English",

pages = "1--2",

booktitle = "SA '20 Posters: SIGGRAPH Asia 2020 Posters",

publisher = "Association for Computing Machinery",

address = "USA United States",

note = "ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia, SIGGRAPH Asia ; Conference date: 04-12-2020 Through 13-12-2020",

url = "https://sa2020.siggraph.org/en/",

}

TY - GEN

T1 - Deferred Neural Rendering for View Extrapolation

AU - Bertel, Tobias

AU - Tomoto, Yusuke

AU - Rao, Srinivas

AU - Ortiz-Cayon, Rodrigo

AU - Holzer, Stefan

AU - Richardt, Christian

N1 - Conference code: 13

PY - 2020/12/31

Y1 - 2020/12/31

N2 - Image-based rendering methods that support visually pleasing specular surface reflections require accurate surface geometry and a large number of input images. Recent advances in neural scene representations show excellent visual quality while requiring only imperfect mesh proxies or no surface-based proxies at all. While providing state-of-the-art visual quality, the inference time of learned models is usually too slow for interactive applications. While using a casually captured circular video sweep as input, we extend Deferred Neural Rendering to extrapolate smooth viewpoints around specular objects like a car.This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 665992

AB - Image-based rendering methods that support visually pleasing specular surface reflections require accurate surface geometry and a large number of input images. Recent advances in neural scene representations show excellent visual quality while requiring only imperfect mesh proxies or no surface-based proxies at all. While providing state-of-the-art visual quality, the inference time of learned models is usually too slow for interactive applications. While using a casually captured circular video sweep as input, we extend Deferred Neural Rendering to extrapolate smooth viewpoints around specular objects like a car.This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 665992

KW - novel-view synthesis

KW - surface light field

KW - extrapolation

U2 - 10.1145/3415264.3425441

DO - 10.1145/3415264.3425441

M3 - Chapter in a published conference proceeding

SP - 1

EP - 2

BT - SA '20 Posters: SIGGRAPH Asia 2020 Posters

PB - Association for Computing Machinery

T2 - ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia

Y2 - 4 December 2020 through 13 December 2020

ER -

Deferred Neural Rendering for View Extrapolation

Abstract

Conference

Keywords

Access to Document

Fingerprint

Projects

Centre for the Analysis of Motion, Entertainment Research and Applications (CAMERA) - 2.0

Fellowship - Towards Immersive 360° VR Video with Motion Parallax

Centre for the Analysis of Motion, Entertainment Research and Applications (CAMERA)

Cite this