Casual Real-World VR using Light Fields

Yusuke Tomoto; Srinivas Rao; Tobias Bertel; Krunal Chande; Christian Richardt; Stefan Holzer; Rodrigo Ortiz-Cayon

doi:10.1145/3415264.3425452

Casual Real-World VR using Light Fields

Yusuke Tomoto, Srinivas Rao, Tobias Bertel, Krunal Chande, Christian Richardt, Stefan Holzer, Rodrigo Ortiz-Cayon

Fyusion Inc.

Research output: Chapter or section in a book/report/conference proceeding › Chapter in a published conference proceeding

1 Citation (SciVal)

157 Downloads (Pure)

Abstract

Virtual reality (VR) would benefit from more end-to-end systems centered around a casual capturing procedure, high-quality visual results, and representations that are viewable on multiple platforms. We present an end-to-end system that is designed for casual creation of real-world VR content, using a smartphone. We use an AR app to capture a linear light field of a real-world object by recording a video sweep around the object. We predict multiplane images for a subset of input viewpoints, from which we extract high-quality textured geometry that are used for real-time image-based rendering suitable for VR. The round-trip time of our system, from guided capture to interactive display, is typically 1–2 minutes per scene.

This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 665992

Original language	English
Title of host publication	SA'20 Posters: SIGGRAPH Asia 2020 Posters
Publisher	Association for Computing Machinery
Pages	1-2
Number of pages	2
DOIs	https://doi.org/10.1145/3415264.3425452
Publication status	Published - 31 Dec 2020
Event	ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia - Online, Online, Korea, Republic of Duration: 4 Dec 2020 → 13 Dec 2020 Conference number: 13 https://sa2020.siggraph.org/en/

Conference

Conference	ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia
Abbreviated title	SIGGRAPH Asia
Country/Territory	Korea, Republic of
City	Online
Period	4/12/20 → 13/12/20
Internet address	https://sa2020.siggraph.org/en/

Keywords

virtual reality
VR photography
view synthesis
multiplane images

Access to Document

10.1145/3415264.3425452

LLFF4VR-TomotoEtAl-SA2020-Poster-Abstract.PDF
© ACM, 2020. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in SA -20 Posters: SIGGRAPH Asia 2020 Posters http://doi.acm.org/10.1145/3415264.3425452
Accepted author manuscript, 9.14 MB

1 Finished
1 Active

Centre for the Analysis of Motion, Entertainment Research and Applications (CAMERA) - 2.0
Campbell, N. (PI), Cosker, D. (PI), Bilzon, J. (CoI), Campbell, N. (CoI), Cazzola, D. (CoI), Colyer, S. (CoI), Cosker, D. (CoI), Lutteroth, C. (CoI), McGuigan, P. (CoI), O'Neill, E. (CoI), Petrini, K. (CoI), Proulx, M. (CoI) & Yang, Y. (CoI)
Engineering and Physical Sciences Research Council
1/11/20 → 31/10/25
Project: Research council
Centre for the Analysis of Motion, Entertainment Research and Applications (CAMERA)
Cosker, D. (PI), Bilzon, J. (CoI), Campbell, N. (CoI), Cazzola, D. (CoI), Colyer, S. (CoI), Fincham Haines, T. (CoI), Hall, P. (CoI), Kim, K. I. (CoI), Lutteroth, C. (CoI), McGuigan, P. (CoI), O'Neill, E. (CoI), Richardt, C. (CoI), Salo, A. (CoI), Seminati, E. (CoI), Tabor, A. (CoI) & Yang, Y. (CoI)
Engineering and Physical Sciences Research Council
1/09/15 → 28/02/21
Project: Research council

Cite this

Tomoto, Y, Rao, S, Bertel, T, Chande, K, Richardt, C, Holzer, S & Ortiz-Cayon, R 2020, Casual Real-World VR using Light Fields. in SA'20 Posters: SIGGRAPH Asia 2020 Posters., 38, Association for Computing Machinery, pp. 1-2, ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia, Online, Korea, Republic of, 4/12/20. https://doi.org/10.1145/3415264.3425452

@inproceedings{70e6577a6fb64bc49db142819089fae5,

title = "Casual Real-World VR using Light Fields",

abstract = "Virtual reality (VR) would benefit from more end-to-end systems centered around a casual capturing procedure, high-quality visual results, and representations that are viewable on multiple platforms. We present an end-to-end system that is designed for casual creation of real-world VR content, using a smartphone. We use an AR app to capture a linear light field of a real-world object by recording a video sweep around the object. We predict multiplane images for a subset of input viewpoints, from which we extract high-quality textured geometry that are used for real-time image-based rendering suitable for VR. The round-trip time of our system, from guided capture to interactive display, is typically 1–2 minutes per scene.This project has received funding from the European Union{\textquoteright}s Horizon 2020 research and innovation programme under the Marie Sk{\l}odowska-Curie grant agreement No 665992 ",

keywords = "virtual reality, VR photography, view synthesis, multiplane images",

author = "Yusuke Tomoto and Srinivas Rao and Tobias Bertel and Krunal Chande and Christian Richardt and Stefan Holzer and Rodrigo Ortiz-Cayon",

year = "2020",

month = dec,

day = "31",

doi = "10.1145/3415264.3425452",

language = "English",

pages = "1--2",

booktitle = "SA'20 Posters: SIGGRAPH Asia 2020 Posters",

publisher = "Association for Computing Machinery",

address = "USA United States",

note = "ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia, SIGGRAPH Asia ; Conference date: 04-12-2020 Through 13-12-2020",

url = "https://sa2020.siggraph.org/en/",

}

TY - GEN

T1 - Casual Real-World VR using Light Fields

AU - Tomoto, Yusuke

AU - Rao, Srinivas

AU - Bertel, Tobias

AU - Chande, Krunal

AU - Richardt, Christian

AU - Holzer, Stefan

AU - Ortiz-Cayon, Rodrigo

N1 - Conference code: 13

PY - 2020/12/31

Y1 - 2020/12/31

N2 - Virtual reality (VR) would benefit from more end-to-end systems centered around a casual capturing procedure, high-quality visual results, and representations that are viewable on multiple platforms. We present an end-to-end system that is designed for casual creation of real-world VR content, using a smartphone. We use an AR app to capture a linear light field of a real-world object by recording a video sweep around the object. We predict multiplane images for a subset of input viewpoints, from which we extract high-quality textured geometry that are used for real-time image-based rendering suitable for VR. The round-trip time of our system, from guided capture to interactive display, is typically 1–2 minutes per scene.This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 665992

AB - Virtual reality (VR) would benefit from more end-to-end systems centered around a casual capturing procedure, high-quality visual results, and representations that are viewable on multiple platforms. We present an end-to-end system that is designed for casual creation of real-world VR content, using a smartphone. We use an AR app to capture a linear light field of a real-world object by recording a video sweep around the object. We predict multiplane images for a subset of input viewpoints, from which we extract high-quality textured geometry that are used for real-time image-based rendering suitable for VR. The round-trip time of our system, from guided capture to interactive display, is typically 1–2 minutes per scene.This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 665992

KW - virtual reality

KW - VR photography

KW - view synthesis

KW - multiplane images

U2 - 10.1145/3415264.3425452

DO - 10.1145/3415264.3425452

M3 - Chapter in a published conference proceeding

SP - 1

EP - 2

BT - SA'20 Posters: SIGGRAPH Asia 2020 Posters

PB - Association for Computing Machinery

T2 - ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia

Y2 - 4 December 2020 through 13 December 2020

ER -

Casual Real-World VR using Light Fields

Abstract

Conference

Keywords

Access to Document

Fingerprint

Projects

Centre for the Analysis of Motion, Entertainment Research and Applications (CAMERA) - 2.0

Centre for the Analysis of Motion, Entertainment Research and Applications (CAMERA)

Cite this