4D Model Flow: Precomputed Appearance Alignment for Real-time 4D Video Interpolation

Dan Casas, Christian Richardt, John Collomosse, Christian Theobalt, Adrian Hilton

Research output: Contribution to journalArticle

4 Citations (Scopus)

Abstract

We introduce the concept of 4D model flow for the precomputed alignment of dynamic surface appearance across 4D video sequences of different motions reconstructed from multi-view video. Precomputed 4D model flow allows the efficient parametrization of surface appearance from the captured videos, which enables efficient real-time rendering of interpolated 4D video sequences whilst accurately reproducing visual dynamics, even when using a coarse underlying geometry. We estimate the 4D model flow using an image-based approach that is guided by available geometry proxies. We propose a novel representation in surface texture space for efficient storage and online parametric interpolation of dynamic appearance. Our 4D model flow overcomes previous requirements for computationally expensive online optical flow computation for data-driven alignment of dynamic surface appearance by precomputing the appearance alignment. This leads to an efficient rendering technique that enables the online interpolation between 4D videos in real time, from arbitrary viewpoints and with visual quality comparable to the state of the art.
Original languageEnglish
Pages (from-to)173-182
Number of pages10
JournalComputer Graphics Forum
Volume34
Issue number7
DOIs
Publication statusPublished - Oct 2015

Fingerprint

Interpolation
Geometry
Optical flows
Textures

Cite this

4D Model Flow: Precomputed Appearance Alignment for Real-time 4D Video Interpolation. / Casas, Dan; Richardt, Christian; Collomosse, John; Theobalt, Christian; Hilton, Adrian.

In: Computer Graphics Forum, Vol. 34, No. 7, 10.2015, p. 173-182.

Research output: Contribution to journalArticle

Casas, Dan ; Richardt, Christian ; Collomosse, John ; Theobalt, Christian ; Hilton, Adrian. / 4D Model Flow: Precomputed Appearance Alignment for Real-time 4D Video Interpolation. In: Computer Graphics Forum. 2015 ; Vol. 34, No. 7. pp. 173-182.
@article{9f6cec057b22460cb8472d9deb6803b8,
title = "4D Model Flow: Precomputed Appearance Alignment for Real-time 4D Video Interpolation",
abstract = "We introduce the concept of 4D model flow for the precomputed alignment of dynamic surface appearance across 4D video sequences of different motions reconstructed from multi-view video. Precomputed 4D model flow allows the efficient parametrization of surface appearance from the captured videos, which enables efficient real-time rendering of interpolated 4D video sequences whilst accurately reproducing visual dynamics, even when using a coarse underlying geometry. We estimate the 4D model flow using an image-based approach that is guided by available geometry proxies. We propose a novel representation in surface texture space for efficient storage and online parametric interpolation of dynamic appearance. Our 4D model flow overcomes previous requirements for computationally expensive online optical flow computation for data-driven alignment of dynamic surface appearance by precomputing the appearance alignment. This leads to an efficient rendering technique that enables the online interpolation between 4D videos in real time, from arbitrary viewpoints and with visual quality comparable to the state of the art.",
author = "Dan Casas and Christian Richardt and John Collomosse and Christian Theobalt and Adrian Hilton",
year = "2015",
month = "10",
doi = "10.1111/cgf.12756",
language = "English",
volume = "34",
pages = "173--182",
journal = "Computer Graphics Forum",
issn = "0167-7055",
publisher = "Wiley-Blackwell",
number = "7",

}

TY - JOUR

T1 - 4D Model Flow: Precomputed Appearance Alignment for Real-time 4D Video Interpolation

AU - Casas, Dan

AU - Richardt, Christian

AU - Collomosse, John

AU - Theobalt, Christian

AU - Hilton, Adrian

PY - 2015/10

Y1 - 2015/10

N2 - We introduce the concept of 4D model flow for the precomputed alignment of dynamic surface appearance across 4D video sequences of different motions reconstructed from multi-view video. Precomputed 4D model flow allows the efficient parametrization of surface appearance from the captured videos, which enables efficient real-time rendering of interpolated 4D video sequences whilst accurately reproducing visual dynamics, even when using a coarse underlying geometry. We estimate the 4D model flow using an image-based approach that is guided by available geometry proxies. We propose a novel representation in surface texture space for efficient storage and online parametric interpolation of dynamic appearance. Our 4D model flow overcomes previous requirements for computationally expensive online optical flow computation for data-driven alignment of dynamic surface appearance by precomputing the appearance alignment. This leads to an efficient rendering technique that enables the online interpolation between 4D videos in real time, from arbitrary viewpoints and with visual quality comparable to the state of the art.

AB - We introduce the concept of 4D model flow for the precomputed alignment of dynamic surface appearance across 4D video sequences of different motions reconstructed from multi-view video. Precomputed 4D model flow allows the efficient parametrization of surface appearance from the captured videos, which enables efficient real-time rendering of interpolated 4D video sequences whilst accurately reproducing visual dynamics, even when using a coarse underlying geometry. We estimate the 4D model flow using an image-based approach that is guided by available geometry proxies. We propose a novel representation in surface texture space for efficient storage and online parametric interpolation of dynamic appearance. Our 4D model flow overcomes previous requirements for computationally expensive online optical flow computation for data-driven alignment of dynamic surface appearance by precomputing the appearance alignment. This leads to an efficient rendering technique that enables the online interpolation between 4D videos in real time, from arbitrary viewpoints and with visual quality comparable to the state of the art.

UR - http://richardt.name/publications/4d-model-flow/

UR - http://richardt.name/publications/4d-model-flow/

U2 - 10.1111/cgf.12756

DO - 10.1111/cgf.12756

M3 - Article

VL - 34

SP - 173

EP - 182

JO - Computer Graphics Forum

JF - Computer Graphics Forum

SN - 0167-7055

IS - 7

ER -