Improving Shape Deformation in Unsupervised Image-to-Image Translation

Aaron  Gokaslan; Vivek Ramanujan; Daniel Ritchie; Kwang In Kim; James Tompkin

doi:10.1007/978-3-030-01258-8_40

Improving Shape Deformation in Unsupervised Image-to-Image Translation

Aaron Gokaslan, Vivek Ramanujan, Daniel Ritchie, Kwang In Kim, James Tompkin

Brown University

Research output: Chapter or section in a book/report/conference proceeding › Chapter in a published conference proceeding

7 Citations (SciVal)

Abstract

Unsupervised image-to-image translation techniques are able to map local texture between two domains, but they are typically unsuccessful when the domains require larger shape change. Inspired by semantic segmentation, we introduce a discriminator with dilated convolutions that is able to use information from across the entire image to train a more context-aware generator. This is coupled with a multi-scale perceptual loss that is better able to represent error in the underlying shape of objects. We demonstrate that this design is more capable of representing shape deformation in a challenging toy dataset, plus in complex mappings with significant dataset variation between humans, dolls, and anime faces, and between cats and dogs.

Original language	English
Title of host publication	Proceedings of European Conference on Computer Vision (ECCV)
Subtitle of host publication	Computer Vision - ECCV 2018
Editors	Vittorio Ferrari, Martial Hebert, Cristian Sminchisescu, Yair Weiss
Publisher	Springer Verlag
Pages	662-678
Number of pages	17
Volume	11216
ISBN (Electronic)	978-3-030-01258-8
ISBN (Print)	978-3-030-01257-1
DOIs	https://doi.org/10.1007/978-3-030-01258-8_40
Publication status	E-pub ahead of print - 6 Oct 2018

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	11216 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Keywords

Generative adversarial networks
Image translation

ASJC Scopus subject areas

Theoretical Computer Science
General Computer Science

Access to Document

10.1007/978-3-030-01258-8_40

Cite this

Gokaslan, A., Ramanujan, V., Ritchie, D., Kim, K. I., & Tompkin, J. (2018). Improving Shape Deformation in Unsupervised Image-to-Image Translation. In V. Ferrari, M. Hebert, C. Sminchisescu, & Y. Weiss (Eds.), Proceedings of European Conference on Computer Vision (ECCV): Computer Vision - ECCV 2018 (Vol. 11216, pp. 662-678). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11216 LNCS). Springer Verlag. Advance online publication. https://doi.org/10.1007/978-3-030-01258-8_40

Improving Shape Deformation in Unsupervised Image-to-Image Translation. / Gokaslan, Aaron ; Ramanujan, Vivek; Ritchie, Daniel et al.
Proceedings of European Conference on Computer Vision (ECCV): Computer Vision - ECCV 2018. ed. / Vittorio Ferrari; Martial Hebert; Cristian Sminchisescu; Yair Weiss. Vol. 11216 Springer Verlag, 2018. p. 662-678 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11216 LNCS).

Research output: Chapter or section in a book/report/conference proceeding › Chapter in a published conference proceeding

Gokaslan, A, Ramanujan, V, Ritchie, D, Kim, KI & Tompkin, J 2018, Improving Shape Deformation in Unsupervised Image-to-Image Translation. in V Ferrari, M Hebert, C Sminchisescu & Y Weiss (eds), Proceedings of European Conference on Computer Vision (ECCV): Computer Vision - ECCV 2018. vol. 11216, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11216 LNCS, Springer Verlag, pp. 662-678. https://doi.org/10.1007/978-3-030-01258-8_40

Gokaslan A, Ramanujan V, Ritchie D, Kim KI, Tompkin J. Improving Shape Deformation in Unsupervised Image-to-Image Translation. In Ferrari V, Hebert M, Sminchisescu C, Weiss Y, editors, Proceedings of European Conference on Computer Vision (ECCV): Computer Vision - ECCV 2018. Vol. 11216. Springer Verlag. 2018. p. 662-678. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). Epub 2018 Oct 6. doi: 10.1007/978-3-030-01258-8_40

Gokaslan, Aaron ; Ramanujan, Vivek ; Ritchie, Daniel et al. / Improving Shape Deformation in Unsupervised Image-to-Image Translation. Proceedings of European Conference on Computer Vision (ECCV): Computer Vision - ECCV 2018. editor / Vittorio Ferrari ; Martial Hebert ; Cristian Sminchisescu ; Yair Weiss. Vol. 11216 Springer Verlag, 2018. pp. 662-678 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{b39b9699cfa04487aca53d7ddd7efad7,

title = "Improving Shape Deformation in Unsupervised Image-to-Image Translation",

abstract = "Unsupervised image-to-image translation techniques are able to map local texture between two domains, but they are typically unsuccessful when the domains require larger shape change. Inspired by semantic segmentation, we introduce a discriminator with dilated convolutions that is able to use information from across the entire image to train a more context-aware generator. This is coupled with a multi-scale perceptual loss that is better able to represent error in the underlying shape of objects. We demonstrate that this design is more capable of representing shape deformation in a challenging toy dataset, plus in complex mappings with significant dataset variation between humans, dolls, and anime faces, and between cats and dogs.",

keywords = "Generative adversarial networks, Image translation",

author = "Aaron Gokaslan and Vivek Ramanujan and Daniel Ritchie and Kim, {Kwang In} and James Tompkin",

year = "2018",

month = oct,

day = "6",

doi = "10.1007/978-3-030-01258-8_40",

language = "English",

isbn = "978-3-030-01257-1",

volume = "11216",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "662--678",

editor = "Vittorio Ferrari and Martial Hebert and Cristian Sminchisescu and Yair Weiss",

booktitle = "Proceedings of European Conference on Computer Vision (ECCV)",

}

TY - GEN

T1 - Improving Shape Deformation in Unsupervised Image-to-Image Translation

AU - Gokaslan, Aaron

AU - Ramanujan, Vivek

AU - Ritchie, Daniel

AU - Kim, Kwang In

AU - Tompkin, James

PY - 2018/10/6

Y1 - 2018/10/6

N2 - Unsupervised image-to-image translation techniques are able to map local texture between two domains, but they are typically unsuccessful when the domains require larger shape change. Inspired by semantic segmentation, we introduce a discriminator with dilated convolutions that is able to use information from across the entire image to train a more context-aware generator. This is coupled with a multi-scale perceptual loss that is better able to represent error in the underlying shape of objects. We demonstrate that this design is more capable of representing shape deformation in a challenging toy dataset, plus in complex mappings with significant dataset variation between humans, dolls, and anime faces, and between cats and dogs.

AB - Unsupervised image-to-image translation techniques are able to map local texture between two domains, but they are typically unsuccessful when the domains require larger shape change. Inspired by semantic segmentation, we introduce a discriminator with dilated convolutions that is able to use information from across the entire image to train a more context-aware generator. This is coupled with a multi-scale perceptual loss that is better able to represent error in the underlying shape of objects. We demonstrate that this design is more capable of representing shape deformation in a challenging toy dataset, plus in complex mappings with significant dataset variation between humans, dolls, and anime faces, and between cats and dogs.

KW - Generative adversarial networks

KW - Image translation

UR - http://www.scopus.com/inward/record.url?scp=85055128063&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-01258-8_40

DO - 10.1007/978-3-030-01258-8_40

M3 - Chapter in a published conference proceeding

SN - 978-3-030-01257-1

VL - 11216

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 662

EP - 678

BT - Proceedings of European Conference on Computer Vision (ECCV)

A2 - Ferrari, Vittorio

A2 - Hebert, Martial

A2 - Sminchisescu, Cristian

A2 - Weiss, Yair

PB - Springer Verlag

ER -

Improving Shape Deformation in Unsupervised Image-to-Image Translation

Abstract

Publication series

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Centre for the Analysis of Motion, Entertainment Research and Applications (CAMERA)

Cite this

Improving Shape Deformation in Unsupervised Image-to-Image Translation

Abstract

Publication series

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Projects

Centre for the Analysis of Motion, Entertainment Research and Applications (CAMERA)

Cite this