Improving Shape Deformation in Unsupervised Image-to-Image Translation

Aaron  Gokaslan; Vivek Ramanujan; Daniel Ritchie; Kwang In Kim; James Tompkin

doi:10.1007/978-3-030-01258-8_40

Improving Shape Deformation in Unsupervised Image-to-Image Translation

Aaron Gokaslan, Vivek Ramanujan, Daniel Ritchie, Kwang In Kim, James Tompkin

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Unsupervised image-to-image translation techniques are able to map local texture between two domains, but they are typically unsuccessful when the domains require larger shape change. Inspired by semantic segmentation, we introduce a discriminator with dilated convolutions that is able to use information from across the entire image to train a more context-aware generator. This is coupled with a multi-scale perceptual loss that is better able to represent error in the underlying shape of objects. We demonstrate that this design is more capable of representing shape deformation in a challenging toy dataset, plus in complex mappings with significant dataset variation between humans, dolls, and anime faces, and between cats and dogs.

Original language	English
Title of host publication	Proceedings of European Conference on Computer Vision (ECCV)
Subtitle of host publication	Computer Vision - ECCV 2018
Editors	Vittorio Ferrari, Martial Hebert, Cristian Sminchisescu, Yair Weiss
Publisher	Springer Verlag
Pages	662-678
Number of pages	17
Volume	11216
ISBN (Electronic)	978-3-030-01258-8
ISBN (Print)	978-3-030-01257-1
DOIs	https://doi.org/10.1007/978-3-030-01258-8_40
Publication status	E-pub ahead of print - 6 Oct 2018

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	11216 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Keywords

Generative adversarial networks
Image translation

ASJC Scopus subject areas

Theoretical Computer Science
Computer Science(all)

Access to Document

10.1007/978-3-030-01258-8_40

Projects

1 Active

Centre for the Analysis of Motion, Entertainment Research and Applications (CAMERA)

Cosker, D., Campbell, N., Fincham Haines, T., Hall, P., Kim, K. I., Lutteroth, C., O'Neill, E., Richardt, C. & Yang, Y.

Engineering and Physical Sciences Research Council

1/09/15 → 28/02/21

Project: Research council

Cite this

Gokaslan, A., Ramanujan, V., Ritchie, D., Kim, K. I., & Tompkin, J. (2018). Improving Shape Deformation in Unsupervised Image-to-Image Translation. In V. Ferrari, M. Hebert, C. Sminchisescu, & Y. Weiss (Eds.), Proceedings of European Conference on Computer Vision (ECCV): Computer Vision - ECCV 2018 (Vol. 11216, pp. 662-678). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11216 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-030-01258-8_40

Improving Shape Deformation in Unsupervised Image-to-Image Translation. / Gokaslan, Aaron ; Ramanujan, Vivek; Ritchie, Daniel; Kim, Kwang In; Tompkin, James.

Proceedings of European Conference on Computer Vision (ECCV): Computer Vision - ECCV 2018. ed. / Vittorio Ferrari; Martial Hebert; Cristian Sminchisescu; Yair Weiss. Vol. 11216 Springer Verlag, 2018. p. 662-678 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11216 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Gokaslan, A, Ramanujan, V, Ritchie, D, Kim, KI & Tompkin, J 2018, Improving Shape Deformation in Unsupervised Image-to-Image Translation. in V Ferrari, M Hebert, C Sminchisescu & Y Weiss (eds), Proceedings of European Conference on Computer Vision (ECCV): Computer Vision - ECCV 2018. vol. 11216, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11216 LNCS, Springer Verlag, pp. 662-678. https://doi.org/10.1007/978-3-030-01258-8_40

Gokaslan A, Ramanujan V, Ritchie D, Kim KI, Tompkin J. Improving Shape Deformation in Unsupervised Image-to-Image Translation. In Ferrari V, Hebert M, Sminchisescu C, Weiss Y, editors, Proceedings of European Conference on Computer Vision (ECCV): Computer Vision - ECCV 2018. Vol. 11216. Springer Verlag. 2018. p. 662-678. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-030-01258-8_40

Gokaslan, Aaron ; Ramanujan, Vivek ; Ritchie, Daniel ; Kim, Kwang In ; Tompkin, James. / Improving Shape Deformation in Unsupervised Image-to-Image Translation. Proceedings of European Conference on Computer Vision (ECCV): Computer Vision - ECCV 2018. editor / Vittorio Ferrari ; Martial Hebert ; Cristian Sminchisescu ; Yair Weiss. Vol. 11216 Springer Verlag, 2018. pp. 662-678 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{b39b9699cfa04487aca53d7ddd7efad7,

title = "Improving Shape Deformation in Unsupervised Image-to-Image Translation",

abstract = "Unsupervised image-to-image translation techniques are able to map local texture between two domains, but they are typically unsuccessful when the domains require larger shape change. Inspired by semantic segmentation, we introduce a discriminator with dilated convolutions that is able to use information from across the entire image to train a more context-aware generator. This is coupled with a multi-scale perceptual loss that is better able to represent error in the underlying shape of objects. We demonstrate that this design is more capable of representing shape deformation in a challenging toy dataset, plus in complex mappings with significant dataset variation between humans, dolls, and anime faces, and between cats and dogs.",

keywords = "Generative adversarial networks, Image translation",

author = "Aaron Gokaslan and Vivek Ramanujan and Daniel Ritchie and Kim, {Kwang In} and James Tompkin",

year = "2018",

month = oct,

day = "6",

doi = "10.1007/978-3-030-01258-8_40",

language = "English",

isbn = "978-3-030-01257-1",

volume = "11216",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "662--678",

editor = "Vittorio Ferrari and Martial Hebert and Cristian Sminchisescu and Yair Weiss",

booktitle = "Proceedings of European Conference on Computer Vision (ECCV)",

}

TY - GEN

T1 - Improving Shape Deformation in Unsupervised Image-to-Image Translation

AU - Gokaslan, Aaron

AU - Ramanujan, Vivek

AU - Ritchie, Daniel

AU - Kim, Kwang In

AU - Tompkin, James

PY - 2018/10/6

Y1 - 2018/10/6

N2 - Unsupervised image-to-image translation techniques are able to map local texture between two domains, but they are typically unsuccessful when the domains require larger shape change. Inspired by semantic segmentation, we introduce a discriminator with dilated convolutions that is able to use information from across the entire image to train a more context-aware generator. This is coupled with a multi-scale perceptual loss that is better able to represent error in the underlying shape of objects. We demonstrate that this design is more capable of representing shape deformation in a challenging toy dataset, plus in complex mappings with significant dataset variation between humans, dolls, and anime faces, and between cats and dogs.

AB - Unsupervised image-to-image translation techniques are able to map local texture between two domains, but they are typically unsuccessful when the domains require larger shape change. Inspired by semantic segmentation, we introduce a discriminator with dilated convolutions that is able to use information from across the entire image to train a more context-aware generator. This is coupled with a multi-scale perceptual loss that is better able to represent error in the underlying shape of objects. We demonstrate that this design is more capable of representing shape deformation in a challenging toy dataset, plus in complex mappings with significant dataset variation between humans, dolls, and anime faces, and between cats and dogs.

KW - Generative adversarial networks

KW - Image translation

UR - http://www.scopus.com/inward/record.url?scp=85055128063&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-01258-8_40

DO - 10.1007/978-3-030-01258-8_40

M3 - Conference contribution

SN - 978-3-030-01257-1

VL - 11216

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 662

EP - 678

BT - Proceedings of European Conference on Computer Vision (ECCV)

A2 - Ferrari, Vittorio

A2 - Hebert, Martial

A2 - Sminchisescu, Cristian

A2 - Weiss, Yair

PB - Springer Verlag

ER -