Improving Shape Deformation in Unsupervised Image-to-Image Translation

Aaron Gokaslan, Vivek Ramanujan, Daniel Ritchie, Kwang In Kim, James Tompkin

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Unsupervised image-to-image translation techniques are able to map local texture between two domains, but they are typically unsuccessful when the domains require larger shape change. Inspired by semantic segmentation, we introduce a discriminator with dilated convolutions that is able to use information from across the entire image to train a more context-aware generator. This is coupled with a multi-scale perceptual loss that is better able to represent error in the underlying shape of objects. We demonstrate that this design is more capable of representing shape deformation in a challenging toy dataset, plus in complex mappings with significant dataset variation between humans, dolls, and anime faces, and between cats and dogs.
LanguageEnglish
Title of host publicationProceedings of European Conference on Computer Vision (ECCV)
Subtitle of host publicationComputer Vision - ECCV 2018
EditorsVittorio Ferrari, Martial Hebert, Cristian Sminchisescu, Yair Weiss
PublisherSpringer Verlag
Pages662-678
Number of pages17
Volume11216
ISBN (Electronic)978-3-030-01258-8
ISBN (Print)978-3-030-01257-1
DOIs
StatusE-pub ahead of print - 6 Oct 2018

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11216 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Keywords

  • Generative adversarial networks
  • Image translation

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Gokaslan, A., Ramanujan, V., Ritchie, D., Kim, K. I., & Tompkin, J. (2018). Improving Shape Deformation in Unsupervised Image-to-Image Translation. In V. Ferrari, M. Hebert, C. Sminchisescu, & Y. Weiss (Eds.), Proceedings of European Conference on Computer Vision (ECCV): Computer Vision - ECCV 2018 (Vol. 11216, pp. 662-678). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11216 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-030-01258-8_40

Improving Shape Deformation in Unsupervised Image-to-Image Translation. / Gokaslan, Aaron ; Ramanujan, Vivek; Ritchie, Daniel; Kim, Kwang In; Tompkin, James.

Proceedings of European Conference on Computer Vision (ECCV): Computer Vision - ECCV 2018. ed. / Vittorio Ferrari; Martial Hebert; Cristian Sminchisescu; Yair Weiss. Vol. 11216 Springer Verlag, 2018. p. 662-678 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11216 LNCS).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Gokaslan, A, Ramanujan, V, Ritchie, D, Kim, KI & Tompkin, J 2018, Improving Shape Deformation in Unsupervised Image-to-Image Translation. in V Ferrari, M Hebert, C Sminchisescu & Y Weiss (eds), Proceedings of European Conference on Computer Vision (ECCV): Computer Vision - ECCV 2018. vol. 11216, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11216 LNCS, Springer Verlag, pp. 662-678. https://doi.org/10.1007/978-3-030-01258-8_40
Gokaslan A, Ramanujan V, Ritchie D, Kim KI, Tompkin J. Improving Shape Deformation in Unsupervised Image-to-Image Translation. In Ferrari V, Hebert M, Sminchisescu C, Weiss Y, editors, Proceedings of European Conference on Computer Vision (ECCV): Computer Vision - ECCV 2018. Vol. 11216. Springer Verlag. 2018. p. 662-678. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-030-01258-8_40
Gokaslan, Aaron ; Ramanujan, Vivek ; Ritchie, Daniel ; Kim, Kwang In ; Tompkin, James. / Improving Shape Deformation in Unsupervised Image-to-Image Translation. Proceedings of European Conference on Computer Vision (ECCV): Computer Vision - ECCV 2018. editor / Vittorio Ferrari ; Martial Hebert ; Cristian Sminchisescu ; Yair Weiss. Vol. 11216 Springer Verlag, 2018. pp. 662-678 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{b39b9699cfa04487aca53d7ddd7efad7,
title = "Improving Shape Deformation in Unsupervised Image-to-Image Translation",
abstract = "Unsupervised image-to-image translation techniques are able to map local texture between two domains, but they are typically unsuccessful when the domains require larger shape change. Inspired by semantic segmentation, we introduce a discriminator with dilated convolutions that is able to use information from across the entire image to train a more context-aware generator. This is coupled with a multi-scale perceptual loss that is better able to represent error in the underlying shape of objects. We demonstrate that this design is more capable of representing shape deformation in a challenging toy dataset, plus in complex mappings with significant dataset variation between humans, dolls, and anime faces, and between cats and dogs.",
keywords = "Generative adversarial networks, Image translation",
author = "Aaron Gokaslan and Vivek Ramanujan and Daniel Ritchie and Kim, {Kwang In} and James Tompkin",
year = "2018",
month = "10",
day = "6",
doi = "10.1007/978-3-030-01258-8_40",
language = "English",
isbn = "978-3-030-01257-1",
volume = "11216",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer Verlag",
pages = "662--678",
editor = "Vittorio Ferrari and Martial Hebert and Cristian Sminchisescu and Yair Weiss",
booktitle = "Proceedings of European Conference on Computer Vision (ECCV)",

}

TY - GEN

T1 - Improving Shape Deformation in Unsupervised Image-to-Image Translation

AU - Gokaslan, Aaron

AU - Ramanujan, Vivek

AU - Ritchie, Daniel

AU - Kim, Kwang In

AU - Tompkin, James

PY - 2018/10/6

Y1 - 2018/10/6

N2 - Unsupervised image-to-image translation techniques are able to map local texture between two domains, but they are typically unsuccessful when the domains require larger shape change. Inspired by semantic segmentation, we introduce a discriminator with dilated convolutions that is able to use information from across the entire image to train a more context-aware generator. This is coupled with a multi-scale perceptual loss that is better able to represent error in the underlying shape of objects. We demonstrate that this design is more capable of representing shape deformation in a challenging toy dataset, plus in complex mappings with significant dataset variation between humans, dolls, and anime faces, and between cats and dogs.

AB - Unsupervised image-to-image translation techniques are able to map local texture between two domains, but they are typically unsuccessful when the domains require larger shape change. Inspired by semantic segmentation, we introduce a discriminator with dilated convolutions that is able to use information from across the entire image to train a more context-aware generator. This is coupled with a multi-scale perceptual loss that is better able to represent error in the underlying shape of objects. We demonstrate that this design is more capable of representing shape deformation in a challenging toy dataset, plus in complex mappings with significant dataset variation between humans, dolls, and anime faces, and between cats and dogs.

KW - Generative adversarial networks

KW - Image translation

UR - http://www.scopus.com/inward/record.url?scp=85055128063&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-01258-8_40

DO - 10.1007/978-3-030-01258-8_40

M3 - Conference contribution

SN - 978-3-030-01257-1

VL - 11216

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 662

EP - 678

BT - Proceedings of European Conference on Computer Vision (ECCV)

A2 - Ferrari, Vittorio

A2 - Hebert, Martial

A2 - Sminchisescu, Cristian

A2 - Weiss, Yair

PB - Springer Verlag

ER -