Unsupervised Attention-guided Image-to-Image Translation

Youssef Alami Mejjati; Christian Richardt; James Tompkin; Darren Cosker; Kwang In Kim

Unsupervised Attention-guided Image-to-Image Translation

Youssef Alami Mejjati, Christian Richardt, James Tompkin, Darren Cosker, Kwang In Kim

Brown University

Research output: Chapter or section in a book/report/conference proceeding › Chapter in a published conference proceeding

227 Citations (SciVal)

Abstract

Current unsupervised image-to-image translation techniques struggle to focus their attention on individual objects without altering the background or the way multiple objects interact within a scene. Motivated by the important role of attention in human perception, we tackle this limitation by introducing unsupervised attention mechanisms that are jointly adversarially trained with the generators and discriminators. We demonstrate qualitatively and quantitatively that our approach attends to relevant regions in the image without requiring supervision, which creates more realistic mappings when compared to those of recent approaches.

This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 665992

Original language	English
Title of host publication	Advances in Neural Information Processing Systems 31 (NIPS), 2018
Editors	S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, R. Garnett
Publisher	Neural Information Processing Systems Foundation, Inc.
Pages	1-22
Number of pages	22
Publication status	Published - 31 Dec 2018
Event	NIPS 2018 - 32nd Conference on Neural Information Processing Systems - Duration: 3 Dec 2018 → 8 Dec 2018

Publication series

Name	NIPS Proceedings
Publisher	Neural Information Processing Systems Foundation, Inc.
ISSN (Electronic)	1049-5258

Conference

Conference	NIPS 2018 - 32nd Conference on Neural Information Processing Systems
Period	3/12/18 → 8/12/18

Access to Document

Fellow for Industrial Research Enhancement (FIRE)
Scott, J. L. (PI) & Yang, Y. (CoI)
EU - Horizon 2020
1/10/15 → 30/03/21
Project: EU Commission
Centre for the Analysis of Motion, Entertainment Research and Applications (CAMERA)
Cosker, D. (PI), Bilzon, J. (CoI), Campbell, N. (CoI), Cazzola, D. (CoI), Colyer, S. (CoI), Fincham Haines, T. (CoI), Hall, P. (CoI), Kim, K. I. (CoI), Lutteroth, C. (CoI), McGuigan, P. (CoI), O'Neill, E. (CoI), Richardt, C. (CoI), Salo, A. (CoI), Seminati, E. (CoI), Tabor, A. (CoI) & Yang, Y. (CoI)
Engineering and Physical Sciences Research Council
1/09/15 → 28/02/21
Project: Research council
EPSRC Centre for Doctoral Training in Digital Entertainment
Willis, P. (PI)
1/04/14 → 30/09/22
Project: Research-related funding

Darren Cosker
- D.P.Coskerbath.acuk
Person: Research & Teaching, Affiliate staff

Cite this

Alami Mejjati, Y., Richardt, C., Tompkin, J., Cosker, D., & Kim, K. I. (2018). Unsupervised Attention-guided Image-to-Image Translation. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, & R. Garnett (Eds.), Advances in Neural Information Processing Systems 31 (NIPS), 2018 (pp. 1-22). (NIPS Proceedings). Neural Information Processing Systems Foundation, Inc.. https://arxiv.org/pdf/1806.02311.pdf

Unsupervised Attention-guided Image-to-Image Translation. / Alami Mejjati, Youssef; Richardt, Christian; Tompkin, James et al.
Advances in Neural Information Processing Systems 31 (NIPS), 2018. ed. / S. Bengio; H. Wallach; H. Larochelle; K. Grauman; N. Cesa-Bianchi; R. Garnett. Neural Information Processing Systems Foundation, Inc., 2018. p. 1-22 (NIPS Proceedings).

Research output: Chapter or section in a book/report/conference proceeding › Chapter in a published conference proceeding

Alami Mejjati, Y, Richardt, C, Tompkin, J, Cosker, D & Kim, KI 2018, Unsupervised Attention-guided Image-to-Image Translation. in S Bengio, H Wallach, H Larochelle, K Grauman, N Cesa-Bianchi & R Garnett (eds), Advances in Neural Information Processing Systems 31 (NIPS), 2018. NIPS Proceedings, Neural Information Processing Systems Foundation, Inc., pp. 1-22, NIPS 2018 - 32nd Conference on Neural Information Processing Systems, 3/12/18. <https://arxiv.org/pdf/1806.02311.pdf>

Alami Mejjati Y, Richardt C, Tompkin J, Cosker D, Kim KI. Unsupervised Attention-guided Image-to-Image Translation. In Bengio S, Wallach H, Larochelle H, Grauman K, Cesa-Bianchi N, Garnett R, editors, Advances in Neural Information Processing Systems 31 (NIPS), 2018. Neural Information Processing Systems Foundation, Inc. 2018. p. 1-22. (NIPS Proceedings). Epub 2018 Dec 3.

Alami Mejjati, Youssef ; Richardt, Christian ; Tompkin, James et al. / Unsupervised Attention-guided Image-to-Image Translation. Advances in Neural Information Processing Systems 31 (NIPS), 2018. editor / S. Bengio ; H. Wallach ; H. Larochelle ; K. Grauman ; N. Cesa-Bianchi ; R. Garnett. Neural Information Processing Systems Foundation, Inc., 2018. pp. 1-22 (NIPS Proceedings).

@inproceedings{4a4207647a1f402c8e428b7e21de2958,

title = "Unsupervised Attention-guided Image-to-Image Translation",

abstract = "Current unsupervised image-to-image translation techniques struggle to focus their attention on individual objects without altering the background or the way multiple objects interact within a scene. Motivated by the important role of attention in human perception, we tackle this limitation by introducing unsupervised attention mechanisms that are jointly adversarially trained with the generators and discriminators. We demonstrate qualitatively and quantitatively that our approach attends to relevant regions in the image without requiring supervision, which creates more realistic mappings when compared to those of recent approaches.This project has received funding from the European Union{\textquoteright}s Horizon 2020 research and innovation programme under the Marie Sk{\l}odowska-Curie grant agreement No 665992 ",

author = "{Alami Mejjati}, Youssef and Christian Richardt and James Tompkin and Darren Cosker and Kim, {Kwang In}",

year = "2018",

month = dec,

day = "31",

language = "English",

series = "NIPS Proceedings",

publisher = "Neural Information Processing Systems Foundation, Inc.",

pages = "1--22",

editor = "S. Bengio and H. Wallach and H. Larochelle and K. Grauman and N. Cesa-Bianchi and R. Garnett",

booktitle = "Advances in Neural Information Processing Systems 31 (NIPS), 2018",

note = "NIPS 2018 - 32nd Conference on Neural Information Processing Systems ; Conference date: 03-12-2018 Through 08-12-2018",

}

TY - GEN

T1 - Unsupervised Attention-guided Image-to-Image Translation

AU - Alami Mejjati, Youssef

AU - Richardt, Christian

AU - Tompkin, James

AU - Cosker, Darren

AU - Kim, Kwang In

PY - 2018/12/31

Y1 - 2018/12/31

N2 - Current unsupervised image-to-image translation techniques struggle to focus their attention on individual objects without altering the background or the way multiple objects interact within a scene. Motivated by the important role of attention in human perception, we tackle this limitation by introducing unsupervised attention mechanisms that are jointly adversarially trained with the generators and discriminators. We demonstrate qualitatively and quantitatively that our approach attends to relevant regions in the image without requiring supervision, which creates more realistic mappings when compared to those of recent approaches.This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 665992

AB - Current unsupervised image-to-image translation techniques struggle to focus their attention on individual objects without altering the background or the way multiple objects interact within a scene. Motivated by the important role of attention in human perception, we tackle this limitation by introducing unsupervised attention mechanisms that are jointly adversarially trained with the generators and discriminators. We demonstrate qualitatively and quantitatively that our approach attends to relevant regions in the image without requiring supervision, which creates more realistic mappings when compared to those of recent approaches.This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 665992

M3 - Chapter in a published conference proceeding

T3 - NIPS Proceedings

SP - 1

EP - 22

BT - Advances in Neural Information Processing Systems 31 (NIPS), 2018

A2 - Bengio, S.

A2 - Wallach, H.

A2 - Larochelle, H.

A2 - Grauman, K.

A2 - Cesa-Bianchi, N.

A2 - Garnett, R.

PB - Neural Information Processing Systems Foundation, Inc.

T2 - NIPS 2018 - 32nd Conference on Neural Information Processing Systems

Y2 - 3 December 2018 through 8 December 2018

ER -

Unsupervised Attention-guided Image-to-Image Translation

Abstract

Publication series

Conference

Access to Document

Fingerprint

Projects

Fellow for Industrial Research Enhancement (FIRE)

Centre for the Analysis of Motion, Entertainment Research and Applications (CAMERA)

EPSRC Centre for Doctoral Training in Digital Entertainment

Profiles

Darren Cosker

Cite this