Projects per year
Abstract
The Generative Adversarial Network (GAN) and its variations have enabled high quality image generation. However, generating reasonable persons in complex scenes (such as MS-COCO images) remains challenging. We propose a novel structure-based and context-aware approach to enhance the person synthesis in complex scenes. The method can success fully predict the person pose and face structures while respecting the weak layout-based context, then leverage the structures to refine the person appearance. Our method involves three parts. First, a memory-based model is used to encode person intrinsic structures including pose and face key points. Second, a context-aware model infers the conditional person structures from the layout context. Third, the structure-guided personappearancerefinersfurtherenhancethefinalimagegeneration.Ourexperiments present convincing person generation results in layout-to-image tasks on a challenging dataset. Person-related evaluations demonstrate our method achieves state-of-the-art performance, especially on person accuracy and face detection metrics.
Original language | English |
---|---|
Publication status | Acceptance date - 1 Oct 2022 |
Event | British Machine Vision Conference 2022 - Duration: 21 Nov 2022 → 24 Nov 2022 |
Conference
Conference | British Machine Vision Conference 2022 |
---|---|
Period | 21/11/22 → 24/11/22 |
Bibliographical note
This work is supported by RCUK grant CAMERA (EP/M023281/1, EP/T022523/1), Centre for Augmented Reasoning (CAR) at the Australian Institute for Machine Learning, and a gift from AdobeFingerprint
Dive into the research topics of 'Enhancing Person Synthesis in Complex Scenes via Intrinsic and Contextual Structure Modeling'. Together they form a unique fingerprint.-
Centre for the Analysis of Motion, Entertainment Research and Applications (CAMERA) - 2.0
Campbell, N. (PI), Cosker, D. (PI), Bilzon, J. (CoI), Campbell, N. (CoI), Cazzola, D. (CoI), Colyer, S. (CoI), Cosker, D. (CoI), Lutteroth, C. (CoI), McGuigan, P. (CoI), O'Neill, E. (CoI), Petrini, K. (CoI), Proulx, M. (CoI) & Yang, Y. (CoI)
Engineering and Physical Sciences Research Council
1/11/20 → 31/10/25
Project: Research council
-
CAMERA MC2 Award
Cosker, D. (PI), Cosker, D. (PI), Campbell, N. (CoI), Fincham Haines, T. (CoI), Hall, P. (CoI), Li, W. (CoI), Lutteroth, C. (CoI), O'Neill, E. (CoI), Richardt, C. (CoI), Yang, Y. (CoI) & Parsons, M. (Researcher)
2/12/19 → 31/03/23
Project: EU Commission