Enhancing Person Synthesis in Complex Scenes via Intrinsic and Contextual Structure Modeling
Xi Tian (University of Bath),* Yongliang Yang (University of Bath), Qi Wu (University of Adelaide)
The 33rd British Machine Vision Conference
Abstract
The Generative Adversarial Network (GAN) and its variations have enabled high-quality image generation. However, generating reasonable persons in complex scenes (such as MS-COCO images) remains challenging. We propose a novel structure-based and context-aware approach to enhance the person synthesis in complex scenes. The method can successfully predict the person pose and face structures while respecting the weak layout-based context, then leverage the structures to refine the person appearance. Our method involves three parts. First, a memory-based model is used to encode person intrinsic structures including pose and face keypoints. Second, a context-aware model infers the conditional person structures from the layout context. Third, the structure-guided person appearance refiners further enhance the final image generation. Our experiments present convincing person generation results in layout-to-image tasks on a challenging dataset. Person-related evaluations demonstrate our method achieves state-of-the-art performance, especially on person accuracy and face detection metrics.Video
Citation
@inproceedings{Tian_2022_BMVC,
author = {Xi Tian and Yongliang Yang and Qi Wu},
title = {Enhancing Person Synthesis in Complex Scenes via Intrinsic and Contextual Structure Modeling},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0491.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by