ShowFace: Coordinated Face Inpainting with Memory-Disentangled Refinement Networks
Zhuojie Wu (Beijing University of Posts and Telecommunications),* Xingqun Qi (University of Technology Sydney), Zijian Wang (Beijing university of Posts and Telecommunications), Wanting Zhou (Beijing University of Posts and Telecommunications), Kun Yuan (Kuaishou Technology), Muyi Sun (CRIPAC, Institute of Automation, Chinese Academy of Sciences), Zhenan Sun (Chinese of Academy of Sciences)
The 33rd British Machine Vision Conference
Abstract
Face inpainting aims to complete the corrupted regions of the face images, which requires coordination between the completed areas and the non-corrupted areas. Recently, memory-oriented methods illustrate great prospects in the generation related tasks by introducing an external memory module to improve image coordination. However, such methods still have limitations in restoring the consistency and continuity for specific facial semantic parts. In this paper, we propose the coarse-to-fine Memory-Disentangled Refinement Networks (MDRNets) for coordinated face inpainting, in which two collaborative modules are integrated, Disentangled Memory Module (DMM) and Mask-Region Enhanced Module (MREM). Specifically, the DMM establishes a group of disentangled memory blocks to store the semantic-decoupled face representations, which could provide the most relevant information to refine the semantic-level coordination. The MREM involves a masked correlation mining mechanism to enhance the feature relationships into the corrupted regions, which could also make up for the correlation loss caused by memory disentanglement. Furthermore, to better improve the inter-coordination between the corrupted and non-corrupted regions and enhance the intra-coordination in corrupted regions, we design InCo$^2$ Loss, a pair of similarity based losses to constrain the feature consistency. Eventually, extensive experiments conducted on CelebA-HQ and FFHQ datasets demonstrate the superiority of our MDRNets compared with previous State-Of-The-Art methods.Video
Citation
@inproceedings{Wu_2022_BMVC,
author = {Zhuojie Wu and Xingqun Qi and Zijian Wang and Wanting Zhou and Kun Yuan and Muyi Sun and Zhenan Sun},
title = {ShowFace: Coordinated Face Inpainting with Memory-Disentangled Refinement Networks},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0052.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by