COAT: Correspondence-driven Object Appearance Transfer
Sangryul Jeon (UC Berkeley / ICSI),* Zhifei Zhang (Adobe Research), Zhe Lin (Adobe Research), Scott Cohen (Adobe Research), Zhihong Ding (Adobe Research), Kwanghoon Sohn (Yonsei Univ.)
The 33rd British Machine Vision Conference
Abstract
Semantic correspondence is playing an increasingly important role in photorealistic style transfer, especially on objects with prior structural patterns like faces and cars. Unlike traditional methods that are blind to object/non-object regions and spatial correspondence between objects, we propose a new model called correspondence-driven object appearance transfer (COAT), which leverages correspondence to spatially align texture features to content features at multiple scales. Our model does not require extra supervision like semantic segmentation or body parsing and can be adapted to any given generic object category. More importantly, our multi-scale strategy achieves richer texture transfer, while at the same time preserving the spatial structure of objects in the content image. We further propose the correspondence contrastive loss (CCL) with hard negative mining during the training, boosting appearance transfer with improved disentanglement of structural and textural features. Exhaustive experimental evaluation on various objects demonstrates our superior robustness and visual quality as compared to state-of-the-art works.Citation
@inproceedings{Jeon_2022_BMVC,
author = {Sangryul Jeon and Zhifei Zhang and Zhe Lin and Scott Cohen and Zhihong Ding and Kwanghoon Sohn },
title = {COAT: Correspondence-driven Object Appearance Transfer},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/1053.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by