Exploring Localization for Self-supervised Fine-grained Contrastive Learning
di wu (Westlake University), Siyuan Li (Westlake University),* Zelin Zang (Zhejiang University & Westlake University), Stan Z. Li (Westlake University)
The 33rd British Machine Vision Conference
Abstract
Self-supervised contrastive learning has demonstrated great potential in learning visual representations. Despite their success in various downstream tasks such as image classification and object detection, self-supervised pre-training for fine-grained scenarios is not fully explored. We point out that current contrastive methods are prone to memorizing background/foreground texture and therefore have a limitation in localizing the foreground object. Analysis suggests that learning to extract discriminative texture information and localization are equally crucial for fine-grained self-supervised pre-training. Based on our findings, we introduce cross-view saliency alignment (CVSA), a contrastive learning framework that first crops and swaps saliency regions of images as a novel view generation and then guides the model to localize on foreground objects via a cross-view alignment loss. Extensive experiments on both small- and large-scale fine-grained classification benchmarks show that CVSA significantly improves the learned representation.Video
Citation
@inproceedings{wu_2022_BMVC,
author = {di wu and Siyuan Li and Zelin Zang and Stan Z. Li},
title = {Exploring Localization for Self-supervised Fine-grained Contrastive Learning},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0268.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by