Towards Robust In-domain and Out-of-Domain Generalization: Contrastive Learning with Prototype Alignment and Collaborative Attention
Yuan-Jhe Kuo (National Tsing Hua University),* Cheng-Yu Yang (National Tsing-Hua University), Chiou-Ting Hsu (National Tsing Hua University)
The 33rd British Machine Vision Conference
Abstract
Domain generalization focuses on generalizing a model learned from multiple source domains to the unseen target domain. Assuming the target domain has different distribution from the source domains, most methods addressed the out-of-domain generalization issue but slightly concern the in-domain performance on the source domains. Because the target domain is unseen and may distribute similarly with the source domains, we believe both the in-domain and out-of-domain performances are equally important. In addition, the noisy ground truth labels in the source domains also raises serious concerns on model robustness. Therefore, in this paper, we propose a contrastive learning framework with prototype alignment and collaborative attention to address the robust in-domain and out-of-domain generalization issue for image classification. We first design a margin-based contrastive learning to boost the out-of-domain performance by pushing the ambiguous classes apart by at least a margin. Next, we propose using prototype alignment to support the in-domain performance by aligning the latent feature representation of each class to the corresponding class prototype. Finally, we propose a novel collaborative attention method by leveraging the strength from both positive and negative learnings to enhance the model robustness. Experimental results on two benchmarks show that our method achieves competitive in-domain performance and outperforms previous methods in the out-of-domain and noisy label scenario.Video
Citation
@inproceedings{Kuo_2022_BMVC,
author = {Yuan-Jhe Kuo and Cheng-Yu Yang and Chiou-Ting Hsu},
title = {Towards Robust In-domain and Out-of-Domain Generalization: Contrastive Learning with Prototype Alignment and Collaborative Attention},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0446.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by