Deep Clustering by Semantic Contrastive Learning
Jiabo Huang (Queen Mary University of London),* Shaogang Gong (Queen Mary University of London)
The 33rd British Machine Vision Conference
Abstract
Whilst contrastive learning has recently brought notable benefits to deep clustering of unlabelled images by learning sample-specific discriminative visual features, its potential for explicitly inferring class decision boundaries is less well understood. This is because its instance discrimination strategy is not class sensitive, therefore, the clusters derived on the resulting sample-specific feature space are not optimised for corresponding to meaningful class decision boundaries. In this work, we solve this problem by introducing Semantic Contrastive Learning (SCL). SCL imposes explicitly distance-based cluster structures on unlabelled training data by formulating a semantic (cluster-aware) contrastive learning objective. Moreover, we introduce a clustering consistency condition to be satisfied jointly by both instance visual similarities and cluster decision boundaries, and concurrently optimising both to reason about the hypotheses of semantic ground-truth classes (unknown/unlabelled) on-the-fly by their consensus. This semantic contrastive learning approach to discovering unknown class decision boundaries has considerable advantages to unsupervised learning of object recognition tasks. Extensive experiments show that SCL outperforms state-of-the-art contrastive learning and deep clustering methods on six object recognition benchmarks, especially on the more challenging finer-grained and larger datasets.Video
Citation
@inproceedings{Huang_2022_BMVC,
author = {Jiabo Huang and Shaogang Gong},
title = {Deep Clustering by Semantic Contrastive Learning},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0039.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by