Consistency-CAM: Towards Improved Weakly Supervised Semantic Segmentation
Sai Rajeswar (University of Montreal),* Issam Hadj Laradji (ServiceNow), Pau Rodriguez (ServiceNow), David Vazquez (Element AI), Aaron Courville (MILA, Université de Montréal)
The 33rd British Machine Vision Conference
Abstract
Semantic segmentation is a popular task that has piqued the interest of many industries and research communities. However, acquiring segmentation labels is costly as it often requires carefully annotating the boundaries of the objects of interest. This has triggered research on weakly-supervised methods with image-level labels that are less costly to obtain. Existing methods leverage pseudo-labels produced from class activation maps (CAM) generated with models pre-trained on ImageNet. Using CAMs as pseudo-labels introduces two different challenges. First, ImageNet pre-training biases models to predict a single object per image. Second, pseudo-labels are noisy. In this work, we address the first problem by pre-training the backbone with multi-label iterated learning. In the literature, the second problem is usually alleviated by introducing an additional consistency loss during the backbone pre-training or as an additional CAM refinement step. Here, we propose a generalization of Puzzle-CAMs consistency loss that supports multiple augmentations and tiling resolutions, which helps to further reduce the noise in CAMs and improve the final segmentation performance. The results show improved results in both PASCAL VOC and COCO in the weakly supervised settings for the mIoU scores compared to existing methods.Video
Citation
@inproceedings{Rajeswar_2022_BMVC,
author = {Sai Rajeswar and Issam Hadj Laradji and Pau Rodriguez and David Vazquez and Aaron Courville},
title = {Consistency-CAM: Towards Improved Weakly Supervised Semantic Segmentation},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0644.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by