Weak-shot Semantic Segmentation by Transferring Semantic Affinity and Boundary
Siyuan Zhou (Shanghai Jiao Tong University), Li Niu (Shanghai Jiao Tong University),* Jianlou Si ( SenseTime), Chen Qian (SenseTime), Liqing Zhang (Shanghai Jiao Tong University)
The 33rd British Machine Vision Conference
Abstract
Weakly-supervised semantic segmentation (WSSS) with image-level labels has been widely studied to relieve the annotation burden of the traditional segmentation task. In this paper, we show that existing fully-annotated base categories can help segment objects of novel categories with only image-level labels, even if base categories and novel categories have no overlap. We refer to this task as weak-shot semantic segmentation, which could also be treated as WSSS with auxiliary fully-annotated categories. Recent advanced WSSS methods usually obtain class activation maps (CAMs) and refine them by affinity propagation. Based on the observation that semantic affinity and boundary are class-agnostic, we propose a method under the WSSS framework to transfer semantic affinity and boundary from base to novel categories. As a result, we find that pixel-level annotation of base categories can facilitate affinity learning and propagation, leading to higher-quality CAMs of novel categories. Extensive experiments on PASCAL VOC 2012 dataset prove that our method significantly outperforms WSSS baselines on novel categories.Video
Citation
@inproceedings{Zhou_2022_BMVC,
author = {Siyuan Zhou and Li Niu and Jianlou Si and Chen Qian and Liqing Zhang},
title = {Weak-shot Semantic Segmentation by Transferring Semantic Affinity and Boundary},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0211.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by