AISFormer: Amodal Instance Segmentation with Transformer
Minh Q Tran (University of Arkansas),* Khoa HV Vo (University of Arkansas), Kashu Yamazaki (University of Arkansas), Arthur Fernandes (Cobb-Vantress / Tyson Foods, Inc), Michael T Kidd (University of Arkansas), Ngan Le (University of Arkansas)
The 33rd British Machine Vision Conference
Abstract
Amodal Instance Segmentation (AIS) aims to segment the region of both visible and possible occluded parts of an object instance. While Mask R-CNN-based AIS approaches have shown promising results, they are unable to model high-level features coherence due to the limited receptive field. The most recent transformer-based models show impressive performance on vision tasks, even better than Convolution Neural Networks (CNN). In this work, we present AISFormer, an AIS framework, with a Transformer-based mask head. AISFormer explicitly models the complex coherence between occluder, visible, amodal, and invisible masks within an object's regions of interest by treating them as learnable queries. Specifically, AISFormer contains four modules: (i) feature encoding: extract ROI and learn both short-range and long-range visual features. (ii) mask transformer decoding: generate the occluder, visible, and amodal mask query embeddings by a transformer decoder (iii) invisible mask embedding: model the coherence between the amodal and visible masks, and (iv) mask predicting: estimate output masks including occluder, visible, amodal and invisible. We conduct extensive experiments and ablation studies on three challenging benchmarks i.e. KINS, D2SA, and COCOA-cls to evaluate the effectiveness of AISFormer.Video
Citation
@inproceedings{Tran_2022_BMVC,
author = {Minh Q Tran and Khoa HV Vo and Kashu Yamazaki and Arthur Fernandes and Michael T Kidd and Ngan Le},
title = {AISFormer: Amodal Instance Segmentation with Transformer},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0712.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by