Scaling up Instance Segmentation using Approximately Localized Phrases
Karan Desai (University of Michigan),* Ishan Misra (Facebook AI Research), Justin Johnson (University of Michigan), Laurens van der Maaten (Facebook)
The 33rd British Machine Vision Conference
Abstract
Training object detectors to segment large numbers of classes is challenging because they require training masks for each class. A potential solution is to partially supervise detectors using only bounding boxes for new object classes. While such boxes are easier to collect than masks, collecting them still requires cumbersome, exhaustive instance labeling from a pre-defined class ontology. We explore using natural language phrases for which a rough localization in the image is available; we refer to such weak supervision as approximately localized phrases (ALPs). We train detectors using masks from COCO dataset and learn to segment 300 Open Images classes, 240 of which do not have any labeled masks/boxes. Results show that ALP-supervised models outperform models that only train with masks for base classes. We also develop a simple one-stage detector to effectively learn from noisy localization of ALPs. Our model outperforms a comparable Mask R-CNN baseline when trained with ALPs. Taken together, our results suggest ALPs may be suitable for learning to segment a large number of object classes.Citation
@inproceedings{Desai_2022_BMVC,
author = {Karan Desai and Ishan Misra and Justin Johnson and Laurens van der Maaten},
title = {Scaling up Instance Segmentation using Approximately Localized Phrases},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0648.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by