Resolving Semantic Confusions for Improved Zero-Shot Detection
Sandipan Sarma (Indian Institute of Technology Guwahati),* SUSHIL KUMAR (Indian Institute of Technology Guwahati), Arijit Sur (IIT Guwahati)
The 33rd British Machine Vision Conference
Abstract
Zero-shot detection (ZSD) is a challenging task where we aim to recognize and localize objects simultaneously, even when our model has not been trained with visual samples of a few target (``unseen'') classes. Recently, methods employing generative models like GANs have shown some of the best results, where unseen-class samples are generated based on their semantics by a GAN trained on seen-class data, enabling vanilla object detectors to recognize unseen objects. However, the problem of semantic confusion still remains, where the model is sometimes unable to distinguish between semantically-similar classes. In this work, we propose to train a generative model incorporating a triplet loss that acknowledges the degree of dissimilarity between classes and reflects them in the generated samples. Moreover, a cyclic-consistency loss is also enforced to ensure that generated visual samples of a class highly correspond to their own semantics. Extensive experiments on two benchmark ZSD datasets -- MSCOCO and PASCAL-VOC -- demonstrate significant gains over the current ZSD methods, reducing semantic confusion and improving detection for the unseen classes.Video
Citation
@inproceedings{Sarma_2022_BMVC,
author = {Sandipan Sarma and SUSHIL KUMAR and Arijit Sur},
title = {Resolving Semantic Confusions for Improved Zero-Shot Detection},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0347.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by