LOCL: Learning Object-Attribute Composition using Localization


Satish Kumar (University of California, Santa Barbara),* ASM Iftekhar (University of California, Santa Barbara), Ekta Prashnani (University of California, Santa Barbara), B. S. Manjunath (University of California Santa Barbara)
The 33rd British Machine Vision Conference

Abstract

This paper describes LOCL: Learning Object-Attribute (O-A) Composition using Localization – that generalizes composition zero shot learning to objects in cluttered/more realistic settings. The problem of unseen O-A associations has been well studied in the field, however, the performance of existing methods is limited in challenging scenes. In this context, our key contribution is a modular approach to localizing objects and attributes of interest in a weakly supervised context that generalizes robustly to unseen configurations. Localization coupled with a composition classifier significantly outperforms state-of-the-art (SOTA) methods, with an improvement of about 12% on currently available challenging datasets. Further, the modularity enables the use of localized feature extractor to be used with existing O-A compositional learning methods to improve their overall performance.

Video



Citation

@inproceedings{Kumar_2022_BMVC,
author    = {Satish Kumar and ASM Iftekhar and Ekta Prashnani and B. S. Manjunath},
title     = {LOCL: Learning Object-Attribute Composition using Localization},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year      = {2022},
url       = {https://bmvc2022.mpi-inf.mpg.de/0037.pdf}
}


Copyright © 2022 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection