CounTR: Transformer-based Generalised Visual Counting
Chang Liu (Shanghai Jiao Tong University), Yujie Zhong (University of Oxford), Andrew Zisserman (University of Oxford), Weidi Xie (Shanghai Jiao Tong University)*
The 33rd British Machine Vision Conference
Abstract
In this paper, we consider the problem of generalised visual object counting, with the goal of developing a computational model for counting the number of objects from arbitrary semantic categories, using arbitrary number of “exemplars”, i.e. zero-shot or few-shot counting. To this end, we make the following four contributions: (1) We introduce a novel transformer-based architecture for generalised visual object counting, termed as Counting TRansformer (CounTR), which explicitly capture the similarity between image patches or with given “exemplars” with the attention mechanism; (2) We adopt a two-stage training regime, that first pre-trains the model with self-supervised learning, and followed by supervised fine-tuning; (3) We propose a simple, scalable pipeline for synthesizing training images with a large number of instances or that from different semantic categories, explicitly forcing the model to make use of the given “exemplars”; (4) We conduct thorough ablation studies on the large-scale counting benchmark, e.g. FSC-147, and demonstrate state-of-the-art performance on both zero and few-shot settings.Video
Citation
@inproceedings{Liu_2022_BMVC,
author = {Chang Liu and Yujie Zhong and Andrew Zisserman and Weidi Xie},
title = {CounTR: Transformer-based Generalised Visual Counting},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0370.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by