Guess What Moves: Unsupervised Video and Image Segmentation by Anticipating Motion
Subhabrata Choudhury (University of Oxford),* Laurynas Karazija (University of Oxford), Iro Laina (University of Oxford), Andrea Vedaldi (Oxford University), Christian Rupprecht (University of Oxford)
The 33rd British Machine Vision Conference
Abstract
Motion, measured via optical flow, provides a powerful cue to discover and learn objects in images and videos. However, compared to using appearance, it has some blind spots, such as the fact that objects become invisible if they do not move. In this work, we propose an approach that combines the strengths of motion-based and appearance-based segmentation. We propose to supervise an image segmentation network with the pretext task of predicting regions that are likely to contain simple motion patterns, and thus likely to correspond to objects. As the model only uses a single image as input, we can apply it in two settings: unsupervised video segmentation, and unsupervised image segmentation. We achieve state-of-the-art results for videos, and demonstrate the viability of our approach on still images containing novel objects. Additionally we experiment with different motion models and optical flow backbones and find the method to be robust to these change. Project page and code available at https://www.robots.ox.ac.uk/~vgg/research/gwm.Video
Citation
@inproceedings{Choudhury_2022_BMVC,
author = {Subhabrata Choudhury and Laurynas Karazija and Iro Laina and Andrea Vedaldi and Christian Rupprecht},
title = {Guess What Moves: Unsupervised Video and Image Segmentation by Anticipating Motion},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0554.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by