Motion-Aware Graph Reasoning Hashing for Self-supervised Video Retrieval
Ziyun Zeng (Tsinghua University),* Jinpeng Wang (Tsinghua University), Bin Chen (Harbin Institute of Technology, Shenzhen), Yuting Wang (Tsinghua University), Shu-Tao Xia (Tsinghua University)
The 33rd British Machine Vision Conference
Abstract
Unsupervised video hashing aims to learn a nonlinear hashing function to map videos into a similarity-preserving hamming space without label supervision. Different from static images, the motion information within videos is crucial for content understanding. However, most existing works merely extract general features from sparsely sampled frames and do not explore motion information adequately. On the other hand, directly extracting clip-wise motion features is not practical in inference because of the heavy computation overhead. In this paper, we propose Motion-Aware Graph Reasoning Hashing (MAGRH), an end-to-end framework that utilizes the motion information explicitly while keeping inference efficiency. Specifically, we design a dual-branch architecture consisting of a main branch and an auxiliary branch. During training, the main (auxiliary) branch receives frame-wise (clip-wise) inputs and produces general (motion) hash codes via delicately designed graph reasoning modules and hash layers. On top of the two branches, we develop a combination of intra- and inter-branch contrastive objectives to simultaneously learn branch-specific hashing functions as well as transfer motion knowledge from the auxiliary branch to the main branch. In inference, the hash codes are solely produced by the main branch, which only requires frame-wise inputs. Benefiting from motion guidance, our MAGRH yields superior performance on two public benchmarks, i.e., FCVID and ActivityNet, even with a small frame rate.Video
Citation
@inproceedings{Zeng_2022_BMVC,
author = {Ziyun Zeng and Jinpeng Wang and Bin Chen and Yuting Wang and Shu-Tao Xia},
title = {Motion-Aware Graph Reasoning Hashing for Self-supervised Video Retrieval},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0082.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by