MAC: Mask-Augmentation for Motion-Aware Video Representation Learning
Arif Akar (Hacettepe University),* Ufuk Umut Senturk (Hacettepe University), Nazli Ikizler-Cinbis (Hacettepe University)
The 33rd British Machine Vision Conference
Abstract
We present MAC, a lightweight, efficient, and novel Mask-Augmentation teChnique and pretext task for self-supervised video representation learning. Most recent and successful methods leverage the instance discrimination approach that requires heavy computation and often leads to inefficient and exhaustive pretraining. We apply MAC augmentation on videos by blending foreground motion using frame-difference-based masks and set up a pretext task to recognize applied transformation. While we incorporate a game of predicting the correct blending multiplier at the pretraining stage, our model is enforced to encode motion-based features which are then successfully transferred to action recognition and video retrieval downstream tasks. Furthermore, we demonstrate the extension of the proposed approach step-by-step to improve representation capabilities in a joint contrastive framework. The proposed method achieves superior performance on UCF-101, HMDB51, and Diving-48 datasets at low resource settings and competitive results with instance discrimination methods at costly computation settings.Video
Citation
@inproceedings{Akar_2022_BMVC,
author = {Arif Akar and Ufuk Umut Senturk and Nazli Ikizler-Cinbis},
title = {MAC: Mask-Augmentation for Motion-Aware Video Representation Learning},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0005.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by