MAC: Mask-Augmentation for Motion-Aware Video Representation Learning

Arif Akar (Hacettepe University),* Ufuk Umut Senturk (Hacettepe University), Nazli Ikizler-Cinbis (Hacettepe University)
The 33rd British Machine Vision Conference


We present MAC, a lightweight, efficient, and novel Mask-Augmentation teChnique and pretext task for self-supervised video representation learning. Most recent and successful methods leverage the instance discrimination approach that requires heavy computation and often leads to inefficient and exhaustive pretraining. We apply MAC augmentation on videos by blending foreground motion using frame-difference-based masks and set up a pretext task to recognize applied transformation. While we incorporate a game of predicting the correct blending multiplier at the pretraining stage, our model is enforced to encode motion-based features which are then successfully transferred to action recognition and video retrieval downstream tasks. Furthermore, we demonstrate the extension of the proposed approach step-by-step to improve representation capabilities in a joint contrastive framework. The proposed method achieves superior performance on UCF-101, HMDB51, and Diving-48 datasets at low resource settings and competitive results with instance discrimination methods at costly computation settings.



author    = {Arif Akar and Ufuk Umut Senturk and Nazli Ikizler-Cinbis},
title     = {MAC: Mask-Augmentation for Motion-Aware Video Representation Learning},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year      = {2022},
url       = {}

Copyright © 2022 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection