Propagating Difference Flows for Efficient Video Super-Resolution


Ruisheng Gao (University of Science and Technology of China), Zeyu Xiao (University of Science and Technology of China), Zhiwei Xiong (University of Science and Technology of China)*
The 33rd British Machine Vision Conference

Abstract

Recent years have witnessed the advancement of video super-resolution (VSR) with elaborately-designed multi-frame alignment and space-time fusion/refinement techniques. However, both techniques require heavy computational burden and memory consumption, hindering existing VSR networks from being deployed on resource-constrained platforms (e.g., smartphones and wearable devices). In this paper, we propose an efficient and lightweight VSR network with two special designs. First, we propose a novel motion propagation scheme which propagates difference flows for efficient feature alignment. The difference flow is sparse and computational-friendly which focuses on texture details. After estimating the preliminary difference flow with an initial motion estimator, we then design an adaptive motion modification module for frame-pair wise adaptation through bidirectional propagation. Second, a dense feature distillation module is designed for further refining the aligned features efficiently. Thanks to both designs, our network achieves comparable performance with state-of-the-art VSR methods while enjoying a clear advantage in model size and computational efficiency.

Video



Citation

@inproceedings{Gao_2022_BMVC,
author    = {Ruisheng Gao and Zeyu Xiao and Zhiwei Xiong},
title     = {Propagating Difference Flows for Efficient Video Super-Resolution},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year      = {2022},
url       = {https://bmvc2022.mpi-inf.mpg.de/0060.pdf}
}


Copyright © 2022 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection