Efficient Feature Extraction for High-resolution Video Frame Interpolation
Moritz Nottebaum (TU Darmstadt), Stefan Roth (TU Darmstadt), Simone Schaub-Meyer (TU Darmstadt)*
The 33rd British Machine Vision Conference
Abstract
Most deep learning methods for video frame interpolation consist of three main components: feature extraction, motion estimation, and image synthesis. Existing approaches are mainly distinguishable in terms of how these modules are designed. However, when interpolating high-resolution images, e.g. at 4K, the design choices for achieving high accuracy within reasonable memory requirements are limited. The feature extraction layers help to compress the input and extract relevant information for the latter stages, such as motion estimation. However, these layers are often costly in parameters, computation time, and memory. We show how ideas from dimensionality reduction combined with a lightweight optimization can be used to compress the input representation while keeping the extracted information suitable for frame interpolation. Further, we require neither a pretrained flow network nor a synthesis network, additionally reducing the number of trainable parameters and required memory. When evaluating on three 4K benchmarks, we achieve state-of-the-art image quality among the methods without pretrained flow while having the lowest network complexity and memory requirements overall.Video
Citation
@inproceedings{Nottebaum_2022_BMVC,
author = {Moritz Nottebaum and Stefan Roth and Simone Schaub-Meyer},
title = {Efficient Feature Extraction for High-resolution Video Frame Interpolation},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0825.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by