MagFormer: Hybrid Video Motion Magnification Transformer from Eulerian and Lagrangian Perspectives
Sicheng Gao (Beihang University), Yutang Feng (Beihang University), Linlin Yang ( University of Bonn),* Xuhui Liu (Beihang University), Zichen Zhu (Harbin Institute of Technology), David Doermann (University at Buffalo), Baochang Zhang (Beihang University)
The 33rd British Machine Vision Conference
Abstract
Video motion magnification methods attract much attention for their strong capability of capturing informative subtle signals from diverse engineering scenes. There are two main types of methods in this field, Eulerian and Lagrangian motion magnification, which have different advantages and perspectives. However, the combination of both remains largely unexplored. In this paper, we develop an end-to-end video motion magnification network, MagFormer, with a well-designed two-branch magnification module, which includes convolutional neural network (CNN) for the Eulerian motion magnification branch and Transformer for the Lagrangian optical flow magnification branch. Our MagFormer can inherit the advantages of two perspectives, by leveraging both Eulerian global motion features from the camera-centered perspective and trajectories of the object-centered from the Lagrangian object perspective in a unified parallel framework. To validate the effectiveness of our method, we collect a new vibration dataset to measure video motion magnification methods via amplitude and frequency.More experiments are conducted on fixed-background subtle motion videos, constantly moving object videos and quantitative vibration videos. Experimental results show that our method achieves a favorable improvement compared to state-of-the-art methods.Video
Citation
@inproceedings{Gao_2022_BMVC,
author = {Sicheng Gao and Yutang Feng and Linlin Yang and Xuhui Liu and Zichen Zhu and David Doermann and Baochang Zhang},
title = {MagFormer: Hybrid Video Motion Magnification Transformer from Eulerian and Lagrangian Perspectives},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0444.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by