Event Transformer FlowNet for optical flow estimation
Yi Tian ( Institut de Robòtica i Informàtica Industrial CSIC-UPC),* Juan Andrade-Cetto (Institut de Robòtica i Informàtica Industrial CSIC-UPC)
The 33rd British Machine Vision Conference
Abstract
Event cameras are bioinspired sensors that produce asynchronous and sparse streams of events at image locations where intensity change is detected. They can detect fast motion with low latency, high dynamic range, and low power consumption. Over the past decade, efforts have been conducted in developing solutions with event cameras for robotics applications. In this work, we address their use for fast and robust computation of optical flow. We present ET-FlowNet, a hybrid RNN-ViT architecture for optical flow estimation. Visual transformers (ViTs) are ideal candidates for the learning of global context in visual tasks, and we argue that rigid body motion is a prime case for the use of ViTs since long-range dependencies in the image hold during rigid body motion. We perform end-to-end training with self-supervised learning method. Our results show comparable and in some cases exceeding performance with state-of-the-art coarse-to-fine event-based optical flow estimation.Video
Citation
@inproceedings{Tian_2022_BMVC,
author = {Yi Tian and Juan Andrade-Cetto},
title = {Event Transformer FlowNet for optical flow estimation},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0577.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by