Momentum Adapt: Robust Unsupervised Adaptation for Improving Temporal Consistency in Video Semantic Segmentation During Test-Time
Amirhossein Hassankhani (Tampere University),* Hamed Rezazadegan Tavakoli (Nokia Technologies), Esa Rahtu (Tampere University)
The 34th British Machine Vision Conference
Abstract
Generating temporally consistent outputs in video semantic segmentation is critical, especially in sensitive applications like self-driving cars. Most approaches attempt to solve the temporal inconsistency issue by using optical flow networks or altering the architecture of the network to extract relevant information from multiple input frames. This paper presents Momentum Adapt, an unsupervised online method for improving the temporal consistency in video semantic segmentation. The method uses two semantic segmentation networks with identical architecture and tries to increase the model's confidence by taking their predictions as ground truth. The first network (AuxNet) is updated by backpropagation, while the weights in the second network (MainNet) are the exponential moving average of the weights from the first network. Our extensive quantitative evaluation shows that our approach significantly improves the performance of the network without adaptation. It also outperforms the state-of-the-art algorithm, especially in more severe conditions, including domain shift and noise. These evaluations are performed on three datasets, Cityscapes, KITTI, and SceneNet RGB-D, with many state-of-the-art semantic networks used as the base network for the adaptation algorithms.Citation
@inproceedings{Hassankhani_2023_BMVC,
author = {Amirhossein Hassankhani and Hamed Rezazadegan Tavakoli and Esa Rahtu},
title = {Momentum Adapt: Robust Unsupervised Adaptation for Improving Temporal Consistency in Video Semantic Segmentation During Test-Time},
booktitle = {34th British Machine Vision Conference 2023, {BMVC} 2023, Aberdeen, UK, November 20-24, 2023},
publisher = {BMVA},
year = {2023},
url = {https://bmvc2022.mpi-inf.mpg.de/BMVC2023/0709.pdf}
}
Copyright © 2023
The British Machine Vision Conference is organised by