MUAD: Multiple Uncertainties for Autonomous Driving, a benchmark for multiple uncertainty types and tasks
Gianni Franchi (ENSTA Paris),* Xuanlong Yu (ENSTA Paris), Andrei Bursuc (valeo.ai), Angel Tena (Next Limit Technologies), Rémi Kazmierczak (ENSTA Paris), Severine Dubuisson (Aix-Marseille University), Emanuel Aldea (Paris-Saclay University), David Filliat (ENSTA Paris)
The 33rd British Machine Vision Conference
Abstract
Predictive uncertainty estimation is essential for safe deployment of Deep Neural Networks in real-world autonomous systems. However, disentangling the different types and sources of uncertainty is non trivial for most datasets, especially since there is no ground truth for uncertainty. In addition, while adverse weather conditions of varying intensities can disrupt neural network predictions, they are usually under-represented in both training and test sets in public datasets. We attempt to mitigate these setbacks and introduce the MUAD dataset (Multiple Uncertainties for Autonomous Driving), consisting of 10,413 realistic synthetic images with diverse adverse weather conditions (night, fog, rain, snow), out-of-distribution objects, and annotations for semantic segmentation, depth estimation, object, and instance detection. MUAD allows to better assess the impact of different sources of uncertainty on model performance. We conduct a thorough experimental study of this impact on several baseline Deep Neural Networks across multiple tasks, and release our dataset to allow researchers to benchmark their algorithm methodically in adverse conditions. More visualizations and the download link for MUAD are available at https://muad-dataset.github.io/.Video
Citation
@inproceedings{Franchi_2022_BMVC,
author = {Gianni Franchi and Xuanlong Yu and Andrei Bursuc and Angel Tena and Rémi Kazmierczak and Severine Dubuisson and Emanuel Aldea and David Filliat},
title = {MUAD: Multiple Uncertainties for Autonomous Driving, a benchmark for multiple uncertainty types and tasks},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0398.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by