Adaptive-TTA: accuracy-consistent weighted test time augmentation method for the uncertainty calibration of deep learning classifiers
Pedro Conde (Institute of Systems and Robotics, University of Coimbra),* Cristiano Premebida (University of Coimbra)
The 33rd British Machine Vision Conference
Abstract
Building deep machine learning systems to classify image data in real-world applications requires not only a quantification of the accuracy of the models but also an understanding of their reliability. With this in mind, the uncertainty calibration of Deep Neural Networks in the task of image classification is addressed in this work. We propose a novel technique based on test time augmentation, called Adaptive-TTA, that - unlike traditional test time augmentation approaches - improves uncertainty calibration without affecting the model’s accuracy. This technique is evaluated with respect to the Brier score - a proper scoring rule for measuring the calibration of predicted probabilities - on the classical CIFAR-10/CIFAR-100 computer vision datasets, as well as on the benchmark satellite imagery dataset AID, using different augmentation policies. Our approach outperforms temperature scaling, a state-of-the-art post-hoc calibration technique, on all the three aforementioned datasets.Video
Citation
@inproceedings{Conde_2022_BMVC,
author = {Pedro Conde and Cristiano Premebida},
title = {Adaptive-TTA: accuracy-consistent weighted test time augmentation method for the uncertainty calibration of deep learning classifiers},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0869.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by