Can Deep Networks be Highly Performant, Efficient and Robust simultaneously?
Madan Ravi Ganesh (BCAI),* Salimeh Yasaei Sekeh (University of Maine), Jason J Corso (University of Michigan)
The 34th British Machine Vision Conference
Abstract
Performance is not enough when it comes to deep neural networks (DNNs); in real-world settings, computational load or efficiency during training and adversarial security are just as or even more important. Often there are critical trade-offs to consider when prioritizing one goal over the others. Instead, we propose to concurrently target Performance, Efficiency, and Robustness, and ask just how far we can push the envelope on simultaneously achieving these goals. Our algorithm, CAPER, follows the intuition that samples highly susceptible to noise strongly affect the decision boundaries learned by DNNs, which in turn degrades their performance and adversarial robustness. By identifying and removing such samples, we demonstrate increased performance and adversarial robustness while using only a subset of the training data, thereby improving the training efficiency. Through our experiments, we highlight CAPER’s high performance across multiple Dataset-DNN combinations, and provide insights into the complementary behavior of CAPER alongside existing adversarial training approaches to increase robustness by over 11.6% while using up to 4% fewer FLOPs during training.Citation
@inproceedings{Ganesh_2023_BMVC,
author = {Madan Ravi Ganesh and Salimeh Yasaei Sekeh and Jason J Corso},
title = {Can Deep Networks be Highly Performant, Efficient and Robust simultaneously?},
booktitle = {34th British Machine Vision Conference 2023, {BMVC} 2023, Aberdeen, UK, November 20-24, 2023},
publisher = {BMVA},
year = {2023},
url = {https://bmvc2022.mpi-inf.mpg.de/BMVC2023/0187.pdf}
}
Copyright © 2023
The British Machine Vision Conference is organised by