SAGE: Saliency-Guided Mixup with Optimal Rearrangements
Avery Ma (University of Toronto and Vector Institute),* NIKITA DVORNIK (Samsung), Ran Zhang (Samsung AI Center Toronto), Leila Pishdad (Samsung), Konstantinos G Derpanis (York University), Afsaneh Fazly (SAIC Toronto)
The 33rd British Machine Vision Conference
Abstract
Data augmentation is a key element for training accurate models by reducing overfitting and improving generalization. For image classification, the most popular data augmentation techniques range from simple photometric and geometrical transformations, to more complex methods that use visual saliency to craft new training examples. As augmentation methods get more complex, their ability to increase the test accuracy improves, yet, such methods become cumbersome, inefficient and lead to poor out-of-domain generalization, as we show in this paper. This motivates a new augmentation technique that allows for high accuracy gains while being simple, efficient (i.e., minimal computation overhead) and generalizable. To this end, we introduce Saliency-Guided Mixup with Optimal Rearrangements (SAGE), which creates new training examples by rearranging and mixing image pairs using visual saliency as guidance. By explicitly leveraging saliency, SAGE promotes discriminative foreground objects and produces informative new images useful for training. We demonstrate on CIFAR-10 and CIFAR-100 that SAGE achieves better or comparable performance to the state of the art while being more efficient. Additionally, evaluations in the out-of-distribution setting show that SAGE achieves improved generalization performance without trading off robustness. Additionally, evaluations in the out-of-distribution setting, and few-shot learning on mini-ImageNet, show that SAGE achieves improved generalization performance without trading off robustness. Our source code is available at https://github.com/SamsungLabs/SAGE.Video
Citation
@inproceedings{Ma_2022_BMVC,
author = {Avery Ma and NIKITA DVORNIK and Ran Zhang and Leila Pishdad and Konstantinos G Derpanis and Afsaneh Fazly},
title = {SAGE: Saliency-Guided Mixup with Optimal Rearrangements},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0484.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by