Revisiting Deep Fisher Vectors: Using Fisher Information to Improve Object Classification
SARAH AHMED (Center of Artificial Intelligence in Health Sciences, ICCBS), Tayyaba Azim (University of Southampton),* Joseph Early (University of Southampton), Sarvapali Ramchurn (University of Southampton)
The 33rd British Machine Vision Conference
Abstract
Although deep learning models have become the gold standard in achieving outstanding results on a large variety of computer vision and machine learning tasks, the use of kernel methods has still not gone out of trend because of its potential to beat deep learning performances at a number of occasions. Given the potential of kernel techniques, prior works have also proposed the use of hybrid approaches combining deep learning with kernel learning to complement their respective strengths and weaknesses. This work develops this idea further by introducing an improved version of Fisher kernels derived from the deep Boltzmann machines (DBM). Our improved deep Fisher kernel (IDFK) utilises an approximation of the Fisher information matrix to derive improved Fisher vectors. We show IDFK can be utilised to retain a high degree of class separability, making it appropriate for classification and retrieval tasks. The efficacy of the proposed approach is evaluated on three benchmark data sets: MNIST, USPS and Alphanumeric, showing an improvement in classification performance over existing kernel approaches, and comparable performance to deep learning methods, but with much reduced computational costs. Using explainable AI methods, we also demonstrate why our IDFK leads to better classification performance.Video
Citation
@inproceedings{AHMED_2022_BMVC,
author = {SARAH AHMED and Tayyaba Azim and Joseph Early and Sarvapali Ramchurn},
title = {Revisiting Deep Fisher Vectors: Using Fisher Information to Improve Object Classification},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0900.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by