Training Binarized Neural Networks the Easy Way
Alasdair J Paren (University of Oxford),* Rudra Poudel (Toshiba Research)
The 33rd British Machine Vision Conference
Abstract
In this work we present a simple but effective method for training Binarized Neural Networks (BNNs). Specifically, models where the majority of both weights and activations are constrained to the set {-1,1}. These models offer significant improvements in memory efficiency, energy usage and inference speed over their floating point counterparts. Our approach to training BNN splits the task into two phases. In the first phase a model with binary activations and floating point weights is trained. In the second, a concave regulariser is added to encourage the weights to become binary. This work is orthogonal to improvements of BNN architectures, and offers an alternative optimisation scheme for these models. Our method doesn't require an auxiliary set of weights during training and can be easily applied to any existing architectures. Finally, we achieve a new state of the art training a BNN on the ImageNet data set.Video
Citation
@inproceedings{Paren_2022_BMVC,
author = {Alasdair J Paren and Rudra Poudel},
title = {Training Binarized Neural Networks the Easy Way},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0035.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by