Training Binarized Neural Networks the Easy Way


Alasdair J Paren (University of Oxford),* Rudra Poudel (Toshiba Research)
The 33rd British Machine Vision Conference

Abstract

In this work we present a simple but effective method for training Binarized Neural Networks (BNNs). Specifically, models where the majority of both weights and activations are constrained to the set {-1,1}. These models offer significant improvements in memory efficiency, energy usage and inference speed over their floating point counterparts. Our approach to training BNN splits the task into two phases. In the first phase a model with binary activations and floating point weights is trained. In the second, a concave regulariser is added to encourage the weights to become binary. This work is orthogonal to improvements of BNN architectures, and offers an alternative optimisation scheme for these models. Our method doesn't require an auxiliary set of weights during training and can be easily applied to any existing architectures. Finally, we achieve a new state of the art training a BNN on the ImageNet data set.

Video



Citation

@inproceedings{Paren_2022_BMVC,
author    = {Alasdair J Paren and Rudra Poudel},
title     = {Training Binarized Neural Networks the Easy Way},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year      = {2022},
url       = {https://bmvc2022.mpi-inf.mpg.de/0035.pdf}
}


Copyright © 2022 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection