Improving Gradient Paths for Binary Convolutional Neural Networks
Baozhou Zhu (Tudelft),* Peter Hofstee (IBM), Jinho Lee (Yonsei University), Zaid Alars (TUDelft)
The 33rd British Machine Vision Conference
Abstract
Our starting point is a closer investigation of Bi-Real ResNet. In our investigation of Bi-Real ResNet, we believe that the superiority of Bi-Real ResNet over binary ResNet requires a different explanation rather than being attributed to the representational capability. Instead, we study the gradient paths rather than representational capability for BCNNs. To our best knowledge, this is the first work to consider gradient paths for BCNNs. Improving gradient paths is realized by reducing the smallest number of operations to compute gradient backpropagation for a gradient path. Regarding Bi-Real ResNet and BinaryDenseNet, the error of BCNNs decreases when the increased shortcuts improve gradient paths. In addition, we design two architectures by improving gradient paths for BCNNs: 1. Improving Gradient Paths for binary ResNet (IGP-ResNet), and 2. Improving Gradient Paths for binary DenseNet (IGP-DenseNet). Specifically, the Top-1 error of proposed IGP-ResNet37(41) and IGP-DenseNet51(53) on ImageNet gets lower than Bi-Real ResNet18(64) and BinaryDenseNet51(32) by 3.29% and 1.41%, respectively, with almost the same computational complexity.Video
Citation
@inproceedings{Zhu_2022_BMVC,
author = {Baozhou Zhu and Peter Hofstee and Jinho Lee and Zaid Alars},
title = {Improving Gradient Paths for Binary Convolutional Neural Networks},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0281.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by