CICC: Channel Pruning via the Concentration of Information and Contributions of Channels
Yihao Chen (Zhejiang University),* Zhishan Li (Zhejiang University), Yingqing Yang (Zhejiang University), Lei Xie (Zhejiang University), Yong Liu (Zhejiang University), longhua ma (NingboTech University), Shanqi Liu (Zhejiang University), Guanzhong Tian (Ningbo Research Institute, Zhejiang University)
The 33rd British Machine Vision Conference
Abstract
Channel pruning provides a promising prospect to compress and accelerate convolutional neural networks. However, existing pruning methods neglect the compression sensitivity of different layers and adjust the pruning rate through engineering tuning. To address this problem, we propose to assign the layer-wise pruning ratio via the concentration of information for the convolutional layers. Specifically, we introduce the rank and entropy of convolutional layers as indicators of the redundancy and amount of information, respectively. After that, we define a fusion function, which compromises these two indicators, to represent the concentration of information for the convolutional layers. Additionally, for pruning filters with interpretability and intuition, we propose to evaluate the importance of channels by leveraging Shapley values, which fairly distribute the average marginal contributions among them. Extensive experiments on various architectures and benchmarks demonstrate the promising performance of our proposed method (CICC). For example, CICC achieves an accuracy increase of 0.21% with FLOPs and parameters reductions of 45.5% and 40.3% on CIFAR-10. Besides, CICC obtains Top-1/Top-5 accuracy of 0.43%/0.11% with FLOPs and parameters reductions of 41.6% and 35.0% on ImageNet. It is worth noting that our method can still achieve excellent accuracy under high acceleration rates for pruning ResNet-110 on CIFAR-10.Video
Citation
@inproceedings{Chen_2022_BMVC,
author = {Yihao Chen and Zhishan Li and Yingqing Yang and Lei Xie and Yong Liu and longhua ma and Shanqi Liu and Guanzhong Tian},
title = {CICC: Channel Pruning via the Concentration of Information and Contributions of Channels},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0243.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by