CICC: Channel Pruning via the Concentration of Information and Contributions of Channels


Yihao Chen (Zhejiang University),* Zhishan Li (Zhejiang University), Yingqing Yang (Zhejiang University), Lei Xie (Zhejiang University), Yong Liu (Zhejiang University), longhua ma (NingboTech University), Shanqi Liu (Zhejiang University), Guanzhong Tian (Ningbo Research Institute, Zhejiang University)
The 33rd British Machine Vision Conference

Abstract

Channel pruning provides a promising prospect to compress and accelerate convolutional neural networks. However, existing pruning methods neglect the compression sensitivity of different layers and adjust the pruning rate through engineering tuning. To address this problem, we propose to assign the layer-wise pruning ratio via the concentration of information for the convolutional layers. Specifically, we introduce the rank and entropy of convolutional layers as indicators of the redundancy and amount of information, respectively. After that, we define a fusion function, which compromises these two indicators, to represent the concentration of information for the convolutional layers. Additionally, for pruning filters with interpretability and intuition, we propose to evaluate the importance of channels by leveraging Shapley values, which fairly distribute the average marginal contributions among them. Extensive experiments on various architectures and benchmarks demonstrate the promising performance of our proposed method (CICC). For example, CICC achieves an accuracy increase of 0.21% with FLOPs and parameters reductions of 45.5% and 40.3% on CIFAR-10. Besides, CICC obtains Top-1/Top-5 accuracy of 0.43%/0.11% with FLOPs and parameters reductions of 41.6% and 35.0% on ImageNet. It is worth noting that our method can still achieve excellent accuracy under high acceleration rates for pruning ResNet-110 on CIFAR-10.

Video



Citation

@inproceedings{Chen_2022_BMVC,
author    = {Yihao Chen and Zhishan Li and Yingqing Yang and Lei Xie and Yong Liu and longhua ma and Shanqi Liu and Guanzhong Tian},
title     = {CICC: Channel Pruning via the Concentration of Information and Contributions of Channels},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year      = {2022},
url       = {https://bmvc2022.mpi-inf.mpg.de/0243.pdf}
}


Copyright © 2022 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection