EAPruning: Evolutionary Pruning for Vision Transformers and CNNs
Qingyuan Li (Meituan), Bo Zhang (Meituan),* Xiangxiang Chu (Meituan)
The 33rd British Machine Vision Conference
Abstract
Structured pruning greatly eases the deployment of large neural networks in resource-constrained environments. However, current methods either involve strong domain expertise, require extra hyperparameter tuning, or are restricted only to a specific type of network, which prevents pervasive industrial applications. In this paper, we undertake a simple and universal approach that can be easily applied to both vision transformers and convolutional neural networks. Specifically, we consider pruning as an evolution process of sub-network structures that inherits weight through reconstruction techniques. We achieve 50% FLOPS reduction for ResNet50 and MobileNetV1, leading to 1.37× and 1.34× speedup respectively. For DeiT-base, we reach 40% FLOPs reduction and 1.4× speedup. Our code will be made available.Video
Citation
@inproceedings{Li_2022_BMVC,
author = {Qingyuan Li and Bo Zhang and Xiangxiang Chu},
title = {EAPruning: Evolutionary Pruning for Vision Transformers and CNNs},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0258.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by