對於卷積層，設置一個固定的剪枝率和/或指定閾值的濾波器剪枝方法已被廣泛用於減少卷積神經網絡（CNN）模型中所需的參數量。然而，它無法完全修剪不同層中量各異的冗餘濾波器。為了克服此缺點，我們提出了一種新的反向濾波器修剪算法，該算法使用排序好的二分圖進行基於二元搜索的分群(SBGBS）以及使用遞減的修剪率（DPR）。我們首先用二分圖K1,n表示最後一層的每個濾波器，其中有一個單點的方均根點集合和一個有n個點的權重集合，其中n是濾波器中的參數量。接下來，根據可以容忍的精度損失，使用基於SBGBS的分群方法將所有濾波器盡可能最大地劃分為多個群集。然後對於每個群集，我們保留在該群集中 具有中值方均根的二分圖所對應的濾波器，並且刪去同一群集中的其他濾波器。遵循著DPR方法，我們反向地對每一層重複上述基於SBGBS的濾波器修剪方法，直到處理完所有層。基於CIFAR10和MNIST數據集，我們將提出的濾波器修剪算法實作到VGG16、AlexNet、LeNet和ResNet上。在準確率相似的條件下，我們提供了詳細的實驗結果，證明了相較於現有的濾波器修剪方法，我們的濾波器修剪算法的在參數量和浮點數運算量減少上的優點。

Setting a fixed pruning rate and/or specified threshold for pruning filters in convolutional layers has been widely used to reduce the number of parameters required in the convolutional neural networks (CNN) model. However, it fails to fully prune redundant filters for different layers whose redundant filters vary with layers. To overcome this disadvantage, we propose a new backward filter pruning algorithm using a sorted bipartite graph- and binary search-based (SBGBS-based) clustering and decreasing pruning rate (DPR) approach. We first represent each filter of the last layer by a bipartite graph K1,n, with one root mean set and one n-weight set, where n denotes the number of weights in the filter. Next, according to the accuracy loss tolerance, an SBGBS-based clustering method is used to partition all filters into clusters as maximal as possible. Then, for each cluster, we retain the filter corresponding to the bipartite graph with the median root mean among n root means in the cluster, but we discard the other filters in the same cluster. Following the DPR approach, we repeat the above SBGBS-based filtering pruning approach to the backward layer until all layers are processed. Based on the CIFAR-10 and MNIST datasets, the proposed filter pruning algorithm has been deployed into VGG-16, AlexNet, LeNet, and ResNet. With similar accuracy, the thorough experimental results have demonstrated the substantial parameters and floating-point operations reduction merits of our filter pruning algorithm relative to the existing filter pruning methods.

[1] B. O. Ayinde and J. M. Zurada, “Building efficient convnets using redundant feature pruning,”arXiv preprint arXiv:1802.07653, Feb. 2018.
[2] V. Badrinarayanan, A. Kendall, and R. Cipolla, “SegNet: A deep convolutional encoder-decoder architecture for image segmentation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 12, pp. 24812495, Dec. 2017.
[3] C. F. Chen, G. G. Lee, V. Sritapan, and C. Y. Lin, “Deep convolutional neural network on iOS mobile devices,” IEEE International Workshop on Signal Processing Systems, pp. 130–135, Oct. 2016.
[4] B. N. Datta, ”Numerical linear algebra and applications”, Brooks/ Cole Publishing Company, pp. 315324, 1995.
[5] M. Denil, B. Shakibi, L. Dinh, M. Ranzato, and N. de Freitas, “Predicting parameters in deep learning,” International Conference on Neural Information Processing Systems, vol. 2, pp. 2148–2156, Dec. 2013.
[6] E. L. Denton, W. Zaremba, J. Bruna, Y. LeCun, and R. Fergus, “Exploiting linear structure within convolutional networks for efficient evaluation,” arXiv preprint arXiv:1404.0736, Apr. 2014.
[7] Execution code. Accessed: 26 Jan. 2019. [Online]. Available: ftp:// 140.118.175.164/Model_Compression/Codes.
[8] I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial networks,” arXiv preprint arXiv:1406.2661, Jun. 2014.
[9] J. Gou, B. Yu, S. J. Maybank, and D. Tao, “Knowledge distillation: A survey,”arXiv preprint arXiv:2006.05525, Jun. 2021.
[10] S. Han, H. Mao, and W. J. Dally, “Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding,” arXiv preprint arXiv:1510.00149, Oct. 2015.
[11] S. Han, J. Pool, J. Tran, and W. J. Dally, “Learning both weights and connections for efficient neural network,” arXiv preprint arXiv:1506.02626, Jun. 2015.
[12] K. He, G. Gkioxari, P. Dollar, and R. Girshick, “Mask r-cnn,” IEEE International Conference on Computer Vision, pp. 29612969, 2017.
[13] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” arXiv preprint arXiv:1512.03385, Dec. 2015.
[14] Y. He, G. Kang, X. Dong, Y. Fu, and Y. Yang, “Soft filter pruning for accelerating deep convolutional neural networks,”arXiv preprint arXiv:1808.06866, Aug. 2018.
[15] Y. He, J. Lin, Z. Liu, H. Wang, L. J. Li, and S. Han, “Amc: Automl for model compression and acceleration on mobile devices,” European Conference on Computer Vision, pp. 784800, 2018.
[16] Y. He, P. Liu, Z. Wang, Z. Hu, and Y. Yang, “Filter pruning via geometric median for deep convolutional neural networks acceleration,” IEEE Conference on Computer Vision and Pattern Recognition, pp. 43404349, 2019.
[17] Y. He, X. Zhang, and J. Sun, “Channel pruning for accelerating very deep neural networks,”IEEE International Conference on Computer Vision, pp. 13981402, Oct. 2017.
[18] G. E. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” NIPS Deep Learning and Representation Learning Workshop, pp. 19, 2015.
[19] A. G. Howard, M. Zhu, B. Chen, and D. Kalenichenko, “MobileNets: Efficient convolutional neural networks for mobile vision applications,” arXiv preprint arXiv:1704.04861, Apr. 2017.
[20] A. Krizhevsky and G. Hinton., “Learning multiple layers of features from tiny images,” Technical Report, Computer Science Department, University of Toronto, 2009.
[21] A. Krizhevsky, I. Sutskever, and G. Hinton, “ImageNet classification with deep convolutional neural networks,” Conference on Neural Information Processing Systems, pp. 1097–1105, 2012.
[22] V. Lebedev, Y. Ganin, M. Rakhuba, I. Oseledets, and V. Lempitsky, “Speeding-up convolutional neural networks using fine-tuned cp-decomposition,” arXiv preprint arXiv:1412.6553, Apr. 2015.
[23] Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner “Gradient-based learning applied to document recognition,” Proceedings of the IEEE, vol.86, no.11, pp. 22782324, Nov. 1998.
[24] H. Li, A. Kadav, I. Durdanovic, H. Samet, and H. P. Graf, “Pruning filters for efficient convnets,”arXiv preprint arXiv:1608.08710, Mar. 2017.
[25] S. Lin, R. Ji, Y. Li, C. Deng, and X. Li, “Toward compact convnets via structure-sparsity regularized filter pruning,” IEEE Transactions on Neural Networks and Learning Systems, vol. 31, no. 2, pp. 574588, Feb. 2020.
[26] J. Luo, J. Wu, and W. Lin, “ThiNet: A filter level pruning method for deep neural network compression,” arXiv preprint arXiv:1707.06342, Jul. 2017.
[27] P. Molchanov, S. Tyree, T. Karras, T. Aila, and J. Kautz, “Pruning convolutional neural networks for resource efficient inference,” arXiv preprint arXiv:1611.06440, Jun. 2017.
[28] O. Ronneberger, P. Fischer, and T. Brox, “Unet: Convolutional networks for biomedical image segmentation,” International Conference On Medical Image Computing & Computer Assisted Intervention, vol. 9351, pp. 234–241, 2015.
[29] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,”arXiv preprint arXiv:1409.1556, Sep. 2014.
[30] S. Swaminathan, D. Garg, R. Kannan, and F. Andres. ”Sparse low rank factorization for deep neural network compression,” Neurocomputing, vol. 398, pp. 185196, Jul. 2020.
[31] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, ”Going deeper with convolutions,” arXiv preprint arXiv:1409.4842, Sep. 2014.
[32] A. Torfi, R. A. Shirvani, S. Soleymani, and N. M. Nasrabadi, ” Attention-based guided structured sparsity of deep neural networks,” arXiv preprint arXiv:1802.09902, Jul. 2018.
[33] K. Wang, Z. Liu, Y. Lin, J. Lin, and S. Han, ”Haq: Hardware-aware automated quantization with mixed precision,” IEEE Conference on Computer Vision and Pattern Recognition, pp. 86128620, 2019.
[34] W. Wen, C. Wu, Y. Wang, Y. Chen, and H. Li, ”Learning structured sparsity in deep neural networks,” In Advances in Neural Information Processing Systems, pp. 20742082, 2016.
[35] J. Yoon, and S. J. Hwang, ”Combined group and exclusive sparsity for deep neural networks,” IET Image Processing, vol. 13, no. 5, pp. 779784, May. 2019.
[36] M. Yuan and Y. Lin, ”Model selection and estimation in regression with grouped variables,” J. Roy. Statist. Soc., B (Statist. Methodol.), vol. 68, no. 1, pp. 49–67, Dec. 2005.