類神經網絡模型的壓縮近年來受到了越來越多的關注。為了減緩卷積核冗餘的問題，這篇論文提出了新的卷積結構：共享核卷積和加權共享核卷積，後者在前者的基礎上增加了額外的共享權重來滿足不同輸入通道訊號間的差異性。傳統的卷積中，每個輸入通道都有各自的卷積核去進行卷積，這樣可能會導致卷積核的冗餘。考慮到此情況，本文提出的兩個架構均將不同的輸入通道訊號進行分組卷積核共享，同一組中來自不同的通道的訊號會共享同一個卷積核。這樣以來，卷積核的數目會相應減少，從而可降低模型參數的大小和提高預測的速度。另外，我們將加權共享卷積核結構和深度可分離卷積結構結合產生一個高度壓縮的網路架構。我們分別在ImageNet分類任務，CIFAR-100，Caltech-256數據集上面進行了大量的實驗，并與最先進的壓縮方法進行了比較，結果顯示我們的架構在參數量和計算代價的壓縮方面均具有廣泛的效益。

Increasing focus has been put on pursuing computation efficient convolutional neural network (CNN) models. To lessen the redundancy of convolutional kernels, this paper proposes two new convolutional structures, i.e., kernel sharing convolution (KSC) and weighted kernel sharing convolution (WKSC), where an extra weighting is imposed for each input in WKSC to manifest the diversity of input channels. Inspired by the fact that in traditional convolution, each input channel has its respective kernel to convolute with, which may lead to redundant kernels, both of the proposed schemes gather the inputs using the same kernel together, so the inputs in each group can share the same convolutional kernel. As a consequence, the number of kernels can be greatly reduced, leading to a reduction of model parameters and the speedup of inference. Moreover, WKSC is also combined with depthwise separable convolutions, resulting in a highly compressed architecture. Extensive experiments on CIFAR-100, Caltech-256 and ImageNet classification demonstrate the effectiveness of the new approach in both computation cost and the parameters required compared with the state-of-the-art works.

[1] T. Zhang, G. J. Qi, B. Xiao, and J. Wang, \Interleaved group convolutions for
deep neural networks," in ICCV, 2017.
[2] K. Simonyan and A. Zisserman, \Very deep convolutional networks for large-
scale image recognition," in ICLR, 2015.
[3] S. Xie, R. Girshick, P. Dollar, Z. Tu, and K. He, \Aggregated residual trans-
formations for deep neural networks," in CVPR, pp. 5987{5995, 2017.
[4] K. He, X. Zhang, S. Ren, and J. Sun, \Deep residual learning for image recog-
nition," in CVPR, pp. 770{778, 2016.
[5] A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand,
M. Andreetto, and H. Adam, \Mobilenets: Ecient convolutional neural net-
works for mobile vision applications," arXiv preprint arXiv:1704.04861, 2017.
[6] F. N. Iandola, S. Han, M. W. Moskewicz, K. Ashraf, W. J. Dally, and
K. Keutzer, \Squeezenet: Alexnet-level accuracy with 50x fewer parameters
and< 0.5 mb model size," arXiv preprint arXiv:1602.07360, 2016.
[7] A. Krizhevsky, I. Sutskever, and G. E. Hinton, \Imagenet classication with
deep convolutional neural networks," in NIPS, pp. 1097{1105, 2012.
[8] S. Srinivas and R. V. Babu, \Data-free parameter pruning for deep neural
networks," in BMVC, 2015.
[9] H. Li, A. Kadav, I. Durdanovic, H. Samet, and H. P. Graf, \Pruning lters for
ecient convnets," arXiv preprint arXiv:1608.08710, 2016.
[10] S. Han, J. Pool, J. Tran, and W. Dally, \Learning both weights and connections
for ecient neural network," in NIPS, pp. 1135{1143, 2015.
[11] S. Han, H. Mao, and W. J. Dally, \Deep compression: Compressing deep neural
networks with pruning, trained quantization and human coding," in ICLR,
2016.
[12] Y. Gong, L. Liu, M. Yang, and L. Bourdev, \Compressing deep convolutional
networks using vector quantization," arXiv preprint arXiv:1412.6115, 2014.
[13] J. Wu, C. Leng, Y. Wang, Q. Hu, and J. Cheng, \Quantized convolutional
neural networks for mobile devices," in CVPR, pp. 4820{4828, 2016.
[14] V. Vanhoucke, A. Senior, and M. Z. Mao, \Improving the speed of neural
networks on cpus," in Proc. Deep Learning and Unsupervised Feature Learning
NIPS Workshop, vol. 1, p. 4, Citeseer, 2011.
[15] G. Hinton, O. Vinyals, and J. Dean, \Distilling the knowledge in a neural net-
work," in NIPS Deep Learning and Representation Learning Workshop, 2014.
[16] T. Chen, I. Goodfellow, and J. Shlens, \Net2net: Accelerating learning via
knowledge transfer," in ICLR, 2016.
[17] A. Romero, N. Ballas, S. E. Kahou, A. Chassang, C. Gatta, and Y. Bengio,
\Fitnets: Hints for thin deep nets," in ICLR, 2015.
[18] M. Jaderberg, A. Vedaldi, and A. Zisserman, \Speeding up convolutional neural
networks with low rank expansions," in BMVC, 2014.
[19] C. Tai, T. Xiao, Y. Zhang, X. Wang, et al., \Convolutional neural networks
with low-rank regularization," in ICLR, 2016.
[20] R. Rigamonti, A. Sironi, V. Lepetit, and P. Fua, \Learning separable lters,"
in CVPR, pp. 2754{2761, 2013.
[21] W. Shang, K. Sohn, D. Almeida, and H. Lee, \Understanding and improving
convolutional neural networks via concatenated rectied linear units," in ICML,
pp. 2217{2225, 2016.
[22] L. Sifre and P. Mallat, Rigid-motion scattering for image classication. PhD
thesis, Citeseer, 2014.
[23] F. Chollet, \Xception: Deep learning with depthwise separable convolutions,"
in CVPR, 2017.
[24] X. Zhang, X. Zhou, M. Lin, and J. Sun, \Shuenet: An extremely ecient con-
volutional neural network for mobile devices," arXiv preprint arXiv:1707.01083,
2017.
[25] D. E. Rumelhart, G. E. Hinton, and R. J. Williams, \Learning representations
by back-propagating errors," nature, vol. 323, no. 6088, p. 533, 1986.
[26] A. Krizhevsky and G. Hinton, \Learning multiple layers of features from tiny
images," Technical report, 2009.
[27] K. He, X. Zhang, S. Ren, and J. Sun, \Delving deep into rectiers: Surpassing
human-level performance on imagenet classication," in ICCV, pp. 1026{1034,
2015.
[28] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, \Imagenet: A
large-scale hierarchical image database," in CVPR, pp. 248{255, 2009.
[29] G. Grin, A. Holub, and P. Perona, \Caltech-256 object category dataset,"
Technical Report 7694, 2007.
[30] N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov,
\Dropout: A simple way to prevent neural networks from overtting," The
Journal of Machine Learning Research, vol. 15, no. 1, pp. 1929{1958, 2014.