隨著深度神經網路(DNN)模型辨識準確率的提高，人工智慧（AI）的應用變得更加豐富多元。AI更被大量應用到物聯網（IoT）系統中，其中邊緣運算技術成為了常見的解決方案，而雲端至邊緣設備上的傳輸延遲將成為系統效能的主要關鍵因素，因此為了解決向邊緣設備部署DNN模型時的傳輸延遲，我們提出了基於深度神經網路模型的深度編碼（MDE），並針對量化壓縮後的模型之權重分佈進行探討，並基於我們的觀察到的分佈特性，進一步在量化的DNN模型上進行傳輸優化。在實驗中，MDE針對三個基準模型AlexNet，GoogleNet和ResNet-50這三種代表性模型進行評估。根據實驗結果，MDE和原始模型進行傳輸時相比可以達到的平均壓縮比為88.72％，傳輸時的位元填充平均節省93.76%。

Along with the development of DNN technique, Artificial Intelligence (AI) application achieves more complex due to the improvement in the capability of emerging DNN model. With edge computing architecture, it offers a general solution to build up AI into Internet of Things (IoT). To solve the transmission latency when deploying Deep Neural Network (DNN) model to the edge device through the USB interface, a Model-based Deep Encoding (MDE) is proposed as a general pipeline to optimize the transmission efficiency by exploiting the distribution of model weights on the quantized DNN model. In the experiment, MDE is performed by three representative model quantization techniques over three benchmarked models AlexNet, GoogleNet and ResNet-50. The experimental results indicate that MDE can achieve the average compression ratio of 88.72% and the average stuffing bit saving of 93.76%

[1]M. Mohammadi, A. Al-Fuqaha, S. Sorour, and M. Guizani, ”Deep learning for IoT big data and streaming analytics: A survey,” IEEE Commun. Surveys Tuts., vol. 20, no. 4, pp. 2923-2960, 4th Quart., 2018.
[2]H. Li, K. Ota, and M. Dong, ’”Learning IoT in Edge: Deep Learning for the Internet of Things with Edge Computing,” in IEEE Network, vol. 32, no. 1, pp. 96-101, 2018.
[3]J. Li, X. Mei, D. Prokhorov and D. Tao, ”Deep Neural Network for Structural Prediction and Lane Detection in Traffic Scene,” in IEEE Transactions on Neural Networks and Learning Systems, vol. 28, no. 3, pp. 690-703, March 2017.
[4]JW. Hu, Q. Zhuo, C. Zhang and J. Li, ”Fast Branch Convolutional Neural Network for Traffic Sign Recognition,” in IEEE Intelligent Transportation Systems Magazine, vol. 9, no. 3, pp. 114-126, Fall 2017.
[5]X. Du, M. El-Khamy, J. Lee and L. Davis, ”Fused DNN: A Deep Neural Network Fusion Approach to Fast and Robust Pedestrian Detection,” 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), Santa Rosa, CA, 2017, pp. 953-961.
[6]V. C. Liang et al., ”Mercury: Metro density prediction with recurrent neural network on streaming CDR data,” in Proc. IEEE 32nd Int. Conf.Data Eng. (ICDE), Helsinki, Finland: IEEE, 2016, pp. 1374-1377.
[7]H. Wei, Y. Xiao, R. Li and X. Liu, ”Crowd Abnormal Detection Using Two-Stream Fully Convolutional Neural Networks,” 2018 10th International Conference on Measuring Technology and Mechatronics Automation (ICMTMA), Changsha, 2018, pp. 332-336.
[8]E. Ahmed, M. Jones and T. K. Marks, ”An improved deep learning architecture for person re-identification,” 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, 2015, pp. 3908-3916.
[9]Google Inc., https://cloud.google.com/. [Online]. Available: https:// cloud.google.com/edge-tpu/.[Accessed: 14- Jun-2018].
[10]”Movidius announces deep learning accelerator and fathom software framework.’ https://developer.movidius.com/.
[11]S. B. Calo, M. Touna, D. C. Verma and A. Cullen, ”Edge computing architecture for applying AI to IoT,” 2017 IEEE International Conference on Big Data (Big Data), Boston, MA. 2017, pp. 3012-3016.
[12]Y. He, X. Zhang, and J. Sun, ”Channel Pruning for Accelerating Very Deep Neural Networks,” in Proc. Int. Conf. Comput. Vis. (ICCV), vol. 2, no. 6, 2017, pp. 1-7.
[13]D. D. Lin, S. S. Talathi,and V.S.Annapureddy, ”Fixed point quantization of deep convolutional networks,” arXiv preprint arXiv:1511.06393,Nov. 2015.
[14] S. Han, H. Mao, and W. J. Dally, ”Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding,”International Conference on Learning Representations 2016.
[15]J. Zhang, W. Xie, F. Yang, and Q. Bi, ”Mobile edge computing and field trial results for 5G low latency scenario,” China Commun., vol. 13, no. 2, pp. 174182, 2016.
[16]W. Hu et al., ”Quantifying the impact of edge computing on mobile applications,” in Proc. 7th ACM SIGOPS AsiaPacific Workshop Syst.,, Hong Kong, 2016, pp. 18.
[17]Y. Choi, M. EI-Khamy, and J. Lee, ”Universal deep neural network compression,”arXiv preprint arXiv:1802.02271,, 2018.
[18]C. Pal, S. Pankaj, W. Akram, A. Acharyya and D. Biswas, ”Modified Huffman based compression methodology for Deep Neural Network Implementation on Resource Constrained Mobile Platforms,2018 IEEE International Symposium on Circuits and Systems (ISCAS),, Florence, Italy, 2018, pp. 1-5.
[19]S. Ge, Z. Luo, S. Zhao, X. Jin and X. Zhang, ”Compressing deep neural networks for efficient visual inference, 2017 IEEE International Conference on Multimedia and Expo (ICME), Hong Kong, 2017, pp. 667-672.
[20]Y. Pai, F. Cheng, S. Lu and S. Ruan, ”Sub-Trees Modification of Huffman Coding for Stuffing Bits Reduction and Efficient NRZI Data Transmission,” in IEEE Transactions on Broadcasting, vol. 58, no. 2, pp. 221-227, June 2012.
[21]Dongqing Zhang, Jiaolong Yang, Dongqiangzi Ye, and Gang Hua, LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks, 15th European Conference., Munich, Germany, September 8-14, 2018, Proceedings
[22]C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, ”Going deeper with convolutions,” arXiv:1409.4842, 2014.
[23]K. He, X. Zhang, S. Ren, and J. Sun, ”Deep residual learning for image recognition,”arXiv:1512.03385, 2015.
[24]A. G. Howard et al. (2017). ”MobileNets: Efficient convolutional neural networks for mobile vision applications.” [Online]. Available:https://arxiv.org/abs/1704.04861
[25]Xiangyu Zhang, Xinyu Zhou, Mengxiao Lin, and Jian Sun, ”Shufflenet: An extremely efficient convolutional neural network for mobile devices,” arXiv preprint arXiv:1707.01083, 2017.
[26]Han, S.; Pool, J.; Tran, J.; and Dally, W., 2015. Learning both weights and connections for efficient neural networks. In Advances in Neural Information Processing Systems.
[27]J.-H. Luo, J. Wu, and W. Lin. (2017). ”ThiNet: A filter level pruning method for deep neural network compression.”[Online]. Available:https://arxiv.org/abs/1707.06342.
[28]M. Denil, B. Shakibi, L. Dinh, and N. De Freitas, ”Predicting parameters in deep learning,” in Proc. 27th Annu. Conf. Neural Inf. Process. Syst., 2013, pp. 2148-2156.
[29]E. L. Denton, W. Zaremba, J. Bruna, Y. LeCun, and R. Fergus, ”Exploiting linear structure within convolutional networks for efficient evaluation,” in in Advances in Neural Information Processing Systems 27, Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K. Weinberger, Eds. Red Hook, NY, USA: Curran Associates, 2014, pp. 1269-1277.
[30]X. Yu, T. Liu, X. Wang and D. Tao, ”On Compressing Deep Models by Low Rank and Sparse Decomposition, ” 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, 2017, pp. 67-76.
[31]X. Zhang, J. Zou, K. He and J. Sun, ”Accelerating Very Deep Convolutional Networks for Classification and Detection, ” in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 38, no. 10, pp. 1943-1955, 1 Oct. 2016.
[32]B. Liu, M. Wang, H. Foroosh, M. Tappen, and M. Pensky, ”Sparse convolutional neural networks, ” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2015, pp. 806814.
[33]V. Lebedev and V. Lempitsky, ”Fast convnets using group-wise brain damage, ”In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2554-2564, 2016.
[34]W. Wen, C. Wu, Y. Wang, Y. Chen, and H. Li, ”Learning structured sparsity in deep neural networks, ”in Proc. NIPS, 2016, pp. 20742082. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 8.
[35]S. Zhou, et.al., DoReFa-Net: training low bitwidth convolutional neural networks with low bitwidth gradients, arXiv:1606.06160, 2016.
[36]I. Hubara, M. Courbariaux, D. Soudry, R. El-Yaniv, and Y. Bengio, ”Quantized neural networks: Training neural networks with low precision weights and activations,” arXiv preprint arXiv:1609.07061, 2016.
[37]D. Miyashita et al., ”Convolutional neural networks using logarithmic data representation,” arXiv preprint arXiv:1603.01025, 2016.
[38]A. Zhou, A. Yao, Y. Guo, L. Xu, and Y. Chen, ”Incremental network quantization: Towards lossless CNNs with low-precision weights,” in Proc. ICLR, 2017.
[39]Z. Cai, X. He, J. Sun, and N. Vasconcelos, ”Deep learning with low precision by halfwave Gaussian quantization,” in Proc. ICLR, 2017.
[40]K. A. S. Immink, Coding Techniques for Digital Recorders. Englewood Cliffs, NJ: PrenticeHall, 1991.
[41]J. Deng, W. Dong, R. Socher, L. Li, Kai Li and Li Fei-Fei, ”ImageNet: A large-scale hierarchical image database,” in Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, pages 248-255, June 2009