卷積神經網路的模組壓縮研究｜國立臺灣科技大學博碩士論文系統

簡易檢索 / 詳目顯示

回結果列表

研究生：	紀玉敏 Yu-Min Chi
論文名稱：	卷積神經網路的模組壓縮研究 The Study of Model Compression for Convolutional Neural Networks
指導教授：	鍾國亮 Kuo-Liang Chung
口試委員:	顏嗣鈞 Hsu-Chun Yen 蔡文祥 Wen-Hsiang Tsai 花凱龍 Kai-Lung Hua
學位類別：	碩士 Master
系所名稱：	電資學院 - 資訊工程系 Department of Computer Science and Information Engineering
論文出版年：	2021
畢業學年度：	109
語文別：	中文
論文頁數：	34
中文關鍵詞：	模組壓縮、剪枝、卷積層、全連接層、準確率
外文關鍵詞：	module compression, pruning, convolutional layer, fully connected layer, accuracy
相關次數：	點閱：215 下載：0
分享至:	分享至facebook 分享至twitter

查詢本校圖書館目錄查詢臺灣博碩士論文知識加值系統勘誤回報

近年來深度學習盛行，許多領域都希望能藉由深度學習的技術達到更好的效果，其中包含醫療、先進駕駛輔助系統等領域。在產品中都離不開嵌入式系統的應用，也變成了不可或缺的搭配。嵌入式系統不同於個人電腦，在商業考量下為了節省成本與硬體體積，記憶體容量與運算能力有其限制，因此需要運用模組壓縮（model compression）的技術，適當的將卷積神經網路(Convolutional Neural Networks)精簡化，減少計算量與容量大小。
本論文研究的主旨，在不影響準確度的前提下，對卷積神經網路的卷積層（convolution layer）與全連接層（fully connected layer）進行最大幅度的剪枝(pruning)。我們將以LeNet為例，詳細說明我們提出的方法所得到的剪枝效果，以及實際記憶體減少的效果。

Deep Learning has significantly prevailed in recent years and achieved state-of-the-art results in many research fields, including medical care, advanced driver assistance systems, etc. In practical applications, it is essential to bring deep learning techniques into embedded systems. However, existing deep learning models are time-consuming and memory-expensive. From the above-mentioned difficulties, model compression is a promising approach to reduce the number of learned weights, leading to memory- and time-saving effects.
The main contribution of this thesis is to propose a multilayer approach to maximizing the number of pruned learned weights for convolutional neural networks. We take the LeNet as an example to justify the merits of our proposed approach.

中文摘要..................................................................................................................................... i
abstract ....................................................................................................................................... ii
致謝.......................................................................................................................................... iii
目錄........................................................................................................................................... iv
圖目錄........................................................................................................................................ v
表目錄....................................................................................................................................... vi
第一章 緒論........................................................................................................................ 1
1.1 研究動機.................................................................................................................... 1
1.2 過去相關研究............................................................................................................ 1
1.3 論文貢獻.................................................................................................................... 3
第二章 環境介紹與架設.................................................................................................... 4
2.1 實驗環境.................................................................................................................... 4
2.2 顯示卡驅動程式安裝................................................................................................ 4
2.3 環境建置-Docker ...................................................................................................... 7
第三章 方法與介紹.......................................................................................................... 10
3.1 網路架構例子：LeNet............................................................................................ 10
3.2 方法介紹.................................................................................................................. 11
第四章 實驗結果.............................................................................................................. 15
4.1 資料集...................................................................................................................... 15
4.2 評估指標：執行FLOP與加權係數使用量 .......................................................... 16
4.3 實驗結果.................................................................................................................. 20
第五章 結論與未來展望.................................................................................................. 24
5.1 結論.......................................................................................................................... 24
5.2 未來展望.................................................................................................................. 24
第六章 參考文獻.............................................................................................................. 25
                                

[1] Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, and Dmitry Kalenichenko. Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference. CVPR, 2704-2713, 2018.
[2] Michael Zhu and Suyog Gupta. To prune, or not to prune: exploring the efficacy of pruning for model compression. arXiv:1710.01878, 2017.
[3] Yihui He, Xiangyu Zhang, and Jian Sun. Channel Pruning for Accelerating Very Deep Neural Networks. ICCV, 2017
[4] Song Han, Huizi Mao, and William J. Dally. Deep compression: Compressing DNNs with pruning, trained quantization and huffman coding. arXiv:1510.00149, 2016
[5] Yann LeCun, Leon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-2324, November 1998.
[6] Hao Li, Asim Kadav, Igor Durdanovic, Hanan Samet and Hans Peter Graf. Pruning Filters for Efficient ConvNets. arXiv:1608.08710, 2017
[7] The MNIST dataset of handwritten digits. http://yann.lecun.com/exdb/MNIST/, 2019
[8] Nvidia. https://developer.nvidia.com/cuda-toolkit, 2019
[9] dockerhub. https://hub.docker.com/, 2019
[10] The CIFAR-10 dataset. https://www.cs.toronto.edu/~kriz/cifar.html, 2019

簡易檢索 / 詳目顯示

相關論文