簡易檢索 / 詳目顯示

研究生: 鄧友豪
You-Hao Deng
論文名稱: 基於遷移式學習之深度學習影像分類參數微調系統
Fine-Tuning Deep Learning Image Classification Parameter based on Transfer Learning
指導教授: 陳建中
Jiann-Jone Chen
口試委員: 杭學鳴
吳怡樂
Yi-Leh Wu
郭天穎
鍾國亮
Kuo-Liang Chung
陳建中
Jiann-Jone Chen
學位類別: 碩士
Master
系所名稱: 電資學院 - 電機工程系
Department of Electrical Engineering
論文出版年: 2018
畢業學年度: 106
語文別: 中文
論文頁數: 63
中文關鍵詞: 深度學習遷移式學習影像分類卷積神經網路系統參數微調
外文關鍵詞: Deep Learning, Transfer Learning, Image Classification, Convolutional Neural Network, System Parameter Fine-Tuning
相關次數: 點閱:340下載:7
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報
  • 隨著應用深度學習法的許多系統成效卓著,如何能夠快速地把個別的問題轉換成應用到深度學習架構來尋求專業的運作,是目前相當熱門的研究項目之一。一般而言,要應用深度學習架構從頭開始訓練一個新的任務,需要很長的時間以及精神來訓練、調整參數,因為要精準地找到方向去調整訓練參數仍必須要大量的實驗累積經驗。但隨著遷移式學習(Transfer Learning)方法的崛起,這個問題得到了解決方案,當使用者有一個經過巨量的資料庫訓練過的模型權重,即所謂的預訓練模型,就可以不需要重複浪費時間及運算資源。例如在運用卷積神經網路(Convolutional Neural Network, CNN)架構來分類影像資料庫時,通常以分1000類的120萬張訓練圖片的權重(VGG16)為起始設定再進行微調。然而,針對不同的資料庫還是需要花不少的時間去實驗找出最佳的參數設置。本論文提出一套參數微調系統架構,幫助使用者能夠找到調整參數的方向,避免不斷地在測試錯誤的參數,而不知道要如何修改。此參數微調系統架構可以有效的帶領使用者從拿到新的影像資料庫開始,從事前預備(切割和分析資料庫),接著到訓練前的參數調整階段,參數包括了期次數、學習率、批次大小等等,最後是結果觀察階段,提供了許多分析的經驗觀點,幫助使用者能夠更清楚的了解調整不同參數的影響。經過實驗證明,本論文之參數微調系統架構能夠精確的調整參數使影像分類準確度達到理想結果。


    Deep learning (DL) methods have been widely applied to different expertise domain and do a good performance. How to transform one big job for solving by the DL method become popular, and one can find many research papers from literatures. In general, to solve a big problem successfully by DL methods from scratch is very time-consuming and tedious, as it needs to take care of training, parameter adjustment, and find a good way to improve the performance. Fortunately, one can adopt a transfer learning approach to eliminate the tedious procedure. System parameters adjusted from a formal learning process with a benchmark database can be took as pre-trained weight for a new database with different scales. For example, it uses weight adjusted from a Convolutional Neural Network (CNN) that deals with one database with one million of images with 1000 categories (VGG-16) as initial weight for other databases. However, it still needs experiments and adjustments to yield the final system parameters. In this research, we proposed a parameter-fine-tune procedure that can help users to bypass the try-and-error process and to achieve the best system configuration. This procedure comprises preparations (database analysis and separation for training, validation and testing), learning rates, iteration number and the size of batch et al, which provides analytical viewpoints from experiments. The proposed parameter-fine-tune procedure can help users to adjust system parameters to achieve ideal recognition rates.

    摘要 I Abstract II 目錄 III 圖目錄 VI 表目錄 VIII 第一章 緒論 1 1.1 研究背景與動機 1 1.2 研究項目與方法概述 1 1.3 論文架構 2 第二章 背景知識與相關文獻探討 3 2.1深度學習 3 2.1.1 卷積層(Convolutional Layer) 3 2.1.2 池化層(Pooling Layer) 3 2.1.3 全連接層(Fully-Connected Layer) 5 2.1.4 丟失函數(Dropout) 6 2.1.5 批次標準化層(Batch Normalization Layer) 6 2.1.6 線性整流層(Rectified Linear Unit Layer) 8 2.1.7 損失層(Loss Layer) 8 2.2 遷移式學習 9 2.2.1 預訓練模型(Pre-Trained Model) 9 2.2.2 微調(Fine-Tuning) 9 2.3 深度架構介紹 10 2.3.1 AlexNet 10 2.3.2 VGG16 12 2.4 優化器(Optimizer) 13 2.4.1隨機梯度下降(Stochastic Gradient Descent) 13 2.4.2 ADAM 14 第三章 參數調整方法 15 3.1 影像分類參數微調系統(Parameter Fine-Tuning System) 15 3.2 參數分析 23 第四章 實驗與分析 27 4.1資料庫介紹 27 4.1.1 Action40 27 4.1.2 Caltech101 27 4.1.3 Caltech256 28 4.2實驗環境設置 29 4.3實驗結果與分析 29 4.3.1 Action40 30 4.3.2 Caltech101 35 4.3.3 Caltech256 40 第五章 結論與未來展望 47 5.1結論 47 5.2未來展望 49 參考文獻 50

    [1] D.-J. Yu et al, “Mixed pooling for convolutional neural networks,” 364-375. 10.1007/978-3-319-11740-9_34, Springer, Cham, 2014.
    [2] https://chtseng.wordpress.com/2017/11/20/ilsvrc-歷屆的深度學習模型/[04 Jan, 2018]
    [3] http://darren1231.pixnet.net/blog/post/336760136-convolutional-neural-net-筆記/[04 Jan, 2018]
    [4] A. Krizhevsky et al, “Imagenet classification with deep convolutional neural networks,” Advances in Neural Information Processing Systems, 2012.
    [5] N. Srivastava et al, “Dropout: a simple way to prevent neural networks from overfitting,” Journal of Machine Learning Research, 15.1:1929-1958, 2014.
    [6] S. Ioffe et al, “Batch normalization: accelerating deep network training by reducing internal covariate shift,” Int. Conf. Machine Learning, 2015.
    [7] https://morvanzhou.github.io/tutorials/machine-learning/tensorflow/5-13-BN/  [04 Jan, 2018]
    [8] S.-J. Pan et al, “A survey on transfer learning,” IEEE Trans. Knowledge and Data Engineering 22.10: 1345-1359, 2010.
    [9] J. Deng et al, “Imagenet: a large-scale hierarchical image database,” IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2009.
    [10] K. Simonyan et al, “Very deep convolutional networks for large-scale image recognition,” arXiv: 1409.1556, 2014.
    [11] O. Russakovsky et al, “Imagenet large scale visual recognition challenge,” International Journal of Computer Vision115.3: 211-252, 2015.
    [12] D. Kingma et al, “Adam: a method for stochastic optimization,” preprint arXiv: 1412.6980, 2014.
    [13] A. Krizhevsky et al, “Learning multiple layers of features from tiny images”, 2009.
    [14] B. Zhou et al, “Places: a 10 million image database for scene recognition,” IEEE Trans. Pattern Analysis and Machine Intelligence, 2017.
    [15] M. Abadi et al, “Tensorflow: large-scale machine learning on heterogeneous distributed systems,” arXiv preprint: 1603.04467, 2016.
    [16] B.-P. Yao et al, “Human action recognition by learning bases of action attributes and parts,” IEEE International Conf. Computer Vision (ICCV), 2011.
    [17] F.-F. Li et al, “Learning generative visual models from few training examples: an incremental Bayesian approach tested on 101 object categories,” Computer Vision and Image Understanding 106.1: 59-70, 2007.
    [18] G. Griffin et al, “Caltech-256 object category dataset,” 2007.
    [19] T. Kobayashi, Sharing convnet across heterogeneous tasks,” International Conf. Neural Information Processing, Springer, Cham, 2017.
    [20] A. P. BĂRAR et al, “Image recognition with deep learning techniques,” Recent Advances in Image, Audio and Signal Processing: Budapest, Hungary, 2013.
    [21] C. Szegedy et al, “Going deeper with convolutions,” Proceedings of the IEEE Conf. Computer Vision and Pattern Recognition, 2015.
    [22] K.-M. He et al, “Deep residual learning for image recognition,” Proceedings of the IEEE Conf. Computer Vision and Pattern Recognition, 2016.

    QR CODE