簡易檢索 / 詳目顯示
| 研究生： |
陳冠霖 Kuan-Ling Chen |
|---|---|
| 論文名稱： |
應用基於遷移學習之卷積神經網路於鋼片缺陷分類及物件偵測 Automatic Classification and Location of Steel Surface Defect Using Transfer Learning Based Convolutional Neural Networks |
| 指導教授： |
王福琨
Fu-Kwun Wang |
| 口試委員: |
羅士哲
Shih-Che Lo 陳子立 Tzu-Li Chen |
| 學位類別： |
碩士 Master |
| 系所名稱： |
管理學院 - 工業管理系 Department of Industrial Management |
| 論文出版年： | 2022 |
| 畢業學年度： | 110 |
| 語文別： | 英文 |
| 論文頁數： | 78 |
| 中文關鍵詞： | 鋼片缺陷偵測 、遷移學習 、Xception 、Cascade-RCNN |
| 外文關鍵詞： | Steel defect detection, Transfer learning, Xception model, Cascade-RCNN |
| 相關次數： | 點閱：365 下載：10 |
| 分享至: |
| 查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報 |
近年來，企業導入工業5.0技術是一種趨勢。對於許多產品來說，表面缺陷可能是一個關鍵點。然而，在傳統的品質管制中，人力成本是高的。此外，由於精度低，運行時間長，不能很好地應用於生產線。隨著時間的推移，由於深度學習的不斷發展，越來越多的檢測方法被提出。雖然，物體檢測是維持產品質管制的一種流行方法，但微小而復雜的物體的識別仍然是我們需要解決的挑戰。金屬表面缺陷檢測是鋼鐵企業經常遇到的問題。有效的品質管制可以降低因鋼材缺陷而產生的成本。在本論文中，我們提出了一種綜合方法，包括數據預處理、二元分類、多重分類和目標檢測。選擇兩個數據集來驗證實驗：SEVERSTAL 數據集和 NEU-DET 數據集。為了使模型具有穩建性和通用性，我們使用遷移學習方法來訓練預訓練模型。使用預訓練模型，我們可以在訓練數據上花費更少的時間。遷移學習模型也可以應用於其他新的鋼鐵缺陷數據集，並且仍然保持較高的準確性。論文使用了幾種評估指標，包括準確度、精確度、召回率、F1分數和mAP。我們所提出的分類模型準確度100%及mAP是76.0，整個流程包含了分類及偵測位置，更貼近真實世界的需求，也在模型在做五次的重複驗證以確保泛用及穩健性。
In recent years, it is a trend for enterprises to introduce Industry 5.0 technology. For many products, surface defects can be a critical point. However, in the traditional quality control, the labor cost is high. In addition, due to low precision and long running time, it cannot be well applied to the production line. Over time, more and more detection methods have been proposed because of the continuous development of artificial intelligence. While object detection is a popular method to maintain product quality control, the identification of tiny and complex objects is still a challenge we need to address. Metal surface defect detection is a common problem encountered by iron and steel enterprises. Effective quality control can reduce costs due to steel defects. In this thesis, we propose a comprehensive approach including data preprocessing, binary classification, multi-class classification, and object detection. Two datasets were chosen to validate the experiments: the SEVERSTAL dataset and the NEU-DET dataset. To make the model robust and general, we use a transfer learning method to train the pretrained model. With pretrained models, we can spend less time on training data. The transfer learning model can also be applied to other new steel defect datasets and still maintain high accuracy. This study uses several evaluation metrics including accuracy, precision, recall, F1 score, and mAP. The accuracy of the classification model is 100%, and the mAP is 76.0. The proposed model has a complete flow which is achieve the real-world needs, and we also use five-iteration rigorous experiments verification.
[1] S. Guan, M. Lei, and H. Lu, “A steel surface defect recognition algorithm based on improved deep learning network model using feature visualization and quality evaluation,” IEEE Access, vol. 8, pp. 49885-49895, Mar. 2020. doi: 10.1109/ACCESS.2020.2979755.
[2] K. He and L. Wang, “A review of energy use and energy-efficient technologies for the iron and steel industry,” Renew. Sustain. Energy Rev., vol. 70, pp. 1022-1039, Apr. 2017. doi: https://doi.org/10.1016/j.rser.2016.12.007.
[3] Y. Jiang, T. C. Ling, C. Shi, and S. Y. Pan, “Characteristics of steel slags and their use in cement and concrete—a review,” Resour. Conserv. Recycl., vol. 136, pp. 187-197, May. 2018. doi: 10.1016/j.resconrec.2018.04.023.
[4] J. Li, Z. Su, J. Geng, and Y. Yin, “Real-time detection of steel strip surface defects based on improved yolo detection network,” IFAC-PapersOnLine, vol. 51, pp. 76-81, Jan. 2018. doi: https://doi.org/10.1016/j.ifacol.2018.09.412.
[5] Q. Luo, X. Fang, L. Liu, C. Yang, and Y. Sun, “Automated visual defect detection for flat steel surface: a survey,” IEEE Trans. Instrum. Meas., vol. 69, no. 3, pp. 626-644, Mar. 2020. doi: 10.1109/TIM.2019.2963555.
[6] F. Li, S. Spagnul, V. Odedo, and M. Soleimani, “Monitoring surface defects deformations and displacements in hot steel using magnetic induction tomography,” Sensors, vol. 19, p. 3005, Jul. 2019. doi: 10.3390/s19133005.
[7] Y. J. Jeon, D. C. Choi, S. J. Lee, J. P. Yun, and S. W. Kim, “Steel-surface defect detection using a switching-lighting scheme,” Appl. Opt., vol. 55, pp. 47-57, Feb. 2016. doi: https://doi.org/10.1364/AO.55.000047.
[8] C. Wang, Y. Yu, J. Yu, and Y. Zhang, “Microstructure evolution and corrosion behavior of dissimilar 304/430 stainless steel welded joints,” J. Manuf. Process., vol. 50, pp. 183-191, Feb. 2020. doi: https://doi.org/10.1016/j.jmapro.2019.12.015.
[9] H. Takino and T. Hosaka, “Shaping of steel mold surface of lens array by electrical discharge machining with single rod electrode,” Appl. Opt., vol. 53, pp. 8002-8005, Nov. 2014. doi: https://doi.org/10.1364/AO.53.008002.
[10] Y. He, K. Song, Q. Meng, and Y. Yan, “An end-to-end steel surface defect detection approach via fusing multiple hierarchical features,” IEEE Trans. Instrum. Meas., vol. 69, no. 4, pp. 1493-1504, Apr. 2020. doi: 10.1109/TIM.2019.2915404.
[11] Q. Yao, J. Feng, J. Tang, W. G. Xu, X. H. Zhu, B. J. Yang, J. LÜ, Y. Z. Xie, B. Yao, S. Z. Wu N. Y. Kuai, and L. J. Wang, “Development of an automatic monitoring system for rice light-trap pests based on machine vision,” J. Integr. Agric., vol. 19, pp. 2500-2513, 2020. doi: https://doi.org/10.1016/S2095-3119(20)63168-9
[12] J. Xi, L. Shentu, J. Hu, and M. Li, “Automated surface inspection for steel products using computer vision approach,” Appl. Opt., vol. 56, p. 184, Jan. 2017. doi: https://doi.org/10.1016/S2095-3119(20)63168-9.
[13] Severstal: Steel Defect Detection. Accessed: July. 26, 2019. [Online]. Available: https://www.kaggle.com/c/severstal-steel-defect-detection.
[14] X. Li, S. K. Tso, X. Guan and Q. Huang, “Improving automatic detection of defects in castings by applying wavelet technique,” IEEE Trans. Ind. Electron., vol. 53, no. 6, pp. 1927-1934, Dec. 2006. doi: 10.1109/TIE.2006.885448.
[15] J. Lu, S. Tang, J. Wang, H. Zhu and Y. Wang, “A review on object detection based on deep convolutional neural networks for autonomous driving,” 2019 Chinese Control And Decision Conference (CCDC), China, Jun. 2019, pp. 5301-5308, doi: 10.1109/CCDC.2019.8832398.
[16] S. M. Chao and D. M. Tsai, “An anisotropic diffusion-based defect detection for low-contrast glass substrates,” Image Vis. Comput., vol. 26, pp. 187-200, Feb. 2008. doi: https://doi.org/10.1016/j.imavis.2007.03.003.
[17] A. Alam, J. Thim, M. ONils, A. Manuilskiy, J. Lindgren, and J. Lidén, “Online surface characterization of paper and paperboards in a wide-range of the spatial wavelength spectrum,” Appl. Surf. Sci., vol. 258, pp. 7928-7935, Aug. 2012. doi: https://doi.org/10.1016/j.apsusc.2012.04.136.
[18] W. B. Li, C. H. Lu, and J. C. Zhang, “A local annular contrast based real-time inspection algorithm for steel bar surface defects,” Appl. Surf. Sci., vol. 258, pp. 6080-6086, Jun. 2012. doi: https://doi.org/10.1016/j.apsusc.2012.03.007.
[19] R. Ren, T. Hung and K. C. Tan, “A generic deep-learning-based approach for automated surface inspection,” IEEE Trans. Cybern., vol. 48, no. 3, pp. 929-940, Mar. 2018. doi: 10.1109/TCYB.2017.2668395.
[20] C. Tastimur, H. Yetis, M. Karaköse, and E. Akin, “Rail defect detection and classification with real time image processing technique,” Int. J. Comput. Softw. Eng., vol. 5, pp. 283-290, Dec. 2016. doi: 10.1109/INDIN.2016.7819194.
[21] X. Li, B. Gao, W. L. Woo, G. Y. Tian, X. Qiu and L. Gu, “Quantitative surface crack evaluation based on eddy current pulsed thermography,” IEEE Sens. J., vol. 17, no. 2, pp. 412-421, Jan. 2017. doi: 10.1109/JSEN.2016.2625815.
[22] X. Bai, Y. Fang, W. Lin, L. Wang and B. Ju, “Saliency-based defect detection in industrial images by using phase spectrum,” IEEE Trans. Industr. Inform., vol. 10, no. 4, pp. 2135-2145, Nov. 2014. doi: 10.1109/TII.2014.2359416.
[23] S. Susan and M. Sharma, “Automatic texture defect detection using gaussian mixture entropy modeling,” Neurocomputing, vol. 239, pp. 232-237. May. 2017. doi: https://doi.org/10.1016/j.neucom.2017.02.021.
[24] S. J. Pan and Q. Yang, “A survey on transfer learning,” IEEE Trans. Knowl. Data Eng., vol. 22, no. 10, pp. 1345-1359, Oct. 2010. doi: 10.1109/TKDE.2009.191.
[25] M. Abu, A. Amir, Y. H. Lean, N. A. H. Zahri and S. A. Azemi, “The performance analysis of transfer learning for steel defect detection by using deep learning,” J. Phys. Conf. Ser., vol. 1755, Feb. 2021. doi: 10.1088/1742-6596/1755/1/012041.
[26] S. Wang, X. Xia, L. Ye and B. Yang, “Steel surface defect detection using transfer learning and image Ssegmentation,” 2020 IEEE 6th International Conference on Computer and Communications (ICCC), Chengdu, China, Dec. 2020, pp. 420-425, doi: 10.1109/ICCC51575.2020.9345151.
[27] Y. Ganin, E. Ustinova, H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, M. Marchand, and V. Lempitsky, “Domain-adversarial training of neural networks,” Adv. Comput. Vis. Pattern Recognit., pp. 189-209, Sep. 2017. doi: 10.1007/978-3-319-58347-1_10.
[28] A Newbie-Friendly Guide to Transfer Learning. Accessed: June. 20, 2022. [Online]. Available: https://www.v7labs.com/blog/transfer-learning-guide.
[29] C. Tan, F. Sun, T. Kong, W. Zhang, C. Yang, and C. Liu, “A survey on deep transfer learning,” In: Kůrková, V., Manolopoulos, Y., Hammer, B., Iliadis, L., Maglogiannis, I. (eds) Artificial Neural Networks and Machine Learning – ICANN 2018. ICANN 2018. Lecture Notes in Computer Science(), vol 11141. Springer, Cham. https://doi.org/10.1007/978-3-030-01424-7_27.
[30] F. Chollet, “Xception: deep learning with depthwise separable convolutions,” 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, USA, Jul. 2017, pp. 1800-1807, doi: 10.1109/CVPR.2017.195.
[31] C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, “Rethinking the inception architecture for computer vision,” 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, USA, Jun. 2016, pp. 2818-2826, doi: 10.1109/CVPR.2016.308.
[32] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, USA, Jun. 2016, pp. 770-778, doi: 10.1109/CVPR.2016.90.
[33] A. G. Howard, M. Zhu, M. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, “Mobilenets: efficient convolutional neural networks for mobile vision applications,” Apr. 2017. [Online]. Available: https://doi.org/10.48550/arXiv.1704.04861.
[34] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L. C. Chen, “Mobilenetv2: inverted residuals and linear bottlenecks,” 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, Jun. 2018, pp. 4510-4520, doi: 10.1109/CVPR.2018.00474.
[35] R. Girshick, J. Donahue, T. Darrell, and J. Malik, “Rich feature hierarchies for accurate object detection and semantic segmentation,” 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, USA, Jun. 2014, pp. 580-587, doi: 10.1109/CVPR.2014.81.
[36] R. Girshick, “Fast r-cnn” 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile, Dec. 2015, pp. 1440-1448, doi: 10.1109/ICCV.2015.169.
[37] S. Ren, K. He, R. Girshick, and J. Sun, “Faster r-cnn: towards real-time object detection with region proposal networks,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 39, pp. 1137-1149, Jun. 2017. doi: 10.1109/TPAMI.2016.2577031.
[38] Keras Applications. Accessed: Jul. 2022. [Online]. Available: https://keras.io/api/applications.
[39] J. Fu, X. Zhu, and Y. Li, “Recognition of surface defects on steel sheet using transfer learning,” Sep. 2019. [Online]. Available: https://doi.org/10.48550/arXiv.1909.03258.
[40] Z. Li, Z. Tian, X. Liu, Y. Liu, and X. Shi, “A two-stage industrial defect detection framework based on improved-yolov5 and optimized-inception-resnetv2 models,” Appl. Sci., vol. 12, no. 2, p. 834, Jan. 2022. doi: https://doi.org/10.3390/app12020834.
[41] A. M. Nagy and L. Czúni, “Classification and fast few-shot learning of steel surface defects with randomized network,” Appl. Sci., vol. 12, no. 8, p. 3967, Apr. 2022. doi: https://doi.org/10.3390/app12083967.
[42] H. Zhao, Z. Yang, and J. Li, “Detection of metal surface defects based on yolov4 algorithm,” J. Phys. Conf. Ser., vol. 1907, 2021. doi: 10.1088/1742-6596/1907/1/012043.
[43] W. Zhao, F. Chen, H. Huang, D. Li, W. Cheng, “A new steel defect detection algorithm based on deep learning,” Comput. Intell. Neurosci., vol. 2021, p. 13, Mar. 2021. doi: https://doi.org/10.1155/2021/5592878.
[44] NEU Surface Defect Database. Accessed: July. 14, 2021. [Online]. Available: https://www.kaggle.com/datasets/kaustubhdikshit/neu-surface-defect-database.
[45] Z. Cai, N. Vasconcelos, “Cascade r-cnn: delving into high quality object detection,” 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, Jun. 2018, pp. 6154-6162, doi: 10.1109/CVPR.2018.00644.
[46] B. Fan and W. Li, “Application of gcb-net based on defect detection algorithm for steel plates,” Research Square, Apr. 2022. doi: https://doi.org/10.21203/rs.3.rs-1550068/v1.
[47] J. Xing and M. Jia, “A convolutional neural network-based method for workpiece surface defect detection,” Measurement, vol. 176, May. 2021. doi: https://doi.org/10.1016/j.measurement.2021.109185.
[48] X. Lv, F. Duan, J. J. Jiang, X. Fu, and L. Gan, “Deep Metallic Surface Defect Detection: The New Benchmark and Detection Network,” Sensors, Mar. 2020. doi: 10.3390/s20061562.
