簡易檢索 / 詳目顯示

研究生: Indira Kar
Indira Kar
論文名稱: 檢測柔性印刷電路板(FPC)表面微小製造缺陷:基於高效的 Faster 區域 卷積神經網路(R-CNN)物件偵測模型
Detection of Small Manufacturing Defects on a Flexible Printed Circuit (FPC) Surface: Based on a Competent Faster Regional Convolutional Neural Network (R-CNN) Object Detection Model
指導教授: 郭中豐
Chung-Feng Jeffrey Kuo
口試委員: 黃昌群
張嘉德
蘇德利
學位類別: 碩士
Master
系所名稱: 工程學院 - 材料科學與工程系
Department of Materials Science and Engineering
論文出版年: 2023
畢業學年度: 111
語文別: 英文
論文頁數: 69
中文關鍵詞: Faster R-CNNFPCManufacturing defectObject detectionclassification
外文關鍵詞: Faster R-CNN, FPC, Manufacturing defect, Object detection, classification
相關次數: 點閱:509下載:0
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報
  • 物件偵測、影像辨識和實例分割,是電腦視覺中的主要概念。這種識別和確認影像中
    的物件,並將它們分成不同特定類別的過程稱為辨識。另一方面,物件偵測是一種在單張
    影像或影片中辨識和標記多個相同物件的方法,它還評估物件的位置和大小。實例分割主
    要將傳統的物件偵測任務與影像的語義分割相結合。重量輕、可靠性高的柔性印刷電路板
    需要一種高效、快速的缺陷檢測方法,尤其是當缺陷很小且難以手動辨識時。本文提出了
    一種基於缺陷類別的柔性印刷電路板(FPC)影像雜質缺陷快速檢測算法,總共有 48 張
    已分類的 FPC 影像,經過五種不同的影像處理技術,形成一個包含 1257 張影像的數據集,
    並作為輸入。 然後基於 Faster 區域卷積神經網絡 (Faster R-CNN) 物件偵測模型,建立了
    FPC 表面缺陷檢測方法。另外,該數據集還與單次檢測器 (SSD)、centernet 和 mobilenet 進
    行了比較。通過調整區域候選網路(RPN)中錨點的比例以及特徵提取架構,提高了檢測
    精度和計算指標的性能。為了確定所建議的方法是否可行和可靠,比較了所有模型的各種
    損失曲線的數值。此外,所建議模型的結果使用 COCO 評估器根據平均精度均值(mAP)
    進行評估,即 mAP(@0.50IOU) 和 AR@100 分別為 0.61 和 0.40,與其他模型相比,Faster
    r-cnn 模型顯示出最好的結果。每張影像的處理時間為 70 毫秒,與其他方法相比,這是非
    常短的時間,本研究中提出的物件偵測模型不僅提高了檢測精度,而且提高了實際效率。


    The main broad concepts in computer vision are object detection, image recognition and
    instance segmentation. This process of recognizing and confirming objects in an image and parting them into different specific classes is referred to as recognition. Object Detection, on the other hand, is a method of recognizing and labeling a number of identical objects within a single image or in a video frame. It also estimates the object's location and its size. Instance segmentation mostly combines traditional object detection tasks with semantic segmentation from an image. An efficient and quick flaw detection approach is required for flexible printed circuit boards that are lightweight and highly reliable, especially when the defects are small and challenging to identify manually with the naked human eye. Based on the defect categories a rapid detection algorithm of impurity defects in flexible printed circuit (FPC) images isproposed. In this paper, total 48 FPC images with classes, which is further gone through five different image processing techniques to make a dataset of 1257 images, and used as an input. Then, based on the faster regional convolutional neural network (Faster R-CNN) object detection model, we build the FPC surface defect detection method. Also, the data set is compared with single shot detector (SSD), centernet and mobilenet. The detection accuracy and performance of computing the metrics are enhanced by adjusting the ratios of anchor in the region proposal network (RPN) as well as the feature extraction architecture. To determine whether the suggested method is workable and reliable, the value of various loss curves of all models is compared. And also, the suggested model's results are evaluated in terms of mean average precision (mAP) using COCO evaluator i.e. mAP(@0.50IOU) and AR@100 of 0.61 and 0.40 respectively. Faster r-cnn model has shown the best results as compare to other. And each image takes 70ms to process, which is a very brief amount of time when compared to the other approaches. The strategy suggested in this research is not only increases inspection accuracy but also increases practical efficiency.

    摘要 iii Abstract iv Acknowledgment iv List of Figures vii List of Tables viii Chapter 1 Introduction 9 1.1 Literature review of some of the previous reports 2 1.2 Problem statement 4 1.3 Our contribution 5 1.4 Structure of project 5 Chapter 2 Research Background 6 2.1 Machine learning 6 2.1.1 Types of machine learning 6 2.1.2 Features 6 2.1.3 Generalization 7 2.2 Neural networks 7 2.2.1 Origins 7 2.2.2 Multi-layer networks 8 2.2.3 Back-propagation 9 2.3 Famous convolutional neural network architectures 10 2.3.1 Lenet 10 2.3.2 Alexnet 11 2.3.3 ZFnet 11 2.3.4 Googlenet 11 2.3.5 Vggnet 11 2.3.6 Resnet 12 2.4 Activation function 12 2.4.1 Rectified linear unit (ReLU) 12 2.4.2 Leaky rectified linear unit (Leaky ReLU) 13 2.4.3 Sigmoid function 13 2.4.4 Hyperbolic tangent function 13 2.4.5 Heaviside step function 13 2.5 Loss function 14 2.5.1 Total mean squared error (TMSE) 14 2.5.2 Cross entropy loss (CEL) 15 2.5.3 SoftMax loss 15 2.6 Deep learning 15 2..6 Computer vision 16 Chapter 3 Convolutional Object Detection 17 3.1 Object detection 17 3.1.1 CNN (convolution neural network) 18 3.1.2 R-cnn 23 3.1.3 Fast r-cnn 24 3.1.4 Faster r-cnn 25 Chapter 4 Methodology 26 4.1 Process involved 26 4.2 Important object detection concepts 27 4.2.1 Feature map extraction 27 4.2.2 RPN region 27 4.2.3 ROI layer 27 4.2.4 Intersection over union (IoU) 28 4.2.5 Non-maximum suppression (NMS) 28 4.2.6 Bounding-box regression (bounding-box refinement) 28 4.3 Detection models 29 4.3.1 Faster r-cnn (regional convolutional neural network) 29 4.3.2 SSD (single shot detector) resnet-50 30 4.3.3 Mobilenet 31 4.3.4 Centernet 32 Chapter 5 Experimental Set-up 33 5.1 Data acquisition 33 5.2 Types of defects 34 5.3 Preprocessing of dataset 35 5.4 Data augmentation 36 5.4.1 Cropping 36 5.4.2 Geometric transformations 37 5.4.3 Filtering 37 5.4.4 Noise injections 38 5.4.5 Intensity operations 39 5.5 Data labeling 40 Chapter 6 Results & Discussions 41 6.1 Results for faster r-cnn 41 6.2 Types of losses 41 6.3 Comparison and performance analysis 45 6.3.1 Precision & recall 45 6.3.2 IoU (intersection over union) 46 6.3.3 COCO mAP 46 6.3.4 AR 46 Chapter 7 Conclusion and Future Outlook 48 7.1 Conclusion 48 7.2 Future work 48 References 49

    [1] Dalal, N., Triggs, B. 2005. Histograms of oriented gradients for human detection. In: Proceedings of IEEE computer society conference on computer vision and pattern recognition, pp. 886-893.
    [2] Girshick, R., Donahue, J., Darrell, T., Malik, J. 2014. Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 580-587.
    [3] Leong, W. C., Abdullah, M. Z., Khor, C. Y., 2012. Application of flexible printed circuit board (FPCB) in personal computer motherboards: focusing on mechanical performance. Microelectron. Reliab. 52(4), 744-756.
    [4] Liu, Z., 2011. Reliability evaluation of FPC under bending stress. In: Proceedings of International Conference on Electronic & Mechanical Engineering and Information Technology, pp. 2955-2958.
    [5] Baygin, M., Karakose, M., Sarimaden, A., Erhan, A. K. I. N., 2017. Machine vision-based defect detection approach using image processing. In: Proceedings of International Artificial Intelligence and Data Processing Symposium (IDAP), pp. 1-5.
    [6] Chen, C. H., Wang, C. C., Lin, C. Y., Shih, Y. S., Tu, C. F., 2007. Realization of defect automatic inspection system for flexible printed circuit (FPC). In: Proceedings of International MATADOR Conference, pp. 225-228.
    [7] Hanzaei, S. H., Afshar, A., Barazandeh, F., 2017. Automatic detection and classification of the ceramic tiles’ surface defects. Pattern Recognit. 66, 174-189.
    [8] Du, J., Chen, F., Hu, Y. M., 2017. Automatic defect inspection of patterned FPC board based on 1-D fourier reconstruction. In Proceedings of Chinese Control Conference (CCC), pp. 10109-10112.
    [9] Ye, F., Li, D., Huang, J. X., Dong, Z. J., 2012. Flaw detection on FPC solder surface. Circuit World.
    [10] Wang, Q., Li, D., Zhang, W., Cao, D., Chen, H., 2010. Unsupervised defect detection of flexible printed circuit board gold surfaces based on wavelet packet frame. In 2010 International Conference on Industrial and Information Systems, pp. 324-327.
    [11] Wang, Q., Li, D., Zhang, W., 2008. Detecting defects in golden surfaces of flexible printed circuits using optimal Gabor filters. In 2008 Second International Symposium on Intelligent Information Technology Application, pp. 321-325.
    [12] Li, D., Li, S.,Yuan, W., 2020. Flexible printed circuit fracture detection based on hypothesis testing strategy. IEEE Access 8, 24457-24470.
    [13] Luo, J., Zou, H., Chen, X., Gao, H., 2019. A fast circle detection method based on a tri-class Thresholding for high detail FPC images. IEEE Trans. Instrum. Meas. 69(4), 1327-1335.
    [14] Janóczki, M., Becker, Á., Jakab, L., Gróf, R., Takács, T., 2013. Automatic optical inspection of soldering. Materials Science-Advanced Topics 2156-3950.
    [15] Richter, J., Streitferdt, D., Rozova, E., 2017. On the development of intelligent optical inspections. In 2017 IEEE 7th annual Computing and Communication Workshop and Conference (CCWC), pp. 1-6.
    [16] Taha, E. M., Emary, E., Moustafa, K., 2014. Automatic optical inspection for PCB manufacturing: A survey. Int. j. sci. eng. res. 5(7), 1095-1102.
    [17] Li, Z., Yang, Q., 2011. System design for PCB defects detection based on AOI technology. In 2011 4th International Congress on Image and Signal Processing, pp. 1988-1991.
    [18] Qiang, G., Shanshan, Z., Yang, Z., Mao, C., 2015. Detection method of PCB component based on automatic optical stitching algorithm. Circuit World.
    [19] Tao, X., Zhang, D., Ma, W., Liu, X., Xu, D., 2018. Automatic metallic surface defect detection and recognition with convolutional neural networks. Appl. Sci. 8(9), 1575.
    [20] Dung, C. V., 2019. Autonomous concrete crack detection using deep fully convolutional neural network. Autom. Constr. 99, 52-58.
    [21] Xie, H., Li, Y., Li, X., He, L., 2021. A Method for Surface Defect Detection of Printed Circuit Board Based on Improved YOLOv4. In 2021 IEEE 2nd International Conference on Big Data, Artificial Intelligence and Internet of Things Engineering (ICBAIE), pp. 851-857.
    [22] Zang, Y., Zhang, J., Billah, M. M., 2021. Defect detection of flexible circuit board based on convolutional neural network. In The 2nd International Conference on Computing and Data Science, pp. 1-5.
    [23] Sun, X., Wu, P., Hoi, S. C., 2018. Face detection using deep learning: An improved faster RCNN approach. Neurocomputing 299, 42-50.
    [24] Wei, X., Yang, Z., Liu, Y., Wei, D., Jia, L., Li, Y., 2019. Railway track fastener defect detection based on image processing and deep learning techniques: A comparative study. Eng Appl Artif Intell. 80, 66-81.
    [25] Pei, L., Sun, Z., Xiao, L., Li, W., Sun, J., Zhang, H., 2021. Virtual generation of pavement crack images based on improved deep convolutional generative adversarial network. Eng Appl Artif Intell. 104, 104376.
    [26] Zeng, L., Sun, B., Zhu, D., 2021. Underwater target detection based on Faster R-CNN and adversarial occlusion network. Eng Appl Artif Intell. 100, 104190.
    [27] Jiang, Q., Jia, M., Bi, L., Zhuang, Z., Gao, K., 2022. Development of a core feature identification application based on the Faster R-CNN algorithm. Eng Appl Artif Intell. 115, 105200.
    [28] Kaur, P., Harnal, S., Gautam, V., Singh, M. P., Singh, S. P., 2022. An approach for characterization of infected area in tomato leaf disease based on deep learning and object detection technique. Eng Appl Artif Intell. 115, 105210.
    [29] Goodfellow, I., Bengio, Y., Courville, A. 2016. Deep learning. MIT press.
    [30] Bishop, C. M., & Nasrabadi, N. M. 2006. Pattern recognition and machine learning. Springer, 4, p. 738.
    [31] Bengio, Y. 2012. Deep learning of representations for unsupervised and transfer learning. In: Proceedings of ICML Workshop on Unsupervised and Transfer Learning, pp. 17-36.
    [32] McCulloch, W. S., Pitts, W. 1943. A logical calculus of the ideas immanent in nervous activity. Bull. math. biophys. 5(4), 115-133.
    [33] Rojas, R. 2013. Neural networks: a systematic introduction. Springer Science & Business Media.
    [34] Long, L. N., Gupta, A. 2008. Scalable massively parallel artificial neural networks. Journal of Aerospace Computing, Information, and Communication, 5(1), 3-15.
    [35] Hornik, K. 1991. Approximation capabilities of multilayer feedforward networks. Neural Netw. 4(2), 251-257.
    [36] Rumelhart, D. E., Hinton, G. E., Williams, R. J. 1988. Learning representations by back-propagating errors Cognitive Modeling ch 8.
    [37] Wilson, D. R., Martinez, T. R. 2003. The general inefficiency of batch training for gradient descent learning. Neural Netw. 16(10), 1429-1451.
    [38] Al-Jawfi, R. 2009. Handwriting Arabic character recognition LeNet using neural network. Int. Arab J. Inf. Technol. 6(3), 304-309.
    [39] Alom, M.Z, Taha, T. M., Yakopcic, C., Westberg, S., Sidike, P., Nasrin, M. S., Esesn, B.C.V., Awwal. A.A.S, Asari. V. K. 2018. "The history began from alexnet: A comprehensive survey on deep learning approaches." arXiv preprint arXiv:1803.01164
    [40] Antioquia, A. M. C., Tan, D. S., Azcarraga, A., Cheng, W. H., Hua, K. L. 2018. ZipNet: ZFNet-level accuracy with 48× fewer parameters. In: IEEE Visual Communications and Image Processing (VCIP), pp. 1-4.
    [41] Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V. and Rabinovich, A., 2015. Going deeper with convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1-9.
    [42] Tammina, S. 2019. Transfer learning using vgg-16 with deep convolutional neural network for classifying images. Int. j. sci. res. Publ. 9(10), 143-150.
    [43] Koonce, B. 2021. ResNet 50. In: Proceedings of Convolutional neural networks with Swift for Tensorflow, pp. 63-72.
    [44] LeCun, Y., Bengio, Y., Hinton, G. 2015. Deep learning. Nature 521(7553), 436-444.
    [45] Steinkraus, D., Buck, I., Simard, P. Y. 2005. Using GPUs for machine learning algorithms. In Eighth International Conference on Document Analysis and Recognition (ICDAR'05), pp. 1115-1120.
    [46] Girshick, R. 2015. Fast r-cnn. In: Proceedings of the IEEE international conference on computer vision, pp. 1440-1448.
    [47] Szeliski, R. 2010. Computer vision: algorithms and applications Springer Science & Business Media.
    [48] Sebe, N., Cohen, I., Garg, A., Huang, T. S. 2005. Machine learning in computer vision Springer Science & Business Media Vol. 29.
    [49] Huang, T. 1996. Computer vision: Evolution and promise.
    [50] Zhao, Z. Q., Zheng, P., Xu, S. T., Wu, X. 2019. Object detection with deep learning: A review. IEEE Trans. Neural Netw. Learn. Syst. 30(11), 3212-3232.
    [51] Walther, D., Itti, L., Riesenhuber, M., Poggio, T., Koch, C. 2002. Attentional selection for object recognition—a gentle way. In International workshop on biologically motivated computer vision. Springer, pp. 472-479.
    [52] Burger, W., Burge, M. J. 2022. Scale-invariant feature transform (SIFT). In Digital Image Processing. Springer, pp. 709-763.
    [53] Lowe, D. G. 1999. Object recognition from local scale-invariant features. In: Proceedings of the seventh IEEE International Conference on Computer Vision, pp. 1150-1157.
    [54] Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C. Y., Berg, A. C. 2016. Ssd: Single shot multibox detector. In European Conference on Computer Vision. Springer, pp. 21-37.
    [55] Liu, M., Jiang, J., Wang, Z. 2019. Colonic polyp detection in endoscopic videos with single shot detection based deep convolutional neural network. IEEE Access 7, 75058-75066.
    [56] Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, Weyand, T., Andreetto, M. Adam, H. 2017. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861.
    [57] Uijlings, J. R., Van De Sande, K. E., Gevers, T., Smeulders, A. W. 2013. Selective search for object recognition. Int J Comput Vis. 104(2), 154-171.
    [58] Alzubaidi, L., Zhang, J., Humaidi., A.J., Al-Dujaili., A., Duan, Y., Al-Shamma., O., Santama., J., Fadhel, M.A., Al-Amidie, M. Farhan, L. 2021. Review of deep learning: Concepts, CNN architectures, challenges, applications, future directions. J. Big Data 8, pp.1-74.
    [59] Parasuraman, R., Cosenzo, K. A., De Visser, E. 2009. Adaptive automation for human supervision of multiple uninhabited vehicles: Effects on change detection, situation awareness, and mental workload. Mil Psychol. 21(2), 270-297.
    [60] Hu, G., Yang, Y., Yi, D., Kittler, J., Christmas, W., Li, S. Z., Hospedales, T. 2015. When face recognition meets with deep learning: an evaluation of convolutional neural networks for face recognition. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 142-150.
    [61] Dathathri, S., Dvijotham, K., Kurakin, A., Raghunathan, A., Uesato, J., Bunel, R.R., Shankar, S., Steinhardt, J., Goodfellow, I., Liang, P.S. Kohli, P. 2020. Enabling certification of verification-agnostic networks via memory-efficient semidefinite programming. Adv Neural Inf Process Syst. 33, pp.5318-5331.
    [62] Chatfield, K., Simonyan, K., Vedaldi, A., Zisserman, A. 2014. Return of the devil in the details: Delving deep into convolutional nets. arXiv preprint arXiv:1405.3531.
    [63] Springenberg, J. T., Dosovitskiy, A., Brox, T., Riedmiller, M. 2014. Striving for simplicity: The all convolutional net. arXiv preprint arXiv:1412.6806.
    [64] Krizhevsky, A., Sutskever, I., Hinton, G. E. 2017. Imagenet classification with deep convolutional neural networks. Commun. ACM 60(6), 84-90.
    [65] Hubel, D. H., Wiesel, T. N. 1968. Receptive fields and functional architecture of monkey striate cortex. J. Physiol. 195(1), 215-243.
    [66] Luo, Weiru, Jiaxiang Luo, Zhiyu Yang. 2020. FPC surface defect detection based on improved Faster R-CNN with decoupled RPN. In 2020 Chinese Automation Congress (CAC), pp. 7035-7039.
    [67] Chen, Yu Peng, Ying Li, Gang Wang. 2018. An Enhanced Region Proposal Network for object detection using deep learning method. PloS one 9, e0203897.
    [68] Ren, S., He, K., Girshick, R., Sun, J. 2015. Faster r-cnn: Towards real-time object detection with region proposal networks. Adv Neural Inf Process Syst. 28.
    [69] Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C. Y., Berg, A. C. 2016. Ssd: Single shot multibox detector. In European conference on computer vision. Springer, pp. 21-37.
    [70] Li, Y., Yu, X., Koudas, N. 2021. Data acquisition for improving machine learning models. arXiv preprint arXiv:2105.14107.
    [71] Chlap, P., Min, H., Vandenberg, N., Dowling, J., Holloway, L., Haworth, A. 2021. A review of medical image data augmentation techniques for deep learning applications. J Med Imaging Radiat Oncol. 65(5), 545-563.
    [72] Kompanek, M., T., Benesova, W. 2019. Volumetrie Data Augmentation as an Effective Tool in MRI Classification Using 3D Convolutional Neural Network. In 2019 International Conference on Systems, Signals and Image Processing (IWSSIP), pp. 115-119.
    [73] Aderghal, K., Khvostikov, A., Krylov, A., Benois-Pineau, J., Afdel, K., Catheline, G. 2018. Classification of Alzheimer disease on imaging modalities with deep CNNs using cross-modal transfer learning. In 2018 IEEE 31st International Symposium on Computer-based Medical Systems (CBMS), pp. 345-350.
    [74] Holmstrom, L., Koistinen, P. 1992. Using additive noise in back-propagation training. IEEE Trans Neural Netw Learn Syst. 3(1), 24-38
    [75] Zhang, S., Zhang, S., Zhang, C., Wang, X., Shi, Y. 2019. Cucumber leaf disease identification with global pooling dilated convolutional neural networkComput. Electron. Agric. 162, 422-430.
    [76] Xu, Z., Hrustic, E., Vivet, D. 2020. Centernet heatmap propagation for real-time video object detection. In European conference on computer vision. Springer, pp. 220-234.

    無法下載圖示 全文公開日期 2028/02/09 (校內網路)
    全文公開日期 2028/02/09 (校外網路)
    全文公開日期 2028/02/09 (國家圖書館:臺灣博碩士論文系統)
    QR CODE