近年來有許多工業自動光學瑕疵檢測的方法被提出，這些方法往往是針對要處理案例的影像提出專用的特徵抽取演算法或是影像前處理方法來處理瑕疵檢測問題，它們在瑕疵分類表現上都可以達到很高的正確率，但是這些方法通常只能針對個別的案例作處理，沒有辦法延展到其他類型的瑕疵檢測問題上。在本論文中，我們提出了一個通用型的工業自動光學檢測框架，可以套用在大部分的瑕疵分類應用上。有鑑於近年深度學習技術被用於許多影像分類相關問題並且在特徵提取上有著優異的表現，我們的方法使用了基於深度學習的模型來學習提取工業影像中的特徵，藉由自動學習影像特徵來達到通用化的目的。由於工業影像通常會有很高的解析度，這造成直接套用深度卷積類神經網路模型會有記憶體過量以及訓練時間過長的問題。另外工業影像中有瑕疵的部分常常只佔了整張圖中的極小一部分，這使得我們沒辦法單純的將整張圖片作縮小再處理，縮小影像會直接造成影像中瑕疵部分的資訊消失。另外通常工業影像的資料都為弱標籤 (weakly labeled) 資料，這代表我們只能知道整張影像中是否有瑕疵，但無法知道其中瑕疵在圖上的哪裡，由於沒有瑕疵位置資訊，這使得我們無法單純切割影像為較小的影像再作訓練。我們的方法利用學習影像內部差異來克服以上提到的三個困難點，以達到利用深度學習處理工業影像瑕疵檢測問題的目的。此方法中我們使用 sliding window 把輸入影像分成各個小區塊影像，並且使用卷積自動編碼器 (convolutional autoencoder) 作資料降維以及第一階段特徵提取，接著使用一個配對機制以將這些小區塊影像組成配對，並且利用孿生深度卷積類神經網路來作為第二階段的特徵提取並且學習配對中兩個區塊影像的差異，最後利用有瑕疵的加工樣品影像以及正常的加工樣品影像之間所得到的配對差異分佈不同，來達到分類的目的。就我們的研究，我們是第一個可以同時克服上述三個困難點的深度學習方法，此外我們也是第一個致力於通用化的工業瑕疵檢測方法。

This thesis presents a general purpose defect detection method based on siamese CNN. Due to the high demand of automated optical inspection application, various AOI methods have been proposed. However, these AOI methods could be applied to limited scenarios or applications. To improve the versatility of AOI algorithm, in this thesis, we propose to utilize deep learning techniques in automatically learning data features to improve the generalization of AOI algorithm.

Generally, there are three challenges when employing deep learning in industrial inspection task. First, high resolution of input image that causes complexity issue; second, imbalanced data that causes training issue; last, weakly labeled data where the data labels only have simple class information (normal or defective) and no defect location information. To overcome the above mentioned three difficulties, we first proposed a framework that learns the intra relations of product sample images. In addition, a sliding window strategy is applied to separate input images to small patches. The patches are used to train a convolutional autoencoder which transfers raw patch data to feature patch data. To learn the inter relations of feature patch data, we employ a siamese CNN to learn the similarity of patches. By using the different distributions of patches' similarities from the same image, the proposed framework can effectively classify normal and defective samples.

[1] A. Krizhevsky, Sutskever, Ilya, Hinton, and G. E, “Imagenet classification with deep convolutional
neural networks,” in Advances in Neural Information Processing Systems 25 (F. Pereira, C. J. C.
Burges, L. Bottou, and K. Q. Weinberger, eds.), pp. 1097–1105, Curran Associates, Inc., 2012.
[2] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,”
CoRR, vol. abs/1409.1556, 2014.
[3] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich,
“Going deeper with convolutions,” in The IEEE Conference on Computer Vision and Pattern
Recognition (CVPR), June 2015.
[4] J. Lee, H.-A. Kao, and S. Yang, “Service innovation and smart analytics for industry 4.0 and big data
environment,” Procedia CIRP, vol. 16, pp. 3 – 8, 2014.
[5] S. Huang, X. Li, Z. M. Zhang, Z. He, F. Wu, W. Liu, J. Tang, and Y. Zhuang, “Deep learning driven
visual path prediction from a single image,” CoRR, vol. abs/1601.07265, 2016.
[6] M. Vakalopoulou, K. Karantzalos, N. Komodakis, and N. Paragios, “Building detection in very high
resolution multispectral data with deep learning features,” in 2015 IEEE International Geoscience and
Remote Sensing Symposium (IGARSS), pp. 1873–1876, July 2015.
[7] D.-M. Tsai and C.-H. Yang, “A quantile–quantile plot based pattern matching for defect detection,”
Pattern Recognition Letters, vol. 26, no. 13, pp. 1948 – 1962, 2005.
[8] H. Kong, J. Yang, and Z. Chen, “Accurate and efficient inspection of speckle and scratch defects on
surfaces of planar products,” vol. PP, pp. 1–1, 2017.
[9] X. Bai, Y. Fang, W. Lin, L. Wang, and B. F. Ju, “Saliency-based defect detection in industrial images
by using phase spectrum,” IEEE Transactions on Industrial Informatics, vol. 10, pp. 2135–2145, Nov.
2014.
[10] Q. Li and S. Ren, “A visual detection system for rail surface defects,” in IEEE Transactions on Systems,
Man and Cypernetics, Part C:Applications and Reviews, vol. 42, no. 6, pp. 1531-1542, Nov. 2012.
[11] D. M. Tsai and J. Y. Luo, “Mean shift-based defect detection in multicrystalline solar wafer surfaces,”
IEEE Transactions on Industrial Informatics, vol. 7, pp. 125–135, Feb. 2011.
[12] A. Kumar and G. K. H. Pang, “Defect detection in textured materials using optimized filters,” IEEE
Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), vol. 32, pp. 553–570, Oct.
2002.
[13] H. Kong, H. C. Akakin, and S. E. Sarma, “A generalized laplacian of gaussian filter for blob detection
and its applications,” IEEE Transactions on Cybernetics, vol. 43, pp. 1719–1733, Dec. 2013.
[14] W. C. Li and D. M. Tsai, “Defect inspection in low-contrast lcd images using hough transform-based
nonstationary line detection,” IEEE Transactions on Industrial Informatics, vol. 7, pp. 136–147, Feb.
2011.
[15] Y. Park and I. S. Kweon, “Ambiguous surface defect image classification of amoled displays in smartphones,”
in IEEE Transactions on Industrial Informatics, vol. 12, pp. 597–607, Apr. 2016.
[16] D. Kang, D. Dhar, and A. B. Chan, “Crowd counting by adapting convolutional neural networks with
side information,” CoRR, vol. abs/1611.06748, 2016.
[17] J. Li, X. Liang, Y. Wei, T. Xu, J. Feng, and S. Yan, “Perceptual generative adversarial networks for
small object detection,” CoRR, vol. abs/1706.05274, 2017.
[18] P. H. O. Pinheiro and R. Collobert, “Weakly supervised semantic segmentation with convolutional
networks,” CoRR, vol. abs/1411.6228, 2014.
[19] M. Oquab, L. Bottou, I. Laptev, and J. Sivic, “Is object localization for free? - weakly-supervised
learning with convolutional neural networks,” in 2015 IEEE Conference on Computer Vision and
Pattern Recognition (CVPR), pp. 685–694, June 2015.
[20] I. T. Jolliffe, “Principal component analysis and factor analysis,” in Principal Component Analysis,
(New York, NY), pp. 115–128, Springer New York, 1986.
[21] P. Baldi, “Autoencoders, unsupervised learning, and deep architectures,” in Proceedings of ICML
Workshop on Unsupervised and Transfer Learning (I. Guyon, G. Dror, V. Lemaire, G. Taylor, and
D. Silver, eds.), vol. 27 of Proceedings of Machine Learning Research, (Bellevue, Washington, USA),
pp. 37–49, PMLR, July 2012.
[22] Y. Usui and K. Kondo, “The sift image feature reduction method using the histogram intersection kernel,”
in 2009 International Symposium on Intelligent Signal Processing and Communication Systems
(ISPACS), pp. 517–520, Jan 2009.
[23] J. Masci, U. Meier, and J. Cireşan, Danand Schmidhuber, “Stacked convolutional auto-encoders for
hierarchical feature extraction,” in Artificial Neural Networks and Machine Learning – ICANN 2011:
21st International Conference on Artificial Neural Networks, Espoo, Finland, June 14-17, 2011, Proceedings,
Part I (T. Honkela, W. Duch, M. Girolami, and S. Kaski, eds.), (Berlin, Heidelberg), pp. 52–
59, Springer Berlin Heidelberg, 2011.
[24] M.-L. Zhang and Z.-H. Zhou, “Ml-knn: A lazy learning approach to multi-label learning,” Pattern
Recognition, vol. 40, no. 7, pp. 2038 – 2048, 2007.
[25] K. P. Murphy, “Naive bayes classifiers,” University of British Columbia, 2006.
[26] S. S. Keerthi, S. K. Shevade, C. Bhattacharyya, and K. R. K. Murthy, “Improvements to platt’s smo
algorithm for svm classifier design,” Neural Computation, vol. 13, no. 3, pp. 637–649, 2001.
[27] S. R. Safavian and D. Landgrebe, “A survey of decision tree classifier methodology,” IEEE Transactions
on Systems, Man, and Cybernetics, vol. 21, pp. 660–674, May 1991.
[28] S. Bell and K. Bala, “Learning visual similarity for product design with convolutional neural networks,”
ACM Trans. Graph., vol. 34, pp. 98:1–98:10, July 2015.
[29] L. Leal-Taixé, C. Canton-Ferrer, and K. Schindler, “Learning by tracking: Siamese CNN for robust
target association,” CoRR, vol. abs/1604.07866, 2016.
[30] S. Chopra, R. Hadsell, and Y. LeCun, “Learning a similarity metric discriminatively, with application
to face verification,” in 2005 IEEE Computer Society Conference on Computer Vision and Pattern
Recognition (CVPR’05), vol. 1, pp. 539–546 vol. 1, June 2005.
[31] X. Hou, J. Harel, and C. Koch, “Image signature: Highlighting sparse salient regions,” IEEE Transactions
on Pattern Analysis and Machine Intelligence, vol. 34, pp. 194–201, Jan 2012.