隨著科技進步在印刷電路板(PCB)上的零件的檢驗上有著新的技術，過往的自動化光學檢測都是透過利用較傳統的影像演算法來進行瑕疵的偵測，如今人工智慧蓬勃發展，許多的自動化光學檢測都會導入AI的技術進行瑕疵檢驗與異常偵測，且所使用的模型趨勢都往非監督式學習的方式，透過非監督式學習的模型可以減少人工對於要訓練的影像進行標註。
本研究主要是透使用變分自編碼器(VAE)這個生成模型來建立異常偵測模型，來判斷PCB上電子零件是否異常，VAE是一個重建式的非監督式學習的模型，透過正常樣本進行訓練，計算出輸入與輸出的影像差異，來判定是否屬於異常，過往VAE的影像生成品質較不穩定，所以本研究提出將自注意力機制層嵌入到VAE使得最終重建的影像品質有所提升，並透過較為敏銳的指標結構相似性來進行判別是否異常，最終提升異常偵測的結果，來協助現場檢驗的人員減少檢驗的時間。
由於本研究的數據中異常數據都集中在部分零件當中，所以本研究只針對部分的零件進行異常偵測，最終透過假設檢定後本研究所使用的方法相對於在原始的變分自動編碼器，在影像的成像以及異常偵測的結果上有較好的成果，在公開數據集中也具有相同的結果，從實驗證明中過往VAE內部使用卷積層都是透過局部的進行卷積核來進行計算缺乏全局的相關資訊，本研究透過加入自注意力機制後可以獲得全局的相關資訊進而改善影像生成的品質與異常偵測的品質。

With the advancement of technology, there are new techniques for component inspection on printed circuit boards (PCB). In the past, automated optical inspection relied on conventional image algorithms for defect detection. However, with the rapid development of artificial intelligence, many automated optical inspection systems now incorporate AI technologies for defect detection and anomaly detection. Moreover, the trend in these systems is moving towards unsupervised learning approaches, where models can be trained without the need for labeled images.
This study focuses on using a VAE, a generative model, to build an anomaly detection model for determining abnormalities in electronic components on PCB boards. VAE is a reconstruction-based unsupervised learning model trained on normal samples. It calculates the difference between the input and output images to determine whether an anomaly is present. Previous VAE exhibited unstable image generation quality. To address this, the study proposes embedding a self-attention mechanism layer into the VAE to improve the quality of the reconstructed images. Additionally, a more sensitive metric, structural similarity, is used to discriminate anomalies, ultimately enhancing the anomaly detection results and reducing inspection time for field personnel. As the abnormal data is concentrated in specific components, anomaly detection is performed only on those parts. The proposed method outperforms the original VAE in terms of image reconstruction and anomaly detection, even with publicly available datasets. The study shows that previous variational encoders lacked global contextual information due to local convolutions. Incorporating the self-attention mechanism provides this contextual information, leading to improved image generation and anomaly detection.

Aggarwal, C. C., & Aggarwal, C. C. (2017). An introduction to outlier analysis. Springer.
Albawi, S., Mohammed, T. A., & Al-Zawi, S. (2017). Understanding of a convolutional neural network. 2017 international conference on engineering and technology (ICET),
An, J., & Cho, S. (2015). Variational autoencoder based anomaly detection using reconstruction probability. Special lecture on IE, 2(1), 1-18.
Bank, D., Koenigstein, N., & Giryes, R. (2020). Autoencoders. arXiv preprint arXiv:2003.05991.
Bello, I., Zoph, B., Vaswani, A., Shlens, J., & Le, Q. V. (2019). Attention augmented convolutional networks. Proceedings of the IEEE/CVF international conference on computer vision,
Bergmann, P., Fauser, M., Sattlegger, D., & Steger, C. (2019). MVTec AD--A comprehensive real-world dataset for unsupervised anomaly detection. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition,
Blei, D. M., Kucukelbir, A., & McAuliffe, J. D. (2017). Variational inference: A review for statisticians. Journal of the American statistical Association, 112(518), 859-877.
Bond-Taylor, S., Leach, A., Long, Y., & Willcocks, C. G. (2021). Deep generative modelling: A comparative review of vaes, gans, normalizing flows, energy-based and autoregressive models. IEEE transactions on pattern analysis and machine intelligence.
Bos, C. S. (2002). A comparison of marginal likelihood computation methods. Compstat: Proceedings in Computational Statistics,
Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., & Askell, A. (2020). Language models are few-shot learners. Advances in neural information processing systems, 33, 1877-1901.
Campbell, C., & Ahmad, F. (2020). Attention-augmented convolutional autoencoder for radar-based human activity recognition. 2020 IEEE International Radar Conference (RADAR),
Casale, F. P., Dalca, A., Saglietti, L., Listgarten, J., & Fusi, N. (2018). Gaussian process prior variational autoencoders. Advances in neural information processing systems, 31.
Chandola, V., Banerjee, A., & Kumar, V. (2009). Anomaly detection: A survey. ACM computing surveys (CSUR), 41(3), 1-58.
Channappayya, S. S., Bovik, A. C., Caramanis, C., & Heath, R. W. (2008). SSIM-optimal linear image restoration. 2008 IEEE International Conference on Acoustics, Speech and Signal Processing,
Chen, T., Liu, H., Ma, Z., Shen, Q., Cao, X., & Wang, Y. (2021). End-to-end learnt image compression via non-local attention optimization and improved context modeling. IEEE Transactions on Image Processing, 30, 3179-3191.
Cui, Y., Liu, Z., & Lian, S. (2022). A survey on unsupervised industrial anomaly detection algorithms. arXiv preprint arXiv:2204.11161.
Dan, H., Azuma, T., & Kohyama, K. (2007). Characterization of spatiotemporal stress distribution during food fracture by image texture analysis methods. Journal of food engineering, 81(2), 429-436.
Dasgupta, D., & Nino, F. (2000). A comparison of negative and positive selection algorithms in novel pattern detection. Smc 2000 conference proceedings. 2000 ieee international conference on systems, man and cybernetics.'cybernetics evolving to systems, humans, organizations, and their complex interactions'(cat. no. 0,
Donahue, E. A., Quach, T.-T., Potter, K., Martinez, C., Smith, M., & Turner, C. D. (2019). Deep learning for automated defect detection in high-reliability electronic parts. Applications of Machine Learning,
Dosselmann, R., & Yang, X. D. (2011). A comprehensive assessment of the structural similarity index. Signal, Image and Video Processing, 5, 81-91.
Edmunds, R., & Feinstein, E. (2017). Deep semi-supervised embeddings for dynamic targeted anomaly detection.
Elmrabit, N., Zhou, F., Li, F., & Zhou, H. (2020). Evaluation of machine learning algorithms for anomaly detection. 2020 international conference on cyber security and protection of digital services (cyber security),
Fridman, Y., Rusanovsky, M., & Oren, G. (2021). ChangeChip: A reference-based unsupervised change detection for PCB defect detection. 2021 IEEE Physical Assurance and Inspection of Electronics (PAINE),
Fujimaki, R., Yairi, T., & Machida, K. (2005). An approach to spacecraft anomaly detection problem using kernel feature space. Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining,
Gao, Y., Lu, W., Si, X., & Lan, Y. (2020). Deep model-based semi-supervised learning way for outlier detection in wireless capsule endoscopy images. IEEE Access, 8, 81621-81632.
Guo, F., & Guan, S.-a. (2011). Research of the machine vision based PCB defect inspection system. 2011 International Conference on Intelligence Science and Information Engineering,
Hadji, I., & Wildes, R. P. (2018). What do we understand about convolutional networks? arXiv preprint arXiv:1803.08834.
He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. Proceedings of the IEEE conference on computer vision and pattern recognition,
Hinton, G. E., & Salakhutdinov, R. R. (2006). Reducing the dimensionality of data with neural networks. science, 313(5786), 504-507.
Hu, W., Liao, Y., & Vemuri, V. R. (2003). Robust Support Vector Machines for Anomaly Detection in Computer Security. ICMLA,
ITS, C. (2004). Aoi testing positions in comparison.
Joyce, J. M. (2011). Kullback-leibler divergence. In International encyclopedia of statistical science (pp. 720-722). Springer.
Kingma, D. P., & Welling, M. (2013). Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114.
Kliger, M., & Fleishman, S. (2018). Novelty detection with gan. arXiv preprint arXiv:1802.10560.
Kwon, H.-J., Shin, D.-H., & Chung, K. (2021). PGGAN-based anomaly classification on chest X-ray using weighted multi-scale similarity. IEEE Access, 9, 113315-113325.
Kwon, H., & Nasrabadi, N. M. (2005). Kernel RX-algorithm: A nonlinear anomaly detector for hyperspectral imagery. IEEE transactions on Geoscience and Remote Sensing, 43(2), 388-397.
LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278-2324.
Li, J. (2022). Data Augmentation Based on Contrastive Learning in Audio Anomaly Detection. Journal of Physics: Conference Series,
Li, X., Yu, S., Chen, S., & Zhao, L. (2022). Normalizing Flow-Based Probability Distribution Representation Detector for Hyperspectral Anomaly Detection. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 15, 4885-4896.
Li, Z., Liu, F., Yang, W., Peng, S., & Zhou, J. (2021). A survey of convolutional neural networks: analysis, applications, and prospects. IEEE transactions on neural networks and learning systems.
Liao, Y., & Vemuri, V. R. (2002). Use of k-nearest neighbor classifier for intrusion detection. Computers & security, 21(5), 439-448.
Liu, F., Ma, X., Wu, J., Yang, J., Xue, S., Beheshti, A., Zhou, C., Peng, H., Sheng, Q. Z., & Aggarwal, C. C. (2022). DAGAD: Data Augmentation for Graph Anomaly Detection. arXiv preprint arXiv:2210.09766.
Liu, J., Song, K., Feng, M., Yan, Y., Tu, Z., & Zhu, L. (2021). Semi-supervised anomaly detection with dual prototypes autoencoder for industrial surface inspection. Optics and Lasers in Engineering, 136, 106324.
Lu, Y., & Xu, P. (2018). Anomaly detection for skin disease images using variational autoencoder. arXiv preprint arXiv:1807.01349.
Lu, Y., Zheng, X., Xin, H., Tang, H., Wang, R., & Nie, F. (2023). Ensemble and random collaborative representation-based anomaly detector for hyperspectral imagery. Signal Processing, 204, 108835.
Mi, L., Shen, M., & Zhang, J. (2018). A probe towards understanding gan and vae models. arXiv preprint arXiv:1812.05676.
Mikołajczyk, A., & Grochowski, M. (2018). Data augmentation for improving deep learning in image classification problem. 2018 international interdisciplinary PhD workshop (IIPhDW),
Muniyandi, A. P., Rajeswari, R., & Rajaram, R. (2012). Network anomaly detection by cascading k-Means clustering and C4. 5 decision tree algorithm. Procedia Engineering, 30, 174-182.
Myung, I. J. (2003). Tutorial on maximum likelihood estimation. Journal of mathematical Psychology, 47(1), 90-100.
Patil, P., Holambe, R., & Waghmare, L. (2022). An Attention Augmented Convolution based Improved Residual UNet for Road Extraction. In: May.
Primartha, R., & Tama, B. A. (2017). Anomaly detection using random forest: A performance revisited. 2017 International conference on data and software engineering (ICoDSE),
Rabinovich, A., Vedaldi, A., Galleguillos, C., Wiewiora, E., & Belongie, S. (2007). Objects in context. 2007 IEEE 11th International Conference on Computer Vision,
Racah, E., Beckham, C., Maharaj, T., Ebrahimi Kahou, S., Prabhat, M., & Pal, C. (2017). Extremeweather: A large-scale climate dataset for semi-supervised detection, localization, and understanding of extreme weather events. Advances in neural information processing systems, 30.
Reshadat, V., & Kapteijns, R. A. (2021). Improving the performance of automated optical inspection (AOI) using machine learning classifiers. 2021 International Conference on Data and Software Engineering (ICoDSE),
Rezende, D., & Mohamed, S. (2015). Variational inference with normalizing flows. International conference on machine learning,
Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.
Soille, P. (1999). Morphological image analysis: principles and applications (Vol. 2). Springer.
Souza, R., Rittner, L., & Lotufo, R. (2014). A comparison between k-optimum path forest and k-nearest neighbors supervised classifiers. Pattern recognition letters, 39, 2-10.
Su, H., Wu, Z., Du, Q., & Du, P. (2018). Hyperspectral anomaly detection using collaborative representation with outlier removal. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 11(12), 5029-5038.
Tang, W., Vian, C. M., Tang, Z., & Yang, B. (2021). Anomaly detection of core failures in die casting X-ray inspection images using a convolutional autoencoder. Machine Vision and Applications, 32(4), 102.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems, 30.
Walsh, D., & Arena, J. (2001). Seeing AOI for Six Sigma PCB Manufacturing. ELECTRONIC PACKAGING AND PRODUCTION-US EDITION-, 41(9), 46-53.
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004). Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4), 600-612.
Widodo, A., & Yang, B.-S. (2007). Support vector machine in machine condition monitoring and fault diagnosis. Mechanical systems and signal processing, 21(6), 2560-2574.
Xu, H., Chen, W., Zhao, N., Li, Z., Bu, J., Li, Z., Liu, Y., Zhao, Y., Pei, D., & Feng, Y. (2018). Unsupervised anomaly detection via variational auto-encoder for seasonal kpis in web applications. Proceedings of the 2018 world wide web conference,
Zhang, H., Goodfellow, I., Metaxas, D., & Odena, A. (2019). Self-attention generative adversarial networks. International conference on machine learning,
Zimmerer, D., Petersen, J., & Maier-Hein, K. (2019). High-and Low-level image component decomposition using VAEs for improved reconstruction and anomaly detection. arXiv preprint arXiv:1911.12161.