本論文討論了智慧製造中的兩個主要問題:第一個是從多變量時間序列（multivariate time-series, MTS）資料中進行異常或不一致(discord)特徵序列的發現;第二個是如何在多變量時間序列從製程監控。Discord 可視為多變量時間序列資料中獨特呈現的異常子序列。在文獻中，偵知不一致子序列是一個非常具有挑戰性的問題，因為不一致子序列的模式、長度和位置通常是未知的。本研究提出了一種基於局部復發率 (local recurrence rate, LRR) 和robust k-means 分群演算法的方法 (LRR-RKMeans)。提出的LRR-RKMeans方法主要基於LRR和變化檢測演算法將MTS轉換和分割成監測信號的子序列，並基於robust k-means分群演算法加以分群以識別不一致的子序列。
所偵知之不一致子序列將可作為製程監控的指標。對於製程監控部分，本論文的研究重點是減少反應延遲，這也是檢測製程偏移的關鍵性能指標。在文獻中，己有研究人員引入了即時對比度管制圖 (real-time contrast control chart, RTC) 來快速檢測製程偏移。然而，大多數 RTC 在大數據環境中表現不佳；因此，本論文提出了一種新的RTC結合堆疊的長短期記憶網絡（ stacked long short-term memory network, SLSTM），稱為SLSTM-RTC，以監測具有短反應延遲的串流MTS。通過堆疊結構，所提出的 SLSTM-RTC 可以捕捉 MTS 內部的層次關係，以提高預測精度。此外，本論文也探討了 LSTM 網絡中的大規模優化問題，其中 LSTM網絡的預測精度在很大程度上取決於其大型網路權重的訓練情況。本研究在 LSTM 網絡內部整合了基於之群集共生有機搜索演算法 (clustering-based symbiotic organisms search, CSOS)，以進一步改進 SLSTM-RTC 中的優化策略，命名為 CSOS-LSTM。 CSOS 是一種在高維度優化問題上表現良好的啟發式演算法，通過其基於分群的局部全局搜索策略，以達到跳脫局部最優的狀況。在高維度合成數據集和四種真實案例中的一系列實驗：如心電圖資料、晶圓監測資料、葡萄酒和紙漿生產的傳感器數據，可發現所提出方法的優勢和本研究在推進智慧製造方法論上的貢獻。

This dissertation discusses two main issues in intelligent manufacturing. One is discord discovery or anomaly detection on multivariate time-series (MTS) data; another is process monitoring on MTS. Discord is considered as the anomalous subsequences that are uniquely presented in the MTS. In the literature, discord discovery is a very challenging problem because the pattern, length, and location of the discords are typically unknown. In this study, a new matrix-based search method for discord discovery based on local recurrence rate (LRR) and robust k-means clustering algorithm, named LRR-RKMeans, was proposed. The proposed LRR-RKMeans converts and segments the MTS into subsequences of monitoring signals based on LRR and change detection algorithm and identify the discords based on the robust k-means clustering algorithm. The detected discord in MTS can be used as the indicator when performing process monitoring. For process monitoring, this dissertation focuses on reducing the response delay, which is a critical performance measure for detecting the process shift. In the literature, researchers introduced the real-time contrast control chart (RTC) to fast detect the process shift. However, most of the RTCs are underperformed in the big data environment; therefore, this dissertation proposes a new RTC combined with a stacked long short-term memory network (SLSTM), named SLSTM-RTC, to monitor streaming MTS with short response delay. With the stacked structure, the proposed SLSTM-RTC can catch the hierarchical relations inside the MTS to improve the prediction accuracy. More on, this dissertation discusses the large-scale optimization issue in an LSTM network, where the prediction accuracy of an LSTM network is heavily dependent on how well its large network weights are trained. This study integrates the clustering-based symbiotic organisms search (CSOS) inside the LSTM network to further improve the optimization strategy in LSTM-RTC, named CSOS-LSTM, to solve the optimization issue. CSOS, a metaheuristic method that performs well for high-dimensional optimization problems, was utilized to escape the local-optimum by its cluster-based local-global search strategy. A series of experiments in high-dimensional synthetic datasets and four types of real-world cases: the electrocardiograph, sensors data for monitoring wafer, wine, and paper-and-pulp productions, highlights the advantages of the proposed methods and the dissertation importance on advancing the methodologies in intelligent manufacturing.

[1] J. Cheng, W. Chen, F. Tao, and C.­L. Lin, “Industrial iot in 5g environment towards smart manufacturing,”Journal of Industrial Information Integration, vol. 10, pp. 10–19, 2018.
[2] F. Tao and Q. Qi, “New it driven service­oriented smart manufacturing: Framework and characteristics,”IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 49, no. 1, pp. 81–91, 2019.
[3] P. Lade, R. Ghosh, and S. Srinivasan, “Manufacturing analytics and industrial internet of things,”IEEE Intelligent Systems, vol. 32, no. 3, pp. 74–79, 2017.
[4] E. Keogh, J. Lin, and A. Fu, “Hot sax: Efficiently finding the most unusual time series subsequence,” inThe Fifth IEEE International Conference on Data Mining, (1106352), pp. 226–233, IEEE Computer Society,2005.
[5] T. Huang, Y. Zhu, Y. Wu, and W. Shi, “J­distance discord: An improved time series discord definition and discovery method, 2015 IEEE International Conference on Data Mining Workshop (ICDMW), pp. 303–310,2015.
[6] A. L. Alfeo, M. G. Cimino, G. Manco, E. Ritacco, and G. Vaglini, “Using an autoencoder in the design of an anomaly detector for smart manufacturing,”Pattern Recognition Letters, vol. 136, pp. 272–278, 2020.
[7] M. Reis and G. Gins, “Industrial process monitoring in the big data/industry 4.0 era: from detection, to diagnosis, to prognosis,”Processes, vol. 5, no. 35, 2017.
[8] S. Yin, S. X. Ding, X. Xie, and H. Luo, “A review on basic data­driven approaches for industrial process monitoring,”IEEE Transactions on Industrial Electronics, vol. 61, no. 11, pp. 6418–6428, 2014.
[9] T. P. Carvalho, F. A. A. M. N. Soares, R. Vita, R. da P. Francisco, J. P. Basto, and S. G. S. Alcalá, “A systematic literature review of machine learning methods applied to predictive maintenance,” Computers & Industrial Engineering, vol. 137, no. 106024, 2019.
[10] Q. P. He and J. Wang, “Statistical process monitoring as a big data analytics tool for smart manufacturing,”Journal of Process Control, vol. 67, pp. 35–43, 2018.
[11] C.­L. Yang, H. Sutrisno, N.­W. Lo, Z.­X. Chen, C.­C. Wei, H.­W. Zhang, C.­T. Lin, C.­L. Wei, and S.­H.Hsieh, “Streaming data analysis framework for cyber­physical system of metal machining processes,” in 2018 IEEE Industrial Cyber­Physical Systems (ICPS), IEEE, 2018.
[12] P. Cortez, A. Cerdeira, F. Almeida, T. Matos, and J. Reis, “Modeling wine preferences by data mining from physicochemical properties,”Decision Support Systems, vol. 47, pp. 547–553, 2009.
[13] C. Ranjan, M. Reddy, M. Mustonen, K. Paynabar, and K. Pourak, “Dataset: Rare event classification in multivariate time series,”arXiv, 2018.
[14] A. Khodabakhsh, I. Ari, M. Bakír, and A. O. Ercan, “Multivariate sensor data analysis for oil refineries andmulti­mode identification of system behavior in real­time,”IEEE Access, vol. 6, pp. 64389–64405, 2018.
[15] T. T. El­Midany, M. A. El­Baz, and M. S. Abd­Elwahed, “A proposed framework for control chart pat­tern recognition in the multivariate process using artificial neural networks,”Expert Systems With Applications,vol. 37, no. 2, pp. 1035–1042, 2010.
[16] S. Şentürk, N. Erginel, ￿. Kaya, and C. Kahramand, “Fuzzy exponentially weighted moving average control chart for univariate data with a real case application,”Applied Soft Computing, vol. 22, pp. 1–10, 2014.68
[17] Z. Li, C. Zou, Z. Wang, and L. Huwang, “A multivariate sign chart for monitoring process shape parameters,”Journal of Quality Technology, vol. 45, no. 2, pp. 149–165, 2017.
[18] A. F. Silva, J. Vercruysse, C. Vervaet, J. P. Remon, J. A. Lopes, T. D. Beera, and M. C. Sarraguça, “Process monitoring and evaluation of a continuous pharmaceutical twin­screw granulation and drying process using multivariate data analysis,”European Journal of Pharmaceutics and Biopharmaceutics, vol. 128, pp. 36–47,2018.
[19] Z. Wang, D. Pi, and Y. Gao, “A novel unsupervised time series discord detection algorithm in aircraft engine gearbox,” inAdvanced Data Mining and Applications(G. Gan, B. Li, X. Li, and S. Wang, eds.), pp. 202–210,Springer International Publishing, 2018.
[20] M. Linardi, Yan Zhu, T. Palpanas, and E. Keogh, “Matrix profile goes mad: variable­length motif and discord discovery in data series,”Data Mining and Knowledge Discovery, vol. 34, p. 1022–1071, 2020.
[21] M. Hu, X. Feng, Z. Ji, K. Yan, and S. Zhou, “A novel computational approach for discord search with local recurrence rates in multivariate time series,”Information Sciences, vol. 477, pp. 220–233, 2019.
[22] C.­L. Yang, F. Darwin, and H. Sutrisno, “Local recurrence rates with automatic time windows for discord search in multivariate time series,”Procedia Manufacturing, vol. 39, pp. 1783–1792, 2019.
[23] J. Lei, T. Jiang, K. Wu, H. Du, G. Zhu, and Z. Wang, “Robust k­means algorithm with automatically splitting and merging clusters and its applications for surveillance data,”Multimedia Tools and Applications, vol. 75,p. 12043–12059, 2016.
[24] H. Sakoe and S. Chiba, “Dynamic programming algorithm optimization for spoken word recognition,”IEEETransactions on Acoustics, Speech, and Signal Processing, vol. 26, no. 1, pp. 43–49, 1978.
[25] Z. Li, C. Zou, Z. Gong, and Z. Wang, “The computation of average run length and average time to signal:An overview,”Journal of Statistical Computation and Simulation, vol. 84, no. 8, pp. 1779–1802, 2013.
[26] C. A. Lowry and D. C. Montgomery, “A review of multivariate control charts,”IIE Transactions, vol. 27,pp. 800–810, 1995.
[27] H. Deng, G. Runger, and E. Tuv, “System monitoring with real­time contrasts,”Journal of Quality Technol­ogy, vol. 44, no. 1, pp. 9–27, 2012.
[28] Q. Wei, W. Huang, W. Jiang, and W. Zhao, “Real­time process monitoring using kernel distances,” Interna­tional Journal of Production Research, vol. 54, no. 21, pp. 6563–6578, 2016.
[29] S. Jang, S. H. Park, and J.­G. Baek, “Real­time contrasts control chart using random forests with weighted voting,”Expert Systems With Applications, vol. 71, pp. 358–369, 2017.
[30] S. He, W. Jiang, and H. Deng, “A distance­based control chart for monitoring multivariate processes using support vector machines,”Ann Oper Res, vol. 263, pp. 191–207, 2018.
[31] F.­K. Wang, B. Bizuneh, and X.­B. Cheng, “One­sided control chart based on support vector machines with differential evolution algorithm,”Quality and Reliability Engineering International, pp. 1–12, 2019.
[32] H. Robbins and S. Monro, “A stochastic approximation method,”The annals of mathematical statistics,pp. 400–407, 1951.[33]J. Duchi, E. Hazan, and Y. Singer, “Adaptive subgradient methods for online learning and stochastic opti­mization,”Journal of Machine Learning Research, vol. 12, pp. 2121–2159, 2011.
[34] T. Tieleman and G. Hinton, “Lecture 6.5­rmsprop: Divide the gradient by a running average of its recent magnitude,” report, 2012.69
[35] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” 2014.
[36] T. Dozat, “Incorporating nesterov momentum into adam,” 2016.
[37] C.­L. Yang and H. Sutrisno, “A clustering-­based symbiotic organisms search algorithm for high­dimensionaloptimization problems,”Applied Soft Computing, vol. 97, p. 106722, 2020.
[38] J. Lin, E. Keogh, S. Lonardi, and B. Chiu, “A symbolic representation of time series, with implications for streaming algorithms,” DMKD ’03, (New York, NY, USA), pp. 2–11, Association for Computing Machinery,2003.
[39] X. Wang, J. Lin, N. Patel, and M. Braun, “Exact variable­length anomaly detection algorithm for univariate and multivariate time series,”Data Mining and Knowledge Discovery, vol. 32, pp. 1806–1844, 2018.
[40] W. Luo and M. Gallagher,Faster and Parameter­Free Discord Search in Quasi­ Periodic Time Series,vol. 6635 of Lecture Notes in Computer Science. Verlag Berlin Heidelberg: Springer, 2011.
[41] W. Luo, M. Gallagher, and J. Wiles, “Parameter­free search of time­series discord,”Journal of ComputerScience and Technology, vol. 28, no. 2, pp. 300–310, 2013.
[42] C. A. Lowry, W. H. Woodall, C. W. Champ, and S. E. Rigdon, “A multivariate exponentially weighted moving average control chart,”Technometrics, vol. 34, no. 1, pp. 46–53, 1992.
[43] J. Hu, G. Runger, and E. Tuv, “Tuned artificial contrasts to detect signals,”International Journal of Produc­tion Research, vol. 45, no. 23, pp. 5527–5534, 2007.[44] K.­S. Shin, I.­s. Lee, and J.­G. Baek, “An improved real­time contrasts control chart using novelty detection and variable importance,”Applied Sciences, vol. 9, no. 173, pp. 1–11, 2019.
[45] B. Zaman, M. Riaz, S. Ahmad, and S. A. Abbasi, “On artificial neural networking­based process monitor­ing under bootstrapping using runs rules schemes,”The International Journal of Advanced manufacturing technology, vol. 76, no. 1­4, pp. 311–327, 2015.
[46] J.­N. Pan, C.­I. Li, and M. Z. Lu, “Detecting the process changes for multivariate nonlinear profile data,”Quality and Reliability Engineering International, vol. 35, no. 6, pp. 1890–1910, 2019.
[47] K. Fukushima, “Neocognitron: A self­organizing neural network model for a mechanism of pattern recogni­tion unaffected by shift in position,”Biological Cybernetics, vol. 36, pp. 193–202, 1980.
[48] Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,”Nature, vol. 521, pp. 436–444, 2015.
[49] K. B. Lee, S. Cheon, and C. O. Kim, “A convolutional neural network for fault classification and diagnosis in semiconductor manufacturing processes,”IEEE Transactions on Semiconductor Manufacturing, vol. 30,no. 2, pp. 135–142, 2017.
[50] K. Wang, K. Li, L. Zhou, Y. Hu, Z. Cheng, J. Liu, and C. Chen, “Multiple convolutional neural networks for multivariate time series prediction,”Neurocomputing, vol. 360, pp. 107–119, 2019.
[51] Y. Chen, Y. Kang, Y. Chen, and Z. Wang, “Probabilistic forecasting with temporal convolutional neuralnetwork,”Neurocomputing, vol. 399, pp. 491–501, 2020.
[52] W. Chen and K. Shi, “A deep learning framework for time series classification using relative position matrix and convolutional neural network,”Neurocomputing, vol. 359, pp. 384–394, 2019.
[53] P. Schäfer, “The boss is concerned with time series classification in the presence of noise,”Data Mining and knowledge Discovery, vol. 29, pp. 1505–1530, 2015.70
[54] A. Bagnall, J. Lines, J. Hills, and A. Bostrom, “Time­series classification with cote: The collective of transformation­ based ensembles,”IEEE Transactions on Knowledge and Data Engineering, vol. 27, no. 9,pp. 2522–2535, 2015.
[55] C.­L. Yang, Z.­X. Chen, and C.­Y. Yang, “Sensor classification using convolutional neural network by en­coding multivariate time series as two­dimensional colored images,”Sensors, vol. 20, pp. 1–15, 2020.
[56] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large­scale image recognition,”arXivpreprint arXiv:1409.1556, 2014.
[57] J. Zhang, P. Wang, R. Yan, and R. X. Gao, “Long short­term memory for machine remaining life prediction,”Journal of Manufacturing Systems, vol. 48, pp. 78–86, 2018.
[58] P. Park, P. D. Marco, H. Shin, and J. Bang, “Fault detection and diagnosis using combined autoencoder andlong short­term memory network,”Sensors, vol. 19, p. 4612, 2019.
[59] A. Sagheer and M. Kotb, “Time series forecasting of petroleum production using deep lstm recurrent net­works,”Neurocomputing, vol. 323, pp. 203–213, 2019.
[60] H. Zhou, Y. Zhang, L. Yang, Q. Liu, K. Yan, and Y. Du, “Short-­term photovoltaic power forecasting based on long short term memory neural network and attention mechanism,”IEEE Access, vol. 7, pp. 78063–78074,2019.
[61] Y. Hua, Z. Zhao, R. Li, X. Chen, Z. Liu, and H. Zhang, “Deep learning with long short­term memory for time-series prediction,”IEEE Communications Magazine, vol. 57, no. 6, pp. 114–119, 2019.
[62] R. Yu, J. Gao, M. Yu, W. Lu, T. Xu, M. Zhao, J. Zhang, R. Zhang, and Z. Zhang, “Lstm­-efg for windpower forecasting based on sequential correlation features,”Future Generation of Computer Systems, vol. 93,pp. 33–42, 2019.
[63] J. Zheng, H. Zhang, Y. Dai, B. Wang, T. Zheng, Q. Li, Y. Liang, F. Zhang, and X. Song, “Time series prediction for output of multi­region solar power plants,”Applied Energy, vol. 257, p. 114001, 2020.
[64] R. Behmanesh and I. Rahimi, “Using combination of optimized recurrent neural network with design of ex­periments and regression for control chart forecasting,” in Business Engineering and Industrial Applications Colloquium, vol. 1, pp. 435–439, 2012.
[65] J. Xu, H. Lv, Z. Zhuang, Z. Lu, D. Zou, and W. Qin, “Control chart pattern recognition method based on improved one­dimensional convolutional neural network,” IFAC­PapersOnLine, vol. 52, no. 13, pp. 1537–1542, 2019.
[66] S. Chen and J. Yu, “Deep recurrent neural network­based residual control chart for autocorrelated processes,”Quality and Reliability Engineering International, vol. 35, no. 8, pp. 2687–2708, 2019.
[67] F. Aparisi and J. C. García­Díaz, “Design and optimization of ewma control charts for in­control, indifference,and out­of­control regions,”Computers & Operations Research, vol. 34, no. 7, pp. 2096–2108, 2007.
[68] M. Bashiri, A. Amiri, M. H. Doroudyan, and A. Asgari, “Multi­objective genetic algorithm for economic statistical design of x control chart,”Scientia Iranica, vol. 20, no. 3, pp. 909–918, 2013.
[69] S. Z. Ramadan, “Joint and control charts optimal design using genetic algorithm,”Mathematical Problems in Engineering, vol. 2018, 2018.
[70] Q. Wan, M. Zhu, and Y. Liu, “Monitoring the process mean using a synthetic x ￿control chart with two sampling intervals,”Journal of Intelligent & Fuzzy Systems, no. Preprint, pp. 1–13, 2020.
[71] T. J. Rato, P. Delgado, C. Martins, and M. S. Reis, “First principles statistical process monitoring of high­dimensional industrial microelectronics assembly processes,”Processes, vol. 8, no. 11, p. 1520, 2020.71
[72] R. J. Hyndman and G. Athanasopoulos,Forecasting: Principlesandpractice. Melbourne, Australia: OTexts,2018.
[73] Z. Aliniya and S. A. Mirroshandel, “A novel combinatorial merge­ split approach for automatic clustering using imperialist competitive algorithm,”Expert Systems with Applications, vol. 117, pp. 243–266, 2019.
[74] G. Xiu and Z. Zhao, “Sustainable development of port economy based on intelligent system dynamics,” IEEE Access, vol. 9, pp. 14070–14077, 2021.
[75] C. A. Ratanamahatana and E. Keogh, “Making time­series classification more accurate using learned con­straints,” in proceedings of the 2004 SIAM International Conference on Data Mining, pp. 11–22, 2004.
[76] M. Shokoohi­Yekta, B. Hu, H. Jin, J. Wang, and E. Keogh, “Generalizing dtw to the multi­dimensional case requires an adaptive approach,”Data Mining and Knowledge Discovery, vol. 31, no. 1, pp. 1–31, 2017.
[77] R. T. Olszewski, “Generalized feature extraction for structural pattern recognition in time­series data,” 2001.
[78] S. Hochreiter and J. Schmidhuber, “Long short­term memory,”Neural Computation, vol. 9, no. 8, pp. 1735–1780, 1997.
[79] N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: A simple way to prevent neural networks from overfitting,”Journal of Machine Learning Research, vol. 15, pp. 1929–1958,2014.
[80] R. Pascanu, C. Gulcehre, K. Cho, and Y. Bengio, “How to construct deep recurrent neural networks,”arXiv:1312.6026v5, 2014.
[81] Y. Gal and Z. Ghahramani, “Dropout as a bayesian approximation: Representing model uncertainty in deep learning,” in 2016 International Conference on Machine Learning, 2016.
[82] T. Fischer and C. Krauss, “Deep learning with long short­term memory networks for financial market pre­dictions,”European Journal of Operational Research, vol. 270, pp. 654–669, 2018.
[83] P. Virtanen, R. Gommers, T. E. Oliphant, M. Haberland, T. Reddy, D. Cournapeau, E. Burovski, P. Peterson,W. Weckesser, J. Bright, S. J. v. d. Walt, M. Brett, J. Wilson, K. J. Millman, N. Mayorov, A. R. J. Nelson,E. Jones, R. Kern, E. Larson, C. Carey, ￿. Polat, Y. Feng, E. W. Moore, J. VanderPlas, D. Laxalde, J. Perktold,R. Cimrman, I. Henriksen, E. Quintero, C. R. Harris, A. M. Archibald, A. H. Ribeiro, F. Pedregosa, P. v.Mulbregt, and S. . Contributors, “Scipy 1.0: Fundamental algorithms for scientific computing in python,”Nature Methods (in press), 2020.
[84] M. M. Najafabadi, F. Villanustre, T. M. Khoshgoftaar, N. Seliya, R. Wald, and E. Muharemagic, “Deep learning applications and challenges in big data analytics,”Journal of Big Data, vol. 2, 2015.
[85] J. G. A. Barbedo, “Impact of dataset size and variety on the effectiveness of deep learning and transfer learningfor plant disease classification,”Computers and Electronics in Agriculture, vol. 153, pp. 46–53, 2018.
[86] C. Zou, W. Jiang, and F. Tsung, “A lasso­ based diagnostic framework for multivariate statistical process control,”Technometrics, vol. 53, no. 3, pp. 297–309, 2011.
[87] C.­L. Yang and H. Sutrisno, “Reducing response delay in multivariate process monitoring by a stackedlong­short term memory network and real­time contrasts,”Computers & Industrial Engineering, vol. 153,p. 107052, 2021