本論文目的為預測非線性系統的時間序列，其中包括了位置與趨勢。為了達成此目的，對於現在正熱門的機器學習甚至是深度學習，有其應用與規劃。首先，我們選定了四種非線性系統：包括一個提出的四維渾沌系統、勞倫茲系統、杜芬振子、羅斯勒吸引子；之後，分別取各系統與各維度之時間序列數值，並記錄數值和趨勢；然後做預處理與假設檢定；以此建構三種學習架構，分別是產生表現模型、限制與控制模型和整體驗證模型；因此可將預處理完成的資料集分成訓練集與測試集；最後再以訓練集訓練出預測模型，並用測試集來判斷模型優劣，進而達成預測的目的。
預處理攸關訓練的正確性並能增加訓練前的確定性；模型的架構為達成學習目的與增加學習中參數迭代的相關性。對於非線性系統的預處理，採用的是集成學習中的K平均算法，對於預測離散的資料方便做分群，以利在後面機器學習的提升與袋裝學習能夠加速學習並避免過擬合。
因為整體學習機制為開放的架構，計算與連接上大多運用到統計學裡的常態分布與假定檢驗，所以主要兩架構為限制與控制模型和產生表現模型，相連的部分為密度層，原因在於將產生模型增加特徵維度的權重。所使用的算法是維度一的卷積神經網路，增加的強度使用的是狄利克雷分布函數。
最後使整體平均誤差(MAE)在整體維度的預定誤差內，且整筆測試資料的準確率在95%以上。沒有達到95%則選定最佳的方法和優化器。分析方式幾乎以直方圖、誤差表和標準差來界定模型的好壞。甚至是將此預處理步驟與整體學習模型架構試用到其他相關資料集。希望在預測未來的方向上能做改進。

In this thesis, we design a framework to predict the value and the trend of time series for nonlinear systems. In order to achieve this goal, there are applications and plans for machine learning and even deep learning that are currently popular. First, we selected four nonlinear systems: including a proposed four-dimensional chaotic system, Lorenz system, Duffing oscillator, and Rössler attractor. The framework has three learning parts as Long Short-Term Memory (LSTM) based Generate Performance Model (GPM), ensemble learning based Restrict and Control Model (RCM), and one-dimensional convolutional neural network (1-DCNN) based Overall Verification Model (OVM). Therefore, we exploit the training set to train the prediction model, and judge the model’s fitness through the testing set. So we can achieve the purpose of prediction. The improvement of machine learning and bag learning can accelerate the learning and avoid over-fitting.
Before learning steps, we exploited K-means method as pre-processing and hypothesis verification to improve the prediction accuracy. After learning steps, we construct four testing progresses as Point by Point Generated Method (PPGM), Sequential Full Generated Method (SFGM), Sequential Multiple Generated Method (SMGM), and Improvement RCM and OVM (IPRO) to predict the value of the time steps. The RCM is constructed by three ensemble learning methods as Gradient Boost Regression Method (GBRM), Random Forest Regression Method (RFRM), and Decision Tree Regression Method (DTRM), and it can select the important features for OVM. Therefore, the constructed dirichlet array is formed by OVM to achieve the enhancement for GPM.
Finally, the Mean Average Error (MAE) is within the predetermined error of the overall dimension, and the accuracy of the entire test data is above 95%. If it does not reach 95%, select the best method and the corresponding optimizer. The analysis method almost uses histogram, error table and standard deviation to define the quality of the model. Even this preprocessing step and the overall learning model architecture are tried to other related datasets. Hope to improve the predictive ability in the future direction.

[1] M.T. Yassen, “Adaptive control and synchronization of a modified Chua’s circuit system,” Applied Mathematics and Computation, vol. 135, no. 1, pp. 113-128, Feb. 2002, DOI: 10.1016/S0096-3003(01)00318-6.
[2] Ernesto P. Borges, Constantino Tsallis, Garin F.J Ananos, and Paulo Murilo C. de Oliverira, “Nonequilibrium probabilistic dynamics of the logistic map at the edge of chaos,” Physical Review Letters, vol. 89, no. 25, Dec. 2002, DOI: 10.1103/PhysRevLett.89.254103.
[3] Sun Junwei, Han Gaoyong, and Wang Yanfeng, “Dynamical Analysis of Memcapacitor Chaotic System and Its Image Encryption Application,” International Journal of Control Automation and Systems, vol. 18, no. 5, pp. 1242-1249, May. 2020, DOI: 10.1007/s12555-019-0015-7.
[4] Yu Fei, Li Lixiang, Tang Qiang, Cai Shuo, Song Yun, and Xu Quan, “A Survey on True Random Number Generators Based on Chaos,” Discrete Dynamics in Nature and Society, vol. 2019, Article ID 2545123, 10 pages, Dec. 2019, DOI: 10.1155/2019/2545123. Available: https://doi.org/10.1155/2019/2545123.
[5] Wen-Xu Wang, Ying-Cheng Lai, Celso Grebogi, “Data based identification and prediction of nonlinear and complex dynamical systems,” Physics Reports, vol. 644, pp. 1-76, Jul. 2016, DOI: 10.1016/j.physrep.2016.06.004.
[6] Jaideep Pathak, Zhixin Lu, Brian R. Hunt, Michelle Girvan, and Edward Ott, “Using Machine Learning to Replicate Chaotic Attractors and Calculate Lyapunov Exponents from Data,” Chaos, vol. 27, no. 12, Dec. 2017, DOI: 10.1063/1.5010300.
[7] Shinichi Oishi, “Numerical inclusion of exact periodic solutions for time delay Duffing equation,” Journal of Computational and Applied Mathematics, vol. 372, Jul. 2020, DOI: 10.1016/j.cam.2019.112620.
[8] Gökhan Yalnız, and Nazmi Burak Budanur, “Inferring symbolic dynamics of chaotic flows from persistence,” Chaos, vol. 30, no. 3, Mar. 2020, DOI: 10.1063/1.5122969.
[9] Jaideep Pathak, Alexander Wikner, Rebeckah Fussell, Sarthak Chandra, Brian R. Hunt, Michelle Girvan, and Edward Ott, “Hybrid Forecasting of Chaotic Processes: Using Machine Learning in Conjunction with a Knowledge-Based Model,” Chaos, vol. 28, no. 4, Apr. 2018, DOI: 10.1063/1.5028373.

[10] Helong Yu, Nannan Zhao, Pengjun Wang, Huiling Chen, Chengye Li, “Chaos-enhanced synchronized bat optimizer,” Applied Mathematical Modelling, vol. 77, no. 2, pp. 1201-1215, Jan. 2019, DOI: 10.1016/j.apm.2019.09.029.
[11] Tian Feng, Shuying Yang, and Feng Han, “Chaotic time series prediction using wavelet transform and multi-model hybrid method,” Vibroengineering, vol. 21, issue 7, pp. 1983-1999, Jul. 2019, DOI: https://doi.org/10.21595/jve.2019.20579.
[12] Holger Kantz and Thomas Schreiber, “Nonlinear Time Series Analysis,” Cambridge University Press, UK. 2nd ed., 2004.
[13] Martin Casdagli, “Nonlinear prediction of chaotic time series,” Physica D, vol. 35, no. 3, pp. 335-356, May. 1989, DOI: 10.1016/0167-2789(89)90074-2.
[14] Thao-Tsen Chen and Shie-Jue Lee, “A weighted LS-SVM based learning system for time series forecasting,” Information Sciences, vol. 299, pp. 99-116, Apr. 2015, DOI: 10.1016/j.ins.2014.12.031.
[15] Chih-Chiang Wei, “Evaluation of Photovoltaic Power Generation by Using Deep Learning in Solar Panels Installed in Buildings,” Energies, vol. 12, no. 18, Sep. 2019, DOI: 10.3390/en12183564.
[16] Xibin DONG, Zhiwen YU, Wenming CAO, Yifan SHI, and QianliMA, “A survey on ensemble learning,” Frontiers of Computer Science, vol. 14, no. 2, Apr. 2020, DOI: 10.1007/s11704-019-8208-z.
[17] Diego Castillo-Barnes, Li Sub, Javier Ramírez, Diego Salas-Gonzalez, Francisco J. Martinez-Murcia, Ignacio A. Illan, Fermin Segovia, Andres Ortiz, Carlos Cruchaga, Martin R. Farlow, Chengjie Xiong, Neil R. Graff-Radford, Peter R. Schofield, Colin L. Masters, Stephen Salloway, Mathias Jucker, Hiroshi Mori, Johannes Levin, and Juan M. Gorriz, “Autosomal dominantly inherited alzheimer disease Analysis of genetic subgroups by machine learning,” Information Fusion, vol. 58, pp. 153-167, Jun. 2020, DOI: 10.1016/j.inffus.2020.01.001.
[18] XU Dong-wei, WANG Yong-dong, JIA Li-min, ZHANG Gui-jun, and GUO Hai-feng, “Real-time road traffic states estimation based on kernel-KNN matching of road traffic spatial characteristics.” Journal of Central South University, vol. 23, no. 9, pp. 2453-2464, Sep. 2016, DOI: 10.1007/s11771-016-3304-9.
[19] Shaodong Zheng, Jinsong Zhao, “A new unsupervised data mining method based on the stacked autoencoder for chemical process fault diagnosis,” Computers & Chemical Engineering, vol. 135, Apr. 2020, DOI: 10.1016/j.compchemeng.2020.106755.

[20] Cong Guo, Yu-Jie Xie, Meng-Ting Zhu, Qian Xiong, Yi Chen, Qiang Yu, and Jian-Hua Xie, “Influence of different cooking methods on the nutritional and potentially harmful components of peanuts,” Food Chemistry, vol. 316, Jun. 2020, DOI: 10.1016/j.foodchem.2020.126269.
[21] Wenyou Gao and Chang Su, “Analysis of earnings forecast of blockchain financial products based on particle swarm optimization,” Journal of Computational and Applied Mathematics, vol. 372, Jul. 2020, DOI: 10.1016/j.cam.2020.112724.
[22] Zhang Chu, Wu Wenyan, Zhou Lei, Cheng Huan, Ye Xingqian, and He Yong, “Developing deep learning based regression approaches for determination of chemical compositions in dry black goji berries (Lycium ruthenicum Murr.) using near-infrared hyperspectral imaging,” Food Chemistry, vol. 319, Jul. 2020, DOI: 10.1016/j.foodchem.2020.126536.
[23] Xin Yao and Cho-Li Wang, “Probabilistic Consistency Guarantee in Partial Quorum-Based Data Store,” IEEE Transactions on Parallel and Distributed Systems, vol. 31, no. 8, pp. 1815-1827, Aug. 2020, DOI: 10.1109/TPDS.2020.2973619.
[24] Michael Hauser, Yiwei Fu, Shashi Phoha, and Asok Ray, “Neural Probabilistic Forecasting of Symbolic Sequences With Long Short-Term Memory,” Journal of Dynamic Systems Measurement and Control-Transactions of the ASME, vol. 140, no. 8, Aug. 2018, DOI: 10.1115/1.4039281.
[25] John Duchi, Elad Hazan, and Yoram Singer, “Adaptive Subgradient Methods for Online Learning and Stochastic Optimization,” Journal of Machine Learning Research, vol. 12, Jul. 2011.
[26] Timothy Dozat, “Incorporating Nesterov Momentum into Adam,” ICLR. Computer Science, Mar. 2016.
[27] Diederik P. Kingma, and Jimmy Lei Ba, “Adam: A Method for Stochastic Optimization,” ICLR. Machine Learning, Jul. 2015.
[28] Matthew D. Zeiler, “Adadelta: An adaptive learning rate method,” Google Inc. Machine Learning, Dec. 2012.
[29] Alex Graves, “Generating Sequences with Recurrent Neural Networks,” Neural and Evolutionary Computing, Jun. 2014.
[30] Kelly Van Lancker, An Vandebosch, and Stijn Vansteelandt, “Improving interim decisions in randomized trials by exploiting information on short-term endpoints and prognostic baseline covariates,” Pharmaceutical Statistics, Apr. 2019, DOI: 10.1002/pst.2014.
[31] Weiming Wu, Cong Wang, and Chengzhi Yuan, “Deterministic learning from sampling data,” Neurocomputing, vol. 358, pp. 456-466, Sep. 2019, DOI: 10.1016/j.neucom.2019.05.044.
[32] Kasun Bandara, Christoph Bergmeir, and Slawek Smyl, “Forecasting across time series databases using recurrent neural networks on groups of similar series: A clustering approach,” Expert Systems With Applications, vol. 140, Feb. 2019, DOI: 10.1016/j.eswa.2019.112896.
[33] Lev V. Utkin, “An imprecise deep forest for classification,” Expert Systems With Applications, vol. 141, Mar. 2020, DOI: 10.1016/j.eswa.2019.112978.
[34] Rahul Kumar Agrawal, Frankle Muchahary, and Madan Mohan Tripathi, “Ensemble of relevance vector machines and boosted trees for electricity price forecasting,” Applied Energy, vol. 250, pp. 540-548, Sep. 2019, DOI: 10.1016/j.apenergy.2019.05.062.
[35] Christopher Krauss, Xuan Anh Do, and Nicolas Huck, “Deep neural networks, gradient-boosted trees, random forests: Statistical arbitrage on the S&P 500,” European Journal of Operational Research, vol. 259, no. 2, pp. 689-702, Jun. 2017, DOI: 10.1016/j.ejor.2016.10.031.
[36] Junho Lee, Wu Wang, Fouzi Harrou, and Ying Sun, “Reliable solar irradiance prediction using ensemble learning-based models: A comparative study,” Energy Conversion and Management, vol. 208, Mar. 2020, DOI: 10.1016/j.enconman.2020.112582.
[37] Yunzhe Liu and Tao Cheng, “Understanding public transit patterns with open geodemographics to facilitate public transport planning,” Transportmetrica A-transport Science, vol. 16, no. 1, Dec. 2020, DOI: 10.1080/23249935.2018.1493549.
[38] Ye Zhang and Rongfang Gong, “Second order asymptotical regularization methods for inverse problems in partial differential equations,” Journal of Computational and Applied Mathematics, vol. 375, Sep. 2020, DOI: 10.1016/j.cam.2020.112798.
[39] Arash Mohamadi, Majid Shahgholi, and Faramarz Ashenai Ghasemi, “Free vibration and stability of an axially moving thin circular cylindrical shell using multiple scales method,” Meccanica, vol. 54, no. 14, pp. 2227-2246, Nov. 2019, DOI: 10.1007/s11012-019-01062-8.
[40] François-XavierDupé, and SandrineAnthoine, “Generalized greedy alternatives,” Applied and Computational Harmonic Analysis, vol. 49, no. 1, pp. 1-29, Jul. 2020, DOI: 10.1016/j.acha.2018.10.005.

[41] Francesco Da Ros, Stenio M. Ranzini,Henning Bülow , and Darko Zibar, “Reservoir-Computing Based Equalization With Optical Pre-Processing for Short-Reach Optical Transmission,” IEEE Journal of Selected Topics In Quantum Electronics, vol. 26, no. 5, Sep. 2020, DOI: 10.1109/JSTQE.2020.2975607.
[42] Uras Mutlu and Ethem Alpaydin, “Training bidirectional generative adversarial networks with hints,” Pattern Recognition, vol. 103, Jul. 2020, DOI: 10.1016/j.patcog.2020.107320.
[43] Cheng-Hsiung Yang, Hou-Cheng Wu, and Shun-Feng Su, “Implementation of Encryption Algorithm and Wireless Image Transmission System on FPGA,” IEEE Access, vol. 7, Mar. 2019, DOI: 10.1109/ACCESS.2019.2910859.
[44] 吳厚呈，”加密演算法和無線圖像傳輸系統之FPGA實現”。國立臺灣科技大學自動化與控制所碩士論文，台北市大安區基隆路四段43號。一零七年七月。
[45] 沈慧瑜，”基於深度學習神經網路對渾沌時間序列分析與預測”。 國立臺灣科技大學自動化與控制所碩士論文，台北市大安區基隆路四段43號。一零八年七月。
[46] Robert Tibshirani, Guenther Walther, and Trevor Hastie, “Estimating the number of clusters in a data set via the gap statistic,” Journal of the Royal Statistical Society Series B-statistical Methodology, vol. 63, pp. 411-423, Feb. 2001, DOI: 10.1111/1467-9868.00293.
[47] Zhu Liya, Huan sheng Song, Zhang Xi, Yan Maode, Zhang Tao, Wang Xiaoyan, and Xu Juan, “A robust meaningful image encryption scheme based on block compressive sensing and SVD embedding,” Signal Processing, vol. 175, Oct. 2020, DOI: 10.1016/j.sigpro.2020.107629.
[48] Jalal Mostafa, Grasley Zachary, Gurganus Charles, and Bullard Jeffrey W, “Experimental investigation and comparative machine-learning prediction of strength behavior of optimized recycled rubber concrete,” Construction and Building Materials, vol. 256, Sep. 2020, DOI: 10.1016/j.conbuildmat.2020.119478.
[49] Viet-Hung Truong, Quang-Viet Vu, Huu-Tai Thai, and Manh-Hung Ha, “A robust method for safety evaluation of steel trusses using Gradient Tree Boosting algorithm,” Advances in Engineering Software, vol. 147, Sep. 2020, DOI: 10.1016/j.advengsoft.2020.102825.
[50] Yu Zhuoxi, Qin Lu, Chen Yunjing, and Parmar Milan Deepak, “Stock price forecasting based on LLE-BP neural network model,” Physica A-statistical Mechanics and Its Applications, vol. 553, Sep. 2020, DOI: 10.1016/j.physa.2020.124197.
[51] Li Jiu, Zang Hongyan, and Wei Xinyuan, “On the construction of one-dimensional discrete chaos theory based on the improved version of Marotto's theorem,” Journal of Computational and Applied Mathematics, vol. 380, Dec. 2020, DOI: 10.1016/j.cam.2020.112952.
[52] J. Escrig, E. Woolley, A. Simeone, and N. J. Watson, “Monitoring the cleaning of food fouling in pipes using ultrasonic measurements and machine learning,” Food Control, vol. 116, Oct. 2020. DOI: 10.1016/j.foodcont.2020.107309.
[53] Wang Liang, Tripathi Yogesh Mani, Wu Shuo-Jye, and Zhang Meng, “Inference for confidence sets of the generalized inverted exponential distribution under k-record values,” Journal of Computational and Applied Mathematics, vol. 380, Dec. 2020, DOI: 10.1016/j.cam.2020.112969.
[54] Khaled Alzaareera, Maarouf Saada, Hasan Mehrjerdib, Claude Ziad El-Bayeha, Dalal Asberc, and Serge Lefebvre, “A new sensitivity approach for preventive control selection in real-time voltage stability assessment,” International Journal of Electrical Power & Energy Systems, vol. 112, Nov. 2020, DOI: 10.1016/j.ijepes.2020.106212.