多元時間序列(Multivariate time series,MTS)資料十分常見於現實生活中，由於大多數問題不只考慮單一變量，而是多個變量影響一個類別標籤(Label)，因此如何有效地解決多元時間序列分類問題成為本次研究方向。近年來隨著人工智慧(Artificial Intelligence,AI)快速的發展，嘗試深度學習框架處理多元時間序列分類問題。
本研究提出一解決多元時間序列分類問題之方法，藉由時間序列擁有趨勢之特性，將多元時間序列資料中，每一個變數所包含的時間序列利用迴歸分析找出其迴歸方程式，並利用迴歸係數與截距項進行分群，使得擁有相似趨勢之時間序列被分在相同一群，並根據過往文獻所發表的四種時間序列轉換圖像之方法，根據分群結果，將具有相似趨勢的時間序列，使用同一種時間序列轉換為圖像方法進行轉換，並實驗多種時間序列轉換圖像方法組合，決定每一群時間序列的轉圖方法。依照上述方法將多元時間序資料列轉換為圖像後，將輸入每一筆資料到3D卷積神經網路進行特徵擷取與圖像辨識，能有效解決多元時間序列分類問題，找出最佳的分類準確度。

Multivariate time series data is very common in real life. Since most problems not only consider a single variable, but also multiple variables affect the label, how to effectively solve the problem of multivariate time series classification remain a major problem in research. In recent years, with the rapid development of Artificial Intelligence (AI), the deep learning framework has been tried to deal with multivariate time series classification problems.
This study proposes a method to solve the problem of MTS classification. The multivariate time series data is used to find the regression equation by regression analysis. We use the regression coefficient and intercept to the cluster so that the time series with similar trends are divided into the same cluster, and the literature proposes to the four frameworks to encode time series data as different types of images. According to the clustering results, the time series with similar trends will be used the same method to encode time series into images and try a variety of experiment to determine encoding method for each cluster of time series. After encoding multivariate time series data as images according to the above method, each data is input into the 3D convolutional neural networks for feature extraction and image recognition, which can effectively solve the multivariate time series classification problem and find the best classification accuracy.

Ali, S., Mehmood, C., Khawja, A., Nasim R., Jawad, M., Usman, S., Khan, S., Salahuddin, S., & Ihsan, M.(2014). Statistical weather data analysis for wide area smart grid operations, in IEEE International Conference on Electro/Information Technology (EIT),doi:10.1109/EIT.2014.6871808.

Bartlett, J., Prabhu, V., & Whaley, J. (2017). Acctionnet: A dataset of human activity recognition using on-phone motion sensors. In Proceedings of the 34th International Conference on Machine Learning.

Bhola, R., Krishna, N. H., Ramesh, K., Senthilnath, J., & Anand, G.(2018). Detection of the power lines in UAV remote sensed images using spectral-spatial methods, J. Environ. Manage., vol. 206, pp. 1233–1242.

Braun, M.R., Altan, H., & Beck, S.B.M.(2014).Using regression analysis to predict
the future energy consumption of a supermarket in the UK Appl. Energy, pp. 305-313.
Byeon, Y.H., Kwak, K.C. (2014). Facial expression recognition using 3d convolutional neural network. Int J Adv Comput Sci Appl 5(12):107–112.

Campanharo, A. S., Sirer, M. I., Malmgren, R. D., Ramos, F. M,. & Amaral, L. A. N. (2011). Duality between time series and networks. PloS one 6(8):e23378.

Eckmann, J. P., Oliffson Kamphorst, S., & Ruelle, D.(1987).Recurrence plots of dynamical systems, Europhys. Lett. 4, 973.

Esmaeilzadeh, S., Belivanis.D.I., Pohl, K.M., & Adeli, E.(2018).End-To-End Alzheimer’s Disease Diagnosis and Biomarker Identification. MICCAI arXiv:1810.00523v1.

Gamboa,J.(2017). Deep Learning for Time-Series Analysis. University of Kaiserslautern, Kaiserslautern, Germany, arXiv:1701.01887.

Hartigan, J.A., & Wong, M.A.(1979). Algorithm AS136: A k-means Clustering Algorithm, Applied Statistics, vol. 28, pp. 100-108.

Hatami, N., Gavet, Y., & Debayle, J. (2017) .Classification of time-series images using deep convolutional neural networks. In: International Conference on Machine Vision.

Ismail Fawaz, H ., Forestier, G., Weber, J., Idoumghar, L.,& Muller, P.-A.( 2018). Deep learning for time series classification: a review, ArXiv. :1809.04356v2.

Ji, S., Xu, W., Yang, M.,& Yu, K.(2013). 3D convolutional neural networks for human action recognition. PAMI, 35(1):221– 231.
Jia, B., Pham, K., Blasch, E., Wang, Z., Shen, D., & Chen, G. (2018). Space Object Classification Using Deep Neural Networks, IEEE Aerospace Conf.

Kim, Y. (2014). Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882.

Krizhevsky, A.(2010). Convolutional deep belief networks on cifar-10. Unpublished manuscript.

Krizhevsky, A., Sutskever, I., & Hinton, G.E.( 2012). Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pp. 1097–1105.

LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989). Backpropagation Applied to Handwritten Zip Code Recognition. Neural Computation, 1(4):541–551.

LeCun, Y., Bottou, L., Bengio Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324.

LeCun, Y., Bengio, Y., & Hinton G.(2015). Deep learning. Nature 521, 436–444 doi:10.1038/nature14539 pmid:2601744.

Li, H.,Lin, Z., Shen, X., Brandt, J., & Hua, G.(2015).A convolutional neural network cascade for face detection,” in IEEE Conf. Comput. Vis. Pattern Recognit. pp. 5325–5334.

Lin, G., & Shen, W. (2018). Research on convolutional neural network based on improved Relu piecewise activation function. Procedia Computer Science Volume 131, Pages 977-984.

Łuczak,M.(2017).Univariate and multivariate time series classification with parametric integral dynamic time warping, Journal of Intelligent and Fuzzy Systems 33(4), 2403–2413.

MacQueen, J. (1967). Some methods for classification and analysis of multivariate observations, Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, eds. L.M. LeCam and J. Neyman, University of California Press, pp. 281–297.

Mitiche, I., Morison, G., Nesbitt, A., Hughes-Narborough, M., Stewart, B. G., & Boreham, P.(2018). Imaging Time Series for the Classification of EMI Discharge Sources,.Sensors. doi: 10.3390/s18093098.

Nie, D., Zhang, H., Ehsan, A., Lyan, L., & Dinggang, S .(2016) .3D deep learning for multi- modal imaging-guided survival time prediction of brain tumor patients. In: MICCAI 14th Int Conf (Vision, Pattern Recognition, Graph) 9900:697. doi:10.1007/978-3-319-46720-7.

Oyelade, O. J., Oladipupo, O. O., & Obagbuwa, I. C. (2010).Application of K-Means Clustering algorithm for prediction of Students’ Academic Performance, (IJCSIS) International Journal of Computer Science and Information Security, Vol. 7, No. 1, pages 292–295.

Sakoe, H., & Chiba, S.(1978).Dynamic programming algorithm optimization for spoken word recognition.IEEE Trans. Acoust., Speech Signal Proc., 26 (1978), pp. 43-49.

Seto, S., Zhang, W.,& Zhou, Y.(2015).Multivariate time series classification using dynamic time warping template selection for human activity recognition,” in Computational Intelligence, 2015 IEEE Symposium Series on. IEEE, pp. 1399–1406.

Simonyan, K., & Zisserman, A. (2014).Two-stream convolutional networks for action recognition in videos. In NIPS.

Springenberg, J.T., Dosovitskiy, A., Brox, T., & Riedmiller, M.(2014).Striving for simplicity: The all convolutional net. arXiv preprint arXiv:1412.6806.

Tran, D., Bourdev, L.D., Fergus, R., Torresani, L., & Paluri, M. (2015). Learning spatiotemporal features with 3D convolutional networks. In: ICCV, pp. 4489–4497.

Tsai, Y.C., Chen, J.H., & Wang, C.C.(2019). Encoding Candlesticks as Images for Patterns Classification Using Convolutional Neural Networks. Computational Engineering, Finance, and Science. arXiv:1901.05237.

Wallnerström, C.J., Setréus, J., Hilber, P., Tong, F., & Bertling , L. (2010). Model of capacity demand under uncertain weather.. IEEE 11th International Conference on Probabilistic Methods Applied to Power Systems.doi: 10.1109/PMAPS.2010.5528841.

Wang, Z., & Oates, T. (2015). Encoding time series as images for visual inspection and classification using tiled convolutional neural networks. In: Workshops at the TwentyNinth AAAI Conference on Artificial Intelligence.