本研究提出一個將時間序列資料轉換成二維圖像並將這些圖像串聯成一個更大圖像的架構，來提升Multivariate Time Series (MTS) 分類問題的準確率。本研究使用了三種方法將時間序列資料轉換成二維圖像：Gramian Angular Summation Field (GASF)、 Gramian Angular Difference Field (GADF) 和Markov Transition Field (MTF)。經由在機器視覺中非常熱門的深度學習架構-卷積神經網路(ConvNet)以轉換後二維圖像作為輸入，找出原始時間序列資料的特徵及模式，進行分類的預測。本研究比較多變量串聯Red, Green, Blue (RGB)圖片法及多變量串聯灰階圖片法兩種資料整合的方式，並以開放資料進行方法的驗證與評估。研究結果經由統計檢定表示利用RGB轉換會比灰階轉換能得到更好的分類預測準確率，而且只需要使用較簡單的ConvNet架構就可以滿足預測上的需求。

This research encodes multivariate time series data into two-dimensional images, and aggregate the images into one bigger image for classification through convolutional neural network (ConvNet). This study applied three methods to encode time series into images: Gramian Angular Summation Field (GASF), Gramian Angular Difference Field (GADF), and Markov Transition Field (MTF). Then the images from multiple sensors or variables were aggregated using the concatenating method. Two open datasets were applied to evaluate the impact of using different transformation methods, input images and ConvNet architectures on prediction accuracy. The result shows that RGB images deliver significantly better classification accuracy than grayscale images and combination of RGB and grayscale images. Also, the simple structure of ConvNet is sufficient to process the images as it performed equally well with the complex structure of VGGNet.

Batal, Iyad, Lucia Sacchi, Riccardo Bellazzi, and Milos Hauskrecht. 2009. "Multivariate Time Series Classification with Temporal Abstractions." In FLAIRS Conference.
Baydogan, Mustafa Gokce, Runger, and George. 2015. 'Learning a symbolic representation for multivariate time series classification', 29: 400-22.
Berndt, Donald J, and James Clifford. 1994. "Using dynamic time warping to find patterns in time series." In KDD workshop, 359-70. Seattle, WA.
Box, George EP, Gwilym M Jenkins, Gregory C Reinsel, and Greta M Ljung. 2015. Time series analysis: forecasting and control (John Wiley & Sons).
Chen, Jou-Fan, Wei-Lun Chen, Chun-Ping Huang, Szu-Hao Huang, and An-Pin Chen. 2016. "Financial Time-Series Data Analysis Using Deep Convolutional Neural Networks." In Cloud Computing and Big Data (CCBD), 2016 7th International Conference on, 87-92. IEEE.
Faouzi, Johann. 2018. 'pyts: a Python package for time series transformation and classification'.
Górecki, Tomasz, and Maciej Łuczak. 2015. 'Multivariate time series classification with parametric derivative dynamic time warping', Expert Systems with Applications, 42: 2305-12.
Glorot, Xavier, Antoine Bordes, and Yoshua Bengio. 2011. "Deep sparse rectifier neural networks." In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, 315-23.
Hahnloser, Richard HR, Rahul Sarpeshkar, Misha A Mahowald, Rodney J Douglas, and H Sebastian %J Nature Seung. 2000. 'Digital selection and analogue amplification coexist in a cortex-inspired silicon circuit', 405: 947.
He, Kaiming, XRSSJ Zhang, S Ren, and J %J arXiv preprint arXiv:. Sun. 2015. 'Deep residual learning for image recognition. eprint'.
Karim, Fazle, Somshubra Majumdar, Houshang Darabi, and Samuel Harford. 2018. 'Multivariate LSTM-FCNs for Time Series Classification', arXiv preprint arXiv:1801.04503.
Keogh, Eamonn, Kaushik Chakrabarti, Michael Pazzani, and Sharad Mehrotra. 2001. 'Dimensionality reduction for fast similarity search in large time series databases', Knowledge and information Systems, 3: 263-86.
Krizhevsky, Alex %J arXiv preprint arXiv:. 2014. 'One weird trick for parallelizing convolutional neural networks'.
Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E Hinton. 2012. "Imagenet classification with deep convolutional neural networks." In Advances in neural information processing systems, 1097-105.
LeCun, Yann, Bernhard E Boser, John S Denker, Donnie Henderson, Richard E Howard, Wayne E Hubbard, and Lawrence D Jackel. 1990. "Handwritten digit recognition with a back-propagation network." In Advances in neural information processing systems, 396-404.
Łuczak, Maciej. 2018. 'Combining raw and normalized data in multivariate time series classification with dynamic time warping', Journal of Intelligent & Fuzzy Systems, 34: 373-80.
Mitiche, Imene, Gordon Morison, Alan Nesbitt, Michael Hughes-Narborough, Brian Stewart, and Philip %J Sensors Boreham. 2018. 'Imaging Time Series for the Classification of EMI Discharge Sources', 18: 3098.
Muth, John F. 1960. 'Optimal properties of exponentially weighted forecasts', Journal of the american statistical association, 55: 299-306.
Nagem, Tarek AM Hamad, Rami Qahwaji, and Stan Ipson. 2017. "Deep learning teachology for the prediction of solar flares from GOES data." In Computing Conference, 2017, 697-700. IEEE.
Olszewski, Robert T. 2001. "Generalized feature extraction for structural pattern recognition in time-series data." In.: Carnegie-mellon univ pittsburgh pa school of computer science.
Prieto, Oscar J, Carlos J Alonso-González, and Juan J Rodríguez. 2015. 'Stacking for multivariate time series classification', Pattern Analysis and Applications, 18: 297-312.
Sübakan, Y Cem, Barış Kurt, A Taylan Cemgil, and Bülent %J Digital Signal Processing Sankur. 2014. 'Probabilistic sequence clustering with spectral learning', 29: 1-19.
Sabour, Sara, Nicholas Frosst, and Geoffrey E Hinton. 2017. "Dynamic routing between capsules." In Advances in neural information processing systems, 3856-66.
Simonyan, Karen, and Andrew %J arXiv preprint arXiv:. Zisserman. 2014. 'Very deep convolutional networks for large-scale image recognition'.
Sánchez, FA Rivera, and JA González Cervera. 2019. "ECG Classification Using Artificial Neural Networks." In Journal of Physics: Conference Series, 012062. IOP Publishing.
Szegedy, Christian, Vincent Vanhoucke, Sergey Ioffe, Jonathon Shlens, and Zbigniew %J arXiv preprint arXiv:.00567 Wojna. 2015. 'Rethinking the inception architecture for computer vision. arXiv 2015', 1512.
Tan, Mingxing, and Quoc V %J arXiv preprint arXiv:.11946 Le. 2019. 'EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks'.
Technical University of Denmark, DTU Informatics, E-mail: reception@imm.dtu.dk. 2012. 'Prediction as a candidate for learning deep hierarchical models of data'.
Wang, Lin, Zhigang Wang, and Shan %J Expert Systems with Applications Liu. 2016. 'An effective multivariate time series classification approach using echo state network and adaptive differential evolution algorithm', 43: 237-49.
Wang, Zhiguang, and Tim Oates. 2015a. "Encoding time series as images for visual inspection and classification using tiled convolutional neural networks." In Workshops at the Twenty-Ninth AAAI Conference on Artificial Intelligence, 40-46.
———. 2015b. 'Imaging time-series to improve classification and imputation', arXiv preprint arXiv:1506.00327.
YANG, Chen-Yi. 2018. 'Multivariate Time Series Data Transformation for Convolutional Neural Network', National Taiwan University of Science and Technology.
Yang, Haw-Ching, Yu-Yung Li, Min-Hsiung Hung, and Fan-Tien Cheng. 2017. 'A cyber-physical scheme for predicting tool wear based on a hybrid dynamic neural network', Journal of the Chinese Institute of Engineers, 40: 614-25.
Yao, Yuan, Lorenzo Rosasco, and Andrea Caponnetto. 2007. 'On early stopping in gradient descent learning', Constructive Approximation, 26: 289-315.
Yazdanbakhsh, Omolbanin, and Scott %J arXiv preprint arXiv:.01697 Dick. 2019. 'Multivariate Time Series Classification using Dilated Convolutional Neural Network'.
Zheng, Yi, Qi Liu, Enhong Chen, Yong Ge, and J Leon Zhao. 2014. "Time series classification using multi-channels deep convolutional neural networks." In International Conference on Web-Age Information Management, 298-310. Springer.
Zhou, Yi-Tong, and Rama Chellappa. 1988. "Computation of optical flow using a neural network." In IEEE International Conference on Neural Networks, 71-78.