近年來因物聯網的蓬勃發展，時間序列類型的資料大幅增加，如何有效地在巨量的時序資料上偵測出異常就變的極為重要，在實務上(1)非監督式學習、(2)概念漂移的調整、(3)不需儲存大量資料，這三點對於處理巨量的時序資料有著關鍵性的效用，也因為這些需求，目前的非監督式時序資料異常偵測多偏向以統計為核心。
階層式時間記憶(Hierarchical Temporal Memory，簡稱HTM)是一種模擬皮質神經元運作的模型，主要用途是預測時序上的資料，再加上統計方法(模型簡稱HTM AL)便能分析出資料中的異常，並且HTM模型符合實務上使用的三大需求，相比於單純統計方法對於序列的不敏感，HTM AL模型更適合拿來處理時序資料。
本研究提出了一種改善HTM AL(Hierarchical Temporal Memory Anomaly Likelihood)的模型，該模型在HTM AL模型的統計方法前加入一層時間步長處理機制，該機制是藉由設立閥值來區分出HTM計算的異常值中需要調整的點，而後經由目前調整點與前一個調整點間的時間步長來縮放異常值。此機制能有效地將序列上包含的模式資訊附加到異常值中，對於原有模型也不需要新添加過多的計算。當新異常值送入下一層統計方法做計算時，就不會忽略掉序列的模式資訊，讓最終的異常值能考量的更加全面。最後本研究也透過NAB(Numenta Anomaly Benchmark)來做驗證，證實該論文提出的新模型—HTM TSD(Hierarchical Temporal Memory Time Step Distribution)確實能降低原有HTM AL模型的誤、漏報情形，提升模型異常偵測能力，並勝過其他使用NAB驗證的方法。

In recent years, due to the vigorous development of the Internet of Things, time series data has increased significantly. How to effectively detect anomalies in a huge amount of time series data has become extremely important. In practice,(1) Unsupervised learning, (2)Adjustment of concept drift, and (3) No need to store a large amount of data, are critical for processing huge amount of time series data. Because of these requirements, the current unsupervised time series data anomaly detection is mostly based on statistics.
Hierarchical Temporal Memory(HTM) is a model that simulates the operation of cortical neurons. Its main purpose is to predict time series data. HTM plus statistical method-Hierarchical Temporal Memory Anomaly Likelihood(HTM AL) can analyze abnormalities in the data and HTM model conforms to the three major requirements used in practice. Compared with the insensitivity of pure statistical methods to the sequence, the HTM AL model is more suitable for processing time series data.
This research proposes a new model-Hierarchical Temporal Memory Time Step Distribution(HTM TSD) for improving the HTM AL model. This model adds a layer of abnormal time step processing mechanism before the statistical method of the HTM AL model. The mechanism is to distinguish the adjustment point by setting up a threshold. Then, the outlier is scaled by the time step between the current adjustment point and the previous adjustment point. This method can effectively add the pattern information contained in the sequence to the outliers and it does not need to add too many new calculations to the original model. When the new outliers are sent to the next level of statistical methods for calculation, the pattern information of the sequence will not be ignored and thus the final outliers can be considered more comprehensively. Finally, the HTM TSD model was successfully verified by Numenta Anomaly Benchmark.

[1] R. P. Adams and D. J.C. MacKay, "Bayesian Online Changepoint Detection," Cornell University, 2007.
[2] S. Ahmad and L. Scheinkman, "How CanWe Be So Dense? The Benefits of Using Highly Sparse Representations," Cornell University, 2019.
[3] S. Ahmad, A. Lavin, S. Purdy and Z. Agha, "Unsupervised real-time anomaly detection for streaming data," Elsevier Neurocomputing, 2017.
[4] N. S. Altman, "An Introduction to Kernel and Nearest-Neighbor Nonparametric Regression," The American Statistician, vol. 46, no. 3, pp. 175-185, 1992.
[5] A. Arning, "A Linear Method for Deviation Detection in Large Databases," KDD-96 Proceedings., 1996.
[6] M. M. Breunig, H. P. Kriegel, R. T. Ng and J. Sander, "LOF: Identifying Density-Based Local Outliers," in ACM SIGMOD international conference on Management of data, 2000.
[7] V. Chandola, A. Banerjee, and V. Kumar, "Anomaly Detection: A Survey," ACM Computing Surveys, vol. 41, no. 3, 2009.
[8] F. Chen, J. C. Agüero, M. Gilson, H. Garnier and T. Liu , "EM-based identification of continuous-time ARMA Models from irregularly sampled data," Elsevier Automatica, pp. 293-301, 2017.
[9] M. Yu. Chesnokov, "Time Series Anomaly Searching Based on DBSCAN Ensembles," Springer Scientific and Technical Information Processing, vol. 46, no. 5, pp. 299-305, 2019.
[10] A. A. Cook, G. Mısırlı and Z. Fan, "Anomaly Detection for IoT Time-Series Data:A Survey," IEEE INTERNET OF THINGS JOURNAL, vol. 7, no. 7, 2020.
[11] J. Coughlin and G Perrone, "Multi-scale Anomaly Detection with Wavelets," in Proceedings of the International Conference on Big Data and Internet of Thing, 2017.
[12] Y. Cui, S. Ahmad and J. Hawkins, "The HTM Spatial Pooler—A Neocortical Algorithm for Online Sparse Distributed Coding," Frontiers in Computational Neuroscience, 2017.
[13] Y. Cui, S. Ahmad and J. Hawkins, "Continuous Online Sequence Learning with an Unsupervised Neural Network Model," Massachusetts Institute of Technology, 2016.
[14] T. Ergen and and S. S. Kozat, "Unsupervised Anomaly Detection With LSTM Neural Networks," IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2019.
[15] A. Gammerman, V. Vovk, Z. Luo and H. Papadopoulos, "Conformal k-NN Anomaly Detector for Univariate Data Streams," Conformal and Probabilistic Prediction and Applications, pp. 1-15, 2017.
[16] T. Grundy, R. Killick and G. Mihaylov, "High-dimensional changepoint detection via a geometrically inspired mapping," Statistics and Computing, 2020.
[17] S. Guha, N. Mishra, G. Roy and Okke Schrijvers, "Robust Random Cut Forest Based Anomaly Detection On Streams," in Proceedings of the 33 rd International Conference on Machine, 2016.
[18] N. Gupta, "Artificial Neural Network," in Inter national Conference on Recent Trends in Applied Sciences with Engineering Applications, 2013.
[19] J. Hawkins, M. Lewis, M. Klukas, S. Purdy and S. Ahmad, "A Framework for Intelligence and Cortical Function Based on Grid Cells in the Neocortex," Frontiers in Neural Circuits, 2019.
[20] J. Hawkins and S. Ahmad, "Why Neurons Have Thousands of Synapses, a Theory of Sequence Memory in Neocortex," Frontiers in Neural Circuits, 2016.
[21] J. Hawkins, S. Ahmad and Y. Cui, "A Theory of How Columns in the Neocortex Enable Learning the Structure of the World," Frontiers in Neural Circuits, 2017.
[22] J. Hochenbaum, O. S. Vallis and A. Kejariwal, "Automatic Anomaly Detection in the Cloud Via Statistical Learning," Cornell University, 2017.
[23] R. J Hyndman and G. Athanasopoulos, Forecasting: principles and practice, OTexts, 2018, pp. 221-274.
[24] D. Jarušková and J. Antoch, "Changepoint analysis of Klementinum temperature series," Wiley online library, 2018.
[25] S. Kannan and M. V. Srinath, "Autoregressive --based Outlier Algorithm to Detect Money Laundering Activities," International Journal of Research and Analytical Reviews, 2018.
[26] T. Y. Kim and S. B. Cho, "Web Traffic Anomaly Detection using C-LSTM Neural Networks," Elsevier Expert Systems With Applications, 2018.
[27] V. Kozitsin, I. Katser and D. Lakontsev, "Online Forecasting and Anomaly Detection Based on the ARIMA Model," MDPI Applied Sciences, 2021.
[28] H. P. Kriegel, M. Schubert and A. Zimek, "Angle-Based Outlier Detection in High-dimensional Data," in ACM SIGKDD international conference on Knowledge discovery and data mining, 2008.
[29] A. Lavin and S. Ahmad, "Evaluating Real-time Anomaly Detection Algorithms – the Numenta Anomaly Benchmark," in IEEE International Conference on Machine Learning and Applications, 2015.
[30] N. Leadholm, M. Lewis and S. Ahmad, "GRID CELL PATH INTEGRATION FOR MOVEMENT-BASED VISUAL OBJECT RECOGNITION," Cornell University, 2021.
[31] M. Lewis, S. Purdy, S. Ahmad and J. Hawkins, "Locations in the Neocortex: A Theory of Sensorimotor Object Recognition Using Cortical Grid Cells," Front. Neural Circuits, 2019.
[32] S. Li, L. D. Xu and S. Zhao, "5G Internet of Things: A survey," Elsevier Journal of Industrial Information Integration, 2018.
[33] H. Li, F. Meng, J. Kong, G. Wang, B. Lu, Y. Liu and T. Li, "An ARMA-based Traffic Abnormal Identification Approach for Remote Smart Scheduling of Optical Fiber Cores," IEEE International Conference on Industrial Internet, 2019.
[34] F. T. Liu and K. M. Ting, "Isolation Forest," in Eighth IEEE International Conference on Data Mining, 2008.
[35] P. S. Maciąg, M. Kryszkiewicz, R. Bembenik, Jesus L. L. and J. D. Ser, "Unsupervised Anomaly Detection in Stream Data with Online Evolving Spiking Neural Networks," Elsevier Neural Networks, pp. 118-139, 2021.
[36] Y. B. Maïnassara and B. Saussereau, "Diagnostic checking in multivariate ARMA models with dependent errors using normalized residual autocorrelations," Journal of the American Statistical Association, 2017.
[37] J. Ma and S. Perkins, "Time-series Novelty Detection Using One-class Support Vector Machines," IEEE Space and Remote Sensing Sciences, 2003.
[38] P. Malhotra, L. Vig, G. Shroff and P. Agarwal, "Long Short Term Memory Networks for Anomaly Detection in Time Series," in Computational Intelligence and Machine Learning, 2015.
[39] H. C. Mandhare, "A Comparative Study of Cluster Based Outlier Detection,Distance Based Outlier Detection and Density Based Outlier Detection Techniques," in International Conference on Intelligent Computing and Control Systems, 2017.
[40] R. Manzano, A. Albasir and K. Naik, "Detection of Anomalous Behavior in Wireless Devices Using Changepoint Analysis," IEEE International Congress on Internet of Things (ICIOT), 2019.
[41] J. Mnatzaganian, E. Fokoué and D. Kudithipudi, "A Mathematical Formalization of Hierarchical Temporal Memory's Spatial Pooler," Frontiers in Robotics and AI | www.frontiersin.org, 2017.
[42] Numenta, HIERARCHICAL TEMPORAL MEMORY including HTM Cortical Learning Algorithms, 2011.
[43] S. Purdy, "Encoding Data for HTM Systems," Cornell University, 2016.
[44] S. Ray, D. S McEvoy, S. Aaron, T. Hickman and A. Wright, "Using statistical anomaly detection models to find clinical decision support malfunctions," Journal of the American Medical Informatics Association, 2018.
[45] M. Schneider, W. Ertel and F. Ramos, "Expected similarity estimation for large-scale batch and streaming anomaly detection," Springer Mach Learn, pp. 305-333, 2016.
[46] E. Schubert, J. Sander, M. Ester, H. P. Kriegel and X. Xu, "DBSCAN Revisited, Revisited:Why and How You Should (Still) Use DBSCAN," ACM Trans. Database Syst, vol. 42, no. 3, p. 21, 2017.
[47] A. Stanway, 2013. [Online]. Available: https://github.com/etsy/skyline.
[48] J. W. Tukey, Exploratory data analysis, 1977.
[49] C. Wang, K. Viswanathan, L. Choudur and V. Talwar, "Statistical Techniques for Online Anomaly Detection in Data Centers," in IEEE Mini Conference, 2011.
[50] J. Wu, W. Zeng and F. Yan, "Hierarchical Temporal Memory method for time-series-based anomaly detection," Elsevier Neurocomputing, 2017.
[51] Y. Yu, Y. Zhu, S. Li and D. Wan, "Time Series Outlier Detection Based on Sliding Window Prediction," Hindawi Publishing Corporation Mathematical Problems in Engineering, 2014.
[52] C. Zhang, S. Li, H. Zhang and Y. Chen, "VELC: A New Variational AutoEncoder Based Model for Time Series Anomaly Detection," Department of Computer Science and Technology Harbin Institute of Technology, 2020.
[53] S. Zhang, X. Li, M. Zong, X. Zhu and R. Wang, "Efficient kNN Classification With Different Numbers of Nearest Neighbors," IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, vol. 29, no. 5, 2018.
[54] D. Zheng, F. Li and T. Zhao, "Self-adaptive statistical process control for anomaly detection in time series," Elsevier Expert Systems With Applications, pp. 324-336, 2016.