本論文之目的在於多人物人體動作偵測及異常動作追蹤，使用人物關鍵點座標資訊結合神經網路分類方法達到多人物之動作辨識，並即時追蹤出畫面中動作異常的人物。為了達到以上之目的本論文僅使用不具深度值的C615 RGB相機，並設計了卷積神經網路以及長短期記憶網路兩個不同的網路架構並比較兩者架構之測試結果。本文設計八種於室內活動可能出現的動作，並包含兩種異常動作，同時也建立了小型的動作數據集，作為神經網路分類訓練的資料。兩個網路模型都是透過人物的關鍵點座標資訊作為動作辨識的依據，首先將訓練資料作前處理；然後接著將神經網路模型訓練成功並以此模型作為即時系統的動作預測之目標。最後再以多物件追蹤的方法達到畫面中每一個人物的追蹤，並同時成功的完成多人物的動作辨識。

The purpose of this paper is to perform multi-person human body motion detection and abnormal action tracking by using the key point coordinate information of the person and combined with the neural network classification method in order to achieve the multi-person action identification issue and track the abnormal action of the person in real time. This paper uses only one RGB camera C615 without the depth value, and designs convolutional neural network and a long short-term memory network to reach the action classification problem and to compare these two different network architectures. This paper has planned eight different actions that may occur in in-door, and including two kinds of abnormal actions, and establishes a small action dataset as the data for neural network classification training. Both network models use the key point coordinate information of the character as the basis for action recognition. First, the training data is preprocessed, and then the neural network model is trained successfully, and then used as a real-time system action prediction tool to achieve the goal. Finally, the multi-object tracking method is used to achieve the tracking of each person in each frame, and at the same time the multi-person action real-time recognition is performed successfully.

1. Yamato, J., J. Ohya, and K. Ishii. Recognizing Human Action in Time-Sequential Images Using Hidden Markov Model. in Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 1992.
2. Schuldt, C., I. Laptev, and B. Caputo. Recognizing Human Actions: a Local SVM Approach. in Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004. 2004.
3. Ren, Z., et al., Robust Part-Based Hand Gesture Recognition Using Kinect Sensor. IEEE Transactions on Multimedia, 2013. 15(5): p. 1110-1120.
4. Liu, T., et al. Human Action Recognition Based on Depth Images from Microsoft Kinect. in 2013 Fourth Global Congress on Intelligent Systems. 2013.
5. Ijjina, E.P. and C.K. Mohan. Human Action Recognition Based on Recognition of Linear Patterns in Action Bank Features Using Convolutional Neural Networks. in 2014 13th International Conference on Machine Learning and Applications. 2014.
6. Wang, X., et al., Beyond Frame-level CNN: Saliency-Aware 3-D CNN With LSTM for Video Action Recognition. IEEE Signal Processing Letters, 2017. 24(4): p. 510-514.
7. Cho, T.Z.W., M.T. Win, and A. Win. Human Action Recognition System Based on Skeleton Data. in 2018 IEEE International Conference on Agents (ICA). 2018.
8. Yang, Y., et al. Human Action Recognition Based on Skeleton and Convolutional Neural Network. in 2019 Photonics & Electromagnetics Research Symposium - Fall (PIERS - Fall). 2019.
9. Redmon, J., et al. You Only Look Once: Unified, Real-Time Object Detection. in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2016.
10. Redmon, J. and A. Farhadi. YOLO9000: Better, Faster, Stronger. in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2017.
11. Redmon, J. and A.J.a.e.-p. Farhadi YOLOv3: An Incremental Improvement. 2018. arXiv:1804.02767.
12. Bochkovskiy, A., C.-Y. Wang, and H.-Y.M.J.a.e.-p. Liao YOLOv4: Optimal Speed and Accuracy of Object Detection. 2020. arXiv:2004.10934.
13. Cao, Z., et al., OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021. 43(1): p. 172-186.
14. Bewley, A., et al. Simple Online and Realtime Tracking. in 2016 IEEE International Conference on Image Processing (ICIP). 2016.
15. Wojke, N., A. Bewley, and D. Paulus. Simple Online and Realtime Tracking with a Deep Association Metric. in 2017 IEEE International Conference on Image Processing (ICIP). 2017.