This work is to build a monitoring system for rehabilitation of Frailty Syndrome with the use of Kinect. There are two parts in the monitoring system developed: motion monitoring and breath rate monitoring. For motion monitoring, a depth calibration process is proposed to calibrate the distortion effects on depth readings caused by the depress angle of Kinect. With the calibration process, the detected angle of monitored joints can have a much higher accuracy than that of the original readings. In addition, re-locating processes are proposed to resolve the unstable situations of the skeleton detection from the Kinect system when occlusion occurs. When unstable skeleton points occur, according to biological features of unstable joints, the skin color detection, contour detection, random circle detection (RCD), and defect points detection are employed to locate possible correct positions of the joints. From our experiments, it can be found that with the proposed motion monitoring system, the motion detected is stably and reliably. For breath monitoring, a non-contact method of detecting the breath rate of patient during rehabilitation is considered. In the proposed method employed, the movement of chest caused by breath is tracked through the images obtained. Useful features in images are defined by Harris corner detector in the ROI. The Lucas-Kanade algorithm is applied for tracking tiny movements of those features. However, the tracking signal is a mixing signal including breath, motion, and noise. In order to acquire the breath signal, the tracking signal is filtered by a band-pass filter of an interval from 0.167 HZ to 0.417 HZ. From our experiments, the breath signal is successfully obtained. In conclusion, by employing the proposed system, it can help doctors to monitor the motions and breath rates of patients reliably and stably.

This work is to build a monitoring system for rehabilitation of Frailty Syndrome with the use of Kinect. There are two parts in the monitoring system developed: motion monitoring and breath rate monitoring. For motion monitoring, a depth calibration process is proposed to calibrate the distortion effects on depth readings caused by the depress angle of Kinect. With the calibration process, the detected angle of monitored joints can have a much higher accuracy than that of the original readings. In addition, re-locating processes are proposed to resolve the unstable situations of the skeleton detection from the Kinect system when occlusion occurs. When unstable skeleton points occur, according to biological features of unstable joints, the skin color detection, contour detection, random circle detection (RCD), and defect points detection are employed to locate possible correct positions of the joints. From our experiments, it can be found that with the proposed motion monitoring system, the motion detected is stably and reliably. For breath monitoring, a non-contact method of detecting the breath rate of patient during rehabilitation is considered. In the proposed method employed, the movement of chest caused by breath is tracked through the images obtained. Useful features in images are defined by Harris corner detector in the ROI. The Lucas-Kanade algorithm is applied for tracking tiny movements of those features. However, the tracking signal is a mixing signal including breath, motion, and noise. In order to acquire the breath signal, the tracking signal is filtered by a band-pass filter of an interval from 0.167 HZ to 0.417 HZ. From our experiments, the breath signal is successfully obtained. In conclusion, by employing the proposed system, it can help doctors to monitor the motions and breath rates of patients reliably and stably.

[1] L. P. Yu, and S. H. Wei, Outcome effects of Produce Outcome for Weakness Elderly Rehabilitation (Power) training on physical and ADL performance among older adults, Department of physical therapy and assistive technology, National Yang-Ming University, 2009.
[2] A. Ota, N. Yasuda, S. Horikawa, and T. Fujimura, H. Ohara, Differential Effects of Power Rehabilitation on Physical Performance and Higher-level Functional Capacity among Community-dwelling Older Adults with a Slight Degree of Frailty, Department of Public Health, Kochi Medical School, 2007.
[3] S. Suzuki, and K. Abe,” Topological Structural Analysis of Digitized Binary Images by Border Following”, CVGIP, vol. 30, no. 1, pp 32-46, December 1985.
[4] T. C. Chen, and K. L. Chung, “An efficient randomized algorithm for detecting circles,” Computer Vision and Image Understanding, pp. 172-191, April, 2001.
[5] C. Harris and M. J. Stephens. “A combined corner and edge detector”, Alvey Vision Conference, pp. 147–152, 1988.
[6] B. D. Lucas, and T. Kanade, “An Iterative Image Registration Technique with an Application to Stereo Vision”, Proceedings of Imaging Understanding Workshop, pp. 121-130, 1981.
[7] J. Y. Bouguet, Pyramidal Implementation of the Lucas Kanade Feature Tracker Description of the algorithm, Intel Corporation Microprocessor Research Labs, 2000.
[8] F. D. Zainordin, L. Hwea Yee, N. A. Sani, W. Y. Min, and C. C. Seng, "Human pose recognition using Kinect and rule-based system", Proc. of the 2012 World Automation Congress, pp. 1-6 Puerto Vallarta, Mexico, June 2012.
[9] A. Ahsan, S. Najam, J. Ahmed, and Z. Najam, “Interactive white board using gestures with KINECT”, Proc. Of the 2016 International Conference on Electrical, Electronics, and Optimization Techniques, pp. 90-95 Chennai, India, March 2016.
[10] S. Monir, S. Rubya, and H. S. Ferdous, "Rotation and scale invariant posture recognition using Microsoft Kinect skeletal tracking feature," Proc. of the 2012 International Conference on Intelligent Systems Design and Applications (ISDA), pp. 404-409, Kochi, November 2012.
[11] T. L. Le, M. Q. Nguyen, and T. T. M. Nguyen, "Human posture recognition using human skeleton provided by Kinect", Proc. of the 2013 International Conference on Computing, Management and Telecommunications, pp. 340-345 Nanchang, Jiangxi, Jan. 2013.
[12] S. I. Park, H. J. Shin, and S. Y. Shin, “On-line locomotion generation based on motion blending”, Proc. of the 2002 ACM SIGGRAPH Symposium on Computer Animation, pp. 130-140, July 2002.
[13] A. Bruderlin and L. Williams, “Motion Signal Processing”, Proc. of the 22nd Annual Conference on Computer graphics and Interactive Techniques, pp. 97-104, September 1995.
[14] C. Rose, P. Sloan, and M. Cohen, “Artist-directed inverse kinematics using radial basis function interpolation”, Proceedings of Eurographics, vol. 20, no. 3, 2001.
[15] R. Boulic, Z. Huang, N. M. Thalmann, and D. Thalmann, “Goal-oriented design and correction of articulated figure motion with the track system”, Computers and Graphics (UK), vol. 18, no. 443, 1994.
[16] H.Y. Huang and S.H. Chang, “A Skeleton-occluded Repair Method from Kinect”, International Symposium on Computer, Consumer and Control, pp. 265-267, June 2014.
[17] X.L. Wei, P.Z. Zhang, and J.X. Chai, “Accurate realtime full-body motion capture using a single depth camera”, ACM Transactions on Graphics, Vol. 31, No. 6, November 2012.
[18] O. Yahya and M. Faezipour, "Automatic detection and classification of acoustic breathing cycles", 2014 Zone 1 Conference of the American Society for Engineering Education (ASEE Zone 1), pp. 1-5, 2014.
[19] K. Perlin, “Real time responsive animation with personality”, IEEE Transactions on Visualization and Computer Graphics, vo. 1, pp.1, March 1995.
[20] Y. W. Bai, W. T. Li, and Y. W. Chen, "Design and implementation of an embedded monitor system for detection of a patient's breath by double Webcams in the dark", 2010 12th IEEE International Conference one-Health Networking Applications and Services (Healthcom), pp. 93-98, 2010.
[21] G. Balakrishnan, F. Durand, and J. Guttag, "Detecting pulse from head motions in video", 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3430-3437, 2013.
[22] T. Mallick, P. P. Das, and A. K. Majumdar, “Characterizations of Noise in Kinect Depth Images: A Review”, IEEE Sensors Journal, vol. 14, no. 6, pp. 1731–1740, 2014.
[23] S. Y. Kim, W. Cho, A. Koschan, and M. A Abidi, “Depth data calibration and enhancement of time-of- flight video-plus-depth camera”, Proc. of the Future of Instrumentation International Workshop, pp. 126–129, November 2011.
[24] M. Sivan, J. Gallagher, S. Makower, D. Keeling, B. Bhakta, R. J O’Connor, and M. Levesley, “Home-based Computer Assisted Arm Rehabilitation (HCAAR) robotic device for upper limb exercise after stroke: results of a feasibility study in home setting”, Journal of NeuroEngineering and Rehabilitation, vol. 11, 2014.
[25] R. Wang, S. Paris, and J. Popovic, “Practical color-based motion capture”, Proc. of the 21th ACM SIGGRAPH/ Eurographics Symposium on Computer Animation, 2011.
[26] W. E. Huang, Kinect Aided X-ray System: System Angle Calibration and Object Thickness Detection, Nation Taiwan University of Science Technology, 2015.
[27] Y. W. Bai, Y. W. Chen, and W. T. Li, "Design of an embedded monitor system with a low-power laser projection for the detection of a patient's breath", 2011 IEEE International Conference on Consumer Electronics (ICCE), pp. 553-554, 2011.
[28] S. Jianbo and C. Tomasi, "Good features to track", Computer Vision and Pattern Recognition, pp. 593-600, 1994.
[29] T. Q. Viet, “Non-Contact Breath Motion Detection Using the Lucas-Kanade Algorithm”, Nation Taiwan University of Science Technology, 2016.
[30] Kinect for Windows v2 C++:
https://kheresy.wordpress.com/kinect-for-windows-v2-cpp-index/
[31] pdm[Vision]® CamCube 3.0: http://www.pmdtec.com/news_media/video/camcube.php
[32] D-Imager 3D sensing technology: http://www2.panasonic.biz/es/densetsu/device/3DImageSensor/en/product.html
[33] Kinect for Xbox one:
http://www.xbox.com/en-us/xbox-one/accessories/kinect-for-xbox-one
[34] Kinect for Xbox 360:
http://www.xbox.com/en-US/xbox-360/accessories/kinect
[35] Asus Xtion Pro Live:
https://www.asus.com/3D-Sensor/Xtion_PRO_LIVE/specifications/
[36] MATLAB Programming for Medical Signal Analysis (Graduate):
http://www.ym.edu.tw/~cflu/CFLu_course_matlabsig.html
[37] Lecture 06 - Optical Flow: https://www.youtube.com/watch?v=5VyLAH8BhF8&t=2689s
[38] Chest movement during breathing:
http://www.balancemotion.com/dont-be-a-victim-of-chest-breathing