基於影像之非接觸式心率量測是一種創新的生理監控技術，其具有高舒適性、高便利性、適合長時間的監控方面等優點，不僅適合用於嬰兒照護、居家長期照護、駕駛生理監控、運動監控，在臨床上也有多種用途，如: 病患生理監控、手術過程中的生命監控等。然而，目前的訊號仍存在著容易受到外在光線或是移動干擾等問題。
在本篇論文中，我們藉由觀察及分析非接觸生理訊號的特性，針對運動時的生理監控需求開發一套對應的訊號處理演算法。除了基本的心律監控外，我們也發展了以非接觸式影像擷取的方式來監控使用者的臉部的位移完成計步等功能，透過單一的影像感測器便能達到多功能的量測。此方法不僅可以取代目前市面上的慣性感測器，也能夠有效降低穿戴上的不便與不適感。經過三個運動器材實驗後，本論文提出的非接觸式計步方法於踏步機及跑步機的平均準確度分別為99.52% 與 99.77%。心率量測的平均絕對誤差/均方根誤差於健身車、踏步機與跑步機分別為2.08/2.86 Bpm、5.68/8.10 Bpm與4.68/6.52 Bpm。
為了增加運動量測時的舒適度，我們進一步利用臉部標誌特徵作為運動時人臉追蹤的依據，此方法可以有效減少在劇烈運動下臉部特徵點容易消失的情況，進而降低心率量測的誤差。除此之外，使用者在量測過程中也不需要持續直盯著鏡頭，頭部可以自然地晃動或是轉動，使得本系統在使用上能夠更加符合使用者的需求。

Contactless pulse rate measurement based on image is an innovative physiological monitoring technology which equips with high comfort level, high convenience and considerable for long-term monitoring. Not only is it suitable for baby care, home care, driver physiological signals monitoring, fitness monitoring, but also has myriad of applications in clinical practice such as patient physiological monitoring, vital signs monitoring during surgery, etc. However, the current development of remote measurement technique is still lack of integrity and the signals are vulnerable to external light or movement interference.
In this dissertation, we observe and analyze the characteristics of non-contact physiological signals to develop the corresponding signal processing algorithms for fitness applications. Besides fundamental pulse rate measurement, a number of contactless images capturing and processing methods to detect user physical parameters such as step counts are also established. Through a signal image sensor, multi-function measurements can be achieved in different applications. This approach can not only ameliorate the existing problems of wearable sensors, but can effectively mitigate the inconvenience and discomfort brought up by wearers. The results reveal that the detection rates of the proposed step count method are 99.57 % and 99.77% for stepping and treadmill exercise, respectively. In pulse rate detection aspect, the provided Mean- absolute error/Root-mean-square error are 2.08/2.86 Bpm in biking; 5.68/8.10 Bpm in stepping; and 4.68/6.52 Bpm in treadmill running.
To enhance the usability of the measurement in fitness, we further apply facial landmark detection as tracking features. This method allows to improve the situation of feature point disappearance under intensive exercise, thereby to reduce the errors between the ground-truth heart rate devices in pulse rate detection. More importantly, the user does not have to keep staring at the camera as always but some casual movements like swaying and rotating are tolerable.

[1] R. E. Glasgow, E. G. Eakin, E. B. Fisher, S. J. Bacak, and R. C. Brownson, “Physician advice and support for physical activity. Results from a national survey,” Am. J. Prev. Med., vol. 21, no. 3, pp. 189–196, 2001.
[2] T. Matsuda and M. Makikawa, “ECG monitoring of a car driver using capacitively-coupled electrodes.,” 30th Annu. Int. Conf. IEEE Eng. Med. Biol. Soc., vol. 2008, pp. 1315–8, 2008.
[3] U. Ryu et al., "Adaptive step detection algorithm for wireless smart step counter," 2013 Int. Conf. Inf. Sci. Appl. ICISA, pp. 1-4, 2013.
[4] Z. Zhou, F. Shi, and W. Wu, “Learning spatial and temporal extents of human actions for action detection,” IEEE Trans. Multimed., vol. 17, no. 4, pp. 512–525, 2015.
[5] G. Yu, N. A. Goussies, J. Yuan, and Z. Liu, "Face action detection via discriminative random forest voting and top-k subvolume search," IEEE Trans. Multimedia, vol. 13, no. 3, pp. 507-517, 2011.
[6] W. Xu, Z. Miao, X. P. Zhang, and Y. Tian, "A hierarchical spatio-temporal model for human activity recognition," IEEE Trans. Multimedia, vol. 19, no. 7, pp. 1494-1509, 2017.
[7] Q. Mao, Q. Rao, Y. Yu, and M. Dong, "Hierarchical bayesian theme models for multipose facial expression recognition," IEEE Trans. Multimedia, vol. 19, no. 4, pp. 861-873, 2017.
[8] K. Ozcan and S. Velipasalar, "Robust and reliable step counting by mobile phone cameras," in Proc. ICDSC, 2015, pp. 164-169.
[9] Z. Li, "Exercises intensity estimation based on the physical activities healthcare system," in Proc. IEEE International Conference on Communications and Mobile Computing, vol. 3, pp. 132-136, 2009.
[10] C. Takano and Y. Ohta, “Heart rate measurement based on a time-lapse image,” Med. Eng. Phys., vol. 29, no. 8, pp. 853–857, 2007.
[11] F. P. Wieringa et al., "Contactless multiple wavelength photoplethysmographic imaging: A first step toward “spo2 camera” technology," Ann. Biomed. Eng., vol. 33, no. 8, pp. 1034-1041, 2005.
[12] C. Yang, G. Cheung, and V. Stankovic, "Estimating heart rate and rhythm via 3D motion tracking in depth video," IEEE Trans. Multimedia, vol. 19, no. 7, pp. 1625-1636, 2017.
[13] M. Lewandowska, J. Ruminski, T. Kocejko, and J. Nowak, "Measuring Pulse Rate with a Webcam - a Non-contact Method for Evaluating Cardiac Activity," in Proc. FedCSIS, 2011, pp. 405-410.
[14] M. Z. Poh, D. J. McDuff, and R. W. Picard, "Non-contact, automated cardiac pulse measurements using video imaging and blind source separation," Opt. Express, vol. 18, no. 10, pp. 10762-10774, 2010.
[15] G. de Haan and V. Jeanne, "Robust pulse-rate from chrominance-based rPPG," IEEE Trans. Biomed. Eng., vol. 60, no. 10, pp. 2878-2886, 2013.
[16] G. de Haan and A.Van Leest, “Improved motion robustness of remote-PPG by using the blood volume pulse signature,” Physiol. Meas., vol. 35, no. 9, pp. 1913–1926, 2014.
[17] W. Wang et al., “Algorithmic principles of remote-PPG,” IEEE Trans. Biomed. Eng., vol. 64, no. 99, pp. 1479–1491, 2017.
[18] R. Y. Huang and L. R. Dung, "A motion-robust contactless photoplethysmography using chrominance and adaptive filtering," in IEEE Biomedical Circuits and Systems Conference (BioCAS), 2015, pp. 1-4.
[19] M. -Z. Poh, D. McDuff, and R. Picard, “Advancements in noncontact, multiparameter physiological measurements using a webcam,” IEEE Trans. Biomed., vol. 58, no. 1, pp. 7–11, 2011.
[20] Y. C. Lin, Y. H. Lin, "Step count and pulse rate detection based on the contactless image measrueemnt method," IEEE Trans. Multimedia, vol. 20, no. 8, pp. 2223-2231, 2018.
[21] A. B. Hertzman and C. Spealman, “Observations on the finger volume pulse recorded photoelectrically,” Amer. J. Physiol., vol. 119, pp. 334–335, 1937.
[22] J. Allen, “Photoplethysmography and its application in clinical physiological measurement,” Physiological Measurement, vol. 28, no. 3, pp. R1–R39, 2007.
[23] S.Vogel, M.Hülsbusch, T.Hennig, V.Blazek, andS.Leonhardt, “In-ear vital signs monitoring using a novel microoptic reflective sensor,” IEEE Trans. Inf. Technol. Biomed., vol. 13, no. 6, pp. 882–889, 2009.
[24] P. Mannheimer, “The light-tissue interaction of pulse oximetry,” Anesthesia and Analgesia, vol. 105, no. 6, pp. S10–S17, 2007.
[25] Y. Sun and N. Thakor, “Photoplethysmography Revisited: From Contact to Noncontact, from Point to Imaging,” IEEE Trans. Biomed. Eng., vol. 63, no. 3, pp. 463–477, 2016.
[26] W. Cui, L. E. Ostrander, and B. Y. Lee, “In Vivo Reflectance of Blood and Tissue as a Function of Light Wavelength,” IEEE Trans. Biomed. Eng., vol. 37, no. 6, pp. 632–639, 1990.
[27] S. Ohta, H. Nakamoto, Y. Shinagawa, and T. Tanikawa, “A health monitoring system for elderly people living alone,” J. Telemed. Telecare, vol. 8, no. 3, pp. 151–156, 2002.
[28] L. Cattani et al., “Monitoring infants by automatic video processing: A unified approach to motion analysis,” Comput. Biol. Med., vol. 80, no. August 2016, pp. 158–165, 2017.
[29] M. -Z. Poh, D. McDuff, and R. Picard, “A medical mirror for non-contact health monitoring,” ACM SIGGRAPH 2011 Emerg. Technol. - SIGGRAPH ’11, vol. 18, no. 10, pp. 1–1, 2011.
[30] J. Allen and K. Howell, “Microvascular imaging: Techniques and opportunities for clinical physiological measurements,” Physiol. Meas., vol. 35, no. 7, 2014.
[31] F. Vicente, Z. Huang, X. Xiong, F. DeLa Torre, W.Zhang, and D.Levi, “Driver Gaze Tracking and Eye Off the Road Detection System,” IEEE Trans. Intell. Transp. Syst., vol. 16, no. 4, pp. 2014–2027, 2015.
[32] A. A. Lenskiy and J.-S.Lee, “Driver’s eye blinking detection using novel color and texture segmentation algorithms,” Int. J. Control. Autom. Syst., vol. 10, no. 2, pp. 317–327, 2012.
[33] A. Lal, S. K. L. and Craig, “A critical review of the psychophysiology of driver fatigue. Biological Psychology,” vol. 55, no. 2001, pp. 173–194, 2001.
[34] Q. Zhang, Q. Wu, Y. Zhou, X. Wu, Y. Ou, and H. Zhou, “Webcam-based, non-contact, real-time measurement for the physiological parameters of drivers,” Meas. J. Int. Meas. Confed., vol. 100, pp. 311–321, 2017.
[35] B. -F. Wu et al., “A motion robust remote-ppg approach to driver’s health state monitoring,” in ACCV 2016 International Workshops, Taipei, Taiwan, November 20-24, 2016, Revised Selected Papers, Part I, 2017, pp. 463–476.
[36] J. Kuo, S. Koppel, J. L. Charlton, and C. M. Rudin-Brown, “Evaluation of a video-based measure of driver heart rate,” J. Safety Res., vol. 54, no. June, pp. 55–59, 2015.
[37] PAI Health: “Why you should never wear a heart rate monitor chest strap again”, http://www.paihealth.com/blog/why-you-should-never-wear-a-heart-rate-monitor-chest-strap-again.htm (accessed on 4 March 2018)
[38] Y. Chen, C. Lin, Y. Lin, and C. Zhao, “Non-invasive detection of alcohol concentration based on photoplethysmogram signals,” pp. 188–193, 2017.
[39] Y. Izawa, S. Tamura, and Y. Omura, “Advanced method to mask smoking effect on alcoholic intake detection based on photoplethysmogram signal analysis,” in Proc. IEEE Sensors Conf., 2012, pp. 370-373.
[40] M. A. Turk and A. P. Pentland, “Face recognition using eigenfaces,” Proceedings. 1991 IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., pp. 586–591, 1991.
[41] W. Zhao et al., “Face Recognition: A Literature Survey,” ACM Comput. Surv., vol. 35, no. 4, pp. 399–458, 2003.
[42] J. Maatta et al., “Face spoofing detection from single images using microtexture analysis,” in 2011 International Joint Conference on Biometrics (IJCB), Oct. 2011, pp. 1–7.
[43] T. Pereira F. et al., “Face liveness detection using dynamic texture,” EURASIP J. Image Video Process., p. 2, 2014.
[44] A. Hadid, “Face biometrics under spoofing attacks: Vulnerabilities, countermeasures, open issues, and research directions,” IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. Work., pp. 113–118, 2014.
[45] Y. Sun, C. Papin, V. Azorin-Peris, R. Kalawsky, S. Greenwald, and S. Hu, “Comparison of scientific CMOS camera and webcam for monitoring cardiac pulse after exercise,” p. 813506, 2011.
[46] L. Lindberg and P. Oberg, “Photoplethysmography. part 2. influence of light source wavelength.” Medical & biological engineering & computing, vol. 29, no. 1, p. 48, 1991.
[47] A. V. Moco, S. Stuijk, and G. de Haan, “Ballistocardiographic Artifacts in PPG Imaging,” IEEE Trans. Biomed. Eng., vol. 63, no. 9, pp. 1804–1811, 2016.
[48] L. Feng et al., “Dynamic roi based on k-means for remote photoplethysmography,” in 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1310–1314, 2015.
[49] S. Kwon, J. Kim, D. Lee, and K. Park, “ROI analysis for remote photoplethysmography on facial video,” Proc. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. EMBS, vol. 2015–Novem, pp. 4938–4941, 2015.
[50] W. Wang, S. Stuijk, and G. de Haan, "Exploiting spatial redundancy of image sensor for remote robust rPPG," IEEE Trans. Biomed. Eng., vol. 62, no. 2, pp. 415-425, 2015.
[51] M. Lewandowska et al., “Measuring pulse rate with a webcam - a noncontact method for evaluating cardiac activity,” in Computer Science and Information Systems (FedCSIS), Federated Conference on, Sept. 2011, pp. 405–410.
[52] Y. C. Lin, G. Y. Lin, and Y. H. Lin, “Development of a FPGA-based contactless pulse rate detection system,” Proc. 2nd Int. Conf. Intell. Green Build. Smart Grid, IGBSG 2016, pp. 2–6, 2016.
[53] B. D. Lucas and T. Kanade, "An iterative image registration technique with an application to stereo vision," in Proc. 7th IJCAI, 1981, pp. 674-679.
[54] J. Shi and C. Tomasi, "Good features to track," in IEEE Conference on Computer Vision and Pattern Recognition (CVPR94), 1994, pp. 593-600.
[55] R. Libby, “A simple method for reliable footstep detection on embedded sensor platforms,” 2008.
[56] Y. H. Lin, C. F. Lin, and H. Z. You, "A driver's physiological monitoring system based on a wearable PPG sensor and a smartphone," in Security-Enriched Urban Computing and Smart Grid, Berlin, Germany, 2011, pp. 326-335.
[57] D. G. Jang, S. Park, and M. Hahn, "A real-time pulse peak detection algorithm for the photoplethysmogram," IJEEE, vol. 2, no. 1, pp. pp. 45-49, 2014.
[58] "Polar H7 Website," [Online]. Available: https://www.polar.com/us-en/products/accessories/H7_heart_rate_sensor.
[59] Intel RealSense Technology: https://software.intel.com/en-us/realsense/sdk (accessed on 4 March 2018)
[60] T. F. Cootes, C. J. Taylor, D. H. Cooper, and J. Graham, “Active Shape Models-Their Training and Application,” Computer Vision and Image Understanding, vol. 61, no. 1. pp. 38–59, 1995.
[61] G. J. Edwards, C. J. Taylor, and T. F. Cootes, “Interpreting face images using active appearance models,” Autom. Face Gesture Recognit., pp. 300–305, 1998.
[62] IIAU-LAB: Active Appearance Model (AAM) http://ice.dlut.edu.cn/lu/AAM.html (accessed on 2 May 2018)
[63] T. F. Cootes, G. J. Edwards, and C. J. Taylor, “Active Appearance Models,” Proc. Eur. Conf. Comput. Vis., vol. 2, pp. 484–498, 1998.
[64] Active Appearance Model: http://www.menpo.org/menpofit/aam.html (accessed on 25 April 2018).