隨著台灣邁入高齡化社會，加速推廣運動休閒提升老人健康，成為我國非常重要的課題。對此，政府也鼓勵民眾養成運動休閒與健康的生活習慣，藉以降低未來老年疾病發生率與減少醫療支出。而運動過程的心跳變化常被視為一項評估運動強度和訓練成果的指標。因此，倘若能將新興的非接觸式心率量測技術與運動場域加以結合，透過其所具備的高舒適、高便利和低風險等優勢，不僅能讓使用者於運動過程中避免因配戴穿戴式裝置而造成的身體不適感或皮膚過敏等現象，亦能降低病毒接觸感染的風險，進而提升整體的運動成效。
目前非接觸式心率量測技術應用於運動情境仍存在部分問題需要改善，主要重點整理如下：
(1) 由於非接觸式生理訊號主要透過量測人體皮膚的微小顏色變化來計算心率，若量測範圍內包含了非皮膚組織的區域(例如：眼睛、眉毛和嘴巴)將影響訊號的品質。
(2) 運動過程中除了週期性晃動雜訊外，影像模糊所造成的雜訊剛好介於心率頻段時，便將可能會造成頻譜上難以明確辨別何者為真實的心跳脈動頻率。
(3) 先前研究在實驗時皆以離線的方式進行量測分析，如此將無法於運動中即時提供量測資訊給予使用者參考。
因此，本論文基於影像感測器實現一套即時的非接觸式心率量測系統，並對於上述相關問題進行改善，而經由腳踏車、踏步機和跑步機等三項實驗項目在即時量測實驗所得到的平均絕對誤差(Mean Absolute Error，MAE)/均方根誤差(Root-Mean-Square Error，RMSE)分別為2.11/2.93、2.43/3.44和2.26/3.45 BPM，而Success Rate-5/ Success Rate-10分別為0.87/0.94、0.83/0.92和0.88/0.96。經由實驗結果，證明本論文所提出的方法有助於提升心率量測準確度。

As Taiwan enters an aging society, accelerating the promotion of sports to improve the health of the elderly has become an issue for our country. In this regard, the government encourages people to develop sports and healthy lifestyles to reduce elderly disease incidence and medical expenditures. The change of heart rate during exercise is often regarded as an index to evaluate exercise intensity and training results. Therefore, taking the advantages of the novel non-contact pulse rate measurement technology, such as high comfort, high convenience, and low risk, to the fitness industry will not only help users to avoid discomfort or skin allergies caused by the wearable devices, but also reduce the risk of infectious diseases, and thereby improves the overall exercise effectiveness.
After studying current camera-based rPPG technology applications in fitness field, we consider some problems need to be solved. The main issues are as follows. 1) Since the non-contact physiological signal mainly calculates the pulse rate by measuring the subtle color changes of the human skin, the signal quality will be affected if the measurement range includes non-skin tissue areas. 2) In addition to the periodic motion artifacts occurred during exercise, it may be difficult to distinguish the noise frequency from the pulse rate frequency on the spectrum when the noise frequency caused by blurred image is close to the pulse rate frequency. 3) Owing to the complexity of the rPPG algorithm, most research chooses to extract the pulse rate from videos offline, which fails to provide the users real-time measurement information in the actual field.
Therefore, this thesis implements a real-time non-contact pulse rate measurement system based on image sensors and improves the above-mentioned related problems, to be practically applied to sports fields. The results reveal that the mean absolute error (MAE) / root-mean-square error (RMSE) of pulse rate measurement in real-time are 2.11/2.93, 2.43/3.44, and 2.26/3.45 bpm for biking, stepper, and treadmill, respectively. In addition, the success rate-5 / success rate-10 are 0.87/0.94, 0.83/0.92, and 0.88/0.96, respectively. The experimental results show that the proposed methods can improve the accuracy of pulse rate measurement.

[1] 國家發展委員會, "中華民國人口推估(2020至2070年)報告," 2020. [Online]. Available: https://pop-proj.ndc.gov.tw/upload/download/中華民國人口推估(2020至2070年)報告.pdf.
[2] 台灣趨勢股份有限公司, "TTR 台灣趨勢研究報告, 產業分析:運動服務業發展趨勢," 2018. [Online]. Available: http://www.twtrend.com/upload/shares/a_15299856880.pdf.
[3] J. H. Wilmore, D. L. Costill, and W. L. Kenney, Physiology of sport and exercise. Human Kinetics, 1994.
[4] 中華民國衛生福利部國民健康署, "促進健康體能的方法," 2018. [Online]. Available: https://www.hpa.gov.tw/Pages/Detail.aspx?nodeid=571&pid=882.
[5] F. P. Wieringa, F. Mastik, and A. F. van der Steen, "Contactless multiple wavelength photoplethysmographic imaging: a first step toward "SpO2 camera" technology," Annual Review of Biomedical Engineering, vol. 33, no. 8, pp. 1034-41, 2005.
[6] K. Humphreys, T. Ward, and C. Markham, "Noncontact simultaneous dual wavelength photoplethysmography: a further step toward noncontact pulse oximetry," Review of Scientific Instruments, vol. 78, no. 4, p. 044304, 2007.
[7] W. Verkruysse, L. O. Svaasand, and J. S. Nelson, "Remote plethysmographic imaging using ambient light," Opt. Express, vol. 16, no. 26, pp. 21434-45, 2008.
[8] L. A. Aarts et al., "Non-contact heart rate monitoring utilizing camera photoplethysmography in the neonatal intensive care unit - a pilot study," Early Human Development, vol. 89, no. 12, pp. 943-8, 2013.
[9] G. A. Ramirez, O. Fuentes, S. L. Crites, M. Jimenez, and J. Ordonez, "Color Analysis of Facial Skin: Detection of Emotional State," in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2014, pp. 474-479.
[10] Y. C. Tsai, P. W. Lai, P. W. Huang, T. M. Lin, and B. F. Wu, "Vision-Based Instant Measurement System for Driver Fatigue Monitoring," IEEE Access, vol. 8, pp. 67342-67353, 2020.
[11] M. Z. Poh, D. J. McDuff, and R. W. Picard, "Non-contact, automated cardiac pulse measurements using video imaging and blind source separation," Opt. Express, vol. 18, no. 10, pp. 10762-10774, 2010.
[12] M. Z. Poh, D. J. McDuff, and R. W. Picard, "Advancements in Noncontact, Multiparameter Physiological Measurements Using a Webcam," IEEE Transactions on Biomedical Engineering, vol. 58, no. 1, pp. 7-11, 2011.
[13] M. Lewandowska, J. Rumiński, T. Kocejko, and J. Nowak, "Measuring pulse rate with a webcam — A non-contact method for evaluating cardiac activity," Proceedings of Federated Conference on Computer Science and Information Systems (FedCSIS), pp. 405-410, 2011.
[14] H. Monkaresi, R. A. Calvo, and H. Yan, "A Machine Learning Approach to Improve Contactless Heart Rate Monitoring Using a Webcam," IEEE Journal of Biomedical and Health Informatics, vol. 18, no. 4, pp. 1153-1160, 2014.
[15] X. Li, J. Chen, G. Zhao, and M. Pietikäinen, "Remote Heart Rate Measurement from Face Videos under Realistic Situations," in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014, pp. 4264-4271.
[16] D. McDuff, S. Gontarek, and R. W. Picard, "Improvements in Remote Cardiopulmonary Measurement Using a Five Band Digital Camera," IEEE Transactions on Biomedical Engineering, vol. 61, no. 10, pp. 2593-2601, 2014.
[17] J. Rumiński, "Reliability of pulse measurements in videoplethysmography," Metrology and Measurement Systems, vol. 23, no. 3, pp. 359-371, 2016.
[18] C. Zhao, C. L. Lin, W. Chen, and Z. Li, "A Novel Framework for Remote Photoplethysmography Pulse Extraction on Compressed Videos," in IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2018, pp. 1380-138009.
[19] E. M. Nowara, T. K. Marks, H. Mansour, and A. Veeraraghavan, "SparsePPG: Towards Driver Monitoring Using Camera-Based Vital Signs Estimation in Near-Infrared," in IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2018, pp. 1353-135309.
[20] C. Wang, T. Pun, and G. Chanel, "A Comparative Survey of Methods for Remote Heart Rate Detection From Frontal Face Videos," Frontiers in Bioengineering and Biotechnology, vol. 6, no. 33, 2018.
[21] Y. Sun and N. Thakor, "Photoplethysmography Revisited: From Contact to Noncontact, From Point to Imaging," IEEE Transactions on Biomedical Engineering, vol. 63, no. 3, pp. 463-477, 2016.
[22] A. Al-Naji, K. Gibson, S. Lee, and J. Chahl, "Monitoring of Cardiorespiratory Signal: Principles of Remote Measurements and Review of Methods," IEEE Access, vol. 5, pp. 15776-15790, 2017.
[23] X. Chen, J. Cheng, R. Song, Y. Liu, R. Ward, and Z. J. Wang, "Video-Based Heart Rate Measurement: Recent Advances and Future Prospects," IEEE Transactions on Instrumentation and Measurement, vol. 68, no. 10, pp. 3600-3615, 2019.
[24] G. de Haan and V. Jeanne, "Robust Pulse Rate From Chrominance-Based rPPG," IEEE Transactions on Biomedical Engineering, vol. 60, no. 10, pp. 2878-2886, 2013.
[25] G. de Haan and A. van Leest, "Improved motion robustness of remote-PPG by using the blood volume pulse signature," Physiological measurement, vol. 35, no. 9, pp. 1913-1926, 2014.
[26] R. Y. Huang and L. R. Dung, "A motion-robust contactless photoplethysmography using chrominance and adaptive filtering," in Proceedings of IEEE Biomedical Circuits and Systems Conference (BioCAS), 2015, pp. 1-4.
[27] L. Feng, L. Po, X. Xu, Y. Li, and R. Ma, "Motion-Resistant Remote Imaging Photoplethysmography Based on the Optical Properties of Skin," IEEE Transactions on Circuits and Systems for Video Technology, vol. 25, no. 5, pp. 879-891, 2015.
[28] W. Wang, S. Stuijk, and G. d. Haan, "A Novel Algorithm for Remote Photoplethysmography: Spatial Subspace Rotation," IEEE Transactions on Biomedical Engineering, vol. 63, no. 9, pp. 1974-1984, 2016.
[29] W. Wang, A. C. d. Brinker, S. Stuijk, and G. d. Haan, "Algorithmic Principles of Remote PPG," IEEE Transactions on Biomedical Engineering, vol. 64, no. 7, pp. 1479-1491, 2017.
[30] B. F. Wu, P. W. Huang, C. H. Lin, M. L. Chung, T. Y. Tsou, and Y. L. Wu, "Motion Resistant Image-Photoplethysmography Based on Spectral Peak Tracking Algorithm," IEEE Access, vol. 6, pp. 21621-21634, 2018.
[31] Y. C. Lin and Y. H. Lin, "Step Count and Pulse Rate Detection Based on the Contactless Image Measurement Method," IEEE Transactions on Multimedia, vol. 20, no. 8, pp. 2223-2231, 2018.
[32] 葉柏毅, "進階的非接觸式心率量測系統在運動器材的應用," 國立臺灣科技大學, 2020.
[33] T. Tamura, Y. Maeda, M. Sekine, and M. Yoshida, "Wearable Photoplethysmographic Sensors—Past and Present," Electronics, vol. 3, no. 2, 2014.
[34] Y. C. Lin and Y. H. Lin, "A study of color illumination effect on the SNR of rPPG signals," in Proceedings of IEEE International Conference of the Engineering in Medicine and Biology Society (EMBC), 2017, pp. 4301-4304.
[35] S. B. Park, G. Kim, H. J. Baek, J. H. Han, and J. H. Kim, "Remote Pulse Rate Measurement From Near-Infrared Videos," IEEE Signal Processing Letters, vol. 25, no. 8, pp. 1271-1275, 2018.
[36] P. Viola and M. Jones, "Robust real-time face detection," International Journal of Computer Vision, vol. 57, no. 2, pp. 137-154, 2004.
[37] H. Bay, T. Tuytelaars, and L. Van Gool, "SURF: Speeded Up Robust Features," in Proceedings of European Conference on Computer Vision (ECCV), 2006, pp. 404-417.
[38] N. Dalal and B. Triggs, "Histograms of oriented gradients for human detection," in Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), 2005, vol. 1, pp. 886-893.
[39] R. Girshick, J. Donahue, T. Darrell, and J. Malik, "Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation," in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014, pp. 580-587.
[40] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, "You Only Look Once: Unified, Real-Time Object Detection," in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 779-788.
[41] W. Liu et al., "SSD: Single Shot MultiBox Detector," in Proceedings of European Conference on Computer Vision (ECCV), 2016, pp. 21-37.
[42] OpenCV, "face detector." [Online]. Available: https://github.com/opencv/opencv/tree/master/samples/dnn/face_detector.
[43] K. He, X. Zhang, S. Ren, and J. Sun, "Deep Residual Learning for Image Recognition," in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770-778.
[44] R. M. Fouad, O. A. Omer, and M. H. Aly, "Optimizing Remote Photoplethysmography Using Adaptive Skin Segmentation for Real-Time Heart Rate Monitoring," IEEE Access, vol. 7, pp. 76513-76528, 2019.
[45] D. Chai and K. N. Ngan, "Face segmentation using skin-color map in videophone applications," IEEE Transactions on Circuits and Systems for Video Technology, vol. 9, no. 4, pp. 551-564, 1999.
[46] G. Kukharev and A. Nowosielski, "Visitor Identification - Elaborating Real Time Face Recognition System," World Society for Computer Graphics, 2004.
[47] D. A. Ross, J. Lim, R. S. Lin, and M. H. Yang, "Incremental Learning for Robust Visual Tracking," International Journal of Computer Vision, vol. 77, no. 1, pp. 125-141, 2008.
[48] X. Jia, H. Lu, and M. H. Yang, "Visual tracking via adaptive structural local sparse appearance model," in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2012, pp. 1822-1829.
[49] P. Pérez, C. Hue, J. Vermaak, and M. Gangnet, "Color-Based Probabilistic Tracking," in Proceedings of European Conference on Computer Vision (ECCV), 2002, pp. 661-675.
[50] A. Adam, E. Rivlin, and I. Shimshoni, "Robust Fragments-based Tracking using the Integral Histogram," in Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), 2006, vol. 1, pp. 798-805.
[51] D. Comaniciu, V. Ramesh, and P. Meer, "Kernel-based object tracking," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25, no. 5, pp. 564-577, 2003.
[52] G. Bradski, "Computer Vision Face Tracking For Use in a Perceptual User Interface," 1998.
[53] J. F. Henriques, R. Caseiro, P. Martins, and J. Batista, "High-Speed Tracking with Kernelized Correlation Filters," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 37, no. 3, pp. 583-596, 2015.
[54] M. Danelljan, G. Häger, F. S. Khan, and M. Felsberg, "Discriminative Scale Space Tracking," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 8, pp. 1561-1575, 2017.
[55] H. Nam and B. Han, "Learning Multi-domain Convolutional Neural Networks for Visual Tracking," in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 4293-4302.
[56] M. Danelljan, G. Häger, F. S. Khan, and M. Felsberg, "Convolutional Features for Correlation Filter Based Visual Tracking," in Proceedings of IEEE International Conference on Computer Vision Workshop (ICCVW), 2015, pp. 621-629.
[57] T. Vojir, J. Noskova, and J. Matas, "Robust scale-adaptive mean-shift for tracking," Pattern Recognition Letters, vol. 49, pp. 250-258, 2014.
[58] M. Kristan et al., "The Visual Object Tracking VOT2015 Challenge Results," in Proceedings of IEEE International Conference on Computer Vision Workshop (ICCVW), 2015, pp. 564-586.
[59] T. B. Fitzpatrick, "The validity and practicality of sun-reactive skin types I through VI," Arch Dermatol, vol. 124, no. 6, pp. 869-71, 1988.
[60] Dlib, "Dlib C++ library." [Online]. Available: https://github.com/davisking/dlib.
[61] C. C. Hsieh, D. H. Liou, and M. K. Jiang, "Fast enhanced face-based adaptive skin color model," in Proceedings of SPIE - The International Society for Optical Engineering, 2010, vol. 7820.
[62] C. Sagonas, G. Tzimiropoulos, S. Zafeiriou, and M. Pantic, "300 Faces in-the-Wild Challenge: The First Facial Landmark Localization Challenge," in Proceedings of IEEE International Conference on Computer Vision Workshops (ICCVW), 2013, pp. 397-403.
[63] "Polar H7 heart rate sensor." [Online]. Available: https://www.polar.com/en/products/accessories/H7_heart_rate_sensor.
[64] "Vernier Go Direct EKG sensor." [Online]. Available: https://www.vernier.com/product/go-direct-ekg-sensor/.