簡易檢索 / 詳目顯示

回結果列表
研究生： 楊崇男
Chung-nan Yang
論文名稱： 用於人機互動系統的即時雙手指尖鑑別與手勢辨識技術
Two-Hand Fingertip Identification and Gesture Recognition Techniques Applied for Human-Computer Interaction Systems in Real Time
指導教授： 范欽雄
Chin-shyurng Fahn
口試委員: 古鴻炎
Hung-yan Gu
王榮華
Jung-hua Wang
林啟芳
none
學位類別： 碩士
Master
系所名稱： 電資學院 - 資訊工程系
Department of Computer Science and Information Engineering
論文出版年： 2009
畢業學年度： 97
語文別： 英文
論文頁數： 115
中文關鍵詞： 人機互動 、手勢辨識 、人臉偵測 、指尖偵測 、掌心偵測 、特徵擷取 、虛擬滑鼠 、分類器
外文關鍵詞： human computer interaction, gesture recognition, face detection, fingertips detection, palm detection, feature extraction, virtual mouse, Classification
相關次數： 點閱：271  下載：3 
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報

    • 現今科技不斷在人機操控介面上力求精進，為強化其功能達到人性化及簡單化，使操控介面越來越朝向便利性，從過去硬體多鍵輸入介面到簡易觸控式面板控制介面，大大地改變人與機器間的溝通模式。以往觸控面板仍需以手拿觸控筆操控，這些年已簡化成人手指直接進行控制所有功能介面，面板也從單點式控制演化到今日的多點式控制介面。鑑於此，我們希望提出更進一步的演化操控介面，即無觸控面板的人機互動操控介面。

    本論文提出一個使用簡便視訊攝影機之影像處理，達到具有即時雙手動態手勢辨識功能的人機互動操控系統。我們的方法是針對人的雙手手指變化進行辨識，並將其辨識出的手勢對應於機器指令執行功能。本系統機器互動介面主要是以操控取代滑鼠與鍵盤的功能，目前辨識的手勢有100多種，其指令數足以取代滑鼠及鍵盤的按鍵數。但手勢與對應指令過多時，容易造成操控者需記憶每一鍵盤按鍵之指令，反而造成反效果。因此，本實驗將以12個手勢對應最常用之應用軟體控制功能，實驗結果顯示本系統的指令辨識度可達95%以上。

    Nowadays the science and technology is pursuing precision and making progress in human-computer interaction (HCI) interface to strengthen its function for user-friendly and simplification. The control panel switches from keyboard to a simple hand touch, which dramatically changes the communication pattern between human and computer. In the past, people had to use a touch pen to trigger the panel, but lately we can use fingers to directly control the panel. Furthermore, the panel is also evolved from a single-touch into today’s multi-touch interface. Based on these progresses, we render a HCI controlling interface which uses no panel at all.

    This thesis proposes an image process through a simple web-camera to achieve a real time HCI system of two-hand fingertips identification and gesture recognition techniques. Our approach is to use the changes of our hand gestures to conduct the identification and use the identified gestures to map the computer’s commands for carrying out different functions. This interface of HCI system mainly uses the controlling panel to replace the functions of mouse and keyboard. At present, there are more than 100 kinds of hand gestures, enough to replace the commands of the mouse and keyboard. However, if the gestures and the mapping commands are too many, users would have to memorize every command of the keyboard that will be a troublesome work. Therefore, this experiment will use 12 kinds of hand gestures to represent the most commonly-used commands. The experimental results demonstrate that the recognition ability of this system can reach over 95%.

    誌謝 i 中文摘要 ii Abstract iii Contents v List of Figures viii List of Tables xii Chapter 1 Introduction 1 1.1 Overview 1 1.2 Background and motivation 2 1.3 Thesis organization and system architecture 5 Chapter 2 Related Works 7 2.1 Reviews of face detection 7 2.2 Reviews of hands detection 10 2.3 Reviews of gesture recognition 12 Chapter 3 Face and Hand Region Detection 17 3.1 Color space transformation 18 3.1.1 Skin color detection using the HSV model 19 3.1.2 Hair color detection using YcbCr model 21 3.2 Connected component labeling 22 3.3 Face and hands separation 24 Chapter 4 Feature Extraction 27 4.1 Scan Converting Circles 28 4.1.1 Eight-Way Symmetry 28 4.1.2 Midpoint Circle Algorithm 29 4.2 Fingertip features extraction 35 4.3 Center of the Palm features extraction 37 Chapter 5 Gesture Recognition 40 5.1 Gesture definition 40 5.1.1 Direction definition 41 5.1.2 Feature definition 43 5.2 Multi-Layer Perceptrons 46 5.2.1 The back-propagation algorithm 46 5.2.2 The MLP-based classifier 50 5.3 Support Vector Machines 53 5.3.1 Linear support vector machines 53 5.3.2 Non-linear support vector machines 58 5.3.3 The SVM-based multi-classifier 60 5.4 Adaboosting Schemes 62 5.4.1 The AdaBoost algorithm 63 5.4.2 The weak classifier 69 5.4.3 The AdaBoost-base multi-classifier 71 Chapter 6 Experimental Results and Discussions 74 6.1 System interface description 75 6.2 The results of face and hand detection 77 6.3 The results of fingertip point and palm point extraction 79 6.4 Comparison of Three Different Classifiers 82 6.5 Experiments on human computer interaction control system 92 Chapter 7 Conclusions and Future Works 108 7.1 Conclusions 108 7.2 Future works 109 References 111

    [1] http://cs.nyu.edu/~jhan/ftirtouch/index.html
    [2] http://www.washingtonpost.com/wp-dyn/content/article/2008/02/04/AR2008020402796.html
    [3] L. Yu, D. Zhang, K. Wang “The relative distance of key point based iris recognition” ScienceDirect Pattern Recognition, Vol. 40 , PP. 423-430, 2007.
    [4] K. Oka, Y. Sato, H. Koike, “Real-time fingertip tracking and gesture recognition” Computer Graphics and Applications, IEEE Vol. 22, PP. 64 – 71, 2002.
    [5] K. Y. Wang, “A real-time face tracking and recognition system based on particle filtering and AdaBoosting techniques,” Master Thesis, Department of Computer Science Information Engineering, National Taiwan University of Science and Technology, Taipei, Taiwan, 2006.
    [6] G. R. Bradski, “Computer vision face tracking for use in a perceptual user interface,” Intel Technology Journal, vol. 2, no. 2, pp. 1-15, 1998.
    [7] C. Shan, Y. Wei, T. Tan, and F. Ojardias, “Real time hand tracking by combining particle filtering and mean shift,” in Proceedings of the Sixth IEEE International Conference on Automatic Face and Gesture Recognition, Seoul, South Korea, pp. 669-674, May 2004.
    [8] K. T. Song and W. J. Chen, “Face recognition and tracking for human-robot interaction,” in Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics, Hauge, Netherlands, pp. 2877-2882, Octorber 2004.
    [9] X. Liu and K. Fujimura, “Hand gesture recognition using depth data,” in Proceedings of the Sixth IEEE International Conference on Automatic Face and Gesture Recognition, Seoul, Korea, pp. 529-534 May, 2004.
    [10] H. Jiang, Z. N. Li, and M. S. Drew, “Human posture recognition with convex programming,” in Proceedings of the IEEE International Conference on Multimedia and Expo, Amsterdam, Netherlands, pp. 574-577, July 2005.
    [11] T. A. C. Bragatto, G. S. I. Ruas, M. V. Lamar, ”Real-time hand postures recognition using low computational complexity Artificial Neural Networks and Support Vector Machines” in Proceedings of the IEEE Communications, Control and Signal Processing, PP. 1530 – 1535, 2008.
    [12] M. H. Yang, D. Kriegman, and N. Ahuja, “Detecting faces in images: a survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 1, pp. 34-58, 2002.
    [13] G. Yang and T. S. Huang, “Human face detection in complex background,” Pattern Recognition, vol. 27, no. 1, pp. 53-64, 1994.
    [14] K. C. Yow and R. Cipolla, “Feature-based human face detection,” Image and Vision Computing, vol. 15, no. 9, pp. 712-735, 1997.
    [15] D. Chai and A. Bouzerdoum, “A Bayesian approach to skin color classification in YCbCr color space,” in Proceedings of the IEEE Region Ten Conference, Kuala Lumpur, Malaysia, vol. 2, pp. 421-424, September 2000.
    [16] A. Lanitis, C. J. Taylor, and T. F. Cootes, “An automatic face identification system using flexible appearance models,” Image and Vision Computing, vol. 13, no. 5, pp. 393-401, 1995.
    [17] R. Vaillant, C. Monrocq, and Y. Le Cun, “An original approach for the localization of objects in images,” in Proceedings of the IEEE Conference on Artificial Neural Networks, Brighton, United Kindom, pp. 26-30, May 1993.
    [18] M. Turk and A. Pentland, “Eigenfaces for recognition,” Journal of Cognitive Neuroscence, vol. 3, no. 1, pp. 71-86, 1991.
    [19] L. Ma and K. Khorasani “Facial expression recognition using constructive feedforward neural networks,” IEEE Transaction on Systems, Man, and Cybernetics, vol. 34, no. 3, pp. 1588-1595, 2004.
    [20] M. J. Lyons, J. Budynek, and S. Akamatsu, “Automatic classification of single facial images,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 21, no. 12, pp. 1357-1362, 1999.
    [21] Y. L. Tian, T. Kanade, and J. F. Cohn, “Evaluation of Gabor-wavelet-based facial action unit recognition in image sequences of increasing complexity,” in Proceedings of the 5th IEEE International Conference on Automatic Face and Gesture Recognition, Yorktown Heights, New York, pp. 229-234, 2002.
    [22] J. Lien, T. Kanade, J. Cohn, and C. Li, “Detection, tracking, and classification of action units in facial expression,” Robotics and Autonomous Systems, vol. 31, no. 3, pp. 131-146, 2000.
    [23] M. Kass, A. Witkin, and D. Terzopoulos, “Snakes: active contour models,” International Journal of Computer Vision, vol. 1, no. 4, pp.321-331, 1988.
    [24] T. F. Cootes, C. J. Taylor, D. H. Cooper, and J. Graham, “Active shape models - their training and application,” Computer Vision and Image Understanding, vol. 61, no. 1, pp.38-59, 1995.
    [25] T. F. Cootes, G. J. Edwards, and C. J. Taylor, “Active appearance models,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, no. 6, pp. 681-685, 2001.
    [26] K. Dorfmüller-Ulhaas and D. Schmalstieg, “Finger tracking for interaction in augmented environments,” in Proceedings of the IEEE and ACM International Symposium on Augmented Reality, pp. 55-64, 2001.
    [27] A. Vacavant and T. Chateau, “Realtime head and hands tracking by monocular vision,” in Proceedings of the IEEE International Conference on Image Processing, vol. 2, pp. 11-14, 2005.
    [28] H. C. Xu, “Principal Component Analysis on Fingertips for Gesture Recognition” Master Thesis, Department of Applied Marine physics & Undersea Technology, National Sun Yat-sen University, Kaohsiung, Taiwan, 2006.
    [29] Y. C. Lin, “Design and Implementation of a Vision-Based Fingertip Writing Interface” Master Thesis, Department of Computer Science and Information Engineering, National Dong-Hwa University, Hualien , Taiwan,2005.
    [30] Xiaoming Yin, Ming Xie, “Finger identification and hand posture recognition for human–robot interaction” ScienceDirect Image and Vision Computing, vol.25 , PP. 1291-1300, 2007.
    [31] M. Soriano, S. Huovinen, B. Martinkauppi, and M. Laaksonen, “Using the skin locus to cope with changing illumination conditions in color-based face tracking,” in Proceedings of the IEEE Nordic Signal Processing Symposium, pp. 383-386, 2000.
    [32] R. C. Gonzalez and R. E. Woods, Digital Image Processing, 2nd Ed., Addison-Wesley, Reading, Massachusetts, 1992.
    [33] S. L. Phung, A. Bouzerdoum, and D. Chai, “A novel skin color model in YCbCr color space and its application to human face detection,” in Proceedings of IEEE International Conference on Image Processing, vol. 1, pp. 289-291, 2002.
    [34] K. Suzuki, I. Horiba, and N. Sugie, “Linear-time connected-component labeling based on sequential local operations,” Source Computer Vision and Image Understanding Archive, vol. 89 , no. 1, pp. 1-23, 2003.
    [35] J. D. Foley, A. V. Dam, S. K. Feiner, J. F. Hughes, Computer Graphics Principles and Practice 2nd in C Ed., Addison-Wesley, 1997.
    [36] Y. Sato, Y. Kobayashi, H. Koike, “Fast Tracking of Hands and Fingertips in Infrared Images for Augmented Desk Interface,”in Proceedings of the IEEE Conference on Automatic Face and Gesture Recognition, pp. 462-467, 2000.
    [37] 王進德，蕭大全，類神經網路與模糊控制理論入門，全華科技股份有限公司，台北市，民國90年。
    [38] S. Theodoridis and K. Koutroumbas, Pattern Recognition, 3rd Ed., Elsevier, Academic Press, San Diego, California, 2006.
    [39] Y. Freund and R. E. Schapire, “Experiments with a new boosting algorithm,” in Proceedings of the 13th International Conference on Machine Learning, Bari, Italy, pp. 148-156, 1996.
    [40] J. Friedman, T. Hastie, and R. Tibshirani, “Additive logistic regression: a statistical view of boosting,” The Annals of Statistics, vol. 28, no. 2, pp. 337-407, 2000.
    [41] R. E. Schapire and Y. Singer, “Improved boosting algorithms using confidence-rated predictions,” Machine Learning, vol. 37, no. 3, pp. 297-336, 1999.
    [42] L. Breiman, J. Friedman, R. Olshen, and C. Stone, Classification and Regression Trees, Chapman and Hall, New York, 1984.

    QR CODE