Basic Search / Detailed Display

Author: 陳威詔
Wei-chao Chen
Thesis Title: 基於前臂輪廓的即時掌心追蹤以及手勢判斷
Real-Time Palm Tracking and Hand Gesture Estimation Based on Fore-Arm Contour
Advisor: 范欽雄
Chin-Shyurng Fahn
Committee: 洪西進
Shi-Jinn Horng
李建德
Jiann-Der Lee
王聖智
Sheng-Jhih Wang
Degree: 碩士
Master
Department: 電資學院 - 資訊工程系
Department of Computer Science and Information Engineering
Thesis Publication Year: 2011
Graduation Academic Year: 99
Language: 英文
Pages: 80
Keywords (in Chinese): 人機互動手勢辨識指尖偵測掌心偵測計算幾何電腦視覺
Keywords (in other languages): HCI, Gesture Recognition, Fingertip Detection, Palm Detection, Computation Geometry, Computer Vision
Reference times: Clicks: 214Downloads: 2
Share:
School Collection Retrieve National Library Collection Retrieve Error Report

在人機介面發展的歷程中,一直朝著人性化與簡單化的方式不斷的持續成長。尤其是近幾年更是有爆炸性的突破,隨著各種新方式的出現,傳統的硬體按鍵輸入已逐漸被取代。觸控面板從單點控制進入到多點控制,體感技術的出現更擺脫了按鍵的束縛,讓人機介面更貼近人類的行為模式。體感中最具重要性的部位便是手部的控制。鑒於此,我們希望提出對手部重要資訊能夠更精確定位的系統。
本論文提供一個使用視訊攝影機之影像處理系統。和以往手勢辨識不同的地方在於,我們並不是將手勢化成數種指定的指令,而是藉由計算幾何的方式將手部的重要資訊:手指、手掌的位置準確的標出,提供資訊讓手部與系統做即時的互動。利用計算幾何帶來的優點,本系統可以在包含前臂的情況下準確的判斷出手心的位置,並且容許手掌和手臂一定程度的翻轉。大大的提高了掌心辨識所能掌握的自由度。
實驗結果顯示指尖的辨識準確度為99.1%,在手掌手臂不旋轉傾斜下的掌心辨識率為99.91%,在手臂傾斜下的掌心辨識率為99.53%,在手掌傾斜下的掌心辨識率為93.57%,在手掌手臂旋轉下的掌心辨識率為90.69%。


The development of HCI (human computer interface) is continuously pursuing an user-friendly interface and simplification system. There are some innovated breakthroughs recently. The traditional keyboard-input has been replaced gradually since many new methods had been invented. The touch panel evolves from single-touch to a multi-touch interface. The body sense technology even gets rid of restriction of input device; make the HCI closer to human's nature action. The most important part of it will be manipulation using hand. Hence, we propose a system which is accurate in locating some important features of hand.
In this thesis, we proposed an image processing system using a web camera. Differently from other hand recognition method, we are not trying to transfer the gesture to some certain instructions. We mark up the important features of hand: fingertips, palm center by computation geometry calculation, provide real-time interaction between gesture and the system. Within the advantages brought by computation geometry method, our system can accurately locate the palm center even when the fore-arm is involved. And the system tolerates a certain rotation of palm and fore-arm, which enhances the freedom of use in palm center estimation.
The experiment result shows that the accuracy is 99.1% for fingertip detection. Accuracy for palm position estimation will be 99.91% when the arm and hand are not tilt and rotate. Accuracy for palm position estimation will be 99.53% when the arm is tilt. Accuracy for palm position estimation will be 93.57% when the hand is tilt. Accuracy will be 90.69% for palm position estimation when the arm and hand are rotated.

中文摘要 i Abstract ii 致謝 iv Table of Contents v Chapter1 Introduction 1 1.1 Overview 1 1.2 Motivation 2 1.3 System Description 2 1.4 Thesis Organization 3 Chapter 2 Related Works 4 Chapter 3 Pre-Processing and Convex-Hull Method 9 3.1 Preliminary Processing 10 3.1.1 Skin color detection using HSV color space 10 3.1.2 Morphology Processing 13 3.1.3 Contour Finding 15 3.2 Convex Hull 21 3.2.1 Convex Polygon 22 3.2.2 Definition of Convex Hull 23 3.2.3 Three-Coin Algorithm 24 3.3 Convexity Defect 32 Chapter 4 Palm Tracking and Fingertip Detection 38 4.1 Minimum Enclosing Circle 39 4.1.1 Find Circle through Three Points 41 4.1.2 Angle Calculation by Dot Product 43 4.1.3 Skyum’s algorithm 44 4.2 Fingertip Detection 48 4.3 Buffer for palm position 55 4.4 Mix of palm position and average depth point position 56 4.5 Add an Extra Point when there is less Than Two Depth Points 58 4.6 Distance Threshold for Fingertips 60 Chapter 5 Experiment Results 62 5.1 Experiment 1 – Straight Arm and Hand 63 5.2 Experiment 2 – Tilt Arm 65 5.3 Experiment 3 – Tilt Hand Palm 68 5.4 Experiment 4 – Rotated Arm 70 5.5 Experiment Data 72 5.6 Extend to two hands 75 Chapter 6 Conclusions and Future Work 76 6.1 Conclusions 76 6.2 Future Work 77 References 78

[1] H. C. Xu, “Principal Component Analysis on Fingertips for Gesture Recognition” Master Thesis, Department of Applied Marine physics & Undersea Technology, National Sun Yat-sen University, Kaohsiung, Taiwan, 2006.
[2] A. Anagnostopoulos, A. Pnevmatikakis, “A real-time mixed reality system for seamless interaction between real and virtual objects”, Proceedings of the 3rd international conference on Digital Interactive Media in Entertainment and Arts, September 10-12, 2008, Athens, Greece
[3] G. Amayeh, G. Bebis, A. Erol, and M. Nicolescu, “Hand-Based Verification and Identification Using Palm-Finger Segmentation and Fusion,” Computer Vision and Image Understanding, vol. 113, pp.477-501, 2009.
[4] Weiying Chen, Fujiki, R.; Arita, D.; Taniguchi, R.-I, “Real-Time 3D hand Shape Estimation based on Image Feature Analysis and Inverse Kinematics”, Image Analysis and Processing, 2007. ICIAP 2007. 14th International Conference, pp.247-252, 2007
[5] Y.H. Chang and C.M. Chang, “Automatic Hand-Pose Trajectory Tracking System Using Video Sequences”, Master Thesis, Department of Information and Computer Engineering, Chung-Yuan Christian University, Chung-Li, Taiwan, 2010.
[6] M. Soriano, S. Huovinen, B. Martinkauppi, and M. Laaksonen, “Using the skin locus to cope with changing illumination conditions in color-based face tracking,” in Proceedings of the IEEE Nordic Signal Processing Symposium, pp. 383-386, 2000.

[7] T. Pavlidis, “Algorithms for Graphics and Image Processing”, Computer Science Press, Rockville, Maryland, 1982.
[8] R. L. Graham. “An efficient algorithm for determining the convex hull of a finite planar set”, Information Processing Letters, 7:175–180, 1972.
[9] Sklansky J., "Measuring Concavity on a Rectangular Mosaic", IEEE Transactions on Computing, 21, p1355, 1972.
[10] M. Soriano, S. Huovinen, B. Martinkauppi, and M. Laaksonen, “Using the skin locus to cope with changing illumination conditions in color-based face tracking,” in Proceedings of the IEEE Nordic Signal Processing Symposium, pp. 383-386, 2000.
[11] R.C. Gonzalez and R.E. Woods, Digital Image Processing, 2nd Ed., Addison Wesley, Reading, Massachusetts, 1992.
[12] S. L. Phung, A. Bouzerdoum, and D. Chai, “A novel skin color model in YCbCr color space and its application to human face detection,” in Proceedings of IEEE International Conference on Image Processing, vol. 1, pp. 289-291, 2002.
[13] X. Wen, Y. Niu, “A Method for Hand Gesture Recognition Based on Morphology and Fingertip- Angle”, The 2nd International Conference on Computer and Automation Engineering (ICCAE), vol. 1, pp. 688-691, 2010
[14] N. Megiddo, "Linear-time algorithms for linear programming in R^3 and related problems," SIAM Journal on Computing 12 (1983), No. 4, 759–776
[15] S. Skyum, “A simple algorithm for computing the smallest enclosing circle”, Information Processing Letters, vol. 37, Issue 3, 1991
[16] P. Bourke, “Equation of a Circle from 3 Points (2 dimensions)”, http://paulbourke.net/geometry/circlefrom3/ , Jan. 1991.
[17] http://en.wikipedia.org/wiki/Dot_product

[18] C. Manresa, J. Varona, R. Mas, F.J. Perales, “Real –Time Hand Tracking and Gesture Recognition for Human-Computer Interaction”, Electronic Letters on Computer Vision and Image Analysis 0(0):1-7, 2000
[19] C.W. Ng, S. Ranganath, “Real-time gesture recognition system and application”, Image and Vision Computing 20, pp. 993–1007, 2002
[20] Xiaoming Yin, Ming Xie, “Finger identification and hand posture recognition for human–robot interaction” ScienceDirect Image and Vision Computing, vol.25 , PP. 1291-1300, 2007.

QR CODE