Sign language recognition research has made significant progresses in recent years. The present progresses provide the basis method for future implementations with the objective of supporting the integration of deaf people into the hearing society. Translation systems, for example, could facilitate communication between deaf and hearing people in public situations. Further applications, such as user interfaces and automatic indexing of signed videos, become feasible. The current state in sign language recognition is roughly 30 years behind [1] speech recognition. Research efforts were mainly focused on robust feature extraction or statistical modeling of signs. However, current recognition systems are still designed for signer-dependent operation under laboratory conditions.
This thesis proposes a scheme to recognize isolated American Sign Language (ASL) by using RGB-D camera. We utilize 10 signs database that distinguished become 5 signs for one-handed sign language and 5 signs for two-handed sign language. The main idea of our proposed method is to recognize sign language by detecting and recognizing hand shapes. After we get the detected hand shape, then we utilize the distance trajectory feature of hand against the signer face center to classify the hand gesture. In this scheme, the hand shape detection is introduced first for further recognition process. The hand shape detection segments one or two hands according to the number of hands appearing in the image frame sequence by using background subtraction and skin color detection. Face detection is utilized to detect the signer and signer face center as threshold point. However, the skin color methods may fail in insufficient light conditions. Therefore, the adaptive lighting compensation is applied to help the skin color detection method become more accurate. Before performing hand shape recognition process, image enhancement is performed first because the obtained blobs consist of holes inside them. After obtaining masked enhanced image in gray scale color space then recognition process is performed. Principal Component Analysis (PCA) is applied to train and recognize obtained hand shapes. The hand distance trajectory is classified by using Dynamic Time Warping (DTW). To analyze the performance of proposed method the datasets established from the RGB-D camera under indoor environment are tested. The experimental results show the effectiveness of our proposed method with high hand shape recognition rate and good hand distance trajectory classification result.

Sign language recognition research has made significant progresses in recent years. The present progresses provide the basis method for future implementations with the objective of supporting the integration of deaf people into the hearing society. Translation systems, for example, could facilitate communication between deaf and hearing people in public situations. Further applications, such as user interfaces and automatic indexing of signed videos, become feasible. The current state in sign language recognition is roughly 30 years behind [1] speech recognition. Research efforts were mainly focused on robust feature extraction or statistical modeling of signs. However, current recognition systems are still designed for signer-dependent operation under laboratory conditions.
This thesis proposes a scheme to recognize isolated American Sign Language (ASL) by using RGB-D camera. We utilize 10 signs database that distinguished become 5 signs for one-handed sign language and 5 signs for two-handed sign language. The main idea of our proposed method is to recognize sign language by detecting and recognizing hand shapes. After we get the detected hand shape, then we utilize the distance trajectory feature of hand against the signer face center to classify the hand gesture. In this scheme, the hand shape detection is introduced first for further recognition process. The hand shape detection segments one or two hands according to the number of hands appearing in the image frame sequence by using background subtraction and skin color detection. Face detection is utilized to detect the signer and signer face center as threshold point. However, the skin color methods may fail in insufficient light conditions. Therefore, the adaptive lighting compensation is applied to help the skin color detection method become more accurate. Before performing hand shape recognition process, image enhancement is performed first because the obtained blobs consist of holes inside them. After obtaining masked enhanced image in gray scale color space then recognition process is performed. Principal Component Analysis (PCA) is applied to train and recognize obtained hand shapes. The hand distance trajectory is classified by using Dynamic Time Warping (DTW). To analyze the performance of proposed method the datasets established from the RGB-D camera under indoor environment are tested. The experimental results show the effectiveness of our proposed method with high hand shape recognition rate and good hand distance trajectory classification result.

1 Kraiss K.-F. (Ed.) Advanced Man-Machine Interaction: Fundamentals and Implementation, Springer Verlag, 2006.
2 Mole, J., McColl, H., and Vale, M., Deaf and Multilingual - A practical guide to teaching and supporting deaf learners in foreign language classes, Direct Learn Services Ltd , 2005.
3 Schneider, M., Stevens, C.: Development and testing of a new magnetic- tracking device for image guidance. Proceedings of SPIE 7035 65090I–65090I–11, 2007.
4 Wang, R.Y., Popović, J.: Real-time hand-tracking with a color glove. ACM Trans. Graph. 28 63:1–63:8, 2009.
5 Brashear, H., Park, K.-H., Lee, S., Henderson, V., Hamilton, H., and Starner, T., American Sign Language Recognition in Game Development for Deaf Children. In Assets ’06:Proceedings of the 8th International ACM SIGACCESS
Conference on Computers and Accessibility, Portland, Oregon. ACM Press, 2006.
6 Fujimura, K., Liu, X.: Sign recognition using depth image streams. In: Procs. of FGR, pp. 381 – 386. Southampton, UK, 2006.
7 Doliotis, P., Athitsos, V., Kosmopoulos, D., Perantonis, S., Hand Shape and 3D Pose Estimation Using Depth Data from a Single Cluttered Frame, Proc. Int. Symp. on Visual Computing (ISVC), 2012.
8 Karami, A., Zanj, B., Azadeh, K. S., Persian sign language (PSL) recognition using wavelet transform and neural networks, Expert Systems with Applications 38 (3) 2661–2667, 2011.
9 Oz, C., Leu, M. C., “American Sign Language word recognition with a sensory glove using artificial neural networks”, Engineering Applications of Artificial Intelligence 24 1204-1213 , 2011.
10 Auephanwiriyakul, S., Phitakwinai, S., Suttapak, W., Chanda, P., Theera, U. N., “Thai sign language translation using Scale Invariant Feature Transform and Hidden Markov Models ”, Pattern Recognition Letters 34 1291–1298 , 2013.

11 Yang, H-D., Lee, S-W., “Robust Sign Language Recognition by Combining Manual and Non-Manual Features Based on Conditional Random Field and Support Vector Machine”, Pattern Recognition Letters doi: http://dx.doi.org/10.1016/j.patrec.2013.06.022, 2013.
12 Zaki, M. M., Shaheen, S. I., “Sign language recognition using a combination of new vision based features”, Pattern Recognition Letters, Vol. 32: 572-577, 2011.
13 Sun, C., Zhang, T., Bao, K-B., Xu, C., Mei, T., “Discriminative Exemplar Coding for Sign Language Recognition with Kinect”, IEEE TRANSACTIONS ON CYBERNETICS, 2168-2267, 2013.
14 Li, Y., Chen, X., Zhang, X., Wang, K., Wang, J. Z., “A sign-component based framework for Chinese sign language recognition using accelerometer and sEMG data.,” IEEE Transactions on Biomedical Engineering, vol. 59, no. 10, pp. 2695–2704, 2012.
15 Kosmidou, E. V., Petrantonakis, C. P., Hadjileontiadis, J. L., “Enhanced sign language recognition using weighted intrinsic-mode entropy and signer’s level of deafness,” IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 41, no. 6, pp. 1531–1543, 2011.
16 Chen, L., and Grecos, C., "Fast skin color detector for face extraction," Proceedings of SPIE - The International Society for Optical Engineering, pp. 93-101, 2005.
17 Funt, B., Barnard, K., and Martin, L., "Is Machine Color Constancy Good Enough?," the 5th European Conference on Computer Vision, pp. 445-459, 1998.
18 Viola P. & Jones M., Rapid Object Detection using a Boosted Cascade of Simple Features, Computer Vision and Pattern Recognition Proceedings of the 2001 IEEE Computer Society Conference vol. 1: I-511 – I-518, 2001.
19 Chai, D., and Ngan, K. N., “Face segmentation using skin-color map in videophone applications,” IEEE Transactions on Circuits and Systems for Video
Technology, Vol. 9(4), pp. 551-564 , 1999.

20 Chen, L., and Grecos, C., "Fast skin color detector for face extraction,"Proceedings of SPIE - The International Society for Optical Engineering, pp. 93-101, 2005.
21 Lin, W.-C., “Skin Color Detection and Face Location in Different sceneries,” Master Thesis, National Central University, Taoyuan, Taiwan, 2009.
22 Phung, S. L., Bouzerdoum, A., and Chai, D., "A novel skin color model in YCbCr color space and its application to human face detection," International
Conference on Image Processing, pp. I-289-I-292 , 2002.
23 Soriano, M., Martinkauppi, B., Huovinen, S., and Laaksonen, M., "Using the skin Locus to cope with Changing Illumination Conditions in Color-Based Face Tracking," Proceedings of IEEE Nordic Signal Processing Symposium, pp. 383-386 , 2000.
24 Pai, Y. T., Ruan, S. J., Shie, M. C., and Liu, Y. C., "A Simple and Accurate Color
Face Detection Algorithm in Complex Background," International Conference on Multimedia and Expo, pp. 1545-1548, 2006.
25 Suzuki, S. and Abe, K., Topological Structural Analysis of Digitized Binary Images by Border Following . CVGIP 30 1, pp 32-46, 1985.