隨著科技越來越發達，人類與機器互動的方式也越來越多，並且也越來越人性化，本論文即時表情辨識系統為一個人機互動系統，透過影像處理的方式，將攝影機擷取出的影像進行人臉偵測以及人臉表情辨識，並進一步利用表情辨識的結果，輸出互動的對話與使用者進行互動。
本論文提出一個以個人電腦做為平台來實現的表情辨識系統，以人臉偵測為基礎，進一步發展出表情辨識系統，最後實現與使用者互動的功能，此系統分為三大部分：(1)人臉候選區域擷取、(2)人臉驗證與表情辨識，及最後的(3)機器語音互動。人臉候選區域主要是以前處理的方式計算出可能的人臉區域並擷取出來，並且交由第二部分，擷取LBP特徵向量後進行人臉候選區域的分類。第二部分的分類分為兩個階段：(1)首先人臉候選區域須先經過人臉驗證之分類，(2)通過人臉驗證並且被分類為人臉之影像，接著進入第二階段進行表情辨識的分類。最後機器語音互動的部分，則是依據表情辨識之結果，進行分析後透過語音輸出互動之對話。
本系統使用C語言進行實作，並且在個人電腦做為平台，實驗結果顯示以表情資料庫進行測試，其交叉驗證之結果可達84.78%。另外以網路攝影機做為影像擷取工具，輸入影像大小為640X480像素的序列影像進行測試，並在實驗過程中嘗試刪除人臉中較不具有表情特徵的部分，以不同維度的人臉表情特徵向量進行辨識率、花費時間等等的實驗比較。

With new technological advances, interaction between human and machine has been varied. In this thesis, an interaction-oriented real-time facial expression recognition system is applied for the usage of interaction. Through image processing, face detection and expression recognition system is used to detect human face in the image and recognize the expression on the face. Further, we analyze the recognition results and output the corresponding speech to certain expression in order to interaction with the user.
In this thesis, we proposed a real-time facial expression recognition system based on PC. The research has developed the expression recognition on the basis of face detection and finally can interact with user. The system consists of three parts: (1) Face candidate extraction, (2) Face verification & expression recognition, and (3) Machine reaction. In the first part, we extract the face candidate in the image using pre-processing method. The face candidate image is then classified in the second part by extracting Local Binary Pattern feature. The classification in the second part is done using two stages: (1) first, the candidate image is classified into face image or non-face image, (2) the face image is then classified into six type of expression. In the last part, the output speech is played according to the classified results produced in a time period.
The system is implemented using C language and based on PC. In the experiments, we use a facial expression database to perform cross validation and recognition rate is 84.78%. Further, we use webcam as the input serial image and compare the recognition rate and run time differences between the non-deleting-blocks case and deleting-blocks case, which contain less expression information.

[1] He, L., Chao, Y., Suzuki, K., & Wu, K. “Fast connected-component labeling,” Pattern Recognition, vol 42, No.9, pp.1977-1987, 2009.
[2] Hsu, R. L., Abdel-Mottaleb, M., & Jain, A. K. “Face detection in color images,” IEEE transactions on pattern analysis and machine intelligence, vol 24, No.5, pp.696-706, 2002.
[3] Viola, P., & Jones, M. J. “Robust real-time face detection,” International journal of computer vision, vol 57 No.2, pp.137-154, 2004.
[4] Ojala, T., Pietikainen, M., & Maenpaa, T. “Multiresolution gray-scale and rotation invariant texture classification with local binary patterns,” IEEE Transactions on pattern analysis and machine intelligence, vol 24, No.7, pp.971-987, 2002.
[5] Ojala, T., Pietikäinen, M., & Harwood, D. “A comparative study of texture measures with classification based on featured distributions,” Pattern recognition, vol 29, No.1, pp.51-59, 1996.
[6] Zhang, L., Chu, R., Xiang, S., Liao, S., & Li, S. Z. “Face detection based on multi-block lbp representation,” In International Conference on Biometrics, pp. 11-18, 2007.
[7] Ahonen, T., Hadid, A., & Pietikainen, M. “Face description with local binary patterns: Application to face recognition,” IEEE transactions on pattern analysis and machine intelligence, vol 28, No.12, pp.2037-2041, 2006.
[8] Shan, C., Gong, S., & McOwan, P. W. “Robust facial expression recognition using local binary patterns,” IEEE International Conference on Image Processing 2005, vol 2, pp. II-370, 2005.
[9] Kanade, T., Cohn, J. F., & Tian, Y. “Comprehensive database for facial expression analysis,” IEEE International Conference on Automatic Face and Gesture Recognition, 2000, pp. 46-53, 2000.
[10] Wang, L., Li, R. F., Wang, K., & Chen, J. “Feature representation for facial expression recognition based on FACS and LBP,” International Journal of Automation and Computing, vol 11, No.5, pp.459-468, 2014.
[11] Shan, C., & Gritti, T. “Learning Discriminative LBP-Histogram Bins for Facial Expression Recognition,” British Machine Vision Conference, pp.1-10, 2008.
[12] Shan, C., Gong, S., & McOwan, P. W. “Facial expression recognition based on local binary patterns: A comprehensive study,” Image and Vision Computing, vol 27 No.6, pp.803-816, 2009.
[13] Zilu, Y., & Xieyan, F. “Combining LBP and Adaboost for facial expression recognition,” In 9th International Conference on Signal Processing, pp.1461-1464, 2008.
[14] Schapire, R. E., & Singer, Y. “Improved boosting algorithms using confidence-rated predictions,” Machine learning, vol 37, No.3, pp.297-336, 1999.
[15] Wang, Y., Ai, H., Wu, B., & Huang, C. “Real time facial expression recognition with adaboost.” In 17th International Conference on Pattern Recognition, vol. 3, pp. 926-929, 2004.
[16] Wu, B., Ai, H., & Huang, C. “LUT-based Adaboost for gender classification,” In International Conference on Audio-and Video-Based Biometric Person Authentication , pp.104-110, 2003.
[17] Phung, S. L., Bouzerdoum, A., & Chai, D. “A novel skin color model in YCbCr color space and its application to human face detection,” International Conference on Image Processing, vol 1, pp.I-289, 2002.
[18] 王乃堅、洪信甫，“即時瞳孔辨識系統” 台灣科技大學碩士論文，民國一百零三年。
[19] Murala, S., Maheshwari, R. P., & Balasubramanian, R. “Local tetra patterns: a new feature descriptor for content-based image retrieval,” IEEE Transactions on Image Processing, vol 21, No.5, pp.2874-2886, 2012.