表情在日常生活中扮演著很重要的角色，是一種非語言的溝通方式，人們常藉由喜怒哀樂等情緒反應來表達與周遭環境之間的心理感受。本論文主要是發展一套自動化的表情辨識系統，可透過擷取數位攝影機的影像，自動分析去人臉的表情。本系統主要可分為三個部分，分別為「人臉偵測」、「特徵擷取」及「表情辨識」。
人臉偵測部分，本研究使用Adaboost訓練學習演算法，挑選適當的Haar-like矩形特徵，最後將多種矩形特徵加以組合，並配合相應的權重值，訓練出一個強健的分類器來偵測及定位出臉部的位置。
特徵擷取部分，透過五官的邊緣特性及灰階值差異，使用一維積分投影將眼睛、眉毛及嘴巴的區域劃分出來，把人臉偵測影像及五官區域影像當作第一層特徵區域，而上述影像取樣成低解析度的影像當作第二層的特徵區域。本論文將在上述兩層的區域使用區域二元結構(LBP)去擷取紋理資訊並結合離散餘旋轉換(DCT)所保存的低頻資訊做為表情辨識的特徵。
表情辨識部分則使用近年來相當熱門的機器學習系統：SVM分類器，來做為表情分類的依據。我們將上述所擷取到的表情特徵，包括DCT低頻的係數值及LBP的直方圖統計轉換成表情特徵向量，輸入到SVM分類器作表情辨識。
本論文所提出的方法用以分類7種不同的表情，包括開心、生氣、悲傷、驚訝、害怕、厭惡及中性表情，並使用著名的JAFFE表情影像資料庫來做測試，實驗結果證實本論文所提出的方法與前人文獻相比即使在較低的解析度下仍可得較好的辨識率，最後本論文使用PTZ攝影機、USB影像擷取卡及筆記型電腦和RS232傳輸線，實現一套可用於監控環境下的自動化人臉表情辨識系統。

Facial expression is a non-verbal communication media, which plays an important role in daily life. People often switch emotions and other emotional reactions according to the surrounding environment and the psychological feelings. This thesis aims to develop an automatic facial expression recognition system which can detect human faces, extract features, and recognize the corresponding expressions.
In face detection phase, the Adaboost learning algorithm is applied to select the appropriate Haar-like features, and combine all weak classifiers with different weights to form a strong classifier for detecting face.
In feature extraction, each detected face image is divided into three local images such as eye, eyebrow and mouth based on organ location, and which are used to form the first layer areas. Then, the original face image and the local images are down-sampled to form the second layer areas. Subsequently, the Local Binary Pattern (LBP) and the Discrete Cosine Transform (DCT) operators are applied on the two types of areas to extract the texture information and the noise-free low frequency information.
The recognition stage is based on the well-known SVM classifier. The extracted global/local information from DCT coefficients and LBP histograms is transformed to expression feature vector and then fed to the SVM classifier for facial expression recognition. In this thesis, the proposed expression classification scheme recognizes seven various facial expressions, such as happy, angry, sad, surprise, fear, disgust and neutral with the JAFFE database. In the experimental results, the proposed method can yield superior performance compared to former approaches even with image of lower resolutions. Finally, the equipments used in this research including PTZ camera, USB video capture card, notebook and RS232 transmission. These equipments realize the proposed automatic facial expression recognition system, and which can be considered as an effective candidate for the surveillance applications.

[1] M. Turk and A. Pentland, "Eigenfaces for recognition," Journal of Cognitive Neuroscience, vol. 3, no. 1, pp. 71-86, Jan. 1991.
[2] E. Hjelmas and B. K. Low, "Face detection: a survey," Computer Vision and Image Understanding, vol. 83, no. 3, pp. 236-274, Sep. 2001.
[3] C. Garica, G. Tziritas and G. ziritas, "Face detection in color image using Wavelet Packet Analysis," IEEE Conference on Multimedia Computing and systems," vol. 1, pp. 703-709, Jul. 1999.
[4] C. Garica and G. ziritas, "Face detection using quantized skin color regions Merging and wavelet packet analysis," IEEE Transactions on Multimedia," vol. 1, no. 3, pp. 264-277, Sep. 1999.
[5] S. Karungaru, M. Fukumi, N. Akamatsu, "Face recognition using genetic algorithm based template matching, " proceeding of the IEEE, vol. 2, pp.1252-1257, Oct. 2004.
[6] P. Viola and M. J. Jones, "Robust real-time face detection," International Journal of Computer Vision, vol. 57, no. 2, pp. 137-154, May 2004.
[7] C. P. Papageorgiou, M. Oren and T. Poggio, "A general framework for object detection," Sixth International Conference on Computer Vision, pp. 555-562, Jan. 1998.
[8] P. Viola and M. J. Jones, "Robust real-time object detection," Second International Workshop on Statistical and Computational Theories of Vision, Jul. 2001.
[9] R. Lienhart, A. Kuranov and V. Pisarevsky, "Empirical analysis of detection cascades of boosted classifiers for rapid object detection," Pattern Recognition, vol. 2781, pp. 297-304, Sep. 2003.
[10] D. H. Kim, S. U. Jung and M. J. Chung, "Extension of cascaded simple feature based face detection to facial expression recognition," Pattern Recognition Letters, vol. 29, no. 11, pp. 1621-1631, Aug. 2008.
[11] Y. Freund and R. E. Schapire, "A decision-theoretic generalization of on-line learning and an application to boosting," Journal of Computer and System Sciences, vol. 55, no. 1, pp.119-139, Aug. 1997.
[12] 趙楠，「基於AdaBoost算法的人臉偵測」，北京大學，北京，2005。
[13] G. Lei, X. H. Li, J. L. Zhou and X. G. Gong, "Geometric feature based facial expression recognition using multiclass support vector machines," IEEE International Conference on Granular Computing, pp. 318-321, Aug. 2009.
[14] Q. Zhang, Z. Liu, G. Quo, D. Terzopoulos, and H. Y. Shum, "Geometry-driven photorealistic facial expression synthesis," IEEE Transactions on Visualization and Computer Graphics, vol. 12, no. 1, pp. 48-60, Feb. 2006.
[15] Q. Y. Zhao, B. C. Pan, J. J. Pan and Y. Y. Tang, "Facial expression recognition based on fusion of Gabor and LBP features ," International Conference on Wavelet Analysis and Pattern Recognition, vol. 1, pp. 362-367, Aug. 2008.
[16] S. M. Lajevardi, M. Lech, "Facial Expression Recognition Using Neural Networks and Log-Gabor Filters," Digital Image Computing: Techniques and Applications, pp. 77-83, Dec. 2008.
[17] J. F. Ye, Y. Z. Zhan and S. L. Song, "Facial expression features extraction based on Gabor wavelet transformation," IEEE International Conference on System, Man and Cybernetics, vol. 3, pp. 2215-2219, Oct. 2004.
[18] W.F. Liu, Z.F. Wang, "Facial Expression Recognition Based on Fusion of Multiple Gabor Features," 18th International Conference on Pattern Recognition, vol. 3, pp.536-539, Sep. 2006.
[19] A. Asthana, J. Saragih, M. Wanger and R. Goecke, "Evaluating AAM fitting methods for facial expression recognition," 3rd International Conference on Affective Computing and Workshops, pp.1-8, Dec. 2009.
[20] T. Okada, T. Takiguchi, Y. Ariki, "Pose robust and person independent facial expression recognition using AAM selection," IEEE 13th International Symposium on Consumer Electronics, pp.637-638, May. 2009
[21] G. U. Kharat, S. V. Dudul, "Human Emotion Recognition System Using Optimally Designed SVM With Different Facial Feature Extraction Techniques," Wseas Transactions on Computers, Issue 6, Volume 7, Jun. 2008.
[22] K. Cheng, Y. Chen, Y. Zhan, "A New Approach for Facial Expression Recognition Based on Burial Markov Model," Fifth International Conference on Fuzzy Systems and Knowledge Discovery, vol. 4, pp.1-3, Oct. 2008.
[23] J. Yang, X. Ling, Y. Zhu, Z. Zheng, "A face detection and recognition system in color image series, "Mathematics and Computers in Simulation," vol. 77, pp. 531-539, 2008.
[24] H. Wang, S. Z. Li, Y. Wang J. Zhang, "Self Quotient Image for Face Recognition," International Conference on Image Processing, vol. 2, pp.1397-1400, Oct. 2004.
[25] J. Cao, C. Tong, "Facial expression recognition based on LBP-EHMM," Congress on Image and Signal Processing, vol. 2, pp.371-375, May. 2008.
[26] K. Ebihara, J. Ohya, F. Kishino, "A Study of Real Time Facial Expression Detection for Virtual Space Teleconferencing," IEEE International Workshop on Robot and Human Communication, pp. 247-252, Jul. 1995.
[27] B. Jiang, G. S. Yang and H. L. Zhang, "Comparative study of dimension reduction and recognition algorithms of DCT and 2DPCA," International Conference on Machine Learning and Cybernetics vol. 1, pp. 407-410, Jul. 2008.
[28] L. B. Cai and Z. L. Ying, "A new approach of facial expression recognition based on Contourlet Transform," ICWAPR, pp. 275-280, Jul. 2009.
[29] Z. Y. Ying and X. Fang, "Combining LBP and Adaboost for facial expression recognition," 9th International Conference on Signal Processing, pp. 1461-1464, Oct. 2008.
[30] M. W. Huang, Z. W. Wang and Z. L. Ying "A new method for facial expression recognition based on sparse representation plus LBP," 3rd International Congress on Image and Signal Processing, pp. 1250-1253, Oct. 2010.
[31] Z. L. Ying, Y. W. Zhang and J. W. Li, "Manifold learning approach to facial expression recognition on local binary pattern features," International Conference on Machine Learning and Cybernetics, vol. 1, pp. 405-410, Jul. 2009.
[32] M. W. Huang, Z. W. Wang and Z. L. Ying "A novel method of facial expression recognition based on GPLVM plus SVM," 10th International Conference on Signal Processing, pp. 916-919, Oct. 2010.
[33] T. Ojala, M Pietikinen and D. Harwood, "A comparative study of texture measures with classification with local binary patterns," Pattern Recognition, vol. 29, no. 1, 1996.
[34] T. Ojala, M Pietikinen and T. Menp, "Multiresolution grayscale and rotation invariant texture classifications with local binary patterns," IEEE PAMI, vol. 24, no. 7, 2002.
[35] C. J. Wen, Y.Z. Zhan, "HMM+KNN classifier for facial expression recognition," 3rd IEEE conference on Industrial Electronic and Applications, pp. 260-263, Jun. 2008.
[36] K. Numata, K. Numata, S. Miyano, "A Structure Learning Algorithm for Inference of Gene Networks from Microarray Gene Expression Data Using Bayesian Networks," Bioinformatics and Bioengineering, pp. 1280-1284, Oct. 2007.
[37] S. M. Lajevardi, M. Lech, "Facial Expression Recognition Using Neural Networks and Log-Gabor Filters," Digital Image Computing: Techniques and Applications, pp. 77-83, Dec. 2008.
[38] X. F. Bai and W. J. Wang, "An approach for facial expression recognition based on neutral network ensemble," vol. 1, pp.19-23, Jul. 2009.
[39] T. V. Gestel, B. Baesens, J. Suykens, M. Espinoza, D.E. Baestaens, J. Vanthienen, B.D. Moor, "Bankruptcy Predictiob with Least Squares Support Vector Machine Classifiers," Proceeding of the IEEE International Conference on Computational Intelligence for Financial Engineering, pp.1-8, Mar. 2003.
[40] S. T. Chen, P. S. Yu, “Pruning of support vector networks on flood forecasting,” Proceeding of Journal of Hydrology, vol.347, pp67-78 15, Dec. 2007.
[41] Y. B. Dibike, S. Velickov, D. Solomatine, M.B. Abbott, "Model Induction With Support Vector Machines: Introduction And Application," Proceeding of the Journal of Computing in Civil Engineering, 2001.
[42] T. Gunes and E. Polat, "Feature selection for multi-SVM classifiers in facial expression classification," 23rd International Symposium on Computer and Information Science, pp. 1-5, Oct. 2008.
[43] C. Cortes and V. Vapnik, "Support-vector networks," Machine Learning, vol. 20, no. 3, pp. 273-297, Sep. 1995.
[44] B. E. Boser, I. M. Guyon and V. N. Vapnik, "A training algorithm for optimal margin classifiers," Fifth Annual ACM Workshop on Computational Learning Theory, pp. 144-152, Jul. 1992.
[45] V. Vapnik, "Estimation of dependences based on empirical data," Springer-Verlag New York, 1982.
[46] E. Osuna, R. Freund and F. Girosi, "An improved training algorithm for support vector machines," IEEE Workshop Neural Networks for Signal Processing, pp. 276-285, Sep. 1997.
[47] J. C. Platt, "Fast training of support vector machines using sequential minimal optimization," Advances in Kernel Methods - Support Vector Learning, pp. 185-208, Jan.1999.
[48] C. C. Chang and C. J. Lin, "LIBSVM : a library for support vector machines," 2001. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm
[49] M. Lyons, S. Akamasku, M. Kamachi, and J. Gyoba. “ Coding facial expressions with gabor wavelets”, In Proceedings of International Conference on Face and Gesture Recognition, pp. 200-205, Apr. 1998.
[50] SONY EVI-D70技術手冊，http://pro.sony.com/bbsc/ssr/product-EVID70P/