簡易檢索 / 詳目顯示

研究生: 孫敬勛
Jing-Xun Sun
論文名稱: 人臉表情辨識技術應用於新生兒照護之研究
Application of Facial Expression Recognition Schemes to Baby Care Systems
指導教授: 徐勝均
Sendren Sheng-Dong Xu
口試委員: 謝劍書
Chien-Shu Hsieh
林顯易
Hsien-I Lin
林紀穎
Chi-Ying Lin
學位類別: 碩士
Master
系所名稱: 工程學院 - 自動化及控制研究所
Graduate Institute of Automation and Control
論文出版年: 2015
畢業學年度: 104
語文別: 中文
論文頁數: 91
中文關鍵詞: 表情辨識Adaboost主成分分析臉部動作單元支持向量機區域賈伯二値化圖型
外文關鍵詞: Action Unit, Support Vector Machime, Local Gabor Binary Pattern
相關次數: 點閱:593下載:1
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報
  • 本研究基於影像處理與型樣分類技術提出一類新生兒照護系統之建構概念,利用辨識新生兒之臉部表情以電腦自動判斷新生兒是否需要旁人之協助照顧,同時減輕照護者的壓力。本研究將攝影機放置在新生兒的正上方,拍攝新生兒影像。本系統分為兩部分,包括:新生兒臉部偵測以及表情辨識及分類。在臉部偵測部分,我們利用Adaboost演算法,挑選出適當的Haar-like矩型特徵,最後訓練出一個強健分類器來取得臉部所在位置。在表情辨識部分,我們提出利用少量得臉部特徵區塊,例如;眼睛、鼻子、嘴巴等區域,藉由特徵區塊的變化組合出特定表情模型,我們可將嬰兒表情分為數個類別,包括哭、笑、呆滯、吐奶….等。為了提升準確率以及辨識出特殊情況下嬰兒的表情,本研究利用主成份分析(Principal Component Analysis, PCA)與線性鑑別分析(Linear Discriminate Analysis, LDA)的方法準確地擷取特徵區域,如此一來,可以避免以臉部幾何形狀特徵方法,所產生五官相對位置因臉型不同造成的誤差,之後利用眼睛的位置將臉部分為上、下兩部份,並將其影像截取後,及獲得基本動作單元,再利用遮罩示Gabor濾波器與區域二元圖型(Local binary patterns, LBP)計算特徵値,最後經由一對一投票示支持向量機辨識動作單元種類,依此做為辨識表情之依據。


    Through image processing and pattern classification technology, the neonatal care system concept was built in this study. The identification of facial expressions of newborns relieved caregiver stress. In this study, a camera was placed right above the newborns to shoot images of the newborns. This system is divided into two parts: detecting the face of the newborn and the identification and classification of their expressions. In the face detection, the AdaBoost algorithm was used to select suitable Haar-like rectangular features. Finally, a robust classifier partook in the training to obtain the location of the face. In the identification of facial expressions, a small number of facial feature blocks, such as eye, nose, mouth, and other areas, were used. Through the changes in the feature blocks, specific facial expression models were formed. The facial expressions of infants can be divided into several types: crying, laughing, sleepiness, disgust, etc. In order to enhance accuracy and identify facial expressions of infants under special circumstances, the Principal Component Analysis (PCA), and the Linear Discriminate Analysis (LDA), were adopted to accurately extract feature areas. In this way, errors in the relative locations of the five features on different face shapes obtained using the geometric features method could be avoided. Then, based on the location of the eyes, the face was divided into two parts: the upper face and the lower face. After extracting images of these two parts and after capturing the basic action units, the mask Gabor filter and the local binary patterns (LGBP) were used to calculate the eigenvalues. Finally, the administration of one-on-one voting in support vector machine’s (SVM) identified action unit types served as a basis for identifying facial expressions.

    中文摘要 I Abstract II 致謝 III 目錄 IV 圖目錄 VII 表目錄 X 第1章 簡介 1 1.1研究背景與動機 1 1.2論文架構 4 第2章 文獻探討 5 2.1人臉偵測 5 2.2表情辨識 6 2.2.1幾何特徵 6 2.2.2外貌特徵 8 第3章 臉部偵測 11 3.1矩形特徵 11 3.2積分影像 16 3.3AdaBoost演算法 18 第4章 臉部特徵定位 23 4.1臉部動作編碼系統(Fical Actional Coding System, FACS) 23 4.2基於主成份分析(PCA)與線性鑑別分析(LDA)眼睛偵測特徵擷取 25 4.2.1成份分析(Principal Component Analysis, PCA) 25 4.2.2線性鑑別分析(Linear Discriminate Analysis, LDA) 30 4.3支持向量機(Support Vector Machome, SVM)之眼睛偵測 34 第5章 表情特徵擷取 50 5.1 動作單元(Action Units, AUs)擷取 50 5.2 動作單元(AUs)特徵值擷取 52 5.2.1 賈伯斯濾波器(Gabor Filiter) 52 5.2.2區域二元化圖型(Lcal Binary Pattern, LBP) 56 5.2.3區域賈伯二元化圖型(Lcal Gabor Binary Pattern, LGBP) 59 第6章 基本動作單元辨識 60 6.1 Action Units 辨識 60 6.2 支持向量機一對一訓練(One-Against-One) 60 6.2.1投票法 61 6.2.2二元樹法 61 6.3支持向量機一對多訓練(One-Against-Rest) 63 6.4 一對一與一對多的比較 65 6.5基本動作單元影像辨識流程 67 第7章 實驗結果與分析 69 7.1 建立表情辨識規則 69 7.2 基本動作單元實驗訓練樣本 74 7.4基本動作單元辨識結果 76 7.5表情辨識結果 78 第8章結論與未來展望 79 8.1結論 79 8.2未來工作 79 參考文獻 81

    [1] 內政部統計司網站: http://statis.moi.gov.tw/micst/stmain.jsp?sys=100
    [2] M. H. Yang and N. Ahuja, “Detecting human faces in color images,” IEEE International Conference on Image Processing, pp. 127-139, Oct. 1998.
    [3] R. L. Hsu, M. Abdel-Mottaleb, and A. K. Jain, “Face detection in color images,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 5, pp. 696-706, 2002.
    [4] K. Sobottka and I. Pitas, “Extraction of facial regions and features using colorand shape information”, International Conference on Pattern Recognition, pp.421-425, Vienna, Austria, Aug. 1996.
    [5] C. Chen and S. P. Chiang, “Detection of human face in colour image,” IEEE International Conference on Image Signal Process, Taichung, Taiwan vol. 144, pp. 384-388, 1997
    [6] H. Wu, Q. Chen, and M. Yachida, “Face detection from color image using afuzzy pattern matching method,” IEEE Trans. Pattern Anlysis and Machine Intelligence, vol. 14, no. 6, pp. 557-563, 1999.
    [7] P. Zhang, “A Video-based face detection and recognition system using cascade face verification modules,” Proceedings of the Applied Imagery Pattern Recognition Workshop, Washington, DC , 2008.
    [8] Q. H. Thu, M. Meguro, and M. Kaneko, “Skin color extraction in images with complex background and varying illumination,” Proceedings of the Sixth IEEE Workshop on Applications of Computer Vision, Chofu, Japan, pp. 280-285, 2002.
    [9] J. C. Terillon, M. David, and S. Akamatus, “Detection of human face in complex scene images by use of skin color model and of invariant Fourier-Mellin moments,” IEEE International Conference, Kyoto, vol. 2,pp. 1350-1355, 1998.
    [10] R. L. Hsu, M. Abdel-Mottable, and A. K. Jain, “Face detection in color images,” IEEE Trans. Pattern Anlysis and Machine Intelligence, vol. 24, no. 5, pp. 696-706, 2002.
    [11] P. Kakumanu, S. Makrogiammis, and N. Bourbakis, “A survey of skin-color modeling and detection methods,” Patter Recognition, vol. 40, pp. 1160-1122, 2007.
    [12] P. Viola and M. J. Jones, “Robust real-time face detection,” International Journal of Computer Vision, vol. 57, pp. 137-154, 2004.
    [13] Y. Freund and R. E. Schapire, “A desicion-theoretic generalization of on-line learning and an application to boosting,” Computational Learning Theory, pp. 23-37, 1995.
    [14] C. P. Papageorgiou, M. Oren, and T. Poggio, “A general framework for object detection,” in Computer vision, International Conference on Computer vision, pp. 555-562, 1998.
    [15] B. Jun, I. Choi, and D. Kim, “Local transform features and hybridization for accurate face and human detection,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 35, no. 6, pp. 1423-1436, 2013.
    [16] Y. Ban, S.-K. Kim, S. Kim, K.-A. Toh, and S. Lee, “Face detection based on skin color likelihood,” Pattern Recognition, vol. 47, pp. 1573-1585, 2014.
    [17] P. Ekman and W. V. Friesen, “The facial action coding system: A technique for the measurement of facial movement,” San Francisco: Consulting Psychologists Press, 1978.
    [18] Y. Tiang, T. Kanade, and J. Cohn, “Recognizing action units for facial expression analysis,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 2, pp. 97-115, 2001.
    [19] N. Vretos, N. Nikolaidis, and I. Pitas, “A model-based facial expression recognition algorithm using principal components analysis,” IEEE International Conference on Image Processing, Cairo, Egypt, vol. 7, pp. 3301-3304, Nov. 7-10, 2009.
    [20] I. Kotisa and I. Paitas, “Facial expression recognition in image sequences using geometric deformation features and support vector machines,” IEEE Transon. Image Processing, vol. 16, no. 1, pp. 172-187, 2007.
    [21] M. Pantic and I. Patras, “Dynamics of facial expression: Recognition of facial ctions and their temporal segments from face profile image sequences,” IEEE Trans. Systems, Man and Cybernetics-Part B, vol. 36(2), pp. 433-449, 2006.
    [22] R. Toth and A. Madabhushi, “Multifeature Landmark-Free Active Appearance Models: Application to Prostate MRI Segmentation,” IEEE Trans. Medical Imaging, vol. 31, no. 8, pp. 1638-1650, 2012.
    [23] I. Kotsia and I. Pitas, “Facial expression recognition in image sequences using geometric deformation features and support vector machines,” IEEE Trans. Image Processing, vol. 16, no. 1, pp. 172-187, Jan. 2007.
    [24] A. Asthana, J. Saragih, M. Wanger, and R. Goecke, “Evaluating AAM fitting methods for facial expression recognition,” IEEE International Conference on Affective Computing and Workshops, Amsterdam, Netherlands, pp.1-8, Dec 10-12, 2009.
    [25] T. Okada, T. Takiguchi, and Y. Ariki, “Pose robust and person independent facial expression recognition using AAM selection,” IEEE International Symposium on Consumer Electronics, pp.637-638, May 25-28, 2009.
    [26] Y. Wang, Z. Zhang, W. Li, and F. Jiang, “Combining tensor space analysis and active appearance models for aging effect simulation on face images,” IEEE Trans. Systems, Man, and Cybernetics, vol. 42, no. 4, pp. 1107-1118, 2012
    [27] Y. Chen, F. Yu, and C. Ai, “Sequential active appearance model based on online instance learning,” IEEE Signal Processing Letters, vol. 20, no. 6, pp. 567-570, 2013.
    [28] W. Gu, C. Xiang, Y. V. Venkatesh, D. Huang, and H. Lin, “Facial expression recognition using radial encoding of local gabor features and classifier synthesis,” Pattern Recognition, vol. 45(1), pp. 80–91, Jan. 2012.
    [29] E. Stefanos, R. Ognjen, and P. Maja, “Discriminative shared gaussian processes for multiview and view-invariant facial expression recognition,” IEEE Trans. Image Processing, vol. 20, no. 1, pp. 189-204, 2015.
    [30] K. H. Wen, T. Y. Han, and H. T. Chiou, “Dual subspace nonnegative graph embedding for identity-independent expression recognition,” IEEE Trans. Information Forensic, vol. 10, no. 3, pp. 626-639, 2015.
    [31] W. F. Liu and Z. F. Wang, “Facial expression recognition based on fusion of multiple gabor features,” International Conference on Pattern Recognition, Hong Kong, vol. 3, pp.536-539, Sep. 2006.
    [32] G. U. Kharat and S. V. Dudul, “Human emotion recognition system using optimally designed SVM With different facial feature extraction techniques,” WSEAS Trans. Computers, Vol 7, no. 6, Jun. 2008.
    [33] P. Yang, Q. Liu, and D. N. Metaxas, “Rankboost with l1 regularization for facial expression recognition and intensity estimation,” International Conference on Computer Vision, pp. 1018–1025, Sept. 29-Oct.2, 2009.
    [34] D. H. Kim, S. U. Jung, and M. J. Chung, “Extension of cascaded simple feature based face detection to facial expression recognition,” Pattern Recognition Letters, vol. 29, pp. 1621-1631, 2008.
    [35] L. H. Cheng, and L. B. Liang, “Multi-view gender classification using multi-resolution local binary patterns and support vector machines,” International Journal of Neural Systems, Vol. 17, No. 6, pp. 479-487, 2007.
    [36] N. Werghi, S. Berretti and A. D. Bimbo, “The Mesh-LBP: A framework for extracting localbinary patterns from discrete manifolds,” IEEE Trans. Image Processing, Vol. 24, no. 1, pp. 220-235, 2015.
    [37] F. J. Xu, and M. Savvides, “Subspace-based discrete transform encoded local binary patterns representations for robust periocular matching on NIST’s face recognition grand challenge,” IEEE Trans. Image Processing, vol. 23, no. 8, pp. 3490-2505, 2014.
    [38] S. Taheri, V. M. Patel, and R. Chellappa, “Component-based recognition of facesand facial expressions,” IEEE Trans. Affective Computing, vol. 4, no. 4, pp. 360-371, 2013.
    [39] T. Senechal, V. Rapp, H. Salam, R. Seguier, K. Bailly, and L. Prevost, “Facial action recognition combining heterogeneous features via multikernel learning,” IEEE Biometrics Compendium, vol. 42, no. 4, pp. 993-1005, 2012.
    [40] L. Yongqiang, W. Shangfei, Z. Yongping, and J. Senior, “Simultaneous facial feature tracking and facial expression recognition,” IEEE Trans. Image Processing, vol. 22, no. 7, pp. 2559-2573, 2013.
    [41] E. Stefanos, R. Ognjen, and P. Maja, “Discriminative shared gaussian processes for multiview and view-invariant facial expression recognition,” IEEE Transon. Image Processing, vol. 24, no. 1, pp. 189-204, 2015.
    [42] P. Ruiz, J. Mateos, G. C. Valls, R. Molina, and A. K. Katsggelos, “Bayesian active remote sensing image classification,” IEEE Trans. Geoscience and Remote Sensing, vol. 52, no. 4, pp. 2186-2196, 2014.
    [43] Z. Yuling, A. Fraysse, and T. Rodet, “Efficient variational bayesian approximation method based on subspace optimization,” IEEE Trans. Image Processing, vol. 204, no. 2, pp. 681-693. 2015.
    [44] K. Kayabol, E. E. Kuruoglu, and B. Sankur, “Bayesian separation of images modeled with MRFs Using MCMC,” IEEE Trans. Image Processing, vol. 18, no. 5, pp. 982-994, 2009.
    [45] S. M. Lajevardi, M. Lech, “Facial expression recognition using neural networks and Log-Gabor Filters,” Digital Image Computing: Techniques and Applications, pp. 77-83, Dec. 2008.
    [46] X.-F. Bai and W-J. Wang, “An approach for facial expression recognition based on neutral network ensemble,” International Joint Conference on Neural Networks (IJCNN), vol. 1, no. 2161-4393, pp.19-23, Jul. 2009.
    [47] J.-M. Keller and Y. Hayashi, “Evidence aggregation networks for fuzzy logic inference,” IEEE Trans. Neural Networks, vol. 3, no. 5, pp. 334-338, 1993.
    [48] J. Hua, “Application of fuzzy neural network in multi-maneuvering target tracking,” International Asia Conference on Informatics in Control, Automation and Robotics, Wuhan, China, vol. 1, pp. 92-95. March 6-7, 2010.
    [49] C.-T. Lin and Y.-C. Lu, “A neural fuzzy system with fuzzy supervised learning,” IEEE Trans. Systems, Man, and Cybernetics, part B: Cybernetics, vol. 26, no. 5, 1996.
    [50] R. Ballini and F. Gomide, “Recurrent fuzzy neural computation: modeling, learning and application,” IEEE International Conference on Fuzzy Systems, Barcelona, Spain, July 18-23, 2010, pp. 1-6.
    [51] R. Ballini and F. Gomide, “Recurrent fuzzy neural computation: modeling, learning and application,” IEEE International Conference on Fuzzy Systems, Barcelona, Spain, pp. 1-6, July 18-23, 2010.
    [52] A.-Z. Wang and G.-F. Ren, “The design of neural network fuzzy controller in washing machine,” International Conference on Computing, Measurement, Control and Sensor Network, Taiyuan, China ,July 7-9, 2012, pp. 136-139.
    [53] W. Jing and C. L. P. Chen, “Finding the near optimal learning rates of fuzzy neural networks (FNNs) via its equivalent fully connected neural networks (FFNNs),” International Conference on System Science and Engineering, Da’lian, China, June 30-July 2, 2012, pp. 137-142, June 30-July 2, 2012.
    [54] A. P. Lemos, W. Caminhas, and F. Gomide, “A fast learning algorithm for uniform-based fuzzy neural networks,” Fuzzy Information Processing Society ,” Annual Meeting of the North American, pp. 1-6, 2012.
    [55] Y.-H. Huang, H. Cheng, L. Huang, and L. Sun, “Soft sensing modeling based on dynamic fuzzy neural network for penicillin fermentation,” Proceedings of the Chinese Control Conference, Hefei, China, pp. 3383- 3388, July 25-27, 2012.
    [56] D. Xue and S. Hao, “Estimation of project costs based on fuzzy neural network,” World Congress on Information and Communication Technologies, Dalian, China, pp. 1177-1181, Oct. 30-Nov. 2, 2012.
    [57] Z. Wanqing, L. Kang, and G.-W. Irwin, “A new gradient descent approach for local learning of fuzzy neural models,” IEEE Trans. Fuzzy Systems, vol. 21, no. 1, 2013.
    [58] S. T. Chen, and P. S. Yu, “Pruning of support vector networks on flood forecasting,” Proceeding of Journal of Hydrology, vol. 347, pp. 67-78, no. 15, Dec. 2007.
    [59] T. D. Nguyen, Q. T. Thanh, T. M. Duc, T. N. Quynh, and T. M. Hoang, “SVM classifier based face detection system using BDIP and BVLC moments,” Advanced Technologies for Communications, Hanoi, Vietnam, pp. 264-267, 2011.
    [60] W. C. Na and Q. J. Zhang, “Automated knowledge-based neural network modeling for microwave applications,” IEEE Microwave and Wireless Components Letters, vol. 24, no. 7, pp. 499-501, 2014.
    [61] J. C. Huang, X. Zhang, Q. Zhou, E. Song, and B. Li, “A practical fundamental frequency extraction algorithm for motion parameters estimation of moving targets,” IEEE Trans. Instrumentation and Measurement, vol. 63, no. 2, pp. 267-276, 2014.
    [62] C. F. Lin and S. D. Wang, “ Fuzzy support vector machines”, IEEE Trans. Neural Networks, vol. 13, no. 2, 2002.
    [63] A.-A. Miranda, Y.-A. Le Borgne, and G. Bontempi, “New routes from minimal approximation error to principal Components,” Neural Processing Letters, vol. 27, no. 3, 2008.
    [64] L. U. Yuan, “Gait recognition based on fuzzy support vector machine,” Computer Engineering, vol. 35, no. 21, pp. 189-191, 2009.
    [65] H. Abdi, and L. J. Williams, “Principal Component Analysis Wiley Interdisciplinary Reviews: Computational Statistics,” vol. 2, pp. 433-459, 2010.
    [66] L. Lei and K.-N. Zhao, “A new intrusion detection system based on rough set theory and fuzzy support vector machine,” International Workshop on Intelligent Systems and Applications, Wuhan, China, pp. 1-5, May 28-29, 2011.
    [67] L. Chen, Y. C. Yang, and M. Yao, “Reliability detection by fuzzy SVM with UBM component feature for emotional speaker recognition,” International Conference on Fuzzy Systems and Knowledge Discovery, Shanghai, China, vol. 1, pp. 458-461, July 26-28, 2011.
    [68] B. Lakshmanan, A. Jeril, Priscilla, S. Ponni, and V. Sankari, “Evaluation of imbalanced datasets using fuzzy support vector machine-class imbalance learning (FSVM-CIL),” IEEE International Conference on Recent Trends in Information Technology, Chennai, Tamil Nadu, pp. 1131-1136 June 3-5, 2011.
    [69] M. Serafeim, M. Giorgos, K. Nikos, and B. John, “SVM-based fuzzy decision trees for classification of high spatial resolution remote sensing images,” IEEE Trans. Geoscience and Remote Sensing, vol. 50, no. 1, pp. 149-169, 2012.
    [70] T. Hao, L. Yuhe, and W. Xiufeng, “A new fuzzy membership function for fuzzy support vector machine and its application in machinery fault diagnosis,” International Conference on Natural Computation, Chongqing, China, pp. 35-39, May 29-31, 2012.
    [71] N. Nuryani, S. H. Ling, and H. T. Nguyen, “Hybrid particle swarm - based fuzzy support vector machine for hypoglycemia detection,” IEEE World Congress on Computational Intelligence, Brisbane, Australia, Brisbane, QLD, pp. 10-15, June 10-15, 2012.
    [72] Z. Wenchao, S. Shiguang, G. Wen, C. Xilin, and Z. Hongming, “Local gabor binary pattern histogram sequence (LGBPHS): A Novel non-statistical model for face representation and recognition,” IEEE International Conference on Computer Vision,China, pp. 786-791, Oct. 17-21, 2005.
    [73] D-Z. Tian, G-B. Peng, and M-H. Ha, “Fuzzy support vector machine based on non-equilibrium data,” International Conference on Machine Learning and Cybernetics, Xi’an, China, pp. 15-17, 2012.
    [74] Q-Y. Zhao, B-C. Pan, J-J. Pan and Y-Y. Tang, “Facial expression recognition based onfusion of Gabor and LBP features,” International Conference on Wavelet Analysis and Pattern Recognition, vol. 1, pp. 362-367, Aug 1-3. 2008.
    [75] S. M. Lajevardi and M. Lech, “Facial expression recognition using neural networks and Log-Gabor Filters,” Digital Image Computing: Techniques and Applications, pp. 77-83, Dec. 2008.
    [76] J-F. Ye, Y-Z. Zhan and S-L. Song, “Facial expression features extraction based on Gabor wavelet transformation,” IEEE International Conference on System, Man and Cybernetics, Zhenjiang, China, vol. 3, pp. 2215-2219, Oct1-13, 2004.
    [77] W-F. Liu and Z-F. Wang, “Facial expression recognition based on fusion of multiple gabor features,” International Conference on Pattern Recognition, vol. 3, pp.536-539, Sep. 2006.
    [78] G. Guo, G. Mu, Y. Fu, and T.S. Huang, “Human age estimation using bioInspired features,” IEEE International Conference on Computer Vision and Pattern Recognition,pp.112-119, 20-25 June 2009.
    [79] T. Ojala, M Pietikinen and T. Menp, “Multiresolution grayscale and rotation invarianttexture classifications with local binary patterns,” IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 24, no. 7, pp. 971-987, 2002.
    [80] C-J. Wen and Y-Z. Zhan, “HMM+KNN classifier for facial expression recognition,” IEEE Conference on Industrial Electronic and Applications, pp. 260-263, Jun3-5, 2008.
    [81] G. Guo, S.Z. Li, and K. L. Chan, “Support vector machines for face recognition,” Image and Vision Computing, vol 19, no 9-10, p 631-638, 2001.
    [82] C-W. Hsu and C-J. Lin, “A comparison of methods for multiclass support vector machines,” IEEE Trans. Neural Networks, vo. 13, no 2, March, pp. 415-425, 2002.

    無法下載圖示 全文公開日期 2020/10/22 (校內網路)
    全文公開日期 本全文未授權公開 (校外網路)
    全文公開日期 本全文未授權公開 (國家圖書館:臺灣博碩士論文系統)
    QR CODE