臉部表情的辨識通常可用來判斷觀影人在觀看影片時，情緒之高低起伏及轉折。本研究有兩個目標：1）判斷臉部表情是否可以用來辨別內心的情緒狀態，2）研究如何利用對個人差異的暸解來對臉部表情辨識系統進行個人化情緒校正。本研究提出了一個新的方法，整合了人工智慧臉部情緒辨識模型與個人情緒校正模型來對人臉部表情進行情緒校正。首先，臉部情緒辨識模型以人工智慧神經網路EfficientNet作為架構，進行情緒分類及面部情緒分數計算，並將辨識出的情緒分數歷程轉換為個人觀影各類情緒分數。接著以隨機森林（Random Forest）演算法作為分類器，從本研究設計的問卷所收集之觀眾個人資料、人格特質與影片資訊作為輸入，來預測觀眾對影片的評分。為了驗證模型之可行性，本研究收集了十二部不同類型的喜劇或戲謔影片，並參考文獻設計了一個人格分析問卷及影片評分表，由觀影者在看完每部短片後立即評分，並在觀影後完成問卷。實驗結果顯示個人化差異因子與臉部情緒表情有顯著關聯，而且利用問卷所收集之個人化差異因子，在95%信賴區間下，可更準確地達到80％以上的準確度。從實驗結果也發現，不同面向的情緒分數能幫助個人情緒校正模型更有效的預測觀影者對影片的評分。

Facial emotion recognition has been used to recognize the emotional fluctuations of an audience while watching videos. There were two objectives in this research: 1) examine whether the facial expression could be used to identify the viewer’s emotion. 2) determine how to utilize the individual discrepancy to calibrate the developed facial emotion recognition individually. This research proposed a novel method integrating artificial intelligent facial emotion recognition model and individual emotion calibration model to calibrate facial expressions based on individual differences. This work applied the architecture of EfficientNet network to recognize the emotion change and classified it to different emotion categories. The data transformation was conducted to calculate the score of each category based on the emotion trajectory. Then, the Random Forest was utilized as the classifier to predict the audience's actual rating after watching the video clip by using the personal information, personality traits and video characteristics as inputs, collected from the conducted questionnaire afterward. To evaluate the model, 12 different comedic clips were collected. A personality traits questionnaire and a video rating form were designed with reference to the literatures. The experimental results reveal that the individual factors collected from the questionnaire were significantly related to recognized facial expressions by EfficientNet. The factors related to individual difference could enhance the model to reach over 80% classification accuracy with 95% confidence interval. The results also showed that various facial emotions scores could help the individual emotion calibration model predict the audience's rating more precisely.

[1] I. Motion Picture Association. (2021). THEME REPORT. Available: https://www.mpa-apac.org/
[2] I. R. Blacker, Elements of Screenwriting: A Guide for Film and Television Writing. Pearson College Div, 1996.
[3] A. Chakravarty, Y. Liu et al., "The differential effects of online word-of-mouth and critics' reviews on pre-release movie evaluation," Journal of Interactive Marketing, vol. 24, no. 3, pp. 185-197, 2010.
[4] A. L. Hadida, J. Lampel et al., "Hollywood studio filmmaking in the age of Netflix: a tale of two institutional logics," Journal of Cultural Economics, vol. 45, no. 2, pp. 213-238, 2021/06/01 2021.
[5] F. Agrafioti, D. Hatzinakos et al., "ECG pattern analysis for emotion detection," IEEE Transactions on affective computing, vol. 3, no. 1, pp. 102-115, 2011.
[6] Z. Cheng, L. Shu et al., "A novel ECG-based real-time detection method of negative emotions in wearable applications," in 2017 International Conference on Security, Pattern Analysis, and Cybernetics (SPAC), Shenzhen, China, 2017, pp. 296-301: IEEE.
[7] C. Darwin, The expression of the emotions in man and animals. University of Chicago press, 2015.
[8] Y.-I. Tian, T. Kanade et al., "Recognizing action units for facial expression analysis," IEEE Transactions on pattern analysis and machine intelligence
vol. 23, no. 2, pp. 97-115, 2001
[9] P. Ekman and W. V. Friesen, "Facial action coding system," Environmental Psychology & Nonverbal Behavior, 1978.
[10] R. Coreau, I. Pépin et al., "Deep learning models and techniques for facial emotion recognition," presented at the Proceedings of the 31st Annual International Conference on Computer Science and Software Engineering, Toronto, Canada, 2021.
[11] N. Mehendale, "Facial emotion recognition using convolutional neural networks (FERC)," SN Applied Sciences, vol. 2, no. 3, p. 446, 2020/02/18 2020.
[12] B. C. Ko, "A Brief Review of Facial Emotion Recognition Based on Visual Information," Sensors vol. 18, no. 2, p. 401, 2018.
[13] A. Majumder, L. Behera et al., "Emotion recognition from geometric facial features using self-organizing map," Pattern Recognition, vol. 47, no. 3, pp. 1282-1293, 2014.
[14] X. Liu, X. Cheng et al., "GA-SVM-Based Facial Emotion Recognition Using Facial Geometric Features," IEEE Sensors Journal, vol. 21, no. 10, pp. 11532-11542, 2020.
[15] M. Dahmane and J. Meunier, "Emotion recognition using dynamic grid-based HoG features," in 2011 IEEE International Conference on Automatic Face & Gesture Recognition (FG), Santa Barbara, CA, USA, 2011, pp. 884-888: IEEE.
[16] H. Meng, B. Romera-Paredes et al., "Emotion recognition by two view SVM_2K classifier on dynamic facial expression features," in 2011 IEEE International Conference on Automatic Face & Gesture Recognition (FG), Santa Barbara, CA, USA, 2011, pp. 854-859: IEEE.
[17] M. Xiaoxi, L. Weisi et al., "Facial emotion recognition," in 2017 IEEE 2nd International Conference on Signal and Image Processing (ICSIP), Singapore, 2017, pp. 77-81.
[18] D. Ghimire, S. Jeong et al., "Facial expression recognition based on local region specific features and support vector machines," Multimedia Tools and Applications, vol. 76, no. 6, pp. 7803-7821, 2017/03/01 2017.
[19] M. Mohammadpour, H. Khaliliardali et al., "Facial emotion recognition using deep convolutional networks," in 2017 IEEE 4th International Conference on Knowledge-Based Engineering and Innovation (KBEI), Tehran, Iran, 2017, pp. 0017-0021.
[20] S. Li and W. Deng, "Deep Facial Expression Recognition: A Survey," IEEE Transactions on Affective Computing, pp. 1-1, 2020.
[21] J. Haddad, O. Lézoray et al., "3D-CNN for Facial Emotion Recognition in Videos," in International Symposium on Visual Computing, 2020, pp. 298-309: Springer.
[22] M. Hu, H. Wang et al., "Video facial emotion recognition based on local enhanced motion history image and CNN-CTSLSTM networks," Journal of Visual Communication and Image Representation, vol. 59, pp. 176-185, 2019/02/01 2019.
[23] S. A. P. R. Sekaran, C. P. Lee et al., "Facial Emotion Recognition Using Transfer Learning of AlexNet," in 2021 9th International Conference on Information and Communication Technology (ICoICT), Yogyakarta, Indonesia, 2021, pp. 170-174.
[24] A. Sepas-Moghaddam, A. Etemad et al., "A Deep Framework for Facial Emotion Recognition using Light Field Images," in 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII), Cambridge, UK, 2019, pp. 1-7.
[25] M. N. A. Wahab, A. Nazir et al., "Efficientnet-Lite and Hybrid CNN-KNN Implementation for Facial Expression Recognition on Raspberry Pi," IEEE Access, vol. 9, pp. 134065-134080, 2021.
[26] M. Tan and Q. Le, "Efficientnet: Rethinking model scaling for convolutional neural networks," in International Conference on Machine Learning, 2019, pp. 6105-6114: PMLR.
[27] K. Simonyan and A. Zisserman, "Very Deep Convolutional Networks for Large-Scale Image Recognition," arXiv, 2014.
[28] S. Hamann and T. Canli, "Individual differences in emotion processing," Current Opinion in Neurobiology, vol. 14, no. 2, pp. 233-238, 2004/04/01 2004.
[29] S. Manyiwa and Z. Jin, "Gender effects on consumers’ attitudes toward comedic violence in advertisements," Journal of Promotion Management, vol. 26, no. 5, pp. 654-673, 2020/07/28 2020.
[30] J. Peter and P. M. Valkenburg, "Adolescents and Pornography: A Review of 20 Years of Research," The Journal of Sex Research, vol. 53, no. 4-5, pp. 509-531, 2016/05/03 2016.
[31] M. Krcmar and L. G. Kean, "Uses and Gratifications of Media Violence: Personality Correlates of Viewing and Liking Violent Genres," Media Psychology, vol. 7, no. 4, pp. 399-420, 2005/11/01 2005.
[32] N. Claessens and A. Dhoest, "Comedy taste: Highbrow/lowbrow comedy and cultural capital," Participations : Journal of Audience & Perception Studies, vol. 7, no. 1, pp. 49-72, May 2010 2010.
[33] A. Palomba, "Consumer personality and lifestyles at the box office and beyond: How demographics, lifestyles and personalities predict movie consumption," Journal of Retailing and Consumer Services, vol. 55, p. 102083, 2020/07/01 2020.
[34] P. J. Rentfrow, L. R. Goldberg et al., "Listening, watching, and reading: The structure and correlates of entertainment preferences," Journal of personality, vol. 79, no. 2, pp. 223-258, 2011.
[35] I. Cantador, I. Fernández-Tobías et al., "Relating personality types with user preferences in multiple entertainment domains," in CEUR workshop proceedings, 2013: Shlomo Berkovsky.
[36] J. W. Shim and B. Paul, "Effects of personality types on the use of television genre," Journal of Broadcasting & Electronic Media, vol. 51, no. 2, pp. 287-304, 2007.
[37] D. Duuren, "The relationship between personality and preference for either arthouse or mainstream movies," University of Twente, 2008.
[38] L. Breiman, "Random Forests," Machine Learning, vol. 45, no. 1, pp. 5-32, 2001/10/01 2001.
[39] W. R. Bristi, Z. Zaman et al., "Predicting IMDb Rating of Movies by Machine Learning Techniques," in 2019 10th International Conference on Computing, Communication and Networking Technologies (ICCCNT), Kanpur, India, 2019, pp. 1-5.
[40] R. Dhir and A. Raj, "Movie Success Prediction using Machine Learning Algorithms and their Comparison," in 2018 First International Conference on Secure Cyber Computing and Communication (ICSCCC), Jalandhar, India, 2018, pp. 385-390.
[41] L. R. Goldberg, "The development of markers for the Big-Five factor structure," Psychological assessment, vol. 4, no. 1, p. 26, 1992.
[42] J. Deng, J. Guo et al., "Retinaface: Single-stage dense face localisation in the wild," arXiv, May 4, 2019 2019.
[43] S. Miao, H. Xu et al., "Recognizing Facial Expressions Using a Shallow Convolutional Neural Network," IEEE Access, vol. 7, pp. 78000-78011, 2019.
[44] R. Bulirsch, J. Stoer et al., Introduction to numerical analysis. Springer, 1991.
[45] G. W. Allport and H. S. Odbert, "Trait-names: A psycho-lexical study," Psychological monographs, vol. 47, no. 1, 1936.
[46] R. B. Cattell, "Psychological measurement: normative, ipsative, interactive," Psychological review, vol. 51, no. 5, p. 292, 1944.
[47] D. W. Fiske, "Two Cheers for the Big Five!," Psychological Inquiry, vol. 5, no. 2, pp. 123-124, 1994/04/01 1994.
[48] J. M. Cheek, "Aggregation, moderator variables, and the validity of personality tests: A peer-rating study," Journal of Personality Social Psychology
vol. 43, no. 6, p. 1254, 1982.
[49] J. C. Loehlin, R. R. McCrae et al., "Heritabilities of common and measure-specific components of the Big Five personality factors," Journal of research in personality, vol. 32, no. 4, pp. 431-453, 1998.
[50] Urban and Regional Development Statistics Republic of China（Taiwan） 2007, U. a. H. D. Department and C. f. E. P. a. Development, eds., Republic of China（Taiwan）, 2007. [Online]. Available.
[51] (2019). International Personality Item Pool: A Scientific Collaboratory for the Development of Advanced Measures of Personality Traits and Other Individual Differences. Available: http://ipip.ori.org/
[52] H. Alkharusi, "Categorical variables in regression analysis: A comparison of dummy and effect coding," International Journal of Education, vol. 4, no. 2, p. 202, 2012.
[53] P. Branco, L. Torgo et al., "SMOGN: a Pre-processing Approach for Imbalanced Regression," presented at the Proceedings of the First International Workshop on Learning with Imbalanced Domains: Theory and Applications, Proceedings of Machine Learning Research, 2017. Available: https://proceedings.mlr.press/v74/branco17a.html
[54] A. Mollahosseini, B. Hasani et al., "Affectnet: A database for facial expression, valence, and arousal computing in the wild," IEEE Transactions on Affective Computing, vol. 10, no. 1, pp. 18-31, 2017.
[55] S. Arlot and A. Celisse, "A survey of cross-validation procedures for model selection," Statistics surveys, vol. 4, pp. 40-79, 2010.
[56] T. K. Kim, "T test as a parametric statistic," (in eng), Korean journal of anesthesiology, vol. 68, no. 6, pp. 540-546, 2015.
[57] G. Ke, Q. Meng et al., "Lightgbm: A highly efficient gradient boosting decision tree," Advances in neural information processing systems, vol. 30, pp. 3146-3154, 2017.
[58] L. Prokhorenkova, G. Gusev et al., "CatBoost: unbiased boosting with categorical features," arXiv, 2017.
[59] T. Chen and C. Guestrin, "XGBoost: A Scalable Tree Boosting System," presented at the Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, California, USA, 2016. Available: https://doi.org/10.1145/2939672.2939785