日常生活中與產品、系統或服務互動時的使用者體驗（UX）逐漸受到各領域重視，現今多數使用者體驗評估方法多為問卷、量表或訪談等主觀評估方法，雖然能夠與測量目標產生高度相關，然而卻可能會受到認知偏差、心理因素等影響評估結果，而藉由蒐集人體生理訊號做為使用者體驗客觀評估方法就可補足主觀評估方法的缺點。因此，本研究的主要目的在研究2D、3D不同任務環境與不同類型軟體對使用者體驗產生的影響，以及使用人體生理訊號對操作環境、軟體類型、優使性（Usability）等級和情緒類別進行分類辨識。
本研究以操作環境、軟體類型為因子（factors），操作環境有2D和3D兩種水準（levels）；軟體類型有創造資訊型與使用資訊型兩種水準。共蒐集20位受試者分別在四種操作情境下的主觀、客觀使用者體驗評估資料；主觀數據包含SUS量表、SAM量表、Borg-CR10量表、使用績效與訪談；客觀評估方法蒐集心率、膚電阻和腦波三種生理數據，輸入機器學習演算法建立分類模型以評估使用者體驗各面向，並探討不同生理數據組合與不同演算法對辨識正確率的影響。
研究結果顯示，操作環境與軟體類型間有交互作用，在不同操作環境不同類型軟體對主觀優使性、情緒體驗、知覺負荷度和使用績效皆有不同的影響；在辨識操作環境和軟體類型時，模型最佳正確率分別可達81.3%和72.5%，辨識優使性等級的最佳正確率為53.8%、辨識情緒類別的最佳正確率則為70.0%。本研究結果將可以做為使用機器學習分類方法辨識不同任務環境與軟體類型下的客觀使用者體驗評估方法之參考。

The importance of user experience (UX) is gradually valued in many fields. The common user experience evaluation methods are subjective methods such as questionnaires and interviews. Although subjective methods are useful, they might be easily affected by psychological factors. However, collecting human physiological signals as objective evaluation methods without the shortcomings of subjective evaluation methods, which can be another way to evaluate user experience. As a result, the main purpose of this research is to collect subjective and objective data when using different types of software in 2D and 3D environments. In addition, this research uses machine learning algorithms to classify environments, software types, usability levels, and emotion categories.
The study applied the theory of Design of Experiments. Environments and software types served as factors. Environment has 2 levels, they are 2D environment and 3D environment. Software type also has 2 levels, they are creating information and using information. SUS scale, SAM scale, Borg CR-10 scale, task performance and interview were used as the subjective evaluation methods in the study. Furthermore, the wearable sensors recording Heart Rate (HR), Galvanic Skin Response (GSR) and Electroencephalography (EEG) signals were deployed during 4 tasks in different software types and environments by 20 participants. The physiological signals were input into the machine learning algorithm to establish the classification model and discuss the effects of different physiological signals combinations and machine learning algorithms.
The research result shows that the interaction of environments and software types is significant. Depending on the type of software in different environments, the effects on usability, emotional experience, perceived load and task performance are different. When classifying the environments and software types, the best accuracy rate can reach 81.3% and 72.5%, respectively. The best accuracy rate of identifying the usability level is 53.8%, and the best accuracy rate of identifying the emotion category is 70.0%. The results of this study can be used as a reference for the use of machine learning algorithms to classify user experience and usability under different task environments and software types.

Antoniou, P. E., Arfaras, G., Pandria, N., Athanasiou, A., Ntakakis, G., Babatsikos, E., Nigdelis, V., & Bamidis, P. (2020). Biosensor real-time affective analytics in virtual and mixed reality medical education serious games: cohort study. JMIR Serious Games, 8(3), e17823.
Ayaz, H., & Dehais, F. (2021). Neuroergonomics. Handbook of human factors and ergonomics, 816-841.
Bangor, A., Kortum, P., & Miller, J. (2009). Determining what individual SUS scores mean: Adding an adjective rating scale. Journal of usability studies, 4(3), 114-123.
Baraybar-Fernández, A., Baños-González, M., Barquero-Pérez, Ó., Goya-Esteban, R., & De-la-Morena-Gómez, A. (2017). Evaluation of emotional responses to television advertising through neuromarketing. Comunicar. Media Education Research Journal, 25(2).
Borg, G. (2010). The Borg CR Scales® Folder. Borg Perception: Hasselby, Sweeden.
Bradley, M. M., & Lang, P. J. (1994). Measuring emotion: the self-assessment manikin and the semantic differential. Journal of behavior therapy and experimental psychiatry, 25(1), 49-59.
Brady, K., Gwon, Y., Khorrami, P., Godoy, E., Campbell, W., Dagli, C., & Huang, T. S. (2016). Multi-modal audio, video and physiological sensor learning for continuous emotion prediction. Proceedings of the 6th International Workshop on Audio/Visual Emotion Challenge.
Brooke, J. (1996). Sus: a “quick and dirty’usability. Usability evaluation in industry, 189(3).
Brooks, P., & Hestnes, B. (2010). User measures of quality of experience: why being objective and quantitative is important. IEEE network, 24(2), 8-13.
Chen, P., & Zhang, J. (2017). Performance comparison of machine learning algorithms for EEG-signal-based emotion recognition. International Conference on Artificial Neural Networks.
Chen, Y.-P., Yang, J.-Y., Liou, S.-N., Lee, G.-Y., & Wang, J.-S. (2008). Online classifier construction algorithm for human activity detection using a tri-axial accelerometer. Applied Mathematics and Computation, 205(2), 849-860.
Desmet, P., & Hekkert, P. (2007). Framework of product experience. International journal of design, 1(1).
Ekman, P. (1992). An argument for basic emotions. Cognition & emotion, 6(3-4), 169-200.
Ekman, P., & Friesen, W. V. (1986). A new pan-cultural facial expression of emotion. Motivation and emotion, 10(2), 159-168.
Ganglbauer, E., Schrammel, J., Deutsch, S., & Tscheligi, M. (2009). Applying psychophysiological methods for measuring user experience: possibilities, challenges and feasibility. Workshop on user experience evaluation methods in product development.
Garbas, J.-U., Ruf, T., Unfried, M., & Dieckmann, A. (2013). Towards robust real-time valence recognition from facial expressions for market research applications. 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction.
Guo, F. (2012). More Than Usability: The Four Elements of User Experience, Part I. UX matters, 24.
Hart, J. D., Piumsomboon, T., Lee, G. A., & Billinghurst, M. (2018). Sharing and Augmenting Emotion in Collaborative Mixed Reality. ISMAR Adjunct.
Hassenzahl, M., & Tractinsky, N. (2006). User experience-a research agenda. Behaviour & Information Technology, 25(2), 91-97.
Helander, M. G., & Khalid, H. M. (2006). Affective and pleasurable design. Handbook of human factors and ergonomics, 3, 543-572.
Hornbæk, K. (2006). Current practice in measuring usability: Challenges to usability studies and research. International journal of human-computer studies, 64(2), 79-102.
Huang, H., & Lee, C.-F. (2019). Factors affecting usability of 3D model learning in a virtual reality environment. Interactive Learning Environments, 1-14.
Huang, T. S., Chen, L. S., Tao, H., Miyasato, T., & Nakatsu, R. (1998). Bimodal emotion recognition by man and machine. ATR Workshop on Virtual Communication Environments.
Hudlicka, E. (2003). To feel or not to feel: The role of affect in human–computer interaction. International journal of human-computer studies, 59(1-2), 1-32.
Hughes, C. E., Stapleton, C. B., Hughes, D. E., & Smith, E. M. (2005). Mixed reality in education, entertainment, and training. IEEE computer graphics and applications, 25(6), 24-30.
Hussein, R., Lin, J., Madden, K., & Wang, Z. J. (2017). Robust recognition of human activities using smartphone sensor data. 2017 International Conference on the Frontiers and Advances in Data Science (FADS).
IBM. (2020). Artificial Neural Network. IBM Cloud Education. https://www.ibm.com/tw-zh/cloud/learn/neural-networks
ISO. (2018). 9241-11: 2018 Ergonomics of Human-System Interaction—Part 11: Usability: Definitions and Concepts. International Organization for Standardization. https://www.iso.org/obp/ui/#iso:std:iso:9241:-11:ed-2:v1:en:en., 9241(11).
ISO, D. (2010). 9241–210: 2010: ergonomics of human-system interaction—part 210: human-centred design for interactive systems (formerly known as 13407). Switzerland: International Standards Organization.
Izard, C. E. (2007). Basic emotions, natural kinds, emotion schemas, and a new paradigm. Perspectives on psychological science, 2(3), 260-280.
Jankowski, J., Wątróbski, J., Witkowska, K., & Ziemba, P. (2016). Eye tracking based experimental evaluation of the parameters of online content affecting the web user behaviour. In Selected Issues in Experimental Economics (pp. 311-332). Springer.
Jordan, P. W. (2000). Designing pleasurable products: An introduction to the new human factors. CRC press.
Kharoub, H., Lataifeh, M., & Ahmed, N. (2019). 3D user interface design and usability for immersive VR. Applied sciences, 9(22), 4861.
Lang, P. J., Greenwald, M. K., Bradley, M. M., & Hamm, A. O. (1993). Looking at pictures: Affective, facial, visceral, and behavioral reactions. Psychophysiology, 30(3), 261-273.
LaViola Jr, J. J., Kruijff, E., McMahan, R. P., Bowman, D., & Poupyrev, I. P. (2017). 3D user interfaces: theory and practice. Addison-Wesley Professional.
Law, E. L.-C., Roto, V., Hassenzahl, M., Vermeeren, A. P., & Kort, J. (2009). Understanding, scoping and defining user experience: a survey approach. Proceedings of the SIGCHI conference on human factors in computing systems.
Lin, T., & Imamiya, A. (2006). Evaluating usability based on multimodal information: an empirical study. Proceedings of the 8th international conference on Multimodal interfaces.
Makowski, D., Pham, T., Lau, Z. J., Brammer, J. C., Lespinasse, F., Pham, H., Schölzel, C., & Chen, S. (2021). NeuroKit2: A Python toolbox for neurophysiological signal processing. Behavior research methods, 53(4), 1689-1696.
Mandryk, R. L., Inkpen, K. M., & Calvert, T. W. (2006). Using psychophysiological techniques to measure user experience with entertainment technologies. Behaviour & Information Technology, 25(2), 141-158.
McCarthy, J., & Wright, P. (2004). Technology as experience. interactions, 11(5), 42-43.
Mehmood, R. M., & Lee, H. J. (2016). A novel feature extraction method based on late positive potential for emotion recognition in human brain signal patterns. Computers & Electrical Engineering, 53, 444-457.
Mehrabian, A., & Russell, J. A. (1974). An approach to environmental psychology. the MIT Press.
Meijers, A. (2020). Immersive Office 365.
Mifsud, J. (2011). The difference (and relationship) between usability and user experience. Usabilitygeek. com, 11(7).
Morville, P. (2004). User experience design. https://semanticstudios.com/user_experience_design/
Nielsen, J. (1994). Usability engineering. Morgan Kaufmann.
Nielsen, J. (2001). Usability Metrics. https://www.nngroup.com/articles/usability-metrics/

Nielsen, J., & Norman, D. (2014). The definition of user experience. Nielsen Norman Group, 191.
Norman, D., Miller, J., & Henderson, A. (1995). What you see, some of what's in the future, and how we go about doing it Conference companion on Human factors in computing systems - CHI '95.
Panchenko, M. (2021). Measuring the Intangible. Usability Metrics. https://www.eleken.co/blog-posts/usability-metrics
Paz, F., & Pow-Sang, J. A. (2016). A systematic mapping review of usability evaluation methods for software development process. International Journal of Software Engineering and Its Applications, 10(1), 165-178.
Picard, R. W. (1997). Affective computing. MIT press.
Picard, R. W., Vyzas, E., & Healey, J. (2001). Toward machine emotional intelligence: Analysis of affective physiological state. IEEE transactions on pattern analysis and machine intelligence, 23(10), 1175-1191.
Robinson, J., Lanius, C., & Weber, R. (2018). The past, present, and future of UX empirical research. Communication Design Quarterly Review, 5(3), 10-23.
Rooney, B., & Hennessy, E. (2013). Actually in the cinema: a field study comparing real 3D and 2D movie patrons' attention, emotion, and film satisfaction. Media Psychology, 16(4), 441-460.
Roto, V., Law, E.-C., Vermeeren, A. P., & Hoonhout, J. (2011). User experience white paper: Bringing clarity to the concept of user experience.
Russell, J. A. (1980). A circumplex model of affect. Journal of Personality and Social Psychology, 39(6), 1161-1178. https://doi.org/10.1037/h0077714
Sacha, J. (2013). Why should one normalize heart rate variability with respect to average heart rate. Front Physiol, 4, 306. https://doi.org/10.3389/fphys.2013.00306
Schmidt, A. (2010). The User Experience. Library Journal, 135(1), 28-29.
Schmidt, L. A., & Trainor, L. J. (2001). Frontal brain electrical activity (EEG) distinguishes valence and intensity of musical emotions. Cognition & emotion, 15(4), 487-500.
Shackel, B. (2009). Usability–Context, framework, definition, design and evaluation. Interacting with computers, 21(5-6), 339-346.
Tcha-Tokey, K., Loup-Escande, E., Christmann, O., & Richir, S. (2017). Effects of interaction level, framerate, field of view, 3D content feedback, previous experience on subjective user experience and objective usability in immersive virtual environment. International Journal of Virtual Reality, 17(3), 27-51.
Tomkins, S. (1962). Affect imagery consciousness: Volume I: The positive affects. Springer publishing company.
Ural, G., Kaçar, F., & Canan, S. (2019). Wavelet phase coherence estimation of EEG signals for neuromarketing studies. NeuroQuantology, 17(2), 112-120.
Wallace, S., Reid, A., Clinciu, D., & Kang, J. S. (2013). Culture and the importance of usability attributes. Information Technology & People.
Winton, W. M., Putnam, L. E., & Krauss, R. M. (1984). Facial and autonomic manifestations of the dimensional structure of emotion. Journal of Experimental Social Psychology, 20(3), 195-216.
Zaki, T., & Islam, M. N. (2021). Neurological and physiological measures to evaluate the usability and user-experience (UX) of information systems: A systematic literature review. Computer Science Review, 40, 100375.
Zhang, J., Yin, Z., Chen, P., & Nichele, S. (2020). Emotion recognition using multi-modal data and machine learning techniques: A tutorial and review. Information Fusion, 59, 103-126.
Zheng, W.-L., Zhu, J.-Y., Peng, Y., & Lu, B.-L. (2014). EEG-based emotion classification using deep belief networks. 2014 IEEE international conference on multimedia and expo (ICME).