Abstract - Emotion has always been an element of human life, expressed by action or conversation. It can influence how individuals regard one another, and events can influence it in our environment. As a result, it is critical to understand how our surroundings feel emotionally so that we can act appropriately in any situation. people can predict each other emotions by listening to their intonation and the words themselves. However, with text is rather hard to predict emotion since it only has text as an attribute. Therefore, a lot of research has been done to make this task more accurate. This task has been challenging because of its imbalanced class on datasets and contextual understanding between utterance. Existing method rely only rely on hierarchy-based model. In this research, we propose a method of detecting emotion by using a hierarchical transformer with a contextual enhancer. Dataset extreme imbalance class also created a problem to create a good model. Therefore, data augmentation AEDA is also implemented to help and fix the class imbalance. The improvement of the proposed will be shown on EmotionPush and Emory NLP dataset.

Abstract - Emotion has always been an element of human life, expressed by action or conversation. It can influence how individuals regard one another, and events can influence it in our environment. As a result, it is critical to understand how our surroundings feel emotionally so that we can act appropriately in any situation. people can predict each other emotions by listening to their intonation and the words themselves. However, with text is rather hard to predict emotion since it only has text as an attribute. Therefore, a lot of research has been done to make this task more accurate. This task has been challenging because of its imbalanced class on datasets and contextual understanding between utterance. Existing method rely only rely on hierarchy-based model. In this research, we propose a method of detecting emotion by using a hierarchical transformer with a contextual enhancer. Dataset extreme imbalance class also created a problem to create a good model. Therefore, data augmentation AEDA is also implemented to help and fix the class imbalance. The improvement of the proposed will be shown on EmotionPush and Emory NLP dataset.

[1] A. Seyeditabari, N. Tabari, S. Gholizadeh, and W. Zadrozny, "Emotion Detection in Text: Focusing on Latent Representation," Jul. 2019, [Online]. Available: http://arxiv.org/abs/1907.09369
[2] P. Ekman, "An argument for basic emotions," Cognition and Emotion, vol. 6, no. 3–4, pp. 169–200, May 1992, doi: 10.1080/02699939208411068.
[3] W. Gerrod Parrott, Emotions in Social Psychology: Essential Readings, Illustrated. 2001.
[4] L. Feldman Barrett and J. A. Russell, "Independence and bipolarity in the structure of current affect.," Journal of Personality and Social Psychology, vol. 74, no. 4, pp. 967–984, 1998, doi: 10.1037/0022-3514.74.4.967.
[5] R. PLUTCHIK, "Chapter 1 - A GENERAL PSYCHOEVOLUTIONARY THEORY OF EMOTION," in Theories of Emotion, R. Plutchik and H. Kellerman, Eds. Academic Press, 1980, pp. 3–33. doi: https://doi.org/10.1016/B978-0-12-558701-3.50007-7.
[6] A. Vaswani et al., "Attention Is All You Need," Jun. 2017, [Online]. Available: http://arxiv.org/abs/1706.03762
[7] J. Devlin, M.-W. Chang, K. Lee, K. T. Google, and A. I. Language, "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding." [Online]. Available: https://github.com/tensorflow/tensor2tensor
[8] Y. Liu et al., "RoBERTa: A Robustly Optimized BERT Pretraining Approach," Jul. 2019, [Online]. Available: http://arxiv.org/abs/1907.11692
[9] Q. Li, C. Wu, Z. Wang, and K. Zheng, "Hierarchical transformer network for utterance-level emotion recognition," Applied Sciences (Switzerland), vol. 10, no. 13, Jul. 2020, doi: 10.3390/app10134447.
[10] L. Luo, H. Seng, H. Yang, and F. Y. L. Chin, "EmotionX-DLC: Self-Attentive BiLSTM for Detecting Sequential Emotions in Dialogues," 2018.
[11] W. Jiao, H. Yang, I. King, and M. R. Lyu, "HiGRU: Hierarchical Gated Recurrent Units for Utterance-level Emotion Recognition," Apr. 2019, [Online]. Available: http://arxiv.org/abs/1904.04446
[12] Y.-H. Huang, S.-R. Lee, M.-Y. Ma, Y.-H. Chen, Y.-W. Yu, and Y.-S. Chen, "EmotionX-IDEA: Emotion BERT -- an Affectional Model for Conversation," Aug. 2019, [Online]. Available: http://arxiv.org/abs/1908.06264
[13] J. Lee and W. Lee, "CoMPM: Context Modeling with Speaker's Pre-trained Memory Tracking for Emotion Recognition in Conversation," Aug. 2021, [Online]. Available: http://arxiv.org/abs/2108.11626
[14] J. Li, M. Zhang, D. Ji, and Y. Liu, "Multi-Task Learning with Auxiliary Speaker Identification for Conversational Emotion Recognition," Mar. 2020, [Online]. Available: http://arxiv.org/abs/2003.01478
[15] O. Simeone, "A Very Brief Introduction to Machine Learning With Applications to Communication Systems," Aug. 2018, [Online]. Available: http://arxiv.org/abs/1808.02342
[16] K. (Kevin N. ) Gurney, An introduction to neural networks. UCL Press, 1997.
[17] C. Janiesch, P. Zschech, and K. Heinrich, “Machine learning and deep learning,” Electronic Markets, vol. 31, no. 3, pp. 685–695, Sep. 2021, doi: 10.1007/s12525-021-00475-2.
[18] C. Nwankpa, W. Ijomah, A. Gachagan, and S. Marshall, "Activation Functions: Comparison of trends in Practice and Research for Deep Learning," Nov. 2018, [Online]. Available: http://arxiv.org/abs/1811.03378
[19] P. Yu, V. Y. Cui, and J. Guan, "Text Classification by using Natural Language Processing," in IOP Conference Series: Earth and Environmental Science, Mar. 2021, vol. 1802, no. 4. doi: 10.1088/1742-6596/1802/4/042010.
[20] S. Aman and S. Szpakowicz, "Identifying expressions of emotion in text," in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2007, vol. 4629 LNAI, pp. 196–205. doi: 10.1007/978-3-540-74628-7_27.
[21] M. Banko and E. Brill, "Mitigating the Paucity-of-Data Problem: Exploring the Effect of Training Corpus Size on Classifier Performance for Natural Language Processing."
[22] B. Li, Y. Hou, and W. Che, "Data augmentation approaches in natural language processing: A survey," AI Open, Mar. 2022, doi: 10.1016/j.aiopen.2022.03.001.
[23] J. Wei and K. Zou, "EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks," Jan. 2019, [Online]. Available: http://arxiv.org/abs/1901.11196
[24] A. Karimi, L. Rossi, and A. Prati, "AEDA: An Easier Data Augmentation Technique for Text Classification," Aug. 2021, [Online]. Available: http://arxiv.org/abs/2108.13230
[25] D. R. Beddiar, M. S. Jahan, and M. Oussalah, "Data expansion using back translation and paraphrasing for hate speech detection," Online Social Networks and Media, vol. 24, Jul. 2021, doi: 10.1016/j.osnem.2021.100153.
[26] A. Sherstinsky, "Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Network," Aug. 2018, doi: 10.1016/j.physd.2019.132306.
[27] S. Hochreiter and J. Schmidhuber, “Long Short-Term Memory,” Neural Computation, vol. 9, no. 8, pp. 1735–1780, Nov. 1997, doi: 10.1162/neco.1997.9.8.1735.
[28] A. Adhikari, A. Ram, R. Tang, and J. Lin, "DocBERT: BERT for Document Classification," Apr. 2019, [Online]. Available: http://arxiv.org/abs/1904.08398
[29] J. S. Lee and J. Hsiang, "PatentBERT: Patent Classification with Fine-Tuning a pre-trained BERT Model," 2019, Accessed: May 21, 2022. [Online]. Available: https://arxiv.org/abs/1906.02124
[30] B. Lee and Y. S. Choi, "Graph Based Network with Contextualized Representations of Turns in Dialogue," Sep. 2021, [Online]. Available: http://arxiv.org/abs/2109.04008