過去，進行情感分析的最常見方法是預測整個句子的情感。但是，我們從其他人那裡獲得的建議和反饋在現實世界中很可能是多方面的。這意味著一個句子中可能有多個方面的事物。某些方面項為正面的，某些方面項為負面的。普通的情感分析模型不能解決用一個句子裡每個方面像的情感問題。因此，我們建立了一個多重面項情感分類模型來解決這個問題。
目標事物可以出現在整個句子的任何位置。多重面向的情感與上下文語義有關。因此，第一步，我們將建立一個LSTM模型，以使用SemEval 2014筆電和餐廳數據集檢測方面方面上下語義的語義。我們使用LSTM模型提取特徵，然後再使用這些特徵進行情感分類。
自2018年底推出變壓器雙向編碼器表示法（BERT）以來，它的功能是如此強大，以至於打破了許多NLP任務記錄，許多研究人員使用它來增強他們的模型。因此，為了使我們的模型有很大的改進，我們還使用了具有相同數據集的BERT模型，並連接了從LSTM模型中提取的特徵以進行方面項情感分類。
在本文中，我們建立了一個LSTM-BERT模型來預測方面術語的情緒。 LSTM可以理解上下文對方面術語的影響，而BERT是功能強大的預訓練模型，有助於獲得更好的結果並縮短訓練時間。 LSTM還可以彌補缺乏相對位置信息的不足。與其他模型相比，我們具有出色的性能。

In the past, the most common way to do sentiment analysis is to predict the sentiment of the whole sentence. However, the advice and feedback that we received from other people are most likely multi-aspect in the real world. It means that there may be more than one aspect term in a sentence. Some aspect terms are positive and some aspect terms are negative. The ordinary sentiment analysis model cannot deal with the problem of finding out the sentiment of each aspect term in one sentence. Thus, we build an aspect term sentiment classification model to solve this problem.
The target aspect term may appear in any place of the whole sentence. The sentiments of aspect terms are related to the sematic of context. Therefore, we build an LSTM model to detect the semantic of context above and below aspect terms with the SemEval 2014 laptop and restaurant dataset in the first step. We use the LSTM model to extract features and later would use the features to do sentiment classification.
Since the Bidirectional Encoder Representations from Transformers (BERT) was launched at the end of 2018, it is so powerful that it breaks many NLP task records, and many researchers use it to enhance their models. Accordingly, to make our model have great improvement, we also use the BERT model with the same dataset and concatenate the features which we extract from the LSTM model to do aspect term sentiment classification.
In this paper, we build an LSTM-BERT model to predict aspect terms sentiment. LSTM can understand the influence of context on aspect term and BERT is a powerful pre-trained model, which help to gain better result and shorten training time. LSTM can also cover the shortage of weak information about the relative position. Comparing with other models, we have a great performance.

[1] Park, H. J., Song, M., & Shin, K. S. (2020). Deep learning models and datasets for aspect term sentiment classification: Implementing holistic recurrent attention on target-dependent memories. Knowledge-Based Systems, 187, 104825.
[2] Yu, J., Zha, Z. J., Wang, M., & Chua, T. S. (2011, June). Aspect ranking: identifying important product aspects from online consumer reviews. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1 (pp. 1496-1505). Association for Computational Linguistics.
[3] Mubarok, M. S., Adiwijaya, & Aldhi, M. D. (2017, August). Aspect-based sentiment analysis to review products using Naïve Bayes. In AIP Conference Proceedings (Vol. 1867, No. 1, p. 020060). AIP Publishing LLC.
[4] Tang, D., Qin, B., Feng, X., & Liu, T. (2015). Effective LSTMs for target-dependent sentiment classification. arXiv preprint arXiv:1512.01100.
[5] Xue, W., & Li, T. (2018). Aspect based sentiment analysis with gated convolutional networks. arXiv preprint arXiv:1805.07043.
[6] Ma, D., Li, S., Zhang, X., & Wang, H. (2017). Interactive attention networks for aspect-level sentiment classification. arXiv preprint arXiv:1709.00893.
[7] Vargas, M. R., dos Anjos, C. E., Bichara, G. L., & Evsukoff, A. G. (2018, July). Deep leaming for stock market prediction using technical indicators and financial news articles. In 2018 International Joint Conference on Neural Networks (IJCNN) (pp. 1-8). IEEE.
[8] Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., & Dean, J. (2013). Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems (pp. 3111-3119).
[9] Pennington, J., Socher, R., & Manning, C. D. (2014, October). Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP) (pp. 1532-1543).
[10] fastText, fastText Library for efficient text classification and representation learning. Retrieved from https://fasttext.cc/docs/en/english-vectors.html
[11] Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1985). Learning internal representations by error propagation (No. ICS-8506). California Univ San Diego La Jolla Inst for Cognitive Science.
[12] Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural computation, 9(8), 1735-1780.
[13] Cho, K., Van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014). Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078.
[14] Bahdanau, D., Cho, K., & Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473.
[15] Sutskever, I., Vinyals, O., & Le, Q. V. (2014). Sequence to sequence learning with neural networks. In Advances in neural information processing systems (pp. 3104-3112).
[16] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). Attention is all you need. In Advances in neural information processing systems (pp. 5998-6008).
[17] Wang, Y., Huang, M., Zhu, X., & Zhao, L. (2016, November). Attention-based LSTM for aspect-level sentiment classification. In Proceedings of the 2016 conference on empirical methods in natural language processing (pp. 606-615).
[18] Chen, P., Sun, Z., Bing, L., & Yang, W. (2017, September). Recurrent attention network on memory for aspect sentiment analysis. In Proceedings of the 2017 conference on empirical methods in natural language processing (pp. 452-461).
[19] Liu, Q., Zhang, H., Zeng, Y., Huang, Z., & Wu, Z. (2018, April). Content attention model for aspect based sentiment analysis. In Proceedings of the 2018 World Wide Web Conference (pp. 1023-1032).
[20] Li, L., Liu, Y., & Zhou, A. (2018, October). Hierarchical attention based position-aware network for aspect-level sentiment analysis. In Proceedings of the 22nd Conference on Computational Natural Language Learning (pp. 181-189).
[21] Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., & Le, Q. V. (2019). XLNet: Generalized Autoregressive Pretraining for Language Understanding. arXiv preprint arXiv:1906.08237.
[22] Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., ... & Stoyanov, V. (2019). Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
[23] Clark, K., Luong, M. T., Le, Q. V., & Manning, C. D. (2020). Electra: Pre-training text encoders as discriminators rather than generators. arXiv preprint arXiv:2003.10555.
[24] Shaw, P., Uszkoreit, J., & Vaswani, A. (2018). Self-attention with relative position representations. arXiv preprint arXiv:1803.02155.