近年來，引入時序概念的序列推薦成為推薦系統中的重要研究方向。在序列推薦模型中，引入自關注機制能改善模型容易遺失序列過久以前資訊的問題，且能對序列的項目根據其相關度給予不同權重，以提升預測準確率；引入嵌入模型能學習使用者偏好與項目的表示，若使用高斯嵌入則能進一步能學習到嵌入表示的變異性；附屬資訊融合則是利用項目附屬資訊輔助來提升模型預測能力。這些改進方法都能有效提升模型的預測效能，然而因其各自有複雜網路結構，目前仍沒有方法能結合這些模型。本論文的研究嘗試結合自關注機制、高斯嵌入模型、與多重附屬訊息融合機制，提出了多訊息自關注高斯嵌入融合模型。
我們在電影推薦資料集MovieLens 1M上對本論文提出的推薦模型進行實驗。首先，我們使用單一附屬資訊「導演」，探討將其融合至高斯嵌入模型的融合架構，比較了「只對平均向量融合」、「只對共變異矩陣融合」以及「同時對兩者融合」等三種架構。實驗結果顯示，同時對平均向量和共變異矩陣都進行融合的架構能獲得最好的效能。接著，我們探討項目與附屬資訊融合的加權方式，比較了「相加」、「串接」、「維度加權」以及「附屬資訊加權」等方法。實驗結果顯示，本論文提出的「附屬資訊加權」的方式有最好的預測效能。基於此融合架構與融合加權方式，考慮所有項目的前5名精確率可達0.0649，MAP可達0.1015，優於未融合附屬資訊的嵌入模型。進一步，我們同時使用「導演」與「風格」兩種附屬資訊進行多訊息融合的實驗；結果發現，多訊息融合比起單一附屬資訊融合，其效能可再往上提升，前5名精確率可達0.0658，MAP可達0.1026。另外，我們將融合模型在資料較充足的遊戲推薦資料集Steam上進行實驗，想了解附屬資訊對此資料集的預測效果。我們發現，在資料充足的情況下，項目資訊就足以訓練出良好的預測模型，若融合鑑別力較低的遊戲類型資訊(23,902個遊戲、13種類型)，效能並沒有提升。為了進一步探究其原因，我們將訓練集依照比例切割為不同大小，進行實驗；實驗結果顯示，遊戲類型資訊在訓練資料量不足時，才會有輔助改進的效果。

Sequential recommendation has recently become an important area of research in recommender systems. State-of-the-art sequential recommendation models were improved with several ways. Self-attention, for example, can make use of long-distance correlation to improve the prediction by weighting the items in the sequence according to their relevance. Embedding model, on the other hand, can learn to represent user preferences and items, and even model their variabilities if Gaussian embedding is further applied. Additionally, fusion of side information for the items can improve the performance since the predictive capability of the model could get benefits from the extra information. Though these models are effective for improve the predictive performance of recommendation systems, there is however no one combining these approaches, perhaps due to their respective complicated network architectures. In this research, a new model integrating self-attention, Gaussian embedding model, and fusion mechanism of multiple messages of side information was proposed and called self-attended Gaussian embedding fusion model.
A series of experiments were conducted for the proposed model on the recommendation datasets of MovieLens 1M. First, we investigate the fusion of single side information of director. The fusion architecture of side information for Gaussian embedding model was investigated. Experimental results show that fusion on both 'mean’ and 'covariance' for Gaussian embedding is superior to fusion on either 'mean' or 'covariance' alone. Additionally, the fusion weighting method for item and side information was investigated. We compared four methods: 'add', 'concat', 'dimension weighting', and 'side information weighting'. Experimental results show that 'side information weighting' is superior to other fusion weighting methods. Based on the fusion architecture and fusion weighting method, experimental results show that top-five precision and mean average precision (MAP) may reach 0.0649 and 0.1015, respectively, which outperform the models without using side information. When multiple messages of side information, director and genre, were fused, top-five precision and MAP can rise up to 0.0658 and 0.1026, respectively.
Moreover, the proposed fusion model was tested on another dataset, Steam, which is a game recommendation dataset with sufficient data relatively. From the experiments it was found that incorporation of less discriminative side information (13 genres for 23,902 games) did not help to improve performance. To explore the reason, we further conducted experiments for the training set in different scales, and experimental results show that the less discriminative side information like genre is effective for improving the prediction performance only when the training data is insufficient .

[1] W.-C. Kang, J. McAuley, “Self-Attentive Sequential Recommendation,” ICDM2018, 20 8 2018.
[2] Z. Fan, Z. Liu, L. Zheng, S. Wang, Yu, “Modeling Sequences as Distributions with Uncertainty for Sequential Recommendation,” CIKM 2021, 2021.
[3] Y. Xie, P. Zhou, S. Kim, “Decoupled Side Information Fusion for Sequential Recommendation,” SIGIR'22, 23 4 2022.
[4] T. Hofmann, “Latent Semantic Models for collaborative filtering,” ACM Transactions on Information SystemsVol. 22, No. 1, pp. 89-115, 1 2004.
[5] X. Ning, G. Karypis, “SLIM: Sparse Linear Methods for Top-N Recommender Systems,” 2011 IEEE 11th International Conference on Data Mining, 12 2011.
[6] H. Fang, D. Zhang, Y. Shu, G. Guo, “Deep Learning for Sequential Recommendation: Algorithms” arXiv:1905.01997, 30 4 2019.
[7] G. Linden, B. Smith., J.York, “Amazon.com Recommendations Item-to-Item Collaborative Filtering,” IEEE Internet Computing, pp. 76-80, 22 1 2003.
[8] S. Rendle, C. Freudenthaler, L. Schmidt-Thieme, “Factorizing Personalized Markov Chains for Next-Basket Recommendation,” WWW 2010, pp. 811-820, 26 4 2010.
[9] R. He, J. McAuley, “Fusing Similarity Models with Markov Chains for Sparse Sequential Recommendation,” Information Retrieval (cs.IR) arXiv:1609.09152, 28 9 2016.
[10] B. Hidasi, A. Karatzoglou, L. Baltrunas, D. Tikk, “Session-based Recommendations with Recurrent Neural Networks,” Machine Learning (cs.LG) arXiv:1511.06939, 17 2 2016.
[11] J. Tang, K. Wang, “Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding,” Information Retrieval (cs.IR); Machine Learning (cs.LG) arXiv:1809.07426, 19 9 2018.
[12] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, I. Polosukhin, “Attention Is All You Need,” NIPS 2017, 2017.
[13] K.-h. Chen, “資訊檢索的績效評估 Performance Evaluations for Information Retrieval,” 於 2004年現代資訊組織與檢索研討會, 2004.
[14] S. Rendle, C. Freudenthaler, Z. Gantner, L. Schmidt-Thieme, “BPR: Bayesian Personalized Ranking from Implicit Feedback,” UAI 2009, 2009.
[15] C. C. Aggarwal, Recommender Systems: The Textbook, 2016.
[16] J. Duchi, “Derivations for Linear Algebra and Optimization”.
[17] R. Kashyap, “The Perfect Marriage and Much More: Combining Dimension Reduction, Distance Measures and Covariance,” 2019.