隨著類神經網路相關理論與技術的蓬勃發展，使其在許多資訊相關領域都佔據了相當重要的地位，資訊檢索領域也包含其中。但若要訓練出一個擁有良好表現的類神經網路模型，如何獲取大量的經過標注的訓練資料往往是一道難以避開的難題，其往往需要花費大量成本才能取得，並不便於實際環境中的應用。因此許多基於類神經網路的資訊檢索(Neural IR)相關研究將目光投射至使用者在檢索系統中留下來的檢索點擊紀錄，以此種便於取得、常被視為使用者的隱性相關回饋的資料來訓練模型。
目前已有許多Neural IR的相關研究藉由檢索點擊紀錄訓練模型，但他們往往只是將此種資料當作一種容易取得的標注訓練資料的替代品，忽略了許多先前研究從中發現的豐富資訊，同時也沒有注意其中所隱藏的雜訊與資料稀疏等問題。
本研究提出了一種新的基於點擊注意圖的類神經資訊檢索模型CAGNIR (Click-Attention Graph for Neural Information Retrieval)，利用圖神經網路讓查詢詞與文件在點擊圖中從周遭鄰居聚合相關資訊，使自身能藉由過往的點擊關係獲得更豐富、更完整的表徵向量，從而降低查詢詞與文件之間常有的語意落差(semantic gap)，並減緩檢索點擊紀錄中資料稀疏問題的影響；同時CAGNIR在聚合的過程中也運用了多視角的注意力機制，讓各節點能從多種面向來衡量自身與鄰居的相關程度，並動態地評估各面向在該次聚合中的重要性。最後，本研究也透過實驗來衡量CAGNIR的表現，並藉由將CAGNIR與相關模型所評估的相關程度結果視覺化來比較其間的差異，驗證CAGNIR確實能藉由點擊注意圖得出更完善的相關程度評估方法，使曾與查詢詞有點擊關係、或在點擊圖中距離較近的文件能有較好、較合理的排序。

With the vigorous development of neural network related theories and technologies, it has occupied a very important position in many information domains, and the domain of information retrieval is no exception. However, when it comes to training a neural network with good performance, how to get a large amount of labeled training data is often one of the unavoidable problems, obtaining such data usually cost a lot, and sometimes not feasible in practice. Therefore, many researchers of neural network based information retrieval (Neural IR) focus on the click logs left by users in the retrieval systems to train models, which is easy to obtain and often seen as a kind of implicit relevance feedback from users.
There has been much research on Neural IR use click logs to train models, but they often just take this kind of data as a substitute which is much easier to obtain, ignoring the rich information discovered by much previous research, as well as the problems of noise and sparsity hidden in the click logs.
This research proposes a new Neural IR model CAGNIR (Click-Attention Graph for Neural Information Retrieval), which uses graph neural network to make queries and documents aggregate relevant information from their neighbors in the click graph, enable them to obtain richer and more complete representation via the click relationships, thereby reducing the semantic gap that often occurs between queries and documents, and alleviating the sparsity in the click logs. At the same time, CAGNIR also uses a multi-view attention mechanism in the aggregation process, so that each node can measure its relevance to its neighbors from multiple perspectives and dynamically evaluate the importance of each perspective in the aggregation. Finally, this research also measures the performance of CAGNIR through experiments, then visualizes the relevance scores measured by CAGNIR and other related models to analyze the differences between them, verifying that CAGNIR does learn a more complete method to measure relevance with Click-Attention Graph, so that documents which are clicked on for the query or are close with it in the click graph will have a better and more reasonable ranking.

[1] Q. Ai, K. Bi, J. Guo, and W. B. Croft, "Learning a Deep Listwise Context Model for Ranking Refinement," presented at the The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, 2018.
[2] D. M. Blei, A. Y. Ng, and M. I. Jordan, "Latent dirichlet allocation," The Journal of Machine Learning Research, vol. 3, pp. 993-1022, 2003.
[3] J. Bruna, W. Zaremba, A. Szlam, and Y. LeCun, "Spectral Networks and Locally Connected Networks on Graphs," in 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings, 2014.
[4] O. Chapelle and Y. Zhang, "A dynamic bayesian network click model for web search ranking," presented at the Proceedings of the 18th international conference on World wide web, Madrid, Spain, 2009.
[5] W.-L. Chiang, X. Liu, S. Si, Y. Li, S. Bengio, and C.-J. Hsieh, "Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks," in Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2019, Anchorage, AK, USA, August 4-8, 2019., 2019, pp. 257-266.
[6] A. Chuklin, I. Markov, and M. d. Rijke, "Click Models for Web Search," Synthesis Lectures on Information Concepts, Retrieval, and Services, vol. 7, no. 3, pp. 1-115, 2015.
[7] N. Craswell and M. Szummer, "Random walks on the click graph," presented at the Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, Amsterdam, The Netherlands, 2007.
[8] N. Craswell, O. Zoeter, M. J. Taylor, and B. Ramsey, "An experimental comparison of click position-bias models," in Proceedings of the International Conference on Web Search and Web Data Mining, WSDM 2008, Palo Alto, California, USA, February 11-12, 2008, 2008, pp. 87-94: ACM.
[9] M. Defferrard, X. Bresson, and P. Vandergheynst, "Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering," in Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spain, 2016, pp. 3837-3845.
[10] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding," in 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2019, pp. 4171-4186.
[11] G. E. Dupret and B. Piwowarski, "A user browsing model to predict search engine click data from past observations," presented at the Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval, Singapore, Singapore, 2008.
[12] R. Gómez-Bombarelli, J. N. Wei, D. Duvenaud, J. M. Hernández-Lobato, B. Sánchez-Lengeling, D. Sheberla, J. Aguilera-Iparraguirre, T. D. Hirzel, R. P. Adams, and A. Aspuru-Guzik, "Automatic Chemical Design Using a Data-Driven Continuous Representation of Molecules," ACS Central Science, vol. 4, no. 2, pp. 268-276, 2018/02/28 2018.
[13] P. Gage, "A new algorithm for data compression," C Users J., vol. 12, no. 2, pp. 23–38, 1994.
[14] M. Gori, G. Monfardini, and F. Scarselli, "A new model for learning in graph domains," in Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005., 2005, vol. 2, pp. 729-734 vol. 2.
[15] J. Guo, Y. Fan, L. Pang, L. Yang, Q. Ai, H. Zamani, C. Wu, W. B. Croft, and X. Cheng, "A Deep Look into neural ranking models for information retrieval," Information Processing & Management, p. 102067, 2019.
[16] W. L. Hamilton, R. Ying, and J. Leskovec, "Representation Learning on Graphs: Methods and Applications," IEEE Data Eng. Bull., vol. 40, no. 3, pp. 52-74, 2017.
[17] W. L. Hamilton, R. Ying, and J. Leskovec, "Inductive Representation Learning on Large Graphs," in NIPS, 2017.
[18] K. He, X. Zhang, S. Ren, and J. Sun, "Deep Residual Learning for Image Recognition," in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770-778.
[19] P.-S. Huang, X. He, J. Gao, L. Deng, A. Acero, and L. Heck, "Learning deep structured semantic models for web search using clickthrough data," presented at the Proceedings of the 22nd ACM international conference on Conference on information & knowledge management, San Francisco, California, USA, 2013.
[20] S. Jiang, Y. Hu, C. Kang, J. Tim Daly, D. Yin, Y. Chang, and C. Zhai, "Learning Query and Document Relevance from a Web-scale Click Graph," presented at the Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval, Pisa, Italy, 2016.
[21] T. Joachims, L. A. Granka, B. Pan, H. Hembrooke, and G. Gay, "Accurately Interpreting Clickthrough Data as Implicit Feedback," SIGIR Forum, vol. 51, no. 1, pp. 4-11, 2017.
[22] K. Järvelin and J. Kekäläinen, "Cumulated gain-based evaluation of IR techniques," ACM Trans. Inf. Syst., vol. 20, no. 4, pp. 422–446, 2002.
[23] D. P. Kingma and J. Ba, "Adam: A Method for Stochastic Optimization," 2015.
[24] T. N. Kipf and M. Welling, "Variational Graph Auto-Encoders," NIPS Workshop on Bayesian Deep Learning, 2016.
[25] T. N. Kipf and M. Welling, "Semi-Supervised Classification with Graph Convolutional Networks," in 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings, 2017.
[26] A. Krizhevsky, I. Sutskever, and G. E. Hinton, "ImageNet classification with deep convolutional neural networks," Commun. ACM, vol. 60, no. 6, pp. 84-90, / 2017.
[27] J. B. Lee, R. A. Rossi, S. Kim, N. K. Ahmed, and E. Koh, "Attention Models in Graphs: A Survey," ACM Trans. Knowl. Discov. Data, vol. 13, no. 6, November 2019.
[28] Y. Li, D. Tarlow, M. Brockschmidt, and R. S. Zemel, "Gated Graph Sequence Neural Networks," in 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings, 2016.
[29] Y. Li, O. Vinyals, C. Dyer, R. Pascanu, and P. W. Battaglia, "Learning Deep Generative Models of Graphs," CoRR, vol. abs/1803.03324, 2018.
[30] Y. Liu, X. Xie, C. Wang, J.-Y. Nie, M. Zhang, and S. Ma, "Time-Aware Click Model," ACM Trans. Inf. Syst., vol. 35, no. 3, pp. 16:1-16:24, / 2017.
[31] J. Mao, C. Luo, M. Zhang, and S. Ma, "Constructing Click Models for Mobile Search," 2018.
[32] A. Micheli, "Neural Network for Graphs: A Contextual Constructive Approach," IEEE Trans. Neural Networks, vol. 20, no. 3, pp. 498-511, 2009.
[33] T. Mikolov, K. Chen, G. Corrado, and J. Dean, "Efficient Estimation of Word Representations in Vector Space," in ICLR, 2013.
[34] H. Palangi, L. Deng, Y. Shen, J. Gao, X. He, J. Chen, X. Song, and R. Ward, "Deep sentence embedding using long short-term memory networks: analysis and application to information retrieval," IEEE/ACM Trans. Audio, Speech and Lang. Proc., vol. 24, no. 4, pp. 694-707, 2016.
[35] B. Perozzi, R. Al-Rfou, and S. Skiena, "DeepWalk: online learning of social representations," in The 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '14, New York, NY, USA - August 24 - 27, 2014, 2014, pp. 701-710.
[36] M. Qu, J. Tang, J. Shang, X. Ren, M. Zhang, and J. Han, "An Attention-based Collaboration Framework for Multi-View Network Representation Learning," presented at the Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, Singapore, Singapore, 2017.
[37] J. Rocchio, "Relevance feedback in information retrieval," The Smart Retrieval System-Experiments in Automatic Document Processing, pp. 313-323, 1971.
[38] G. Salton, A. Wong, and C. S. Yang, "A vector space model for automatic indexing," Communications of the ACM, vol. 18, no. 11, pp. 613-620, 1975.
[39] F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini, "The graph neural network model," Trans. Neur. Netw, vol. 20, no. 1, pp. 61-80, 2009.
[40] R. Sennrich, B. Haddow, and A. Birch, "Neural Machine Translation of Rare Words with Subword Units," in Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Berlin, Germany, 2016, pp. 1715-1725: Association for Computational Linguistics.
[41] Y. Shen, X. He, J. Gao, L. Deng, and G. Mesnil, "A Latent Semantic Model with Convolutional-Pooling Structure for Information Retrieval," presented at the Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management, Shanghai, China, 2014.
[42] D. I. Shuman, S. K. Narang, P. Frossard, A. Ortega, and P. Vandergheynst, "The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains," IEEE Signal Processing Magazine, vol. 30, no. 3, pp. 83-98, 2013.
[43] M. Simonovsky and N. Komodakis, "GraphVAE: Towards Generation of Small Graphs Using Variational Autoencoders," in Artificial Neural Networks and Machine Learning - ICANN 2018 - 27th International Conference on Artificial Neural Networks, Rhodes, Greece, October 4-7, 2018, Proceedings, Part I, 2018, pp. 412-422.
[44] M. D. Smucker, J. Allan, and B. Carterette, "A comparison of statistical significance tests for information retrieval evaluation," presented at the Proceedings of the sixteenth ACM conference on Conference on information and knowledge management, Lisbon, Portugal, 2007.
[45] J. Tang, M. Qu, M. Wang, M. Zhang, J. Yan, and Q. Mei, "LINE: Large-scale Information Network Embedding," presented at the Proceedings of the 24th International Conference on World Wide Web, Florence, Italy, 2015.
[46] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. u. Kaiser, and I. Polosukhin, "Attention is All you Need," in Advances in Neural Information Processing Systems 30, 2017, pp. 5998--6008.
[47] P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Liò, and Y. Bengio, "Graph Attention Networks," in International Conference on Learning Representations, 2018.
[48] C. Wang, Y. Liu, M. Wang, K. Zhou, J.-y. Nie, and S. Ma, "Incorporating Non-Sequential Behavior into Click Models," in Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, New York, NY, USA, 2015, pp. 283–292: Association for Computing Machinery.
[49] F. Wu, X. Lu, J. Song, S. Yan, Z. Zhang, Y. Rui, and Y. Zhuang, "Learning of Multimodal Representations With Random Walks on the Click Graph," IEEE Trans. Image Process., vol. 25, no. 2, pp. 630-642, 2016.
[50] Z. Wu, S. Pan, F. Chen, G. Long, C. Zhang, and P. S. Yu, "A Comprehensive Survey on Graph Neural Networks," IEEE Transactions on Neural Networks and Learning Systems, pp. 1-21, 2020.
[51] C. Xiong, J. Callan, and T.-Y. Liu, "Word-Entity Duet Representations for Document Ranking," presented at the Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, Shinjuku, Tokyo, Japan, 2017.
[52] C. Xiong, Z. Dai, J. Callan, Z. Liu, and R. Power, "End-to-End Neural Ad-hoc Ranking with Kernel Pooling," in Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, Shinjuku, Tokyo, Japan, August 7-11, 2017, 2017, pp. 55-64: ACM.
[53] W. Xu, E. Manavoglu, and E. Cantu-Paz, "Temporal Click Model for Sponsored Search," in Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval, New York, NY, USA, 2010, pp. 106–113: Association for Computing Machinery.
[54] D. Yin, Y. Hu, J. Tang, T. Daly, M. Zhou, H. Ouyang, J. Chen, C. Kang, H. Deng, C. Nobata, J.-M. Langlois, and Y. Chang, "Ranking Relevance in Yahoo Search," presented at the Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, California, USA, 2016.
[55] Y. Zhang, D. Wang, and Y. Zhang, "Neural IR Meets Graph Embedding: A Ranking Model for Product Search," in The World Wide Web Conference, WWW 2019, San Francisco, CA, USA, May 13-17, 2019, 2019, pp. 2390-2400: ACM.
[56] Y. Zheng, Z. Fan, Y. Liu, C. Luo, M. Zhang, and S. Ma, "Sogou-QCL: A New Dataset with Click Relevance Label," in The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, New York, NY, USA, 2018, pp. 1117-1120: ACM.
[57] S. Zhou, J. Bu, X. Wang, J. Chen, B. Hu, D. Chen, and C. Wang, "HAHE: Hierarchical Attentive Heterogeneous Information Network Embedding," CoRR, vol. abs/1902.01475, 2019