簡易檢索 / 詳目顯示

研究生: 林彥碩
YAN-SHUO LIN
論文名稱: 利用專注性代理及概念路徑分辨器整合評分融合之知識圖譜推論
Utilizing Attentional Agent and Conceptual-Path Discriminator with Score Fusion for Knowledge Graph Reasoning
指導教授: 陳怡伶
Yi-Ling Chen
口試委員: 葉彌妍
Mi-Yen Yeh
陳冠宇
Kuan-Yu Chen
學位類別: 碩士
Master
系所名稱: 電資學院 - 資訊工程系
Department of Computer Science and Information Engineering
論文出版年: 2022
畢業學年度: 110
語文別: 英文
論文頁數: 60
中文關鍵詞: 強化式學習知識圖譜推論
外文關鍵詞: Reinforcement Learning, Knowledge Graph Reasoning
相關次數: 點閱:243下載:0
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報


Abstract in Chinese . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii Abstract in English . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v Contents . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . vi List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix List of Tables . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . x 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 2 Related Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.1 Embedding-based Method in Knowledge Graph Completion . . . . . . . 5 2.2 RL-based Method in Knowledge Graph Completion . . . . . . . . . . . . 6 3 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 3.1 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 3.2 Architecture Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 3.3 Adversarial Knowledge Graph Reasoning . . . . . . . . . . . . . . . . . 13 3.3.1 Policy Network . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 3.3.2 Action Attention . . . . . . . . . . . . . . . . . . . . . . . . . . 15 3.3.3 Conceptual-Path Discriminator . . . . . . . . . . . . . . . . . . . 16 3.4 Demonstration Extraction . . . . . . . . . . . . . . . . . . . . . . . . . . 18 3.5 Reward Design and Training Optimization . . . . . . . . . . . . . . . . . 20 3.5.1 Reward Design . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 3.5.2 Training Optimization . . . . . . . . . . . . . . . . . . . . . . . 22 3.6 Prediction Ranking Optimization . . . . . . . . . . . . . . . . . . . . . . 23 4 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 4.1 Experiment Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 4.1.1 Datasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 4.1.2 Baselines and Variations . . . . . . . . . . . . . . . . . . . . . . 27 4.1.3 Hyperparameters . . . . . . . . . . . . . . . . . . . . . . . . . . 28 4.1.4 Evaluation Protocol . . . . . . . . . . . . . . . . . . . . . . . . . 28 4.2 Inference Result . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 4.2.1 Performance of KG inference . . . . . . . . . . . . . . . . . . . 30 4.2.2 The Effect of Reward Factor β . . . . . . . . . . . . . . . . . . . 32 4.3 Ablation Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 4.4 Converge Rate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 4.5 Interpretability Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . 36 4.6 Analysis of PRO Variants . . . . . . . . . . . . . . . . . . . . . . . . . . 39 5 Conclusion . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . 41 References . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 42 Appendix A:The Example of Demonstration Extraction . . . . . . . . . . . . . . . 46 A.1 path sampling stage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 A.2 path ranking stage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

[1] Antoine Bordes, Nicolas Usunier, Alberto GarciaDuran, Jason Weston, and Oksana Yakhnenko. Translating embeddings for modeling multirelational data. In Advances in Neural Information Processing Systems, volume 26, 2013.
[2] Rajarshi Das, Shehzaad Dhuliawala, Manzil Zaheer, Luke Vilnis, Ishan Durugkar, Akshay Krishnamurthy, Alex Smola, and Andrew McCallum. Go for a walk and arrive at the answer: Reasoning over paths in knowledge bases using reinforcement learning. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 May 3, 2018, Conference Track Proceedings, 2018.
[3] Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. Convolutional 2d knowledge graph embeddings. In Proceedings of the ThirtySecond AAAI Conference on Artificial Intelligence, (AAAI18), the 30th innovative Applications of Artificial Intelligence (IAAI18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI18), New Orleans, Louisiana, USA, February 27, 2018 , pages 1811–1818, 2018.
[4] Sepp Hochreiter and Jürgen Schmidhuber. Long shortterm memory. Neural Comput., 9(8):1735–1780, 1997.
[5] Guoliang Ji, Shizhu He, Liheng Xu, Kang Liu, and Jun Zhao. Knowledge graph embedding via dynamic mapping matrix. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 687–696, 2015.
[6] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 79, 2015, Conference Track Proceedings, 2015.
[7] Deren Lei, Gangrong Jiang, Xiaotao Gu, Kexuan Sun, Yuning Mao, and Xiang Ren. Learning collaborative agents with rule guidance for knowledge graph reasoning. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 1620,2020, pages 8541–8547, 2020.
[8] Xi Victoria Lin, Richard Socher, and Caiming Xiong. Multihop knowledge graph reasoning with reward shaping. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 November 4, 2018 , pages 3243–3253, 2018.
[9] Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. Learning entity and relation embeddings for knowledge graph completion. In Proceedings of the TwentyNinth AAAI Conference on Artificial Intelligence, January 2530, 2015, Austin, Texas, USA , pages 2181–2187, 2015.
[10] Xin Lv, Xu Han, Lei Hou, Juanzi Li, Zhiyuan Liu, Wei Zhang, Yichi Zhang, Hao Kong, and Suhui Wu. Dynamic anticipation and completion for multihop reasoning over sparse knowledge graph. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 1620, 2020 , pages 5694–5703, 2020.
[11] Xin Lv, Yixin Cao, Lei Hou, Juanzi Li, Zhiyuan Liu, Yichi Zhang, and Zelin Dai. Is multihop reasoning really explainable? towards benchmarking reasoning interpretability. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 711 November, 2021 , pages 8899–8911, 2021.
[12] Maximilian Nickel, Volker Tresp, and HansPeter Kriegel. A threeway model for collective learning on multirelational data. In Proceedings of the 28th International Conference on Machine Learning,ICML 2011, Bellevue, Washington, USA, June 28 July 2, 2011 , 2011.
[13] Zhiqing Sun, ZhiHong Deng, JianYun Nie, and Jian Tang. Rotate: Knowledge graph embedding by relational rotation in complex space. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 69, 2019 , 2019.
[14] Richard S. Sutton and Andrew G. Barto. Reinforcement learning: An introduction. IEEE Trans. Neural Networks, 9(5):1054–1054, 1998.
[15] Kristina Toutanova, Danqi Chen, Patrick Pantel, Hoifung Poon, Pallavi Choudhury, and Michael Gamon. Representing text for joint embedding of text and knowledge bases. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015, Lisbon, Portugal, September 1721, 2015 , pages 1499–1509, 2015.
[16] Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, and Guillaume Bouchard. Complex embeddings for simple link prediction. In Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York City, NY, USA, June 1924, 2016 , volume 48 of JMLR Workshop and Conference Proceedings, pages 2071–2080, 2016.
[17] Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 May 3, 2018, Conference Track Proceedings , 2018.
[18] Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. Knowledge graph embedding by translating on hyperplanes. In Proceedings of the TwentyEighth AAAI Conference on Artificial Intelligence, July 27 31, 2014, Québec City, Québec, Canada , pages 1112–1119, 2014.
[19] Ronald J. Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach. Learn., 8:229–256, 1992.
[20] Wenhan Xiong, Thien Hoang, and William Yang Wang. Deeppath: A reinforcement learning method for knowledge graph reasoning. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP 2017, Copenhagen, Denmark, September 911, 2017 , pages 564–573, 2017.
[21] Bishan Yang, Wentau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. Embedding entities and relations for learning and inference in knowledge bases. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 79, 2015, Conference Track Proceedings , 2015.
[22] Xunlin Zhan, Yinya Huang, Xiao Dong, Qingxing Cao, and Xiaodan Liang. Pathreasoner: Explainable reasoning paths for commonsense question answering. Knowl. Based Syst., 235:107612, 2022.
[23] Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. Bertscore: Evaluating text generation with BERT. In 8th International Conference on Learning Representations, ICLR 2020,Addis Ababa, Ethiopia, April 2630, 2020 , 2020.
[24] Kangzhi Zhao, Xiting Wang, Yuren Zhang, Li Zhao, Zheng Liu, Chunxiao Xing, and Xing Xie. Leveraging demonstrations for reinforcement recommendation reasoning over knowledge graphs. In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, SIGIR 2020, Virtual Event, China, July 2530, 2020 , pages 239–248, 2020.

無法下載圖示 全文公開日期 2025/09/05 (校內網路)
全文公開日期 本全文未授權公開 (校外網路)
全文公開日期 本全文未授權公開 (國家圖書館:臺灣博碩士論文系統)
QR CODE