研究生: |
Jyostnamayee Sahoo Jyostnamayee Sahoo |
---|---|
論文名稱: |
應用T5大型語言模型及遷移學習於自然災害之自然語言預測SQL指令 T5 Large Language Models with Transfer Learning in Natural Disasters for Natural Language to SQL Prediction |
指導教授: |
蔡孟涵
Meng-Han Tsai |
口試委員: |
蔡孟涵
Meng-Han Tsai 詹皓詠 Hao-Yung Chan 梁期鈞 Ci-Jyun Liang 林之謙 Jacob Je-Chian Lin |
學位類別: |
碩士 Master |
系所名稱: |
工程學院 - 營建工程系 Department of Civil and Construction Engineering |
論文出版年: | 2024 |
畢業學年度: | 112 |
語文別: | 英文 |
論文頁數: | 84 |
中文關鍵詞: | NL2SQL 、Transformers 、BiLSTM 、災難管理 、T5 |
外文關鍵詞: | NL2SQL, Transformers, BiLSTM, Disaster management, T5 |
相關次數: | 點閱:229 下載:0 |
分享至: |
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報 |
Natural Language to SQL (NL2SQL) 系統讓使用者能夠使用日常語言產生複雜的查詢指令,從而徹底改變了與資料庫互動的方式。本研究提出一個改進的 NL2SQL 框架,彌合了人類語言和結構化 SQL 查詢之間的差距,同時引入資料不可知的功能。此框架允許使用者以自然語言任意發問,提供一種有效的資料庫互動方式,對於人類語言中細緻語意的理解方面表現出色。此框架不需要使用整個資料庫進行訓練與預測,從而提高效率並解決資料隱私與安全問題。本研究以臺灣為案例進行實驗,探討了本研究所提出的 NL2SQL 框架的複雜性,以及其在增強亞太地區災害韌性方面的作用。該系統使非技術人員能夠使用的自然語言查詢資料庫,增強災害韌性。本研究使用 WikiSQL 資料與自訂的自然災害資料 (NDD),比較將本研究所提出的 T5-BiLSTM 模型與其他其他最先進模型的效能表現,結果表明,在 WikiSQL 資料上進行預訓練,並搭配 NDD 資料進行微調時,本研究提出的框架達到了96.21% 的最高測試準確率,優於SQLova, HydraNet, SDSQL, SeaD 和CatSQL 等其他模型。此框架為使用者提供近似聊天機器人和虛擬助理的操作體驗,使他們能夠使用自然語言查詢資料庫內容, 而無需 SQL 專業知識。對此 NL2SQL 框架的綜合評估顯示,使用者能夠透過自然語言更有效地查詢、存取、利用關鍵資訊,具備提高臺灣災害韌性的潛能。
Natural Language to SQL (NL2SQL) systems have revolutionized database interactions by enabling users to formulate complex queries using everyday language. This study presents an improved NL2SQL framework that bridges the gap between human language and structured SQL queries while introducing data-agnostic capabilities. This framework allows users to pose under specified natural language questions, offering an efficient means of database interaction and excelling in nuanced human language understanding. This eliminates the need to transfer the entire database for predictions, enhancing efficiency and addressing privacy concerns. This experimental study explores the intricacies of the proposed NL2SQL framework and its role in enhancing disaster resilience in the Asia-Pacific region, using Taiwan as a case study. The comparison of the proposed T5-BiLSTM model's performance with other state-of-the-art models on the WikiSQL and custom Natural Disaster data (NDD) highlights that when pre-trained on the WikiSQL data and fine-tuned on the NDD dataset, it achieves the highest testing accuracy of 96.21% and outperforms other models such as SQLova, HydraNet, SDSQL, SeaD, and CatSQL. The framework is designed to offer users an experience similar to popular chatbots and virtual assistants, allowing them to query databases using natural language without the need for SQL expertise. The comprehensive evaluation of the proposed NL2SQL framework has exhibited the potential to significantly improve disaster resilience in Taiwan by empowering users to access and utilize critical information more effectively through natural language queries.
[1] G. Katsogiannis-Meimarakis and G. Koutrika, “A survey on deep learning approaches for
text-to-sql,” The VLDB Journal, vol. 32, no. 4, pp. 905–936, 2023.
[2] D. Pal, H. Sharma, and K. Chaudhuri, “Data agnostic roberta-based natural language to sql query
generation,” in 2021 6th International Conference for Convergence in Technology (I2CT), pp. 1–5,
IEEE, 2021.
[3] S. Sun, Y. Gao, Y. Zhang, J. Su, B. Chen, Y. Lin, and S. Sun, “An exploratory study on model
compression for text-to-sql,” in Findings of the Association for Computational Linguistics: ACL
2023, pp. 11647–11654, 2023.
[4] M.-H. Tsai, J. Y. Chen, and S.-C. Kang, “Ask diana: A keyword-based chatbot system for
water-related disaster management,” Water, vol. 11, no. 2, p. 234, 2019.
[5] H.-Y. Chan and M.-H. Tsai, “Question-answering dialogue system for emergency operations,”
International Journal of Disaster Risk Reduction, vol. 41, p. 101313, 2019.
[6] D. Philanthropy, “2024 taiwan earthquake,” 2024.
[7] L. Monde, “Taiwan earthquake: Natural disaster expertise helped limit death toll,” Le Monde, 2024.
[8] K. Everington, “Massive landslide near xiulin: Impacting suhua highway and other roads, affecting
transportation and causing damage,” Taiwan News, 2024.
[9] K. Everington, “Massive landslide near xiulin: Impacting suhua highway and other roads, affecting
transportation and causing damage,” Taiwan News, 2024.
[10] B.-S. Huang, C.-S. Ku, C.-J. Lin, S.-J. Lee, Y.-L. E. Chen, J.-S. Jiang, and W.-F. Sun, “The first 30
min hidden aftershocks of the 2022 september 17, ml 6.4, guanshan, taiwan earthquake and its
seismological implications,” Terrestrial, Atmospheric and Oceanic Sciences, vol. 35, no. 1, p. 1,
2024.
[11] C.-C. Wang, M.-S. Li, C.-S. Chang, P.-Y. Chuang, S.-H. Chen, and K. Tsuboki, “Ensemble-based
sensitivity analysis and predictability of an extreme rainfall event over northern taiwan in the mei-yu
season: The 2 june 2017 case,” Atmospheric Research, vol. 259, p. 105684, 2021.
[12] C.-w. Choy, T. H. K. F. of Insurers, M.-c. Wu, and T.-c. Lee, “Assessment of the damages and direct
economic loss in hong kong due to super typhoon mangkhut in 2018,” Tropical Cyclone Research
and Review, vol. 9, no. 4, pp. 193–205, 2020.
[13] Peace Winds America, “Japan Typhoon Hagibis,” 2019.
[14] D. Susilo, M. Indrasari, J. Iristian, E. Yunus, et al., “Managing uncertainty during disaster: Case on
typhoon hagibis japan,” in IOP Conference Series: Earth and Environmental Science, vol. 519,
p. 012015, IOP Publishing, 2020.
59[15] L. H. Marcelin, T. Cela, and J. M. Shultz, “Haiti and the politics of governance and community
responses to hurricane matthew,” Disaster Health, vol. 3, no. 4, pp. 151–161, 2016.
[16] V. Zhong, C. Xiong, and R. Socher, “Seq2sql: Generating structured queries from natural language
using reinforcement learning,” 2017.
[17] J. Liu, Q. Cui, H. Cao, T. Shi, and M. Zhou, “Auto-conversion from natural language to structured
query language using neural networks embedded with pre-training and fine-tuning mechanism,” in
2020 Chinese Automation Congress (CAC), pp. 6651–6654, IEEE, 2020.
[18] W. Wan, Q. Dou, X. Zhou, P. Jiang, and B. Zhang, “Natural language-to-sql based on relationship
extraction,” in 2019 IEEE 14th International Conference on Intelligent Systems and Knowledge
Engineering (ISKE), pp. 1219–1225, IEEE, 2019.
[19] Y. Fan, Z. He, T. Ren, D. Guo, L. Chen, R. Zhu, G. Chen, Y. Jing, K. Zhang, and X. S. Wang, “Gar:
A generate-and-rank approach for natural language to sql translation,” in 2023 IEEE 39th
International Conference on Data Engineering (ICDE), pp. 110–122, IEEE, 2023.
[20] K. Lahoti, M. Paryani, and A. Patil, “Deep learning based text to sql conversion on wikisql dataset:
Comparative analysis,” in 2022 3rd International Conference on Issues and Challenges in Intelligent
Computing Techniques (ICICT), pp. 1–6, IEEE, 2022.
[21] X. Tang, S. Wu, M. Song, S. Ying, F. Li, and G. Chen, “Preqr: Pre-training representation for sql
understanding,” in Proceedings of the 2022 International Conference on Management of Data,
pp. 204–216, 2022.
[22] H. Fu, C. Liu, B. Wu, F. Li, J. Tan, and J. Sun, “Catsql: Towards real world natural language to sql
applications,” Proceedings of the VLDB Endowment, vol. 16, no. 6, pp. 1534–1547, 2023.
[23] K. Xu, Y. Wang, Y. Wang, Z. Wen, and Y. Dong, “Sead: End-to-end text-to-sql generation with
schema-aware denoising,” 2023.
[24] W. Hwang, J. Yim, S. Park, and M. Seo, “Achieving 90% accuracy in wikisql,” 2019.
[25] Q. Lyu, K. Chakrabarti, S. Hathi, S. Kundu, J. Zhang, and Z. Chen, “Hybrid ranking network for
text-to-sql,” arXiv preprint arXiv:2008.04759, 2020.
[26] B. Hui, X. Shi, R. Geng, B. Li, Y. Li, J. Sun, and X. Zhu, “Improving text-to-sql with schema
dependency learning,” arXiv preprint arXiv:2103.04399, 2021.
[27] A. Mousa and B. Schuller, “Contextual bidirectional long short-term memory recurrent neural
network language models: A generative approach to sentiment analysis,” in Proceedings of the 15th
Conference of the European Chapter of the Association for Computational Linguistics: Volume 1,
Long Papers, pp. 1023–1032, 2017.
[28] C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu,
“Exploring the limits of transfer learning with a unified text-to-text transformer,” The Journal of
Machine Learning Research, vol. 21, no. 1, pp. 5485–5551, 2020.
60[29] L. Dong and M. Lapata, “Language to logical form with neural attention,” arXiv preprint
arXiv:1601.01280, 2016.
[30] K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “Bleu: a method for automatic evaluation of
machine translation,” in Proceedings of the 40th annual meeting of the Association for
Computational Linguistics, pp. 311–318, 2002.
[31] S. Banerjee and A. Lavie, “Meteor: An automatic metric for mt evaluation with improved
correlation with human judgments,” in Proceedings of the acl workshop on intrinsic and extrinsic
evaluation measures for machine translation and/or summarization, pp. 65–72, 2005.
[32] C.-Y. Lin, “Rouge: A package for automatic evaluation of summaries,” in Text summarization
branches out, pp. 74–81, 2004.
[33] J. Guo, Z. Zhan, Y. Gao, Y. Xiao, J.-G. Lou, T. Liu, and D. Zhang, “Towards complex text-to-sql in
cross-domain database with intermediate representation,” arXiv preprint arXiv:1905.08205, 2019.
[34] A. Kumar, P. Nagarkar, P. Nalhe, and S. Vijayakumar, “Deep learning driven natural languages text
to sql query conversion: A survey,” arXiv preprint arXiv:2208.04415, 2022.
[35] L. Xusheng, A. Yeteng, L. Jingxian, Z. Huimin, Z. Yumeng, L. Min, Z. Wei, H. Wei, S. Liangfei, and
L. Huiqin, “Research on bert-based text2sql multi-task learning,” in 2023 IEEE 3rd International
Conference on Power, Electronics and Computer Applications (ICPECA), pp. 864–868, IEEE, 2023.
[36] G. Sun, R. Shen, L. Jin, Y. Wang, S. Xu, J. Chen, and W. Jiang, “Instruction tuning text-to-sql with
large language models in the power grid domain,” in Proceedings of the 2023 4th International
Conference on Control, Robotics and Intelligent System, pp. 59–63, 2023.
[37] X. Ning, Y. Zhao, and J. liu, “Learning seq2seq model with dynamic schema linking for nl2sql,” in
China Conference on Knowledge Graph and Semantic Computing, pp. 148–153, Springer, 2022.
[38] T. Yu, M. Yasunaga, K. Yang, R. Zhang, D. Wang, Z. Li, and D. Radev, “Syntaxsqlnet: Syntax tree
networks for complex and cross-domaintext-to-sql task,” arXiv preprint arXiv:1810.05237, 2018.