隨著網際網路日益普及，網路攻擊也隨之不斷而來，駭客攻擊如雨後春筍般的出現，是現在資訊社會面臨相對重要的議題。企業經常面臨進階持續性威脅 (Advanced Persistent Threat, APT)攻擊，於企業內進行後滲透橫向移動與提權，造成企業財產損失、資料外洩的風險。本研究透過建構於Q-Learning強化學習(Reinforcement Learning, RL)之上的紅藍隊網路攻防演練訓練紅藍隊人員資安專業技術與防禦能力。

本研究設計一紅藍隊網路攻防機制，並藉由該攻防機制針對資安人員進行演練，熟悉駭客常見的攻擊手法，針對攻擊進行調查與防禦，再透過演練評鑑系統ATT&CK矩陣來評鑑參與者資安技術能量，最後透過Q-Learning強化學習來模擬紅隊人員進行攻擊，並再每回合生成一最佳攻擊路線回饋給紅隊與藍隊人員進行參考。

As the Internet becomes increasingly widespread, what follows next is the endless cyber-attacks. The rapid emergence of hacker attacks has become a relatively important issue facing the information society today. Enterprises are often faced with advanced persistent threats, which gains privilege escalation through post-penetration and lateral movement within the enterprises' network, causing risks of property loss and data breach. This research goal to improve the professional cybersecurity techniques and defense abilities of both red team and blue team professionals, through the with a cyber offensive and defensive exercise constructed on the basis of Q-Learning, a model-free reinforcement learning algorithm.

This research designed a cyber offensive and defensive exercise mechanism, which was provided for professionals to practice their cybersecurity techniques and learn about hackers’ common attack techniques. Firstly, investigation and defense will be implemented on the attacks. Then the ATT&CK matrix system will be applied to evaluate the participant’s cybersecurity techniques and capabilities. Finally, the mechanism will simulate the red team to attack through the Q-Learning algorithm and provide a generated optimal attack chain at each round back for both teams as their reference.

[1] 行政院，"國家資通安全情勢報告"，pp. 9-10，https://nicst.ey.gov.tw/File/63370CCEA3C7A667?A=C，2019 (accessed 06/13, 2020).
[2] 行政院國土安全辦公室，"國家關鍵基礎設施防護--演習參考手冊" ，https://ohs.ey.gov.tw/File/1B29A608E64CD4F5，2019 (accessed 06/13, 2020).
[3] 行政院國家資通安全會報技術服務中心，"資安威脅趨勢與案例分享" ，https://download.nccst.nat.gov.tw/attachfilehandout/%E8%AD%B0%E9%A1%8C%E4%B8%80%EF%BC%9A%E8%B3%87%E5%AE%89%E5%A8%81%E8%84%85%E8%B6%A8%E5%8B%A2%E8%88%87%E6%A1%88%E4%BE%8B3.pdf，2019 (accessed 6/13, 2020).
[4] 李凱江，"基於Q-learning機制的網路安全動態防禦研究"，河南中原工學院，2018.
[5] 周詩洋與傅鸝，"CVSS 環境指標變數對系統安全的影響研究"，電腦工程與科學， vol. 38, no. 12, pp. 2463-2470，2016.
[6] 周誠等人，"一種網路安全脆弱性評估方法"，江蘇大學學報 (自然科學版), ，no. 2017 年 01, pp. 68-77, 85，2017.
[7] 徐偉華, "基於 CVSS 的漏洞風險評估方法研究," 中國民航大學, 2017.
[8] 國家高速網路與計算中心. "TWCC智慧資安升級 保障有價與無價資產." https://www.nchc.org.tw/Message/Print/3324?mid=42,2019 (accessed 6/13, 2020).
[9] 張必彥 and 王孟, "基於 CVSS 漏洞評分標準的網路攻防量化方法研究," 兵器裝備工程學報, vol. 39, no. 4, pp. 147-150, 2018.
[10] (ISC)², "Global Cybersecurity Workforce Shortage to Reach 1.8 Million as Threats Loom Larger and Stakes Rise Higher." https://www.isc2.org/News-and-Events/Press-Room/Posts/2017/06/07/2017-06-07-Workforce-Shortage,2017 (accessed 06/13, 2020).
[11] A. Ashok et al., "A multi-level fidelity microgrid testbed model for cybersecurity experimentation," in 12th {USENIX} Workshop on Cyber Security Experimentation and Test ({CSET} 19), 2019.
[12] B. P. Adrià Puigdomènech et al., "Agent57: Outperforming the human Atari benchmark." https://deepmind.com/blog/article/Agent57-Outperforming-the-human-Atari-benchmark (accessed 06/13, 2020).
[13] C. Chen and S. Shieh, "CTF：Alternative Training for Offensive Security", Rs.ieee.org, 2015.https://rs.ieee.org/images/files/techact/Reliability/2015-08/2015-08-a05.pdf. (accessed 06/13, 2020).
[14] C. Taylor et al., "{CTF}: State-of-the-Art and Building the Next Generation," in 2017 {USENIX} Workshop on Advances in Security Education ({ASE} 17), 2017.
[15] D. Jacobson et al., "Design and implementation of a cyber physical testbed for security training," in 12th {USENIX} Workshop on Cyber Security Experimentation and Test ({CSET} 19), 2019.
[16] E. Bursztein et al., "Webseclab Security Education Workbench," in CSET, 2010.
[17] E. Trickel et al., "Shell we play a game? ctf-as-a-service for security education," in 2017 {USENIX} Workshop on Advances in Security Education ({ASE} 17), 2017.
[18] FIRST. "CVSS v3.1 Specification Document." https://www.first.org/cvss/specification-document. (accessed 06/13, 2020).
[19] G. Louthan et al., "The Blunderdome: An Offensive Exercise for Building Network, Systems, and Web Security Awareness," in CSET, 2010.
[20] J. C. Acosta et al., "A platform for evaluator-centric cybersecurity training and data acquisition," in MILCOM 2017-2017 IEEE Military Communications Conference (MILCOM), IEEE, pp. 394-399, 2017.
[21] M. Keramati, A. Akbari and M. Keramati, "CVSS-based security metrics for quantitative analysis of attack graphs," ICCKE 2013, Mashhad, pp. 178-183, doi: 10.1109/ICCKE.2013.6682816,2013.
[22] MITRE, "Corporate Overview." https://www.mitre.org/about/corporate-overview. (accessed 06/13, 2020).
[23] MITRE, "Enterprise Matrix", https://attack.mitre.org/matrices/enterprise/. 2019 (accessed 06/13, 2020).
[24] MITRE, "MITRE ATT&CK® EVALUATIONS", https://attackevals.mitre.org/. (accessed 06/13, 2020).
[25] N. Crabtree and J. Orr, "Cyber Red/Blue and Gamified Military Cyberspace Operations", Ll.mit.edu, 2019. http://www.ll.mit.edu/media/9021. (accessed 06/13, 2020).
[26] NIST, "CVE-2019-0708 Detail", https://nvd.nist.gov/vuln/detail/CVE-2019-0708, 2019. (accessed 06/13, 2020).
[27] P. Celeda et al., "Lessons learned from complex hands-on defence exercises in a cyber range," 2017 IEEE Frontiers in Education Conference (FIE), Indianapolis, IN, pp. 1-8, doi: 10.1109/FIE.2017.8190713,2017.
[28] Peng Xie et al., "Using Bayesian networks for cyber security analysis," 2010 IEEE/IFIP International Conference on Dependable Systems & Networks (DSN), Chicago, IL, pp. 211-220, doi: 10.1109/DSN.2010.5544924,2010.
[29] T. J. Burns et al., "Analysis and Exercises for Engaging Beginners in Online {CTF} Competitions for Security Education," in 2017 {USENIX} Workshop on Advances in Security Education ({ASE} 17), 2017.
[30] Z. C. Schreuders et al, "Security Scenario Generator (SecGen): A Framework for Generating Randomly Vulnerable Rich-scenario VMs for Learning Computer Security and Hosting {CTF} Events," in 2017 {USENIX} Workshop on Advances in Security Education ({ASE} 17), 2017.