無線感測網路(WSN)已廣泛用於不同類型的監控應用中以檢測是否有入侵存在。在此類應用中，無線感測網路可以依據需求採用各種部署策略。在無線感測網路中，覆蓋範圍是一種評估無線感測網路監控能力的性能指標。被監控區域中的最小曝光路徑(MEP)可以對應於無線感測網路中覆蓋範圍的最壞情況以評估無線感測網路偵測移動物體的能力。不同於現有研究大多數關注於靜態感測網路中的最小曝光路徑查找問題，本論文研究了移動感測網路中的最小曝光路徑查找問題。本論文考慮之場景為一入侵者入侵存在障礙物的被監控區域以進行間諜活動。目標為找到最小曝光路徑使入侵者能夠以最低的風險完成間諜活動。由於在移動感測網路中查找最小曝光路徑的現有解決方案不但缺乏對移動感測器動態特性的考量，也缺乏每次移動對未來造成之影響的考量因而不能很好地解決問題。因此本論文提出了一種基於深度強化學習的演算法來查找最小曝光路徑。實驗結果顯示本論文提出之演算法的有效性並且優於基線演算法。

The wireless sensor network (WSN) has been widely used in different types of surveillance applications to detect intrusion. In such applications, the WSN can be deployed by various deployment strategies as demand. In WSN, the coverage is a performance metric to evaluate how well the WSN can monitor a region of interest. The minimal exposure path (MEP) in the monitored region can correspond to the worst-case coverage of the WSN to evaluate how well the WSN can detect a moving object. Unlike most of the existing studies focused on the MEP finding problem in static sensor networks, this thesis studies the problem of finding MEP in mobile sensor networks. A scenario where an intruder intrudes into a monitored region with the presence of obstacles to conduct espionage activity is considered. The objective is to find the MEP so that the intruder can accomplish the espionage activity with the lowest risk. Since the existing solutions for finding MEP in a mobile sensor network can not well tackle the problem due to the lack of considering the mobile sensors' dynamics and the future consequence of each movement, a deep reinforcement learning-based algorithm is proposed to determine the MEP. The simulation results show that our proposed algorithm is effective and outperforms the baseline algorithms.

[1] J. A. Manrique, J. S. Rueda-Rueda, and J. M. Portocarrero, “Contrasting internet of things and wireless sensor network from a conceptual overview,” in 2016 IEEE International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData), pp. 252–257, 2016.
[2] S. Meguerdichian, F. Koushanfar, G. Qu, and M. Potkonjak, “Exposure in wireless ad-hoc sensor networks,” in Proceedings of the 7th Annual International Conference on Mobile Computing and Networking, MobiCom ’01, p. 139 150, Association for Computing Machinery, 2001.
[3] T. M. B. Nguyen, C. M. Thang, D. N. Nguyen, and T. T. B. Huynh, “Genetic algorithm for solving minimal exposure path in mobile sensor networks,” in 2017 IEEE Symposium Serieson Computational Intelligence (SSCI), pp. 1–8, 2017.
[4] N. Thi My Binh, A. Mellouk, H. Thi Thanh Binh, L. Vu Loi, D. Lam San, and T. Hai Anh, “An elite hybrid particle swarm optimization for solving minimal exposure path problem in mobile wireless sensor networks,” Sensors, vol. 20, no. 9, 2020.
[5] S. Meguerdichian, F. Koushanfar, M. Potkonjak, and M. Srivastava, “Coverage problems in wireless ad-hoc sensor networks,” in Proceedings IEEE INFOCOM 2001. Conference on Computer Communications. Twentieth Annual Joint Conference of the IEEE Computer and Communications Society (Cat. No.01CH37213), vol. 3, pp. 1380–1387 vol.3, 2001.
[6] X.-Y. Li, P.-J. Wan, and O. Frieder, “Coverage in wireless ad hoc sensor networks,” IEEE Transactions on Computers, vol. 52, no. 6, pp. 753–763, 2003.
[7] J. Ai and A. A. Abouzeid, “Coverage by directional sensors in randomly deployed wireless sensor networks,” Journal of Combinatorial Optimization, vol. 11, pp. 21–41, Feb 2006.
[8] T. Clouqueur, V. Phipatanasuphorn, P. Ramanathan, and K. K. Saluja, “Sensor deployment strategy for target detection,” in Proceedings of the 1st ACM International Workshop on Wireless Sensor Networks and Applications, WSNA ’02, (New York, NY, USA), p. 42–48, Association for Computing Machinery, 2002.
[9] Y. Song, L. Liu, H. Ma, and A. V. Vasilakos, “A biology-based algorithm to minimal exposure problem of wireless sensor networks,” IEEE Transactions on Network and Service Management, vol. 11, no. 3, pp. 417–430, 2014.
[10] L. Zhang, X. Chen, J. Fan, D. Wang, and C.-K. Lin, “The minimal exposure path in mobile wireless sensor network,” in 2015 Seventh International Symposium on Parallel Architectures, Algorithms and Programming (PAAP), pp. 73–79, 2015.
[11] T.-L. Chin, P. Ramanathan, K. Saluja, and K.-C. Wang, “Exposure for collaborative detection using mobile sensor networks,” in IEEE International Conference on Mobile Adhoc and Sensor Systems Conference, 2005., pp. 8 pp.–750, 2005.
[12] A. A. Neto, V. C. d. S. Campos, and D. G. Macharet, “Minimal exposure paths in time-varying fields: A semi lagrangian approach,” IEEE Robotics and Automation Letters, vol. 8, no. 2, pp. 664–671, 2023.
[13] X. Meng, H. Inaltekin, and B. Krongold, “Deep reinforcement learning-based topology optimization for self organized wireless sensor networks,” in 2019 IEEE Global Communications Conference (GLOBECOM), pp. 1–6, 2019.
[14] X. Cao, W. Xu, X. Liu, J. Peng, and T. Liu, “A deep reinforcement learning-based on-demand charging algorithm for wireless rechargeable sensor networks,” Ad Hoc Networks, vol. 110, p. 102278, 2021.
[15] B. Li and Y. Wu, “Path planning for uav ground target tracking via deep reinforcement learning,” IEEE Access, vol. 8, pp. 29064–29074, 2020.
[16] Q. Liu, L. Shi, L. Sun, J. Li, M. Ding, and F. Shu, “Path planning for uav-mounted mobile edge computing with deep reinforcement learning,” IEEE Transactions on Vehicular Technology, vol. 69, no. 5, pp. 5723–5728, 2020.
[17] K. Li, W. Ni, and F. Dressler, “Lstm-characterized deep reinforcement learning for continuous flight control and resource allocation in uav-assisted sensor network,” IEEE Internet of Things Journal, vol. 9, no. 6, pp. 4179–4189, 2022.
[18] B. Zhu, E. Bedeer, H. H. Nguyen, R. Barton, and J. Henry, “Uav trajectory planning in wireless sensor networks for energy consumption minimization by deep reinforcement learning,” IEEE Transactions on Vehicular Technology, vol. 70, no. 9, pp. 9540–9554, 2021.
[19] U. Challita, W. Saad, and C. Bettstetter, “Deep reinforcement learning for interference-aware path planning of cellular-connected uavs,” in 2018 IEEE International Conference on Communications (ICC), pp. 1–7, 2018.
[20] V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller, “Playing atari with deep reinforcement learning,” 2013.
[21] H. van Hasselt, A. Guez, and D. Silver, “Deep reinforcement learning with double q-learning,” 2015.
[22] Z. Wang, T. Schaul, M. Hessel, H. van Hasselt, M. Lanctot, and N. de Freitas, “Dueling network architectures for deep reinforcement learning,” 2015.