隨著無線網路逐漸的普及,許多無線新技術在近年來有重大的發展與轉變,其中之一就是機器型態通訊 。基於網路架構所提供廣大涵蓋率以及高 負載能力下,使得機器型態通訊的連結性變得更為可行。然而,大量的機器型態 通訊裝置試圖去與網路連接,將對網路端附載能力造成重大威脅。因此,如何設計一個有效率的演算法以解決基地台之負載平衡將是機器型態通訊中一項重要議題。在此篇論文裡,我們提議一個基於增強學習技術的演算法『正差值導向探索 模式』,來解決機器型態通訊在網路端所造成的高負載議題。而根據模擬結果顯示,我們所提出的演算法在網路連接成功率相對於傳統演算法亦或是其他常見之增強學習演算法中都是較為突出的。

Machine Type Communication (MTC), as one of the most promising technologies in the future wireless communication, has brought mobile communication network into a new level. The breakthrough of cutting-edge technology and broad coverage of cellular networks in Long Term Evolution Advanced (LTE- A) network architecture constitute an ideal platform for ubiquitous MTC service provisioning on large scale. However, the access requests from a massive number of MTC de- vices pose a great threat on the radio access network (RAN), causing the network congested and overloaded. As a result, it is necessary to design an efficient eNB selection mechanism to avoid overload issue. In this paper, MTC arrivals are assumed to be non-homogeneous Poisson process (NHPP) cases and we formulate the eNB selection problem as Markov Decision Process (MDP) which comes neatly with NHPP in forms of memorylessness. Also, a learning-based algorithm with policy, Value-Difference Based Exploration (VDBE), is integrated to reinforce the weakness of MDP. Numerical results show that our proposed mechanism outperforms other learning-based approaches and traditional approaches which are commonly used in network selection in terms of robustness, stability and efficiency.

[1] 3GPP TR 22.368 v13.0.0, “Service requirements for Machine-Type Communi- cations,” June 2014.
[2] S.-Y. Lien, K.-C. Chen, and Y. Lin, “Toward ubiquitous massive accesses in 3GPP Machine-to-Machine Communications,” IEEE Commun. Mag., vol. 49, no. 4, pp. 66–74, Apr. 2011.
[3] F. Ghavimi and H.-H. Chen, “M2M communications in 3GPP LTE/LTE-A networks: architectures, service requirements, challenges, and applications,” IEEE Communications Surveys & Tutorials, vol. 17, no. 2, pp. 525–549, Oct. 2015.
[4] 3GPP TR 22.868 v8.0.0, “Study on facilitating machine to machine communi- cation in 3GPP systems,” July 2007.
[5] 3GPP TR 23.888 v11.0.0, “System improvements for Machine-Type Commu- nications (MTC),” Sept. 2012.
[6] ABI Research, “Cellular M2M connectivity services: the market opportunity for moblie operators, MVNOs, and other connectivity service providers,” 2012.
[7] 3GPP TR 37.868 v11.0.0, “Study on RAN improvements for Machine-type Communications,” Oct. 2011.
[8] M. Hasan, E. Hossain, and D. Niyato, “Random access for machine-to-machine communication in LTE-advanced networks: issues and approaches,” IEEE Commun Mag., vol. 51, no. 6, pp. 86–93, Jun. 2013.
[9] F. Cao and Z. Fan, “Cellular M2M network access congestion: Performance analysis and solutions,” in Proc. IEEE WiMob 2013, Oct. 2013, pp. 39–44.
[10] A. G. Gotsis, A. S. Lioumpas, and A. Alexiou, “M2M scheduling over LTE: Challenges and new perspectives,” IEEE Vehicular Technology Magazine, vol. 7, no. 3, pp. 34–39, Sept. 2012.
[11] J.-P. Cheng, C.-H. Lee, and T.-M. Lin, “Prioritized random access with dy- namic access barring for MTC in 3GPP LTE-A networks,” in Proc. IEEE GLOBECOM 2011 workshops, Dec. 2011, pp. 368–372.
[12] T. P. de Andrade, C. A. Astudillo, and N. L. da Fonseca, “Random access mechanism for RAN overload control in LTE/LTE-A networks,” in Proc. IEEE ICC 2015, June. 2015, pp. 7607–7612.
[13] M.-Y. Cheng, G.-Y. Lin, H.-Y. Wei, and A. C.-C. Hsu, “Overload control for Machine-Type-Communications in LTE-Advanced system,” IEEE Communi- cations Magazine, vol. 50, no. 6, pp. 38–45, Jun. 2012.
[14] M. Tokic, “Adaptive ε-greedy exploration in reinforcement learning based on value di erences,” in Proc. AAAI 2010, Sept. 2010, pp. 203–210.
[15] H.-L. He, Q.-H. Du, H.-B. Song, W.-Y. Li, Y.-C. Wang, and P.-Y. Ren, “Tra c- aware ACB scheme for massive access in Machine-to-Machine networks,” in Proc. IEEE ICC 2015, June. 2015, pp. 2226–2231.
[16] M. K. Giluka, A. Prasannakumar, N. Rajoria, and B. R. Tamma, “Adaptive RACH congestion management to support M2M communication in 4G LTE networks,” in Proc. IEEE ANTS 2013, Dec. 2013, pp. 1–6.
[17] S.-T. Sheu, C.-H. Chiu, Y.-C. Cheng, and K.-H. Kuo, “Self-adaptive persistent contention scheme for scheduling based machine type communications in LTE system,” in Proc. IEEE iCOST 2012, Jul. 2012, pp. 77–82.
[18] G. C. Madueno, S. Stefanovic, and P. Popovski, “E cient LTE access with colli- sion resolution for massive M2M communications,” in Proc. IEEE GLOBECOM 2014 workshops, Dec. 2014, pp. 1433–1438.
[19] C.-W. Chang, J.-C. Chen, C. Chen, and R.-H. Jan, “Scattering random-access intensity in LTE Machine-to-Machine (M2M) communications,” in Proc. IEEE GLOBECOM 2013 workshops, Dec. 2013, pp. 4729–4734.
[20] N. Abbas, S. Taleb, H. Hajj, and Z. Dawy, “A learning-based approach for network selection in WLAN/3G heterogeneous network,” in Proc. IEEE ICCIT 2013, Jun. 2013, pp. 309–313.
[21] J. Suga and R. Tafazolli, “Joint resource management with reinforcement learn- ing in heterogeneous networks,” in Proc. IEEE VTC Fall 2013, Sept. 2013, pp. 1–5.
[22] J. Buhler and G. Wunder, “Tra c-aware optimization of heterogeneous access management,” IEEE Transactions on Communications, vol. 58, no. 6, pp. 1737– 1747, Jun. 2010.
[23] K. Kittiwaytang, P. Chanloha, and C. Aswakul, “CTM-Based Reinforcement Learning Strategy for Optimal Heterogeneous Wireless Network Selection,” in Proc. SICCIMS 2010, Sept. 2010, pp. 73–78.
[24] M. El Helou, M. Ibrahim, S. Lahoud, and K. Khawam, “Radio access selection approaches in heterogeneous wireless networks,” in Proc. IEEE WiMob 2013, Oct. 2013, pp. 8–p.
[25] M. El Helou, M. Ibrahim, S. Lahoud, K. Khawam, D. Mezher, and B. Cousin, “A network-assisted approach for rat selection in heterogeneous cellular net- works,” IEEE Journal on Selected Areas in Communications, vol. 33, no. 6, pp. 1055–1067, Mar. 2015.
[26] M. Laner, P. Svoboda, N. Nikaein, and M. Rupp, “Tra c models for machine type communications,” in Proc. ISWCS 2013, Aug. 2013, pp. 1–5.
[27] M.-Y. Cheng, G.-Y. Lin, H.-Y. Wei, and C.-C. Hsu, “Performance evaluation of radio access network overloading from machine type communications in LTE-A networks,” in Proc. IEEE WCNCW 2012, Apr. 2012, pp. 248–252.
[28] “Open cell id,” http://opencellid.org/, accessed: 2016-06-01.