[1] K. Khawam, S. Lahoud, M. E. Helou, S. Martin, and F. Gang, “Coordinated framework for spectrum
allocation and user association in 5G HetNets with mmWave,” IEEE Transactions on Mobile Computing, vol. 21, no. 4, pp. 1226–1243, Apr. 2022.
[2] L. Bai, Z. Huang, X. Zhang, and X. Cheng, “A non-stationary 3D model for 6G massive MIMO
mmWave UAV channels,” IEEE Transactions on Wireless Communications, vol. 21, no. 6, pp. 4325–4339, Jun. 2022.
[3] M. W. Akhtar, S. A. Hassan, A. Mahmood, H. Jung, H. K. Qureshi, and M. Gidlund, “Q2ANOMA:
A Q-learning-based QoS-aware NOMA system design for diverse data rate requirements,” IEEE Transactions on Industrial Informatics, vol. 18, no. 11, pp. 7549–7559, Nov. 2022.
[4] I. Union, “IMT traffic estimates for the years 2020 to 2030,” Report ITU, vol. 2370, 2015.
[5] L. Liu, B. Sun, X. Tan, and D. H. K. Tsang, “Energy-efficient resource allocation and subchannel
assignment for NOMA-enabled multiaccess edge computing,” IEEE Systems Journal, vol. 16, no. 1, pp. 1558–1569, Mar. 2022.
[6] G. Interdonato, S. Buzzi, C. D’Andrea, L. Venturino, C. D’Elia, and P. Vendittelli, “On the coexistence
of eMBB and URLLC in multi-cell massive MIMO,” IEEE Open Journal of the Communications Society, vol. 4, pp. 1040–1059, Apr. 2023.
[7] L. M. P. Larsen, H. L. Christiansen, S. Ruepp, and M. S. Berger, “Toward greener 5G and beyond
radio access networks—A survey,” IEEE Open Journal of the Communications Society, vol. 4, pp.
768–797, Mar. 2023.
[8] T. Islam, D. Lee, and S. S. Lim, “Enabling network power savings in 5G-advanced and beyond,” IEEE Journal on Selected Areas in Communications, vol. 41, no. 6, pp. 1888–1899, Jun. 2023.
[9] M. S. Ali, E. Hossain, and D. I. Kim, “Coordinated multipoint transmission in downlink multi-cell NOMA systems: Models and spectral efficiency performance,” IEEE Wireless Communications, vol. 25, no. 2, pp. 24–31, Apr. 2018.
[10] A. J. Muhammed, Z. Ma, Z. Ding, M. Xiao, P. Fan, W. Xu, G. Liu, and Z. Zhang, “Resource allocation
for energy-efficient NOMA system in coordinated multipoint networks,” IEEE Transactions on Vehicular Technology, vol. 70, no. 2, pp. 1577–1591, Feb. 2021.
[11] S. Mukherjee, D. Kim, and J. Lee, “Base station coordination scheme for multi-tier ultra-dense networks,” IEEE Transactions on Wireless Communications, vol. 20, no. 11, pp. 7317–7332, Nov. 2021.
[12] K. Wang, H. Li, Z. Ding, and P. Xiao, “Reinforcement learning based latency minimization in secure
NOMA-MEC systems with hybrid SIC,” IEEE Transactions on Wireless Communications, vol. 22, no. 1, pp. 408–422, Jan. 2023.
[13] E. Oh and K. Son, “A unified base station switching framework considering both uplink and downlink
traffic,” IEEE Wireless Communications Letters, vol. 6, no. 1, pp. 30–33, Feb. 2017.
[14] W. K. Lai, C.S. Shieh, C.S. Ho, and Y.R. Chen, “A clustering-based energy saving scheme for dense
small cell networks,” IEEE Access, vol. 7, pp. 2880–2893, Dec. 2019.
[15] C. Luo and J. Liu, “Load based dynamic small cell on/off strategy in ultra-dense
networks,” in Proc. International Conf. on Wireless Communications and Signal Processing (WCSP), Oct. 2018, pp. 1–6.
[16] B. Zhuang, D. Guo, and M. L. Honig, “Energy-efficient cell activation, user association, and spectrum allocation in heterogeneous networks,” IEEE Journal on Selected Areas in Communications, vol. 34, no. 4, pp. 823–831, Apr. 2016.
[17] J. Liu, B. Krishnamachari, S. Zhou, and Z. Niu, “DeepNap: Data-driven base station sleeping operations through deep reinforcement learning,” IEEE Internet of Things Journal, vol. 5, no. 6, pp. 4273–4282, Dec. 2018.
[18] J. Ye and Y.J. A. Zhang, “DRAG: Deep reinforcement learning based base station activation in heterogeneous networks,” IEEE Transactions on Mobile Computing, vol. 19, no. 9, pp. 2076–2087, Sep. 2020.
[19] M. Ozturk, A. I. Abubakar, J. P. B. Nadas, R. N. B. Rais, S. Hussain, and M. A. Imran, “Energy
optimization in ultra-dense radio access networks via traffic-aware cell switching,” IEEE Transactions on Green Communications and Networking, vol. 5, no. 2, pp. 832–845, Jun. 2021.
[20] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization
algorithms,” vol. abs/1707.06347, 2017. [Online]. Available: http://arxiv.org/abs/1707.06347
[21] C. Yu, A. Velu, E. Vinitsky, J. Gao, Y. Wang, A. Bayen, and Y. WU, “The surprising effectiveness of PPO in cooperative, multi-agent games,” in Advances in Neural Information Processing Systems, Dec. 2022, pp. 24 611–24 624.
[22] C. Yu, “The surprising effectiveness of PPO in cooperative, multi-agent games,” Nov. 2021. [Online].
Available: https://github.com/marlbenchmark/onpolicy
[23] F. E. Salem, Z. Altman, A. Gati, T. Chahed, and E. Altman, “Reinforcement learning approach for advanced sleep modes management in 5G networks,” in Proc. IEEE Vehicular Technology Conf. (VTCFall), Aug. 2018, pp. 1–5.
[24] R. Li, Z. Zhao, X. Chen, J. Palicot, and H. Zhang, “TACT: A transfer actor-critic learning framework
for energy saving in cellular radio access networks,” IEEE Transactions on Wireless Communications, vol. 13, no. 4, pp. 2000–2011, Apr. 2014.
[25] A. E. Amine, P. Dini, and L. Nuaymi, “Reinforcement learning for delay-constrained energy-aware small cells with multi-sleeping control,” in Proc. IEEE International Conf. on Communications Workshops (ICC Workshops), Jun. 2020, pp. 1–6.
[26] X. Chen, J. Wu, Y. Cai, H. Zhang, and T. Chen, “Energy-efficiency oriented traffic offloading in wireless networks: A brief survey and a learning approach for heterogeneous cellular networks,” IEEE Journal on Selected Areas in Communications, vol. 33, no. 4, pp. 627–640, Apr. 2015.
[27] X. Dai, Z. Xiao, H. Jiang, M. Alazab, J. C. S. Lui, S. Dustdar, and J. Liu, “Task co-offloading
for D2D-assisted mobile edge computing in Industrial Internet of Things,” IEEE Transactions on Industrial
Informatics, vol. 19, no. 1, pp. 480–490, Jan. 2023.
[28] Z. Li, N. Zhu, D. Wu, H. Wang, and R. Wang, “Energy-efficient mobile edge computing under delay
constraints,” IEEE Transactions on Green Communications and Networking, vol. 6, no. 2, pp. 776–786, Jun. 2022.
[29] M. Gao, R. Shen, J. Li, S. Yan, Y. Li, J. Shi, Z. Han, and L. Zhuo, “Computation offloading with
instantaneous load billing for mobile edge computing,” IEEE Transactions on Services Computing,
vol. 15, no. 3, pp. 1473–1485, May 2022.
[30] C. Kai, H. Zhou, Y. Yi, and W. Huang, “Collaborative cloud-edge-end
task offloading in mobile-edge computing networks with limited communication capability,” IEEE Transactions on Cognitive Communications and Networking, vol. 7, no. 2, pp. 624–634, Jun. 2021.
[31] Z. Jin, C. Zhang, Y. Jin, L. Zhang, and J. Su, “A resource allocation scheme for joint optimizing energy
consumption and delay in collaborative edge computing-based
Industrial IoT,” IEEE Transactions on
Industrial Informatics, vol. 18, no. 9, pp. 6236–6243, Sep. 2022.
[32] X. Deng, J. Yin, P. Guan, N. N. Xiong, L. Zhang, and S. Mumtaz, “Intelligent delay-aware partial
computing task offloading for multiuser Industrial Internet of Things through edge computing,” IEEE
Internet of Things Journal, vol. 10, no. 4, pp. 2954–2966, Feb. 2023.
[33] L. Ale, N. Zhang, X. Fang, X. Chen, S. Wu, and L. Li, “Delay-aware and energy-efficient computation
offloading in mobile-edge computing using deep reinforcement learning,” IEEE Transactions on
Cognitive Communications and Networking, vol. 7, no. 3, pp. 881–892, Sep. 2021.
[34] H. Zhou, K. Jiang, X. Liu, X. Li, and V. C. M. Leung, “Deep reinforcement learning for energy-efficient
computation offloading in mobile-edge computing,” IEEE Internet of Things Journal, vol. 9, no. 2, pp. 1517–1530, Jan. 2022.
[35] M. Tang and V. W. Wong, “Deep reinforcement learning for task offloading in mobile edge computing
systems,” IEEE Transactions on Mobile Computing, vol. 21, no. 6, pp. 1985–1997, Jun. 2022.
[36] F. Zhang, G. Han, L. Liu, M. MartinezGarcia, and Y. Peng, “Deep reinforcement learning based cooperative partial task offloading and resource allocation for IIoT applications,” IEEE Transactions on Network Science and Engineering, early access, pp. 1–1, 2022, doi:10.1109/TNSE.2022.3167949.
[37] X. Qi, M. Peng, and H. Zhang, “Joint mmWave beamforming and resource allocation in NOMA-MEC
network for Internet of Things,” IEEE Transactions on Vehicular Technology, vol. 72, no. 4, pp. 4969–4980, Apr. 2023.
[38] Z. Chen, L. Zhang, Y. Pei, C. Jiang, and L. Yin, “NOMAbased multiuser
mobile edge computation offloading via cooperative multi-agent deep reinforcement learning,” IEEE Transactions on Cognitive Communications and Networking, vol. 8, no. 1, pp. 350–364, Mar. 2022.
[39] V. D. Tuong, W. Noh, and S. Cho, “Delay minimization for NOMA-enabled
mobile edge computing in Industrial Internet of Things,” IEEE Transactions on Industrial Informatics, vol. 18, no. 10, pp. 7321–7331, Oct. 2022.
[40] S. SobhiGivi, M. G. Shayesteh, and H. Kalbkhani, “Energy-efficient power allocation and user selection
for mmWave-NOMA transmission in M2M communications underlaying cellular heterogeneous
networks,” IEEE Transactions on Vehicular Technology, vol. 69, no. 9, pp. 9866–9881, Jun. 2020.
[41] N. Otao, Y. Kishiyama, and K. Higuchi, “Performance of nonorthogonal access with SIC in cellular
downlink using proportional fair-based resource allocation,” in Proc. International Symposium on
Wireless Communication Systems (ISWCS) 2012, Aug. 2012, pp. 476–480.
[42] M. E. Tanab and W. Hamouda, “Efficient resource allocation in fast-uplink grant for machine-type
communications with NOMA,” IEEE Internet of Things Journal, vol. 9, no. 18, pp. 18 113–18 129, Sep. 2022.
[43] B. Rashid, A. Ahmad, S. Ali, Z. Kaleem, A. Alkhayyat, and C. Yuen, “Energy efficient resource allocation for uplink MCNOMA based heterogeneous small cell networks with wireless backhaul,” IEEE Transactions on Vehicular Technology, vol. 72, no. 3, pp. 3419–3429, Mar. 2023.
[44] I. Azam, M. B. Shahab, and S. Y. Shin, “Energy-efficient pairing and power allocation for NOMA
UAV network under QoS constraints,” IEEE Internet of Things Journal, 2022.
[45] X. Wang, Y. Zhang, R. Shen, Y. Xu, and F.C. Zheng, “DRL-based energy-efficient resource allocation frameworks for uplink NOMA systems,” IEEE Internet of Things Journal, vol. 7, no. 8, pp. 7279–7294, Aug. 2020.
[46] F. Wu, L. Zhang, and Y. He, “Energy efficiency optimization in downlink NOMA-enabled fog radio
access network based on deep reinforcement learning,” in Proc. Computing, Communications and IoT
Applications (ComComAp2021), Nov. 2021, pp. 14–19.
[47] S. Wang, X. Wang, Y. Zhang, and Y. Xu, “Resource allocation in multi-cell NOMA systems with multi-agent deep reinforcement learning,” in Proc. IEEE Wireless Communications and Networking Conf. (WCNC) 2021, Mar. 2021, pp. 1–6.
[48] Y. Cao, G. Zhang, G. Li, and J. Zhang, “A deep Q-Network based-resource
allocation scheme for massive MIMO-NOMA,”
IEEE Communications Letters, vol. 25, no. 5, pp. 1544–1548, May 2021.
[49] J. Zheng, X. Tang, X. Wei, H. Shen, and L. Zhao, “Channel assignment for hybrid NOMA systems
with deep reinforcement learning,” IEEE Wireless Communications Letters, vol. 10, no. 7, pp. 1370–
1374, Jul. 2021.
[50] W. Ahsan, W. Yi, Z. Qin, Y. Liu, and A. Nallanathan, “Resource allocation in uplink NOMA-IoT networks:
A reinforcement-learning approach,” IEEE Transactions on Wireless Communications, vol. 20,
no. 8, pp. 5083–5098, Aug. 2021.
[51] A. B. M. Adam, L. Lei, S. Chatzinotas, and N. U. R. Junejo, “Deep convolutional self-attention network
for energy-efficient power control in NOMA networks,” IEEE Transactions on Vehicular Technology, vol. 71, no. 5, pp. 5540–5545, May 2022.
[52] C. Chaieb, F. Abdelkefi, and W. Ajib, “Deep reinforcement learning for resource allocation in multiband
and hybrid OMANOMA
wireless networks,” IEEE Transactions on Communications, vol. 71,
no. 1, pp. 187–198, Jan. 2023.
[53] S. Wang, T. Lv, W. Ni, N. C. Beaulieu, and Y. J. Guo, “Joint resource management for MCNOMA:
A deep reinforcement learning approach,” IEEE Transactions on Wireless Communications, vol. 20,
no. 9, pp. 5672–5688, Sep. 2021.
[54] A. Abdelnasser, E. Hossain, and D. I. Kim, “Clustering and resource allocation for dense femtocells in
a twotier
cellular OFDMA network,” IEEE Transactions on Wireless Communications, vol. 13, no. 3,
pp. 1628–1641, Mar. 2014.
[55] A. I. Sulyman, A. Alwarafy, G. R. MacCartney, T. S. Rappaport, and A. Alsanie, “Directional radio
propagation path loss models for millimeterwave
wireless networks in the 28,
60,
and 73GHz
bands,” IEEE Transactions on Wireless Communications, vol. 15, no. 10, pp. 6939–6947, Oct. 2016.
[56] B. Soleimani and M. Sabbaghian, “Clusterbased
resource allocation and user association in mmWave
femtocell networks,” IEEE Transactions on Communications, vol. 68, no. 3, pp. 1746–1759, Mar.
2020.
[57] T. S. Rappaport, “Wireless communications – Principles and practice, (the book end),” Microwave
Journal, vol. 45, no. 12, pp. 128–129, Jul. 2002.
[58] M. R. Akdeniz, Y. Liu, M. K. Samimi, S. Sun, S. Rangan, T. S. Rappaport, and E. Erkip, “Millimeter
wave channel modeling and cellular capacity evaluation,” IEEE Journal on Selected Areas in Communications,
vol. 32, no. 6, pp. 1164–1179, Jun. 2014.
[59] T. S. Rappaport, G. R. MacCartney, M. K. Samimi, and S. Sun, “Wideband millimeterwave
propagation
measurements and channel models for future wireless communication system design,” IEEE
Transactions on Communications, vol. 63, no. 9, pp. 3029–3056, Sep. 2015.
[60] A. Alkhateeb, O. El Ayach, G. Leus, and R. W. Heath, “Channel estimation and hybrid precoding for
millimeter wave cellular systems,” IEEE Journal of Selected Topics in Signal Processing, vol. 8, no. 5, pp. 831–846, Oct. 2014.
[61] B. Li, Y. Dai, Z. Dong, E. Panayirci, H. Jiang, and H. Jiang, “Energy-efficient
resources allocation with millimeterwave massive MIMO in ultra dense HetNets by SWIPT and CoMP,” IEEE Transactions on Wireless Communications, vol. 20, no. 7, pp. 4435–4451, Jul. 2021.
[62] F. B. Mismar, B. L. Evans, and A. Alkhateeb, “Deep reinforcement learning for 5G networks: Joint
beamforming, power control, and interference coordination,” IEEE Transactions on Communications,
vol. 68, no. 3, pp. 1581–1592, Mar. 2020.
[63] J. Zhao, Y. Liu, K. K. Chai, A. Nallanathan, Y. Chen, and Z. Han, “Spectrum allocation and power
control for nonorthogonal multiple access in HetNets,” IEEE Transactions on Wireless Communications, vol. 16, no. 9, pp. 5825–5837, Sep. 2017.
[64] Y. Xu and S. Mao, “User association in massive MIMO HetNets,” IEEE Systems Journal, vol. 11, no. 1, pp. 7–19, Mar. 2017.
[65] T. Zhang, K. Zhu, J. Wang, and Z. Han, “Cost-efficient beam management and resource allocation in millimeter wave backhaul HetNets with hybrid energy supply,” IEEE Transactions on Wireless Communications, vol. 21, no. 5, pp. 3291–3306, May 2022.
[66] S. Han, C. Yang, and A. F. Molisch, “Spectrum and energy efficient cooperative base station doze,”
IEEE Journal on Selected Areas in Communications, vol. 32, no. 2, pp. 285–296, Feb. 2014.
[67] D. Zhao, H. Lu, Y. Wang, H. Sun, and Y. Gui, “Joint power allocation and user association optimization for IRS-assisted mmWave systems,” IEEE Transactions on Wireless Communications, vol. 21,
no. 1, pp. 577–590, Jan. 2022.
[68] L. A. Wolsey and G. L. Nemhauser, Integer and combinatorial optimization. John Wiley & Sons, 1999, vol. 55.
[69] M. Ester, H.P. Kriegel, J. Sander, and X. Xu, “A density-based algorithm for discovering clusters in large spatial databases with noise.” in Proc. KDD, vol. 96, no. 34, Aug. 1996, pp. 226–231.
[70] F. A. Oliehoek and C. Amato, A Concise Introduction to Decentralized POMDPs. Springer, 2016.
[71] S. Aboagye, A. Ibrahim, and T. M. N. Ngatched, “Frameworks for energy efficiency maximization
in HetNets with millimeter wave backhaul links,” IEEE Transactions on Green Communications and Networking, vol. 4, no. 1, pp. 83–94, Mar. 2020.
[72] S. K. S. Tyagi, A. Mukherjee, Q. Boyang, and D. K. Jain, “Computing resource optimization of big
data in optical cloud radio access networked Industrial Internet of Things,” IEEE Transactions on
Industrial Informatics, vol. 17, no. 11, pp. 7734–7742, Nov. 2021.
[73] Z. Ning, X. Wang, J. J. P. C. Rodrigues, and F. Xia, “Joint computation offloading, power allocation,
and channel assignment for 5G-enabled traffic management systems,” IEEE Transactions on Industrial Informatics, vol. 15, no. 5, pp. 3058–3067, May 2019.
[74] T. M. Ho and K.K. Nguyen, “Joint server selection, cooperative offloading and handover in multiaccess edge computing wireless network: A deep reinforcement learning approach,” IEEE Transactions on Mobile Computing, vol. 21, no. 7, pp. 2421–2435, Jul. 2022.
[75] Y. Mao, C. You, J. Zhang, K. Huang, and K. B. Letaief, “A survey on mobile edge computing: The
communication perspective,” IEEE Communications Surveys & Tutorials, vol. 19, no. 4, pp. 2322–2358, Aug. 2017.
[76] X. Qi, G. Xie, and Y. Liu, “Energy-efficient power allocation in multiuser mmWaveNOMA systems
with finite resolution analog precoding,” IEEE Transactions on Vehicular Technology, vol. 71, no. 4, pp. 3750–3759, Apr. 2022.
[77] B. Xu, Y. Chen, J. R. Carrión, and T. Zhang, “Resource allocation in energy-cooperation
enabled two-tier NOMA HetNets toward green 5G,” IEEE Journal on Selected Areas in Communications, vol. 35, no. 12, pp. 2758–2770, Dec. 2017.
[78] M. F. Hanif, Z. Ding, T. Ratnarajah, and G. K. Karagiannidis, “A minorization-maximization
method for optimizing sum rate in the downlink of nonorthogonal multiple access systems,” IEEE Transactions on Signal Processing, vol. 64, no. 1, pp. 76–88, Jan. 2016.
[79] M. Elhattab, M. A. Arfaoui, and C. Assi, “Joint clustering and power allocation in coordinated multipoint
assisted CNOMA cellular networks,” IEEE Transactions on Communications, vol. 70, no. 5, pp. 3483–3498, May. 2022.
[80] X. Wei, H. AlObiedollah, K. Cumanan, Z. Ding, and O. A. Dobre, “Energy efficiency maximization
for hybrid TDMANOMA system with opportunistic time assignment,” IEEE Transactions on Vehicular Technology, vol. 71, no. 8, pp. 8561–8573, Aug. 2022.
[81] M. Elhattab, M. A. Arfaoui, C. Assi, A. Ghrayeb, and M. Qaraqe, “Power allocation optimization and
decoding order selection in uplink CNOMA networks,” IEEE Communications Letters, vol. 27, no. 1, pp. 352–356, Jan. 2023.
[82] W. Ahsan, W. Yi, Y. Liu, and A. Nallanathan, “A reliable reinforcement learning for resource allocation in uplink NOMA-URLLC networks,” IEEE Transactions on Wireless Communications, vol. 21,
no. 8, pp. 5989–6002, Aug. 2022.
[83] M. S. Ali, H. Tabassum, and E. Hossain, “Dynamic user clustering and power allocation for uplink
and downlink nonorthogonal multiple access (NOMA) systems,” IEEE Access, vol. 4, pp. 6325–6343, Aug 2016.
[84] A. Graves, A.r. Mohamed, and G. Hinton, “Speech recognition with deep recurrent neural networks,” in IEEE International Conf. on Acoustics, Speech and Signal Processing, Oct. 2013, pp. 6645–6649.
[85] R. Child, S. Gray, A. Radford, and I. Sutskever, “Generating long sequences with sparse transformers,” CoRR, vol. abs/1904.10509, Apr. 2019. [Online]. Available: http://arxiv.org/abs/1904.10509