Improved Q-learning algorithm for load balance in millimeter wave backhaul networks

doi:10.19682/j.cnki.1005-8885.2018.0020

JOURNAL OF CHINA UNIVERSITIES OF POSTS AND TELECOM ›› 2018, Vol. 25 ›› Issue (3): 8-16.doi: 10.19682/j.cnki.1005-8885.2018.0020

• Wireless • Previous Articles Next Articles

Improved Q-learning algorithm for load balance in millimeter wave backhaul networks

Received:2018-02-02 Revised:2018-05-19 Online:2018-06-29 Published:2018-06-30
Contact: Dan fengMeng E-mail:570717835@qq.com

Abstract

Abstract: With the intensive deployment of users and the drastic increase of traffic load, a millimeter wave (mmWave) back-haul network was widely investigated. A typical mmWave back-haul network consists of the macro base station (MBS) and the small base stations (SBSs). How to efficiently associate users with the MBS and the SBSs for load balancing is a key issue in the network. By adding a virtual power bias to the SBSs, more users can access to the SBSs to share the load of the MBS. The bias values shall be set reasonably to guarantee the back-haul efficiency and the quality of service (QoS). An improved Q-learning algorithm is proposed to effectively adjust the bias value for each SBS. In the proposed algorithm, each SBS becomes an agent with independent learning and can achieve the best behavior, namely the optimal bias value through a series of training. Besides, an improved behavior selection mechanism is adopted to improve the learning efficiency and accelerate the convergence of the algorithm. Finally, simulations conducted in the 60 GHz band demonstrate the superior performance of the proposed algorithm in back-haul efficiency and user outage probability.

Key words: millimeter wave backhaul networks, load balance, user association, Q-learning

References

1. Feng W, Li Y, Jin D P, et al. Millimetre-wave backhaul for 5G networks: Challenges and solutions. Sensors, 2016, 16(6): Article 892.

2. Athanasiou G, Weeraddana P C, Fischione C, et al. Optimizing client association for load balancing and fairness in millimeter-wave wireless networks. IEEE/ACM Transactions on Networking, 2015, 23(3): 836–850.

3. Koizumi T, Higuchi K. A simple decentralized cell association method for heterogeneous networks. IEICE Transactions on Communications, 2013, E96.B(6): 1358–1366.

4. Xu Y, Yin R, Yu G D. Adaptive biasing scheme for load balancing in backhaul constrained small cell networks. IET Communications, 2015, 9(7): 999–1005.

5. Thakur R, Sengupta A, Murthy C S R. Improving capacity and energy efficiency of femtocell based cellular network through cell biasing. Proceedings of the 11th International Symposium on Modeling and Optimization in Mobile, Ad Hoc and Wireless Networks (WiOpt’13), May 13-17, 2013, Tsukuba Science City, Japan. Piscataway, NJ, USA: IEEE, 2013: 436–443.

6. Ye Q Y, Rong B Y, Chen Y D, et al. User association for load balancing in heterogeneous cellular networks. IEEE Transactions on Wireless Communications, 2013, 12(6): 2706–2716.

7. Kudo T, Ohtsuki T. Cell selection using distributed Q-learning in heterogeneous networks. Proceedings of the 2013 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA’13), Oct 29-Nov 1, 2013, Kaohsiung, China. Piscataway, NJ, USA: IEEE, 2013 6p.

8. Tam H H M, Tuan H D, Ngo D T, et al. Joint load balancing and interference management for small-cell heterogeneous networks with limited backhaul capacity. IEEE Transactions on Wireless Communications, 2017, 16(2): 872–884.

9. Hao W M, Zeng M, Chu Z, et al. Energy-efficient resource allocation for mmWave massive MIMO HetNets with wireless backhaul. IEEE Access, 2018, 6: 2457–2471.

10. Sangiamwong J, Saito Y, Miki N, et al. Investigation on cell selection methods associated with inter-cell interference coordination in heterogeneous networks for LTE-advanced downlink. Proceedings of the 17th European Wireless Conference on Sustainable Wireless Technologies, Apr 27-29, 2011, Vienna, Austria. Piscataway, NJ, USA: IEEE, 2011: 6p.

11. Sun Y X, Liu C, Qiu H K. The research on patterns and UCT algorithm in NoGo game. Proceedings of the 25th Chinese Control and Decision Conference (CCDC’13), May 25-27, 2013, Guiyang, China. Piscataway, NJ, USA: IEEE, 2013: 1178–1182.

12. Watkins C J C H, Dayan P. Q-learning. Machine Learning, 1992, 8(3/4): 279–292.

13. Niu Y, Gao C H, Li Y, et al. Energy-efficient scheduling for mmWave backhauling of small cells in heterogeneous cellular networks. IEEE Transactions on Vehicular Technology, 2017, 66(3): 2674–2687.

Metrics

Comments

Copyright © 2020 The Journal of China Universities of Posts and Telecommunications
　 Adress: P.O. Box 231,Beijing University of Posts and Telecommunications,10 Xi Tucheng Road,Beijing 100876,P.R.China　Post Code: 100081
Tel：86-010-62282493　Fax： 86-010-62283461　E-mail: jchupt@bupt.edu.cn
Support by: Beijing Magtech Co.Ltd

Improved Q-learning algorithm for load balance in millimeter wave backhaul networks

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 0

Recommended Articles

Metrics

Comments