中国邮电高校学报(英文) ›› 2018, Vol. 25 ›› Issue (3): 8-16.doi: 10.19682/j.cnki.1005-8885.2018.0020

• Wireless • 上一篇    下一篇

Improved Q-learning algorithm for load balance in millimeter wave backhaul networks

蒙丹凤1,李晓辉1,蒲文娟2   

  1. 1. 陕西省 西安电子科技大学
    2.
  • 收稿日期:2018-02-02 修回日期:2018-05-19 出版日期:2018-06-29 发布日期:2018-06-30
  • 通讯作者: 蒙丹凤 E-mail:570717835@qq.com

Improved Q-learning algorithm for load balance in millimeter wave backhaul networks

  • Received:2018-02-02 Revised:2018-05-19 Online:2018-06-29 Published:2018-06-30
  • Contact: Dan fengMeng E-mail:570717835@qq.com

摘要: With the intensive deployment of users and the drastic increase of traffic load, a millimeter wave (mmWave) back-haul network was widely investigated. A typical mmWave back-haul network consists of the macro base station (MBS) and the small base stations (SBSs). How to efficiently associate users with the MBS and the SBSs for load balancing is a key issue in the network. By adding a virtual power bias to the SBSs, more users can access to the SBSs to share the load of the MBS. The bias values shall be set reasonably to guarantee the back-haul efficiency and the quality of service (QoS). An improved Q-learning algorithm is proposed to effectively adjust the bias value for each SBS. In the proposed algorithm, each SBS becomes an agent with independent learning and can achieve the best behavior, namely the optimal bias value through a series of training. Besides, an improved behavior selection mechanism is adopted to improve the learning efficiency and accelerate the convergence of the algorithm. Finally, simulations conducted in the 60 GHz band demonstrate the superior performance of the proposed algorithm in back-haul efficiency and user outage probability.

关键词: millimeter wave backhaul networks, load balance, user association, Q-learning

Abstract: With the intensive deployment of users and the drastic increase of traffic load, a millimeter wave (mmWave) back-haul network was widely investigated. A typical mmWave back-haul network consists of the macro base station (MBS) and the small base stations (SBSs). How to efficiently associate users with the MBS and the SBSs for load balancing is a key issue in the network. By adding a virtual power bias to the SBSs, more users can access to the SBSs to share the load of the MBS. The bias values shall be set reasonably to guarantee the back-haul efficiency and the quality of service (QoS). An improved Q-learning algorithm is proposed to effectively adjust the bias value for each SBS. In the proposed algorithm, each SBS becomes an agent with independent learning and can achieve the best behavior, namely the optimal bias value through a series of training. Besides, an improved behavior selection mechanism is adopted to improve the learning efficiency and accelerate the convergence of the algorithm. Finally, simulations conducted in the 60 GHz band demonstrate the superior performance of the proposed algorithm in back-haul efficiency and user outage probability.

Key words: millimeter wave backhaul networks, load balance, user association, Q-learning