中国邮电高校学报(英文) ›› 2007, Vol. 14 ›› Issue (3): 48-55.doi: 1005-8885 (2007) 03-0048-08

• Network • 上一篇    下一篇

Markov game for autonomic joint radio resource management in a multi-operator scenario

张永靖;LIN Yue-wei   

  1. School of Telecommunication Engineering, Beijing University of Posts and Telecommunications, Beijing 100876, China
  • 收稿日期:2006-03-03 修回日期:1900-01-01 出版日期:2007-09-30

Markov game for autonomic joint radio resource management in a multi-operator scenario

ZHANG Yong-jing; LIN Yue-wei   

  1. School of Telecommunication Engineering, Beijing University of Posts and Telecommunications, Beijing 100876, China
  • Received:2006-03-03 Revised:1900-01-01 Online:2007-09-30

摘要:

This article addresses the autonomy of joint radio resource management (JRRM) between heterogeneous radio access technologies (RATs) owned by multiple operators. By modeling the inter-operator competition as a general-sum Markov game, correlated-Q learning (CE-Q) is introduced to generate the operators’ pricing and admission policies at the correlated equilibrium autonomically. The heterogeneity in terms of coverage, service suitability, and cell capacity amongst different RATs are considered in the input state space, which is generalized using multi-layer feed-forward neural networks for less memory requirement. Simulation results indicate that the proposed algorithm can produce rational JRRM polices for each network under different load conditions through the autonomic learning process. Such policies guide the traffic toward an optimized distribution and improved resource utilization, which results in the highest network profits and lowest blocking probability compared to other self-learning algorithms.

关键词:

autonomic; JRRM; multi-operator; reinforcement learning (RL); Markov game

Abstract:

This article addresses the autonomy of joint radio resource management (JRRM) between heterogeneous radio access technologies (RATs) owned by multiple operators. By modeling the inter-operator competition as a general-sum Markov game, correlated-Q learning (CE-Q) is introduced to generate the operators’ pricing and admission policies at the correlated equilibrium autonomically. The heterogeneity in terms of coverage, service suitability, and cell capacity amongst different RATs are considered in the input state space, which is generalized using multi-layer feed-forward neural networks for less memory requirement. Simulation results indicate that the proposed algorithm can produce rational JRRM polices for each network under different load conditions through the autonomic learning process. Such policies guide the traffic toward an optimized distribution and improved resource utilization, which results in the highest network profits and lowest blocking probability compared to other self-learning algorithms.

Key words:

autonomic; JRRM; multi-operator; reinforcement learning (RL); Markov game

中图分类号: