Dynamic computation offloading in time-varying environment for ultra-dense networks: a stochastic game approach

doi:10.19682/j.cnki.1005-8885.2021.1003

The Journal of China Universities of Posts and Telecommunications ›› 2021, Vol. 28 ›› Issue (2): 24-37.doi: 10.19682/j.cnki.1005-8885.2021.1003

Previous Articles Next Articles

Dynamic computation offloading in time-varying environment for ultra-dense networks: a stochastic game approach

Xie Renchao, Liu Xu, Duan Xuefei, Tang Qinqin, Yu Fei Richard, Huang Tao

1. State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing 100876, China
2. Guangdong Communications and Networks Institute, Guangzhou 510700, China
3. Department of Future Networks, Purple Mountain Laboratory, Nanjing 211111, China
4. Department of Systems and Computer Engineering, Carleton University, Ottawa, Canada

Received:2020-08-19 Revised:2021-02-24 Online:2021-04-30 Published:2021-04-30
Contact: Xie Renchao E-mail:renchao_xie@bupt.edu.cn

Abstract

Abstract: To meet the demands of large-scale user access with computation-intensive and delay-sensitive applications,
combining ultra-dense networks (UDNs) and mobile edge computing (MEC)are considered as important solutions.
In the MEC enabled UDNs, one of the most important issues is computation offloading. Although a number of work
have been done toward this issue, the problem of dynamic computation offloading in time-varying environment,
especially the dynamic computation offloading problem for multi-user, has not been fully considered. Therefore, in
order to fill this gap, the dynamic computation offloading problem in time-varying environment for multi-user is
considered in this paper. By considering the dynamic changes of channel state and users queue state, the dynamic
computation offloading problem for multi-user is formulated as a stochastic game, which aims to optimize the delay
and packet loss rate of users. To find the optimal solution of the formulated optimization problem, Nash Q-learning
(NQLN) algorithm is proposed which can be quickly converged to a Nash equilibrium solution. Finally, extensive
simulation results are presented to demonstrate the superiority of NQLN algorithm. It is shown that NQLN algorithm
has better optimization performance than the benchmark schemes.

Key words: dynamic computation offloading, time-varying environment, stochastic game, ultra-dense networks (UDNs), mobile edge computing (MEC)

CLC Number:

TN915.01

Xie Renchao, Liu Xu, Duan Xuefei, Tang Qinqin, Yu Fei Richard, Huang Tao. Dynamic computation offloading in time-varying environment for ultra-dense networks: a stochastic game approach[J]. The Journal of China Universities of Posts and Telecommunications, 2021, 28(2): 24-37.

References

1. Ge X H, Tu S, Mao G Q, et al. 5G ultra-dense cellular networks. IEEE Wireless Communications, 2016, 23(1): 72-79

2. Kamel M, Hamouda W, Youssef A. Ultra-dense networks: a survey. IEEE Communications Surveys & Tutorials, 2016, 18(4): 2522-2545

3. An J P, Yang K, Wu J S, et al. Achieving sustainable ultra-dense heterogeneous networks for 5G. IEEE Communications Magazine, 2017, 55(12): 84-90

4. Mobile edge computing (MEC); framework and reference architecture. ETSI GS MEC 003 V1.1.1. Sophia Antipolis, Franc: ETSI, 2016

5. Mach P, Becvar Z. Mobile edge computing: a survey on architecture and computation offloading. IEEE Communications Surveys & Tutorials, 2017, 19(3): 1628-1656

6. Wang X F, Han Y W, Wang C Y, et al. In-edge AI: intelligentizing mobile edge computing, caching and communication by federated learning. IEEE Network, 2019, 33(5): 156-165

7. Ren J K, Yu G D, He Y H, et al. Collaborative cloud and edge computing for latency minimization. IEEE Trans on Vehicular Technology, 2019, 68(5): 5031-5044

8. Zhang H B, Wang R, Liu J J. Mobility management for ultra-dense edge computing: a reinforcement learning approach. Proceedings of the IEEE 90th Vehicular Technology Conference (VTC’19-Fall), Sept 22-25, 2019, Honolulu, HI, USA. Piscataway, NJ, USA: IEEE, 2019: 1-5

9. Chen S Y, Zhao T Y, Chen H H. et al. Downlink coordinated multi-point transmission in ultra-dense networks with mobile edge computing. IEEE Network, 2019, 33(2): 152-159

10. Wang Q, Zhou F H. Fair resource allocation in an MEC-enabled ultra-dense IoT network with NOMA. Proceedings of the 2019 IEEE International Conference on Communications Workshops (ICC Workshops’19), May 20-24, 2019, Shanghai, China. Piscataway, NJ, USA: IEEE, 2019: 1-6

11. Seng S M, Li X, Luo C Q, et al. A D2D-assisted MEC computation offloading in the blockchain-based framework for UDNs. Proceedings of the 2019 IEEE International Conference on Communications (ICC’19), May 20-24, 2019, Shanghai, China. Piscataway, NJ, USA: IEEE, 2019: 1-6

12. Guo H Z, Zhang J, Liu J J, et al. Energy-efficient task offloading and transmit power allocation for ultra-dense edge computing. Proceedings of the 2018 IEEE Global Communications Conference (GLOBECOM’18), Dec 9-13, 2018, Abu Dhabi, United Arab Emirates. Piscataway, NJ, USA: IEEE, 2018: 1-6

13. Li L J, Zhou H M, Xiong S X, et al. Compound model of task arrivals and load-aware offloading for vehicular mobile edge computing networks. IEEE Access, 2019, 7: 26631-26640

14. Guo F X, Zhang H L, Ji H, et al. An efficient computation offloading management scheme in the densely deployed small cell networks with mobile edge computing. IEEE/ACM Trans on Networking, 2018, 26(6): 2651-2664

15. Liu C B, Li K L, Liang J, et al. COOPER-MATCH: job offloading with a cooperative game for guaranteeing strict deadlines in MEC. IEEE Trans on Mobile Computing, 2019, DOI: 10.1109/TMC.2019.2921713

16. Asheralieva A, Niyato D. Hierarchical game-theoretic and reinforcement learning framework for computational offloading in UAV-enabled mobile edge computing networks with multiple service providers. IEEE Internet of Things Journal, 2019, 6(5): 8753-8769

17. Chen X, Liu Z Y, Chen Y, et al. Mobile edge computing based task offloading and resource allocation in 5G ultra-dense networks. IEEE Access, 2019, 7: 184172-184182

18. Xie R C, Tang Q Q, Liang C H, et al. Dynamic computation offloading in IoT fog systems with imperfect channel-state information: a POMDP approach. IEEE Internet of Things Journal, 2021, 8(1): 345-356

19. Van Le D, Tham C. Quality of service aware computation offloading in an ad-hoc mobile cloud. IEEE Trans on Vehicular Technology, 2018, 67(9): 8890-8904

20. Guo H Z, Liu J J, Qin H L. Collaborative mobile edge computation offloading for IoT over fiber-wireless networks. IEEE Network, 2018, 32(1): 66-71

21. Fu F W, van der Schaar M. Learning to compete for resources in wireless stochastic games. IEEE Trans on Vehicular Technology, 2009, 58(4): 1904-1919

22. Bao X C, Liang H, Liu Y, et al. A stochastic game approach for collaborative beamforming in SDN-based energy harvesting wireless sensor networks. IEEE Internet of Things Journal, 2019, 6(6): 9583-9595

23. Wei Z L, Zhao B K, Su J S, et al. Dynamic edge computation offloading for internet of things with energy harvesting: a learning method. IEEE Internet of Things Journal, 2019, 6(3): 4436-4447

24. Wang K L, Tan Y Y, Shao Z Y, et al. Learning-based task offloading for delay-sensitive applications in dynamic fog networks. IEEE Trans on Vehicular Technology, 2019, 68(11): 11399-11403

25. Chen L X, Zhou S, Xu J. Computation peer offloading for energy-constrained mobile edge computing in small-cell networks. IEEE/ACM Trans on Networking, 2018, 26(4): 1619-1632

26. Zhang H J, Duan Y N, Long K P, et al. Energy Efficient resource allocation in terahertz downlink NOMA systems. IEEE Trans on Communications, 2021, 69(2): 1375-1384

27. Zheng J C, Cai Y M, Wu Y, et al. Dynamic computation offloading for mobile cloud computing: a stochastic game-theoretic approach. IEEE Trans on Mobile Computing, 2019, 18(4): 771-786

28. Zhang L, Cao B. Stochastic programming method for offloading in mobile edge computing based internet of vehicle. Proceedings of the 2019 IEEE International Conference on Communications (ICC’19), May 20-24, 2019, Shanghai, China. Piscataway, NJ, USA: IEEE, 2019: 1-6

29. Wang B B, Wu Y L, Ray Liu K J. Game theory for cognitive radio networks: an overview. Computer Networks, 2010, 54(14): 2537-2561

30. Chen X, Jiao L, Li W Z, et al. Efficient multi-user computation offloading for mobile-edge cloud computing. IEEE/ACM Trans on Networking, 2016, 24(5): 2795-2808

31. Sun Y X, Zhou S, Xu J. EMM: energy-aware mobility management for mobile edge computing in ultra-dense networks. IEEE Journal on Selected Areas in Communications, 2017, 35(11): 2637-2646

32. Huang Y F, Tan T H, Wang N C, et al. Resource allocation for D2D communications with a novel distributed Q-learning algorithm in heterogeneous networks. Proceedings of the 2018 International Conference on Machine Learning and Cybernetics (ICMLC’18): Vol 2, Jul 15-18, 2018, Chengdu, China. Piscataway, NJ, USA: IEEE, 2018: 533-537

33. Dab B, Aitsaadi N, Langar R. Q-learning algorithm for joint computation offloading and resource allocation in edge cloud. Proceedings of the 2019 IFIP/IEEE Symposium on Integrated Network and Service Management (IM’19), Apr 8-12, 2019, Arlington, VA, USA. Piscataway, NJ, USA: IEEE, 2019: 45-52

34. Hu J L, Wellman M P. Nash Q-learning for general-sum stochastic games. Journal of Machine Learning Research, 2003, 4: 1039-1069

35. Shams F, Bacci G, Luise M. Energy-efficient power control for multiple-relay cooperative networks using Q-learning. IEEE Trans on Wireless Communications, 2015, 14(3): 1567-1580

36. Ke H C, Wang J, Wang H, et al. Joint optimization of data offloading and resource allocation with renewable energy aware for IoT devices: a deep reinforcement learning approach. IEEE Access, 2019, 7: 179349-179363

37. Guo K, Yang M C, Zhang Y B, et al. An efficient dynamic offloading approach based on optimization technique for mobile edge computing. Proceedings of the 6th IEEE International Conference on Mobile Cloud Computing, Services, and Engineering (MobileCloud’18), Mar 26-29, 2018, Bamberg, Germany. Piscataway, NJ, USA: IEEE, 2018: 29-36

38. Xiao L, Xie C X, Chen T H, et al. A mobile offloading game against smart attacks. IEEE Access, 2016, 4: 2281-2291

39. Liu X L, Yu J D, Wang J, et al. Resource allocation with edge computing in IoT networks via machine learning. IEEE Internet of Things Journal, 2020, 7(4): 3415-3426

Metrics

Comments

Copyright © 2020 The Journal of China Universities of Posts and Telecommunications
　 Adress: P.O. Box 231,Beijing University of Posts and Telecommunications,10 Xi Tucheng Road,Beijing 100876,P.R.China　Post Code: 100081
Tel：86-010-62282493　Fax： 86-010-62283461　E-mail: jchupt@bupt.edu.cn
Support by: Beijing Magtech Co.Ltd

[1]	Yang Yujia, Liu Yiming, Zhang Wenjia, Zhang Zhi. SNR-adaptive deep joint source-channel coding scheme for imagesemantic transmission with convolutional block attention module [J]. The Journal of China Universities of Posts and Telecommunications, 2024, 31(1): 1-11.
[2]	Chai Rong, Duan Xiaofang, Wang Lixuan. GNN-based temporal knowledge reasoning for UAV mission planning systems [J]. The Journal of China Universities of Posts and Telecommunications, 2024, 31(1): 12-25.
[3]	Ren Chao, He Zongrui, Sun Chen, Li Haojin, Zhang Haijun. Wireless semantic communication based on semantic matching multiple access and intent bias multiplexing [J]. The Journal of China Universities of Posts and Telecommunications, 2024, 31(1): 26-36.
[4]	Li Linpei, Zhao Chuan, Su Yu, Huo Jiahao, Huang Yao, Li Haojin. Energy-efficient computation offloading assisted by RIS-based UAV [J]. The Journal of China Universities of Posts and Telecommunications, 2024, 31(1): 37-48.
[5]	Kang Xiaofei, Wang Tian, Liang Xian. Intelligent reflecting surfaces-assisted millimeter wave communication: Channel estimation based on deep learning [J]. The Journal of China Universities of Posts and Telecommunications, 2024, 31(1): 49-56.
[6]	Zheng Guangming, Zhou Tianle, Lu Haiwei, Long Yifei, Bai Jing. Study on high-order frequency selective surface with interdigital capacitance loading [J]. The Journal of China Universities of Posts and Telecommunications, 2024, 31(1): 57-63.
[7]	Cai Xiumei, He Ningning, Wu Chengmao, Liu Xiao, Liu Hang. Fractional order distance regularized level set method with bias correction [J]. The Journal of China Universities of Posts and Telecommunications, 2024, 31(1): 64-82.
[8]	Cheng Yi, Zhao Yan, Yin Peiwen. Radar false alarm plots elimination based on multi-feature extraction and classification [J]. The Journal of China Universities of Posts and Telecommunications, 2024, 31(1): 83-92.
[9]	Zhang Xiaojiao, Wu Xiang. Distributed consensus of Lurie multi-agent systems under directed topology: a contraction approach [J]. The Journal of China Universities of Posts and Telecommunications, 2023, 30(6): 11-21.
[10]	Wu Yue, Chen Xiangyong, Qiu Jianlong, Hu Shunwei, Zhao Feng. Dynamic event-triggered leader-follower consensus of nonlinear multi-agent systems under directed weighted topology [J]. The Journal of China Universities of Posts and Telecommunications, 2023, 30(6): 3-10.
[11]	Lv Pengchao, Huang Junjie, Liu Bo. Linear-quadratic optimal control for time-varying descriptor systems via space decompositions [J]. The Journal of China Universities of Posts and Telecommunications, 2023, 30(6): 38-48.
[12]	Xu Xingtao, Tao Jiagui. Parameter optimization of complex network based on the change-point identification [J]. The Journal of China Universities of Posts and Telecommunications, 2023, 30(6): 22-29.
[13]	Cheng Zunshui, Jiang Jingna, Sun Dongsheng. Stability and Hopf bifurcation analysis in DCTCP congestion control [J]. The Journal of China Universities of Posts and Telecommunications, 2023, 30(6): 30-37.
[14]	Liang Xiaolin, Ma Jiaxu, Cao Wangbin, Xu Jianpeng, Liu Shuaiqi, Zhao Xiongwen. Characteristics and modeling of UAV-vehicle MIMO wideband channels [J]. The Journal of China Universities of Posts and Telecommunications, 2023, 30(6): 60-67.
[15]	Zhang Sicong, Dai Jianzhuo, Huang Wenjing, Mi Xinping. Behavioral finance between the spot and futures markets based on multilayer network [J]. The Journal of China Universities of Posts and Telecommunications, 2023, 30(6): 82-88.

Dynamic computation offloading in time-varying environment for ultra-dense networks: a stochastic game approach

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics

Comments