Reinforced virtual optical network embedding algorithm in EONs for edge computing

doi:10.19682/j.cnki.1005-8885.2022.1020

The Journal of China Universities of Posts and Telecommunications ›› 2022, Vol. 29 ›› Issue (6): 18-29.doi: 10.19682/j.cnki.1005-8885.2022.1020

Previous Articles Next Articles

Reinforced virtual optical network embedding algorithm in EONs for edge computing

Zhu Ruijie, Li Gong, Wang Peisen, Zhang Wenchao

1. School of Computer and Artificial Intelligence, Zhengzhou University, Zhengzhou 450001, China 2. Henan Institute of Advanced Technology, Zhengzhou University, Zhengzhou 450002, China

Received:2022-07-21 Revised:2022-10-18 Online:2022-12-30 Published:2022-12-30
Contact: Zhu Ruijie E-mail:zhuruijie@zzu.edu.cn
Supported by:
This work was supported in part by the National Natural Science Foundation of China (62001422), and Henan Scientific and Technology Innovation Talents (22HASTIT016).

Abstract

Abstract: As the core technology of optical networks virtualization, virtual optical network embedding ( VONE) enables multiple virtual network requests to share substrate elastic optical network ( EON) resources simultaneously and hence has been applicated in edge computing scenarios. In this paper, we propose a reinforced virtual optical network embedding ( R-VONE ) algorithm based on deep reinforcement learning ( DRL) to optimize network embedding policies automatically. The network resource attributes are extracted as the environment state for model training, based on which DRL agent can deduce the node embedding probability. Experimental results indicate that R-VONE presents a significant advantage with lower blocking probability and higher resource utilization.

Key words: elastic optical network (EON), deep reinforcement learning (DRL), virtual optical network embedding (VONE), edge computing

CLC Number:

TP393.0

Zhu Ruijie, Li Gong, Wang Peisen, Zhang Wenchao. Reinforced virtual optical network embedding algorithm in EONs for edge computing[J]. The Journal of China Universities of Posts and Telecommunications, 2022, 29(6): 18-29.

References

1. ZHANG Y, WEI H Y. Risk-aware cloud-edge computing framework for delay-sensitive industrial IoTs. IEEE Transactions on Network and Service Management, 2021, 18(3): 2659-2671

2. SHIH Y Y, LIN H P, PANG A C, et al. An NFV-based service framework for IoT applications in edge computing environments. IEEE Transactions on Network and Service Management, 2019, 16(4): 1419-1434

3. ZHU R J, LI S H, WANG P S, et al. Energy-efficient deep reinforced traffic grooming in elastic optical networks for cloud–fog computing. IEEE Internet of Things Journal, 2021, 8(15): 12410-12421

4. CAO H T, XIAO A L, HU Y, et al. On virtual resource allocation of heterogeneous networks in virtualization environment: A service oriented perspective. IEEE Transactions on Network Science and Engineering, 2020, 7(4): 2468-2480

5. LI B, HE Q, CUI G M, et al. READ: Robustness-oriented edge application deployment in edge computing environment. IEEE Transactions on Services Computing, 2022, 15(3): 1746-1759

6. YU A, YANG H, NGUYEN K K, et al. Burst traffic scheduling for hybrid E/O switching DCN: An error feedback spiking neural network approach. IEEE Transactions on Network and Service Management, 2021, 18(1):882-893

7. TAEB S, SHAHRIAR N, CHOWDHURY S R, et al. Virtual network embedding with path-based latency guarantees in elastic optical networks. Proceedings of the IEEE 27th International Conference on Network Protocols (ICNP’19), 2019, Oct 8-10, Chicago, CA, USA. Piscataway, NJ, USA: IEEE, 2019: 12p

8. ZHU M, SUN Q, ZHANG S Y, et al. Energy-aware virtual optical network embedding in sliceable-transponder-enabled elastic optical networks. IEEE Access, 2019, 7: 41897-41912

9. NONDE L, EL-GORASHI T E H, ELMIRGHANI J M H. Energy efficient virtual network embedding for cloud networks. IEEE Journal of Lightwave Technology, 2015, 33(9): 1828-1849

10. SHAHRIAR N, TAEB S, CHOWDHURY S R, et al. Achieving a fully-flexible virtual network embedding in elastic optical networks. Proceedings of the 2019 IEEE Conference on Computer Communications (INFOCOM’19), 2019, Apr 29-May 2, Paris, France. Piscataway, NJ, USA: IEEE, 2019: 1756-1764

11. CHAI R, XIE D S, LUO L, et al. Multi-objective optimization based virtual network embedding algorithm for software-defined networking. IEEE Transactions on Network and Service Management. 2020, 17(1): 532-546

12. ZONG Y, OU Y N, HAMMAD A, et al. Location-aware energy efficient virtual network embedding in software-defined optical data center networks. IEEE/OSA Journal of Optical Communications and Networking, 2018, 10(7): 58-70

13. WANG J J, JIANG C X, ZHANG H J, et al. Thirty years of machine learning: The road to Pareto-optimal wireless networks. IEEE Communications Surveys & Tutorials, 2020, 22(3): 1472-1514

14. WU H T, ZHOU F, CHEN Y J, et al. On virtual network embedding: Paths and cycles. IEEE Transactions on Network and Service Management, 2020, 17(3): 1487-1500

15. YANG H, YU A, ZHANG J, et al. Data-driven network slicing from core to RAN for 5G broadcasting services. IEEE Transactions on Broadcasting, 2021, 67(1): 23-32

16. YANG H, ZHAO X D, YAO Q Y, et al. Accurate fault location using deep neural evolution network in cloud data center interconnection. IEEE Transactions on Cloud Computing, 2022, 10(2): 1402-1412

17. ZHU R J, SAMUEL A, WANG P S, et al. Survival multipath energy-aware resource allocation in SDM-EONs during fluctuating traffic. Journal of Lightwave Technology, 2021, 39(7): 1900-1912

18. WANG J J, JIANG C X, ZHANG H J, et al. Learning-aided network association for hybrid indoor LiFi-WiFi systems. IEEE Transactions on Vehicular Technology, 2018, 67(4): 3561-3574

19. ZHU R J, SAMUEL A, WANG P S, et al. Protected resource allocation in space division multiplexing elastic optical networks with fluctuating traffic. Journal of Network and Computer Applications, 2021, 174: Article 102887/1-14

20. CHEN X L, LI B J, PROIETTI R, et al. DeepRMSA: A deep reinforcement learning framework for routing, modulation and spectrum assignment in elastic optical networks. Journal of Lightwave Technology, 2019, 37(16): 4155-4163

21. YAN Z X, GE J G, WU Y L, et al. Automatic virtual network embedding: A deep reinforcement learning approach with graph convolutional networks. IEEE Journal on Selected Areas in Communications, 2020, 38(6): 1040-1057

22. ZHANG P Y, YAO H P, LIU Y J. Virtual network embedding based on computing, network, and storage resource constraints. IEEE Internet of Things Journal, 2018, 5(5): 3298-3304

23. HAERI S, TRAJKOVIC L. Virtual network embedding via Monte Carlo tree search. IEEE Transactions on Cybernetics, 2018, 48(2): 510-521

24. ZHU R J, LI S S, WANG P S, et al. Time and spectrum fragmentation aware virtual optical network embedding in elastic optical networks. Optical Fiber Technology, 2020, 54: Article 102117

25. WEI W T, GU H X, WANG K, et al. Improving cloud-based IoT services through virtual network embedding in elastic optical inter-dc networks. IEEE Internet of Things Journal, 2019, 6(1): 986-996

26. HOU W G, NING Z L, GUO L, et al. Novel framework of risk-aware virtual network embedding in optical data center networks. IEEE Systems Journal, 2018, 12(3): 2473-2482

27. CAO H T, YANG L X, ZHU H B. Novel node-ranking approach and multiple topology attributes-based embedding algorithm for single-domain virtual network embedding. IEEE Internet of Things Journal, 2018, 5(1): 108-120

28. ZHAO C G, PARHAMI B. Virtual network embedding through graph eigenspace alignment. IEEE Transactions on Network and Service Management, 2019, 16(2): 632-646

29. LI M Y, CHEN C L, HUA C Q, et al. Intelligent latency-aware virtual network embedding for industrial wireless networks. IEEE Internet of Things Journal, 2019, 6(5): 7484-7496

30. BLENK A, KALMBACH P, ZERWAS J, et al. NeuroViNE: A neural preprocessor for your virtual network embedding algorithm. Proceedings of the 2018 IEEE Conference on Computer Communications (INFOCOM’18), 2018, Apr 16-19, Honolulu, HI, USA. Piscataway, NJ, USA: IEEE, 2018: 405-413

31. CHEN X L, PROIETTI R, LIU C Y, et al. A multi-task-learning-based transfer deep reinforcement learning design for autonomic optical networks. IEEE Journal on Selected Areas in Communications, 2021, 39(9): 2878-2889

32. LIU W X. Intelligent routing based on deep reinforcement learning in software-defined data-center networks. Proceedings of the 2019 IEEE Symposium on Computers and Communications (ISCC’19), 2019, Jun 29-Jul 3, Barcelona, Spain. Piscataway, NJ, USA: IEEE, 2019: 6p

33. YAO H P, ZHANG B, ZHANG P Y, et al. RDAM: A reinforcement learning based dynamic attribute matrix representation for virtual network embedding. IEEE Transactions on Emerging Topics in Computing, 2021, 9(2): 901-914

34. DOLATI M, HASSANPOUR S B, GHADERI M, et al. DeepViNE: Virtual network embedding with deep reinforcement learning. Proceedings of the 2019 IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS’19), 2019, Apr 29-May 2, Paris, France. Piscataway, NJ, USA: IEEE, 2019: 879-885

35. ZHANG X, LI B J, PENG J Q, et al. You calculate and I provision: A DRL-assisted service framework to realize distributed and tenant-driven virtual network slicing. Journal of Lightwave Technology, 2021, 39(1): 4-16

36. YAO H P, CHEN X, LI M Z, et al. A novel reinforcement learning algorithm for virtual network embedding. Neurocomputing, 2018, 284: 1-9

37. CHIANG Y, CHAO Y H, HSU C H, et al. Virtual network embedding with dynamic speed switching orchestration in fog/edge network. IEEE Access, 2020, 8: 84753-84768

38. CHEN Q, JIANG Y F, LEI Y, et al. A profit-maximized approach based on link importance degree for virtual optical networks mapping. Proceedings of the 2019 Asia Communications and Photonics Conference (ACP’19), 2019, Nov 2-5, Chengdu, China. Piscataway, NJ, USA: IEEE, 2019: 3p

39. FU X Y, YU F R, WANG J Y, et al. Dynamic service function chain embedding for NFV-enabled IoT: A deep reinforcement learning approach. IEEE Transactions on Wireless Communications, 2020, 19(1): 507-519

Metrics

Comments

Copyright © 2020 The Journal of China Universities of Posts and Telecommunications
　 Adress: P.O. Box 231,Beijing University of Posts and Telecommunications,10 Xi Tucheng Road,Beijing 100876,P.R.China　Post Code: 100081
Tel：86-010-62282493　Fax： 86-010-62283461　E-mail: jchupt@bupt.edu.cn
Support by: Beijing Magtech Co.Ltd

[1]	Zhang Luying, Liu Xiaokai, Li Zhao, Xu Fangmin, Zhao Chenglin. Prediction based dynamic resource allocation method for edge computing first networking [J]. The Journal of China Universities of Posts and Telecommunications, 2023, 30(3): 78-87.
[2]	Liu Kun, Wang Hui , Shen Zihao. Prediction of network attack profit path based on NAPG model [J]. The Journal of China Universities of Posts and Telecommunications, 2020, 27(5): 91-102.
[3]	Tan Xiaoying, Huang Dan, Guo Yuchun, Chen Changjia. Dynamic resource allocation in cloud download service [J]. JOURNAL OF CHINA UNIVERSITIES OF POSTS AND TELECOM, 2017, 24(5): 53-59.
[4]	. Road traffic and geography topology based opportunistic routing for VANETs [J]. Acta Metallurgica Sinica(English letters), 2014, 21(4): 32-39.
[5]	. Novel MAC protocol for terahertz ultra-high data-rate wireless networks [J]. Acta Metallurgica Sinica(English letters), 2013, 20(6): 69-76.
[6]	WANG Yang, WANG Ru-chuan, CHENG Xi. Improved multipath routing with WNN for the open networks [J]. Acta Metallurgica Sinica(English letters), 2008, 15(2): 107-113.
[7]	. KFDA and clustering based multiclass SVM for intrusion detection [J]. Acta Metallurgica Sinica(English letters), 2008, 15(1): 123-128.
[8]	HU Lan-lan;NIU Xin-xin;YANG Yi-xian . Weaknesses and improvements of a remote user authentication scheme using smart cards [J]. Acta Metallurgica Sinica(English letters), 2007, 14(3): 91-94.
[9]	WANG Hai-yan; WANG Ru-chuan. CPK-based grid authentication: a step forward [J]. Acta Metallurgica Sinica(English letters), 2007, 14(1): 26-31.
[10]	LI Chao;YANG Yi-xian;NIU Xin-xin. Biometric-based personal identity-authentication system and security analysis [J]. Acta Metallurgica Sinica(English letters), 2006, 13(4): 43-47.
[11]	LI Ling-juan;SHEN Ling-tong. An improved multilevel fuzzy comprehensive evaluation algorithm for security performance [J]. Acta Metallurgica Sinica(English letters), 2006, 13(4): 48-53.
[12]	YANG Geng;RONG Chun-ming;VEIGNER Christian;WANG Jiang-tao;CHENG Hong-bing. Identity-based key agreement and encryption for wireless sensor networks [J]. Acta Metallurgica Sinica(English letters), 2006, 13(4): 54-60.
[13]	ZHANG Jian-feng;WANG Ru-chuan. An agent-based incentive mechanism for P2P systems [J]. Acta Metallurgica Sinica(English letters), 2006, 13(4): 81-85.
[14]	JIANG Ling-yun;WANG Ru-chuan;WANG Hai-yan. Code generation framework for grid development [J]. Acta Metallurgica Sinica(English letters), 2006, 13(2): 39-42.
[15]	CHEN Jian-gang, WANG Ru-chuan, WANG Hai-yan. The extended RBAC model based on grid computing [J]. Acta Metallurgica Sinica(English letters), 2006, 13(3): 93-97.

Reinforced virtual optical network embedding algorithm in EONs for edge computing

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics

Comments