中国邮电高校学报(英文版) ›› 2022, Vol. 29 ›› Issue (1): 13-26.doi: 10.19682/j.cnki.1005-8885.2022.2003
Zhang Zezhong, Chen Mingzhe, Xu Jie, Cui Shuguang
收稿日期:2021-12-13
修回日期:2022-01-30
接受日期:2022-02-05
出版日期:2022-02-26
发布日期:2022-02-28
通讯作者:
Corresponding author: Xu Jie
E-mail:xujie@cuhk.edu.cn
Zhang Zezhong, Chen Mingzhe, Xu Jie, Cui Shuguang
Received:2021-12-13
Revised:2022-01-30
Accepted:2022-02-05
Online:2022-02-26
Published:2022-02-28
Contact:
Corresponding author: Xu Jie
E-mail:xujie@cuhk.edu.cn
摘要： Recent breakthroughs in artificial intelligence (AI) give rise to a plethora of intelligent applications and services based on machine learning algorithms such as deep neural networks (DNNs). With the proliferation of Internet of things (IoT) and mobile edge computing, these applications are being pushed to the network edge, thus enabling a new paradigm termed as edge intelligence. This provokes the demand for decentralized implementation of learning algorithms over edge networks to distill the intelligence from distributed data, and also calls for new communication-efficient designs in air interfaces to improve the privacy by avoiding raw data exchange. This paper provides a comprehensive overview on edge intelligence, by particularly focusing on two paradigms named edge learning and edge inference, as well as the corresponding communication-efficient solutions for their implementations in wireless systems. Several insightful theoretical results and design guidelines are also provided.
中图分类号:
Zhang Zezhong, Chen Mingzhe, Xu Jie, Cui Shuguang. Air interface design for edge intelligence[J]. The Journal of China Universities of Posts and Telecommunications, 2022, 29(1): 13-26.
| References [1] MAO Y, Y YOU C S, ZHANG J, et al. A survey on mobile edge computing: The communication perspective. IEEE Communications Surveys and Tutorials, 2017, 19(4): 2322 - 2358. [2] ZHOU Z, CHEN X, LI E, et al. Edge intelligence: Paving the last mile of artificial intelligence with edge computing. Proceedings of the IEEE, 2019, 107(8): 1738 - 1762. [3] SAAD W, BENNIS M, CHEN M Z. A vision of 6G wireless systems: Applications, trends, technologies, and open research problems. IEEE Network, 2020, 34(3): 134 - 142. [4] CHEN M Z, GUNDUZ D, HUANG K B, et al. Distributed learning in wireless networks: Recent progress and future challenges. IEEE Journal on Selected Areas in Communications, 2021, 39(12): 3579 - 3605. [5] PREDD J B, KULKARNI S B, POOR H V. Distributed learning in wireless sensor networks. IEEE Signal Processing Magazine, 2006, 23(4): 56 - 69. [6] MCMAHAN B, MOORE E, RAMAGE D, et al. Communication-efficient learning of deep networks from decentralized data. Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, 2017, Apr 20 - 22, Fort Lauderdale, FL, USA. New York, NY, USA: ACM, 2017: 1273 - 1282. [7] YANG Q, LIU Y, CHEN T J, et al. Federated machine learning: Concept and applications. ACM Transactions on Intelligent Systems and Technology, 2019, 10(2): Article 12. [8] ZHUO H H, FENG W F, LIN Y F, et al. Federated deep reinforcement learning. ArXiv: 1901. 08277, 2019. [9] FINN C, ABBEEL P, LEVINE S. Model-agnostic meta-learning for fast adaptation of deep networks. Proceedings of the 34th International Conference on Machine Learning (ICML'17), 2017, Aug 6 - 11, Sydney, Australia. New York, NY, USA: ACM, 2017: 1126 - 1135. [10] SMITH V, CHIANG C K, SANJABI M, et al. Federated multi-task learning. Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS'17), 2017, Dec 4 - 9, Long Beach, CA, USA. Red Hook, NY, USA: Curran Associates Inc, 2017: 4427 - 4437. [11] ABAD M S H, OZFATURA E, GUNDUZ D, et al. Hierarchical federated learning across heterogeneous cellular networks. Proceeding of the 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP'20), 2020, May 4 - 8, Barcelona, Spain. Piscataway, NJ, USA: IEEE, 2020: 8866 - 8870. [12] YANG H H, LIU Z Z, QUEK T Q S, et al. Scheduling policies for federated learning in wireless networks. IEEE Transactions on Communications, 2020, 68(1): 317 - 333. [13] CHEN M Z, SHLEZINGER N, POOR H V, et al. Communication-efficient federated learning. Proceedings of the National Academy of Sciences, 2021, 118(17): Article e2024789118. [14] STICH S U, CORDONNIER J B, JAGGI M. Sparsified SGD with memory. Proceeding of the 32nd International Conference on Neural Information Processing Systems (NIPS'18), 2018, Dec 2 - 8, Montreal, Canada. Red Hook, NY, USA: Curran Associates Inc, 2018: 4447 - 4458. [15] ZHU G G, WANG Y, HUANG K B. Broadband analog aggregation for low-latency federated edge learning. IEEE Transactions on Wireless Communications, 2020, 19(1): 491 - 506. [16] SHAO J W, ZHANG J. Communication-computation trade-off in resource-constrained edge inference. IEEE Communications Magazine, 2020, 58(12): 20 - 26. [17] SHAO J W, MAO Y Y, ZHANG J. Task-oriented communication for multi-device cooperative edge inference. ArXiv: 2109. 00172, 2021. [18] LAN Q, DU Y Q, POPOVSKI P, et al. Capacity of remote classification over wireless channels. IEEE Transactions on Communications, 2021, 69(7): 4489 - 4503. [19] BOYD S, GHOSH A, PRABHAKAR B, et al. Randomized gossip algorithms. IEEE Transactions on Information Theory, 2006, 52(6): 2508 - 2530. [20] ZHU G X, LIU D Z, DU Y Q, et al. Toward an intelligent edge: Wireless communication meets machine learning. IEEE Communications Magazine, 2020, 58(1): 19 - 25. [21] WANG F, XU J, WANG X, et al. Joint offloading and computing optimization in wireless powered mobile-edge computing systems. IEEE Transactions on Wireless Communications, 2018, 17(3): 1784 - 1797. [22] KIM H, PARK J, BENNIS M, et al. On-device federated learning via blockchain and its latency analysis. ArXiv: 1808. 03949, 2018. [23] BLOT M, PICARD D, CORD M, et al. Gossip training for deep learning. ArXiv: 1611. 09726, 2016. [24] JIN P H, YUAN Q C, IANDOLA F, et al. How to scale distributed deep learning? ArXiv: 1611. 04581, 2016. [25] DAILY J, VISHNU A, SIEGEL C, et al. GossipGraD: Scalable deep learning using gossip communication based asynchronous gradient descent. ArXiv: 1803. 05880, 2018. [26] LI X Y, ZHU G G, SHEN K M, et al. Joint annotator-and-spectrum allocation in wireless networks for crowd labeling. IEEE Transactions on Wireless Communications, 2020, 19(9): 6116 - 6129. [27] ZHAO H, JI F, GUAN Q S, et al. Federated meta learning enhanced acoustic radio cooperative framework for ocean of things underwater acoustic communications. ArXiv: 2105. 13296, 2021. [28] SEO H, PARK J, OH S, et al. Federated knowledge distillation. ArXiv: 2011. 02367, 2020. [29] ZHANG Z J, WANG S, HONG Y C, et al. Distributed dynamic map fusion via federated learning for intelligent networked vehicles. Proceeding of the 2021 IEEE International Conference on Robotics and Automation (ICRA'21), 2021, May 30 - Jun 5, Xi'an, China. Piscataway, NJ, USA: IEEE, 2021. [30] LIU Y, KANG Y, ZHANG X W, et al. A communication efficient collaborative learning framework for distributed features. ArXiv: 1912. 11187, 2019. [31] ZHANG Y R, WU Q R, SHIKH-BAHAEI M. Vertical federated learning based privacy-preserving cooperative sensing in cognitive radio networks. Proceeding of the 2020 IEEE Globecom Workshops (GC Wkshps'20), 2020, Dec 7 - 11, Taipei, China. Piscataway, NJ, USA: IEEE, 2020: 1 - 6. [32] FENG S W, YU H. Multi-participant multi-class vertical federated learning. ArXiv: 2001. 11154, 2020. [33] HU Y, CHEN M Z, SAAD W, et al. Distributed multi-agent meta learning for trajectory design in wireless drone networks. IEEE Journal on Selected Areas in Communications, 2021, 39(10): 3177 - 3192. [34] LIU D Z, ZHU G X, ZHANG J, et al. Data-importance aware user scheduling for communication-efficient edge machine learning. IEEE Transactions on Cognitive Communications and Networking, 2021, 7(1): 265 - 278. [35] ALISTARH D, HOEFLER T, JOHANSSON M, et al. The convergence of sparsified gradient methods. Proceeding of the 32nd International Conference on Neural Information Processing Systems (NIPS'18), 2018, Dec 2 - 8, Montreal, Canada. Red Hook, NY, USA: Curran Associates Inc, 2018: 5973 - 5983. [36] EGHLIDI N F, JAGGI M. Sparse communication for training deep networks. ArXiv: 2009. 09271, 2020. [37] OZFATURA E, OZFATURA K, GUNDUZ D. Time-correlated sparsification for communication-efficient federated learning. Proceeding of the 2021 IEEE International Symposium on Information Theory (ISIT'21), 2021, Jul 12 - 20, Melbourne, Australia. Piscataway, NJ, USA: IEEE, 2021. [38] WEN D Z, JEON K J, HUANG K B. Federated dropout-A simple approach for enabling federated learning on resource constrained devices. IEEE Wireless Communications Letters, 2022, Early Access Article. [39] ALISTARH D, GRUBIC D, LI J, et al. QSGD: Communication-efficient SGD via gradient quantization and encoding. Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS'17), 2017, Dec 4 - 9, Long Beach, CA, USA. Red Hook, NY, USA: Curran Associates Inc, 2017: 1709 - 1720. [40] DU Y Q, YANG S, HUANG K B. High-dimensional stochastic gradient quantization for communication-efficient edge learning. IEEE Transactions on Signal Processing, 2020, 68: 2128 - 2142. [41] CHEN M Z, YANG Z H, SAAD W, et al. A joint learning and communications framework for federated learning over wireless networks. IEEE Transactions on Wireless Communications, 2021, 20(1): 269 - 283. [42] BALAKRISHNAN R, AKDENIZ M, DHAKAL S, et al. Resource management and fairness for federated learning over wireless edge networks. Proceeding of the IEEE 21st International Workshop on Signal Processing Advances in Wireless Communications (SPAWC'20), 2020, May 26 - 29, Atlanta, GA, USA. Piscataway, NJ, USA: IEEE, 2020: 1 - 5. [43] SHI W Q, ZHOU S, NIU Z S, et al. Joint device scheduling and resource allocation for latency constrained wireless federated learning. IEEE Transactions on Wireless Communications, 2021, 20(1): 453 - 467. [44] ZHU G X, XU J, HUANG K B, et al. Over-the-air computing for wireless data aggregation in massive IoT. IEEE Wireless Communications, 2021, 28(4): 57 - 65. [45] CAO X W, ZHU G X, XU J, et al. Optimized power control design for over-the-air federated edge learning. IEEE Journal on Selected Areas in Communications, 2022, 40(1): 342 - 358. [46] CAO X W, ZHU G X, XU J, et al. Transmission power control for over-the-air federated averaging at network edge. IEEE Journal on Selected Areas in Communications, 2022, Early Access Article. [47] LI X Y, ZHU G X, GONG Y, et al. Wirelessly powered data aggregation for IoT via over-the-air function computation: Beamforming and power control. IEEE Transactions on Wireless Communications, 2019, 18(7): 3437 - 3452. [48] HU Y T, CHEN M, CHEN M Z, et al. Energy minimization for federated learning with IRS-assisted over-the-air computation. Proceeding of the 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP'21), 2021, Jun 6 - 11, Toronto, Canada. Piscataway, NJ, USA: IEEE, 2021: 3105 - 3109. [49] ZHAI X F, CHEN X H, XU J, et al. Hybrid beamforming for massive MIMO over-the-air computation. IEEE Transactions on Communications, 2021, 69(4): 2737 - 2751. [50] ZHU G X, DU Y Q, GUNDUZ D, et al. One-bit over-the-air aggregation for communication-efficient federated edge learning: Design and convergence analysis. IEEE Transactions on Wireless Communications, 2021, 20(3): 2120 - 2135. [51] ZHANG Z Z, ZHU G X, WANG R, et al. Turning channel noise into an accelerator for over-the-air principal component analysis. ArXiv: 2104. 10095, 2021. [52] XING H, SIMEONE O, BI S Z. Federated learning over wireless device-to-device networks: Algorithms and convergence analysis. IEEE Journal on Selected Areas in Communications, 2021, 39(12): 3723 - 3741. [53] CHEN M Z, POOR H V, SAAD W, et al. Wireless communications for collaborative federated learning. IEEE Communications Magazine, 2020, 58(12): 48 - 54. [54] CHENG Y, WANG D, ZHOU P, et al. Model compression and acceleration for deep neural networks: The principles, progress, and challenges. IEEE Signal Processing Magazine, 2018, 35(1): 126 - 136. [55] CAI H, GAN C, WANG T X, et al. Once-for-all: Train one network and specialize it for efficient deployment. Proceeding of the 8th International Conference on Learning Representations (ICLR'20), 2020, Apr 26 - 30, Addis Ababa, Ethiopia. 2020: 1 - 13. [56] YOU C S, HUANG K B, CHAE H, et al. Energy-efficient resource allocation for mobile-edge computation offloading. IEEE Transactions on Wireless Communications, 2017, 16(3): 1397 - 1411. [57] HUANG D, WANG P, NIYATO D. A dynamic offloading algorithm for mobile computing. IEEE Transactions on Wireless Communications, 2012, 11(6): 1991 - 1995. [58] KANG Y P, HAUSWALD J, GAO C, et al. Neurosurgeon: Collaborative intelligence between the cloud and mobile edge. ACM SIGARCH Computer Architecture News, 2017, 45(1): 615 - 629. [59] WEAVER W. Recent contributions to the mathematical theory of communication. ETC: A Review of General Semantics, 1953, 10(4): 261 - 281. [60] SHI G M, XIAO Y, LI Y Y, et al. From semantic communication to semantic-aware networking: Model, architecture, and open problems. IEEE Communications Magazine, 2021, 59(8): 44 - 50. [61] LAN Q, WEN D Z, ZHANG Z Z, et al. What is semantic communication? A view on conveying meaning in the era of machine intelligence. Journal of Communications and Information Networks, 2021, 6(4): 336 - 371. |
| 阅读次数 | ||||||
|
全文 |
|
|||||
|
摘要 |
|
|||||
