Resource allocation and hybrid prediction scheme for low-latency visual feedbacks to support tactile Internet multimodal perceptions

doi:10.19682/j.cnki.1005-8885.2021.2002

The Journal of China Universities of Posts and Telecommunications ›› 2021, Vol. 28 ›› Issue (4): 13-28.doi: 10.19682/j.cnki.1005-8885.2021.2002

• Artificial intelligence • Previous Articles Next Articles

Resource allocation and hybrid prediction scheme for low-latency visual feedbacks to support tactile Internet multimodal perceptions

Kang Mancong, Li Xi, Ji Hong, Zhang Heli

School of Telecommunication Engineering, Beijing University of Posts and Telecommunications, Beijing 100876, China

Received:2020-08-25 Revised:2021-06-07 Accepted:2021-07-29 Online:2021-08-31 Published:2021-10-11
Contact: Corresponding author: Li Xi, E-mail: lixi@bupt.edu.cn E-mail:lixi@bupt.edu.cn
Supported by:
This work was supported by the National Natural Science Foundation of China (61771070).

Abstract

Abstract: Predicting user states in future and rendering visual feedbacks accordingly can effectively reduce the visual experienced delay in the tactile Internet (TI). However, most works omit the fact that different parts in an image may have distinct prediction requirements, based on which different prediction models can be used in the predicting process, and then it can further improve predicting quality especially under resources-limited environment. In this paper, a hybrid prediction scheme is proposed for the visual feedbacks in a typical TI scenario with mixed visuo-haptic interactions, in which haptic traffic needs sufficient wireless resources to meet its stringent communication requirement, leaving less radio resources for the visual feedback. First, the minimum required number of radio resources for haptic traffic is derived based on the haptic communication requirements, and wireless resources are allocated to the haptic and visual traffics afterwards. Then, a grouping strategy is designed based on the deep neural network (DNN) to allocate different parts from an image feedback into two groups to use different prediction models, which jointly considers the prediction deviation thresholds, latency and reliability requirements, and the bit sizes of different image parts. Simulations show that, the hybrid prediction scheme can further reduce the visual experienced delay under haptic traffic requirements compared with existing strategies.

Key words: tactile Internet, ultra-reliable and low-latency communications, enhanced mobile broadband, visuo-haptic feedbacks, layered rendering

CLC Number:

TN92

Kang Mancong, Li Xi, Ji Hong, Zhang Heli. Resource allocation and hybrid prediction scheme for low-latency visual feedbacks to support tactile Internet multimodal perceptions[J]. The Journal of China Universities of Posts and Telecommunications, 2021, 28(4): 13-28.

References

1. Aijaz A, Sooriyabandara M. The tactile Internet for industries: a review. Proceedings of the IEEE, 2019, 107(2): 414 -435
2. Maier M, Ebrahimzadeh A, Chowdhury M. The tactile Internet: automation or augmentation of the human. IEEE Access, 2018, 6: 41607 -41618
3. Popovski P, Trillingsgaard K F, Simeone O, et al. 5G wireless network slicing for eMBB, URLLC, and mMTC: a communication-theoretic view. IEEE Access, 2018, 6: 55765 -55779
4. Chang C S, Thomas J A. Effective bandwidth in highspeed digital networks. IEEE Journal on Selected Areas in Communications, 1995, 13(6): 1091 -1100
5. Holland O, Steinbach E, Prasad R V, et al. The IEEE 1918. 1 'tactile internet' standards working group and its standards. Proceedings of the IEEE, 2019, 107(2): 256 –279
6. Lai Z, Hu Y C, Cui Y, et al. Furion: engineering high-quality immersive virtual reality on today's mobile devices. IEEE
Transactions on Mobile Computing, 2020, 19(7): 1586 -1602
7. Guo F X, Yu F R, Zhang H L, et al. An adaptive wireless virtual reality framework in future wireless networks: a distributed learning approach. IEEE Transactions on Vehicular Technology, 2020, 69(8): 8514 -8528
8. Liu Y M, Yu F R, Li X, et al. Decentralized resource allocation for video transcoding and delivery in blockchain-based system with mobile edge computing. IEEE Transactions on Vehicular Technology, 2019, 68(11): 11169 -11185
9. Feng X L, Bao Z Y, Wei S. Exploring CNN-based viewport prediction for live virtual reality streaming. Proceedings of the 2019 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR'19), 2018, Dec 9 -11, San Diego, CA, USA. Piscataway, NJ, USA: IEEE, 2019: 183 -186
10. Vielhaben J, Camalan H, Samek W, et al. Viewport forecasting in 360-degree virtual reality videos with machine learning. Proceedings of the 2019 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR'19), 2019, Dec 9 -11, San Diego, CA, USA. Piscataway, NJ, USA: IEEE, 2019: 740 -747
11. Pohl S, Becher A, Grauschopf T, et al. Neural network 3D body pose tracking and prediction for motion-to-photon latency compensation in distributed virtual reality. Proceedings of the 2019 International Conference on Artificial Neural Networks and Machine Learning (ICANN'19), 2019, Sep 17 -19, Munich, Germany. LNCS 11729. Berlin, Germany: Springe, 2019: 429 -442

12. Aykut T, Xu J, Steinbach E. Realtime 3D 360-degree telepresence with deep-learning-based head-motion prediction. IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 2019, 9(1): 231 -244

13. Wu E W, Koike H. Future pose-mixed reality martial arts training using real-time 3D human pose forecasting with a RGB camera. Proceedings of the 2019 IEEE Winter Conference on Applications of Computer Vision (WACV'19), 2019, Jan 7 -11, Waikoloa, HI, USA. Piscataway, NJ, USA: IEEE, 2019: 1384 -1392

14. Zhang Z L, Ma Z H, Sun Y, et al. Wireless multicast of virtual reality videos with MPEG - I format. IEEE Access, 2019, 7: 176693 -176705
15. Vega M T, van der Hooft J, Heyse J, et al. Exploring new york in 8k: an adaptive tile-based virtual reality video streaming experience. Proceedings of the 10th ACM Multimedia Systems Conference (MMSys'19), 2019, Jun 18 -21, Amherst, MA, USA. New York, NJ, USA: ACM, 2019: 330 -333
16. Smirnov S, Battisti F, Gotchev A. Layered approach for improving the quality of free-viewpoint depth-image-based rendering images. Journal of Electronic Imaging, 2019, 28(1): 1 -17
17. Hou Z W, She C, Li Y H, et al. Prediction and communication co-design for ultra-reliable and low-latency communications. IEEE Transactions on Wireless Communications, 2020, 19(2): 1196 -1209
18. She C Y, Dong R, Gu Z Y, et al. Deep learning for ultra-reliable and low-latency communications in 6G networks. IEEE Network, 2020, 34(5): 219 -225
19. Ateya A A, Vybornova A, Muthanna A, et al. Key solutions for light limitations: toward tactile Internet system realization. Proceedings of the 2nd International Conference on Future Networks and Distributed Systems (ICFNDS'18), 2018, Jun 26 -27, Amman, Jordan. New York, NY, USA: ACM, 2018: 1 -6
20. Cox J, Wong K. Predictive feedback augmentation for manual control of an unmanned aerial vehicle with latency. International Journal of Micro Air Vehicles, 2019, 11

21. Axenie C, Pohl S, Becher A, et al. Dataset for neural network 3D body pose tracking and prediction for motion-to-photon latency compensation in distributed virtual reality (Version 1.0). http://zenodo.org/record/3716093#.YSN7e3DDX7F, 2020-03-19
22. Park J, Bennis M. URLLC-eMBB slicing to support VR multimodal perceptions over wireless cellular systems. Proceedings of the 2018 IEEE Global Communications Conference (GLOBECOM'18), 2018, Dec 9 -13, Abu Dhabi, United Arab Emirates. Piscataway, NJ, USA: IEEE, 2018: 1 -7
23. Yu S, Wang X, Langar R. Computation offloading for mobile edge computing: a deep learning approach. Proceedings of the IEEE 28th Annual International Symposium on Personal, Indoor, and Mobile Radio Communications (PIMRC'17), Oct 8 -13, Montreal, Canada. Piscataway, NJ, USA: IEEE, 2017: 1 -6

Metrics

Comments

Copyright © 2020 The Journal of China Universities of Posts and Telecommunications
　 Adress: P.O. Box 231,Beijing University of Posts and Telecommunications,10 Xi Tucheng Road,Beijing 100876,P.R.China　Post Code: 100081
Tel：86-010-62282493　Fax： 86-010-62283461　E-mail: jchupt@bupt.edu.cn
Support by: Beijing Magtech Co.Ltd

[1]	Zhang Ping, Xu Xiaodong, Dong Chen, Han Shujun, Wang Bizhu. Intellicise communication system: model-driven semantic communications [J]. The Journal of China Universities of Posts and Telecommunications, 2022, 29(1): 2-12.
[2]	Zhang Zezhong, Chen Mingzhe, Xu Jie, Cui Shuguang. Air interface design for edge intelligence [J]. The Journal of China Universities of Posts and Telecommunications, 2022, 29(1): 13-26.
[3]	Liu Guangyi, Deng Juan, Zheng Qingbi, Li Gang, Sun Xin, Huang Yuhong. Native intelligence for 6G mobile network: technical challenges, architecture and key features [J]. The Journal of China Universities of Posts and Telecommunications, 2022, 29(1): 27-40.
[4]	Sun Junshuai, Zhu Xinghui, Xiao Yeqiu, Cheng Ke, Zhao Shuangrui. Adaptive TTI bundling with self-healing scheme for 5G [J]. The Journal of China Universities of Posts and Telecommunications, 2022, 29(1): 64-70.
[5]	Guo Hui, Zhao Xuehui. Maximum throughput design for IRS aided WPCN system based on NOMA [J]. The Journal of China Universities of Posts and Telecommunications, 2022, 29(1): 93-101.
[6]	Song Xin, Huang Xue, Gao Yiming, Qian Haijun. Energy-efficient power allocation for NOMA heterogeneous networks with imperfect CSI [J]. The Journal of China Universities of Posts and Telecommunications, 2022, 29(1): 102-112.
[7]	Wang Hang, Li Xi, Ji Hong, Zhang Heli. QoE-based video segments caching strategy in urban public transportation system [J]. The Journal of China Universities of Posts and Telecommunications, 2021, 28(4): 29-38.
[8]	Li Zongyan, Li Jiahui, Yu Honglu, Li Shiyin. New spatial quadrature modulation scheme for indoor visible light communication systems [J]. The Journal of China Universities of Posts and Telecommunications, 2021, 28(2): 89-96.
[9]	Zhou Yutong, Li Xi, Ji Hong, Zhang Heli. Blockchain-based collaborative edge caching scheme for trustworthy content sharing [J]. The Journal of China Universities of Posts and Telecommunications, 2021, 28(2): 38-47.
[10]	Liu Xu, Xie Yang. Channel estimation for multi-panel millimeter wave MIMO based on joint compressed sensing [J]. The Journal of China Universities of Posts and Telecommunications, 2020, 27(6): 1-7.
[11]	Zheng Lin, Wang Zhen, Chen Jianmei, Lin Mengying, Deng Xiaofang. MIMO-FSK non-coherent detection with spatial multiplexing in fast-fading environment [J]. The Journal of China Universities of Posts and Telecommunications, 2020, 27(5): 47-54.
[12]	Zhang Yongchang. Game-Based Distributed Noncooperation Interference Coordination Scheme in Ultra-Dense Networks [J]. The Journal of China Universities of Posts and Telecommunications, 2020, 27(5): 55-62.
[13]	Li Xinmin, Li Guomin, Liu Yang, Guo Tian, Li Pu, Li Yaru. Low-complexity transmit antenna selection algorithm for massive MIMO [J]. The Journal of China Universities of Posts and Telecommunications, 2020, 27(5): 63-68.
[14]	Li Shaoxian, Lv Fan, Wang Chengrui, Hou Yanzhao. UWB positioning enhancement using Markov chain in indoor NLOS environment [J]. The Journal of China Universities of Posts and Telecommunications, 2020, 27(4): 54-58.
[15]	Yao Yindi, Wang Lei, He Junjin. Hybrid time slot allocation algorithm based on LoRa Internet of things [J]. The Journal of China Universities of Posts and Telecommunications, 2020, 27(4): 83-90.

Resource allocation and hybrid prediction scheme for low-latency visual feedbacks to support tactile Internet multimodal perceptions

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics

Comments