DRO-SLAM: Real-time object-aware SLAM for navigation robots and autonomous driving in dynamic environments

doi:10.19682/j.cnki.1005-8885.2022.1011

中国邮电高校学报(英文) ›› 2023, Vol. 30 ›› Issue (3): 14-24.doi: 10.19682/j.cnki.1005-8885.2022.1011

• Artificial Intelligence • 上一篇下一篇

DRO-SLAM: Real-time object-aware SLAM for navigation robots and autonomous driving in dynamic environments

Wang Zixian, Zhang Miao, Yan Danfeng

School of Computer Science, Beijing University of Posts and Telecommunications, Beijing 100876, China

收稿日期:2021-10-27 修回日期:2022-04-07 出版日期:2023-06-30 发布日期:2023-06-30
通讯作者: 闫丹凤 E-mail:yandf@bupt.edu.cn

DRO-SLAM: Real-time object-aware SLAM for navigation robots and autonomous driving in dynamic environments

Wang Zixian, Zhang Miao, Yan Danfeng

School of Computer Science, Beijing University of Posts and Telecommunications, Beijing 100876, China

Received:2021-10-27 Revised:2022-04-07 Online:2023-06-30 Published:2023-06-30
Contact: Yan Danfeng E-mail:yandf@bupt.edu.cn

摘要/Abstract

摘要：

Traditional simultaneous localization and mapping ( SLAM) mostly performs under the assumption of an ideal static environment, which is not suitable for dynamic environments in the real world. Dynamic real-time object-aware SLAM ( DRO-SLAM) is proposed in this paper, which is a visual SLAM that can realize simultaneous localizing and mapping and tracking of moving objects indoor and outdoor at the same time. It can use target recognition, oriented fast and rotated brief (ORB) feature points, and optical flow assistance to track multi-target dynamic objects and remove them during dense point cloud reconstruction while estimating their pose. By verifying the algorithm effect on the public dataset and comparing it with other methods, it can be obtained that the proposed algorithm has certain guarantees in real-time and accuracy, it also provides more functions. DRO-SLAM can provide the solution to automatic navigation which can realize lightweight deployment, provide more vehicles, pedestrians and other environmental information for navigation.

关键词: simultaneous localization and mapping (SLAM), object tracking, stereo vision, dynamic environment, data association

Abstract: Traditional simultaneous localization and mapping ( SLAM) mostly performs under the assumption of an ideal static environment, which is not suitable for dynamic environments in the real world. Dynamic real-time object-aware SLAM ( DRO-SLAM) is proposed in this paper, which is a visual SLAM that can realize simultaneous localizing and mapping and tracking of moving objects indoor and outdoor at the same time. It can use target recognition, oriented fast and rotated brief (ORB) feature points, and optical flow assistance to track multi-target dynamic objects and remove them during dense point cloud reconstruction while estimating their pose. By verifying the algorithm effect on the public dataset and comparing it with other methods, it can be obtained that the proposed algorithm has certain guarantees in real-time and accuracy, it also provides more functions. DRO-SLAM can provide the solution to automatic navigation which can realize lightweight deployment, provide more vehicles, pedestrians and other environmental information for navigation.

Key words: simultaneous localization and mapping (SLAM), object tracking, stereo vision, dynamic environment, data association

中图分类号:

TP39

Wang Zixian, Zhang Miao, Yan Danfeng . DRO-SLAM: Real-time object-aware SLAM for navigation robots and autonomous driving in dynamic environments[J]. The Journal of China Universities of Posts and Telecommunications, 2023, 30(3): 14-24.

参考文献

1. MUR-ARTAL R, TARDÓS J D. ORB-SLAM2: An open-source SLAM system for monocular, stereo, and RGB-D cameras. IEEE Transactions on Robotics, 2017, 33(5): 1255-1262.

2. ENGEL J, KOLTUN V, CREMERS D. Direct sparse odometry. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(3): 611-625.

3. FORSTER C, PIZZOLI M, SCARAMUZZA D. SVO: Fast semi-direct monocular visual odometry. Proceedings of the 2014 IEEE International Conference on Robotics and Automation (ICRA’14), 2014, May 31-Jun 7, Hong Kong, China. Piscataway, NJ, USA: IEEE, 2014: 15-22.

4. CAMPOS C, ELVIRA R, RODRÍGUEZ J J G, et al. ORB-SLAM3: An accurate open-source library for visual, visual-inertial, and multimap SLAM. IEEE Transactions on Robotics, 2021, 37(6): 1874-1890.

5. BESCOS B, FÁCIL J M, CIVERA J, et al. DynaSLAM: Tracking, mapping, and inpainting in dynamic scenes. IEEE Robotics and Automation Letters, 2018, 3(4): 4076-4083.

6. YU C, LIU Z X, LIU X J, et al. DS-SLAM: A semantic visual SLAM towards dynamic environments. Proceedings of the 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’18), 2018, Oct 1-5, Madrid, Spain. Piscataway, NJ, USA: IEEE, 2018: 1168-1174.

7. ZHANG T W, ZHANG H Y, LI Y, et al. Flowfusion: Dynamic dense RGB-D SLAM based on optical flow. Proceedings of the 2020 IEEE International Conference on Robotics and Automation (ICRA’20), 2020, May 31-Aug 31, Paris, France. Piscataway, NJ, USA: IEEE, 2020: 7322-7328.

8. YUAN X, CHEN S. SaD-SLAM: A visual SLAM based on semantic and depth information. Proceedings of the 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’20), 2020, Oct 25-29, Las Vegas, NV, USA. Piscataway, NJ, USA: IEEE, 2020: 4930-4935.

9. LI P L, QIN T, SHEN S J. Stereo vision-based semantic 3D object and ego-motion tracking for autonomous driving. Computer Vision: Proceedings of the 15th European Conference on Computer Vision (ECCV): Part II, 2018, Sept 8-14, Munich, Germany. LNCS 11206. Berlin, Germany: Springer, 2018: 664-679.

10. YANG S C, SCHERER S. CubeSLAM: Monocular 3-D object SLAM. IEEE Transactions on Robotics, 2019, 35(4): 925-938.

11. HUANG J H, YANG S, MU T J, et al. ClusterVO: Clustering moving instances and estimating visual odometry for self and surroundings. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR'20), 2020, Jun 14-19, Seattle, WA, USA. Piscataway, NJ, USA: IEEE, 2020: 2168-2177.

12. ZHANG J, HENEIN M, MAHONY R, et al. VDO-SLAM: A visual dynamic object-aware SLAM system. arXiv Preprint, arXiv: 2005.11052, 2020.

13. HENEIN M, ZHANG J, MAHONY R, et al. Dynamic SLAM: The need for speed. Proceedings of the 2020 IEEE International Conference on Robotics and Automation (ICRA’20), 2020, May 31-Aug 31, Paris, France. Piscataway, NJ, USA: IEEE, 2020: 2123-2129.

14. BESCOS B, CAMPOS C, TARDÓS J D, et al. DynaSLAM II: Tightly-coupled multi-object tracking and SLAM. IEEE Robotics and Automation Letters, 2021, 6(3): 5191-5198.

15. WANG C C, THORPE C, THRUN S. Online simultaneous localization and mapping with detection and tracking of moving objects: Theory and results from a ground vehicle in crowded urban areas. Proceedings of the 2003 IEEE International Conference on Robotics and Automation (ICRA’03): Vol 1, 2003, Sept 14-19, Taipei, China. Piscataway, NJ, USA: IEEE, 2003: 842-849.

16. WANGSIRIPITAK S, MURRAY D W. Avoiding moving outliers in visual SLAM by tracking moving objects. Proceedings of the 2009 IEEE International Conference on Robotics and Automation (ICRA’09), 2009, May 12-17, Kobe, Japan. Piscataway, NJ, USA: IEEE, 2009: 375-380.

17. ROGERS J G, TREVOR A J B, NIETO-GRANDA C, et al. SLAM with expectation maximization for moveable object tracking. Proceedings of the 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’10), 2010, Oct 18-22, Taipei, China. Piscataway, NJ, USA: IEEE, 2010: 2077-2082.

18. BÂRSAN I A, LIU P D, POLLEFEYS M, et al. Robust dense mapping for large-scale dynamic environments. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA’18), 2018, May 21-25, Brisbane, Australia. Piscataway, NJ, USA: IEEE, 2018: 7510-7517.

19. ROSINOL A, GUPTA A, ABATE M, et al. 3D dynamic scene graphs: Actionable spatial perception with places, objects, and humans. arXiv Preprint, arXiv: 2002.06289, 2020.

20. GEIGER A, LENZ P, STILLER C, et al. Vision meets robotics: The KITTI dataset. International Journal of Robotics Research, 2013, 32(11): 1231-1237.

21. WANG C C, THORPE C, THRUN S, et al. Simultaneous localization, mapping and moving object tracking. International Journal of Robotics Research, 2007, 26(9): 889-916.

22. HENEIN M, KENNEDY G, MAHONY V I R, et al. Exploiting rigid body motion for SLAM in dynamic environments. http://www.gerard-kennedy.com/2018_icra.

23. Eppenberger T, Cesari G, Dymczyk M, et al. Leveraging stereo-camera data for real-time dynamic obstacle detection and tracking[C]//2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2020: 10528-10535.

24. HIRSCHMULLER H. Stereo processing by semiglobal matching and mutual information. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2007, 30(2): 328-341.

25. GEIGER A, ROSER M, URTASUN R. Efficient large-scale stereo matching. Computer Vision: Proceedings of the 10th Asian Conference on Computer vision (ACCV’10): Part I, 2010, Nov 8-12, Queenstown, New Zealand. LNIP 6492. Berlin, Germany: Springer, 2010: 25-38.

26. REDMON J, FARHADI A. YOLOv3: An incremental improvement. arXiv Preprint, arXiv:1804.02767.

27. MEINHARDT-LLOPIS E, SÁNCHEZ J. Horn-schunck optical flow with a multi-scale strategy. Image Processing On Line, 2012-06-13.

[1]	Zhang Luying, Liu Xiaokai, Li Zhao, Xu Fangmin, Zhao Chenglin. Prediction based dynamic resource allocation method for edge computing first networking [J]. 中国邮电高校学报(英文), 2023, 30(3): 78-87.
[2]	Chen Hui, Jiang Xiaoling, Wu Tianting, Mou Xingyu. Crowd sensing data delivery based on tangle DAG network [J]. 中国邮电高校学报(英文), 2023, 30(3): 88-98.
[3]	Guo Xiangbo, Wang Jian, Huang Mengjie, Wang Minghui, Yang Jian, Yu Yongtao. Deep knowledge tracking algorithm based on forgetting law[J]. 中国邮电高校学报(英文版), 2023, 30(1): 17-27.
[4]	Wang Xianlun, Wang Guangyu, Cui Yuxia. Facial expression recognition based on improved ResNet[J]. 中国邮电高校学报(英文版), 2023, 30(1): 28-38.
[5]	Wu Hongxin, Lin Zhijian, Chen Pingping, Chen Feng. Joint partial computation offloading and resource allocation in MEC-enable networks[J]. 中国邮电高校学报(英文版), 2023, 30(1): 80-86.
[6]	Zhu Ruijie, Li Gong, Wang Peisen, Zhang Wenchao. Reinforced virtual optical network embedding algorithm in EONs for edge computing [J]. 中国邮电高校学报(英文), 2022, 29(6): 18-29.
[7]	Kong Chao, Ou Weihua, Gong Xiaofeng, Li Weian, Han Jie, Yao Yi, Xiong Jiahao. Face anti-spoofing based on multi-modal and multi-scale features fusion [J]. 中国邮电高校学报(英文), 2022, 29(6): 73-82.
[8]	Yang Jingjing, Guo Yuchun, Zhao Yongxiang, Chen Yishuai. How time latency in navigation Apps affects traffic [J]. 中国邮电高校学报(英文), 2022, 29(6): 53-63.
[9]	张函井音吉赵永利. FS-LSTM: sales forecasting in e-commerce on feature selection [J]. 中国邮电高校学报(英文版), 2022, 29(5): 92-98.
[10]	Shi Jinjing, Wang Wenxuan, Xiao Zimeng, Mu Shuai, Li Qin. Quantum classifier with parameterized quantum circuit based on the isolated quantum system[J]. 中国邮电高校学报(英文版), 2022, 29(4): 21-31.
[11]	Liu Hailing, Zhang Jie, Qin Sujuan, Gao Fei. Quantum algorithm for soft margin support vector machine with hinge loss function[J]. 中国邮电高校学报(英文版), 2022, 29(4): 32-41.
[12]	Wang Jian, Qiao Kuoyuan, Yuan Yanlei, Liu Xiaole, Yang Jian. Adaptive learning path recommendation model for examination-oriented education[J]. 中国邮电高校学报(英文版), 2022, 29(4): 77-88.
[13]	Meng Wei, Wang Liting, Lu Meng. Summary of research on recommendation system based on serendipity[J]. 中国邮电高校学报(英文版), 2022, 29(4): 89-105.
[14]	Jia Wei, Gong Chao. Precise and efficient Chinese license plate recognition in the real monitoring scene of intelligent transportation system[J]. 中国邮电高校学报(英文), 2022, 29(3): 1-14.
[15]	Song Yue, Wu Chengmao, Tian Xiaoping, Song Qiuyu. Enhanced kernel-based fuzzy local information clustering integrating neighborhood membership [J]. 中国邮电高校学报(英文版), 2021, 28(6): 65-81.

DRO-SLAM: Real-time object-aware SLAM for navigation robots and autonomous driving in dynamic environments

DRO-SLAM: Real-time object-aware SLAM for navigation robots and autonomous driving in dynamic environments

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价