Steering control in autonomous vehicles using deep reinforcement learning

doi:10.19682/j.cnki.1005-8885.2018.1027

中国邮电高校学报(英文) ›› 2018, Vol. 25 ›› Issue (6): 58-64.doi: 10.19682/j.cnki.1005-8885.2018.1027

• Artificial Intelligence • 上一篇下一篇

Steering control in autonomous vehicles using deep reinforcement learning

Xue Chong, Jia Peng, Zhang Xinyu

1. Department of Computer Science and Technique, Tsinghua University, Beijing 300401, China
2. Joint Logistics College, Beijing 100858, China

收稿日期:2018-08-07 修回日期:2019-01-04 出版日期:2018-12-30 发布日期:2019-02-26
通讯作者: Jia Peng, E-mail: jiapeng1018@163.com E-mail:jiapeng1018@163.com
作者简介:Jia Peng, E-mail: jiapeng1018@163.com

Steering control in autonomous vehicles using deep reinforcement learning

Xue Chong, Jia Peng, Zhang Xinyu

1. Department of Computer Science and Technique, Tsinghua University, Beijing 300401, China
2. Joint Logistics College, Beijing 100858, China

Received:2018-08-07 Revised:2019-01-04 Online:2018-12-30 Published:2019-02-26
Contact: Jia Peng, E-mail: jiapeng1018@163.com E-mail:jiapeng1018@163.com
About author:Jia Peng, E-mail: jiapeng1018@163.com

摘要/Abstract

摘要： A novel deep reinforcement learning-based steering control method of autonomous vehicles is proposed. A distortionless compressing method of action space is presented. Convolutional neural networks (CNNs) are designed to serve as an action policy. Driver experience is investigated and modeled to optimize policy of new actions exploration. Experimental results show that the proposed algorithm has better robustness and smoothness. Moreover, it is applicable to different roads, velocities or wire-control systems.

关键词: autonomous vehicles, deep reinforcement learning, steering control, CNNs

Abstract: A novel deep reinforcement learning-based steering control method of autonomous vehicles is proposed. A distortionless compressing method of action space is presented. Convolutional neural networks (CNNs) are designed to serve as an action policy. Driver experience is investigated and modeled to optimize policy of new actions exploration. Experimental results show that the proposed algorithm has better robustness and smoothness. Moreover, it is applicable to different roads, velocities or wire-control systems.

Key words: autonomous vehicles, deep reinforcement learning, steering control, CNNs

中图分类号:

TP30

Xue Chong, Jia Peng, Zhang Xinyu. Steering control in autonomous vehicles using deep reinforcement learning[J]. JOURNAL OF CHINA UNIVERSITIES OF POSTS AND TELECOM, 2018, 25(6): 58-64.

参考文献

References
1. Marino R, Scalzi S, Orlando G, et al. A nested PID steering control for lane keeping in vision based autonomous vehicles. Proc IEEE Amer Control Conf, St Louis, MO, USA, Jun, 2009: 2885 -2890
2. Du X, Tan K K, Htet K K K. Vision approach towards fully self-reverse parking system. Proc IEEE Conf Mechatronics Autom, Tianjin, China, Aug, 2014: 186 -191
3. Yeh Y C, Li T H S, Chen C Y. Adaptive fuzzy sliding-mode control of dynamic model based car-like mobile robot. Int J Fuzzy Syst, 2009(11): 272 -286
4. Chen H, Allgower F. A quasi infinite horizon nonlinear model predictive control scheme with guaranteed stability. Automatica, 1998, 34: 1205 -1217
5. Erkan K, Herman R, Wouter S. Robust trajectory tracking error model-based predictive control for unmanned ground vehicles. IEEE/ ASME Transactions on Mechatronics, 2016, 21: 806 -814
6. Du X X, Htet K K K, Tan K K. Development of a genetic-algorithm-based nonlinear model predictive control scheme on velocity and steering of autonomous vehicles. IEEE Transactions on Industrial Electronics, 2016, 63: 6970 -6977
7. Volodymyr M, Koray K, David S, et al. Playing atari with deep reinforcement learning. Computer Science, 2013
8. Timothy P L, Jonathan J H, Alexander P, et al. Continuous control with deep reinforcement learning. ICLR, 2016
9. Paul F C, Jan L, Tom B B, et al. Deep reinforcement learning from human preferences. ArXiv E-Prints, 2017
10. Jonathan H, Stefano E. Generative adversarial imitation learning. In Advances in Neural Information Processing Systems, 2016: 4565 -4573
11. Finn C, Yu T H, Fu J, et al. Generalizing skills with semi-supervised reinforcement learning. International Conference on Learning Representations, 2017
12. Simon K, Mattias K. Autonomous vehicle control via deep reinforcement learning. Master's Thesis, Chalmers University of Technology, 2017
13. https://github.com/yenchenlin/DeepLearningFlappyBird
14. https://www.youtube.com/watch?v=THhUXIhjkCM
15. Alexey D, German R, Felipe C, et al. CARLA: an open urban driving simulator
16. Ouyang Z C, Niu J, Guizani M. Improved vehicle steering pattern recognition by using selected sensor data. IEEE Transactions on Mobile Computing, 2018,17(6): 1383 -1396
17. Omead A. Integrated mobile robot control. Technical Report CMU-RI-TR-90-17, Carnegie Mellon University Robotics Institute, 1990
18. Stanley. The robot that won the DARPA grand challenge

[1]	孟慧任利娜赵宗渠. Identity-based proxy re-encryption scheme from RLWE assumption with ciphertext evolution[J]. 中国邮电高校学报(英文版), 2023, 30(5): 51-60.
[2]	Peng Weiping, Cui Shuang, Song Cheng, Han Ning. Enhanced secure medical data sharing with traceable and direct revocation[J]. 中国邮电高校学报(英文版), 2023, 30(1): 66-79.
[3]	Zhang Jiangbo, Zheng Jiali, Quan Yixuan, Lin Zihan, Xie Xiaode. Improved sparrow search algorithm for RFID network planning[J]. 中国邮电高校学报(英文版), 2023, 30(1): 93-102.
[4]	谢孝德郑嘉利林子涵何思怡冯敏瑜. Random mating mayfly algorithm for RFID network planning[J]. 中国邮电高校学报(英文版), 2022, 29(5): 40-50.
[5]	林子涵郑嘉利谢孝德冯敏瑜何思怡. RFID network planning based on improved brain storm optimization algorithm[J]. 中国邮电高校学报(英文版), 2022, 29(5): 30-39.
[6]	Lv Lihui, Wang Hong, Ma Zhi, Duan Qianheng, Fei Yangyang, Meng Xiangdong. Methods for solving equations with errors based on the HHL algorithm[J]. 中国邮电高校学报(英文版), 2022, 29(4): 9-20.
[7]	Han Yushan, Che Bichen, Liu Jiali, Dou Zhao, Di Junyu. Nearly universal and efficient quantum secure multi-party computation protocol[J]. 中国邮电高校学报(英文版), 2022, 29(4): 51-68.
[8]	Han Gang, Xing Qixuan, Zhang Yinghui. Fine-grained cooperative access control scheme with hidden policies[J]. 中国邮电高校学报(英文版), 2021, 28(6): 13-25.
[9]	Tao Yunting, Kong Fanyu, Yu Jia. EPMDA: an efficient privacy-preserving multi-dimensional data aggregation scheme for edge computing-based IoT system [J]. 中国邮电高校学报(英文版), 2021, 28(6): 26-35.
[10]	Xu Yan, Li Zheng, Ding Long, Xu Rui. Cross-domain data cloud storage auditing scheme based on certificateless cryptography [J]. 中国邮电高校学报(英文版), 2021, 28(6): 36-47.
[11]	Quan Yixuan, Zheng Jiali, Xie Xiaode, Lin Zihan, Luo Wencong. Hybrid gray wolf optimization-cuckoo search algorithm for RFID network planning [J]. 中国邮电高校学报(英文版), 2021, 28(6): 91-102.
[12]	顾亦然周鹏杨海根. News recommendation based on time factor and word embedding [J]. 中国邮电高校学报(英文版), 2021, 28(5): 82-90.
[13]	赵国生张婧婷王健. Research on location privacy protection method of sensor-cloud base station [J]. 中国邮电高校学报(英文版), 2021, 28(1): 64-77.
[14]	赵宗渠马少提王永军汤永利叶青. Two-factor ( biometric and password) authentication key exchange on lattice based on key consensus [J]. 中国邮电高校学报(英文版), 2020, 27(6): 42-53.
[15]	李丽, 郑嘉利, 全艺璇, 林子涵, 李映超, 黄天星. RFID Indoor positioning based on Semi-supervised Actor-Critic Co-training[J]. 中国邮电高校学报(英文版), 2020, 27(5): 69-81.

Steering control in autonomous vehicles using deep reinforcement learning

Steering control in autonomous vehicles using deep reinforcement learning

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价