Autonomous driving in the uncertain traffic -- a deep reinforcement learning approach

doi:10.19682/j.cnki.1005-8885.2018.1024

中国邮电高校学报(英文) ›› 2018, Vol. 25 ›› Issue (6): 21-30.doi: 10.19682/j.cnki.1005-8885.2018.1024

• Artificial Intelligence • 上一篇下一篇

Autonomous driving in the uncertain traffic -- a deep reinforcement learning approach

Yang Shun, Wu Jian, Zhang Sumin, Han Wei

1. State Key Laboratory of Automotive Simulation and Control, Jilin University, Changchun 130022, China
2. Institute of Microelectronics, Chinese Academy of Sciences, Beijing 100029, China

收稿日期:2018-10-23 修回日期:2018-12-27 出版日期:2018-12-30 发布日期:2019-02-26
通讯作者: Zhang Sumin, E-mail: suminzhang@163.com E-mail:suminzhang@163.com
作者简介:Zhang Sumin, E-mail: suminzhang@163.com
基金资助:
Sample DRL training and demo sequences are provided as supplementary material for the review process. The URL are directly input below.
Agent driving without traffic participants:https://youtu.be/dMMi3a_BaqU.
Agent driving with traffic participants:https://youtu.be/gnSzw9c2TuM.

Autonomous driving in the uncertain traffic -- a deep reinforcement learning approach

Yang Shun, Wu Jian, Zhang Sumin, Han Wei

1. State Key Laboratory of Automotive Simulation and Control, Jilin University, Changchun 130022, China
2. Institute of Microelectronics, Chinese Academy of Sciences, Beijing 100029, China

Received:2018-10-23 Revised:2018-12-27 Online:2018-12-30 Published:2019-02-26
Contact: Zhang Sumin, E-mail: suminzhang@163.com E-mail:suminzhang@163.com
About author:Zhang Sumin, E-mail: suminzhang@163.com
Supported by:
Sample DRL training and demo sequences are provided as supplementary material for the review process. The URL are directly input below.
Agent driving without traffic participants:https://youtu.be/dMMi3a_BaqU.
Agent driving with traffic participants:https://youtu.be/gnSzw9c2TuM.

摘要/Abstract

摘要： Driving in the complex traffic safely and efficiently is a difficult task for autonomous vehicle because of the stochastic characteristics of engaged human drivers. Deep reinforcement learning (DRL), which combines the abstract representation capability of deep learning (DL) and the optimal decision making and control capability of reinforcement learning (RL), is a good approach to address this problem. Traffic environment is built up by combining intelligent driver model (IDM) and lane-change model as behavioral model for vehicles. To increase the stochastic of the established traffic environment, tricks such as defining a speed distribution with cutoff for traffic cars and using various politeness factors to represent distinguished lane-change style, are taken. For training an
artificial agent to achieve successful strategies that lead to the greatest long-term rewards and sophisticated maneuver, deep deterministic policy gradient (DDPG) algorithm is deployed for learning. Reward function is designed to get a trade-off between the vehicle speed, stability and driving safety. Results show that the proposed approach can achieve good autonomous maneuvering in a scenario of complex traffic behavior through interaction with the environment.

关键词: autonomous driving, complex traffic scenario, DRL, DDPG

Abstract: Driving in the complex traffic safely and efficiently is a difficult task for autonomous vehicle because of the stochastic characteristics of engaged human drivers. Deep reinforcement learning (DRL), which combines the abstract representation capability of deep learning (DL) and the optimal decision making and control capability of reinforcement learning (RL), is a good approach to address this problem. Traffic environment is built up by combining intelligent driver model (IDM) and lane-change model as behavioral model for vehicles. To increase the stochastic of the established traffic environment, tricks such as defining a speed distribution with cutoff for traffic cars and using various politeness factors to represent distinguished lane-change style, are taken. For training an
artificial agent to achieve successful strategies that lead to the greatest long-term rewards and sophisticated maneuver, deep deterministic policy gradient (DDPG) algorithm is deployed for learning. Reward function is designed to get a trade-off between the vehicle speed, stability and driving safety. Results show that the proposed approach can achieve good autonomous maneuvering in a scenario of complex traffic behavior through interaction with the environment.

Key words: autonomous driving, complex traffic scenario, DRL, DDPG

中图分类号:

U495

Yang Shun, Wu Jian, Zhang Sumin, Han Wei. Autonomous driving in the uncertain traffic -- a deep reinforcement learning approach[J]. JOURNAL OF CHINA UNIVERSITIES OF POSTS AND TELECOM, 2018, 25(6): 21-30.

参考文献

References
1. Broadhurst A, Baker S, Kanade T. Monte Carlo road safety reasoning. IEEE Proceedings Intelligent Vehicles Symposium, 2005, Las Vegas, NV, USA, 2005: 319 -324
2. Kolmanovsky I V, Filev D P. Stochastic optimal control of systems with soft constraints and opportunities for automotive applications. Control Applications ( CCA) and Intelligent Control ( ISIC), IEEE, 2009: 1265 -1270
3. Carvalho A, Lefevre S, Schildbach G, et al. Automated driving: the role of forecasts and uncertainty-a control perspective. European Journal of Control, 2015, 24: 14 -32
4. Ulbrich S, Maurer M. Probabilistic online POMDP decision making for lane changes in fully automated driving. 16th International IEEE Conference on Intelligent Transportation Systems
(ITSC 2013), Hague, Netherlands, 2013: 2063 -2067
5. Hubmann C, Schulz J, Becker M, et al. Automated driving in uncertain environments: planning with interaction and uncertain maneuver prediction. IEEE Transactions on Intelligent Vehicles, 2018(99): 1p
6. Schwarting W, Alonsomora J, Rus D. Planning and decision-making for autonomous vehicles. Annual Review of Control, Robotics, and Autonomous Systems, 2018(1): 187 -210
7. Bojarski M, Del T D, Dworakowski D, et al. End to end learning for self-driving cars. ArXiv preprint arXiv: 1604. 07316. 2016 Apr 25
8. Voosen P. How AI detectives are cracking open the black box of deep learning. Science, July 2017. http://www.sciencemag.org/news/2017/07/-how-ai-detectives-are-cracking-open-blackbox-deep-learning
9. Isele D, Rahimi R, Cosgun A, et al. Navigating occluded intersections with autonomous vehicles using deep reinforcement learning. In 2018 IEEE International Conference on Robotics and Automation (ICRA), IEEE, May 21, 2018: 2034 -2039
10. Mukadam M, Cosgun A, Nakhaei A, et al. Tactical decision making for lane changing with deep reinforcement learning. 31st Conference on Neural Information Processing Systems ( NIPS 2017), Long Beach, CA, USA, 2017. https://openreview.net/pdf?id=B1G6uM0WG
11. Mnih V, Kavukcuoglu K, Silver D, et al. Playing atari with deep reinforcement learning. ArXiv preprint arXiv: 1312. 5602. 2013 Dec 19
12. Paxton C, Raman V, Hager G D, et al. Combining neural networks and tree search for task and motion planning in challenging environments. ArXiv preprint arXiv: 1703. 07887.
2017 Mar 22
13. Shalevshwartz S, Shammah S, Shashua A. On a formal model of safe and scalable self-driving cars. ArXiv preprint arXiv: 1708. 06374. 2017 Aug 21
14. Treiber M, Hennecke A, Helbing D. Congested traffic states in empirical observations and microscopic simulations. Physical Review E Stat Phys Plasmas Fluids Relat Interdiscip Topics, 2000, 62(2 Pt A): 1805 -1824
15. Kesting A, Treiber M, Helbing D. General lane-changing model MOBIL for car-following models. Transportation Research Record Journal of the Transportation Research Board, 2007, 1999(1): 86 -94
16. Sorstedt J, Svensson L, Sandblom F, et al. A new vehicle motion model for improved predictions and situation assessment. IEEE Transactions on Intelligent Transportation Systems, 2011, 12(4): 1209 -1219
17. Svensson L, Jenny E. Tuning for ride quality in autonomous vehicle: application to linear quadratic path planning algorithm. Uppsala University, Uppsala, Sweden, 2015
18. Chen C, Seff A, Kornhauser A, et al. Deepdriving: learning affordance for direct perception in autonomous driving. Proceedings of 15th IEEE International Conference on Computer Vision (ICCV 2015), Santiago, Chile, 2015: 2722 -2730

Autonomous driving in the uncertain traffic -- a deep reinforcement learning approach

Autonomous driving in the uncertain traffic -- a deep reinforcement learning approach

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 0

编辑推荐

Metrics

本文评价