Precise Decision-Making Learning for Automated Vehicles in Lane-Change Scenario Based on Parameter Description
CSTR:
Author:
Affiliation:

State Key Laboratory of Automotive Simulation and Control, Jilin University, Changchun 130022, China.

Clc Number:

U471.1

  • Article
  • | |
  • Metrics
  • |
  • Reference [16]
  • |
  • Related [20]
  • | | |
  • Comments
    Abstract:

    To promote safety and fully consider human drivers' acceptance, precise decision-making is realized for automated vehicles under the lane-change scenario in this paper. More specifically, automated vehicles not only decide to change lanes or not but also decide specific microcosmic behaviors, such as lane-change time and expected acceleration. Thus, precise decisions for lane-change are described with three parameters and learned by reinforcement learning. The rationality of such parameter-based precise decisions is shown in two aspects. First, different values of decision parameters will notably influence the planned trajectory, which means other microcosmic behaviors will be a significant uncertainty when they are not precisely decided in the decision-making layer. Secondly, based on the analysis of real traffic data, NGSIM, changeable lane-change time, and expected acceleration are revealed in lane-change behaviors, which is seldom explicitly considered in the decision-making layer of current researches. The decision parameters that include lane-change time and expected acceleration are learned with kernel-based least-squares policy iteration reinforcement learning (KLSPI). Safety, current driver's willingness, and average human driving style are considered in the reward function. Simulation results demonstrate that using reinforcement learning (RL) to learn decision parameters can realize more precise decisions, promote safety performance, and imitate human drivers' behaviors in the lane-change scenario.

    Reference
    [1] PADEN B , ?áP M , YONG S Z , et al . A survey of motion planning and control techniques for self-driving urban vehicles[J]. IEEE Transactions on Intelligent Vehicles, 2016, 1(1): 33.
    [2] LI X H , SUN Z P , CAO D P , et al . Real-time trajectory planning for autonomous urban driving: Framework, algorithms, and verifications[J]. IEEE/ASME Transactions on mechatronics, 2015, 21(2): 740.
    [3] GUO C Z , KIDONO K , TERASHIMA R , et al . Toward human-like behavior generation in urban environment based on Markov decision process with hybrid potential maps[C]// 2018 IEEE Intelligent Vehicles Symposium (IV). Changshu: IEEE, 2018: 2209.
    [4] CHU H Q , GUO L L , YAN Y J , et al . Self-learning optimal cruise control based on individual car-following style[J], IEEE Transactions on Intelligent Transportation Systems, 2020, 99: 1.
    [5] GINDELE T , BRECHTEL S , DILLMANN R , et al . Learning driver behavior models from traffic observations for decision making and planning[J]. IEEE Intelligent Transportation Systems Magazine, 2015, 7(1): 69.
    [6] MARTINEZ C M , HEUCKE M , WANG F Y , et al . Driving style recognition for intelligent vehicle control and advanced driver assistance: A survey[J]. IEEE Transactions on Intelligent Transportation Systems, 2018, 19(3): 666.
    [7] GONZáLEZ D , PéREZ J , MILANéS V , et al . A review of motion planning techniques for automated vehicles[J]. IEEE Trans. Intelligent Transportation Systems, 2016, 17(4): 1135.
    [8] VALLON C , ERCAN Z , CARVALHO A , et al . A machine learning approach for personalized autonomous lane change initiation and control[C]// 2017 IEEE Intelligent Vehicles Symposium (IV). Los Angeles: IEEE, 2017: 1590.
    [9] HE G L , LI X , LYU Y, et al . Probabilistic intention prediction and trajectory generation based on dynamic bayesian networks[C]// 2019 Chinese Automation Congress (CAC), Hangzhou: IEEE, 2019: 2646.
    [10] TAN Y V , ELLIOTT M R , FLANNAGAN C A C , et al . Development of a real-time prediction model of driver behavior at intersections using kinematic time series data[J]. Accident Analysis & Prevention, 2017, 106: 428.
    [11] YOU C X , LU J B , FILEV D , et al . Highway traffic modeling and decision making for autonomous vehicle using reinforcement learning[C]// 2018 IEEE Intelligent Vehicles Symposium (IV). Changshu: IEEE, 2018: 1227.
    [12] SHALEV-SHWARTZ S , SHAMMAH S , SHASHUA A , et al . On a formal model of safe and scalable self-driving cars [DB/OL]. arXiv: 1708.06374, 2017. https://doi.org/10.48550/arXiv.1708.06374.
    [13] ZHANG Y X , GAO B Z , GUO L L , et al . Adaptive decision-making for automated vehicles under roundabout scenarios using optimization embedded reinforcement learning[J]. IEEE Transactions on Neural Networks and Learning Systems, 2020, 99: 1.
    [14] ARIKERE A , YANG D , KLOMP M , et al . Integrated evasive manoeuvre assist for collision mitigation with oncoming vehicles[J]. Vehicle System Dynamics, 2018, 56(10): 1.
    [15] ZHANG Y X , GAO B Z , GUO L L , et al . A novel trajectory planning method for automated vehicles under parameter decision framework[J]. IEEE Access, 2019, 7: 88264.
    [16] XU X , HU D W , LU X C , et al . Kernel-based least squares policy iteration for reinforcement learning[J]. IEEE Transactions on Neural Networks, 2007, 18(4): 973.
    Cited by
    Comments
    Comments
    分享到微博
    Submit
Get Citation

ZHANG Yuxiang, HE Ganglei, LI Xin, LIU Qifang, CONG Yanfeng, WANG Yuhai. Precise Decision-Making Learning for Automated Vehicles in Lane-Change Scenario Based on Parameter Description[J].同济大学学报(自然科学版),2021,49(S1):132~140

Copy
Share
Article Metrics
  • Abstract:208
  • PDF: 508
  • HTML: 32
  • Cited by: 0
History
  • Received:September 25,2021
  • Online: February 28,2023
Article QR Code