skip to main content

Title: Dampen the Stop-and-Go Traffic with Connected and Automated Vehicles – A Deep Reinforcement Learning Approach
Stop-and-go traffic poses significant challenges to the efficiency and safety of traffic operations, and its impacts and working mechanism have attracted much attention. Recent studies have shown that Connected and Automated Vehicles (CAVs) with carefully designed longitudinal control have the potential to dampen the stop-and-go wave based on simulated vehicle trajectories. In this study, Deep Reinforcement Learning (DRL) is adopted to control the longitudinal behavior of CAVs and real-world vehicle trajectory data is utilized to train the DRL controller. It considers a Human-Driven (HD) vehicle tailed by a CAV, which are then followed by a platoon of HD vehicles. Such an experimental design is to test how the CAV can help to dampen the stop-and-go wave generated by the lead HD vehicle and contribute to smoothing the following HD vehicles’ speed profiles. The DRL control is trained using real-world vehicle trajectories, and eventually evaluated using SUMO simulation. The results show that the DRL control decreases the speed oscillation of the CAV by 54% and 8%-28% for those following HD vehicles. Significant fuel consumption savings are also observed. Additionally, the results suggest that CAVs may act as a traffic stabilizer if they choose to behave slightly altruistically.
Authors:
; ; ; ; ;
Award ID(s):
1932921 1826162
Publication Date:
NSF-PAR ID:
10318661
Journal Name:
2021 7th International Conference on Models and Technologies for Intelligent Transportation Systems
Sponsoring Org:
National Science Foundation
More Like this
  1. Stop-and-go traffic poses significant challenges to the efficiency and safety of traffic operations, and its impacts and working mechanism have attracted much attention. Recent studies have shown that Connected and Automated Vehicles (CAVs) with carefully designed longitudinal control have the potential to dampen the stop-and-go wave based on simulated vehicle trajectories. In this study, Deep Reinforcement Learning (DRL) is adopted to control the longitudinal behavior of CAVs and real-world vehicle trajectory data is utilized to train the DRL controller. It considers a Human-Driven (HD) vehicle tailed by a CAV, which are then followed by a platoon of HD vehicles. Suchmore »an experimental design is to test how the CAV can help to dampen the stop-and-go wave generated by the lead HD vehicle and contribute to smoothing the following HD vehicles’ speed profiles. The DRL control is trained using realworld vehicle trajectories, and eventually evaluated using SUMO simulation. The results show that the DRL control decreases the speed oscillation of the CAV by 54% and 8%-28% for those following HD vehicles. Significant fuel consumption savings are also observed. Additionally, the results suggest that CAVs may act as a traffic stabilizer if they choose to behave slightly altruistically.« less
  2. Motivated by connected and automated vehicle (CAV) technologies, this paper proposes a data-driven optimization-based Model Predictive Control (MPC) modeling framework for the Cooperative Adaptive Cruise Control (CACC) of a string of CAVs under uncertain traffic conditions. The proposed data-driven optimization-based MPC modeling framework aims to improve the stability, robustness, and safety of longitudinal cooperative automated driving involving a string of CAVs under uncertain traffic conditions using Vehicle-to-Vehicle (V2V) data. Based on an online learning-based driving dynamics prediction model, we predict the uncertain driving states of the vehicles preceding the controlled CAVs. With the predicted driving states of the preceding vehicles,more »we solve a constrained Finite-Horizon Optimal Control problem to predict the uncertain driving states of the controlled CAVs. To obtain the optimal acceleration or deceleration commands for the CAVs under uncertainties, we formulate a Distributionally Robust Stochastic Optimization (DRSO) model (i.e. a special case of data-driven optimization models under moment bounds) with a Distributionally Robust Chance Constraint (DRCC). The predicted uncertain driving states of the immediately preceding vehicles and the controlled CAVs will be utilized in the safety constraint and the reference driving states of the DRSO-DRCC model. To solve the minimax program of the DRSO-DRCC model, we reformulate the relaxed dual problem as a Semidefinite Program (SDP) of the original DRSO-DRCC model based on the strong duality theory and the Semidefinite Relaxation technique. In addition, we propose two methods for solving the relaxed SDP problem. We use Next Generation Simulation (NGSIM) data to demonstrate the proposed model in numerical experiments. The experimental results and analyses demonstrate that the proposed model can obtain string-stable, robust, and safe longitudinal cooperative automated driving control of CAVs by proper settings, including the driving-dynamics prediction model, prediction horizon lengths, and time headways. Computational analyses are conducted to validate the efficiency of the proposed methods for solving the DRSO-DRCC model for real-time automated driving applications within proper settings.« less
  3. Connected and automated vehicle (CAV) technology is providing urban transportation managers tremendous opportunities for better operation of urban mobility systems. However, there are significant challenges in real-time implementation as the computational time of the corresponding operations optimization model increases exponentially with increasing vehicle numbers. Following the companion paper (Chen et al. 2021), which proposes a novel automated traffic control scheme for isolated intersections, this study proposes a network-level, real-time traffic control framework for CAVs on grid networks. The proposed framework integrates a rhythmic control method with an online routing algorithm to realize collision-free control of all CAVs on a networkmore »and achieve superior performance in average vehicle delay, network traffic throughput, and computational scalability. Specifically, we construct a preset network rhythm that all CAVs can follow to move on the network and avoid collisions at all intersections. Based on the network rhythm, we then formulate online routing for the CAVs as a mixed integer linear program, which optimizes the entry times of CAVs at all entrances of the network and their time–space routings in real time. We provide a sufficient condition that the linear programming relaxation of the online routing model yields an optimal integer solution. Extensive numerical tests are conducted to show the performance of the proposed operations management framework under various scenarios. It is illustrated that the framework is capable of achieving negligible delays and increased network throughput. Furthermore, the computational time results are also promising. The CPU time for solving a collision-free control optimization problem with 2,000 vehicles is only 0.3 second on an ordinary personal computer.« less
  4. For energy-efficient Connected and Automated Vehicle (CAV) Eco-driving control on signalized arterials under uncertain traffic conditions, this paper explicitly considers traffic control devices (e.g., road markings, traffic signs, and traffic signals) and road geometry (e.g., road shapes, road boundaries, and road grades) constraints in a data-driven optimization-based Model Predictive Control (MPC) modeling framework. This modeling framework uses real-time vehicle driving and traffic signal data via Vehicle-to-Infrastructure (V2I) and Vehicle-to-Vehicle (V2V) communications. In the MPC-based control model, this paper mathematically formulates location-based traffic control devices and road geometry constraints using the geographic information from High-Definition (HD) maps. The location-based traffic controlmore »devices and road geometry constraints have the potential to improve the safety, energy, efficiency, driving comfort, and robustness of connected and automated driving on real roads by considering interrupted flow facility locations and road geometry in the formulation. We predict a set of uncertain driving states for the preceding vehicles through an online learning-based driving dynamics prediction model. We then solve a constrained finite-horizon optimal control problem with the predicted driving states to obtain a set of Eco-driving references for the controlled vehicle. To obtain the optimal acceleration or deceleration commands for the controlled vehicle with the set of Eco-driving references, we formulate a Distributionally Robust Stochastic Optimization (DRSO) model (i.e., a special case of data-driven optimization models under moment bounds) with Distributionally Robust Chance Constraints (DRCC) with location-based traffic control devices and road geometry constraints. We design experiments to demonstrate the proposed model under different traffic conditions using real-world connected vehicle trajectory data and Signal Phasing and Timing (SPaT) data on a coordinated arterial with six actuated intersections on Fuller Road in Ann Arbor, Michigan from the Safety Pilot Model Deployment (SPMD) project.« less
  5. Connected and automated vehicles (CAVs) will undoubtedly transform many aspects of transportation systems in the future. In the meantime, transportation agencies must make investment and policy decisions to address the future needs of the transportation system. This research provides much-needed guidance for agencies about planning-level capacities in a CAV future and quantify Highway Capacity Manual (HCM) capacities as a function of CAV penetration rates and vehicle behaviors such as car-following, lane change, and merge. As a result of numerous uncertainties on CAV implementation policies, the study considers many scenarios including variations in parameters (including CAV gap/headway settings), roadway geometry, andmore »traffic characteristics. More specifically, this study considers basic freeway, freeway merge, and freeway weaving segments in which various simulation scenarios are evaluated using two major CAV applications: cooperative adaptive cruise control and advanced merging. Data from microscopic traffic simulation are collected to develop capacity adjustment factors for CAVs. Results show that the existence of CAVs in the traffic stream can significantly enhance the roadway capacity (by as much as 35% to 40% under certain cases), not only on basic freeways but also on merge and weaving segments, as the CAV market penetration rate increases. The human driver behavior of baseline traffic also affects the capacity benefits, particularly at lower CAV market penetration rates. Finally, tables of capacity adjustment factors and corresponding regression models are developed for HCM implementation of the results of this study.« less