skip to main content

Title: Cooperative Highway Work Zone Merge Control Based on Reinforcement Learning in a Connected and Automated Environment
Given the aging infrastructure and the anticipated growing number of highway work zones in the U.S.A., it is important to investigate work zone merge control, which is critical for improving work zone safety and capacity. This paper proposes and evaluates a novel highway work zone merge control strategy based on cooperative driving behavior enabled by artificial intelligence. The proposed method assumes that all vehicles are fully automated, connected, and cooperative. It inserts two metering zones in the open lane to make space for merging vehicles in the closed lane. In addition, each vehicle in the closed lane learns how to adjust its longitudinal position optimally to find a safe gap in the open lane using an off-policy soft actor critic reinforcement learning (RL) algorithm, considering its surrounding traffic conditions. The learning results are captured in convolutional neural networks and used to control individual vehicles in the testing phase. By adding the metering zones and taking the locations, speeds, and accelerations of surrounding vehicles into account, cooperation among vehicles is implicitly considered. This RL-based model is trained and evaluated using a microscopic traffic simulator. The results show that this cooperative RL-based merge control significantly outperforms popular strategies such as late merge more » and early merge in terms of both mobility and safety measures. It also performs better than a strategy assuming all vehicles are equipped with cooperative adaptive cruise control. « less
; ;
Award ID(s):
Publication Date:
Journal Name:
Transportation Research Record: Journal of the Transportation Research Board
Page Range or eLocation-ID:
363 to 374
Sponsoring Org:
National Science Foundation
More Like this
  1. This study focuses on how to improve the merge control prior to lane reduction points due to either accidents or constructions. A Cooperative Car-following and Merging (CCM) control strategy is proposed considering the coexistence of Automated Vehicles (AVs) and Human-4 Driven Vehicles (HDVs). CCM introduces a modified/generalized Cooperative Adaptive Cruise Control (CACC) for vehicle longitudinal control prior to lane reduction points. It also takes courtesy into account to ensure that AVs behave responsibly and ethically. CCM is evaluated using microscopic traffic simulation and compared with no control and CACC merge strategies. The results show that CCM consistently generates the lowestmore »delays and highest throughputs approaching the theoretical capacity. Its safety benefits are also found to be significant based on vehicle trajectories and density maps. AVs in this study do not need to be fully automated and can be at Level-1 automation. CCM only requires automated longitudinal control such as Adaptive Cruise Control (ACC) and information sharing among vehicles, and ACC is already commercially available on many new vehicles. Also, it does not need 100% ACC penetration, presenting itself as a promising and practical solution for improving traffic operations in lane reduction transition areas such as highway work zones.« less
  2. Connected and automated vehicles (CAVs) will undoubtedly transform many aspects of transportation systems in the future. In the meantime, transportation agencies must make investment and policy decisions to address the future needs of the transportation system. This research provides much-needed guidance for agencies about planning-level capacities in a CAV future and quantify Highway Capacity Manual (HCM) capacities as a function of CAV penetration rates and vehicle behaviors such as car-following, lane change, and merge. As a result of numerous uncertainties on CAV implementation policies, the study considers many scenarios including variations in parameters (including CAV gap/headway settings), roadway geometry, andmore »traffic characteristics. More specifically, this study considers basic freeway, freeway merge, and freeway weaving segments in which various simulation scenarios are evaluated using two major CAV applications: cooperative adaptive cruise control and advanced merging. Data from microscopic traffic simulation are collected to develop capacity adjustment factors for CAVs. Results show that the existence of CAVs in the traffic stream can significantly enhance the roadway capacity (by as much as 35% to 40% under certain cases), not only on basic freeways but also on merge and weaving segments, as the CAV market penetration rate increases. The human driver behavior of baseline traffic also affects the capacity benefits, particularly at lower CAV market penetration rates. Finally, tables of capacity adjustment factors and corresponding regression models are developed for HCM implementation of the results of this study.« less
  3. Motivated by connected and automated vehicle (CAV) technologies, this paper proposes a data-driven optimization-based Model Predictive Control (MPC) modeling framework for the Cooperative Adaptive Cruise Control (CACC) of a string of CAVs under uncertain traffic conditions. The proposed data-driven optimization-based MPC modeling framework aims to improve the stability, robustness, and safety of longitudinal cooperative automated driving involving a string of CAVs under uncertain traffic conditions using Vehicle-to-Vehicle (V2V) data. Based on an online learning-based driving dynamics prediction model, we predict the uncertain driving states of the vehicles preceding the controlled CAVs. With the predicted driving states of the preceding vehicles,more »we solve a constrained Finite-Horizon Optimal Control problem to predict the uncertain driving states of the controlled CAVs. To obtain the optimal acceleration or deceleration commands for the CAVs under uncertainties, we formulate a Distributionally Robust Stochastic Optimization (DRSO) model (i.e. a special case of data-driven optimization models under moment bounds) with a Distributionally Robust Chance Constraint (DRCC). The predicted uncertain driving states of the immediately preceding vehicles and the controlled CAVs will be utilized in the safety constraint and the reference driving states of the DRSO-DRCC model. To solve the minimax program of the DRSO-DRCC model, we reformulate the relaxed dual problem as a Semidefinite Program (SDP) of the original DRSO-DRCC model based on the strong duality theory and the Semidefinite Relaxation technique. In addition, we propose two methods for solving the relaxed SDP problem. We use Next Generation Simulation (NGSIM) data to demonstrate the proposed model in numerical experiments. The experimental results and analyses demonstrate that the proposed model can obtain string-stable, robust, and safe longitudinal cooperative automated driving control of CAVs by proper settings, including the driving-dynamics prediction model, prediction horizon lengths, and time headways. Computational analyses are conducted to validate the efficiency of the proposed methods for solving the DRSO-DRCC model for real-time automated driving applications within proper settings.« less
  4. The Intelligent Transportation System has become one of the most globally researched topics, with Connected and Autonomous Vehicles(CAV) at its core. The CAV applications can be improved by the study of vehicle platooning immune to realtime traffic and vehicular network losses. In this work, we explore the need to integrate the Network model and Platooning system model for highway environments. The proposed platoon model is designed to be adaptive in length, providing the node vehicles to merge and exit. This overcomes the assumption that all the platoon nodes should have a common source and destination. The challenges of the existingmore »platoon model, such as relay selection, acceleration threshold, are addressed for highly modular platoon design. The presented algorithm for merge and exit events optimizes the trade-off between network parameters such as communication range and vehicle dynamic parameters such as velocity and acceleration threshold. It considers the network bounds like SINR and link stability and vehicle trajectory parameters like the duration of the vehicle in the platoon. This optimizes the traffic throughput while maintaining stability using the PID controller. The work tries to increase the vehicle inclusion time in the platoon while preserving the overall traffic throughput.« less
  5. Vehicle to Vehicle (V2V) communication allows vehicles to wirelessly exchange information on the surrounding environment and enables cooperative perception. It helps prevent accidents, increase the safety of the passengers, and improve the traffic flow efficiency. However, these benefits can only come when the vehicles can communicate with each other in a fast and reliable manner. Therefore, we investigated two areas to improve the communication quality of V2V: First, using beamforming to increase the bandwidth of V2V communication by establishing accurate and stable collaborative beam connection between vehicles on the road; second, ensuring scalable transmission to decrease the amount of datamore »to be transmitted, thus reduce the bandwidth requirements needed for collaborative perception of autonomous driving vehicles. Beamforming in V2V communication can be achieved by utilizing image-based and LIDAR’s 3D data-based vehicle detection and tracking. For vehicle detection and tracking simulation, we tested the Single Shot Multibox Detector deep learning-based object detection method that can achieve a mean Average Precision of 0.837 and the Kalman filter for tracking. For scalable transmission, we simulate the effect of varying pixel resolutions as well as different image compression techniques on the file size of data. Results show that without compression, the file size for only transmitting the bounding boxes containing detected object is up to 10 times less than the original file size. Similar results are also observed when the file is compressed by lossless and lossy compression to varying degrees. Based on these findings using existing databases, the impact of these compression methods and methods of effectively combining feature maps on the performance of object detection and tracking models will be further tested in the real-world autonomous driving system.« less