skip to main content

Title: Cooperative Highway Work Zone Merge Control Based on Reinforcement Learning in a Connected and Automated Environment
Given the aging infrastructure and the anticipated growing number of highway work zones in the U.S.A., it is important to investigate work zone merge control, which is critical for improving work zone safety and capacity. This paper proposes and evaluates a novel highway work zone merge control strategy based on cooperative driving behavior enabled by artificial intelligence. The proposed method assumes that all vehicles are fully automated, connected, and cooperative. It inserts two metering zones in the open lane to make space for merging vehicles in the closed lane. In addition, each vehicle in the closed lane learns how to adjust its longitudinal position optimally to find a safe gap in the open lane using an off-policy soft actor critic reinforcement learning (RL) algorithm, considering its surrounding traffic conditions. The learning results are captured in convolutional neural networks and used to control individual vehicles in the testing phase. By adding the metering zones and taking the locations, speeds, and accelerations of surrounding vehicles into account, cooperation among vehicles is implicitly considered. This RL-based model is trained and evaluated using a microscopic traffic simulator. The results show that this cooperative RL-based merge control significantly outperforms popular strategies such as late merge and early merge in terms of both mobility and safety measures. It also performs better than a strategy assuming all vehicles are equipped with cooperative adaptive cruise control.  more » « less
Award ID(s):
Author(s) / Creator(s):
; ;
Date Published:
Journal Name:
Transportation Research Record: Journal of the Transportation Research Board
Page Range / eLocation ID:
363 to 374
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. null (Ed.)
    This study focuses on how to improve the merge control prior to lane reduction points due to either accidents or constructions. A Cooperative Car-following and Merging (CCM) control strategy is proposed considering the coexistence of Automated Vehicles (AVs) and Human-4 Driven Vehicles (HDVs). CCM introduces a modified/generalized Cooperative Adaptive Cruise Control (CACC) for vehicle longitudinal control prior to lane reduction points. It also takes courtesy into account to ensure that AVs behave responsibly and ethically. CCM is evaluated using microscopic traffic simulation and compared with no control and CACC merge strategies. The results show that CCM consistently generates the lowest delays and highest throughputs approaching the theoretical capacity. Its safety benefits are also found to be significant based on vehicle trajectories and density maps. AVs in this study do not need to be fully automated and can be at Level-1 automation. CCM only requires automated longitudinal control such as Adaptive Cruise Control (ACC) and information sharing among vehicles, and ACC is already commercially available on many new vehicles. Also, it does not need 100% ACC penetration, presenting itself as a promising and practical solution for improving traffic operations in lane reduction transition areas such as highway work zones. 
    more » « less
  2. Temporary traffic control (TTC) in highway work zones has significant implications and challenges in terms of safety for road users and workers. Work zone workers are increasingly concerned about the risks they face due to their proximity to live traffic on the road. Drivers tend to be less aware of the risks faced by workers in highway work zones. The public should develop empathy to increase awareness about the danger construction workers are exposed in highway work zones. The research objective was to use virtual reality (VR) with a role-playing situation with almost complete sensory immersion in a controlled environment and a driving simulator to investigate if exposing drivers to the work hazards that highway construction workers typically encounter influences their behaviour while driving through work zones. The study compared the driving behaviours in the simulator between subjects sensitized using VR to the subjects who were not sensitized using VR. The simulation included the use of a GPS device that instructed drivers to turn on a road that was blocked by the TTC of the work zone as a distraction strategy. The results indicate that participants exposed to VR made safer driving decisions than participants without the VR intervention. The results suggest that drivers' empathy towards highway construction workers in a work zone can positively impact safety, communication, and well-being. 
    more » « less
  3. Autonomous Vehicles (AVs) are an emerging and highly impactful technology on today's roads. When assessing the performance of AVs, it is useful to study their improvement relative to common metrics such as fuel economy/emissions, safety, and congestion. But metrics of the vehicle's performance alone may not be complete; an AV that is affecting and reacting to a smart traffic light, for example, may improve its own performance, but may cause the same intersection to degrade the performance of other vehicles around the AV. Similar concerns arise in nearly all AV topics: platooning, light pre-emption, lane tracking, etc. Thus, the assessment of the vehicle's impacts on surrounding traffic is important, possibly even more important than the improvements enabled on the AV alone. But what boundary, or factors, define the vehicles, equipment, etc. “surrounding” an AV? The goal of this work is to characterize the boundary of vehicles “surrounding” an AV, referred to as Region of Influence, or ROI. Specifically, this work focuses on the problem that considering a perturbation is exerted into a traffic system, how far in time and space the perturbation from an AV’s decision can influence the surrounding system’s behavior. To achieve the goal, we utilized AIMSUN, a microscopic traffic simulator, to perform baseline and perturbed simulations. The ROI was evaluated by comparing trajectories of traffic surrounding the ego vehicle using different metrics, including difference in trajectories, Euclidian distance, rate of change of Euclidian distance, total number of lane changes over the whole simulation space versus time and total number of lane changes over the whole simulation time versus distance to ego vehicle. The results show that the ROI can be viewed from different perspectives using these metrics, and it is dependent on speed variance of the traffic. 
    more » « less
  4. null (Ed.)
    Connected and automated vehicles (CAVs) will undoubtedly transform many aspects of transportation systems in the future. In the meantime, transportation agencies must make investment and policy decisions to address the future needs of the transportation system. This research provides much-needed guidance for agencies about planning-level capacities in a CAV future and quantify Highway Capacity Manual (HCM) capacities as a function of CAV penetration rates and vehicle behaviors such as car-following, lane change, and merge. As a result of numerous uncertainties on CAV implementation policies, the study considers many scenarios including variations in parameters (including CAV gap/headway settings), roadway geometry, and traffic characteristics. More specifically, this study considers basic freeway, freeway merge, and freeway weaving segments in which various simulation scenarios are evaluated using two major CAV applications: cooperative adaptive cruise control and advanced merging. Data from microscopic traffic simulation are collected to develop capacity adjustment factors for CAVs. Results show that the existence of CAVs in the traffic stream can significantly enhance the roadway capacity (by as much as 35% to 40% under certain cases), not only on basic freeways but also on merge and weaving segments, as the CAV market penetration rate increases. The human driver behavior of baseline traffic also affects the capacity benefits, particularly at lower CAV market penetration rates. Finally, tables of capacity adjustment factors and corresponding regression models are developed for HCM implementation of the results of this study. 
    more » « less
  5. Model uncertainties are considered in a learning-based control framework that combines control dependent barrier function (CDBF), time-varying control barrier function (TCBF), and control Lyapunov function (CLF). Tracking control is achieved by CLF, while safety-critical constraints during tracking are guaranteed by CDBF and TCBF. A reinforcement learning (RL) method is applied to jointly learn model uncertainties that related to CDBF, TCBF, and CLF. The learning-based framework eventually formulates a quadratic programming (QP) with different constraints of CDBF, TCBF and CLF involving model uncertainties. It is the first time to apply the proposed learning-based framework for safety-guaranteed tracking control of automated vehicles with uncertainties. The control performances are validated for two different single-lane change maneuvers via Simulink/CarSim® co-simulation and compared for the cases with and without learning. Moreover, the learning effects are discussed through explainable constraints in the QP formulation. 
    more » « less