skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: Cooperative Highway Work Zone Merge Control Based on Reinforcement Learning in a Connected and Automated Environment
Given the aging infrastructure and the anticipated growing number of highway work zones in the U.S.A., it is important to investigate work zone merge control, which is critical for improving work zone safety and capacity. This paper proposes and evaluates a novel highway work zone merge control strategy based on cooperative driving behavior enabled by artificial intelligence. The proposed method assumes that all vehicles are fully automated, connected, and cooperative. It inserts two metering zones in the open lane to make space for merging vehicles in the closed lane. In addition, each vehicle in the closed lane learns how to adjust its longitudinal position optimally to find a safe gap in the open lane using an off-policy soft actor critic reinforcement learning (RL) algorithm, considering its surrounding traffic conditions. The learning results are captured in convolutional neural networks and used to control individual vehicles in the testing phase. By adding the metering zones and taking the locations, speeds, and accelerations of surrounding vehicles into account, cooperation among vehicles is implicitly considered. This RL-based model is trained and evaluated using a microscopic traffic simulator. The results show that this cooperative RL-based merge control significantly outperforms popular strategies such as late merge and early merge in terms of both mobility and safety measures. It also performs better than a strategy assuming all vehicles are equipped with cooperative adaptive cruise control.  more » « less
Award ID(s):
1734521
PAR ID:
10257285
Author(s) / Creator(s):
; ;
Date Published:
Journal Name:
Transportation Research Record: Journal of the Transportation Research Board
Volume:
2674
Issue:
10
ISSN:
0361-1981
Page Range / eLocation ID:
363 to 374
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. null (Ed.)
    This study focuses on how to improve the merge control prior to lane reduction points due to either accidents or constructions. A Cooperative Car-following and Merging (CCM) control strategy is proposed considering the coexistence of Automated Vehicles (AVs) and Human-4 Driven Vehicles (HDVs). CCM introduces a modified/generalized Cooperative Adaptive Cruise Control (CACC) for vehicle longitudinal control prior to lane reduction points. It also takes courtesy into account to ensure that AVs behave responsibly and ethically. CCM is evaluated using microscopic traffic simulation and compared with no control and CACC merge strategies. The results show that CCM consistently generates the lowest delays and highest throughputs approaching the theoretical capacity. Its safety benefits are also found to be significant based on vehicle trajectories and density maps. AVs in this study do not need to be fully automated and can be at Level-1 automation. CCM only requires automated longitudinal control such as Adaptive Cruise Control (ACC) and information sharing among vehicles, and ACC is already commercially available on many new vehicles. Also, it does not need 100% ACC penetration, presenting itself as a promising and practical solution for improving traffic operations in lane reduction transition areas such as highway work zones. 
    more » « less
  2. This paper proposes a reinforcement learning-based framework for mandatory lane changing of automated vehicles in a non-cooperative environment. The objective is to create a reinforcement learning (RL) agent that is able to perform lane-changing maneuvers successfully and efficiently and with minimal impact on traffic flow in the target lane. For this purpose, this study utilizes the double deep Q-learning algorithm structure, which takes relevant traffic states as input and outputs the optimal actions (policy) for the automated vehicle. We put forward a realistic approach for dealing with this problem where, for instance, actions selected by the automated vehicle include steering angles and acceleration/deceleration values. We show that the RL agent is able to learn optimal policies for the different scenarios it encounters and performs the lane-changing task safely and efficiently. This work illustrates the potential of RL as a flexible framework for developing superior and more comprehensive lane-changing models that take into consideration multiple aspects of the road environment and seek to improve traffic flow as a whole. 
    more » « less
  3. Temporary traffic control (TTC) in highway work zones has significant implications and challenges in terms of safety for road users and workers. Work zone workers are increasingly concerned about the risks they face due to their proximity to live traffic on the road. Drivers tend to be less aware of the risks faced by workers in highway work zones. The public should develop empathy to increase awareness about the danger construction workers are exposed in highway work zones. The research objective was to use virtual reality (VR) with a role-playing situation with almost complete sensory immersion in a controlled environment and a driving simulator to investigate if exposing drivers to the work hazards that highway construction workers typically encounter influences their behaviour while driving through work zones. The study compared the driving behaviours in the simulator between subjects sensitized using VR to the subjects who were not sensitized using VR. The simulation included the use of a GPS device that instructed drivers to turn on a road that was blocked by the TTC of the work zone as a distraction strategy. The results indicate that participants exposed to VR made safer driving decisions than participants without the VR intervention. The results suggest that drivers' empathy towards highway construction workers in a work zone can positively impact safety, communication, and well-being. 
    more » « less
  4. Autonomous Vehicles (AVs) are an emerging and highly impactful technology on today's roads. When assessing the performance of AVs, it is useful to study their improvement relative to common metrics such as fuel economy/emissions, safety, and congestion. But metrics of the vehicle's performance alone may not be complete; an AV that is affecting and reacting to a smart traffic light, for example, may improve its own performance, but may cause the same intersection to degrade the performance of other vehicles around the AV. Similar concerns arise in nearly all AV topics: platooning, light pre-emption, lane tracking, etc. Thus, the assessment of the vehicle's impacts on surrounding traffic is important, possibly even more important than the improvements enabled on the AV alone. But what boundary, or factors, define the vehicles, equipment, etc. “surrounding” an AV? The goal of this work is to characterize the boundary of vehicles “surrounding” an AV, referred to as Region of Influence, or ROI. Specifically, this work focuses on the problem that considering a perturbation is exerted into a traffic system, how far in time and space the perturbation from an AV’s decision can influence the surrounding system’s behavior. To achieve the goal, we utilized AIMSUN, a microscopic traffic simulator, to perform baseline and perturbed simulations. The ROI was evaluated by comparing trajectories of traffic surrounding the ego vehicle using different metrics, including difference in trajectories, Euclidian distance, rate of change of Euclidian distance, total number of lane changes over the whole simulation space versus time and total number of lane changes over the whole simulation time versus distance to ego vehicle. The results show that the ROI can be viewed from different perspectives using these metrics, and it is dependent on speed variance of the traffic. 
    more » « less
  5. null (Ed.)
    Connected and automated vehicles (CAVs) will undoubtedly transform many aspects of transportation systems in the future. In the meantime, transportation agencies must make investment and policy decisions to address the future needs of the transportation system. This research provides much-needed guidance for agencies about planning-level capacities in a CAV future and quantify Highway Capacity Manual (HCM) capacities as a function of CAV penetration rates and vehicle behaviors such as car-following, lane change, and merge. As a result of numerous uncertainties on CAV implementation policies, the study considers many scenarios including variations in parameters (including CAV gap/headway settings), roadway geometry, and traffic characteristics. More specifically, this study considers basic freeway, freeway merge, and freeway weaving segments in which various simulation scenarios are evaluated using two major CAV applications: cooperative adaptive cruise control and advanced merging. Data from microscopic traffic simulation are collected to develop capacity adjustment factors for CAVs. Results show that the existence of CAVs in the traffic stream can significantly enhance the roadway capacity (by as much as 35% to 40% under certain cases), not only on basic freeways but also on merge and weaving segments, as the CAV market penetration rate increases. The human driver behavior of baseline traffic also affects the capacity benefits, particularly at lower CAV market penetration rates. Finally, tables of capacity adjustment factors and corresponding regression models are developed for HCM implementation of the results of this study. 
    more » « less