skip to main content

Explore Research Products in the NSF-PAR

Title: Cooperative Highway Work Zone Merge Control Based on Reinforcement Learning in a Connected and Automated Environment
Given the aging infrastructure and the anticipated growing number of highway work zones in the U.S.A., it is important to investigate work zone merge control, which is critical for improving work zone safety and capacity. This paper proposes and evaluates a novel highway work zone merge control strategy based on cooperative driving behavior enabled by artificial intelligence. The proposed method assumes that all vehicles are fully automated, connected, and cooperative. It inserts two metering zones in the open lane to make space for merging vehicles in the closed lane. In addition, each vehicle in the closed lane learns how to adjust its longitudinal position optimally to find a safe gap in the open lane using an off-policy soft actor critic reinforcement learning (RL) algorithm, considering its surrounding traffic conditions. The learning results are captured in convolutional neural networks and used to control individual vehicles in the testing phase. By adding the metering zones and taking the locations, speeds, and accelerations of surrounding vehicles into account, cooperation among vehicles is implicitly considered. This RL-based model is trained and evaluated using a microscopic traffic simulator. The results show that this cooperative RL-based merge control significantly outperforms popular strategies such as late merge and early merge in terms of both mobility and safety measures. It also performs better than a strategy assuming all vehicles are equipped with cooperative adaptive cruise control.  more » « less
Award ID(s):
1734521
NSF-PAR ID:
10257285
Author(s) / Creator(s):
; ;
Date Published:
Journal Name:
Transportation Research Record: Journal of the Transportation Research Board
Volume:
2674
Issue:
10
ISSN:
0361-1981
Page Range / eLocation ID:
363 to 374
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ( , Proceedings of the 100th Transportation Research Board Annual Meeting)
    null (Ed.)
    This study focuses on how to improve the merge control prior to lane reduction points due to either accidents or constructions. A Cooperative Car-following and Merging (CCM) control strategy is proposed considering the coexistence of Automated Vehicles (AVs) and Human-4 Driven Vehicles (HDVs). CCM introduces a modified/generalized Cooperative Adaptive Cruise Control (CACC) for vehicle longitudinal control prior to lane reduction points. It also takes courtesy into account to ensure that AVs behave responsibly and ethically. CCM is evaluated using microscopic traffic simulation and compared with no control and CACC merge strategies. The results show that CCM consistently generates the lowest delays and highest throughputs approaching the theoretical capacity. Its safety benefits are also found to be significant based on vehicle trajectories and density maps. AVs in this study do not need to be fully automated and can be at Level-1 automation. CCM only requires automated longitudinal control such as Adaptive Cruise Control (ACC) and information sharing among vehicles, and ACC is already commercially available on many new vehicles. Also, it does not need 100% ACC penetration, presenting itself as a promising and practical solution for improving traffic operations in lane reduction transition areas such as highway work zones. 
    more » « less
  2. ; ; ; ; ; ; ( , Transportation Research Record: Journal of the Transportation Research Board)
    null (Ed.)
    Connected and automated vehicles (CAVs) will undoubtedly transform many aspects of transportation systems in the future. In the meantime, transportation agencies must make investment and policy decisions to address the future needs of the transportation system. This research provides much-needed guidance for agencies about planning-level capacities in a CAV future and quantify Highway Capacity Manual (HCM) capacities as a function of CAV penetration rates and vehicle behaviors such as car-following, lane change, and merge. As a result of numerous uncertainties on CAV implementation policies, the study considers many scenarios including variations in parameters (including CAV gap/headway settings), roadway geometry, and traffic characteristics. More specifically, this study considers basic freeway, freeway merge, and freeway weaving segments in which various simulation scenarios are evaluated using two major CAV applications: cooperative adaptive cruise control and advanced merging. Data from microscopic traffic simulation are collected to develop capacity adjustment factors for CAVs. Results show that the existence of CAVs in the traffic stream can significantly enhance the roadway capacity (by as much as 35% to 40% under certain cases), not only on basic freeways but also on merge and weaving segments, as the CAV market penetration rate increases. The human driver behavior of baseline traffic also affects the capacity benefits, particularly at lower CAV market penetration rates. Finally, tables of capacity adjustment factors and corresponding regression models are developed for HCM implementation of the results of this study. 
    more » « less
  3. ( , XXII Conferencia Panamericana de Ingeniería y Logística de Tránsito y Transporte)
    Temporary traffic control (TTC) in highway work zones has significant implications and challenges in terms of safety for road users and workers. Work zone workers are increasingly concerned about the risks they face due to their proximity to live traffic on the road. Drivers tend to be less aware of the risks faced by workers in highway work zones. The public should develop empathy to increase awareness about the danger construction workers are exposed in highway work zones. The research objective was to use virtual reality (VR) with a role-playing situation with almost complete sensory immersion in a controlled environment and a driving simulator to investigate if exposing drivers to the work hazards that highway construction workers typically encounter influences their behaviour while driving through work zones. The study compared the driving behaviours in the simulator between subjects sensitized using VR to the subjects who were not sensitized using VR. The simulation included the use of a GPS device that instructed drivers to turn on a road that was blocked by the TTC of the work zone as a distraction strategy. The results indicate that participants exposed to VR made safer driving decisions than participants without the VR intervention. The results suggest that drivers' empathy towards highway construction workers in a work zone can positively impact safety, communication, and well-being. 
    more » « less
  4. ; ; ; ; ( , 2022 Road Safety and Simulation International Conference (RSS))
    Autonomous Vehicles (AVs) are an emerging and highly impactful technology on today's roads. When assessing the performance of AVs, it is useful to study their improvement relative to common metrics such as fuel economy/emissions, safety, and congestion. But metrics of the vehicle's performance alone may not be complete; an AV that is affecting and reacting to a smart traffic light, for example, may improve its own performance, but may cause the same intersection to degrade the performance of other vehicles around the AV. Similar concerns arise in nearly all AV topics: platooning, light pre-emption, lane tracking, etc. Thus, the assessment of the vehicle's impacts on surrounding traffic is important, possibly even more important than the improvements enabled on the AV alone. But what boundary, or factors, define the vehicles, equipment, etc. “surrounding” an AV? The goal of this work is to characterize the boundary of vehicles “surrounding” an AV, referred to as Region of Influence, or ROI. Specifically, this work focuses on the problem that considering a perturbation is exerted into a traffic system, how far in time and space the perturbation from an AV’s decision can influence the surrounding system’s behavior. To achieve the goal, we utilized AIMSUN, a microscopic traffic simulator, to perform baseline and perturbed simulations. The ROI was evaluated by comparing trajectories of traffic surrounding the ego vehicle using different metrics, including difference in trajectories, Euclidian distance, rate of change of Euclidian distance, total number of lane changes over the whole simulation space versus time and total number of lane changes over the whole simulation time versus distance to ego vehicle. The results show that the ROI can be viewed from different perspectives using these metrics, and it is dependent on speed variance of the traffic. 
    more » « less
  5. ; ( , Transportation Research Record: Journal of the Transportation Research Board)

    Connected and automated vehicles (CAVs) extend urban traffic control from temporal to spatiotemporal by enabling the control of CAV trajectories. Most of the existing studies on CAV trajectory planning only consider longitudinal behaviors (i.e., in-lane driving), or assume that the lane changing can be done instantaneously. The resultant CAV trajectories are not realistic and cannot be executed at the vehicle level. The aim of this paper is to propose a full trajectory planning model that considers both in-lane driving and lane changing maneuvers. The trajectory generation problem is modeled as an optimization problem and the cost function considers multiple driving features including safety, efficiency, and comfort. Ten features are selected in the cost function to capture both in-lane driving and lane changing behaviors. One major challenge in generating a trajectory that reflects certain driving policies is to balance the weights of different features in the cost function. To address this challenge, it is proposed to optimize the weights of the cost function by imitation learning. Maximum entropy inverse reinforcement learning is applied to obtain the optimal weight for each feature and then CAV trajectories are generated with the learned weights. Experiments using the Next Generation Simulation (NGSIM) dataset show that the generated trajectory is very close to the original trajectory with regard to the Euclidean distance displacement, with a mean average error of less than 1 m. Meanwhile, the generated trajectories can maintain safety gaps with surrounding vehicles and have comparable fuel consumption.

     
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email