skip to main content


Title: Traffic signal control in mixed traffic environment based on advance decision and reinforcement learning
Abstract

Reinforcement learning-based traffic signal control systems (RLTSC) can enhance dynamic adaptability, save vehicle travelling time and promote intersection capacity. However, the existing RLTSC methods do not consider the driver's response time requirement, so the systems often face efficiency limitations and implementation difficulties. We propose the advance decision-making reinforcement learning traffic signal control (AD-RLTSC) algorithm to improve traffic efficiency while ensuring safety in mixed traffic environment. First, the relationship between the intersection perception range and the signal control period is established and the trust region state (TRS) is proposed. Then, the scalable state matrix is dynamically adjusted to decide the future signal light status. The decision will be displayed to the human-driven vehicles (HDVs) through the bi-countdown timer mechanism and sent to the nearby connected automated vehicles (CAVs) using the wireless network rather than be executed immediately. HDVs and CAVs optimize the driving speed based on the remaining green (or red) time. Besides, the Double Dueling Deep Q-learning Network algorithm is used for reinforcement learning training; a standardized reward is proposed to enhance the performance of intersection control and prioritized experience replay is adopted to improve sample utilization. The experimental results on vehicle micro-behaviour and traffic macro-efficiency showed that the proposed AD-RLTSC algorithm can simultaneously improve both traffic efficiency and traffic flow stability.

 
more » « less
PAR ID:
10383109
Author(s) / Creator(s):
; ;
Publisher / Repository:
Oxford University Press
Date Published:
Journal Name:
Transportation Safety and Environment
Volume:
4
Issue:
4
ISSN:
2631-4428
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. In the urban corridor with a mixed traffic composition of connected and automated vehicles (CAVs) alongside human-driven vehicles (HDVs), vehicle operations are intricately influenced by both individual driving behaviors and the presence of signalized intersections. Therefore, the development of a coordinated control strategy that effectively accommodates these dual factors becomes imperative to enhance the overall quality of traffic flow. This study proposes a bi-level structure crafted to decouple the joint effects of the vehicular driving behaviors and corridor signal offsets setting. The objective of this structure is to optimize both the average travel time (ATT) and fuel consumption (AFC). At the lower-level, three types of car-following models while considering driving modes are presented to illustrate the desired driving behaviors of HDVs and CAVs. Moreover, a trigonometry function method combined with a rolling horizon scheme is proposed to generate the eco-trajectory of CAVs in the mixed traffic flow. At the upper-level, a multi-objective optimization model for corridor signal offsets is formulated to minimize ATT and AFC based on the lower-level simulation outputs. Additionally, a revised Non-Dominated Sorting Genetic Algorithm II (NSGA-II) is adopted to identify the set of Pareto-optimal solutions for corridor signal offsets under different CAV penetration rates (CAV PRs). Numerical experiments are conducted within a corridor that encompasses three signalized intersections. The performance of our proposed eco-driving strategy is validated in comparison to the intelligent driver model (IDM) and green light optimal speed advisory (GLOSA) algorithm in single-vehicle simulation. Results show that our proposed strategy yields reduced travel time and fuel consumption to both IDM and GLOSA. Subsequently, the effectiveness of our proposed coordinated control strategy is validated across various CAV PRs. Results indicated that the optimal AFC can be reduced by 4.1%–32.2% with CAV PRs varying from 0.2 to 1, and the optimal ATT can be saved by 2.3% maximum. Furthermore, sensitivity analysis is conducted to evaluate the impact of CAV PRs and V/C ratios on the optimal ATT and AFC. 
    more » « less
  2. Abstract

    Rapid advances in vehicle automation and communication technologies enable connected autonomous vehicles (CAVs) to cross intersections cooperatively, which could significantly improve traffic throughput and safety at intersections. Virtual platooning, designed upon car‐following behavior, is one of the promising control methods to promote cooperative intersection crossing of CAVs. Nevertheless, demand variation raises safety and stability concerns when CAVs adopt a virtual platooning control approach. Along this line, this study proposes an adaptive vehicle control method to facilitate the formation of a virtual platoon and the cooperative crossing of CAVs, factoring demand variations at an isolated intersection. This study derives the stability conditions of virtual CAV platoons depending on the time‐varying traffic demand. Based on the derived stability conditions, an optimization model is proposed to adaptively control CAVs dynamics by balancing approaching traffic mobility and safety to enhance the reliability of cooperative crossing at intersections. The simulation results show that, compared to the nonadaptive control, our proposed method can increase the intersection throughput by 18.2%. Also, time‐to‐collision results highlight the advantages of the proposed adaptive control in securing traffic safety.

     
    more » « less
  3. Cooperative adaptive cruise control (CACC) is one of the popular connected and automated vehicle (CAV) applications for cooperative driving automation with combined connectivity and automation technologies to improve string stability. This study aimed to derive the string stability conditions of a CACC controller and analyze the impacts of CACC on string stability for both a fleet of homogeneous CAVs and for heterogeneous traffic with human-driven vehicles (HDVs), connected vehicles (CVs) with connectivity technologies only, and autonomous vehicles (AVs) with automation technologies only. We mathematically analyzed the impact of CACC on string stability for both homogeneous and heterogeneous traffic flow. We adopted parameters from literature for HDVs, CVs, and AVs for the heterogeneous traffic case. We found there was a minimum constant time headway required for each parameter design to ensure stability in homogeneous CACC traffic. In addition, the constant time headway and the length of control time interval had positive correlation with stability, but the control parameter had a negative correlation with stability. The numerical analysis also showed that CACC vehicles could maintain string stability better than CVs and AVs under low HDV market penetration rates for the mixed traffic case. 
    more » « less
  4. null (Ed.)
    In this paper, we investigate the intersection traffic management for connected automated vehicles (CAVs). In particular, a decentralized autonomous intersection management scheme that takes into account both the traffic efficiency and scheduling flexibility is proposed, which adopts a novel intersection–vehicle model to check conflicts among CAVs in the entire intersection area. In addition, a priority-based collision-avoidance rule is set to improve the performance of traffic efficiency and shorten the delays of emergency CAVs. Moreover, a multi-objective function is designed to obtain the optimal trajectories of CAVs, which considers ride comfort, velocities of CAVs, fuel consumption, and the constraints of safety, velocity, and acceleration. Simulation results demonstrate that our proposed scheme can achieve good performance in terms of traffic efficiency and shortening the delays of emergency CAVs. 
    more » « less
  5. In this paper, we showcase a framework for cooperative mixed traffic platooning that allows the platooning vehicles to realize multiple benefits from using vehicle-to- everything (V2X) communications and advanced controls on urban arterial roads. A mixed traffic platoon, in general, can be formulated by a lead and ego connected automated vehicles (CAVs) with one or more unconnected human-driven vehicles (UHVs) in between. As this platoon approaches an intersection, the lead vehicle uses signal phase and timing (SPaT) messages from the connected intersection to optimize its trajectory for travel time and energy efficiency as it passes through the intersection. These benefits carry over to the UHVs and the ego vehicle as they follow the lead vehicle. The ego vehicle then uses information from the lead vehicle received through basic safety messages (BSMs) to further optimize its safety, driving comfort, and energy consumption. This is accomplished by the recently designed cooperative adaptive cruise control with unconnected vehicles (CACCu). The performance benefits of our framework are proven and demonstrated by simulations using real-world platooning data from the CACC Field Operation Test (FOT) Dataset from the Netherlands. 
    more » « less