An official website of the United States government
The work explores how Reinforcement Learning can be used to re-time traffic signals around cordoned neighborhoods. An RL-based controller is developed by representing traffic states as graph-structured data and customizing corresponding neural network architectures to handle those data. The customizations enable the controller to: (i) model neighborhood-wide traffic based on directed-graph representations; (ii) use the representations to identify patterns in real-time traffic measurements; and (iii) capture those patterns to a spatial representation needed for selecting optimal cordon-metering rates. Input to the selection process also includes a total inflow to be admitted through a cordon. The rate is optimized in a separate process that is not part of the present work. Our RL-controller distributes that separately-optimized rate across the signalized street links that feed traffic through the cordon. The resulting metering rates vary from one feeder link to the next. The selection process can reoccur at short time intervals in response to changing traffic patterns. Once trained on a few cordons, the RL-controller can be deployed on cordons elsewhere in a city without additional training. This portability feature is confirmed via simulations of traffic on an idealized street network. The tests also indicate that the controller can reduce the network’s vehicle hours traveled well beyond what can be achieved via spatially-uniform cordon metering. The extra reductions in VHT are found to grow larger when traffic exhibits greater in-homogeneities over the network.
more »
« less
City-wide traffic control: Modeling impacts of cordon queues
Optimal cordon-metering rates are obtained using Macroscopic Fundamental Diagrams in combination with flow conservation laws. A model-predictive control algorithm is also used so that time-varying metering rates are generated based on their forecasted impacts. Our scalable algorithm can do this for an arbitrary number of cordoned neighborhoods within a city. Unlike its predecessors, the proposed model accounts for the time-varying constraining effects that cordon queues impose on a neighborhood’s circulating traffic, as those queues expand and recede over time. The model does so at every time step by approximating a neighborhood’s street space occupied by cordon queues, and re-scaling the MFD to describe the state of circulating traffic that results. The model also differentiates between saturated and under-saturated cordon-metering operations. Computer simulations of an idealized network show that these enhancements can substantially improve the predictions of both, the trip completion rates in a neighborhood and the rates that vehicles cross metered cordons. Optimal metering policies generated as a result are similarly shown to do a better job in reducing the Vehicle Hours Traveled on the network. The VHT reductions stemming from the proposed model and from its predecessors differed by as much as 14%.
more »
« less
- Award ID(s):
- 1760971
- PAR ID:
- 10156611
- Date Published:
- Journal Name:
- Transportation research Part C Emerging technologies
- Volume:
- 113
- ISSN:
- 1879-2359
- Page Range / eLocation ID:
- 164-175
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
null (Ed.)Abstract Unlimited access to a motorway network can, in overloaded conditions, cause a loss of throughput. Ramp metering, by controlling access to the motorway at onramps, can help avoid this loss of throughput. The queues that form at onramps are dependent on the metering rates chosen at the onramps, and these choices affect how the capacities of different motorway sections are shared amongst competing flows. In this paper we perform an analytical study of a fluid, or differential equation, model of a linear network topology with onramp queues. The model allows for adaptive arrivals, in the sense that the rate at which external traffic enters the queue at an onramp can depend on the current perceived delay in that queue. The model also includes a ramp metering policy which uses global onramp queue length information to determine the rate at which traffic enters the motorway from each onramp. This ramp metering policy minimizes the maximum delay over all onramps and produces equal delay times over many onramps. The paper characterizes both the dynamics and the equilibrium behavior of the system under this policy. While we consider an idealized model that leaves out many practical details, an aim of the paper is to develop analytical methods that yield interesting qualitative insights and might be adapted to more general contexts. The paper can be considered as a step in developing an analytical approach towards studying more complex network topologies and incorporating other model features.more » « less
-
Service systems abound with queues, but the most natural direct models are often time-varying queues, which may require nonstandard analysis methods beyond stochastic textbooks. This paper provides an overview of time-varying queues. Most of the recent literature concerns many-server queues, which arise in large-scale service systems, such as in customer contact centers and hospital emergency departments, but there also has been some new work on single-server queues with time-varying arrivals, which arise in some settings, such as airplanes coming to land at an airport, cars coming to a traffic intersection and medical staff waiting for the availability of special operating rooms in a hospital. The understanding of many-server queues and single-server queues is enhanced by heavy-traffic limits, which have been extended to time-varying models as well as stationary models.more » « less
-
Recent studies have leveraged the existence of network macroscopic fundamental diagrams (MFD) to develop regional control strategies for urban traffic networks. Existing MFD-based control strategies focus on vehicle movement within and across regions of an urban network and do not consider how freeway traffic can be controlled to improve overall traffic operations in mixed freeway and urban networks. The purpose of this study is to develop a coordinated traffic management scheme that simultaneously implements perimeter flow control on an urban network and variable speed limits (VSL) on a freeway to reduce total travel time in such a mixed network. By slowing down vehicles traveling along the freeway, VSL can effectively meter traffic exiting the freeway into the urban network. This can be particularly useful since freeways often have large storage capacities and vehicles accumulating on freeways might be less disruptive to overall system operations than on urban streets. VSL can also be used to change where freeway vehicles enter the urban network to benefit the entire system. The combined control strategy is implemented in a model predictive control framework with several realistic constraints, such as gradual reductions in freeway speed limit. Numerical tests suggest that the combined implementation of VSL and perimeter metering control can improve traffic operations compared with perimeter metering alone.more » « less
-
Throughout the past decades, many different versions of the widely used first-order Cell-Transmission Model (CTM) have been proposed for optimal traffic control. Highway traffic management techniques such as Ramp Metering (RM) are typically designed based on an optimization problem with nonlinear constraints originating in the flow-density relation of the Fundamental Diagram (FD). Most of the extended CTM versions are based on the trapezoidal approximation of the flow-density relation of the Fundamental Diagram (FD) in an attempt to simplify the optimization problem. However, this relation is naturally nonlinear, and crude approximations can greatly impact the efficiency of the optimization solution. In this study, we propose a class of extended CTMs that are based on piecewise affine approximations of the flow-density relation such that (a) the integrated squared error with respect to the true relation is greatly reduced in comparison to the trapezoidal approximation, and (b) the optimization problem remains tractable for real-time application of ramp metering optimal controllers. A two-step identification method is used to approximate the FD with piecewise affine functions resulting in what we refer to as PWA-CTMs. The proposed models are evaluated by the performance of the optimal ramp metering controllers, e.g. using the widely used PI-ALINEA approach, in complex highway traffic networks. Simulation results show that the optimization problems based on the PWA-CTMs require less computation time compared to other CTM extensions while achieving higher accuracy of the flow and density evolution. Hence, the proposed PWA-CTMs constitute one of the best approximation approaches for first-order traffic flow models that can be used in more general and challenging modeling and control applications.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/https://doi.org/10.1016/j.trc.2019.04.024
