The work explores how Reinforcement Learning can be used to re-time traffic signals around cordoned neighborhoods. An RL-based controller is developed by representing traffic states as graph-structured data and customizing corresponding neural network architectures to handle those data. The customizations enable the controller to: (i) model neighborhood-wide traffic based on directed-graph representations; (ii) use the representations to identify patterns in real-time traffic measurements; and (iii) capture those patterns to a spatial representation needed for selecting optimal cordon-metering rates. Input to the selection process also includes a total inflow to be admitted through a cordon. The rate is optimized in a separate process that is not part of the present work. Our RL-controller distributes that separately-optimized rate across the signalized street links that feed traffic through the cordon. The resulting metering rates vary from one feeder link to the next. The selection process can reoccur at short time intervals in response to changing traffic patterns. Once trained on a few cordons, the RL-controller can be deployed on cordons elsewhere in a city without additional training.
This portability feature is confirmed via simulations of traffic on an idealized street network. The tests also indicate that the controller can reduce the network’s vehicle hours traveled well beyond what can be achieved via spatially-uniform cordon metering. The extra reductions in VHT are found to grow larger when traffic exhibits greater in-homogeneities over the network.
more »
« less
City-wide traffic control: Modeling impacts of cordon queues
Optimal cordon-metering rates are obtained using Macroscopic Fundamental Diagrams in combination with flow conservation laws. A model-predictive control algorithm is also used so that time-varying metering rates are generated based on their forecasted impacts. Our scalable algorithm can do this for an arbitrary number of cordoned neighborhoods within a city. Unlike its predecessors, the proposed model accounts for the time-varying constraining effects that cordon queues impose on a neighborhood’s circulating traffic, as those queues expand and recede over time. The model does so at every time step by approximating a neighborhood’s street space occupied by cordon queues, and re-scaling the MFD to describe the state of circulating traffic that results. The model also differentiates between saturated and under-saturated cordon-metering operations. Computer simulations of an idealized network show that these enhancements can substantially improve the predictions of both, the trip completion rates in a neighborhood and the rates that vehicles cross metered cordons. Optimal metering policies generated as a result are similarly shown to do a better job in reducing the Vehicle Hours Traveled on the network. The VHT reductions stemming from the proposed model and from its predecessors differed by as much as 14%.
more »
« less
- Award ID(s):
- 1760971
- NSF-PAR ID:
- 10156611
- Date Published:
- Journal Name:
- Transportation research Part C Emerging technologies
- Volume:
- 113
- ISSN:
- 1879-2359
- Page Range / eLocation ID:
- 164-175
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
null (Ed.)Abstract Unlimited access to a motorway network can, in overloaded conditions, cause a loss of throughput. Ramp metering, by controlling access to the motorway at onramps, can help avoid this loss of throughput. The queues that form at onramps are dependent on the metering rates chosen at the onramps, and these choices affect how the capacities of different motorway sections are shared amongst competing flows. In this paper we perform an analytical study of a fluid, or differential equation, model of a linear network topology with onramp queues. The model allows for adaptive arrivals, in the sense that the rate at which external traffic enters the queue at an onramp can depend on the current perceived delay in that queue. The model also includes a ramp metering policy which uses global onramp queue length information to determine the rate at which traffic enters the motorway from each onramp. This ramp metering policy minimizes the maximum delay over all onramps and produces equal delay times over many onramps. The paper characterizes both the dynamics and the equilibrium behavior of the system under this policy. While we consider an idealized model that leaves out many practical details, an aim of the paper is to develop analytical methods that yield interesting qualitative insights and might be adapted to more general contexts. The paper can be considered as a step in developing an analytical approach towards studying more complex network topologies and incorporating other model features.more » « less
-
null (Ed.)Although urban transit systems (UTS) often have fixed vehicle capacity and relatively constant departure headways, they may need to accommodate dramatically fluctuating passenger demands over space and time, resulting in either excessive passenger waiting or vehicle capacity and energy waste. Therefore, on the one hand, optimal operations of UTS rely on accurate modeling of passenger queuing dynamics, which is particularly complex on a multistop transit corridor. On the other hand, capacities of transit vehicles can be made variable and adaptive to time-variant passenger demand so as to minimize energy waste, especially with the emergence of modular vehicle technologies. This paper investigates operations of a multistop transit corridor in which vehicles may have different capacities across dispatches. We specify skewed time coordinates to simplify the problem structure while incorporating traffic congestion. Then, we propose a mixed integer linear programming model that determines the optimal dynamic headways and vehicle capacities over the study time horizon to minimize the overall system cost for the transit corridor. In particular, the proposed model considers a realistic multistop first-in, first-out (MSFIFO) rule that gives the same boarding priority to passengers arriving at a station in the same time interval yet with different destinations. A customized dynamic programming (DP) algorithm is proposed to solve this model efficiently. To circumvent the rapid increase of the state space of a typical DP algorithm, we analyze the theoretical properties of the investigated problem and identify upper and lower bounds to a feasible solution. The bounds largely reduce the state space during the DP iterations and greatly improve the efficiency of the proposed DP algorithm. The state dimensions are also reduced with the MSFIFO rule such that all queues with different destinations at the same origin can be tracked with a single boarding position state variable at each stage. A hypothetical example and a real-world case study show that the proposed DP algorithm greatly outperforms a state-of-the-art commercial solver (Gurobi) in both solution quality and time.more » « less
-
Dynamic user equilibrium (DUE) is the most widely studied form of dynamic traffic assignment (DTA), in which road travelers engage in a non-cooperative Nash-like game with departure time and route choices. DUE models describe and predict the time-varying traffic flows on a network consistent with traffic flow theory and travel behavior. This paper documents theoretical and numerical advances in synthesizing traffic flow theory and DUE modeling, by presenting a holistic computational theory of DUE, which is numerically implemented in a MATLAB package. In particular, the dynamic network loading (DNL) sub-problem is formulated as a system of differential algebraic equations based on the Lighthill-Whitham-Richards fluid dynamic model, which captures the formation, propagation and dissipation of physical queues as well as vehicle spillback on networks. Then, the fixed-point algorithm is employed to solve the DUE problems with simultaneous route and departure time choices on several large-scale networks. We make openly available the MATLAB package, which can be used to solve DUE problems on user-defined networks, aiming to not only facilitate benchmarking a wide range of DUE algorithms and solutions, but also offer researchers a platform to further develop their own models and applications. The MATLAB package and computational examples are available online.more » « less
-
Recent studies have leveraged the existence of network macroscopic fundamental diagrams (MFD) to develop regional control strategies for urban traffic networks. Existing MFD-based control strategies focus on vehicle movement within and across regions of an urban network and do not consider how freeway traffic can be controlled to improve overall traffic operations in mixed freeway and urban networks. The purpose of this study is to develop a coordinated traffic management scheme that simultaneously implements perimeter flow control on an urban network and variable speed limits (VSL) on a freeway to reduce total travel time in such a mixed network. By slowing down vehicles traveling along the freeway, VSL can effectively meter traffic exiting the freeway into the urban network. This can be particularly useful since freeways often have large storage capacities and vehicles accumulating on freeways might be less disruptive to overall system operations than on urban streets. VSL can also be used to change where freeway vehicles enter the urban network to benefit the entire system. The combined control strategy is implemented in a model predictive control framework with several realistic constraints, such as gradual reductions in freeway speed limit. Numerical tests suggest that the combined implementation of VSL and perimeter metering control can improve traffic operations compared with perimeter metering alone.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/https://doi.org/10.1016/j.trc.2019.04.024
