Search for: All records

Perimeter metering control has long been an active research topic since well-defined relationships between network productivity and usage, that is, network macroscopic fundamental diagrams (MFDs), were shown to be capable of describing regional traffic dynamics. Numerous methods have been proposed to solve perimeter metering control problems, but these generally require knowledge of the MFDs or detailed equations that govern traffic dynamics. Recently, a study applied model-free deep reinforcement learning (Deep-RL) methods to two-region perimeter control and found comparable performances to the model predictive control scheme, particularly when uncertainty exists. However, the proposed methods therein provide very low initial performances during the learning process, which limits their applicability to real life scenarios. Furthermore, the methods may not be scalable to more complicated networks with larger state and action spaces. To combat these issues, this paper proposes to integrate the domain control knowledge (DCK) of congestion dynamics into the agent designs for improved learning and control performances. A novel agent is also developed that builds on the Bang-Bang control policy. Two types of DCK are then presented to provide knowledge-guided exploration strategies for the agents such that they can explore around the most rewarding part of the action spaces. The results from extensive numerical experiments on two- and three-region urban networks show that integrating DCK can (a) effectively improve learning and control performances for Deep-RL agents, (b) enhance the agents’ resilience against various types of environment uncertainties, and (c) mitigate the scalability issue for the agents. 

Two key aggregated traffic models are the relationship between average network flow and density (known as the network or flow macroscopic fundamental diagram [flow-MFD]) and the relationship between trip completion and density (known as network exit function or the outflow-MFD [o-FMD]). The flow- and o-MFDs have been shown to be related by average network length and average trip distance under steady-state conditions. However, recent studies have demonstrated that these two relationships might have different patterns when traffic conditions are allowed to vary: the flow-MFD exhibits a clockwise hysteresis loop, while the o-MFD exhibits a counter-clockwise loop. One recent study attributes this behavior to the presence of bottlenecks within the network. The present paper demonstrates that this phenomenon may arise even without bottlenecks present and offers an alternative, but more general, explanation for these findings: a vehicle’s entire trip contributes to a network’s average flow, while only its end contributes to the trip completion rate. This lag can also be exaggerated by trips with different lengths, and it can lead to other patterns in the o-MFD such as figure-eight patterns. A simple arterial example is used to demonstrate this explanation and reveal the expected patterns, and they are also identified in real networks using empirical data. Then, simulations of a congestible ring network are used to unveil features that might increase or diminish the differences between the flow- and o-MFDs. Finally, more realistic simulations are used to confirm that these behaviors arise in real networks. 

This paper proposes a novel decentralized signal control algorithm that seeks to improve traffic delay equity, measured as the variation of delay experienced by individual vehicles. The proposed method extends the recently developed delay-based max pressure (MP) algorithm by using the sum of cumulative delay experienced by all vehicles that joined a given link as the metric for weight calculation. Doing so ensures the movements with lower traffic loads have a higher chance of being served as their delay increases. Three existing MP models are used as baseline models with which to compare the proposed algorithm in microscopic simulations of both a single intersection and a grid network. The results indicate that the proposed algorithm can improve the delay equity for various traffic conditions, especially for highly unbalanced traffic flows. Moreover, this improvement in delay equity does not come with a significant increase to average delay experienced by all vehicles. In fact, the average delay from the proposed algorithm is close to—and sometimes even lower than—the baseline models. Therefore, the proposed algorithm can maintain both objectives at the same time. In addition, the performance of the proposed control strategy was tested in a connected vehicle environment. The results show that the proposed algorithm outperforms the other baseline models in both reducing traffic delay and increasing delay equity when the penetration rate is less or equal to 60%, which would not be exceeded in reality in the near future.