An official website of the United States government
Mean field games (MFG) and mean field control (MFC) are critical classes of multiagent models for the efficient analysis of massive populations of interacting agents. Their areas of application span topics in economics, finance, game theory, industrial engineering, crowd motion, and more. In this paper, we provide a flexible machine learning framework for the numerical solution of potential MFG and MFC models. State-of-the-art numerical methods for solving such problems utilize spatial discretization that leads to a curse of dimensionality. We approximately solve high-dimensional problems by combining Lagrangian and Eulerian viewpoints and leveraging recent advances from machine learning. More precisely, we work with a Lagrangian formulation of the problem and enforce the underlying Hamilton–Jacobi–Bellman (HJB) equation that is derived from the Eulerian formulation. Finally, a tailored neural network parameterization of the MFG/MFC solution helps us avoid any spatial discretization. Our numerical results include the approximate solution of 100-dimensional instances of optimal transport and crowd motion problems on a standard work station and a validation using a Eulerian solver in two dimensions. These results open the door to much-anticipated applications of MFG and MFC models that are beyond reach with existing numerical methods.
more »
« less
This content will become publicly available on November 1, 2025
A Game-Theoretic Framework for Generic Second-Order Traffic Flow Models Using Mean Field Games and Adversarial Inverse Reinforcement Learning
A traffic system can be interpreted as a multiagent system, wherein vehicles choose the most efficient driving approaches guided by interconnected goals or strategies. This paper aims to develop a family of mean field games (MFG) for generic second-order traffic flow models (GSOM), in which cars control individual velocity to optimize their objective functions. GSOMs do not generally assume that cars optimize self-interested objectives, so such a game-theoretic reinterpretation offers insights into the agents’ underlying behaviors. In general, an MFG allows one to model individuals on a microscopic level as rational utility-optimizing agents while translating rich microscopic behaviors to macroscopic models. Building on the MFG framework, we devise a new class of second-order traffic flow MFGs (i.e., GSOM-MFG), which control cars’ acceleration to ensure smooth velocity change. A fixed-point algorithm with fictitious play technique is developed to solve GSOM-MFG numerically. In numerical examples, different traffic patterns are presented under different cost functions. For real-world validation, we further use an inverse reinforcement learning approach (IRL) to uncover the underlying cost function on the next-generation simulation (NGSIM) data set. We formulate the problem of inferring cost functions as a min-max game and use an apprenticeship learning algorithm to solve for cost function coefficients. The results show that our proposed GSOM-MFG is a generic framework that can accommodate various cost functions. The Aw Rascle and Zhang (ARZ) and Light-Whitham-Richards (LWR) fundamental diagrams in traffic flow models belong to our GSOM-MFG when costs are specified. History: This paper has been accepted for the Transportation Science Special Issue on ISTTT25 Conference. Funding: X. Di is supported by the National Science Foundation [CAREER Award CMMI-1943998]. E. Iacomini is partially supported by the Italian Research Center on High-Performance Computing, Big Data and Quantum Computing (ICSC) funded by MUR Missione 4-Next Generation EU (NGEU) [Spoke 1 “FutureHPC & BigData”]. C. Segala and M. Herty thank the Deutsche Forschungsgemeinschaft (DFG) for financial support [Grants 320021702/GRK2326, 333849990/IRTG-2379, B04, B05, and B06 of 442047500/SFB1481, HE5386/18-1,19-2,22-1,23-1,25-1, ERS SFDdM035; Germany’s Excellence Strategy EXC-2023 Internet of Production 390621612; and Excellence Strategy of the Federal Government and the Länder]. Support through the EU DATAHYKING is also acknowledged. This work was also funded by the DFG [TRR 154, Mathematical Modelling, Simulation and Optimization Using the Example of Gas Networks, Projects C03 and C05, Project No. 239904186]. Moreover, E. Iacomini and C. Segala are members of the Indam GNCS (Italian National Group of Scientific Calculus).
more »
« less
- Award ID(s):
- 1943998
- PAR ID:
- 10560466
- Publisher / Repository:
- INFORMS
- Date Published:
- Journal Name:
- Transportation Science
- Volume:
- 58
- Issue:
- 6
- ISSN:
- 0041-1655
- Page Range / eLocation ID:
- 1403 to 1426
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
The framework of mean-field games (MFGs) is used for modeling the collective dynamics of large populations of non-cooperative decision-making agents. We formulate and analyze a kinetic MFG model for an interacting system of non-cooperative motile agents with inertial dynamics and finite-range interactions, where each agent is minimizing a biologically inspired cost function. By analyzing the associated coupled forward–backward in a time system of nonlinear Fokker–Planck and Hamilton–Jacobi–Bellman equations, we obtain conditions for closed-loop linear stability of the spatially homogeneous MFG equilibrium that corresponds to an ordered state with non-zero mean speed. Using a combination of analysis and numerical simulations, we show that when energetic cost of control is reduced below a critical value, this equilibrium loses stability, and the system transitions to a traveling wave solution. Our work provides a game-theoretic perspective to the problem of collective motion in non-equilibrium biological and bio-inspired systems.more » « less
-
This paper proposes a scalable learning framework to solve a system of coupled forward–backward partial differential equations (PDEs) arising from mean field games (MFGs). The MFG system incorporates a forward PDE to model the propagation of population dynamics and a backward PDE for a representative agent’s optimal control. Existing work mainly focus on solving the mean field game equilibrium (MFE) of the MFG system when given fixed boundary conditions, including the initial population state and terminal cost. To obtain MFE efficiently, particularly when the initial population density and terminal cost vary, we utilize a physics-informed neural operator (PINO) to tackle the forward–backward PDEs. A learning algorithm is devised and its performance is evaluated on one application domain, which is the autonomous driving velocity control. Numerical experiments show that our method can obtain the MFE accurately when given different initial distributions of vehicles. The PINO exhibits both memory efficiency and generalization capabilities compared to physics-informed neural networks (PINNs).more » « less
-
null (Ed.)This article applies the existing Markovian traffic assignment framework to novel traffic control strategies. In the Markovian traffic assignment framework, transition matrices are used to derive the traffic flow allocation. In contrast to the static traffic assignment, the framework only requires flow split ratio at every intersection, bypassing the need of computing path flow allocation. Consequently, compared to static traffic assignment, drivers’ routing behaviors can be modeled with fewer variables. As a result, it could be used to improve the efficiency of traffic management, especially in large scale applications. To begin with, the article introduces Markovian traffic assignment and connects it to the classic static traffic assignment. Then, the framework is extended to dynamic traffic assignment using microscopic traffic simulator Simulation of Urban Mobility (SUMO). In a case study, the framework is applied to a standard benchmark network, where optimal routing behaviors are independently learned through grid search, random search, and evolution strategies, under three different reward functions (network outflow, total vehicle hours of travel, and average marginal regret). The case study shows that the this novel traffic control strategy is promising, as Markov chain theory supports the ability to scale up to larger networks.more » « less
-
We consider a class of deterministic mean field games, where the state associated with each player evolves according to an ODE which is linear w.r.t. the control. Existence, uniqueness, and stability of solutions are studied from the point of viewof generic theory. Within a suitable topological space of dynamics and cost functionals, we prove that, for “nearly all” mean field games (in the Baire category sense) the best reply map is single-valued for a.e. player. As a consequence, the mean field game admits a strong (not randomized) solution. Examples are given of open sets of games admitting a single solution, and other open sets admitting multiple solutions. Further examples show the existence of an open set of MFG having a unique solution which is asymptotically stable w.r.t. the best reply map, and another open set of MFG having a unique solution which is unstable. We conclude with an example of a MFG with terminal constraints which does not have any solution, not even in the mild sense with randomized strategies.more » « less
