Mean field games (MFG) and mean field control (MFC) are critical classes of multiagent models for the efficient analysis of massive populations of interacting agents. Their areas of application span topics in economics, finance, game theory, industrial engineering, crowd motion, and more. In this paper, we provide a flexible machine learning framework for the numerical solution of potential MFG and MFC models. Stateoftheart numerical methods for solving such problems utilize spatial discretization that leads to a curse of dimensionality. We approximately solve highdimensional problems by combining Lagrangian and Eulerian viewpoints and leveraging recent advances from machine learning. More precisely, we work with a Lagrangian formulation of the problem and enforce the underlying Hamilton–Jacobi–Bellman (HJB) equation that is derived from the Eulerian formulation. Finally, a tailored neural network parameterization of the MFG/MFC solution helps us avoid any spatial discretization. Our numerical results include the approximate solution of 100dimensional instances of optimal transport and crowd motion problems on a standard work station and a validation using a Eulerian solver in two dimensions. These results open the door to muchanticipated applications of MFG and MFC models that are beyond reach with existing numerical methods.
 Award ID(s):
 1751636
 Publication Date:
 NSFPAR ID:
 10143814
 Journal Name:
 Proceedings of the National Academy of Sciences
 Volume:
 117
 Issue:
 17
 Page Range or eLocationID:
 p. 91839193
 ISSN:
 00278424
 Publisher:
 Proceedings of the National Academy of Sciences
 Sponsoring Org:
 National Science Foundation
More Like this

We develop a general reinforcement learning framework for mean field control (MFC) problems. Such problems arise for instance as the limit of collaborative multiagent control problems when the number of agents is very large. The asymptotic problem can be phrased as the optimal control of a nonlinear dynamics. This can also be viewed as a Markov decision process (MDP) but the key difference with the usual RL setup is that the dynamics and the reward now depend on the state's probability distribution itself. Alternatively, it can be recast as a MDP on the Wasserstein space of measures. In this work,more »

Prior mathematical work of Constantin & Iyer ( Commun. Pure Appl. Maths , vol. 61, 2008, pp. 330–345; Ann. Appl. Probab. , vol. 21, 2011, pp. 1466–1492) has shown that incompressible Navier–Stokes solutions possess infinitely many stochastic Lagrangian conservation laws for vorticity, backward in time, which generalize the invariants of Cauchy ( Sciences mathématiques et physique , vol. I, 1815, pp. 33–73) for smooth Euler solutions. We reformulate this theory for the case of wallbounded flows by appealing to the Kuz'min ( Phys. Lett. A , vol. 96, 1983, pp. 88–90)–Oseledets ( Russ. Math. Surv. , vol. 44, 1989, p.more »

This project investigates numerical methods for solving fully coupled forwardbackward stochastic differential equations (FBSDEs) of McKeanVlasov type. Having numerical solvers for such mean field FBSDEs is of interest because of the potential application of these equations to optimization problems over a large population, say for instance mean field games (MFG) and optimal mean field control problems . Theory for this kind of problems has met with great success since the early works on mean field games by Lasry and Lions, see [29], and by Huang, Caines, and Malhame, see [26]. Generally speaking, the purpose is to understand the continuum limitmore »

This project investigates numerical methods for solving fully coupled forwardbackward stochastic differential equations (FBSDEs) of McKeanVlasov type. Having numerical solvers for such mean field FBSDEs is of interest because of the potential application of these equations to optimization problems over a large population, say for instance mean field games (MFG) and optimal mean field control problems. Theory for this kind of problems has met with great success since the early works on mean field games by Lasry and Lions, and by Huang, Caines, and Malhame. Generally speaking, the purpose is to understand the continuum limit of optimizers or of equilibriamore »

Ocean volume and tracer transports are commonly computed on density surfaces because doing so approximates the semiLagrangian mean advective transport. The resulting densityaveraged transport can be related approximately to Eulerianaveraged quantities via the Temporal Residual Mean (TRM), valid in the limit of small isopycnal height fluctuations. This article builds on a formulation of the TRM for volume fluxes within Neutral Density surfaces, (the “NDTRM”), selected because Neutral Density surfaces are constructed to be as neutral as possible while still forming welldefined surfaces. This article derives a TRM, referred to as the “Neutral TRM” (NTRM), that approximates volume fluxes within surfacesmore »