 Award ID(s):
 1716673
 NSFPAR ID:
 10169099
 Date Published:
 Journal Name:
 ArXivorg
 ISSN:
 23318422
 Format(s):
 Medium: X
 Sponsoring Org:
 National Science Foundation
More Like this

We consider the problem of representing collective behavior of large popula tions and predicting the evolution of a population distribution over a discrete state space. A discrete time mean field game (MFG) is motivated as an interpretable model founded on game theory for understanding the aggregate effect of individ ual actions and predicting the temporal evolution of population distributions. We achieve a synthesis of MFG and Markov decision processes (MDP) by showing that a special MFG is reducible to an MDP. This enables us to broaden the scope of mean field game theory and infer MFG models of large realworld systems via deep inverse reinforcement learning. Our method learns both the reward function and forward dynamics of an MFG from real data, and we report the first empirical test of a mean field game model of a realworld social media population.more » « less

We investigate reinforcement learning for mean field control problems in discrete time, which can be viewed as Markov decision processes for a large number of exchangeable agents interacting in a mean field manner. Such problems arise, for instance when a large number of robots communicate through a central unit dispatching the optimal policy computed by minimizing the overall social cost. An approximate solution is obtained by learning the optimal policy of a generic agent interacting with the statistical distribution of the states of the other agents. We prove rigorously the convergence of exact and modelfree policy gradient methods in a meanfield linearquadratic setting. We also provide graphical evidence of the convergence based on implementations of our algorithms.more » « less

In this paper, we study the maximum principle of mean field type control problems when the volatility function depends on the state and its measure and also the control, by using our recently developed method in [Bensoussan, A., Huang, Z. and Yam, S. C. P. [2023] Control theory on Wasserstein space: A new approach to optimality conditions, Ann. Math. Sci. Appl.; Bensoussan, A., Tai, H. M. and Yam, S. C. P. [2023] Mean field type control problems, some Hilbertspacevalued FBSDEs, and related equations, preprint (2023), arXiv:2305.04019; Bensoussan, A. and Yam, S. C. P. [2019] Control problem on space of random variables and master equation, ESAIM Control Optim. Calc. Var. 25, 10]. Our method is to embed the mean field type control problem into a Hilbert space to bypass the evolution in the Wasserstein space. We here give a necessary condition and a sufficient condition for these control problems in Hilbert spaces, and we also derive a system of forward–backward stochastic differential equations.more » « less

In this paper, we investigate how the selfsynchronization property of a swarm of Kuramoto oscillators can be controlled and exploited to achieve target densities and target phase coherence. In the limit of an infinite number of oscillators, the collective dynamics of the agents’ density is described by a meanfield model in the form of a nonlocal PDE, where the nonlocality arises from the synchronization mechanism. In this meanfield setting, we introduce two spacetime dependent control inputs to affect the density of the oscillators: an angular velocity field that corresponds to a state feedback law for individual agents, and a control parameter that modulates the strength of agent interactions over space and time, i.e., a multiplicative control with respect to the integral nonlocal term. We frame the density tracking problem as a PDEconstrained optimization problem. The controlled synchronization and phaselocking are measured with classical polar order metrics. After establishing the mass conservation property of the meanfield model and bounds on its nonlocal term, a system of firstorder necessary conditions for optimality is recovered using a Lagrangian method. The optimality system, comprising a nonlocal PDE for the state dynamics equation, the respective nonlocal adjoint dynamics, and the Euler equation, is solved iteratively following a standard OptimizethenDiscretize approach and an efficient numerical solver based on spectral methods. We demonstrate our approach for each of the two control inputs in simulation.more » « less

Mean field games (MFG) and mean field control (MFC) are critical classes of multiagent models for the efficient analysis of massive populations of interacting agents. Their areas of application span topics in economics, finance, game theory, industrial engineering, crowd motion, and more. In this paper, we provide a flexible machine learning framework for the numerical solution of potential MFG and MFC models. Stateoftheart numerical methods for solving such problems utilize spatial discretization that leads to a curse of dimensionality. We approximately solve highdimensional problems by combining Lagrangian and Eulerian viewpoints and leveraging recent advances from machine learning. More precisely, we work with a Lagrangian formulation of the problem and enforce the underlying Hamilton–Jacobi–Bellman (HJB) equation that is derived from the Eulerian formulation. Finally, a tailored neural network parameterization of the MFG/MFC solution helps us avoid any spatial discretization. Our numerical results include the approximate solution of 100dimensional instances of optimal transport and crowd motion problems on a standard work station and a validation using a Eulerian solver in two dimensions. These results open the door to muchanticipated applications of MFG and MFC models that are beyond reach with existing numerical methods.