More Like this

1. Reinforcement Learning for Mean Field Games with Strategic Complementarities (AISTATS 2021)

   https://doi.org/PMLR 130:2458-2466  

   Lee, Kiyeob; Rengarajan, Desik; Kalathil, Dileep; Shakkottai, Srinivas (January 2021, Proceedings of Machine Learning Research)

   null (Ed.)

   Mean Field Games (MFG) are the class of games with a very large number of agents and the standard equilibrium concept is a Mean Field Equilibrium (MFE). Algorithms for learning MFE in dynamic MFGs are unknown in general. Our focus is on an important subclass that possess a monotonicity property called Strategic Complementarities (MFG-SC). We introduce a natural refinement to the equilibrium concept that we call Trembling-Hand-Perfect MFE (T-MFE), which allows agents to employ a measure of randomization while accounting for the impact of such randomization on their payoffs. We propose a simple algorithm for computing T-MFE under a known model. We also introduce a model-free and a model-based approach to learning T-MFE and provide sample complexities of both algorithms. We also develop a fully online learning scheme that obviates the need for a simulator. Finally, we empirically evaluate the performance of the proposed algorithms via examples motivated by real-world applications. 

   more » « less
2. Scalable Learning for Spatiotemporal Mean Field Games Using Physics-Informed Neural Operator

   https://doi.org/10.3390/math12060803  

   Liu, Shuo; Chen, Xu; Di, Xuan (March 2024, Mathematics)

   This paper proposes a scalable learning framework to solve a system of coupled forward–backward partial differential equations (PDEs) arising from mean field games (MFGs). The MFG system incorporates a forward PDE to model the propagation of population dynamics and a backward PDE for a representative agent’s optimal control. Existing work mainly focus on solving the mean field game equilibrium (MFE) of the MFG system when given fixed boundary conditions, including the initial population state and terminal cost. To obtain MFE efficiently, particularly when the initial population density and terminal cost vary, we utilize a physics-informed neural operator (PINO) to tackle the forward–backward PDEs. A learning algorithm is devised and its performance is evaluated on one application domain, which is the autonomous driving velocity control. Numerical experiments show that our method can obtain the MFE accurately when given different initial distributions of vehicles. The PINO exhibits both memory efficiency and generalization capabilities compared to physics-informed neural networks (PINNs). 

   more » « less
3. On Time‐Inconsistency in Mean‐Field Games

   https://doi.org/10.1111/mafi.12456  

   Bayraktar, Erhan; Wang, Zhenhua (July 2025, Mathematical Finance)

   ABSTRACT We investigate an infinite‐horizon time‐inconsistent mean‐field game (MFG) in a discrete time setting. We first present a classic equilibrium for the MFG and its associated existence result. This classic equilibrium aligns with the conventional equilibrium concept studied in MFG literature when the context is time‐consistent. Then we demonstrate that while this equilibrium produces an approximate optimal strategy when applied to the related ‐agent games, it does so solely in a precommitment sense. Therefore, it cannot function as a genuinely approximate equilibrium strategy from the perspective of a sophisticated agent within the ‐agent game. To address this limitation, we propose a newconsistentequilibrium concept in both the MFG and the ‐agent game. We show that a consistent equilibrium in the MFG can indeed function as an approximate consistent equilibrium in the ‐agent game. Additionally, we analyze the convergence of consistent equilibria for ‐agent games toward a consistent MFG equilibrium as tends to infinity. 

   more » « less
4. Caballero–Engel meet Lasry–Lions: A uniqueness result

   https://doi.org/10.1007/s11579-024-00370-2  

   Alvarez, Fernando; Lippi, Francesco; Souganidis, Panagiotis (July 2024, Mathematics and Financial Economics)

   Abstract In a Mean Field Game (MFG) each decision maker cares about the cross sectional distribution of the state and the dynamics of the distribution is generated by the agents’ optimal decisions. We prove the uniqueness of the equilibrium in a class of MFG where the decision maker controls the state at optimally chosen times. This setup accommodates several problems featuring non-convex adjustment costs, and complements the well known drift-control case studied by Lasry–Lions. Examples of such problems are described by Caballero and Engel in several papers, which introduce the concept of the generalized hazard function of adjustment. We extend the analysis to a general “impulse control problem” by introducing the concept of the “Impulse Hamiltonian”. Under the monotonicity assumption (a form of strategic substitutability), we establish the uniqueness of equilibrium. In this context, the Impulse Hamiltonian and its derivative play a similar role to the classical Hamiltonian that arises in the drift-control case. 

   more » « less
5. Phase transition in a kinetic mean-field game model of inertial self-propelled agents

   https://doi.org/10.1063/5.0230729  

   Grover, Piyush; Huo, Mandy (December 2024, Chaos: An Interdisciplinary Journal of Nonlinear Science)

   The framework of mean-field games (MFGs) is used for modeling the collective dynamics of large populations of non-cooperative decision-making agents. We formulate and analyze a kinetic MFG model for an interacting system of non-cooperative motile agents with inertial dynamics and finite-range interactions, where each agent is minimizing a biologically inspired cost function. By analyzing the associated coupled forward–backward in a time system of nonlinear Fokker–Planck and Hamilton–Jacobi–Bellman equations, we obtain conditions for closed-loop linear stability of the spatially homogeneous MFG equilibrium that corresponds to an ordered state with non-zero mean speed. Using a combination of analysis and numerical simulations, we show that when energetic cost of control is reduced below a critical value, this equilibrium loses stability, and the system transitions to a traveling wave solution. Our work provides a game-theoretic perspective to the problem of collective motion in non-equilibrium biological and bio-inspired systems. 

   more » « less