More Like this

1. Topological bifurcations in a mean-field game

   https://doi.org/10.1098/rspa.2024.0348  

   Lori, Ali_Akbar Rezaei; Grover, Piyush (March 2025, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences)

   Mean-field games (MFGs) provide a statistical physics-inspired modelling framework for decision-making in large populations of strategic, non-cooperative agents. Mathematically, these systems consist of a forwards–backwards in time-system of two coupled nonlinear partial differential equations (PDEs), namely, the Fokker–Plank (FP) and the Hamilton–Jacobi–Bellman (HJB) equations, governing the agent state and control distribution, respectively. In this work, we study a finite-time MFG with a rich global bifurcation structure using a reduced-order model (ROM). The ROM is a four-dimensional (4D) two-point boundary value problem (BVP) obtained by restricting the controlled dynamics to the first two moments of the agent state distribution, i.e. the mean and the variance. Phase space analysis of the ROM reveals that the invariant manifolds of periodic orbits around the so-called ‘ergodic MFG equilibrium’ play a crucial role in determining the bifurcation diagram and imparting a topological signature to various solution branches. We show a qualitative agreement of these results with numerical solutions of the full-order MFG PDE system. 

   more » « less
2. Reinforcement Learning for Mean Field Games with Strategic Complementarities

   Lee, Kiyeob; Rengarajan, Desik; Kalathil, Dileep; Shakkottai, Srinivas. (January 2021, Proceedings of Machine Learning Research)

   null (Ed.)

   Mean Field Games (MFG) are the class of games with a very large number of agents and the standard equilibrium concept is a Mean Field Equilibrium (MFE). Algorithms for learning MFE in dynamic MFGs are un- known in general. Our focus is on an important subclass that possess a monotonicity property called Strategic Complementarities (MFG-SC). We introduce a natural refinement to the equilibrium concept that we call Trembling-Hand-Perfect MFE (T-MFE), which allows agents to employ a measure of randomization while accounting for the impact of such randomization on their payoffs. We propose a simple algorithm for computing T-MFE under a known model. We also introduce a model-free and a model-based approach to learning T-MFE and provide sample complexities of both algorithms. We also develop a fully online learning scheme that obviates the need for a simulator. Finally, we empirically evaluate the performance of the proposed algorithms via examples motivated by real-world applications. 

   more » « less
3. Reinforcement Learning for Mean Field Games with Strategic Complementarities (AISTATS 2021)

   https://doi.org/PMLR 130:2458-2466  

   Lee, Kiyeob; Rengarajan, Desik; Kalathil, Dileep; Shakkottai, Srinivas (January 2021, Proceedings of Machine Learning Research)

   null (Ed.)

   Mean Field Games (MFG) are the class of games with a very large number of agents and the standard equilibrium concept is a Mean Field Equilibrium (MFE). Algorithms for learning MFE in dynamic MFGs are unknown in general. Our focus is on an important subclass that possess a monotonicity property called Strategic Complementarities (MFG-SC). We introduce a natural refinement to the equilibrium concept that we call Trembling-Hand-Perfect MFE (T-MFE), which allows agents to employ a measure of randomization while accounting for the impact of such randomization on their payoffs. We propose a simple algorithm for computing T-MFE under a known model. We also introduce a model-free and a model-based approach to learning T-MFE and provide sample complexities of both algorithms. We also develop a fully online learning scheme that obviates the need for a simulator. Finally, we empirically evaluate the performance of the proposed algorithms via examples motivated by real-world applications. 

   more » « less
4. Scalable Learning for Spatiotemporal Mean Field Games Using Physics-Informed Neural Operator

   https://doi.org/10.3390/math12060803  

   Liu, Shuo; Chen, Xu; Di, Xuan (March 2024, Mathematics)

   This paper proposes a scalable learning framework to solve a system of coupled forward–backward partial differential equations (PDEs) arising from mean field games (MFGs). The MFG system incorporates a forward PDE to model the propagation of population dynamics and a backward PDE for a representative agent’s optimal control. Existing work mainly focus on solving the mean field game equilibrium (MFE) of the MFG system when given fixed boundary conditions, including the initial population state and terminal cost. To obtain MFE efficiently, particularly when the initial population density and terminal cost vary, we utilize a physics-informed neural operator (PINO) to tackle the forward–backward PDEs. A learning algorithm is devised and its performance is evaluated on one application domain, which is the autonomous driving velocity control. Numerical experiments show that our method can obtain the MFE accurately when given different initial distributions of vehicles. The PINO exhibits both memory efficiency and generalization capabilities compared to physics-informed neural networks (PINNs). 

   more » « less
5. Caballero–Engel meet Lasry–Lions: A uniqueness result

   https://doi.org/10.1007/s11579-024-00370-2  

   Alvarez, Fernando; Lippi, Francesco; Souganidis, Panagiotis (July 2024, Mathematics and Financial Economics)

   Abstract In a Mean Field Game (MFG) each decision maker cares about the cross sectional distribution of the state and the dynamics of the distribution is generated by the agents’ optimal decisions. We prove the uniqueness of the equilibrium in a class of MFG where the decision maker controls the state at optimally chosen times. This setup accommodates several problems featuring non-convex adjustment costs, and complements the well known drift-control case studied by Lasry–Lions. Examples of such problems are described by Caballero and Engel in several papers, which introduce the concept of the generalized hazard function of adjustment. We extend the analysis to a general “impulse control problem” by introducing the concept of the “Impulse Hamiltonian”. Under the monotonicity assumption (a form of strategic substitutability), we establish the uniqueness of equilibrium. In this context, the Impulse Hamiltonian and its derivative play a similar role to the classical Hamiltonian that arises in the drift-control case. 

   more » « less