More Like this

1. Reinforcement Learning for Mean Field Games with Strategic Complementarities

   Lee, Kiyeob; Rengarajan, Desik; Kalathil, Dileep; Shakkottai, Srinivas. (January 2021, Proceedings of Machine Learning Research)

   null (Ed.)

   Mean Field Games (MFG) are the class of games with a very large number of agents and the standard equilibrium concept is a Mean Field Equilibrium (MFE). Algorithms for learning MFE in dynamic MFGs are un- known in general. Our focus is on an important subclass that possess a monotonicity property called Strategic Complementarities (MFG-SC). We introduce a natural refinement to the equilibrium concept that we call Trembling-Hand-Perfect MFE (T-MFE), which allows agents to employ a measure of randomization while accounting for the impact of such randomization on their payoffs. We propose a simple algorithm for computing T-MFE under a known model. We also introduce a model-free and a model-based approach to learning T-MFE and provide sample complexities of both algorithms. We also develop a fully online learning scheme that obviates the need for a simulator. Finally, we empirically evaluate the performance of the proposed algorithms via examples motivated by real-world applications. 

   more » « less
2. Scalable Learning for Spatiotemporal Mean Field Games Using Physics-Informed Neural Operator

   https://doi.org/10.3390/math12060803  

   Liu, Shuo; Chen, Xu; Di, Xuan (March 2024, Mathematics)

   This paper proposes a scalable learning framework to solve a system of coupled forward–backward partial differential equations (PDEs) arising from mean field games (MFGs). The MFG system incorporates a forward PDE to model the propagation of population dynamics and a backward PDE for a representative agent’s optimal control. Existing work mainly focus on solving the mean field game equilibrium (MFE) of the MFG system when given fixed boundary conditions, including the initial population state and terminal cost. To obtain MFE efficiently, particularly when the initial population density and terminal cost vary, we utilize a physics-informed neural operator (PINO) to tackle the forward–backward PDEs. A learning algorithm is devised and its performance is evaluated on one application domain, which is the autonomous driving velocity control. Numerical experiments show that our method can obtain the MFE accurately when given different initial distributions of vehicles. The PINO exhibits both memory efficiency and generalization capabilities compared to physics-informed neural networks (PINNs). 

   more » « less
3. Phase transition in a kinetic mean-field game model of inertial self-propelled agents

   https://doi.org/10.1063/5.0230729  

   Grover, Piyush; Huo, Mandy (December 2024, Chaos: An Interdisciplinary Journal of Nonlinear Science)

   The framework of mean-field games (MFGs) is used for modeling the collective dynamics of large populations of non-cooperative decision-making agents. We formulate and analyze a kinetic MFG model for an interacting system of non-cooperative motile agents with inertial dynamics and finite-range interactions, where each agent is minimizing a biologically inspired cost function. By analyzing the associated coupled forward–backward in a time system of nonlinear Fokker–Planck and Hamilton–Jacobi–Bellman equations, we obtain conditions for closed-loop linear stability of the spatially homogeneous MFG equilibrium that corresponds to an ordered state with non-zero mean speed. Using a combination of analysis and numerical simulations, we show that when energetic cost of control is reduced below a critical value, this equilibrium loses stability, and the system transitions to a traveling wave solution. Our work provides a game-theoretic perspective to the problem of collective motion in non-equilibrium biological and bio-inspired systems. 

   more » « less
4. Physics-Informed Graph Neural Operator for Mean Field Games on Graph: A Scalable Learning Approach

   https://doi.org/10.3390/g15020012  

   Chen, Xu; Liu, Shuo; Di, Xuan (April 2024, Games)

   Mean-field games (MFGs) are developed to model the decision-making processes of a large number of interacting agents in multi-agent systems. This paper studies mean-field games on graphs (G-MFGs). The equilibria of G-MFGs, namely, mean-field equilibria (MFE), are challenging to solve for their high-dimensional action space because each agent has to make decisions when they are at junction nodes or on edges. Furthermore, when the initial population state varies on graphs, we have to recompute MFE, which could be computationally challenging and memory-demanding. To improve the scalability and avoid repeatedly solving G-MFGs every time their initial state changes, this paper proposes physics-informed graph neural operators (PIGNO). The PIGNO utilizes a graph neural operator to generate population dynamics, given initial population distributions. To better train the neural operator, it leverages physics knowledge to propagate population state transitions on graphs. A learning algorithm is developed, and its performance is evaluated on autonomous driving games on road networks. Our results demonstrate that the PIGNO is scalable and generalizable when tested under unseen initial conditions. 

   more » « less
5. Topological bifurcations in a mean-field game

   https://doi.org/10.1098/rspa.2024.0348  

   Lori, Ali_Akbar Rezaei; Grover, Piyush (March 2025, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences)

   Mean-field games (MFGs) provide a statistical physics-inspired modelling framework for decision-making in large populations of strategic, non-cooperative agents. Mathematically, these systems consist of a forwards–backwards in time-system of two coupled nonlinear partial differential equations (PDEs), namely, the Fokker–Plank (FP) and the Hamilton–Jacobi–Bellman (HJB) equations, governing the agent state and control distribution, respectively. In this work, we study a finite-time MFG with a rich global bifurcation structure using a reduced-order model (ROM). The ROM is a four-dimensional (4D) two-point boundary value problem (BVP) obtained by restricting the controlled dynamics to the first two moments of the agent state distribution, i.e. the mean and the variance. Phase space analysis of the ROM reveals that the invariant manifolds of periodic orbits around the so-called ‘ergodic MFG equilibrium’ play a crucial role in determining the bifurcation diagram and imparting a topological signature to various solution branches. We show a qualitative agreement of these results with numerical solutions of the full-order MFG PDE system. 

   more » « less