More Like this

1. Model-Free Mean-Field Reinforcement Learning: Mean-Field MDP and Mean-Field Q-Learning

   Carmona, Rene; Lauriere, Mathieu (October 2019, ArXivorg)

   We develop a general reinforcement learning framework for mean field control (MFC) problems. Such problems arise for instance as the limit of collaborative multi-agent control problems when the number of agents is very large. The asymptotic problem can be phrased as the optimal control of a non-linear dynamics. This can also be viewed as a Markov decision process (MDP) but the key difference with the usual RL setup is that the dynamics and the reward now depend on the state's probability distribution itself. Alternatively, it can be recast as a MDP on the Wasserstein space of measures. In this work, we introduce generic model-free algorithms based on the state-action value function at the mean field level and we prove convergence for a prototypical Q-learning method. We then implement an actor-critic method and report numerical results on two archetypal problems: a finite space model motivated by a cyber security application and a continuous space model motivated by an application to swarm motion. 

   more » « less
2. A Mean Field Competition

   https://doi.org/10.1287/moor.2018.0966  

   Nutz, Marcel; Zhang, Yuchong (November 2019, Mathematics of Operations Research)
3. Mean field verification theorem

   https://doi.org/10.4310/CIS.2021.v21.n2.a4  

   Bensoussan, Alain; Hoe, SingRu; Kim, Joohyun; Yan, Zhongfeng (January 2021, Communications in Information and Systems)

   null (Ed.)
4. Graphon mean field systems

   https://doi.org/10.1214/22-AAP1901  

   Bayraktar, Erhan; Chakraborty, Suman; Wu, Ruoyu (October 2023, The Annals of Applied Probability)

   We consider heterogeneously interacting diffusive particle systems and their large population limit. The interaction is of mean field type with weights characterized by an underlying graphon. A law of large numbers result is established as the system size increases and the underlying graphons converge. The limit is given by a graphon mean field system consisting of independent but heterogeneous nonlinear diffusions whose probability distributions are fully coupled. Well-posedness, continuity and stability of such systems are provided. We also consider a not-so-dense analogue of the finite particle system, obtained by percolation with vanishing rates and suitable scaling of interactions. A law of large numbers result is proved for the convergence of such systems to the corresponding graphon mean field system. 

   more » « less
5. The critical mean-field Chayes–Machta dynamics

   https://doi.org/10.1017/S0963548322000037  

   Blanca, Antonio; Sinclair, Alistair; Zhang, Xusheng (May 2022, Combinatorics, Probability and Computing)