This content will become publicly available on July 2, 2025
 Award ID(s):
 2204795
 NSFPAR ID:
 10520107
 Publisher / Repository:
 World Scientific
 Date Published:
 Journal Name:
 International game theory review
 ISSN:
 17936675
 Format(s):
 Medium: X
 Sponsoring Org:
 National Science Foundation
More Like this

We study the deterministic control problem in the Wasserstein space, following the recent works of Bonnet and Frankowska, but with a new approach. One of the major advantages of our approach is that it reconciles the closed loop and the open loop approaches, without the technicalities of the traditional feedback control methodology. It allows also to embed the control problem in the Wasserstein space into a control problem in a Hilbert space, similar to the lifting method introduced by P. L. Lions, used already in our previous works. The Hilbert space is different from that proposed by P. L. Lions, and it allows to recover the control problem in the Wasserstein space as a particular case.more » « less

Dynamic programming equations for mean field control problems with a separable structure are Eikonal type equations on the Wasserstein space. Standard differentiation using linear derivatives yield a direct extension of the classical viscosity theory. We use Fourier representation of the Sobolev norms on the space of measures, together with the standard techniques from the finite dimensional theory to prove a comparison result among semicontinuous sub and super solutions, obtaining a unique characterization of the value function.more » « less

We develop a general reinforcement learning framework for mean field control (MFC) problems. Such problems arise for instance as the limit of collaborative multiagent control problems when the number of agents is very large. The asymptotic problem can be phrased as the optimal control of a nonlinear dynamics. This can also be viewed as a Markov decision process (MDP) but the key difference with the usual RL setup is that the dynamics and the reward now depend on the state's probability distribution itself. Alternatively, it can be recast as a MDP on the Wasserstein space of measures. In this work, we introduce generic modelfree algorithms based on the stateaction value function at the mean field level and we prove convergence for a prototypical Qlearning method. We then implement an actorcritic method and report numerical results on two archetypal problems: a finite space model motivated by a cyber security application and a continuous space model motivated by an application to swarm motion.more » « less

Abstract The goal of this paper is to prove a comparison principle for viscosity solutions of semilinear Hamilton–Jacobi equations in the space of probability measures. The method involves leveraging differentiability properties of the 2Wasserstein distance in the doubling of variables argument, which is done by introducing a further entropy penalization that ensures that the relevant optima are achieved at positive, Lipschitz continuous densities with finite Fischer information. This allows to prove uniqueness and stability of viscosity solutions in the class of bounded Lipschitz continuous (with respect to the 1Wasserstein distance) functions. The result does not appeal to a mean field control formulation of the equation, and, as such, applies to equations with nonconvex Hamiltonians and measuredependent volatility. For convex Hamiltonians that derive from a potential, we prove that the value function associated with a suitable meanfield optimal control problem with nondegenerate idiosyncratic noise is indeed the unique viscosity solution.

Linear quadratic stochastic optimal control problems with operator coefficients: openloop solutionsAn optimal control problem is considered for linear stochastic differential equations with quadratic cost functional. The coefficients of the state equation and the weights in the cost functional are bounded operators on the spaces of square integrable random variables. The main motivation of our study is linear quadratic (LQ, for short) optimal control problems for meanfield stochastic differential equations. Openloop solvability of the problem is characterized as the solvability of a system of linear coupled forwardbackward stochastic differential equations (FBSDE, for short) with operator coefficients, together with a convexity condition for the cost functional. Under proper conditions, the wellposedness of such an FBSDE, which leads to the existence of an openloop optimal control, is established. Finally, as applications of our main results, a general meanfield LQ control problem and a concrete meanvariance portfolio selection problem in the openloop case are solved.more » « less