skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: Convergence of deep fictitious play for stochastic differential games
Stochastic differential games have been used extensively to model agents' competitions in finance, for instance, in P2P lending platforms from the Fintech industry, the banking system for systemic risk, and insurance markets. The recently proposed machine learning algorithm, deep fictitious play, provides a novel and efficient tool for finding Markovian Nash equilibrium of large \begin{document}$ N $$\end{document}-player asymmetric stochastic differential games [J. Han and R. Hu, Mathematical and Scientific Machine Learning Conference, pages 221-245, PMLR, 2020]. By incorporating the idea of fictitious play, the algorithm decouples the game into \begin{document}$$ N $$\end{document} sub-optimization problems, and identifies each player's optimal strategy with the deep backward stochastic differential equation (BSDE) method parallelly and repeatedly. In this paper, we prove the convergence of deep fictitious play (DFP) to the true Nash equilibrium. We can also show that the strategy based on DFP forms an \begin{document}$$ \epsilon $$\end{document}$-Nash equilibrium. We generalize the algorithm by proposing a new approach to decouple the games, and present numerical results of large population games showing the empirical convergence of the algorithm beyond the technical assumptions in the theorems.  more » « less
Award ID(s):
1953035
PAR ID:
10343586
Author(s) / Creator(s):
; ;
Date Published:
Journal Name:
Frontiers of Mathematical Finance
Volume:
1
Issue:
2
ISSN:
2769-6715
Page Range / eLocation ID:
287
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Jin, Shi (Ed.)
    In this paper, we apply the idea of fictitious play to design deep neural networks (DNNs), and develop deep learning theory and algorithms for computing the Nash equilibrium of asymmetric N-player non-zero-sum stochastic differential games, for which we refer as deep fictitious play, a multi-stage learning process. Specifically at each stage, we propose the strategy of letting individual player optimize her own payoff subject to the other players’ previous actions, equivalent to solving N decoupled stochastic control optimization problems, which are approximated by DNNs. Therefore, the fictitious play strategy leads to a structure consisting of N DNNs, which only communicate at the end of each stage. The resulting deep learning algorithm based on fictitious play is scalable, parallel and model-free, i.e., using GPU parallelization, it can be applied to any N-player stochastic differential game with different symmetries and heterogeneities (e.g., existence of major players). We illustrate the performance of the deep learning algorithm by comparing to the closed-form solution of the linear quadratic game. Moreover, we prove the convergence of fictitious play under appropriate assumptions, and verify that the convergent limit forms an open-loop Nash equilibrium. We also discuss the extensions to other strategies designed upon fictitious play and closed-loop Nash equilibrium in the end. 
    more » « less
  2. For any finite horizon Sinai billiard map \begin{document}$ T $$\end{document} on the two-torus, we find \begin{document}$$ t_*>1 $$\end{document} such that for each \begin{document}$$ t\in (0,t_*) $$\end{document} there exists a unique equilibrium state \begin{document}$$ \mu_t $$\end{document} for \begin{document}$$ - t\log J^uT $$\end{document}, and \begin{document}$$ \mu_t $$\end{document} is \begin{document}$$ T $$\end{document}-adapted. (In particular, the SRB measure is the unique equilibrium state for \begin{document}$$ - \log J^uT $$\end{document}.) We show that \begin{document}$$ \mu_t $$\end{document} is exponentially mixing for Hölder observables, and the pressure function \begin{document}$$ P(t) = \sup_\mu \{h_\mu -\int t\log J^uT d \mu\} $$\end{document} is analytic on \begin{document}$$ (0,t_*) $$\end{document}. In addition, \begin{document}$$ P(t) $$\end{document} is strictly convex if and only if \begin{document}$$ \log J^uT $$\end{document} is not \begin{document}$$ \mu_t $$\end{document}-a.e. cohomologous to a constant, while, if there exist \begin{document}$$ t_a\ne t_b $$\end{document} with \begin{document}$$ \mu_{t_a} = \mu_{t_b} $$\end{document}, then \begin{document}$$ P(t) $$\end{document} is affine on \begin{document}$$ (0,t_*) $$\end{document}. An additional sparse recurrence condition gives \begin{document}$$ \lim_{t\downarrow 0} P(t) = P(0) $$\end{document}$. 
    more » « less
  3. It is shown that for any positive integer \begin{document}$$ n \ge 3 $$\end{document}, there is a stable irreducible \begin{document}$$ n\times n $$\end{document} matrix \begin{document}$ A $$\end{document} with \begin{document}$$ 2n+1-\lfloor\frac{n}{3}\rfloor $$\end{document} nonzero entries exhibiting Turing instability. Moreover, when \begin{document}$$ n = 3 $$\end{document}, the result is best possible, i.e., every \begin{document}$$ 3\times 3 $$\end{document} stable matrix with five or fewer nonzero entries will not exhibit Turing instability. Furthermore, we determine all possible \begin{document}$$ 3\times 3 $$\end{document} irreducible sign pattern matrices with 6 nonzero entries which can be realized by a matrix \begin{document}$$ A $$\end{document}$ that exhibits Turing instability. 
    more » « less
  4. We consider the well-known Lieb-Liniger (LL) model for \begin{document}$ N $$\end{document} bosons interacting pairwise on the line via the \begin{document}$$ \delta $$\end{document} potential in the mean-field scaling regime. Assuming suitable asymptotic factorization of the initial wave functions and convergence of the microscopic energy per particle, we show that the time-dependent reduced density matrices of the system converge in trace norm to the pure states given by the solution to the one-dimensional cubic nonlinear Schrödinger equation (NLS) with an explict rate of convergence. In contrast to previous work [3] relying on the formalism of second quantization and coherent states and without an explicit rate, our proof is based on the counting method of Pickl [65,66,67] and Knowles and Pickl [44]. To overcome difficulties stemming from the singularity of the \begin{document}$$ \delta $$\end{document} potential, we introduce a new short-range approximation argument that exploits the Hölder continuity of the \begin{document}$$ N $$\end{document}-body wave function in a single particle variable. By further exploiting the \begin{document}$$ L^2 $$\end{document}-subcritical well-posedness theory for the 1D cubic NLS, we can prove mean-field convergence when the limiting solution to the NLS has finite mass, but only for a very special class of \begin{document}$$ N $$\end{document}$-body initial states. 
    more » « less
  5. Genetic variations in the COVID-19 virus are one of the main causes of the COVID-19 pandemic outbreak in 2020 and 2021. In this article, we aim to introduce a new type of model, a system coupled with ordinary differential equations (ODEs) and measure differential equation (MDE), stemming from the classical SIR model for the variants distribution. Specifically, we model the evolution of susceptible \begin{document}$ S $$\end{document} and removed \begin{document}$$ R $$\end{document} populations by ODEs and the infected \begin{document}$$ I $$\end{document} population by a MDE comprised of a probability vector field (PVF) and a source term. In addition, the ODEs for \begin{document}$$ S $$\end{document} and \begin{document}$$ R $$\end{document} contains terms that are related to the measure \begin{document}$$ I $$\end{document}$. We establish analytically the well-posedness of the coupled ODE-MDE system by using generalized Wasserstein distance. We give two examples to show that the proposed ODE-MDE model coincides with the classical SIR model in case of constant or time-dependent parameters as special cases. 
    more » « less