More Like this

1. Linear quadratic stochastic optimal control problems with operator coefficients: open-loop solutions

   https://doi.org/10.1051/cocv/2018013  

   Wei, Qingmeng ; Yong, Jiongmin ; Yu, Zhiyong ( January 2019 , ESAIM: Control, Optimisation and Calculus of Variations)

   An optimal control problem is considered for linear stochastic differential equations with quadratic cost functional. The coefficients of the state equation and the weights in the cost functional are bounded operators on the spaces of square integrable random variables. The main motivation of our study is linear quadratic (LQ, for short) optimal control problems for mean-field stochastic differential equations. Open-loop solvability of the problem is characterized as the solvability of a system of linear coupled forward-backward stochastic differential equations (FBSDE, for short) with operator coefficients, together with a convexity condition for the cost functional. Under proper conditions, the well-posedness of such an FBSDE, which leads to the existence of an open-loop optimal control, is established. Finally, as applications of our main results, a general mean-field LQ control problem and a concrete mean-variance portfolio selection problem in the open-loop case are solved. 

   more » « less
2. Linear-quadratic zero-sum mean-field type games: Optimality conditions and policy optimization

   https://doi.org/10.3934/jdg.2021023  

   Carmona, René ; Hamidouche, Kenza ; Laurière, Mathieu ; Tan, Zongjun ( January 2021 , Journal of Dynamics & Games)

   null (Ed.)

   In this paper, zero-sum mean-field type games (ZSMFTG) with linear dynamics and quadratic cost are studied under infinite-horizon discounted utility function. ZSMFTG are a class of games in which two decision makers whose utilities sum to zero, compete to influence a large population of indistinguishable agents. In particular, the case in which the transition and utility functions depend on the state, the action of the controllers, and the mean of the state and the actions, is investigated. The optimality conditions of the game are analysed for both open-loop and closed-loop controls, and explicit expressions for the Nash equilibrium strategies are derived. Moreover, two policy optimization methods that rely on policy gradient are proposed for both model-based and sample-based frameworks. In the model-based case, the gradients are computed exactly using the model, whereas they are estimated using Monte-Carlo simulations in the sample-based case. Numerical experiments are conducted to show the convergence of the utility function as well as the two players' controls.

    

   more » « less
3. Deep fictitious play for stochastic differential games

   https://doi.org/https://dx.doi.org/10.4310/CMS.2021.v19.n2.a2  

   Hu, Ruimeng ( January 2021 , Communications in mathematical sciences)

   Jin, Shi (Ed.)

   In this paper, we apply the idea of fictitious play to design deep neural networks (DNNs), and develop deep learning theory and algorithms for computing the Nash equilibrium of asymmetric N-player non-zero-sum stochastic differential games, for which we refer as deep fictitious play, a multi-stage learning process. Specifically at each stage, we propose the strategy of letting individual player optimize her own payoff subject to the other players’ previous actions, equivalent to solving N decoupled stochastic control optimization problems, which are approximated by DNNs. Therefore, the fictitious play strategy leads to a structure consisting of N DNNs, which only communicate at the end of each stage. The resulting deep learning algorithm based on fictitious play is scalable, parallel and model-free, i.e., using GPU parallelization, it can be applied to any N-player stochastic differential game with different symmetries and heterogeneities (e.g., existence of major players). We illustrate the performance of the deep learning algorithm by comparing to the closed-form solution of the linear quadratic game. Moreover, we prove the convergence of fictitious play under appropriate assumptions, and verify that the convergent limit forms an open-loop Nash equilibrium. We also discuss the extensions to other strategies designed upon fictitious play and closed-loop Nash equilibrium in the end. 

   more » « less
4. Covariance steering in zero-sum linear-quadratic two-player differential games

   Yongxin Chen, Tryphon T. ( December 2019 , 2019 IEEE 58th Conference on Decision and Control (CDC))

   Abstract—We formulate a new class of two-person zero- sum differential games, in a stochastic setting, where a specifi- cation on a target terminal state distribution is imposed on the players. We address such added specification by introducing incentives to the game that guides the players to steer the join distribution accordingly. In the present paper, we only address linear quadratic games with Gaussian target distribution. The solution is characterized by a coupled Riccati equations system, resembling that in the standard linear quadratic differential games. Indeed, once the incentive function is calculated, our problem reduces to a standard one. Tthe framework developed in this paper extends previous results in covariance control, a fast growing research area. On the numerical side, problems herein are reformulated as convex-concave minimax problems for which efficient and reliable algorithms are available. 

   more » « less
5. Linear-Quadratic Stochastic Differential Games on Directed Chain Networks

   Yichen Feng, Jean-Pierre Fouque ( January 2020 , Journal of mathematics and statistical science)

   We study linear-quadratic stochastic differential games on directed chains inspired by the directed chain stochastic differential equations introduced by Detering, Fouque and Ichiba. We solve explicitly for Nash equilibria with a finite number of players and we study more general finite-player games with a mixture of both directed chain in- teraction and mean field interaction. We investigate and compare the corresponding games in the limit when the number of players tends to infinity. The limit is characterized by Catalan functions and the dy- namics under equilibrium is an infinite-dimensional Gaussian process described by a Catalan Markov chain, with or without the presence of mean field interaction. 

   more » « less