 Award ID(s):
 1812921
 NSFPAR ID:
 10341970
 Date Published:
 Journal Name:
 ESAIM: Control, Optimisation and Calculus of Variations
 Volume:
 27
 ISSN:
 12928119
 Page Range / eLocation ID:
 81
 Format(s):
 Medium: X
 Sponsoring Org:
 National Science Foundation
More Like this

Linear quadratic stochastic optimal control problems with operator coefficients: openloop solutionsAn optimal control problem is considered for linear stochastic differential equations with quadratic cost functional. The coefficients of the state equation and the weights in the cost functional are bounded operators on the spaces of square integrable random variables. The main motivation of our study is linear quadratic (LQ, for short) optimal control problems for meanfield stochastic differential equations. Openloop solvability of the problem is characterized as the solvability of a system of linear coupled forwardbackward stochastic differential equations (FBSDE, for short) with operator coefficients, together with a convexity condition for the cost functional. Under proper conditions, the wellposedness of such an FBSDE, which leads to the existence of an openloop optimal control, is established. Finally, as applications of our main results, a general meanfield LQ control problem and a concrete meanvariance portfolio selection problem in the openloop case are solved.more » « less

null (Ed.)
In this paper, zerosum meanfield type games (ZSMFTG) with linear dynamics and quadratic cost are studied under infinitehorizon discounted utility function. ZSMFTG are a class of games in which two decision makers whose utilities sum to zero, compete to influence a large population of indistinguishable agents. In particular, the case in which the transition and utility functions depend on the state, the action of the controllers, and the mean of the state and the actions, is investigated. The optimality conditions of the game are analysed for both openloop and closedloop controls, and explicit expressions for the Nash equilibrium strategies are derived. Moreover, two policy optimization methods that rely on policy gradient are proposed for both modelbased and samplebased frameworks. In the modelbased case, the gradients are computed exactly using the model, whereas they are estimated using MonteCarlo simulations in the samplebased case. Numerical experiments are conducted to show the convergence of the utility function as well as the two players' controls.

Jin, Shi (Ed.)In this paper, we apply the idea of fictitious play to design deep neural networks (DNNs), and develop deep learning theory and algorithms for computing the Nash equilibrium of asymmetric Nplayer nonzerosum stochastic differential games, for which we refer as deep fictitious play, a multistage learning process. Specifically at each stage, we propose the strategy of letting individual player optimize her own payoff subject to the other players’ previous actions, equivalent to solving N decoupled stochastic control optimization problems, which are approximated by DNNs. Therefore, the fictitious play strategy leads to a structure consisting of N DNNs, which only communicate at the end of each stage. The resulting deep learning algorithm based on fictitious play is scalable, parallel and modelfree, i.e., using GPU parallelization, it can be applied to any Nplayer stochastic differential game with different symmetries and heterogeneities (e.g., existence of major players). We illustrate the performance of the deep learning algorithm by comparing to the closedform solution of the linear quadratic game. Moreover, we prove the convergence of fictitious play under appropriate assumptions, and verify that the convergent limit forms an openloop Nash equilibrium. We also discuss the extensions to other strategies designed upon fictitious play and closedloop Nash equilibrium in the end.more » « less

Abstract—We formulate a new class of twoperson zero sum differential games, in a stochastic setting, where a specifi cation on a target terminal state distribution is imposed on the players. We address such added specification by introducing incentives to the game that guides the players to steer the join distribution accordingly. In the present paper, we only address linear quadratic games with Gaussian target distribution. The solution is characterized by a coupled Riccati equations system, resembling that in the standard linear quadratic differential games. Indeed, once the incentive function is calculated, our problem reduces to a standard one. Tthe framework developed in this paper extends previous results in covariance control, a fast growing research area. On the numerical side, problems herein are reformulated as convexconcave minimax problems for which efficient and reliable algorithms are available.more » « less

We study linearquadratic stochastic differential games on directed chains inspired by the directed chain stochastic differential equations introduced by Detering, Fouque and Ichiba. We solve explicitly for Nash equilibria with a finite number of players and we study more general finiteplayer games with a mixture of both directed chain in teraction and mean field interaction. We investigate and compare the corresponding games in the limit when the number of players tends to infinity. The limit is characterized by Catalan functions and the dy namics under equilibrium is an infinitedimensional Gaussian process described by a Catalan Markov chain, with or without the presence of mean field interaction.more » « less