A class of nonlinear, stochastic staticization control problems (including minimization problems with smooth, convex, coercive payoffs) driven by diffusion dynamics with constant diffusion coefficient is considered. A fundamental solution form is obtained where the same solution can be used for a limited variety of terminal costs without resolution of the problem. One may convert this fundamental solution form from a stochastic control problem form to a deterministic control problem form. This yields an equivalence between certain secondorder (in space) HamiltonJacobi partial differential equations (HJ PDEs) and associated firstorder HJ PDEs. This reformulation has substantial numerical implications.
more »
« less
Conversion of a Class of Stochastic Control Problems to FundamentalSolution Deterministic Control Problems
A class of nonlinear, stochastic staticization control
problems (including minimization problems with smooth, convex,
coercive payoffs) driven by diffusion dynamics and constant
diffusion coefficient is considered. Using dynamic programming
and tools from static duality, a fundamental solution form is
obtained where the same solution can be used for a variety of
terminal costs without resolution of the problem. Further, this
fundamental solution takes the form of a deterministic control
problem rather than a stochastic control problem.
more »
« less
 Award ID(s):
 1908918
 NSFPAR ID:
 10171194
 Date Published:
 Journal Name:
 Proceedings of the American Control Conference
 ISSN:
 07431619
 Page Range / eLocation ID:
 28142819
 Format(s):
 Medium: X
 Sponsoring Org:
 National Science Foundation
More Like this


By exploiting minplus linearity, semiconcavity, and semigroup properties of dynamic programming, a fundamental solution semigroup for a class of approximate finite horizon linear infinite dimensional optimal control problems is constructed. Elements of this fundamental solution semigroup are parameterized by the time horizon, and can be used to approximate the solution of the corresponding finite horizon optimal control problem for any terminal cost. They can also be composed to compute approximations on longer horizons. The value function approximation provided takes the form of a minplus convolution of a kernel with the terminal cost. A general construction for this kernel is provided, along with a spectral representation for a restricted class of subproblems.more » « less

We present a neural network approach for approximating the value function of high dimensional stochastic control problems. Our training process simultaneously updates our value function estimate and identifies the part of the state space likely to be visited by optimal trajectories. Our approach leverages insights from optimal control theory and the fundamental relation between semilinear parabolic partial differential equations and forwardbackward stochastic differential equations. To focus the sampling on relevant states during neural network training, we use the stochastic Pontryagin maximum principle (PMP) to obtain the optimal controls for the current value function estimate. By design, our approach coincides with the method of characteristics for the nonviscous HamiltonJacobiBellman equation arising in deterministic control problems. Our training loss consists of a weighted sum of the objective functional of the control problem and penalty terms that enforce the HJB equations along the sampled trajectories. Importantly, training is unsupervised in that it does not require solutions of the control problem. Our numerical experiments highlight our scheme’s ability to identify the relevant parts of the state space and produce meaningful value estimates. Using a twodimensional model problem, we demonstrate the importance of the stochastic PMP to inform the sampling and compare to a finite element approach. With a nonlinear control affine quadcopter example, we illustrate that our approach can handle complicated dynamics. For a 100dimensional benchmark problem, we demonstrate that our approach improves accuracy and timetosolution and, via a modification, we show the wider applicability of our scheme.more » « less

null (Ed.)A new method is developed for solving optimal control problems whose solutions are nonsmooth. The method developed in this paper employs a modified form of the Legendre–Gauss–Radau orthogonal direct collocation method. This modified Legendre–Gauss–Radau method adds two variables and two constraints at the end of a mesh interval when compared with a previously developed standard Legendre– Gauss–Radau collocation method. The two additional variables are the time at the interface between two mesh intervals and the control at the end of each mesh inter val. The two additional constraints are a collocation condition for those differential equations that depend upon the control and an inequality constraint on the control at the endpoint of each mesh interval. The additional constraints modify the search space of the nonlinear programming problem such that an accurate approximation to the location of the nonsmoothness is obtained. The transformed adjoint system of the modified Legendre–Gauss–Radau method is then developed. Using this transformed adjoint system, a method is developed to transform the Lagrange multipliers of the nonlinear programming problem to the costate of the optimal control problem. Fur thermore, it is shown that the costate estimate satisfies one of the Weierstrass–Erdmann optimality conditions. Finally, the method developed in this paper is demonstrated on an example whose solution is nonsmooth.more » « less

Linear quadratic stochastic optimal control problems with operator coefficients: openloop solutionsAn optimal control problem is considered for linear stochastic differential equations with quadratic cost functional. The coefficients of the state equation and the weights in the cost functional are bounded operators on the spaces of square integrable random variables. The main motivation of our study is linear quadratic (LQ, for short) optimal control problems for meanfield stochastic differential equations. Openloop solvability of the problem is characterized as the solvability of a system of linear coupled forwardbackward stochastic differential equations (FBSDE, for short) with operator coefficients, together with a convexity condition for the cost functional. Under proper conditions, the wellposedness of such an FBSDE, which leads to the existence of an openloop optimal control, is established. Finally, as applications of our main results, a general meanfield LQ control problem and a concrete meanvariance portfolio selection problem in the openloop case are solved.more » « less