More Like this

1. Connecting Hamilton-Jacobi Partial Differential Equations with Maximum a Posteriori and Posterior Mean Estimators for Some Non-convex Priors.

   https://doi.org/10.1007/978-3-030-03009-4_56-1  

   Darbon, J.; Langlois, G.P.; Meng, T. (May 2021, Handbook of Mathematical Models and Algorithms in Computer Vision and Imaging. Springer,)

   null; null; null; null (Ed.)

   Many imaging problems can be formulated as inverse problems expressed as finite-dimensional optimization problems. These optimization problems generally consist of minimizing the sum of a data fidelity and regularization terms. In Darbon (SIAM J. Imag. Sci. 8:2268–2293, 2015), Darbon and Meng, (On decomposition models in imaging sciences and multi-time Hamilton-Jacobi partial differential equations, arXiv preprint arXiv:1906.09502, 2019), connections between these optimization problems and (multi-time) Hamilton-Jacobi partial differential equations have been proposed under the convexity assumptions of both the data fidelity and regularization terms. In particular, under these convexity assumptions, some representation formulas for a minimizer can be obtained. From a Bayesian perspective, such a minimizer can be seen as a maximum a posteriori estimator. In this chapter, we consider a certain class of non-convex regularizations and show that similar representation formulas for the minimizer can also be obtained. This is achieved by leveraging min-plus algebra techniques that have been originally developed for solving certain Hamilton-Jacobi partial differential equations arising in optimal control. Note that connections between viscous Hamilton-Jacobi partial differential equations and Bayesian posterior mean estimators with Gaussian data fidelity terms and log-concave priors have been highlighted in Darbon and Langlois, (On Bayesian posterior mean estimators in imaging sciences and Hamilton-Jacobi partial differential equations, arXiv preprint arXiv:2003.05572, 2020). We also present similar results for certain Bayesian posterior mean estimators with Gaussian data fidelity and certain non-log-concave priors using an analogue of min-plus algebra techniques. 

   more » « less
2. Non-local Hamilton–Jacobi–Bellman equations for the stochastic optimal control of path-dependent piecewise deterministic processes

   https://doi.org/10.1016/j.spa.2025.104813  

   Bandini, Elena; Keller, Christian (October 2025, Stochastic Processes and their Applications)

   We study the optimal control of path-dependent piecewise deterministic processes. An appropriate dynamic programming principle is established. We prove that the associated value function is the unique minimax solution of the corresponding non-local path-dependent Hamilton-Jacobi-Bellman equation. This is the first well-posedness result for nonsmooth solutions of fully nonlinear non-local path-dependent partial differential equations. 

   more » « less
3. Viability for locally monotone evolution inclusions and lower semicontinuous solutions of Hamilton–Jacobi–Bellman equations in infinite dimensions

   https://doi.org/10.1051/cocv/2024086  

   Jiang, Jichao; Keller, Christian (January 2025, ESAIM: Control, Optimisation and Calculus of Variations)

   We establish necessary and sufficient conditions for viability of evolution inclusions with locally monotone operators in the sense of Liu and Röckner [J. Funct. Anal., 259 (2010), pp. 2902-2922]. This allows us to prove wellposedness of lower semicontinuous solutions of Hamilton-Jacobi-Bellman equations associated to the optimal control of evolution inclusions. Thereby, we generalize results in Bayraktar and Keller [J. Funct. Anal., 275 (2018), pp. 2096-2161] on Hamilton-Jacobi equations in infinite dimensions with monotone operators in several ways. First, we permit locally monotone operators. This extends the applicability of our theory to a wider class of equations such as Burgers' equations, reaction-diffusion equations, and 2D Navier-Stokes equations. Second, our results apply to optimal control problems with state constraints. Third, we have uniqueness of viscosity solutions. Our results on viability and lower semicontinuous solutions are new even in the case of monotone operators. 

   more » « less
4. A Neural Network Approach for Stochastic Optimal Control

   Li, Xingjian; Verma, Deepanshu; Ruthotto, Lars (June 2024, SIAM journal on scientific computing)

   We present a neural network approach for approximating the value function of high- dimensional stochastic control problems. Our training process simultaneously updates our value function estimate and identifies the part of the state space likely to be visited by optimal trajectories. Our approach leverages insights from optimal control theory and the fundamental relation between semi-linear parabolic partial differential equations and forward-backward stochastic differential equations. To focus the sampling on relevant states during neural network training, we use the stochastic Pontryagin maximum principle (PMP) to obtain the optimal controls for the current value function estimate. By design, our approach coincides with the method of characteristics for the non-viscous Hamilton-Jacobi-Bellman equation arising in deterministic control problems. Our training loss consists of a weighted sum of the objective functional of the control problem and penalty terms that enforce the HJB equations along the sampled trajectories. Importantly, training is unsupervised in that it does not require solutions of the control problem. Our numerical experiments highlight our scheme’s ability to identify the relevant parts of the state space and produce meaningful value estimates. Using a two-dimensional model problem, we demonstrate the importance of the stochastic PMP to inform the sampling and compare to a finite element approach. With a nonlinear control affine quadcopter example, we illustrate that our approach can handle complicated dynamics. For a 100-dimensional benchmark problem, we demonstrate that our approach improves accuracy and time-to-solution and, via a modification, we show the wider applicability of our scheme. 

   more » « less
5. A Neural Network Approach for High-Dimensional Optimal Control Applied to Multi-Agent Path Finding

   Onken, Derek; Nurbekyan, Levon; Li, Xingjian; Wu Fung, Samy; Osher, Stan; Ruthotto, Lars (July 2022, IEEE transactions on control systems technology)

   We propose a neural network approach that yields approximate solutions for high-dimensional optimal control problems and demonstrate its effectiveness using examples from multi-agent path finding. Our approach yields controls in a feedback form, where the policy function is given by a neural network (NN). Specifically, we fuse the Hamilton-Jacobi-Bellman (HJB) and Pontryagin Maximum Principle (PMP) approaches by parameterizing the value function with an NN. Our approach enables us to obtain approximately optimal controls in real-time without having to solve an optimization problem. Once the policy function is trained, generating a control at a given space-time location takes milliseconds; in contrast, efficient nonlinear programming methods typically perform the same task in seconds. We train the NN offline using the objective function of the control problem and penalty terms that enforce the HJB equations. Therefore, our training algorithm does not involve data generated by another algorithm. By training on a distribution of initial states, we ensure the controls' optimality on a large portion of the state-space. Our grid-free approach scales efficiently to dimensions where grids become impractical or infeasible. We apply our approach to several multi-agent collision-avoidance problems in up to 150 dimensions. Furthermore, we empirically observe that the number of parameters in our approach scales linearly with the dimension of the control problem, thereby mitigating the curse of dimensionality. 

   more » « less