More Like this

1. Stochastic control/stopping problem with expectation constraints

   https://doi.org/10.1016/j.spa.2024.104430  

   Bayraktar, Erhan; Yao, Song (October 2024, Stochastic Processes and their Applications)

   We study a stochastic control/stopping problem with a series of inequality-type and equality-type expectation constraints in a general non-Markovian framework. We demonstrate that the stochastic control/stopping problem with expectation constraints (CSEC) is independent of a specific probability setting and is equivalent to the constrained stochastic control/stopping problem in weak formulation (an optimization over joint laws of Brownian motion, state dynamics, diffusion controls and stopping rules on an enlarged canonical space). Using a martingale-problem formulation of controlled SDEs in spirit of Stroock and Varadhan (2006), we characterize the probability classes in weak formulation by countably many actions of canonical processes, and thus obtain the upper semi-analyticity of the CSEC value function. Then we employ a measurable selection argument to establish a dynamic programming principle (DPP) in weak formulation for the CSEC value function, in which the conditional expected costs act as additional states for constraint levels at the intermediate horizon. This article extends (El Karoui and Tan, 2013) to the expectation-constraint case. We extend our previous work (Bayraktar and Yao, 2024) to the more complicated setting where the diffusion is controlled. Compared to that paper the topological properties of diffusion-control spaces and the corresponding measurability are more technically involved which complicate the arguments especially for the measurable selection for the super-solution side of DPP in the weak formulation. 

   more » « less
2. A Framework for the Dynamic Programming Principle and Martingale-Generated Control Correspondences

   https://doi.org/10.1007/s00245-019-09589-8  

   Fayvisovich, Roman; Žitković, Gordan (June 2019, Applied Mathematics & Optimization)

   We construct an abstract framework in which the dynamic programming principle (DPP) can be readily proven. It encompasses a broad range of common stochastic control problems in the weak formulation, and deals with problems in the “martingale formulation” with particular ease. We give two illustrations; first, we establish the DPP for general controlled diffusions and show that their value functions are viscosity solutions of the associated Hamilton–Jacobi–Bellman equations under minimal conditions. After that, we show how to treat singular control on the example of the classical monotone-follower problem. 

   more » « less
3. Exit Time Risk-Sensitive Control for Systems of Cooperative Agents

   https://doi.org/10.1007/s00498-019-0239-3  

   Dupuis, P. (January 2019, Mathematics of control signals and systems)

   null (Ed.)

   We study a sequence of many-agent exit time stochastic control problems, parameterized by the number of agents, with risk-sensitive cost structure. We identify a fully characterizing assumption, under which each such control problem corresponds to a risk-neutral stochastic control problem with additive cost, and sequentially to a risk-neutral stochastic control problem on the simplex that retains only the distribution of states of agents, while discarding further specific information about the state of each agent. Under some additional assumptions, we also prove that the sequence of value functions of these stochastic control problems converges to the value function of a deterministic control problem, which can be used for the design of nearly optimal controls for the original problem, when the number of agents is sufficiently large. 

   more » « less
4. Convergence Analysis for an Online Data-Driven Feedback Control Algorithm

   https://doi.org/10.3390/math12162584  

   Liang, Siming; Sun, Hui; Archibald, Richard; Bao, Feng (August 2024, Mathematics)

   This paper presents convergence analysis of a novel data-driven feedback control algorithm designed for generating online controls based on partial noisy observational data. The algorithm comprises a particle filter-enabled state estimation component, estimating the controlled system’s state via indirect observations, alongside an efficient stochastic maximum principle-type optimal control solver. By integrating weak convergence techniques for the particle filter with convergence analysis for the stochastic maximum principle control solver, we derive a weak convergence result for the optimization procedure in search of optimal data-driven feedback control. Numerical experiments are performed to validate the theoretical findings. 

   more » « less
5. A Probabilistic Approach to Extended Finite State Mean Field Games

   Carmona, Rene; Wang, Peiqi (August 2018, ArXivorg)

   We develop a probabilistic approach to continuous-time finite state mean field games. Based on an alternative description of continuous-time Markov chain by means of semimartingale and the weak formulation of stochastic optimal control, our approach not only allows us to tackle the mean field of states and the mean field of control in the same time, but also extend the strategy set of players from Markov strategies to closed-loop strategies. We show the existence and uniqueness of Nash equilibrium for the mean field game, as well as how the equilibrium of mean field game consists of an approximative Nash equilibrium for the game with finite number of players under different assumptions of structure and regularity on the cost functions and transition rate between states. 

   more » « less