More Like this

1. Nonparametric Adaptive Robust Control under Model Uncertainty

   https://doi.org/10.1137/22M1481609  

   Bayraktar, Erhan; Chen, Tao (October 2023, SIAM Journal on Control and Optimization)

   Yin, George (Ed.)

   We consider a discrete time stochastic Markovian control problem under model uncertainty. Such uncertainty not only comes from the fact that the true probability law of the underlying stochastic process is unknown, but the parametric family of probability distributions which the true law belongs to is also unknown. We propose a nonparametric adaptive robust control methodology to deal with such problem where the relevant system random noise is, for simplicity, assumed to be i.i.d. and onedimensional. Our approach hinges on the following building concepts: first, using the adaptive robust paradigm to incorporate online learning and uncertainty reduction into the robust control problem; second, learning the unknown probability law through the empirical distribution, and representing uncertainty reduction in terms of a sequence of Wasserstein balls around the empirical distribution; third, using Lagrangian duality to convert the optimization over Wasserstein balls to a scalar optimization problem, and adopting a machine learning technique to achieve efficient computation of the optimal control. We illustrate our methodology by considering a utility maximization problem. Numerical comparisons show that the nonparametric adaptive robust control approach is preferable to the traditional robust frameworks 

   more » « less
2. Data-Driven Chance Constrained Control using Kernel Distribution Embeddings

   Thorpe, Adam; and Oishi, Meeko (May 2021, Proceedings of The 4th Annual Learning for Dynamics and Control Conference)

   We present a data-driven algorithm for efficiently computing stochastic control policies for general joint chance constrained optimal control problems. Our approach leverages the theory of kernel distribution embeddings, which allows representing expectation operators as inner products in a reproducing kernel Hilbert space. This framework enables approximately reformulating the original problem using a dataset of observed trajectories from the system without imposing prior assumptions on the parameterization of the system dynamics or the structure of the uncertainty. By optimizing over a finite subset of stochastic open-loop control trajectories, we relax the original problem to a linear program over the control parameters that can be efficiently solved using standard convex optimization techniques. We demonstrate our proposed approach in simulation on a system with nonlinear non-Markovian dynamics navigating in a cluttered environment. 

   more » « less
3. Finite-Horizon Separation-Based Covariance Control for Discrete-Time Stochastic Linear Systems

   https://doi.org/10.1109/CDC.2018.8619542  

   Bakolas, Efstathios (December 2018, 2018 IEEE Conference on Decision and Control (CDC))

   In this paper, we address a finite-horizon stochastic optimal control problem with covariance assignment and input energy constraints for discrete-time stochastic linear systems with partial state information. In our approach, we consider separation-based control policies that correspond to sequences of control laws that are affine functions of either the complete history of the output estimation errors, that is, the differences between the actual output measurements and their corresponding estimated outputs produced by a discrete-time Kalman filter, or a truncation of the same history. This particular feedback parametrization allows us to associate the stochastic optimal control problem with a tractable semidefinite (convex) program. We argue that the proposed procedure for the reduction of the stochastic optimal control problem to a convex program has significant advantages in terms of improved scalability and tractability over the approaches proposed in the relevant literature. 

   more » « less
4. A Fixed-Time Stable Adaptation Law for Safety-Critical Control under Parametric Uncertainty

   Black, Mitchell; Arabi, Ehsan; Panagou, Dimitra (January 2021, 2021 European Control Conference)

   We present a novel technique for solving the problem of safe control for a general class of nonlinear, control-affine systems subject to parametric model uncertainty. Invoking Lyapunov analysis and the notion of fixed-time stability (FxTS), we introduce a parameter adaptation law which guarantees convergence of the estimates of unknown parameters in the system dynamics to their true values within a fixed-time independent of the initial parameter estimation error. We then synthesize the adaptation law with a robust, adaptive control barrier function (RaCBF) based quadratic program to compute safe control inputs despite the considered model uncertainty. To corroborate our results, we undertake a comparative case study on the efficacy of this result versus other recent approaches in the literature to safe control under uncertainty, and close by highlighting the value of our method in the context of an automobile overtake scenario. 

   more » « less
5. Exit Time Risk-Sensitive Control for Systems of Cooperative Agents

   https://doi.org/10.1007/s00498-019-0239-3  

   Dupuis, P. (January 2019, Mathematics of control signals and systems)

   null (Ed.)

   We study a sequence of many-agent exit time stochastic control problems, parameterized by the number of agents, with risk-sensitive cost structure. We identify a fully characterizing assumption, under which each such control problem corresponds to a risk-neutral stochastic control problem with additive cost, and sequentially to a risk-neutral stochastic control problem on the simplex that retains only the distribution of states of agents, while discarding further specific information about the state of each agent. Under some additional assumptions, we also prove that the sequence of value functions of these stochastic control problems converges to the value function of a deterministic control problem, which can be used for the design of nearly optimal controls for the original problem, when the number of agents is sufficiently large. 

   more » « less