More Like this

1. Optimal Sensor Gain Control for Minimum-Information Estimation of Continuous-Time Gauss-Markov Processes

   Zinage, V. ; Tanaka, T. ; & Ugrinovskii, V. ( June 2022 , American Control Conference)

   We consider the scenario in which a continuous-time Gauss-Markov process is estimated by the Kalman-Bucy filter over a Gaussian channel (sensor) with a variable sensor gain. The problem of scheduling the sensor gain over a finite time interval to minimize the weighted sum of the data rate (the mutual information between the sensor output and the underlying Gauss-Markov process) and the distortion (the mean-square estimation error) is formulated as an optimal control problem. A necessary optimality condition for a scheduled sensor gain is derived based on Pontryagin’s minimum principle. For a scalar problem, we show that an optimal sensor gain control is of bang-bang type, except the possibility of taking an intermediate value when there exists a stationary point on the switching surface in the phase space of canonical dynamics. Furthermore, we show that the number of switches is at most two and the time instants at which the optimal gain must be switched can be computed from the analytical solutions to the canonical equations. 

   more » « less
2. Duality-Based Stochastic Policy Optimization for Estimation with Unknown Noise Covariances

   Shahriar Talebi, Amirhossein Taghvaei ( May 2023 , Proceedings of the American Control Conference)

   Duality of control and estimation allows mapping recent advances in data-guided control to the estimation setup. This paper formalizes and utilizes such a mapping to consider learning the optimal (steady-state) Kalman gain when process and measurement noise statistics are unknown. Specifically, building on the duality between synthesizing optimal control and estimation gains, the filter design problem is formalized as direct policy learning. In this direction, the duality is used to extend existing theoretical guarantees of direct policy updates for Linear Quadratic Regulator (LQR) to establish global convergence of the Gradient Descent (GD) algorithm for the estimation problem–while addressing subtle differences between the two synthesis problems. Subsequently, a Stochastic Gradient Descent (SGD) approach is adopted to learn the optimal Kalman gain without the knowledge of noise covariances. The results are illustrated via several numerical examples. 

   more » « less
3. Multi-agent infinite horizon persistent monitoring of targets with uncertain states in multi-dimensional environments

   Pinto, Samuel C ; Andersson, Sean B ; Hendrickx, Julien M ; Cassandras, Christos G ( January 2020 , Proc. IFAC World Congress)

   This paper investigates the problem of persistent monitoring, where a finite set of mobile agents persistently visits a finite set of targets in a multi-dimensional environment. The agents must estimate the targets’ internal states and the goal is to minimize the mean squared estimation error over time. The internal states of the targets evolve with linear stochastic dynamics and thus the optimal estimator is a Kalman-Bucy Filter. We constrain the trajectories of the agents to be periodic and represented by a truncated Fourier series. Taking advantage of the periodic nature of this solution, we define the infinite horizon version of the problem and explore the property that the mean estimation squared error converges to a limit cycle. We present a technique to compute online the gradient of the steady state mean estimation error of the targets’ states with respect to the parameters defining the trajectories and use a gradient descent scheme to obtain locally optimal movement schedules. This scheme allows us to address the infinite horizon problem with only a small number of parameters to be optimized. 

   more » « less
4. Proximal Recursion for the Wonham Filter

   https://doi.org/10.1109/CDC40024.2019.9030018  

   Halder, Abhishek ; Georgiou, Tryphon T. ( December 2019 , 2019 IEEE 58th Conference on Decision and Control (CDC))

   This paper contributes to the emerging viewpoint that governing equations for dynamic state estimation, conditioned on the history of noisy measurements, can be viewed as gradient flow on the manifold of joint probability density functions with respect to suitable metrics. Herein, we focus on the Wonham filter where the prior dynamics is given by a continuous time Markov chain on a finite state space; the measurement model includes noisy observation of the (possibly nonlinear function of) state. We establish that the posterior flow given by the Wonham filter can be viewed as the small time-step limit of proximal recursions of certain functionals on the probability simplex. The results of this paper extend our earlier work where similar proximal recursions were derived for the Kalman-Bucy filter. 

   more » « less
5. Optimal Control of a Large Population of Randomly Interrogated Interacting Agents

   https://doi.org/10.1109/TAC.2022.3202838  

   Komaee, Arash ( August 2022 , IEEE Transactions on Automatic Control)

   This article investigates a stochastic optimal control problem with linear Gaussian dynamics, quadratic performance measure, but non-Gaussian observations. The linear Gaussian dynamics characterizes a large number of interacting agents evolving under a centralized control and external disturbances. The aggregate state of the agents is only partially known to the centralized controller by means of the samples taken randomly in time and from anonymous randomly selected agents. Due to removal of the agent identity from the samples, the observation set has a non-Gaussian structure, and as a consequence, the optimal control law that minimizes a quadratic cost is essentially nonlinear and infinite-dimensional, for any finite number of agents. For infinitely many agents, however, this paper shows that the optimal control law is the solution to a reduced order, finite-dimensional linear quadratic Gaussian problem with Gaussian observations sampled only in time. For this problem, the separation principle holds and is used to develop an explicit optimal control law by combining a linear quadratic regulator with a separately designed finite-dimensional minimum mean square error state estimator. Conditions are presented under which this simple optimal control law can be adopted as a suboptimal control law for finitely many agents. 

   more » « less