More Like this

1. Meta-learning linear quadratic regulators: a policy gradient MAML approach for model-free LQR

   Toso, Leonardo; Zhan, Donglin; Anderson, James; Wang, Han (July 2024, Proceedings of Machine Learning Research (PMLR))

   Abate, A; Cannon, M; Margellos, K; Papachristodoulou, A (Ed.)

   We investigate the problem of learning linear quadratic regulators (LQR) in a multi-task, heterogeneous, and model-free setting. We characterize the stability and personalization guarantees of a policy gradient-based (PG) model-agnostic meta-learning (MAML) (Finn et al., 2017) approach for the LQR problem under different task-heterogeneity settings. We show that our MAML-LQR algorithm produces a stabilizing controller close to each task-specific optimal controller up to a task-heterogeneity bias in both model-based and model-free learning scenarios. Moreover, in the model-based setting, we show that such a controller is achieved with a linear convergence rate, which improves upon sub-linear rates from existing work. Our theoretical guarantees demonstrate that the learned controller can efficiently adapt to unseen LQR tasks. 

   more » « less
2. Lie Algebraic Cost Function Design for Control on Lie Groups

   Sangli Teng, William Clark (January 2022, Proc. IEEE CDC 2022)

   This paper presents a control framework on Lie groups by designing the control objective in its Lie algebra. Control on Lie groups is challenging due to its nonlinear nature and difficulties in system parameterization. Existing methods to design the control objective on a Lie group and then derive the gradient for controller design are non-trivial and can result in slow convergence in tracking control. We show that with a proper left-invariant metric, setting the gradient of the cost function as the tracking error in the Lie algebra leads to a quadratic Lyapunov function that enables globally exponential convergence. In the PD control case, we show that our controller can maintain an exponential convergence rate even when the initial error is approaching π in SO(3). We also show the merit of this proposed framework in trajectory optimization. The proposed cost function enables the iterative Linear Quadratic Regulator (iLQR) to converge much faster than the Differential Dynamic Programming (DDP) with a well-adopted cost function when the initial trajectory is poorly initialized on SO(3). 

   more » « less
3. A Piecewise Lyapunov Analysis of Sub-quadratic SGD: Applications to Robust and Quantile Regression

   https://doi.org/10.1145/3744970.3727269  

   Zhang, Yixuan; Huo, Dongyan Lucy; Chen, Yudong; Xie, Qiaomin (June 2025, ACM SIGMETRICS Performance Evaluation Review)

   Motivated by robust and quantile regression problems, we investigate the stochastic gradient descent (SGD) algorithm for minimizing an objective functionfthat is locally strongly convex with a sub--quadratic tail. This setting covers many widely used online statistical methods. We introduce a novel piecewise Lyapunov function that enables us to handle functionsfwith only first-order differentiability, which includes a wide range of popular loss functions such as Huber loss. Leveraging our proposed Lyapunov function, we derive finite-time moment bounds under general diminishing stepsizes, as well as constant stepsizes. We further establish the weak convergence, central limit theorem and bias characterization under constant stepsize, providing the first geometrical convergence result for sub--quadratic SGD. Our results have wide applications, especially in online statistical methods. In particular, we discuss two applications of our results. 1) Online robust regression: We consider a corrupted linear model with sub--exponential covariates and heavy--tailed noise. Our analysis provides convergence rates comparable to those for corrupted models with Gaussian covariates and noise. 2) Online quantile regression: Importantly, our results relax the common assumption in prior work that the conditional density is continuous and provide a more fine-grained analysis for the moment bounds. 

   more » « less
4. On Global and Local Convergence of Iterative Linear Quadratic Optimization Algorithms for Discrete Time Nonlinear Control

   Roulet, V; Srinivasa, S; Fazel, M; Harchaoui, Z (May 2025, Journal of machine learning research)

   Jaggi, Martin (Ed.)

   A classical approach for solving discrete time nonlinear control on a nite horizon consists in repeatedly minimizing linear quadratic approximations of the original problem around current candidate solutions. While widely popular in many domains, such an approach has mainly been analyzed locally. We provide detailed convergence guarantees to stationary points as well as local linear convergence rates for the Iterative Linear Quadratic Regulator (ILQR) algorithm and its Di erential Dynamic Programming (DDP) variant. For problems without costs on control variables, we observe that global convergence to minima can be ensured provided that the linearized discrete time dynamics are surjective, costs on the state variables are gradient dominated. We further detail quadratic local convergence when the costs are self-concordant. We show that surjectivity of the linearized dynamics hold for appropriate discretization schemes given the existence of a feedback linearization scheme. We present complexity bounds of algorithms based on linear quadratic approximations through the lens of generalized Gauss-Newton methods. Our analysis uncovers several convergence phases for regularized generalized Gauss-Newton algorithms. 

   more » « less
5. Weighted L ² -contractivity of Langevin dynamics with singular potentials

   https://doi.org/10.1088/1361-6544/ac4152  

   Camrud, Evan; Herzog, David P; Stoltz, Gabriel; Gordina, Maria (December 2021, Nonlinearity)

   Abstract Convergence to equilibrium of underdamped Langevin dynamics is studied under general assumptions on the potential U allowing for singularities. By modifying the direct approach to convergence in L 2 pioneered by Hérau and developed by Dolbeault et al , we show that the dynamics converges exponentially fast to equilibrium in the topologies L 2 (d μ ) and L 2 ( W * d μ ), where μ denotes the invariant probability measure and W * is a suitable Lyapunov weight. In both norms, we make precise how the exponential convergence rate depends on the friction parameter γ in Langevin dynamics, by providing a lower bound scaling as min( γ , γ −1 ). The results hold for usual polynomial-type potentials as well as potentials with singularities such as those arising from pairwise Lennard-Jones interactions between particles. 

   more » « less