More Like this

1. Context-aware controller inference for stabilizing dynamical systems from scarce data

   https://doi.org/10.1098/rspa.2022.0506  

   Werner, Steffen W.; Peherstorfer, Benjamin (February 2023, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences)

   This work introduces a data-driven control approach for stabilizing high-dimensional dynamical systems from scarce data. The proposed context-aware controller inference approach is based on the observation that controllers need to act locally only on the unstable dynamics to stabilize systems. This means it is sufficient to learn the unstable dynamics alone, which are typically confined to much lower dimensional spaces than the high-dimensional state spaces of all system dynamics and thus few data samples are sufficient to identify them. Numerical experiments demonstrate that context-aware controller inference learns stabilizing controllers from orders of magnitude fewer data samples than traditional data-driven control techniques and variants of reinforcement learning. The experiments further show that the low data requirements of context-aware controller inference are especially beneficial in data-scarce engineering problems with complex physics, for which learning complete system dynamics is often intractable in terms of data and training costs. 

   more » « less
2. Energetically-optimal Discrete And Continuous Stabilization Of The Rimless Wheel With Torso

   Sirichotiyakul, Wankun; Satici, Aykut C; Sanchez, Sebastian; Bhounsule, Pranav A (October 2019, ASME-International Design Engineering & Technical Conference)

   In this work, we discuss the modeling, control, and implementation of a rimless wheel with a torso. We derive and compare two control methodologies: a discrete-time controller (DT) that updates the controls once-per-step and a continuous-time controller (CT) that updates gains continuously. For the discrete controller, we use least-squares estimation method to approximate the Poincare ́ map on a certain section and use discrete- linear-quadratic-regulator (DQLR) to stabilize a (closed-form) linearization of this map. For the continuous controller, we introduce moving Poincare ́ sections and stabilize the transverse dynamics along these moving sections. For both controllers, we estimate the region of attraction of the closed-loop system using sum-of-squares methods. Analysis of the impact map yields a refinement of the controller that stabilizes a steady-state walking gait with minimal energy loss. We present both simulation and experimental results that support the validity of the proposed approaches. We find that the CT controller has a larger region of attraction and smoother stabilization as compared with the DT controller. 

   more » « less
3. How Are Learned Perception-Based Controllers Impacted by the Limits of Robust Control?

   Xu, Jingxi; Lee, Bruce; Matni, Nikolai; Jayaraman, Dinesh (July 2021, Proceedings of the 3rd Conference on Learning for Dynamics and Control)

   null (Ed.)

   The difficulty of optimal control problems has classically been characterized in terms of system properties such as minimum eigenvalues of controllability/observability gramians. We revisit these characterizations in the context of the increasing popularity of data-driven techniques like reinforcement learning (RL) in control settings where input observations are high-dimensional images and transition dynamics are not known beforehand. Specifically, we ask: to what extent are quantifiable control and perceptual difficulty metrics of a control task predictive of the performance of various families of data-driven controllers? We modulate two different types of partial observability in a cartpole “stick-balancing” problem–the height of one visible fixation point on the cartpole, which can be used to tune fundamental limits of performance achievable by any controller, and by using depth or RGB image observations of the scene, we add different levels of perception noise without affecting system dynamics. In these settings, we empirically study two popular families of controllers: RL and system identification-based H-infinity control, using visually estimated system state. Our results show the fundamental limits of robust control have corresponding implications for the sample-efficiency and performance of learned perception-based controllers. 

   more » « less
4. Can Direct Latent Model Learning Solve Linear Quadratic Gaussian Control?

   Tian, Yi; Zhang, Kaiqing; Tedrake, Russ; Sra, Suvrit (January 2023, Conference on Learning for Dynamics and Control)

   We study the task of learning state representations from potentially high-dimensional observations, with the goal of controlling an unknown partially observable system. We pursue a direct latent model learning approach, where a dynamic model in some latent state space is learned by predicting quantities directly related to planning (e.g., costs) without reconstructing the observations. In particular, we focus on an intuitive cost-driven state representation learning method for solving Linear Quadratic Gaussian (LQG) control, one of the most fundamental partially observable control problems. As our main results, we establish finite-sample guarantees of finding a near-optimal state representation function and a near-optimal controller using the directly learned latent model. To the best of our knowledge, despite various empirical successes, prior to this work it was unclear if such a cost-driven latent model learner enjoys finite-sample guarantees. Our work underscores the value of predicting multi-step costs, an idea that is key to our theory, and notably also an idea that is known to be empirically valuable for learning state representations. 

   more » « less
5. Unsupervised Learning of Visual 3D Keypoints for Control

   Chen, Boyuan; Abbeel, Pieter; Pathak, Deepak (January 2021, International Conference on Machine Learning)

   Learning sensorimotor control policies from high-dimensional images crucially relies on the quality of the underlying visual representations. Prior works show that structured latent space such as visual keypoints often outperforms unstructured representations for robotic control. However, most of these representations, whether structured or unstructured are learned in a 2D space even though the control tasks are usually performed in a 3D environment. In this work, we propose a framework to learn such a 3D geometric structure directly from images in an end-to-end unsupervised manner. The input images are embedded into latent 3D keypoints via a differentiable encoder which is trained to optimize both a multi-view consistency loss and downstream task objective. These discovered 3D keypoints tend to meaningfully capture robot joints as well as object movements in a consistent manner across both time and 3D space. The proposed approach outperforms prior state-of-art methods across a variety of reinforcement learning benchmarks. Code and videos at https://buoyancy99.github.io/unsup-3d-keypoints/ 

   more » « less