An official website of the United States government
We tackle the problem of generalization to unseen configurations for dynamic tasks in the real world while learning from high-dimensional image input. The family of nonlinear dynamical system-based methods have successfully demonstrated dynamic robot behaviors but have difficulty in generalizing to unseen configurations as well as learning from image inputs. Recent works approach this issue by using deep network policies and reparameterize actions to embed the structure of dynamical systems but still struggle in domains with diverse configurations of image goals, and hence, find it difficult to generalize. In this paper, we address this dichotomy by leveraging embedding the structure of dynamical systems in a hierarchical deep policy learning framework, called Hierarchical Neural Dynamical Policies (H-NDPs). Instead of fitting deep dynamical systems to diverse data directly, H-NDPs form a curriculum by learning local dynamical system-based policies on small regions in state-space and then distill them into a global dynamical system-based policy that operates only from high-dimensional images. H-NDPs additionally provide smooth trajectories, a strong safety benefit in the real world. We perform extensive experiments on dynamic tasks both in the real world (digit writing, scooping, and pouring) and simulation (catching, throwing, picking). We show that H-NDPs are easily integrated with both imitation as well as reinforcement learning setups and achieve state-of-the-art results.
more »
« less
Active Learning for Nonlinear System Identification with Guarantees
While the identification of nonlinear dynamical systems is a fundamental building block of model-based reinforcement learning and feedback control, its sample complexity is only understood for systems that either have discrete states and actions or for systems that can be identified from data generated by i.i.d. random inputs. Nonetheless, many interesting dynamical systems have continuous states and actions and can only be identified through a judicious choice of inputs. Motivated by practical settings, we study a class of nonlinear dynamical systems whose state transitions depend linearly on a known feature embedding of state-action pairs. To estimate such systems in finite time identification methods must explore all directions in feature space. We propose an active learning approach that achieves this by repeating three steps: trajectory planning, trajectory tracking, and re-estimation of the system from all available data. We show that our method estimates nonlinear dynamical systems at a parametric rate, similar to the statistical rate of standard linear regression.
more »
« less
- Award ID(s):
- 1931853
- PAR ID:
- 10338178
- Editor(s):
- Fukumizu, Kenji
- Date Published:
- Journal Name:
- Journal of machine learning research
- Volume:
- 23
- ISSN:
- 1532-4435
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
This paper investigates a control strategy in which the state of a dynamical system is driven slowly along a trajectory of stable equilibria. This trajectory is a continuum set of points in the state space, each one representing a stable equilibrium of the system under some constant control input. Along the continuous trajectory of such constant control inputs, a slowly varying control is then applied to the system, aimed to create a stable quasistatic equilibrium that slowly moves along the trajectory of equilibria. As a stable equilibrium attracts the state of system within its vicinity, by moving the equilibrium slowly along the trajectory of equilibria, the state of system travels near this trajectory alongside the equilibrium. Despite the disadvantage of being slow, this control strategy is attractive for certain applications, as it can be implemented based only on partial knowledge of the system dynamics. This feature is in particular important for the complex systems for which detailed dynamical models are not available.more » « less
-
Data-driven approaches are increasingly popular for identifying dynamical systems due to improved accuracy and availability of sensor data. However, relying solely on data for identification does not guarantee that the identified systems will maintain their physical properties or that the predicted models will generalize well. In this paper, we propose a novel method for data-driven system identification by integrating a neural network as the first-order derivative of the learned dynamics in a Taylor series instead of learning the dynamical function directly. In addition, for dynamical systems with known monotonic properties, our approach can ensure monotonicity by constraining the neural network derivative to be non-positive or non-negative to the respective inputs, resulting in Monotonic Taylor Neural Networks (MTNN). Such constraints are enforced by either a specialized neural network architecture or regularization in the loss function for training. The proposed method demonstrates better performance compared to methods without the physics-based monotonicity constraints when tested on experimental data from an HVAC system and a temperature control testbed. Furthermore, MTNN shows good performance in a control application of a model predictive controller for a nonlinear MIMO system, illustrating the practical application of our method.more » « less
-
This paper presents a variational Bayesian inference Neural Network (BNN) approach to quantify uncertainties in matrix function estimation for the state-space linear parameter-varying (LPV) model identification problem using only inputs/outputs data. The proposed method simultaneously estimates states and posteriors of matrix functions given data. In particular, states are estimated by reaching a consensus between an estimator based on past system trajectory and an estimator by recurrent equations of states; posteriors are approximated by minimizing the Kullback–Leibler (KL) divergence between the parameterized posterior distribution and the true posterior of the LPV model parameters. Furthermore, techniques such as transfer learning are explored in this work to reduce computational cost and prevent convergence failure of Bayesian inference. The proposed data-driven method is validated using experimental data for identification of a control-oriented reactivity controlled compression ignition (RCCI) engine model.more » « less
-
Abstract Identifying the governing equations of a nonlinear dynamical system is key to both understanding the physical features of the system and constructing an accurate model of the dynamics that generalizes well beyond the available data. Achieving this kind of interpretable system identification is even more difficult for partially observed systems. We propose a machine learning framework for discovering the governing equations of a dynamical system using only partial observations, combining an encoder for state reconstruction with a sparse symbolic model. The entire architecture is trained end-to-end by matching the higher-order symbolic time derivatives of the sparse symbolic model with finite difference estimates from the data. Our tests show that this method can successfully reconstruct the full system state and identify the equations of motion governing the underlying dynamics for a variety of ordinary differential equation (ODE) and partial differential equation (PDE) systems.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- The DOI is not currently available.
