An official website of the United States government
This paper proposes a novel learning-based adaptive optimal controller design method for a class of continuous-time linear time-delay systems. A key strategy is to exploit the state-of-the-art reinforcement learning (RL) techniques and adaptive dynamic programming (ADP), and propose a data-driven method to learn the near-optimal controller without the precise knowledge of system dynamics. Specifically, a value iteration (VI) algorithm is proposed to solve the infinite-dimensional Riccati equation for the linear quadratic optimal control problem of time-delay systems using finite samples of input-state trajectory data. It is rigorously proved that the proposed VI algorithm converges to the near-optimal solution. Compared with the previous literature, the nice features of the proposed VI algorithm are that it is directly developed for continuous-time systems without discretization and an initial admissible controller is not required for implementing the algorithm. The efficacy of the proposed methodology is demonstrated by two practical examples of metal cutting and autonomous driving.
more »
« less
Data-Driven Modeling of Linear Dynamical Systems with Quadratic Output in the AAA Framework
Abstract We extend the Adaptive Antoulas-Anderson () algorithm to develop a data-driven modeling framework for linear systems with quadratic output (). Such systems are characterized by two transfer functions: one corresponding to the linear part of the output and another one to the quadratic part. We first establish the joint barycentric representations and the interpolation theory for the two transfer functions of systems. This analysis leads to the proposed algorithm. We show that by interpolating the transfer function values on a subset of samples together with imposing a least-squares minimization on the rest, we construct reliable data-driven models. Two numerical test cases illustrate the efficiency of the proposed method.
more »
« less
- Award ID(s):
- 1819110
- PAR ID:
- 10364212
- Publisher / Repository:
- Springer Science + Business Media
- Date Published:
- Journal Name:
- Journal of Scientific Computing
- Volume:
- 91
- Issue:
- 1
- ISSN:
- 0885-7474
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract Periodic event‐triggered control (PETC) evaluates triggering conditions only at periodic sampling times, based on which it is decided whether the controller needs to be updated. This article investigates the global stabilization of nonlinear systems that are affected by external disturbances under PETC mechanisms. Sufficient conditions are provided to ensure the resulting closed‐loop system is input‐to‐state stable (ISS) for the state feedback and the observer‐based output feedback configurations separately. The sampling period and the triggering functions are chosen such that the ISS‐Lyapunov function of continuous dynamics is also the ISS‐Lyapunov function of the overall system. Based on that, sufficient conditions in the form of linear matrix inequalities are provided for the PETC design of incrementally quadratic nonlinear systems. Two simulation examples are provided to illustrate the effectiveness of the proposed method.more » « less
-
This paper studies the learning-based optimal control for a class of infinite-dimensional linear time-delay systems. The aim is to fill the gap of adaptive dynamic programming (ADP) where adaptive optimal control of infinite-dimensional systems is not addressed. A key strategy is to combine the classical model-based linear quadratic (LQ) optimal control of time-delay systems with the state-of-art reinforcement learning (RL) technique. Both the model-based and data-driven policy iteration (PI) approaches are proposed to solve the corresponding algebraic Riccati equation (ARE) with guaranteed convergence. The proposed PI algorithm can be considered as a generalization of ADP to infinite-dimensional time-delay systems. The efficiency of the proposed algorithm is demonstrated by the practical application arising from autonomous driving in mixed traffic environments, where human drivers’ reaction delay is considered.more » « less
-
Backward transfer is defined as the influence that new learning has on individuals’ prior ways of reasoning. In this article, we report on an exploratory study that examined the influences that quadratic functions instruction in real classrooms had on students’ prior ways of reasoning about linear functions. Two algebra classes and their teachers at two comprehensive high schools served as the participants. Both schools drew from lowsocioeconomic urban populations. The study involved paper-and-pencil assessments about linear functions that were administered before and after a four- to five-week instructional unit on quadratic functions. The teachers were instructed to teach the quadratic functions unit using their regular approach. Qualitative analysis revealed three kinds of backward transfer influences and each influence was related to a shift in how the students reasoned about functions in terms of an action or process view of functions. Additionally, features of the instruction in each class provided plausible explanations for the similarities and differences in backward transfer effects across the two classrooms. These results offer insights into backward transfer, the relationship between prior knowledge and new learning, aspects of reasoning about linear functions, and instructional approaches to teaching functions.more » « less
-
null (Ed.)This paper presents a data-driven algorithm to solve the problem of infinite-horizon linear quadratic regulation (LQR), for a class of discrete-time linear time-invariant systems subjected to state and control constraints. The problem is divided into a constrained finite-horizon LQR subproblem and an unconstrained infinite-horizon LQR subproblem, which can be solved directly from collected input/state data, separately. Under certain conditions, the combination of the solutions of the subproblems converges to the optimal solution of the original problem. The effectiveness of the proposed approach is validated by a numerical example.more » « less
