Search for: All records

In this article, a new framework for the resilient control of continuous-time linear systems under denial-of-service (DoS) attacks and system uncertainty is presented. Integrating techniques from reinforcement learning and output regulation theory, it is shown that resilient optimal controllers can be learned directly from real-time state and input data collected from the systems subjected to attacks. Sufficient conditions are given under which the closed-loop system remains stable given any upper bound of DoS attack duration. Simulation results are used to demonstrate the efficacy of the proposed learning-based framework for resilient control under DoS attacks and model uncertainty. 

This paper studies the problem of data-driven combined longitudinal and lateral control of autonomous vehicles (AVs) such that the AV can stay within a safe but minimum distance from its leading vehicle and, at the same time, in the lane. Most of the existing methods for combined longitudinal and lateral control are either model-based or developed by purely data-driven methods such as reinforcement learning. Traditional model-based control approaches are insufficient to address the adaptive optimal control design issue for AVs in dynamically changing environments and subject to model uncertainty. Moreover, the conventional reinforcement learning approaches require a large volume of data, and cannot guarantee the stability of the vehicle. These limitations are addressed by integrating the advanced control theory with reinforcement learning techniques. To be more specific, by utilizing adaptive dynamic programming techniques and using the motion data collected from the vehicles, a policy iteration algorithm is proposed such that the control policy is iteratively optimized in the absence of the precise knowledge of the AV’s dynamical model. Furthermore, the stability of the AV is guaranteed with the control policy generated at each iteration of the algorithm. The efficiency of the proposed approach is validated by SUMO simulation, a microscopic traffic simulation platform, for different traffic scenarios. 

Feedback optimization aims at regulating the output of a dynamical system to a value that minimizes a cost function. This problem is beyond the reach of the traditional output regulation theory, because the desired value is generally unknown and the reference signal evolves according to a gradient flow using the system’s real-time output. This paper complements the output regulation theory with the nonlinear small-gain theory to address this challenge. Specifically, the authors assume that the cost function is strongly convex and the nonlinear dynamical system is in lower triangular form and is subject to parametric uncertainties and a class of external disturbances. An internal model is used to compensate for the effects of the disturbances while the cyclic small-gain theorem is invoked to address the coupling between the reference signal, the compensators, and the physical system. The proposed solution can guarantee the boundedness of the closed-loop signals and regulate the output of the system towards the desired minimizer in a global sense. Two numerical examples illustrate the effectiveness of the proposed method. 

In this paper, we have proposed a resilient reinforcement learning method for discrete-time linear systems with unknown parameters, under denial-of-service (DoS) attacks. The proposed method is based on policy iteration that learns the optimal controller from input-state data amidst DoS attacks. We achieve an upper bound for the DoS duration to ensure closed-loop stability. The resilience of the closed-loop system, when subjected to DoS attacks with the learned controller and an internal model, has been thoroughly examined. The effectiveness of the proposed methodology is demonstrated on an inverted pendulum on a cart. 

This paper studies the distributed feedback optimization problem for linear multi-agent systems without precise knowledge of local costs and agent dynamics. The proposed solution is based on a hierarchical approach that uses upper-level coordinators to adjust reference signals toward the global optimum and lower-level controllers to regulate agents’ outputs toward the reference signals. In the absence of precise information on local gradients and agent dynamics, an extremum-seeking mechanism is used to enforce a gradient descent optimization strategy, and an adaptive dynamic programming approach is taken to synthesize an internal-model-based optimal tracking controller. The whole procedure relies only on measurements of local costs and input-state data along agents’ trajectories. Moreover, under appropriate conditions, the closed-loop signals are bounded and the output of the agents exponentially converges to a small neighborhood of the desired extremum. A numerical example is conducted to validate the efficacy of the proposed method. 

This paper introduces a learning-based optimal control strategy enhanced with nonmodel-based state estimation to manage the complexities of lane-changing maneuvers in autonomous vehicles. Traditional approaches often depend on comprehensive system state information, which may not always be accessible or accurate due to dynamic traffic environments and sensor limitations. Our methodology dynamically adapts to these uncertainties and sensor noise by iteratively refining its control policy based on real-time sensor data and reconstructed states. We implemented an experimental setup featuring a scaled vehicle equipped with GPS, IMUs, and cameras, all processed through an Nvidia Jetson AGX Xavier board. This approach is pivotal as it addresses the limitations of simulations, which often fail to capture the complexity of dynamic real-world conditions. The results from real-world experiments demonstrate that our learning-based control system achieves smoother and more consistent lane-changing behavior compared to traditional direct measurement approaches. This paper underscores the effectiveness of integrating Adaptive Dynamic Programming (ADP) with state estimation techniques, as demonstrated through small-scale experiments. These experiments are crucial as they provide a practical validation platform that simulates real-world complexities, representing a significant advancement in the control systems used for autonomous driving. 

The majority of the past research dealing with lane-changing controller design of autonomous vehicles (𝐴𝑉 s) is based on the assumption of full knowledge of the model dynamics of the 𝐴𝑉 and the surrounding vehicles. However, in the real world, this is not a very realistic assumption as accurate dynamic models are difficult to obtain. Also, the dynamic model parameters might change over time due to various factors. Thus, there is a need for a learning-based lane change controller design methodology that can learn the optimal control policy in real time using sensor data. In this paper, we have addressed this need by introducing an optimal learningbased control methodology that can solve the real-time lane-changing problem of 𝐴𝑉 s, where the input-state data of the 𝐴𝑉 is utilized to generate a near-optimal lane-changing controller by approximate/adaptive dynamic programming (ADP) technique. In the case of this type of complex lane-changing maneuver, the lateral dynamics depend on the longitudinal velocity of the vehicle. If the longitudinal velocity is assumed constant, a linear parameter invariant model can be used. However, assuming constant velocity while performing a lane-changing maneuver is not a realistic assumption. This assumption might increase the risk of accidents, especially in the case of lane abortion when the surrounding vehicles are not cooperative. Thus, in this paper, the dynamics of the 𝐴𝑉 are assumed to be a linear parameter-varying system. Thus we have two challenges for the lane-changing controller design: parameter-varying, and unknown dynamics. With the help of both gain scheduling and ADP techniques combined, a learning-based control algorithm that can generate a near-optimal lane-changing controller without having to know the accurate dynamic model of the 𝐴𝑉 is proposed. The inclusion of a gain scheduling approach with ADP makes the controller applicable to non-linear and/or parameter-varying 𝐴𝑉 dynamics. The stability of the learning-based gain scheduling controller has also been rigorously proved. Moreover, a data-driven lane-changing decision-making algorithm is introduced that can make the 𝐴𝑉 perform a lane abortion if safety conditions are violated during a lane change. Finally, the proposed learning-based gain scheduling controller design algorithm and the lane-changing decision-making methodology are numerically validated using MATLAB, SUMO simulations, and the NGSIM dataset. 

This paper studies the effect of perturbations on the gradient flow of a general nonlinear programming problem, where the perturbation may arise from inaccurate gradient estimation in the setting of data-driven optimization. Under suitable conditions on the objective function, the perturbed gradient flow is shown to be small-disturbance input-to-state stable (ISS), which implies that, in the presence of a small-enough perturbation, the trajectories of the perturbed gradient flow must eventually enter a small neighborhood of the optimum. This work was motivated by the question of robustness of direct methods for the linear quadratic regulator problem, and specifically the analysis of the effect of perturbations caused by gradient estimation or round-off errors in policy optimization. We show small-disturbance ISS for three of the most common optimization algorithms: standard gradient flow, natural gradient flow, and Newton gradient flow. 

In this letter, we present an active learningbased control method for discrete-time linear systems with unknown parameters under denial-of-service (DoS) attacks. For any DoS duration parameter, using switching systems theory and adaptive dynamic programming, an active learning-based control technique is developed. A critical DoS average dwell-time is learned from online inputstate data, guaranteeing stability of the equilibrium point of the closed-loop system in the presence of DoS attacks with average dwell-time greater than or equal to the critical DoS average dwell-time. The effectiveness of the proposed methodology is illustrated via a numerical example. 

This paper presents a learning-based methodology for developing an optimal lane-changing control policy for a Remote Controlled (RC) car using real-time sensor data. The RC car is equipped with sensors including GPS, IMU devices, and a camera integrated in an Nvidia Jetson AGX Xavier board. By a novel Adaptive Dynamic Programming (ADP) algorithm, our RC car is capable of learning the optimal lane-changing strategies based on the real-time processed measurement from the sensors. The experimental outcomes show that our learning-based control algorithm can be effectively implemented, adapt to parameter changes, and complete the lane changing tasks in a short learning time with satisfactory performance.