More Like this

1. Learning Parametric Constraints in High Dimensions from Demonstrations

   Chou, Glen ; Ozay, Necmiye ; Berenson, Dmitry ( October 2019 , Conference on Robot Learning (CoRL))

   We present a scalable algorithm for learning parametric constraints in high dimensions from safe expert demonstrations. To reduce the ill-posedness of the constraint recovery problem, our method uses hit-and-run sampling to generate lower cost, and thus unsafe, trajectories. Both safe and unsafe trajectories are used to obtain a representation of the unsafe set that is compatible with the data by solving an integer program in that representation’s parameter space. Our method can either leverage a known parameterization or incrementally grow a parameterization while remaining consistent with the data, and we provide theoretical guarantees on the conservativeness of the recovered unsafe set. We evaluate our method on high-dimensional constraints for high-dimensional systems by learning constraints for 7-DOF arm, quadrotor, and planar pushing examples, and show that our method outperforms baseline approaches. 

   more » « less
2. Uncertainty-Aware Constraint Learning for Adaptive Safe Motion Planning from Demonstrations

   Chou, Glen ; Ozay, Necmiye ; Berenson, Dmitry ( January 2020 , Conference on Robot Learning)

   null (Ed.)

   We present a method for learning to satisfy uncertain constraints fromdemonstrations. Our method uses robust optimization to obtain a belief over thepotentially infinite set of possible constraints consistent with the demonstrations,and then uses this belief to plan trajectories that trade off performance with sat-isfying the possible constraints. We use these trajectories in a closed-loop policythat executes and replans using belief updates, which incorporate data gatheredduring execution. We derive guarantees on the accuracy of our constraint beliefand probabilistic guarantees on plan safety. We present results on a 7-DOF armand 12D quadrotor, showing our method can learn to satisfy high-dimensional (upto 30D) uncertain constraints, and outperforms baselines in safety and efficiency. 

   more » « less
3. Adaptive safe reinforcement learning‐enabled optimization of battery fast‐charging protocols

   https://doi.org/10.1002/aic.18605  

   Chowdhury, Myisha_A ; Al‐Wahaibi, Saif_SS ; Lu, Qiugang ( September 2024 , AIChE Journal)

   Optimizing charging protocols is critical for reducing battery charging time and decelerating battery degradation in applications such as electric vehicles. Recently, reinforcement learning (RL) methods have been adopted for such purposes. However, RL‐based methods may not ensure system (safety) constraints, which can cause irreversible damages to batteries and reduce their lifetime. To this end, this article proposes an adaptive and safe RL framework to optimize fast charging strategies while respecting safety constraints with a high probability. In our method, any unsafe action that the RL agent decides will be projected into a safety region by solving a constrained optimization problem. The safety region is constructed using adaptive Gaussian process (GP) models, consisting of static and dynamic GPs, that learn from online experience to adaptively account for any changes in battery dynamics. Simulation results show that our method can charge the batteries rapidly with constraint satisfaction under varying operating conditions.

    

   more » « less
4. Safe and Efficient Reinforcement Learning using Disturbance-Observer-Based Control Barrier Functions

   Cheng, Yikun ; Zhao, Pan ; Hovakimyan, Naira ( June 2023 , Proceedings of The 5th Annual Learning for Dynamics and Control Conference)

   Matni, Nikolai ; Morari, Manfred ; Pappas, George J. (Ed.)

   Safe reinforcement learning (RL) with assured satisfaction of hard state constraints during training has recently received a lot of attention. Safety filters, e.g., based on control barrier functions (CBFs), provide a promising way for safe RL via modifying the unsafe actions of an RL agent on the fly. Existing safety filter-based approaches typically involve learning of uncertain dynamics and quantifying the learned model error, which leads to conservative filters before a large amount of data is collected to learn a good model, thereby preventing efficient exploration. This paper presents a method for safe and efficient RL using disturbance observers (DOBs) and control barrier functions (CBFs). Unlike most existing safe RL methods that deal with hard state constraints, our method does not involve model learning, and leverages DOBs to accurately estimate the pointwise value of the uncertainty, which is then incorporated into a robust CBF condition to generate safe actions. The DOB-based CBF can be used as a safety filter with model-free RL algorithms by minimally modifying the actions of an RL agent whenever necessary to ensure safety throughout the learning process. Simulation results on a unicycle and a 2D quadrotor demonstrate that the proposed method outperforms a state-of-the-art safe RL algorithm using CBFs and Gaussian processes-based model learning, in terms of safety violation rate, and sample and computational efficiency. 

   more » « less
5. Safe and Efficient Reinforcement Learning Using Disturbance-Observer-Based Control Barrier Functions

   Cheng, Yikun ; Zhao, Pan ; Hovakimyan, Naira ( June 2023 , Proceedings of Machine Learning Research)

   Safe reinforcement learning (RL) with assured satisfaction of hard state constraints during training has recently received a lot of attention. Safety filters, e.g., based on control barrier functions (CBFs), provide a promising way for safe RL via modifying the unsafe actions of an RL agent on the fly. Existing safety filter-based approaches typically involve learning of uncertain dynamics and quantifying the learned model error, which leads to conservative filters before a large amount of data is collected to learn a good model, thereby preventing efficient exploration. This paper presents a method for safe and efficient RL using disturbance observers (DOBs) and control barrier functions (CBFs). Unlike most existing safe RL methods that deal with hard state constraints, our method does not involve model learning, and leverages DOBs to accurately estimate the pointwise value of the uncertainty, which is then incorporated into a robust CBF condition to generate safe actions. The DOB-based CBF can be used as a safety filter with model-free RL algorithms by minimally modifying the actions of an RL agent whenever necessary to ensure safety throughout the learning process. Simulation results on a unicycle and a 2D quadrotor demonstrate that the proposed method outperforms a state-of-the-art safe RL algorithm using CBFs and Gaussian processes-based model learning, in terms of safety violation rate, and sample and computational efficiency. 

   more » « less