An official website of the United States government
We propose a reinforcement learning framework where an agent uses an internal nominal model for stochastic model predictive control (MPC) while compensating for a disturbance. Our work builds on the existing risk-aware optimal control with stochastic differential equations (SDEs) that aims to deal with such disturbance. However, the risk sensitivity and the noise strength of the nominal SDE in the riskaware optimal control are often heuristically chosen. In the proposed framework, the risk-taking policy determines the behavior of the MPC to be risk-seeking (exploration) or riskaverse (exploitation). Specifcally, we employ the risk-aware path integral control that can be implemented as a Monte-Carlo (MC) sampling with fast parallel simulations using a GPU. The MC sampling implementations of the MPC have been successful in robotic applications due to their real-time computation capability. The proposed framework that adapts the noise model and the risk sensitivity outperforms the standard model predictive path integ
more »
« less
Risk-Aware Model Predictive Control Enabled by Bayesian Learning
The performance of a model predictive controller depends on the accuracy of the objective and prediction model of the system. Although significant efforts have been dedicated to improving the robustness of model predictive control (MPC), they typically do not take a risk-averse perspective. In this paper, we propose a risk-aware MPC framework, which estimates the underlying parameter distribution using online Bayesian learning and derives a risk-aware control policy by reformulating classical MPC problems as Bayesian Risk Optimization (BRO) problems. The consistency of the Bayesian estimator and the convergence of the control policy are rigorously proved. Furthermore, we investigate the consistency requirement and propose a risk monitoring mechanism to guarantee the satisfaction of the consistency requirement. Simulation results demonstrate the effectiveness of the proposed approach.
more »
« less
- PAR ID:
- 10359107
- Date Published:
- Journal Name:
- Proceedings of 2022 American Control Conference
- Page Range / eLocation ID:
- 108 to 113
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
null (Ed.)This paper reports on developing an integrated framework for safety-aware informative motion planning suitable for legged robots. The information-gathering planner takes a dense stochastic map of the environment into account, while safety constraints are enforced via Control Barrier Functions (CBFs). The planner is based on the Incrementally-exploring Information Gathering (IIG) algorithm and allows closed-loop kinodynamic node expansion using a Model Predictive Control (MPC) formalism. Robotic exploration and information gathering problems are inherently path-dependent problems. That is, the information collected along a path depends on the state and observation history. As such, motion planning solely based on a modular cost does not lead to suitable plans for exploration. We propose SAFE-IIG, an integrated informative motion planning algorithm that takes into account: 1) a robot’s perceptual field of view via a submodular information function computed over a stochastic map of the environment, 2) a robot’s dynamics and safety constraints via discrete-time CBFs and MPC for closedloop multi-horizon node expansions, and 3) an automatic stopping criterion via setting an information-theoretic planning horizon. The simulation results show that SAFE-IIG can plan a safe and dynamically feasible path while exploring a dense map.more » « less
-
In this study, we address the problem of safe control in systems subject to state and input constraints by integrating the Control Barrier Function (CBF) into the Model Predictive Control (MPC) formulation. While CBF offers a conservative policy and traditional MPC lacks the safety guarantee beyond the finite horizon, the proposed scheme takes advantage of both MPC and CBF approaches to provide a guaranteed safe control policy with reduced conservatism and a shortened horizon. The proposed methodology leverages the sum-of-square (SOS) technique to construct CBFs that make forward invariant safe sets in the state space that are then used as a terminal constraint on the last predicted state. CBF invariant sets cover the state space around system fixed points. These islands of forward invariant CBF sets will be connected to each other using MPC. To do this, we proposed a technique to handle the MPC optimization problem subject to the combination of intersections and union of constraints. Our approach, termed Model Predictive Control Barrier Functions (MPCBF), is validated using numerical examples to demonstrate its efficacy, showing improved performance compared to classical MPC and CBF.more » « less
-
In this paper, we develop an optimal weight adap- tation strategy of model predictive control (MPC) for connected and automated vehicles (CAVs) in mixed traffic. We model the interaction between a CAV and a human-driven vehicle (HDV) as a simultaneous game and formulate a game-theoretic MPC problem to find a Nash equilibrium of the game. In the MPC problem, the weights in the HDV’s objective function can be learned online using moving horizon inverse reinforcement learning. Using Bayesian optimization, we propose a strategy to optimally adapt the weights in the CAV’s objective function so that the expected true cost when using MPC in simulations can be minimized. We validate the effectiveness of the optimal strategy by numerical simulations of a vehicle crossing example at an unsignalized intersection.more » « less
-
The active control of stormwater systems is a potential solution to increased street flooding in low-lying, low-relief coastal cities due to climate change and accompanying sea level rise. Model predictive control (MPC) has been shown to be a successful control strategy generally and as well as for managing urban drainage specifically. This research describes and demonstrates the implementation of MPC for urban drainage systems using open source software (Python and The United States Environmental Protection Agency (EPA) Storm Water Management Model (SWMM5). The system was demonstrated using a simplified use case in which an actively-controlled outlet of a detention pond is simulated. The control of the pond’s outlet influences the flood risk of a downstream node. For each step in the SWMM5 model, a series of policies for controlling the outlet are evaluated. The best policy is then selected using an evolutionary algorithm. The policies are evaluated against an objective function that penalizes primarily flooding and secondarily deviation of the detention pond level from a target level. Freely available Python libraries provide the key functionality for the MPC workflow: step-by-step running of the SWMM5 simulation, evolutionary algorithm implementation, and leveraging parallel computing. For perspective, the MPC results were compared to results from a rule-based approach and a scenario with no active control. The MPC approach produced a control policy that largely eliminated flooding (unlike the scenario with no active control) and maintained the detention pond’s water level closer to a target level (unlike the rule-based approach).more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.23919/ACC53348.2022.9867207
