Search for: All records

Inverse reinforcement Learning (IRL) has emerged as a powerful paradigm for extracting expert skills from observed behavior, with applications ranging from autonomous systems to humanrobot interaction. However, the identifiability issue within IRL poses a significant challenge, as multiple reward functions can explain the same observed behavior. This paper provides a linear algebraic characterization of several identifiability notions for an entropy-regularized finite horizon Markov decision process (MDP). Moreover, our approach allows for the seamless integration of prior knowledge, in the form of featurized reward functions, to enhance the identifiability of IRL problems. The results are demonstrated with experiments on a grid world environment. 

Existing safety control methods for non-stochastic systems become undefined when the system operates outside the maximal robust controlled invariant set (RCIS), making those methods vulnerable to unexpected initial states or unmodeled disturbances. In this work, we propose a novel safety control framework that can work both inside and outside the maximal RCIS, by identifying a worst-case disturbance that can be handled at each state and constructing the control inputs robust to that worst-case disturbance model. We show that such disturbance models and control inputs can be jointly computed by considering an invariance problem for an auxiliary system. Finally, we demonstrate the efficacy of our method both in simulation and in a drone experiment. 

Inspired by the work of Tsiamis et al. [1], in this paper we study the statistical hardness of learning to stabilize linear time-invariant systems. Hardness is measured by the number of samples required to achieve a learning task with a given probability. The work in [1] shows that there exist system classes that are hard to learn to stabilize with the core reason being the hardness of identification. Here we present a class of systems that can be easy to identify, thanks to a non-degenerate noise process that excites all modes, but the sample complexity of stabilization still increases exponentially with the system dimension. We tie this result to the hardness of co-stabilizability for this class of systems using ideas from robust control. 

Linear temporal logic (LTL) with the knowledge operator, denoted as KLTL, is a variant of LTL that incorporates what an agent knows or learns at run-time into its specification. Therefore it is an appropriate logical formalism to specify tasks for systems with unknown components that are learned or estimated at run-time. In this paper, we consider a linear system whose system matrices are unknown but come from an a priori known finite set. We introduce a form of KLTL that can be interpreted over the trajectories of such systems. Finally, we show how controllers that guarantee satisfaction of specifications given in fragments of this form of KLTL can be synthesized using optimization techniques. Our results are demonstrated in simulation and on hardware in a drone scenario where the task of the drone is conditioned on its health status, which is unknown a priori and discovered at run-time. 

Falls during sit-to-stand are a common cause of injury. The ability to perform this movement with ease is itself correlated with a lower likelihood of falling. However, a rigorous mathematical understanding of stability during sit-to-stand does not currently exist, particularly in different environments and under different movement control strategies. Having the means to isolate the different factors contributing to instability during sit-to-stand could have great clinical utility, guiding the treatment of fall-prone individuals. In this work, we show that the region of stable human movement during sit-to-stand can be formulated as the backward reachable set of a controlled invariant target, even under state-dependent input constraints representing variability in the environment. This region represents the ‘best-case’ boundaries of stable sit-to-stand motion. We call this the stabilizable region and show that it can be easily computed using existing backward reachability tools. Using a dataset of humans performing sit-to-stand under perturbations, we also demonstrate that the controlled invariance and backward reachability approach is better able to differentiate between a true loss of stability versus a change in control strategy, as compared with other methods.