Search for: All records

Abstract Using the context of trajectory estimation and tracking for multirotor unmanned aerial vehicles (UAVs), we explore the challenges in applying high-gain observers to highly dynamic systems. The multirotor will operate in the presence of external disturbances and modeling errors. At the same time, the reference trajectory is unknown and generated from a reference system with unknown or partially known dynamics. We assume the only measurements that are available are the position and orientation of the multirotor and the position of the reference system. We adopt an extended high-gain observer (EHGO) estimation framework to estimate the unmeasured multirotor states, modeling errors, external disturbances, and the reference trajectory. We design a robust output feedback controller for trajectory tracking that comprises a feedback linearizing controller and the EHGO. The proposed control method is rigorously analyzed to establish its stability properties. Finally, we illustrate our theoretical results through numerical simulation and experimental validation in which a multirotor tracks a moving ground vehicle with an unknown trajectory and dynamics and successfully lands on the vehicle while in motion. 

Abstract Localization of mobile robots is essential for navigation and data collection. This work presents an optical localization scheme for mobile robots during the robot’s continuous movement, despite that only one bearing angle can be captured at a time. In particular, this paper significantly improves upon our previous works where the robot has to pause its movement in order to acquire the two bearing angle measurements needed for position determination. The latter restriction forces the robot to work in a stop-and-go mode, which constrains the robot’s mobilitty. The proposed scheme exploits the velocity prediction from Kalman filtering, to properly correlate two consecutive measurements of bearing angles with respect to the base nodes (beacons) to produce location measurement. The proposed solution is evaluated in simulation and its advantage is demonstrated through the comparison with the traditional approach where the two consecutive angle measurements are directly used to compute the location. 

Abstract The problem of localizing a moving target arises in various forms in wireless sensor networks. Deploying multiple sensing receivers and using the time-difference-of-arrival (TDOA) of the target’s emitted signal is widely considered an effective localization technique. Traditionally, TDOA-based algorithms adopt a centralized approach where all measurements are sent to a predefined reference node for position estimation. More recently, distributed TDOA-based localization algorithms have been shown to improve the robustness of these estimates. For target models governed by highly stochastic processes, the method of nonlinear filtering and state estimation must be carefully considered. In this work, a distributed TDOA-based particle filter algorithm is proposed for localizing a moving target modeled by a discrete-time correlated random walk (DCRW). We present a method for using data collected by the particle filter to estimate the unknown probability distributions of the target’s movement model, and then apply the distribution estimates to recursively update the particle filter’s propagation model. The performance of the distributed approach is evaluated through numerical simulation, and we show the benefit of using a particle filter with online model learning by comparing it with the non-adaptive approach. 

Cognitive rehabilitation, STEM (science, technology, engineering, and math) skill acquisition, and coaching games such as chess often require tutoring decision-making strategies. The advancement of AI-driven tutoring systems for facilitating human learning requires an understanding of the impact of evaluative feedback on human decision-making and skill development. To this end, we conduct human experiments using Amazon Mechanical Turk to study the influence of evaluative feedback on human decision-making in sequential tasks. In these experiments, participants solve the Tower of Hanoi puzzle and receive AI-generated feedback while solving it. We examine how this feedback affects their learning and skill transfer to related tasks. Additionally, treating humans as noisy optimal agents, we employ maximum entropy inverse reinforcement learning to analyze the effect of feedback on the implicit human reward structure that guides their decision making. Lastly, we explore various computational models to understand how people incorporate evaluative feedback into their decision-making processes. Our findings underscore that humans perceive evaluative feedback as indicative of their long-term strategic success, thus aiding in skill acquisition and transfer in sequential decision-making tasks. Moreover, we demonstrate that evaluative feedback fosters a more structured and organized learning experience compared to learning without feedback. Furthermore, our results indicate that providing intermediate goals alone does not significantly enhance human learning outcomes. 

We propose Deterministic Sequencing of Exploration and Exploitation (DSEE) algorithm with interleaving exploration and exploitation epochs for model-based RL problems that aim to simultaneously learn the system model, i.e., a Markov decision process (MDP), and the associated optimal policy. During exploration, DSEE explores the environment and updates the estimates for expected reward and transition probabilities. During exploitation, the latest estimates of the expected reward and transition probabilities are used to obtain a robust policy with high probability. We design the lengths of the exploration and exploitation epochs such that the cumulative regret grows as a sub-linear function of time. 

We consider a prototypical path planning problem on a graph with uncertain cost of mobility on its edges. At a given node, the planning agent can access the true cost for edges to its neighbors and uses a noisy simulator to estimate the cost-to-go from the neighboring nodes. The objective of the planning agent is to select a neighboring node such that, with high probability, the cost-to-go is minimized for the worst possible realization of uncertain parameters in the simulator. By modeling the cost-to-go as a Gaussian process (GP) for every realization of the uncertain parameters, we apply a scenario approach in which we draw fixed independent samples of the uncertain parameter. We present a scenario-based iterative algorithm using the upper confidence bound (UCB) of the fixed independent scenarios to compute the choice of the neighbor to go to. We characterize the performance of the proposed algorithm in terms of a novel notion of regret defined with respect to an additional draw of the uncertain parameter, termed as scenario regret under re-draw. In particular, we characterize a high probability upper bound on the regret under re-draw for any finite number of iterations of the algorithm, and show that this upper bound tends to zero asymptotically with the number of iterations. We supplement our analysis with numerical results.