More Like this

1. Fast and Accurate Trajectory Tracking for Unmanned Aerial Vehicles based on Deep Reinforcement Learning

   https://doi.org/10.1109/RTCSA.2019.8864571  

   Li, Yilan; Li, Hongjia; Li, Zhe; Fang, Haowen; Sanyal, Amit K.; Wang, Yanzhi; Qiu, Qinru (August 2019, IEEE International Conference on Embedded and Real-Time Computing Systems and Applications (RTCSA))

   Continuous trajectory control of fixed-wing unmanned aerial vehicles (UAVs) is complicated when considering hidden dynamics. Due to UAV multi degrees of freedom, tracking methodologies based on conventional control theory, such as Proportional-Integral-Derivative (PID) has limitations in response time and adjustment robustness, while a model based approach that calculates the force and torques based on UAV’s current status is complicated and rigid.We present an actor-critic reinforcement learning framework that controls UAV trajectory through a set of desired waypoints. A deep neural network is constructed to learn the optimal tracking policy and reinforcement learning is developed to optimize the resulting tracking scheme. The experimental results show that our proposed approach can achieve 58.14% less position error, 21.77% less system power consumption and 9:23% faster attainment than the baseline. The actor network consists of only linear operations, hence Field Programmable Gate Arrays (FPGA) based hardware acceleration can easily be designed for energy efficient real-time control. 

   more » « less
2. Trajectory Generation on SE(3) for an Underactuated Vehicle with Pointing Direction Constraints

   https://doi.org/10.23919/ACC.2019.8815238  

   Dhullipalla, Mani H.; Hamrah, Reza; Warier, Rakesh R.; Sanyal, Amit K. (July 2019, 2019 American Control Conference (ACC))

   This paper addresses the problem of generating a position trajectory with pointing direction constraints at given waypoints for underactuated unmanned vehicles. The problem is initially posed on the configuration space ℝ 3 × ℝ 2 and thereafter, upon suitable modifications, is re-posed as a problem on the Lie group SE(3). This is done by determining a vector orthogonal to the pointing direction and using it as the vehicle's thrust direction. This translates to converting reduced attitude constraints to full attitude constraints at the waypoints. For the position trajectory, in addition to position constraints, this modification adds acceleration constraints at the waypoints. For real-time implementation with low computational expenses, a linear-quadratic regulator (LQR) approach is adopted to determine the position trajectory with smoothness upto the fourth time derivative of position (snap). For the attitude trajectory, the thrust direction extracted from the position trajectory is used to first propagate the attitude to the subsequent waypoint and then correct it over time to achieve the desired attitude at this waypoint. Finally, numerical simulation results are presented to validate the trajectory generation scheme. 

   more » « less
3. Discrete Time Optimal Trajectory Generation and Transversality Condition with Free Final Time

   https://doi.org/10.1109/ICUAS51884.2021.9476719  

   Eslamiat, Hossein; Sanyal, Amit K.; Lindsay, Clark (June 2021, 2021 International Conference on Unmanned Aircraft Systems)

   A discrete time, optimal trajectory planning scheme for position trajectory generation of a vehicle is given here, considering the mission duration as a free variable. The vehicle is actuated in three rotational degrees of freedom and one translational degree of freedom. This model is applicable to vehicles that have a body-fixed thrust vector direction for translational motion control, including fixed-wing and rotorcraft unmanned aerial vehicles (UAVs), unmanned underwater vehicles (UUVs) and spacecraft. The lightweight scheme proposed here generates the trajectory in inertial coordinates, and is intended for real time, on-the-go applications. The unspecified terminal time can be considered as an additional design parameter. This is done by deriving the optimality conditions in a discrete time setting, which results in the discrete transversality condition. The trajectory starts from an initial position and reaches a desired final position in an unspecified final time that ensures the cost on state and control is optimized. The trajectory generated by this scheme can be considered as the desired trajectory for a tracking control scheme. Numerical simulation results validate the performance of this trajectory generation scheme used in conjunction with a nonlinear tracking control scheme. 

   more » « less
4. LEVIOSA: Natural Language-Based Uncrewed Aerial Vehicle Trajectory Generation

   https://doi.org/10.3390/electronics13224508  

   Aikins, Godwyll; Dao, Mawaba Pascal; Moukpe, Koboyo Josias; Eskridge, Thomas C; Nguyen, Kim-Doang (November 2024, Electronics)

   This paper presents LEVIOSA, a novel framework for text- and speech-based uncrewed aerial vehicle (UAV) trajectory generation. By leveraging multimodal large language models (LLMs) to interpret natural language commands, the system converts text and audio inputs into executable flight paths for UAV swarms. The approach aims to simplify the complex task of multi-UAV trajectory generation, which has significant applications in fields such as search and rescue, agriculture, infrastructure inspection, and entertainment. The framework involves two key innovations: a multi-critic consensus mechanism to evaluate trajectory quality and a hierarchical prompt structuring for improved task execution. The innovations ensure fidelity to user goals. The framework integrates several multimodal LLMs for high-level planning, converting natural language inputs into 3D waypoints that guide UAV movements and per-UAV low-level controllers to control each UAV in executing its assigned 3D waypoint path based on the high-level plan. The methodology was tested on various trajectory types with promising accuracy, synchronization, and collision avoidance results. The findings pave the way for more intuitive human–robot interactions and advanced multi-UAV coordination. 

   more » « less
5. Waypoint-Based Reinforcement Learning for Robot Manipulation Tasks

   https://doi.org/10.1109/IROS58592.2024.10802681  

   Mehta, Shaunak A; Habibian, Soheil; Losey, Dylan P (October 2024, Proceedings of the International Conference on Intelligent Robots and Systems)

   Robot arms should be able to learn new tasks. One framework here is reinforcement learning, where the robot is given a reward function that encodes the task, and the robot autonomously learns actions to maximize its reward. Existing approaches to reinforcement learning often frame this problem as a Markov decision process, and learn a policy (or a hierarchy of policies) to complete the task. These policies reason over hundreds of fine-grained actions that the robot arm needs to take: e.g., moving slightly to the right or rotating the end-effector a few degrees. But the manipulation tasks that we want robots to perform can often be broken down into a small number of high-level motions: e.g., reaching an object or turning a handle. In this paper we therefore propose a waypoint-based approach for model-free reinforcement learning. Instead of learning a low-level policy, the robot now learns a trajectory of waypoints, and then interpolates between those waypoints using existing controllers. Our key novelty is framing this waypoint-based setting as a sequence of multi-armed bandits: each bandit problem corresponds to one waypoint along the robot’s motion. We theoretically show that an ideal solution to this reformulation has lower regret bounds than standard frameworks. We also introduce an approximate posterior sampling solution that builds the robot’s motion one waypoint at a time. Results across benchmark simulations and two real-world experiments suggest that this proposed approach learns new tasks more quickly than state-of-the-art baselines. See our website here: https://collab.me.vt.edu/rl-waypoints/ 

   more » « less