More Like this

1. Robust Multiagent Reinforcement Learning for UAV Systems: Countering Byzantine Attacks

   https://doi.org/10.3390/info14110623  

   Medhi, Jishu K; Liu, Rui; Wang, Qianlong; Chen, Xuhui (November 2023, Information)

   Multiple unmanned aerial vehicle (multi-UAV) systems have gained significant attention in applications, such as aerial surveillance and search and rescue missions. With the recent development of state-of-the-art multiagent reinforcement learning (MARL) algorithms, it is possible to train multi-UAV systems in collaborative and competitive environments. However, the inherent vulnerabilities of multiagent systems pose significant privacy and security risks when deploying general and conventional MARL algorithms. The presence of even a single Byzantine adversary within the system can severely degrade the learning performance of UAV agents. This work proposes a Byzantine-resilient MARL algorithm that leverages a combination of geometric median consensus and a robust state update model to mitigate, or even eliminate, the influence of Byzantine attacks. To validate its effectiveness and feasibility, the authors include a multi-UAV threat model, provide a guarantee of robustness, and investigate key attack parameters for multiple UAV navigation scenarios. Results from the experiments show that the average rewards during a Byzantine attack increased by up to 60% for the cooperative navigation scenario compared with conventional MARL techniques. The learning rewards generated by the baseline algorithms could not converge during training under these attacks, while the proposed method effectively converged to an optimal solution, proving its viability and correctness. 

   more » « less
2. Bird's eye view: Cooperative exploration by UGV and UAV

   https://doi.org/10.1109/ICUAS.2017.7991513  

   Hood, Shannon; Benson, Kelly; Hamod, Patrick; Madison, Daniel; O'Kane, Jason M.; Rekleitis, Ioannis (June 2017, International Conference on Unmanned Aircraft Systems (ICUAS))

   This paper proposes a solution to the problem of cooperative exploration using an Unmanned Ground Vehicle (UGV) and an Unmanned Aerial Vehicle (UAV). More specifically, the UGV navigates through the free space, and the UAV provides enhanced situational awareness via its higher vantage point. The motivating application is search and rescue in a damaged building. A camera atop the UGV is used to track a fiducial tag on the underside of the UAV, allowing the UAV to maintain a fixed pose relative to the UGV. Furthermore, the UAV uses its front facing camera to provide a birds-eye-view to the remote operator, allowing for observation beyond obstacles that obscure the UGV's sensors. The proposed approach has been tested using a TurtleBot 2 equipped with a Hokuyo laser ranger finder and a Parrot Bebop 2. Experimental results demonstrate the feasibility of this approach. This work is based on several open source packages and the generated code is available on-line. 

   more » « less
3. LEVIOSA: Natural Language-Based Uncrewed Aerial Vehicle Trajectory Generation

   https://doi.org/10.3390/electronics13224508  

   Aikins, Godwyll; Dao, Mawaba Pascal; Moukpe, Koboyo Josias; Eskridge, Thomas C; Nguyen, Kim-Doang (November 2024, Electronics)

   This paper presents LEVIOSA, a novel framework for text- and speech-based uncrewed aerial vehicle (UAV) trajectory generation. By leveraging multimodal large language models (LLMs) to interpret natural language commands, the system converts text and audio inputs into executable flight paths for UAV swarms. The approach aims to simplify the complex task of multi-UAV trajectory generation, which has significant applications in fields such as search and rescue, agriculture, infrastructure inspection, and entertainment. The framework involves two key innovations: a multi-critic consensus mechanism to evaluate trajectory quality and a hierarchical prompt structuring for improved task execution. The innovations ensure fidelity to user goals. The framework integrates several multimodal LLMs for high-level planning, converting natural language inputs into 3D waypoints that guide UAV movements and per-UAV low-level controllers to control each UAV in executing its assigned 3D waypoint path based on the high-level plan. The methodology was tested on various trajectory types with promising accuracy, synchronization, and collision avoidance results. The findings pave the way for more intuitive human–robot interactions and advanced multi-UAV coordination. 

   more » « less
4. Applicability of Relatively Low-Cost Multispectral Uncrewed Aerial Systems for Surface Characterization of the Cryosphere

   https://doi.org/10.3390/rs16193662  

   Rand, Colby F; Khan, Alia L (October 2024, Remote Sensing)

   This paper investigates the ability of a relatively low cost, commercially available uncrewed aerial vehicle (UAV), the DJI Mavic 3 Multispectral, to perform cryospheric research. The performance of this UAV, where applicable, is compared to a similar but higher cost system, the DJI Matrice 350, equipped with a Micasense RedEdge-MX Multispectral dual-camera system. The Mavic 3 Multispectral was tested at three field sites: the Lemon Creek Glacier, Juneau Icefield, AK; the Easton Glacier, Mt. Baker, WA; and Bagley Basin, Mt. Baker, WA. This UAV proved capable of mapping the spatial distribution of red snow algae on the surface of the Lemon Creek Glacier using both spectral indices and a random forest supervised classification method. The UAV was able to assess the timing of snowmelt and changes in suncup morphology on snow-covered areas within the Bagley Basin. Finally, the UAV was able to classify glacier surface features using a random forest algorithm with an overall accuracy of 68%. The major advantages of this UAV are its low weight, which allows it to be easily transported into the field, its low cost compared to other alternatives, and its ease of use. One limitation would be the omission of a blue multispectral band, which would have allowed it to more easily classify glacial ice and snow features. 

   more » « less
5. Development of a Peripheral-Central Vision System for Small UAS Tracking

   https://doi.org/10.2514/6.2019-2074  

   Kang, Changkoo; Chaudhry, Haseeb; Woolsey, Craig A.; Kochersberger, Kevin B. (January 2019, AIAA SciTech)

   With the rapid proliferation of small unmanned aircraft systems (UAS), the risk of mid-air collisions is growing, as is the risk associated with the malicious use of these systems. Airborne Detect-and-Avoid (ABDAA) and counter-UAS technologies have similar sensing requirements to detect and track airborne threats, albeit for different purposes: to avoid a collision or to neutralize a threat, respectively. These systems typically include a variety of sensors, such as electro-optical or infrared (EO/IR) cameras, RADAR, or LiDAR, and they fuse the data from these sensors to detect and track a given threat and to predict its trajectory. Camera imagery can be an effective method for detection as well as for pose estimation and threat classification, though a single camera cannot resolve range to a threat without additional information, such as knowledge of the threat geometry. To support ABDAA and counter-UAS applications, we consider a merger of two image-based sensing methods that mimic human vision: (1) a "peripheral vision" camera (i.e., with a fisheye lens) to provide a large field-of-view and (2) a "central vision" camera (i.e., with a perspective lens) to provide high resolution imagery of a specific target. Beyond the complementary ability of the two cameras to support detection and classification, the pair form a heterogeneous stereo vision system that can support range resolution. This paper describes the initial development and testing of a peripheral-central vision system to detect, localize, and classify an airborne threat and finally to predict its path using knowledge of the threat class. 

   more » « less