skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: Deep Learning–based Reassembling of an Aerial & Legged Marsupial Robotic System–of–Systems
In this work we address the System-of-Systems reassembling operation of a marsupial team comprising a hybrid Unmanned Aerial Vehicle and a Legged Locomotion robot, relying solely on vision-based systems and assisted by Deep Learning. The target application domain is that of large-scale field surveying operations under the presence of wireless communication disruptions. While most real-world field deployments of multi-robot systems assume some degree of wireless communication to coordinate key tasks such as multi-agent rendezvous, a desirable feature against unrecoverable communication failures or radio degradation due to jamming cyber-attacks is the ability for autonomous systems to robustly execute their mission with onboard perception. This is especially true for marsupial air / ground teams, wherein landing onboard the ground robot is required. We propose a pipeline that relies on Deep Neural Network-based Vehicle-to-Vehicle detection based on aerial views acquired by flying at typical altitudes for Micro Aerial Vehicle-based real-world surveying operations, such as near the border of the 400ft Above Ground Level window. We present the minimal computing and sensing suite that supports its execution onboard a fully autonomous micro-Tiltrotor aircraft which detects, approaches, and lands onboard a Boston Dynamics Spot legged robot. We present extensive experimental studies that validate this marsupial aerial / ground robot’s capacity to safely reassemble while in the airborne scouting phase without the need for wireless communication.  more » « less
Award ID(s):
2150394
PAR ID:
10491759
Author(s) / Creator(s):
; ; ; ; ; ;
Publisher / Repository:
IEEE
Date Published:
Journal Name:
International Conference on Unmanned Aircraft Systems
ISBN:
979-8-3503-1037-5
Page Range / eLocation ID:
626 to 633
Format(s):
Medium: X
Location:
Warsaw, Poland
Sponsoring Org:
National Science Foundation
More Like this
  1. In this work we address the flexible physical docking-and-release as well as recharging needs for a marsupial system comprising an autonomous tiltrotor hybrid Micro Aerial Vehicle and a high-end legged locomotion robot. Within persistent monitoring and emergency response situations, such aerial / ground robot teams can offer rapid situational awareness by taking off from the mobile ground robot and scouting a wide area from the sky. For this type of operational profile to retain its long-term effectiveness, regrouping via landing and docking of the aerial robot onboard the ground one is a key requirement. Moreover, onboard recharging is a necessity in order to perform systematic missions. We present a framework comprising: a novel landing mechanism with recharging capabilities embedded into its design, an external battery-based recharging extension for our previously developed power-harvesting Micro Aerial Vehicle module, as well as a strategy for the reliable landing and the docking-and-release between the two robots. We specifically address the need for this system to be ferried by a quadruped ground system while remaining reliable during aggressive legged locomotion when traversing harsh terrain. We present conclusive experimental validation studies by deploying our solution on a marsupial system comprising the MiniHawk micro tiltrotor and the Boston Dynamics Spot legged robot. 
    more » « less
  2. null (Ed.)
    This paper addresses the problem of autonomously deploying an unmanned aerial vehicle in non-trivial settings, by leveraging a manipulator arm mounted on a ground robot, acting as a versatile mobile launch platform. As real-world deployment scenarios for micro aerial vehicles such as searchand- rescue operations often entail exploration and navigation of challenging environments including uneven terrain, cluttered spaces, or even constrained openings and passageways, an often arising problem is that of ensuring a safe take-off location, or safely fitting through narrow openings while in flight. By facilitating launching from the manipulator end-effector, a 6- DoF controllable take-off pose within the arm workspace can be achieved, which allows to properly position and orient the aerial vehicle to initialize the autonomous flight portion of a mission. To accomplish this, we propose a sampling-based planner that respects a) the kinematic constraints of the ground robot / manipulator / aerial robot combination, b) the geometry of the environment as autonomously mapped by the ground robots perception systems, and c) accounts for the aerial robot expected dynamic motion during takeoff. The goal of the proposed planner is to ensure autonomous collision-free initialization of an aerial robotic exploration mission, even within a cluttered constrained environment. At the same time, the ground robot with the mounted manipulator can be used to appropriately position the take-off workspace into areas of interest, effectively acting as a carrier launch platform. We experimentally demonstrate this novel robotic capability through a sequence of experiments that encompass a micro aerial vehicle platform carried and launched from a 6-DoF manipulator arm mounted on a four-wheel robot base. 
    more » « less
  3. This works deals with the problem of long-term autonomy in the context of multi-day field deployments of Micro Aerial Vehicle (MAV) systems. To truly depart from the necessity for human intervention for the crucial task of providing battery recharging, and to liberate from the need to operate in a confined range around specially installed infrastructure such as recharging pods, the MAV robot is required to harvest power on its own, but equally importantly also sustain prolonged periods of ambient power scarcity. This implies being able to sustain the battery charge overnight when using solar recharging, or even during multiple days of illumination inadequacy (e.g., due to degraded atmospheric lucidity and heavy overcast). We address this by presenting a Self-Sustainable Autonomous System architecture for MAVs centered around a specially tailored Power Management Stack, which is capable of achieving deep system hibernation, a feature that facilitates the aforementioned functionalities. We present a) continuous, b) multi-day successive, and c) externally-powered recharging that uses a legged robot-mounted Mobile Recharging Station. We conclude by demonstrating a challenging zero-intervention multi-day field deployment mission in the N.Nevada region. 
    more » « less
  4. In large scale coverage operations, such as marine exploration or aerial monitoring, single robot approaches are not ideal, as they may take too long to cover a large area. In such scenarios, multi-robot approaches are preferable. Furthermore, several real world vehicles are non-holonomic, but can be modeled using Dubins vehicle kinematics. This paper focuses on environmental monitoring of aquatic environments using Autonomous Surface Vehicles (ASVs). In particular, we propose a novel approach for solving the problem of complete coverage of a known environment by a multi-robot team consisting of Dubins vehicles. It is worth noting that both multi-robot coverage and Dubins vehicle coverage are NP-complete problems. As such, we present two heuristics methods based on a variant of the traveling salesman problem-k-TSP-formulation and clustering algorithms that efficiently solve the problem. The proposed methods are tested both in simulations to assess their scalability and with a team of ASVs operating on a 200 km 2 lake to ensure their applicability in real world. 
    more » « less
  5. With the rapid development of technology and the proliferation of uncrewed aerial systems (UAS), there is an immediate need for security solutions. Toward this end, we propose the use of a multi-robot system for autonomous and cooperative counter-UAS missions. In this paper, we present the design of the hardware and software components of different complementary robotic platforms: a mobile uncrewed ground vehicle (UGV) equipped with a LiDAR sensor, an uncrewed aerial vehicle (UAV) with a gimbal-mounted stereo camera for air-to-air inspections, and a UAV with a capture mechanism equipped with radars and camera. Our proposed system features 1) scalability to larger areas due to the distributed approach and online processing, 2) long-term cooperative missions, and 3) complementary multimodal perception for the detection of multirotor UAVs. In field experiments, we demonstrate the integration of all subsystems in accomplishing a counter-UAS task within an unstructured environment. The obtained results confirm the promising direction of using multi-robot and multi-modal systems for C-UAS. 
    more » « less