skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: PRGFlow: Unified SWAP‐aware deep global optical flow for aerial robot navigation
Abstract Global optical flow estimation is the foundation stone for obtaining odometry which is used to enable aerial robot navigation. However, such a method has to be of low latency and high robustness whilst also respecting the size, weight, area and power (SWAP) constraints of the robot. A combination of cameras coupled with inertial measurement units (IMUs) has proven to be the best combination in order to obtain such low latency odometry on resource‐constrained aerial robots. Recently, deep learning approaches for visual inertial fusion have gained momentum due to their high accuracy and robustness. However, an equally noteworthy benefit for robotics of these techniques are their inherent scalability (adaptation to different sized aerial robots) and unification (same method works on different sized aerial robots). To this end, we present a deep learning approach called PRGFlow for obtaining global optical flow and then loosely fuse it with an IMU for full 6‐DoF (Degrees of Freedom) relative pose estimation (which is then integrated to obtain odometry). The network is evaluated on the MSCOCO dataset and the dead‐reckoned odometry on multiple real‐flight trajectories without any fine‐tuning or re‐training. A detailed benchmark comparing different network architectures and loss functions to enable scalability is also presented. It is shown that the method outperforms classical feature matching methods by 2 under noisy data. The supplementary material and code can be found athttp://prg.cs.umd.edu/PRGFlow.  more » « less
Award ID(s):
2020624
PAR ID:
10371425
Author(s) / Creator(s):
 ;  ;  ;  
Publisher / Repository:
DOI PREFIX: 10.1049
Date Published:
Journal Name:
Electronics Letters
Volume:
57
Issue:
16
ISSN:
0013-5194
Format(s):
Medium: X Size: p. 614-617
Size(s):
p. 614-617
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; (, Proceedings IEEE International Conference on Robotics and Automation)
    IEEE (Ed.)
    This paper addresses the robustness problem of visual-inertial state estimation for underwater operations. Underwater robots operating in a challenging environment are required to know their pose at all times. All vision-based localization schemes are prone to failure due to poor visibility conditions, color loss, and lack of features. The proposed approach utilizes a model of the robot's kinematics together with proprioceptive sensors to maintain the pose estimate during visual-inertial odometry (VIO) failures. Furthermore, the trajectories from successful VIO and the ones from the model-driven odometry are integrated in a coherent set that maintains a consistent pose at all times. Health-monitoring tracks the VIO process ensuring timely switches between the two estimators. Finally, loop closure is implemented on the overall trajectory. The resulting framework is a robust estimator switching between model-based and visual-inertial odometry (SM/VIO). Experimental results from numerous deployments of the Aqua2 vehicle demonstrate the robustness of our approach over coral reefs and a shipwreck. 
    more » « less
  2. ; ; (, 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS))
    Recovering rigid registration between successive camera poses lies at the heart of 3D reconstruction, SLAM and visual odometry. Registration relies on the ability to compute discriminative 2D features in successive camera images for determining feature correspondences, which is very challenging in feature-poor environments, i.e. low-texture and/or low-light environments. In this paper, we aim to address the challenge of recovering rigid registration between successive camera poses in feature-poor environments in a Visual Inertial Odometry (VIO) setting. In addition to inertial sensing, we instrument a small aerial robot with an RGBD camera and propose a framework that unifies the incorporation of 3D geometric entities: points, lines, and planes. The tracked 3D geometric entities provide constraints in an Extended Kalman Filtering framework. We show that by directly exploiting 3D geometric entities, we can achieve improved registration. We demonstrate our approach on different texture-poor environments, with some containing only flat texture-less surfaces providing essentially no 2D features for tracking. In addition, we evaluate how the addition of different 3D geometric entities contributes to improved pose estimation by comparing an estimated pose trajectory to a ground truth pose trajectory obtained from a motion capture system. We consider computationally efficient methods for detecting 3D points, lines and planes, since our goal is to implement our approach on small mobile robots, such as drones. 
    more » « less
  3. ; (, Proceedings of International Conference on Unmanned Aircraft Systems (ICUAS))
    This paper investigates the resilience of perception-based multi-robot coordination with wireless communication to online adversarial perception. A systematic study of this problem is essential for many safety-critical robotic applications that rely on the measurements from learned perception modules. We consider a (small) team of quadrotor robots that rely only on an Inertial Measurement Unit (IMU) and the visual data measurements obtained from a learned multi-task perception module (e.g., object detection) for downstream tasks, including relative localization and coordination. We focus on a class of adversarial perception attacks that cause misclassification, mislocalization, and latency. We propose that the effects of adversarial misclassification and mislocalization can be modeled as sporadic (intermittent) and spurious measurement data for the downstream tasks. To address this, we present a framework for resilience analysis of multi-robot coordination with adversarial measurements. The framework integrates data from Visual-Inertial Odometry (VIO) and the learned perception model for robust relative localization and state estimation in the presence of adversarially sporadic and spurious measurements. The framework allows for quantifying the degradation in system observability and stability in relation to the success rate of adversarial perception. Finally, experimental results on a multi-robot platform demonstrate the real-world applicability of our methodology for resource-constrained robotic platforms. 
    more » « less
  4. ; ; ; ; (, OCEANS 2023 - MTS/IEEE U.S. Gulf Coast)
    Vision-based state estimation is challenging in underwater environments due to color attenuation, low visibility and floating particulates. All visual-inertial estimators are prone to failure due to degradation in image quality. However, underwater robots are required to keep track of their pose during field deployments. We propose robust estimator fusing the robot's dynamic and kinematic model with proprioceptive sensors to propagate the pose whenever visual-inertial odometry (VIO) fails. To detect the VIO failures, health tracking is used, which enables switching between pose estimates from VIO and a kinematic estimator. Loop closure implemented on weighted posegraph for global trajectory optimization. Experimental results from an Aqua2 Autonomous Underwater Vehicle field deployments demonstrates the robustness of our approach over different underwater environments such as over shipwrecks and coral reefs. The proposed hybrid approach is robust to VIO failures producing consistent trajectories even in harsh conditions. 
    more » « less
  5. ; ; ; ; (, Journal of Geophysical Research: Solid Earth)
    Abstract A foundational assumption in paleomagnetism is that the Earth's magnetic field behaves as a geocentric axial dipole (GAD) when averaged over sufficient timescales. Compilations of directional data averaged over the past 5 Ma yield a distribution largely compatible with GAD, but the distribution of paleointensity data over this timescale is incompatible. Reasons for the failure of GAD include: (a) Arbitrary “selection criteria” to eliminate “unreliable” data vary among studies, so the paleointensity database may include biased results. (b) The age distribution of existing paleointensity data varies with latitude, so different latitudinal averages represent different time periods. (c) The time‐averaged field could be truly non‐dipolar. Here, we present a consistent methodology for analyzing paleointensity results and comparing time‐averaged paleointensities from different studies. We apply it to data from Plio/Pleistocene Hawai'ian igneous rocks, sampled from fine‐grained, quickly cooled material (lava flow tops, dike margins and scoria cones) and subjected to the IZZI‐Thellier technique; the data were analyzed using the Bias Corrected Estimation of Paleointensity method of Cych et al. (2021,https://doi.org/10.1029/2021GC009755), which produces accurate paleointensity estimates without arbitrarily excluding specimens from the analysis. We constructed a paleointensity curve for Hawai'i over the Plio/Pleistocene using the method of Livermore et al. (2018,https://doi.org/10.1093/gji/ggy383), which accounts for the age distribution of data. We demonstrate that even with the large uncertainties associated with obtaining a mean field from temporally sparse data, our average paleointensities obtained from Hawai'i and Antarctica (reanalyzed from Asefaw et al., 2021,https://doi.org/10.1029/2020JB020834) are not GAD‐like from 0 to 1.5 Ma but may be prior to that. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email