skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: UAV-based Foliage Plant Species Classification for Semantic Characterization of Pre-Fire Landscapes
In this work we deal with the problem of establishing a system architecture to facilitate the real-time autonomous volumetric mapping alongside the semantic characterization of sagebrush ecosystem landscapes, in order to support the pre-fire modeling and analysis required to plan for wildfire prevention and/or suppression. The world, and more specifically the broader region of N. Nevada has been facing one of its most challenging periods over the course of the last decade, as far as uncontrolled wildfires are concerned. This has led to the development of research initiatives aimed at the ecosystem-specific modeling of the pre-, during-, and post-fire process effects in order to better understand, predict, and address these phenomena. However, to collect the required wide-field information that contains both centimeter-level volumetric mapping fidelity, as well as semantic details related to plant (sub)-species, which for the common case of sagebrush can only be identified based on close-up inspection of their foliage fine structure, satellite photography remains insufficient. To this end, we propose a perception and mapping architecture of an aerial robotic system that is capable of: a) LiDAR-based centimeter-level reconstruction, b) robust multi-modal sensor fusion Simultaneous Localization and Mapping (SLAM) lever-aging LiDAR, IMU, Visual-Inertial Odometry, and Differential GPS in a global optimization mapping framework, as well as c) a gimbal-driven point-zoom camera for the efficient real-time collection of close-up imagery of foliage pertaining to specific target plants, in order to allow their real-time identification based on their leaf micro-structure, by leveraging Deep-Learned classification deployed on a Neural Processing Unit. We present the associated systems, the overall hardware and software architecture, as well as a series of field deployment studies validating the proposed aerial robotic capabilities.  more » « less
Award ID(s):
2150394
PAR ID:
10573826
Author(s) / Creator(s):
; ; ; ; ;
Publisher / Repository:
IEEE
Date Published:
ISBN:
979-8-3503-5788-2
Page Range / eLocation ID:
792 to 799
Format(s):
Medium: X
Location:
Chania - Crete, Greece
Sponsoring Org:
National Science Foundation
More Like this
  1. null (Ed.)
    Due to Wildfire's huge destructive impacts on agriculture and food production, wildlife habitat, climate, human life and ecosystem, timely discovery of fires enable swift response to fires before they go out of control, in order to minimize the resulting damage and impacts. One of the emerging technologies for fire monitoring is deploying Unmanned Aerial Vehicles, due to their high flexibility and maneuverability, less human risk, and on-demand high quality imaging capabilities. In order to realize a real-time system for fire detection and expansion analysis, fast and high-accuracy image-processing algorithms are required. Several studies have shown that deep learning methods can provide the most accurate response, however the training time can be prohibitively long, especially when using online learning for constant refinement of the developed model. Another challenge is the lack of large datasets for training a deep learning algorithm. In this respect, we propose to use a pretrained mobileNetV2 architecture to implement transfer learning, which requires a smaller dataset and reduces the computational complexity while not compromising the accuracy. In addition, we conduct an effective data augmentation pipeline to simulate some extreme scenarios, which could promise the robustness of our approach. The testing results illustrate that our method maintains a high identification accuracy in different situations - original dataset (99.7%), adding Gaussian blurred (95.3%), and additive Gaussian noise (99.3%). 
    more » « less
  2. Current forest monitoring technologies including satellite remote sensing, manned/piloted aircraft, and observation towers leave uncertainties about a wildfire’s extent, behavior, and conditions in the fire’s near environment, particularly during its early growth. Rapid mapping and real-time fire monitoring can inform in-time intervention or management solutions to maximize beneficial fire outcomes. Drone systems’ unique features of 3D mobility, low flight altitude, and fast and easy deployment make them a valuable tool for early detection and assessment of wildland fires, especially in remote forests that are not easily accessible by ground vehicles. In addition, the lack of abundant, well-annotated aerial datasets – in part due to unmanned aerial vehicles’ (UAVs’) flight restrictions during prescribed burns and wildfires – has limited research advances in reliable data-driven fire detection and modeling techniques. While existing wildland fire datasets often include either color or thermal fire images, here we present (1) a multi-modal UAV-collected dataset of dual-feed side-by-side videos including both RGB and thermal images of a prescribed fire in an open canopy pine forest in Northern Arizona and (2) a deep learning-based methodology for detecting fire and smoke pixels at accuracy much higher than the usual single-channel video feeds. The collected images are labeled to “fire” or “no-fire” frames by two human experts using side-by-side RGB and thermal images to determine the label. To provide context to the main dataset’s aerial imagery, the included supplementary dataset provides a georeferenced pre-burn point cloud, an RGB orthomosaic, weather information, a burn plan, and other burn information. By using and expanding on this guide dataset, research can develop new data-driven fire detection, fire segmentation, and fire modeling techniques. 
    more » « less
  3. Abstract Surface rupture from the 2019 Ridgecrest earthquake sequence, initially associated with the Mw 6.4 foreshock, occurred on 4 July on a ∼17  km long, northeast–southwest-oriented, left-lateral zone of faulting. Following the Mw 7.1 mainshock on 5 July (local time), extensive northwest–southeast-oriented, right-lateral faulting was then also mapped along a ∼50  km long zone of faults, including subparallel splays in several areas. The largest slip was observed in the epicentral area and crossing the dry lakebed of China Lake to the southeast. Surface fault rupture mapping by a large team, reported elsewhere, was used to guide the airborne data acquisition reported here. Rapid rupture mapping allowed for accurate and efficient flight line planning for the high-resolution light detection and ranging (lidar) and aerial photography. Flight line planning trade-offs were considered to allocate the medium (25 pulses per square meter [ppsm]) and high-resolution (80 ppsm) lidar data collection polygons. The National Center for Airborne Laser Mapping acquired the airborne imagery with a Titan multispectral lidar system and Digital Modular Aerial Camera (DiMAC) aerial digital camera, and U.S. Geological Survey acquired Global Positioning System ground control data. This effort required extensive coordination with the Navy as much of the airborne data acquisition occurred within their restricted airspace at the China Lake ranges. 
    more » « less
  4. This paper proposes a bandwidth tunable technique for real-time probabilistic scene modeling and mapping to enable co-robotic exploration in communication constrained environments such as the deep sea. The parameters of the system enable the user to characterize the scene complexity represented by the map, which in turn determines the bandwidth requirements. The approach is demonstrated using an underwater robot that learns an unsupervised scene model of the environment and then uses this scene model to communicate the spatial distribution of various high-level semantic scene constructs to a human operator. Preliminary experiments in an artificially constructed tank environment, as well as simulated missions over a 10m x 10m coral reef using real data, show the tunability of the maps to different bandwidth constraints and science interests. To our knowledge this is the first paper to quantify how the free parameters of the unsupervised scene model impact both the scientific utility of and bandwidth required to communicate the resulting scene model. 
    more » « less
  5. Abstract Interannual variation, especially weather, is an often‐cited reason for restoration “failures”; yet its importance is difficult to experimentally isolate across broad spatiotemporal extents, due to correlations between weather and site characteristics. We examined post‐fire treatments within sagebrush‐steppe ecosystems to ask: (1) Is weather following seeding efforts a primary reason why restoration outcomes depart from predictions? and (2) Does the management‐relevance of weather differ across space and with time since treatment? Our analysis quantified range‐wide patterns of sagebrush (Artemisiaspp.) recovery, by integrating long‐term records of restoration and annual vegetation cover estimates from satellite imagery following thousands of post‐fire seeding treatments from 1984 to 2005. Across the Great Basin, sagebrush growth increased in wetter, cooler springs; however, the importance of spring weather varied with sites' long‐term climates, suggesting differing ecophysiological limitations across sagebrush's range. Incorporation of spring weather, including from the “planting year,” improved predictions of sagebrush recovery, but these advances were small compared to contributions of time‐invariant site characteristics. Given extreme weather conditions threatening this ecosystem, explicit consideration of weather could improve the allocation of management resources, such as by identifying areas requiring repeated treatments; but improved forecasts of shifting mean conditions with climate change may more significantly aid the prediction of sagebrush recovery. 
    more » « less