skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


  • NSF Public Access
  • Search Results
  • Automated 3D Reconstruction Using Optimized View-Planning Algorithms for Iterative Development of Structure-from-Motion Models
Title: Automated 3D Reconstruction Using Optimized View-Planning Algorithms for Iterative Development of Structure-from-Motion Models
Unsupervised machine learning algorithms (clustering, genetic, and principal component analysis) automate Unmanned Aerial Vehicle (UAV) missions as well as the creation and refinement of iterative 3D photogrammetric models with a next best view (NBV) approach. The novel approach uses Structure-from-Motion (SfM) to achieve convergence to a specified orthomosaic resolution by identifying edges in the point cloud and planning cameras that “view” the holes identified by edges without requiring an initial model. This iterative UAV photogrammetric method successfully runs in various Microsoft AirSim environments. Simulated ground sampling distance (GSD) of models reaches as low as 3.4 cm per pixel, and generally, successive iterations improve resolution. Besides analogous application in simulated environments, a field study of a retired municipal water tank illustrates the practical application and advantages of automated UAV iterative inspection of infrastructure using 63 % fewer photographs than a comparable manual flight with analogous density point clouds obtaining a GSD of less than 3 cm per pixel. Each iteration qualitatively increases resolution according to a logarithmic regression, reduces holes in models, and adds details to model edges.  more » « less
Award ID(s):
1650547
PAR ID:
10316796
Author(s) / Creator(s):
; ; ; ; ; ;
Date Published:
Journal Name:
Remote Sensing
Volume:
12
Issue:
13
ISSN:
2072-4292
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; ; ; (, Drones)
    Unmanned aerial vehicles (UAV) enable detailed historical preservation of large-scale infrastructure and contribute to cultural heritage preservation, improved maintenance, public relations, and development planning. Aerial and terrestrial photo data coupled with high accuracy GPS create hyper-realistic mesh and texture models, high resolution point clouds, orthophotos, and digital elevation models (DEMs) that preserve a snapshot of history. A case study is presented of the development of a hyper-realistic 3D model that spans the complex 1.7 km2 area of the Brigham Young University campus in Provo, Utah, USA and includes over 75 significant structures. The model leverages photos obtained during the historic COVID-19 pandemic during a mandatory and rare campus closure and details a large scale modeling workflow and best practice data acquisition and processing techniques. The model utilizes 80,384 images and high accuracy GPS surveying points to create a 1.65 trillion-pixel textured structure-from-motion (SfM) model with an average ground sampling distance (GSD) near structures of 0.5 cm and maximum of 4 cm. Separate model segments (31) taken from data gathered between April and August 2020 are combined into one cohesive final model with an average absolute error of 3.3 cm and a full model absolute error of <1 cm (relative accuracies from 0.25 cm to 1.03 cm). Optimized and automated UAV techniques complement the data acquisition of the large-scale model, and opportunities are explored to archive as-is building and campus information to enable historical building preservation, facility maintenance, campus planning, public outreach, 3D-printed miniatures, and the possibility of education through virtual reality (VR) and augmented reality (AR) tours. 
    more » « less
  2. ; ; ; ; ; ; ; ; ; ; et al (, Remote Sensing in Ecology and Conservation)
    Abstract Non‐forest ecosystems, dominated by shrubs, grasses and herbaceous plants, provide ecosystem services including carbon sequestration and forage for grazing, and are highly sensitive to climatic changes. Yet these ecosystems are poorly represented in remotely sensed biomass products and are undersampled by in situ monitoring. Current global change threats emphasize the need for new tools to capture biomass change in non‐forest ecosystems at appropriate scales. Here we developed and deployed a new protocol for photogrammetric height using unoccupied aerial vehicle (UAV) images to test its capability for delivering standardized measurements of biomass across a globally distributed field experiment. We assessed whether canopy height inferred from UAV photogrammetry allows the prediction of aboveground biomass (AGB) across low‐stature plant species by conducting 38 photogrammetric surveys over 741 harvested plots to sample 50 species. We found mean canopy height was strongly predictive of AGB across species, with a median adjustedR2of 0.87 (ranging from 0.46 to 0.99) and median prediction error from leave‐one‐out cross‐validation of 3.9%. Biomass per‐unit‐of‐height was similarwithinbut differentamong,plant functional types. We found that photogrammetric reconstructions of canopy height were sensitive to wind speed but not sun elevation during surveys. We demonstrated that our photogrammetric approach produced generalizable measurements across growth forms and environmental settings and yielded accuracies as good as those obtained from in situ approaches. We demonstrate that using a standardized approach for UAV photogrammetry can deliver accurate AGB estimates across a wide range of dynamic and heterogeneous ecosystems. Many academic and land management institutions have the technical capacity to deploy these approaches over extents of 1–10 ha−1. Photogrammetric approaches could provide much‐needed information required to calibrate and validate the vegetation models and satellite‐derived biomass products that are essential to understand vulnerable and understudied non‐forested ecosystems around the globe. 
    more » « less
  3. ; (, NSF Arctic Data Center)
    To assess coastal erosion dynamics during the entire 2018 and 2019 open water seasons at Drew Point, Beaufort Sea Coast, Alaska, we derived 16 coastlines position using satellite, airborne and unmanned aerial vehicle (UAV) sensors. Sensors with associated image dates are: Worldview 1 imagery ©Maxar (14 April 2019), Worldview 2 panchromatic imagery ©Maxar (5 April 2019, 26 September 2019, and 3 April 2020); Modular Aerial Camera System (MACS-Polar) during the Polar-6 airborne operations during the ThawTrend-Air campaign (13 July 2019, 23 July, 2019, and 30 July 2019) and DJI Phantom 4 UAV surveys (24 July 2018, 29 July 2018, 3 August 2018, 30 September 2018, 2 August 2019, 6 August 2019, 10 August 2019, 12 August 2019 and 15 August 2019). Pixel resolution for the satellite, airborne and UAV imagery was 50 cm (Worldview 1), 46 centimeter (cm) (Worldview 2), 10 cm and 4 cm, respectively. The satellite-image derived coastlines span the 9 kilometer (Km) segment described in Jones et al. (2018; DOI: 10.1088/1748-9326/aae471), while the other coastline spans a 1.5 Km sub-section of the study area that includes the coastline, part of inland coastal area (~125 meters (m)) and fallen toppled permafrost blocks in front of the bluff. Fallen toppled permafrost blocks were digitized using the airborne and UAV images. The satellite imagery was too coarse to digitize blocks. All datasets are in WGS84 UTM Zone 5N. 
    more » « less
  4. ; ; ; (, Remote Sensing)
    The integration of structure from motion (SFM) and unmanned aerial vehicle (UAV) technologies has allowed for the generation of very high-resolution three-dimensional (3D) point cloud data (up to millimeters) to detect and monitor surface changes. However, a bottleneck still exists in accurately and rapidly registering the point clouds at different times. The existing point cloud registration algorithms, such as the Iterative Closest Point (ICP) and the Fast Global Registration (FGR) method, were mainly developed for the registration of small and static point cloud data, and do not perform well when dealing with large point cloud data with potential changes over time. In particular, registering large data is computationally expensive, and the inclusion of changing objects reduces the accuracy of the registration. In this paper, we develop an AI-based workflow to ensure high-quality registration of the point clouds generated using UAV-collected photos. We first detect stable objects from the ortho-photo produced by the same set of UAV-collected photos to segment the point clouds of these objects. Registration is then performed only on the partial data with these stable objects. The application of this workflow using the UAV data collected from three erosion plots at the East Tennessee Research and Education Center indicates that our workflow outperforms the existing algorithms in both computational speed and accuracy. This AI-based workflow significantly improves computational efficiency and avoids the impact of changing objects for the registration of large point cloud data. 
    more » « less
  5. ; ; ; ; ; (, Sensors)
    This study presents a novel multi-scale view-planning algorithm for automated targeted inspection using unmanned aircraft systems (UAS). In industrial inspection, it is important to collect the most relevant data to keep processing demands, both human and computational, to a minimum. This study investigates the viability of automated targeted multi-scale image acquisition for Structure from Motion (SfM)-based infrastructure modeling. A traditional view-planning approach for SfM is extended to a multi-scale approach, planning for targeted regions of high, medium, and low priority. The unmanned aerial vehicle (UAV) can traverse the entire aerial space and facilitates collection of an optimized set of views, both close to and far away from areas of interest. The test case for field validation is the Tibble Fork Dam in Utah. Using the targeted multi-scale flight planning, a UAV automatically flies a tiered inspection using less than 25% of the number of photos needed to model the entire dam at high-priority level. This results in approximately 75% reduced flight time and model processing load, while still maintaining high model accuracy where needed. Models display stepped improvement in visual clarity and SfM reconstruction integrity by priority level, with the higher priority regions more accurately modeling smaller and finer features. A resolution map of the final tiered model is included. While this study focuses on multi-scale view planning for optical sensors, the methods potentially extend to other remote sensors, such as aerial LiDAR. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email