skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: Combining Remote and In-situ Sensing for Autonomous Underwater Vehicle Localization and Navigation
Scientists continue to study the red tide and fish-kill events happening in Florida. Machine learning applications using remote sensing data on coastal waters to monitor water quality parameters and detect harmful algal blooms are also being studied. Unmanned Surface Vehicles (USVs) and Autonomous Underwater Vehicles (AUVs) are often deployed on data collection and disaster response missions. To enhance study and mitigation efforts, robots must be able to use available data to navigate these underwater environments. In this study, we compute a satellite-derived underwater environment (SDUE) model by implementing a supervised machine learning model where remote sensing reflectance (Rrs) indices are labeled with in-situ data they correlate with. The models predict bathymetry and water quality parameters given a recent remote sensing image. In our experiment, we use Sentine1-2 (S2) images and in-situ data of the Biscayne Bay to create an SDUE that can be used as a Chlorophyll-a map. The SDUE is then used in an Extended Kalman Filter (EKF) application that solves an underwater vehicle localization and navigation problem.  more » « less
Award ID(s):
2111661 2024733
PAR ID:
10417177
Author(s) / Creator(s):
; ; ; ; ;
Publisher / Repository:
OCEANS 2022
Date Published:
Journal Name:
OCEANS 2022
Page Range / eLocation ID:
1 to 7
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Many studies suggest that water quality parameters can be estimated by applying statistical and machine learning methods using remote sensing or in-situ data. However, identifying best practices for implementing solutions appears to be done on a case-by-case basis. In our case, we have in-situ data that covers a large period, but only small areas of Biscayne Bay, Florida. In this paper, we combine available in-situ data with remote sensing data captured by Landsat 8 OLI-TIRS Collection 2 Level 2(L8), Sentinel-2 L2A(S2), and Sentinel-3 OLCI L1B(S3). The combined data set is for use in a water quality parameter estimation application. Our contributions are two-fold. First, we present a pipeline for data collection, processing, and co-location that results in a usable data set of combined remote sensing and in-situ data. Second, we propose a classification model using the combined data set to identify areas of interest for future data collection missions based on chlorophyll-a in-situ measurements. To further prove our methodology, we conduct a data collection mission using one of the predicted paths from our model. 
    more » « less
  2. Abstract: Aim In this study, we present the results of a project which used Landsat Collection 2 Surface Reflectance data and European Centre for Medium-Range Weather Forecasts (ECMWF) Reanalysis v5 (ERA5) data to develop a machine learning model to estimate Secchi depth in Lake Yojoa, Honduras. Methods Satellite remote sensing data obtained within a 7-day window of an in situ measurement were matched with in situ Secchi depth measurements and were partitioned into train-test-validate data sets for model development. Results The machine learning model had good (R2= 0.57) agreement and reasonable uncertainty (MAE = 0.58 m) between remotely estimated and in situ observed Secchi depth. Application of the machine learning model increased the monitoring record of Lake Yojoa from 6 years of measured data to a 23-year record. Conclusions This model demonstrates the utility of coordinating in situ sampling schedules of short-term research projects with satellite imagery acquisition schedules in order to increase the temporal coverage of remote sensing derived estimates of water quality in understudied lakes. 
    more » « less
  3. The emerging sector of offshore kelp aquaculture represents an opportunity to produce biofuel feedstock to help meet growing energy demand. Giant kelp represents an attractive aquaculture crop due to its rapid growth and production, however precision farming over large scales is required to make this crop economically viable. These demands necessitate high frequency monitoring to ensure outplant success, maximum production, and optimum quality of harvested biomass, while the long distance from shore and large necessary scales of production makes in person monitoring impractical. Remote sensing offers a practical monitoring solution and nascent imaging technologies could be leveraged to provide daily products of the kelp canopy and subsurface structures over unprecedented spatial scales. Here, we evaluate the efficacy of remote sensing from satellites and aerial and underwater autonomous vehicles as potential monitoring platforms for offshore kelp aquaculture farms. Decadal-scale analyses of the Southern California Bight showed that high offshore summertime cloud cover restricts the ability of satellite sensors to provide high frequency direct monitoring of these farms. By contrast, daily monitoring of offshore farms using sensors mounted to aerial and underwater drones seems promising. Small Unoccupied Aircraft Systems (sUAS) carrying lightweight optical sensors can provide estimates of canopy area, density, and tissue nitrogen content on the time and space scales necessary for observing changes in this highly dynamic species. Underwater color imagery can be rapidly classified using deep learning models to identify kelp outplants on a longline farm and high acoustic returns of kelp pneumatocysts from side scan sonar imagery signal an ability to monitor the subsurface development of kelp fronds. Current sensing technologies can be used to develop additional machine learning and spectral algorithms to monitor outplant health and canopy macromolecular content, however future developments in vehicle and infrastructure technologies are necessary to reduce costs and transcend operational limitations for continuous deployment in an offshore setting. 
    more » « less
  4. Real-time computer vision and remote visual sensing platforms are increasingly used in numerous underwater applications such as shipwreck mapping, subsea inspection, coastal water monitoring, surveillance, coral reef surveying, invasive fish tracking, and more. Recent advancements in robot vision and powerful single-board computers have paved the way for an imminent revolution in the next generation of subsea technologies. In this chapter, we present these exciting emerging applications and discuss relevant open problems and practical considerations. First, we delineate the specific environmental and operational challenges of underwater vision and highlight some prominent scientific and engineering solutions to ensure robust visual perception. We specifically focus on the characteristics of underwater light propagation from the perspective of image formation and photometry. We also discuss the recent developments and trends in underwater imaging literature to facilitate the restoration, enhancement, and filtering of inherently noisy visual data. Subsequently, we demonstrate how these ideas are extended and deployed in the perception pipelines of Autonomous Underwater Vehicles (AUVs) and Remotely Operated Vehicles (ROVs). In particular, we present several use cases for marine life monitoring and conservation, human-robot cooperative missions for inspecting submarine cables and archaeological sites, subsea structure or cave mapping, aquaculture, and marine ecology. We elaborately discuss how the deep visual learning and on-device AI breakthroughs are transforming the perception, planning, localization, and navigation capabilities of visually-guided underwater robots. Along this line, we also highlight the prospective future research directions and open problems at the intersection of computer vision and underwater robotics domains. 
    more » « less
  5. Inland waters pose a unique challenge for water quality monitoring by remote sensing techniques due to their complicated spectral features and small-scale variability. At the same time, collecting the reference data needed to calibrate remote sensing data products is both time consuming and expensive. In this study, we present the further development of a robotic team composed of an uncrewed surface vessel (USV) providing in situ reference measurements and an unmanned aerial vehicle (UAV) equipped with a hyperspectral imager. Together, this team is able to address the limitations of existing approaches by enabling the simultaneous collection of hyperspectral imagery with precisely collocated in situ data. We showcase the capabilities of this team using data collected in a northern Texas pond across three days in 2020. Machine learning models for 13 variables are trained using the dataset of paired in situ measurements and coincident reflectance spectra. These models successfully estimate physical variables including temperature, conductivity, pH, and turbidity as well as the concentrations of blue–green algae, colored dissolved organic matter (CDOM), chlorophyll-a, crude oil, optical brighteners, and the ions Ca2+, Cl−, and Na+. We extend the training procedure to utilize conformal prediction to estimate 90% confidence intervals for the output of each trained model. Maps generated by applying the models to the collected images reveal small-scale spatial variability within the pond. This study highlights the value of combining real-time, in situ measurements together with hyperspectral imaging for the rapid characterization of water composition. 
    more » « less