skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: Estimating spatio-temporal fields through reinforcement learning
Prediction and estimation of phenomena of interest in aquatic environments are challenging since they present complex spatio-temporal dynamics. Over the past few decades, advances in machine learning and data processing contributed to ocean exploration and sampling using autonomous robots. In this work, we formulate a reinforcement learning framework to estimate spatio-temporal fields modeled by partial differential equations. The proposed framework addresses problems of the classic methods regarding the sampling process to determine the path to be used by the agent to collect samples. Simulation results demonstrate the applicability of our approach and show that the error at the end of the learning process is close to the expected error given by the fitting process due to added noise.  more » « less
Award ID(s):
2024733 2034123
PAR ID:
10358100
Author(s) / Creator(s):
; ; ;
Date Published:
Journal Name:
Frontiers in Robotics and AI
Volume:
9
ISSN:
2296-9144
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Spatio-temporal data indexed by sampling locations and sampling time points are encountered in many scientific disciplines such as climatology, environ- mental sciences, and public health. Here, we propose a novel spatio-temporal expanding distance (STED) asymptotic framework for studying the proper- ties of statistical inference for nonstationary spatio-temporal models. In particular, to model spatio-temporal dependence, we develop a new class of locally stationary spatio-temporal covariance functions. The STED asymp- totic framework has a fixed spatio-temporal domain for spatio-temporal pro- cesses that are globally nonstationary in a rescaled fixed domain and locally stationary in a distance expanding domain. The utility of STED is illus- trated by establishing the asymptotic properties of the maximum likelihood estimation for a general class of spatio-temporal covariance functions. A simulation study suggests sound finite-sample properties and the method is applied to a sea-surface temperature dataset. 
    more » « less
  2. Recurrent neural networks (RNNs) are nonlinear dynamical models commonly used in the machine learning and dynamical systems literature to represent complex dynamical or sequential relationships between variables. Recently, as deep learning models have become more common, RNNs have been used to forecast increasingly complicated systems. Dynamical spatio-temporal processes represent a class of complex systems that can potentially benefit from these types of models. Although the RNN literature is expansive and highly developed, uncertainty quantification is often ignored. Even when considered, the uncertainty is generally quantified without the use of a rigorous framework, such as a fully Bayesian setting. Here we attempt to quantify uncertainty in a more formal framework while maintaining the forecast accuracy that makes these models appealing, by presenting a Bayesian RNN model for nonlinear spatio-temporal forecasting. Additionally, we make simple modifications to the basic RNN to help accommodate the unique nature of nonlinear spatio-temporal data. The proposed model is applied to a Lorenz simulation and two real-world nonlinear spatio-temporal forecasting applications. 
    more » « less
  3. The use of video-imaging data for in-line process monitoring applications has become popular in industry. In this framework, spatio-temporal statistical process monitoring methods are needed to capture the relevant information content and signal possible out-of-control states. Video-imaging data are characterized by a spatio-temporal variability structure that depends on the underlying phenomenon, and typical out-of-control patterns are related to events that are localized both in time and space. In this article, we propose an integrated spatio-temporal decomposition and regression approach for anomaly detection in video-imaging data. Out-of-control events are typically sparse, spatially clustered and temporally consistent. The goal is not only to detect the anomaly as quickly as possible (“when”) but also to locate it in space (“where”). The proposed approach works by decomposing the original spatio-temporal data into random natural events, sparse spatially clustered and temporally consistent anomalous events, and random noise. Recursive estimation procedures for spatio-temporal regression are presented to enable the real-time implementation of the proposed methodology. Finally, a likelihood ratio test procedure is proposed to detect when and where the anomaly happens. The proposed approach was applied to the analysis of high-sped video-imaging data to detect and locate local hot-spots during a metal additive manufacturing process. 
    more » « less
  4. Satellite sensors have been widely used for precipitation retrieval, and a number of precipitation retrieval algorithms have been developed using observations from various satellite sensors. The current operational rainfall rate quantitative precipitation estimate (RRQPE) product from the geostationary operational environmental satellite (GOES) offers full disk rainfall rate estimates based on the observations from the advanced baseline imager (ABI) aboard the GOES-R series. However, accurate precipitation retrieval using satellite sensors is still challenging due to the limitations on spatio-temporal sampling of the satellite sensors and/or the uncertainty associated with the applied parametric retrieval algorithms. In this article, we propose a deep learning framework for precipitation retrieval using the combined observations from the ABI and geostationary lightning mapper (GLM) on the GOES-R series to improve the current operational RRQPE product. Particularly, the proposed deep learning framework is composed of two deep convolutional neural networks (CNNs) that are designed for precipitation detection and quantification. The cloud-top brightness temperature from multiple ABI channels and the lightning flash rate from the GLM measurement are used as inputs to the deep learning framework. To train the designed CNNs, the precipitation product multiradar multi-sensor (MRMS) system from the National Oceanic and Atmospheric Administration (NOAA) is used as target labels to optimize the network parameters. The experimental results show that the precipitation retrieval performance of the proposed framework is superior to the currently operational GOES RRQPE product in the selected study domain, and the performance is dramatically enhanced after incorporating the lightning data into the deep learning model. Using the independent MRMS product as a reference, the deep learning model can reduce the retrieval uncertainty in the operational RRQPE product by at least 31% in terms of the mean squared error and normalized mean absolute error, and the improvement is more significant in moderate to heavy rain regions. Therefore, the proposed deep learning framework can potentially serve as an alternative approach for GOES precipitation retrievals. 
    more » « less
  5. Successfully tackling many urgent challenges in socio-economically critical domains, such as public health and sustainability, requires a deeper understanding of causal relationships and interactions among a diverse spectrum of spatio-temporally distributed entities. In these applications, the ability to leverage spatio-temporal data to obtain causally based situational awareness and to develop informed forecasts to provide resilience at different scales is critical. While the promise of a causally grounded approach to these challenges is apparent, the core data technologies needed to achieve these are in the early stages and lack a framework to help realize their potential. In this article, we argue that there is an urgent need for a novel paradigm of spatio-causal research built on computational advances in spatio-temporal data and model integration, causal learning and discovery, large scale data- and model-driven simulations, emulations, and forecasting, as well as spatio-temporal data-driven and model-centric operational recommendations, and effective causally driven visualization and explanation. We thus provide a vision, and a road map, for spatio-causal situation awareness, forecasting, and planning. 
    more » « less