skip to main content


Title: Estimating spatio-temporal fields through reinforcement learning
Prediction and estimation of phenomena of interest in aquatic environments are challenging since they present complex spatio-temporal dynamics. Over the past few decades, advances in machine learning and data processing contributed to ocean exploration and sampling using autonomous robots. In this work, we formulate a reinforcement learning framework to estimate spatio-temporal fields modeled by partial differential equations. The proposed framework addresses problems of the classic methods regarding the sampling process to determine the path to be used by the agent to collect samples. Simulation results demonstrate the applicability of our approach and show that the error at the end of the learning process is close to the expected error given by the fitting process due to added noise.  more » « less
Award ID(s):
2024733 2034123
NSF-PAR ID:
10358100
Author(s) / Creator(s):
; ; ;
Date Published:
Journal Name:
Frontiers in Robotics and AI
Volume:
9
ISSN:
2296-9144
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract

    Citizen and community science datasets are typically collected using flexible protocols. These protocols enable large volumes of data to be collected globally every year; however, the consequence is that these protocols typically lack the structure necessary to maintain consistent sampling across years. This can result in complex and pronounced interannual changes in the observation process, which can complicate the estimation of population trends because population changes over time are confounded with changes in the observation process.

    Here we describe a novel modelling approach designed to estimate spatially explicit species population trends while controlling for the interannual confounding common in citizen science data. The approach is based on Double machine learning, a statistical framework that uses machine learning (ML) methods to estimate population change and the propensity scores used to adjust for confounding discovered in the data. ML makes it possible to use large sets of features to control for confounding and to model spatial heterogeneity in trends. Additionally, we present a simulation method to identify and adjust for residual confounding missed by the propensity scores.

    To illustrate the approach, we estimated species trends using data from the citizen science project eBird. We used a simulation study to assess the ability of the method to estimate spatially varying trends when faced with realistic confounding and temporal correlation. Results demonstrated the ability to distinguish between spatially constant and spatially varying trends. There were low error rates on the estimated direction of population change (increasing/decreasing) at each location and high correlations on the estimated magnitude of population change.

    The ability to estimate spatially explicit trends while accounting for confounding inherent in citizen science data has the potential to fill important information gaps, helping to estimate population trends for species and/or regions lacking rigorous monitoring data.

     
    more » « less
  2. Spatio-temporal data indexed by sampling locations and sampling time points are encountered in many scientific disciplines such as climatology, environ- mental sciences, and public health. Here, we propose a novel spatio-temporal expanding distance (STED) asymptotic framework for studying the proper- ties of statistical inference for nonstationary spatio-temporal models. In particular, to model spatio-temporal dependence, we develop a new class of locally stationary spatio-temporal covariance functions. The STED asymp- totic framework has a fixed spatio-temporal domain for spatio-temporal pro- cesses that are globally nonstationary in a rescaled fixed domain and locally stationary in a distance expanding domain. The utility of STED is illus- trated by establishing the asymptotic properties of the maximum likelihood estimation for a general class of spatio-temporal covariance functions. A simulation study suggests sound finite-sample properties and the method is applied to a sea-surface temperature dataset. 
    more » « less
  3. Abstract

    Distributed Acoustic Sensing (DAS) is an emerging technology for earthquake monitoring and subsurface imaging. However, its distinct characteristics, such as unknown ground coupling and high noise level, pose challenges to signal processing. Existing machine learning models optimized for conventional seismic data struggle with DAS data due to its ultra-dense spatial sampling and limited manual labels. We introduce a semi-supervised learning approach to address the phase-picking task of DAS data. We use the pre-trained PhaseNet model to generate noisy labels of P/S arrivals in DAS data and apply the Gaussian mixture model phase association (GaMMA) method to refine these noisy labels and build training datasets. We develop PhaseNet-DAS, a deep learning model designed to process 2D spatio-temporal DAS data to achieve accurate phase picking and efficient earthquake detection. Our study demonstrates a method to develop deep learning models for DAS data, unlocking the potential of integrating DAS in enhancing earthquake monitoring.

     
    more » « less
  4. The applications for wide area monitoring, protection, and control systems (WAMPC) at the control center, help with providing resilient, efficient, and secure operation of the transmission system of the smart grid. The increased proliferation of phasor measurement units (PMUs) in this space has inspired many prudent applications to assist in the process of decision making in the control centers. Machine learning (ML) based decision support systems have become viable with the availability of abundant high-resolution wide area operational PMU data. We propose a deep neural network (DNN) based supervisory protection and event diagnosis system and demonstrate that it works with very high degree of confidence. The system introduces a supervisory layer that processes the data streams collected from PMUs and detects disturbances in the power systems that may have gone unnoticed by the local monitoring and protection system. Then, we investigate compromise of the insights of this ML based supervisory control by crafting adversaries that corrupt the PMU data via minimal coordinated manipulation and identification of the spatio-temporal regions in the multidimensional PMU data in a way that the DNN classifier makes wrong event predictions. This dataset contains images that represent PMU data described in the reference paper. Each image has a dimension of [300X20X3] comprising of 300 time points, 10 voltage and 10 frequency measurements, and 3 fundamental color intensities. Each of the image represents the instance of a disturbance. We consider a disturbance pattern length of 5s, with 0.5 s before the trigger and 4.5 s after the trigger. Voltage and frequency data streams from 10 PMUs at a sampling rate of 60 frames per second, were aggregated to form these pseudo color images. The data-set consisted of three sub-folders: 1. 344 instances of faults located in the sub-folder “DB_FLT” 2. 140 instances of loss of generation located in the sub-folder “DB_GNL” 3. 21 instances of synchronous motor switching events located in the sub-folder “DB_SMS”. 
    more » « less
  5. Abstract

    Condensation is ubiquitous in nature and industry. Heterogeneous condensation on surfaces is typified by the continuous cycle of droplet nucleation, growth, and departure. Central to the mechanistic understanding of the thermofluidic processes governing condensation is the rapid and high‐fidelity extraction of interpretable physical descriptors from the highly transient droplet population. However, extracting quantifiable measures out of dynamic objects with conventional imaging technologies poses a challenge to researchers. Here, an intelligent vision‐based framework is demonstrated that unites classical thermofluidic imaging techniques with deep learning to fundamentally address this challenge. The deep learning framework can autonomously harness physical descriptors and quantify thermal performance at extreme spatio‐temporal resolutions of 300 nm and 200 ms, respectively. The data‐centric analysis conclusively shows that contrary to classical understanding, the overall condensation performance is governed by a key tradeoff between heat transfer rate per individual droplet and droplet population density. The vision‐based approach presents a powerful tool for the study of not only phase‐change processes but also any nucleation‐based process within and beyond the thermal science community through the harnessing of big data.

     
    more » « less