More Like this

1. Beyond Point Prediction: Capturing Zero-Inflated & Heavy-Tailed Spatiotemporal Data with Deep Extreme Mixture Models

   https://doi.org/10.1145/3534678.3539464  

   Wilson, Tyler; McDonald, Andrew; Galib, Asadullah Hill; Tan, Pang-Ning; Luo, Lifeng (August 2022, KDD '22: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining)

   Zhang, Aidong; Rangwala, Huzefa (Ed.)

   Zero-inflated, heavy-tailed spatiotemporal data is common across science and engineering, from climate science to meteorology and seismology. A central modeling objective in such settings is to forecast the intensity, frequency, and timing of extreme and non-extreme events; yet in the context of deep learning, this objective presents several key challenges. First, a deep learning framework applied to such data must unify a mixture of distributions characterizing the zero events, moderate events, and extreme events. Second, the framework must be capable of enforcing parameter constraints across each component of the mixture distribution. Finally, the framework must be flexible enough to accommodate for any changes in the threshold used to define an extreme event after training. To address these challenges, we propose Deep Extreme Mixture Model (DEMM), fusing a deep learning-based hurdle model with extreme value theory to enable point and distribution prediction of zero-inflated, heavy-tailed spatiotemporal variables. The framework enables users to dynamically set a threshold for defining extreme events at inference-time without the need for retraining. We present an extensive experimental analysis applying DEMM to precipitation forecasting, and observe significant improvements in point and distribution prediction. All code is available at https://github.com/andrewmcdonald27/DeepExtremeMixtureModel. 

   more » « less
2. Context-aware Deep Representation Learning for Geo-spatiotemporal Analysis

   Mao, Hanzi; Liu, Xi; Duffield, Nick; Yuan, Hao; Ji, Shuiwang Ji; Mohanty, Binayak (January 2020, International Conference on Data Mining)

   null (Ed.)

   Abstract—The emergence of remote sensing technologies cou- pled with local monitoring workstations enables us the un- precedented ability to monitor the environment in large scale. Information mining from multi-channel geo-spatiotemporal data however poses great challenges to many computational sustainability applications. Most existing approaches adopt various dimensionality reduction techniques without fully taking advantage of the spatiotemporal nature of the data. In addition, the lack of labeled training data raises another challenge for modeling such data. In this work, we propose a novel semi-supervised attention-based deep representation model that learns context-aware spatiotemporal representations for prediction tasks. A combination of convolutional neural networks with a hybrid attention mechanism is adopted to extract spatial and temporal variations in the geo-spatiotemporal data. Recognizing the importance of capturing more complete temporal dependencies, we propose the hybrid attention mechanism which integrates a learnable global query into the classic self-attention mechanism. To overcome the data scarcity issue, sampled spatial and temporal context that naturally reside in the largely-available unlabeled geo-spatiotemporal data are exploited to aid meaningful representation learning. We conduct experiments on a large-scale real-world crop yield prediction task. The results show that our methods significantly outperforms existing state-of-the-art yield prediction methods, especially under the stress of training data scarcity. 

   more » « less
3. WindSR-4D: Super-Resolution Spatiotemporal Wind Forecasting at Wind Farm Scale

   https://doi.org/10.1115/FEDSM2025-156477  

   Chen, Fuhao; Gao, Linyue (July 2025, American Society of Mechanical Engineers)

   Abstract Existing weather prediction products suffer from low spatiotemporal resolution, which fails to meet the requirements for fine-grained power forecasting and intelligent control of wind farms. This limitation adversely impacts wind turbine load management and the operational reliability of power systems. To tackle this, the study explores four-dimensional wind field superresolution, aiming to reconstruct high-resolution wind fields that fulfill both temporal and spatial resolution requirements through deep learning methods. The study emphasizes model architecture innovations to effectively capture the spatiotemporal coupling characteristics of wind field variations within wind farm regions. A novel model, termed 4dST-DMS, is proposed, comprising a spatial module and a temporal module, both built upon a 3D extension of the downsampled skip-connection/multi-scale framework as the backbone. Additionally, an iterative interpolation prediction method is proposed in the temporal module to extract multi-scale temporal information and capture dynamic variations, ensuring high fidelity in the reconstructed wind field. The proposed method is validated using the WTK-LED-5min dataset under experimental settings, including a 10-fold spatial resolution enhancement (both longitude and latitude), a 4-fold height resolution enhancement, and a 12-fold temporal resolution enhancement. The results demonstrate that the 4dST-DMS outperforms conventional interpolation methods and existing deep learning approaches in terms of numerical accuracy, vector field continuity and smoothness, structural similarity to real wind fields, and physical fidelity. 

   more » « less
4. Unsupervised Gaze Prediction in Egocentric Videos by Energy-based Surprise Modeling

   Aakur, Sathyanarayanan N.; Bagavathi, Arunkumar (April 2021, International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications)

   null (Ed.)

   Egocentric perception has grown rapidly with the advent of immersive computing devices. Human gaze prediction is an important problem in analyzing egocentric videos and has primarily been tackled through either saliency-based modeling or highly supervised learning. We quantitatively analyze the generalization capabilities of supervised, deep learning models on the egocentric gaze prediction task on unseen, out-of-domain data. We find that their performance is highly dependent on the training data and is restricted to the domains specified in the training annotations. In this work, we tackle the problem of jointly predicting human gaze points and temporal segmentation of egocentric videos without using any training data. We introduce an unsupervised computational model that draws inspiration from cognitive psychology models of event perception. We use Grenander's pattern theory formalism to represent spatial-temporal features and model surprise as a mechanism to predict gaze fixation points. Extensive evaluation on two publicly available datasets - GTEA and GTEA+ datasets-shows that the proposed model can significantly outperform all unsupervised baselines and some supervised gaze prediction baselines. Finally, we show that the model can also temporally segment egocentric videos with a performance comparable to more complex, fully supervised deep learning baselines. 

   more » « less
5. Survey of Federated Learning Models for Spatial-Temporal Mobility Applications

   https://doi.org/10.1145/3666089  

   Belal, Yacine; Ben_Mokhtar, Sonia; Haddadi, Hamed; Wang, Jaron; Mashhadi, Afra (September 2024, ACM Transactions on Spatial Algorithms and Systems)

   Federated learning involves training statistical models over edge devices such as mobile phones such that the training data are kept local. Federated Learning (FL) can serve as an ideal candidate for training spatial temporal models that rely on heterogeneous and potentially massive numbers of participants while preserving the privacy of highly sensitive location data. However, there are unique challenges involved with transitioning existing spatial temporal models to federated learning. In this survey article, we review the existing literature that has proposed FL-based models for predicting human mobility, traffic prediction, community detection, location-based recommendation systems, and other spatial-temporal tasks. We describe the metrics and datasets these works have been using and create a baseline of these approaches in comparison to the centralized settings. Finally, we discuss the challenges of applying spatial-temporal models in a decentralized setting and by highlighting the gaps in the literature we provide a road map and opportunities for the research community. 

   more » « less