An official website of the United States government
Forecasting the block maxima of a future time window is a challenging task due to the difficulty in inferring the tail distribution of a target variable. As the historical observations alone may not be sufficient to train robust models to predict the block maxima, domain-driven process models are often available in many scientific domains to supplement the observation data and improve the forecast accuracy. Unfortunately, coupling the historical observations with process model outputs is a challenge due to their disparate temporal coverage. This paper presents Self-Recover, a deep learning framework to predict the block maxima of a time window by employing self-supervised learning to address the varying temporal data coverage problem. Specifically Self-Recover uses a combination of contrastive and generative self-supervised learning schemes along with a denoising autoencoder to impute the missing values. The framework also combines representations of the historical observations with process model outputs via a residual learning approach and learns the generalized extreme value (GEV) distribution characterizing the block maxima values. This enables the framework to reliably estimate the block maxima of each time window along with its confidence interval. Extensive experiments on real-world datasets demonstrate the superiority of Self-Recover compared to other state-of-the-art forecasting methods.
more »
« less
DeepExtrema: A Deep Learning Approach for Forecasting Block Maxima in Time Series Data
Accurate forecasting of extreme values in time series is critical due to the significant impact of extreme events on human and natural systems. This paper presents DeepExtrema, a novel framework that combines a deep neural network (DNN) with generalized extreme value (GEV) distribution to forecast the block maximum value of a time series. Implementing such a network is a challenge as the framework must preserve the inter-dependent constraints among the GEV model parameters even when the DNN is initialized. We describe our approach to address this challenge and present an architecture that enables both conditional mean and quantile prediction of the block maxima. The extensive experiments performed on both real-world and synthetic data demonstrated the superiority of DeepExtrema compared to other baseline methods.
more »
« less
- Award ID(s):
- 2006633
- PAR ID:
- 10358675
- Date Published:
- Journal Name:
- Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence
- Page Range / eLocation ID:
- 2980 to 2986
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
It is appealing but challenging to achieve real-time deep neural network (DNN) inference on mobile devices because even the powerful modern mobile devices are considered “resource-constrained” when executing large-scale DNNs. It necessitates the sparse model inference via weight pruning, i.e., DNN weight sparsity, and it is desirable to design a new DNN weight sparsity scheme that can facilitate real-time inference on mobile devices while preserving a high sparse model accuracy. This paper designs a novel mobile inference acceleration framework GRIM that is General to both convolutional neural networks (CNNs) and recurrent neural networks (RNNs) and that achieves Real-time execution and high accuracy, leveraging fine-grained structured sparse model Inference and compiler optimizations for Mobiles. We start by proposing a new fine-grained structured sparsity scheme through the Block-based Column-Row (BCR) pruning. Based on this new fine-grained structured sparsity, our GRIM framework consists of two parts: (a) the compiler optimization and code generation for real-time mobile inference; and (b) the BCR pruning optimizations for determining pruning hyperparameters and performing weight pruning. We compare GRIM with Alibaba MNN, TVM, TensorFlow-Lite, a sparse implementation based on CSR, PatDNN, and ESE (a representative FPGA inference acceleration framework for RNNs), and achieve up to 14.08× speedup.more » « less
-
Extreme storm surges can overwhelm many coastal flooding protection measures in place and cause severe damages to private communities, public infrastructure, and natural ecosystems. In the US Mid-Atlantic, a highly developed and commercially active region, coastal flooding is one of the most significant natural hazards and a year-round threat from both tropical and extra-tropical cyclones. Mean sea levels and high-tide flood frequency has increased significantly in recent years, and major storms are projected to increase into the foreseeable future. We estimate extreme surges using hourly water level data and harmonic analysis for 1980–2019 at 12 NOAA tide gauges in and around the Delaware and Chesapeake Bays. Return levels (RLs) are computed for 1.1, 3, 5, 10, 25, 50, and 100-year return periods using stationary extreme value analysis on detrended skew surges. Two traditional approaches are investigated, Block Maxima fit to General Extreme Value distribution and Points-Over-Threshold fit to Generalized Pareto distribution, although with two important enhancements. First, the GEV r -largest order statistics distribution is used; a modified version of the GEV distribution that allows for multiple maximum values per year. Second, a systematic procedure is used to select the optimum value for r (for the BM/GEVr approach) and the threshold (for the POT/GP approach) at each tide gauge separately. RLs have similar magnitudes and spatial patterns from both methods, with BM/GEVr resulting in generally larger 100-year and smaller 1.1-year RLs. Maximum values are found at the Lewes (Delaware Bay) and Sewells Point (Chesapeake Bay) tide gauges, both located in the southwest region of their respective bays. Minimum values are found toward the central bay regions. In the Delaware Bay, the POT/GP approach is consistent and results in narrower uncertainty bands whereas the results are mixed for the Chesapeake. Results from this study aim to increase reliability of projections of extreme water levels due to extreme storms and ultimately help in long-term planning of mitigation and implementation of adaptation measures.more » « less
-
Deep neural networks (DNNs) have emerged as the most powerful machine learning technique in numerous artificial intelligent applications. However, the large sizes of DNNs make themselves both computation and memory intensive, thereby limiting the hardware performance of dedicated DNN accelerators. In this paper, we propose a holistic framework for energy-efficient high-performance highly-compressed DNN hardware design. First, we propose block-circulant matrix-based DNN training and inference schemes, which theoretically guarantee Big-O complexity reduction in both computational cost (from O(n2) to O(n log n)) and storage requirement (from O(n2) to O(n)) of DNNs. Second, we dedicatedly optimize the hardware architecture, especially on the key fast Fourier transform (FFT) module, to improve the overall performance in terms of energy efficiency, computation performance and resource cost. Third, we propose a design flow to perform hardware-software co-optimization with the purpose of achieving good balance between test accuracy and hardware performance of DNNs. Based on the proposed design flow, two block-circulant matrix-based DNNs on two different datasets are implemented and evaluated on FPGA. The fixed-point quantization and the proposed block-circulant matrix-based inference scheme enables the network to achieve as high as 3.5 TOPS computation performance and 3.69 TOPS/W energy efficiency while the memory is saved by 108X ~ 116X with negligible accuracy degradation.more » « less
-
PurposeThe purpose of this study is to develop a deep learning framework for additive manufacturing (AM), that can detect different defect types without being trained on specific defect data sets and can be applied for real-time process control. Design/methodology/approachThis study develops an explainable artificial intelligence (AI) framework, a zero-bias deep neural network (DNN) model for real-time defect detection during the AM process. In this method, the last dense layer of the DNN is replaced by two consecutive parts, a regular dense layer denoted (L1) for dimensional reduction, and a similarity matching layer (L2) for equal weight and non-biased cosine similarity matching. Grayscale images of 3D printed samples acquired during printing were used as the input to the zero-bias DNN. FindingsThis study demonstrates that the approach is capable of successfully detecting multiple types of defects such as cracks, stringing and warping with high accuracy without any prior training on defective data sets, with an accuracy of 99.5%. Practical implicationsOnce the model is set up, the computational time for anomaly detection is lower than the speed of image acquisition indicating the potential for real-time process control. It can also be used to minimize manual processing in AI-enabled AM. Originality/valueTo the best of the authors’ knowledge, this is the first study to use zero-bias DNN, an explainable AI approach for defect detection in AM.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.24963/ijcai.2022/413
