To find the best method of predicting when daily relativistic electron flux (>2 MeV) will rise at geosynchronous orbit, we compare model predictive success rates (true positive rate or TPR) for multiple regression, ARMAX, logistic regression, a feed‐forward multilayer perceptron (MLP), and a recurrent neural network (RNN) model. We use only those days on which flux could rise, removing days when flux is already high from the data set. We explore three input variable sets: (1) ground‐based data (
Short‐term forecasting of wind gusts, particularly those of higher intensity, is of great societal importance but is challenging due to the presence of multiple gust generation mechanisms. Wind gust observations from eight high‐passenger‐volume airports across the continental United States (CONUS) are summarized and used to develop predictive models of wind gust occurrence and magnitude. These short‐term (same hour) forecast models are built using multiple logistic and linear regression, as well as artificial neural networks (ANNs) of varying complexity. A suite of 19 upper‐air predictors drawn from the ERA5 reanalysis and an autoregressive (AR) term are used. Stepwise procedures instruct predictor selection, and resampling is used to quantify model stability. All models are developed separately for the warm (April–September) and cold (October–March) seasons. Results show that ANNs of 3–5 hidden layers (HLs) generally exhibit higher hit rates than logistic regression models and also improve skill with respect to wind gust magnitudes. However, deeper networks with more HLs increase false alarm rates in occurrence models and mean absolute error in magnitude models due to model overfitting. For model skill, inclusion of the AR term is critical while the majority of the remaining skill derives from wind speeds and lapse rates. A predictive ceiling is also clearly demonstrated, particularly for the strong and damaging gust magnitudes, which appears to be partially due to ERA5 predictor characteristics and the presence of mixed wind climates.
more » « less- NSF-PAR ID:
- 10443068
- Publisher / Repository:
- DOI PREFIX: 10.1029
- Date Published:
- Journal Name:
- Earth and Space Science
- Volume:
- 9
- Issue:
- 12
- ISSN:
- 2333-5084
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
more » « less
Abstract Kp ,Dst , and sunspot number), (2) a full set of easily available solar wind and interplanetary magnetic field parameters (|B |,Bz ,V ,N ,P ,Ey ,Kp ,Dst , and sunspot number, and (3) this full set with the addition of previous day's flux. Despite high validation correlations in the multiple regression and ARMAX predictions, these regression models had low predictive ability (TPR < 45%) and are not recommended for use. The three classifier model types (logistic regression, MLP, and RNN) performed better (TPR: 50.8–74.6%). These rates were increased further if the cost of missing an event was set at 4 times that of predicting an event that did not happen (TPR: 73.1–89.6%). The area under the receiver operating characteristic curves did not, for the most part, differ between the classifier models (logistic, MLP, and RNN), indicating that any of the three could be used to discriminate between events and nonevents, but validation suggests a full RNN model performs best. The addition of previous day's flux as a predictor provided only a slight advantage. -
more » « less
Despite the widespread application of statistical downscaling tools, uncertainty remains regarding the role of model formulation in determining model skill for daily maximum and minimum temperature (
T maxandT min), and precipitation occurrence and intensity. Impacts of several key aspects of statistical transfer function form on model skill are evaluated using a framework resistant to model overspecification. We focus on: (a) model structure: simple (generalized linear models, GLMs) versus complex (artificial neural networks, ANNs) models. (b) Predictor selection: Fixed number of predictors chosena priori versus stepwise selection of predictors and inclusion of grid point values versus predictors derived from application of principal components analysis (PCA) to spatial fields. We also examine the influence of domain size on model performance. For precipitation downscaling, we consider the role of the threshold used to characterize a wet day and apply three approaches (Poisson and Gamma distributions in GLM and ANN) to downscale wet‐day precipitation amounts. While no downscaling formulation is optimal for all predictands and at 10 locations representing diverse U.S. climates, and due to the exclusion of variance inflation all of the downscaling formulations fail to reproduce the range of observed variability, models with larger suites of prospective predictors generally have higher skill. For temperature downscaling, ANNs generally outperform GLM, with greater improvements forT minthanT max. Use of PCA‐derived predictors does not systematically improve model skill, but does improve skill for temperature extremes. Model skill for precipitation occurrence generally increases as the wet‐day threshold increases and models using PCA‐derived predictors tend to outperform those based on grid cell predictors. Each model for wet‐day precipitation intensity overestimates annual total precipitation and underestimates the proportion derived from extreme precipitation events, but ANN‐based models and those with larger predictor suites tend to have the smallest bias. -
Machine learning (ML) methods, such as artificial neural networks (ANN), k-nearest neighbors (kNN), random forests (RF), support vector machines (SVM), and boosted decision trees (DTs), may offer stronger predictive performance than more traditional, parametric methods, such as linear regression, multiple linear regression, and logistic regression (LR), for specific mapping and modeling tasks. However, this increased performance is often accompanied by increased model complexity and decreased interpretability, resulting in critiques of their “black box” nature, which highlights the need for algorithms that can offer both strong predictive performance and interpretability. This is especially true when the global model and predictions for specific data points need to be explainable in order for the model to be of use. Explainable boosting machines (EBM), an augmentation and refinement of generalize additive models (GAMs), has been proposed as an empirical modeling method that offers both interpretable results and strong predictive performance. The trained model can be graphically summarized as a set of functions relating each predictor variable to the dependent variable along with heat maps representing interactions between selected pairs of predictor variables. In this study, we assess EBMs for predicting the likelihood or probability of slope failure occurrence based on digital terrain characteristics in four separate Major Land Resource Areas (MLRAs) in the state of West Virginia, USA and compare the results to those obtained with LR, kNN, RF, and SVM. EBM provided predictive accuracies comparable to RF and SVM and better than LR and kNN. The generated functions and visualizations for each predictor variable and included interactions between pairs of predictor variables, estimation of variable importance based on average mean absolute scores, and provided scores for each predictor variable for new predictions add interpretability, but additional work is needed to quantify how these outputs may be impacted by variable correlation, inclusion of interaction terms, and large feature spaces. Further exploration of EBM is merited for geohazard mapping and modeling in particular and spatial predictive mapping and modeling in general, especially when the value or use of the resulting predictions would be greatly enhanced by improved interpretability globally and availability of prediction explanations at each cell or aggregating unit within the mapped or modeled extent.more » « less
-
Solar flare prediction is a central problem in space weather forecasting and has captivated the attention of a wide spectrum of researchers due to recent advances in both remote sensing as well as machine learning and deep learning approaches. The experimental findings based on both machine and deep learning models reveal significant performance improvements for task specific datasets. Along with building models, the practice of deploying such models to production environments under operational settings is a more complex and often time-consuming process which is often not addressed directly in research settings. We present a set of new heuristic approaches to train and deploy an operational solar flare prediction system for ≥M1.0-class flares with two prediction modes: full-disk and active region-based. In full-disk mode, predictions are performed on full-disk line-of-sight magnetograms using deep learning models whereas in active region-based models, predictions are issued for each active region individually using multivariate time series data instances. The outputs from individual active region forecasts and full-disk predictors are combined to a final full-disk prediction result with a meta-model. We utilized an equal weighted average ensemble of two base learners’ flare probabilities as our baseline meta learner and improved the capabilities of our two base learners by training a logistic regression model. The major findings of this study are: 1) We successfully coupled two heterogeneous flare prediction models trained with different datasets and model architecture to predict a full-disk flare probability for next 24 h, 2) Our proposed ensembling model, i.e., logistic regression, improves on the predictive performance of two base learners and the baseline meta learner measured in terms of two widely used metrics True Skill Statistic (TSS) and Heidke Skill Score (HSS), and 3) Our result analysis suggests that the logistic regression-based ensemble (Meta-FP) improves on the full-disk model (base learner) by ∼9% in terms TSS and ∼10% in terms of HSS. Similarly, it improves on the AR-based model (base learner) by ∼17% and ∼20% in terms of TSS and HSS respectively. Finally, when compared to the baseline meta model, it improves on TSS by ∼10% and HSS by ∼15%.more » « less
-
During periods of rapidly changing geomagnetic conditions electric fields form within the Earth’s surface and induce currents known as geomagnetically induced currents (GICs), which interact with unprotected electrical systems our society relies on. In this study, we train multi-variate Long-Short Term Memory neural networks to predict magnitude of north-south component of the geomagnetic field (| B N |) at multiple ground magnetometer stations across Alaska provided by the SuperMAG database with a future goal of predicting geomagnetic field disturbances. Each neural network is driven by solar wind and interplanetary magnetic field inputs from the NASA OMNI database spanning from 2000–2015 and is fine tuned for each station to maximize the effectiveness in predicting | B N |. The neural networks are then compared against multivariate linear regression models driven with the same inputs at each station using Heidke skill scores with thresholds at the 50, 75, 85, and 99 percentiles for | B N |. The neural network models show significant increases over the linear regression models for | B N | thresholds. We also calculate the Heidke skill scores for d| B N |/dt by deriving d| B N |/dt from | B N | predictions. However, neural network models do not show clear outperformance compared to the linear regression models. To retain the sign information and thus predict B N instead of | B N |, a secondary so-called polarity model is utilized. The polarity model is run in tandem with the neural networks predicting geomagnetic field in a coupled model approach and results in a high correlation between predicted and observed values for all stations. We find this model a promising starting point for a machine learned geomagnetic field model to be expanded upon through increased output time history and fast turnaround times.more » « less
