An official website of the United States government
Abstract We develop an Imbalanced Regression Artificial Neural Network model for the Auroral electrojet index (IRANNA) to predict the SuperMAG SML index, addressing the heavily imbalanced distribution of the SML data set. The data set contains mostly quiet‐time values of lesser importance and very few strong‐to‐extreme values of interest, such as those associated with super substorms. Traditional prediction models, which minimize mean squared error uniformly across the whole data set, are often skewed by this imbalance, prioritizing the lower, quiet‐time values and consequently underestimating strong geomagnetic events. The IRANNA model addresses this issue by using a customized weighting scheme in the loss function, enabling it to predict strong‐to‐extreme events accurately for the first time. The model takes solar wind parameters as inputs and predicts the logarithm of the absolute SML values. It does not rely on past values of the SML index, differentiating it from other models that use historical data for prediction. The model has demonstrated its ability to predict the peak amplitudes of strong‐to‐extreme events across various statistical analyses, event studies, and virtual experiments. Despite this success, challenges remain, particularly during localized electrojet events and when upstream solar wind data propagation is unreliable. This study emphasizes the importance of using imbalanced regression techniques, especially in space physics, where data sets are inherently skewed. It also highlights the potential of the IRANNA model to provide valuable insights into the magnetosphere's response to solar wind driving, improving space weather forecasting and offering new tools for investigating magnetospheric dynamics.
more »
« less
Model Evaluation Guidelines for Geomagnetic Index Predictions
Abstract Geomagnetic indices are convenient quantities that distill the complicated physics of some region or aspect of near‐Earth space into a single parameter. Most of the best‐known indices are calculated from ground‐based magnetometer data sets, such as Dst, SYM‐H, Kp, AE, AL, and PC. Many models have been created that predict the values of these indices, often using solar wind measurements upstream from Earth as the input variables to the calculation. This document reviews the current state of models that predict geomagnetic indices and the methods used to assess their ability to reproduce the target index time series. These existing methods are synthesized into a baseline collection of metrics for benchmarking a new or updated geomagnetic index prediction model. These methods fall into two categories: (1) fit performance metrics such as root‐mean‐square error and mean absolute error that are applied to a time series comparison of model output and observations and (2) event detection performance metrics such as Heidke Skill Score and probability of detection that are derived from a contingency table that compares model and observation values exceeding (or not) a threshold value. A few examples of codes being used with this set of metrics are presented, and other aspects of metrics assessment best practices, limitations, and uncertainties are discussed, including several caveats to consider when using geomagnetic indices.
more »
« less
- Award ID(s):
- 1663770
- PAR ID:
- 10375682
- Author(s) / Creator(s):
- ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; more »
- Publisher / Repository:
- DOI PREFIX: 10.1029
- Date Published:
- Journal Name:
- Space Weather
- Volume:
- 16
- Issue:
- 12
- ISSN:
- 1542-7390
- Page Range / eLocation ID:
- p. 2079-2102
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract Space weather indices are used commonly to drive forecasts of thermosphere density, which affects objects in low‐Earth orbit (LEO) through atmospheric drag. One commonly used space weather proxy,F10.7cm, correlates well with solar extreme ultra‐violet (EUV) energy deposition into the thermosphere. Currently, the USAF contracts Space Environment Technologies (SET), which uses a linear algorithm to forecastF10.7cm. In this work, we introduce methods using neural network ensembles with multi‐layer perceptrons (MLPs) and long‐short term memory (LSTMs) to improve on the SET predictions. We make predictions only from historicalF10.7cmvalues. We investigate data manipulation methods (backwards averaging and lookback) as well as multi step and dynamic forecasting. This work shows an improvement over the popular persistence and the operational SET model when using ensemble methods. The best models found in this work are ensemble approaches using multi step or a combination of multi step and dynamic predictions. Nearly all approaches offer an improvement, with the best models improving between 48% and 59% on relative MSE with respect to persistence. Other relative error metrics were shown to improve greatly when ensembles methods were used. We were also able to leverage the ensemble approach to provide a distribution of predicted values; allowing an investigation into forecast uncertainty. Our work found models that produced less biased predictions at elevated and high solar activity levels. Uncertainty was also investigated through the use of a calibration error score metric (CES), our best ensemble reached similar CES as other work.more » « less
-
Abstract This article analyzes various color quantization methods using multiple image quality assessment indices. Experiments were conducted with ten color quantization methods and eight image quality indices on a dataset containing 100 RGB color images. The set of color quantization methods selected for this study includes well-known methods used by many researchers as a baseline against which to compare new methods. On the other hand, the image quality assessment indices selected are the following: mean squared error, mean absolute error, peak signal-to-noise ratio, structural similarity index, multi-scale structural similarity index, visual information fidelity index, universal image quality index, and spectral angle mapper index. The selected indices not only include the most popular indices in the color quantization literature but also more recent ones that have not yet been adopted in the aforementioned literature. The analysis of the results indicates that the conventional assessment indices used in the color quantization literature generate different results from those obtained by newer indices that take into account the visual characteristics of the images. Therefore, when comparing color quantization methods, it is recommended not to use a single index based solely on pixelwise comparisons, as is the case with most studies to date, but rather to use several indices that consider the various characteristics of the human visual system.more » « less
-
Abstract EUropean Heliospheric FORecasting Information Asset (EUHFORIA) is a physics‐based data‐driven solar wind and coronal mass ejections (CMEs) propagation model designed for space weather forecasting and event analysis investigations. Although EUHFORIA can predict the solar wind plasma and magnetic field properties at Earth, it is not equipped to quantify the geo‐effectiveness of the solar transients in terms of geomagnetic indices like the disturbance storm time (Dst) index and the auroral indices, that quantify the impact of the magnetized plasma encounters on Earth's magnetosphere. Therefore, we couple EUHFORIA with the Open Geospace General Circulation Model (OpenGGCM), a magnetohydrodynamic model of the response of Earth's magnetosphere, ionosphere, and thermosphere to transient solar wind characteristics. In this coupling, OpenGGCM is driven by the solar wind and interplanetary magnetic field obtained from EUHFORIA simulations to produce the magnetospheric and ionospheric response to the CMEs. This coupling is validated with two observed geo‐effective CME events driven with the spheromak flux‐rope CME model. We compare these simulation results with the indices obtained from OpenGGCM simulations driven by the measured solar wind data from spacecraft. We further employ the dynamic time warping (DTW) technique to assess the model performance in predicting Dst. The main highlight of this study is to use EUHFORIA simulated time series to predict the Dst and auroral indices 1–2 days in advance, as compared to using the observed solar wind data at L1, which only provides predictions 1–2 hr before the actual impact.more » « less
-
Abstract Understanding and forecasting outer radiation belt electron flux dropouts is one of the top concerns in space physics. By constructing Support Vector Machine (SVM) models to predict storm‐time dropouts for both relativistic and ultra‐relativistic electrons overL = 4.0–6.0, we investigate the nonlinear correlations between various driving factors (model inputs) and dropouts (model output) and rank their relative importance. Only time series of geomagnetic indices and solar wind parameters are adopted as model inputs. A comparison of the performance of the SVM models that uses only one driving factor at a time enables us to identify the most informative parameter and its optimal length of time history. Its accuracy and the ability to correctly predict dropouts identifies the SYM‐H index as the governing factor atL = 4.0–4.5, while solar wind parameters dominate the dropouts at higher L‐shells (L = 6.0). Our SVM model also gives good prediction of dropouts during completely out‐of‐sample storms.more » « less
