An official website of the United States government
Abstract Solar energetic particle (SEP) events and their major subclass, solar proton events (SPEs), can have unfavorable consequences on numerous aspects of life and technology, making them one of the most harmful effects of solar activity. Garnering knowledge preceding such events by studying operational data flows is essential for their forecasting. Considering only solar cycle (SC) 24 in our previous study, we found that it may be sufficient to only utilize proton and soft X-ray (SXR) parameters for SPE forecasts. Here, we report a catalog recording ≥10 MeV ≥10 particle flux unit SPEs with their properties, spanning SCs 22–24, using NOAA’s Geostationary Operational Environmental Satellite flux data. We report an additional catalog of daily proton and SXR flux statistics for this period, employing it to test the application of machine learning (ML) on the prediction of SPEs using a support vector machine (SVM) and extreme gradient boosting (XGBoost). We explore the effects of training models with data from oneandtwo SCs, evaluating how transferable a model might be across different time periods. XGBoost proved to be more accurate than SVMs for almost every test considered, while also outperforming operational SWPC NOAA predictions and a persistence forecast. Interestingly, training done with SC 24 produces weaker true skill statistic and Heidke skill scores2, even when paired with SC 22 or SC 23, indicating transferability issues. This work contributes toward validating forecasts using long-spanning data—an understudied area in SEP research that should be considered to verify the cross cycle robustness of ML-driven forecasts.
more »
« less
Investigating Performance Trends of Simulated Real-time Solar Flare Predictions: The Impacts of Training Windows, Data Volumes, and the Solar Cycle
Abstract This study explores the behavior of machine-learning-based flare forecasting models deployed in a simulated operational environment. Using Georgia State University’s Space Weather Analytics for Solar Flares benchmark data set, we examine the impacts of training methodology and the solar cycle on decision tree, support vector machine, and multilayer perceptron performance. We implement our classifiers using three temporal training windows: stationary, rolling, and expanding. The stationary window trains models using a single set of data available before the first forecasting instance, which remains constant throughout the solar cycle. The rolling window trains models using data from a constant time interval before the forecasting instance, which moves with the solar cycle. Finally, the expanding window trains models using all available data before the forecasting instance. For each window, a number of input features (1, 5, 10, 25, 50, and 120) and temporal sizes (5, 8, 11, 14, 17, and 20 months) were tested. To our surprise, we found that, for a window of 20 months, skill scores were comparable regardless of the window type, feature count, and classifier selected. Furthermore, reducing the size of this window only marginally decreased stationary and rolling window performance. This implies that, given enough data, a stationary window can be chosen over other window types, eliminating the need for model retraining. Finally, a moderately strong positive correlation was found to exist between a model’s false-positive rate and the solar X-ray background flux. This suggests that the solar cycle phase has a considerable influence on forecasting.
more »
« less
- Award ID(s):
- 1936361
- PAR ID:
- 10497506
- Publisher / Repository:
- DOI PREFIX: 10.3847
- Date Published:
- Journal Name:
- The Astrophysical Journal
- Volume:
- 964
- Issue:
- 2
- ISSN:
- 0004-637X
- Format(s):
- Medium: X Size: Article No. 163
- Size(s):
- Article No. 163
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract This work explores the impacts of magnetogram projection effects on machine-learning-based solar flare forecasting models. Utilizing a methodology proposed by D. A. Falconer et al., we correct for projection effects present in Georgia State University’s Space Weather Analytics for Solar Flares benchmark data set. We then train and test a support vector machine classifier on the corrected and uncorrected data, comparing differences in performance. Additionally, we provide insight into several other methodologies that mitigate projection effects, such as stacking ensemble classifiers and active region location-informed models. Our analysis shows that data corrections slightly increase both the true-positive (correctly predicted flaring samples) and false-positive (nonflaring samples predicted as flaring) prediction rates, averaging a few percent. Similarly, changes in performance metrics are minimal for the stacking ensemble and location-based model. This suggests that a more complicated correction methodology may be needed to see improvements. It may also indicate inherent limitations when using magnetogram data for flare forecasting.more » « less
-
Abstract In this paper, a method for real-time forecasting of the dynamics of structures experiencing nonstationary inputs is described. This is presented as time series predictions across different timescales. The target applications include hypersonic vehicles, space launch systems, real-time prognostics, and monitoring of high-rate and energetic systems. This work presents numerical analysis and experimental results for the real-time implementation of a Fast Fourier Transform (FFT)-based approach for time series forecasting. For this preliminary study, a testbench structure that consists of a cantilever beam subjected to nonstationary inputs is used to generate experimental data. First, the data is de-trended, then the time series data is transferred into the frequency domain, and measures for frequency, amplitude, and phase are obtained. Thereafter, select frequency components are collected, transformed back to the time domain, recombined, and then the trend in the data is restored. Finally, the recombined signals are propagated into the future to the selected prediction horizon. This preliminary time series forecasting work is done offline using pre-recorded experimental data, and the FFT-based approach is implemented in a rolling window configuration. Here learning windows of 0.1, 0.5, and 1 s are considered with different computation times simulated. Results demonstrate that the proposed FFT-based approach can maintain a constant prediction horizon at 1 s with sufficient accuracy for the considered system. The performance of the system is quantified using a variety of metrics. Computational speed and prediction accuracy as a function of training time and learning window lengths are examined in this work. The algorithm configuration with the shortest learning window (0.1 s) is shown to converge faster following the nonstationary when compared to algorithm configuration with longer learning windows.more » « less
-
ABSTRACT Efforts are underway to use high-precision timing of pulsars in order to detect low-frequency gravitational waves. A limit to this technique is the timing noise generated by dispersion in the plasma along the line of sight to the pulsar, including the solar wind. The effects due to the solar wind vary with time, influenced by the change in solar activity on different time-scales, ranging up to ∼11 yr for a solar cycle. The solar wind contribution depends strongly on the angle between the pulsar line of sight and the solar disc, and is a dominant effect at small separations. Although solar wind models to mitigate these effects do exist, they do not account for all the effects of the solar wind and its temporal changes. Since low-frequency pulsar observations are most sensitive to these dispersive delays, they are most suited to test the efficacy of these models and identify alternative approaches. Here, we investigate the efficacy of some solar wind models commonly used in pulsar timing using long-term, high-cadence data on six pulsars taken with the Long Wavelength Array, and compare them with an operational solar wind model. Our results show that stationary models of the solar wind correction are insufficient to achieve the timing noise desired by pulsar timing experiments, and we need to use non-stationary models, which are informed by other solar wind observations, to obtain accurate timing residuals.more » « less
-
Abstract Regime shifts have large consequences for ecosystems and the services they provide. However, understanding the potential for, causes of, proximity to, and thresholds for regime shifts in nearly all settings is difficult. Generic statistical indicators of resilience have been proposed and studied in a wide range of ecosystems as a method to detect when regime shifts are becoming more likely without direct knowledge of underlying system dynamics or thresholds. These early warning statistics (EWS) have been studied separately but there have been few examples that directly compare temporal and spatial EWS in ecosystem‐scale empirical data. To test these methods, we collected high‐frequency time series and high‐resolution spatial data during a whole‐lake fertilization experiment while also monitoring an adjacent reference lake. We calculated two common EWS, standard deviation and autocorrelation, in both time series and spatial data to evaluate their performance prior to the resulting algal bloom. We also applied the quickest detection method to generate binary alarms of resilience change from temporal EWS. One temporal EWS, rolling window standard deviation, provided advanced warning in most variables prior to the bloom, showing trends and between‐lake patterns consistent with theory. In contrast, temporal autocorrelation and both measures of spatial EWS (spatial SD, Moran's I) provided little or no warning. By compiling time series data from this and past experiments with and without nutrient additions, we were able to evaluate temporal EWS performance for both constant and changing resilience conditions. True positive alarm rates were 2.5–8.3 times higher for rolling window standard deviation when a lake was being pushed towards a bloom than the rate of false positives when it was not. For rolling window autocorrelation, alarm rates were much lower and no variable had a higher true positive than false positive alarm rate. Our findings suggest temporal EWS provide advanced warning of algal blooms and that this approach could help managers prepare for and/or minimize negative bloom impacts.more » « less
