An official website of the United States government
Abstract To assess the effect of uncertainties in solar wind driving on the predictions from the operational configuration of the Space Weather Modeling Framework, we have developed a nonparametric method for generating multiple possible realizations of the solar wind just upstream of the bow shock, based on observations near the first Lagrangian point. We have applied this method to the solar wind inputs at the upstream boundary of Space Weather Modeling Framework and have simulated the geomagnetic storm of 5 April 2010. We ran a 40‐member ensemble for this event and have used this ensemble to quantify the uncertainty in the predicted Sym‐H index and ground magnetic disturbances due to the uncertainty in the upstream boundary conditions. Both the ensemble mean and the unperturbed simulation tend to underpredict the magnitude of Sym‐H in the quiet interval before the storm and overpredict in the storm itself, consistent with previous work. The ensemble mean is a more accurate predictor of Sym‐H, improving the mean absolute error by nearly 2 nT for this interval and displaying a smaller bias. We also examine the uncertainty in predicted maxima in ground magnetic disturbances. The confidence intervals are typically narrow during periods where the predicted dBH/dtis low. The confidence intervals are often much wider where the median prediction is for enhanced dBH/dt. The ensemble also allows us to identify intervals of activity that cannot be explained by uncertainty in the solar wind driver, driving further model improvements. This work demonstrates the feasibility and importance of ensemble modeling for space weather applications.
more »
« less
Probabilistic Solar Proxy Forecasting With Neural Network Ensembles
Abstract Space weather indices are used commonly to drive forecasts of thermosphere density, which affects objects in low‐Earth orbit (LEO) through atmospheric drag. One commonly used space weather proxy,F10.7cm, correlates well with solar extreme ultra‐violet (EUV) energy deposition into the thermosphere. Currently, the USAF contracts Space Environment Technologies (SET), which uses a linear algorithm to forecastF10.7cm. In this work, we introduce methods using neural network ensembles with multi‐layer perceptrons (MLPs) and long‐short term memory (LSTMs) to improve on the SET predictions. We make predictions only from historicalF10.7cmvalues. We investigate data manipulation methods (backwards averaging and lookback) as well as multi step and dynamic forecasting. This work shows an improvement over the popular persistence and the operational SET model when using ensemble methods. The best models found in this work are ensemble approaches using multi step or a combination of multi step and dynamic predictions. Nearly all approaches offer an improvement, with the best models improving between 48% and 59% on relative MSE with respect to persistence. Other relative error metrics were shown to improve greatly when ensembles methods were used. We were also able to leverage the ensemble approach to provide a distribution of predicted values; allowing an investigation into forecast uncertainty. Our work found models that produced less biased predictions at elevated and high solar activity levels. Uncertainty was also investigated through the use of a calibration error score metric (CES), our best ensemble reached similar CES as other work.
more »
« less
- Award ID(s):
- 2149747
- PAR ID:
- 10456552
- Publisher / Repository:
- DOI PREFIX: 10.1029
- Date Published:
- Journal Name:
- Space Weather
- Volume:
- 21
- Issue:
- 9
- ISSN:
- 1542-7390
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract Machine learning (ML) has been applied to space weather problems with increasing frequency in recent years, driven by an influx of in-situ measurements and a desire to improve modeling and forecasting capabilities throughout the field. Space weather originates from solar perturbations and is comprised of the resulting complex variations they cause within the numerous systems between the Sun and Earth. These systems are often tightly coupled and not well understood. This creates a need for skillful models with knowledge about the confidence of their predictions. One example of such a dynamical system highly impacted by space weather is the thermosphere, the neutral region of Earth’s upper atmosphere. Our inability to forecast it has severe repercussions in the context of satellite drag and computation of probability of collision between two space objects in low Earth orbit (LEO) for decision making in space operations. Even with (assumed) perfect forecast of model drivers, our incomplete knowledge of the system results in often inaccurate thermospheric neutral mass density predictions. Continuing efforts are being made to improve model accuracy, but density models rarely provide estimates of confidence in predictions. In this work, we propose two techniques to develop nonlinear ML regression models to predict thermospheric density while providing robust and reliable uncertainty estimates: Monte Carlo (MC) dropout and direct prediction of the probability distribution, both using the negative logarithm of predictive density (NLPD) loss function. We show the performance capabilities for models trained on both local and global datasets. We show that the NLPD loss provides similar results for both techniques but the direct probability distribution prediction method has a much lower computational cost. For the global model regressed on the Space Environment Technologies High Accuracy Satellite Drag Model (HASDM) density database, we achieve errors of approximately 11% on independent test data with well-calibrated uncertainty estimates. Using an in-situ CHAllenging Minisatellite Payload (CHAMP) density dataset, models developed using both techniques provide test error on the order of 13%. The CHAMP models—on validation and test data—are within 2% of perfect calibration for the twenty prediction intervals tested. We show that this model can also be used to obtain global density predictions with uncertainties at a given epoch.more » « less
-
Abstract Observational evidence shows changes to North American weather regime occurrence depending on the strength of the lower-stratospheric polar vortex. However, it is not yet clear how this occurs or to what extent an improved stratospheric forecast would change regime predictions. Here we analyze four North American regimes at 500 hPa, constructed in principal component (PC) space. We consider both the location of the regimes in PC space and the linear regression between each PC and the lower-stratospheric zonal-mean winds, yielding a theory of which regime transitions are likely to occur due to changes in the lower stratosphere. Using a set of OpenIFS simulations, we then test the effect of relaxing the polar stratosphere to ERA-Interim on subseasonal regime predictions. The model start dates are selected based on particularly poor subseasonal regime predictions in the European Centre for Medium-Range Weather Forecasts CY43R3 hindcasts. While the results show only a modest improvement to the number of accurate regime predictions, there is a substantial reduction in Euclidean distance error in PC space. The average movement of the forecasts within PC space is found to be consistent with expectation for moderate-to-large lower-stratospheric zonal wind perturbations. Overall, our results provide a framework for interpreting the stratospheric influence on North American regime behavior. The results can be applied to subseasonal forecasts to understand how stratospheric uncertainty may affect regime predictions, and to diagnose which regime forecast errors are likely to be related to stratospheric errors. Significance Statement Predicting the weather several weeks ahead is a major challenge with large potential benefits to society. The strength of the circulation more than 10 km above the Arctic during winter (i.e., the polar vortex) is one source of predictability. This study investigates how forecast error and uncertainty in the polar vortex can impact predictions of large-scale weather patterns called “regimes” over North America. Through statistical analysis of observations and experiments with a weather forecast model, we develop an understanding of which regime changes are more likely to be due to changes in the polar vortex. The results will help forecasters and researchers understand the contribution of the stratosphere to changes in weather patterns, and in assessing and improving weather forecast models.more » « less
-
Abstract Robust ecological forecasting of tree growth under future climate conditions is critical to anticipate future forest carbon storage and flux. Here, we apply three ingredients of ecological forecasting that are key to improving forecast skill: data fusion, confronting model predictions with new data, and partitioning forecast uncertainty. Specifically, we present the first fusion of tree‐ring and forest inventory data within a Bayesian state‐space model at a multi‐site, regional scale, focusing onPinus ponderosavar.brachypterain the southwestern US. Leveraging the complementarity of these two data sources, we parsed the ecological complexity of tree growth into the effects of climate, tree size, stand density, site quality, and their interactions, and quantified uncertainties associated with these effects. New measurements of trees, an ongoing process in forest inventories, were used to confront forecasts of tree diameter with observations, and evaluate alternative tree growth models. We forecasted tree diameter and increment in response to an ensemble of climate change projections, and separated forecast uncertainty into four different causes: initial conditions, parameters, climate drivers, and process error. We found a strong negative effect of fall–spring maximum temperature, and a positive effect of water‐year precipitation on tree growth. Furthermore, tree vulnerability to climate stress increases with greater competition, with tree size, and at poor sites. Under future climate scenarios, we forecast increment declines of 22%–117%, while the combined effect of climate and size‐related trends results in a 56%–91% decline. Partitioning of forecast uncertainty showed that diameter forecast uncertainty is primarily caused by parameter and initial conditions uncertainty, but increment forecast uncertainty is mostly caused by process error and climate driver uncertainty. This fusion of tree‐ring and forest inventory data lays the foundation for robust ecological forecasting of aboveground biomass and carbon accounting at tree, plot, and regional scales, including iterative improvement of model skill.more » « less
-
Barambones, Oscar (Ed.)Accurate quantification of uncertainty in solar photovoltaic (PV) generation forecasts is imperative for the efficient and reliable operation of the power grid. In this paper, a data-driven non-parametric probabilistic method based on the Naïve Bayes (NB) classification algorithm and Dempster–Shafer theory (DST) of evidence is proposed for day-ahead probabilistic PV power forecasting. This NB-DST method extends traditional deterministic solar PV forecasting methods by quantifying the uncertainty of their forecasts by estimating the cumulative distribution functions (CDFs) of their forecast errors and forecast variables. The statistical performance of this method is compared with the analog ensemble method and the persistence ensemble method under three different weather conditions using real-world data. The study results reveal that the proposed NB-DST method coupled with an artificial neural network model outperforms the other methods in that its estimated CDFs have lower spread, higher reliability, and sharper probabilistic forecasts with better accuracy.more » « less
