An official website of the United States government
Abstract Machine learning (ML) models are universal function approximators and—if used correctly—can summarize the information content of observational data sets in a functional form for scientific and engineering applications. A benefit to ML over parametric models is that there are no a priori assumptions about particular basis functions which can potentially limit the phenomena that can be modeled. In this work, we develop ML models on three data sets: the Space Environment Technologies High Accuracy Satellite Drag Model (HASDM) density database, a spatiotemporally matched data set of outputs from the Jacchia‐Bowman 2008 Empirical Thermospheric Density Model (JB2008), and an accelerometer‐derived density data set from CHAllenging Minisatellite Payload (CHAMP). These ML models are compared to the Naval Research Laboratory Mass Spectrometer and Incoherent Scatter radar (NRLMSIS 2.0) model to study the presence of post‐storm cooling in the middle‐thermosphere. We find that both NRLMSIS 2.0 and JB2008‐ML do not account for post‐storm cooling and consequently perform poorly in periods following strong geomagnetic storms (e.g., the 2003 Halloween storms). Conversely, HASDM‐ML and CHAMP‐ML do show evidence of post‐storm cooling indicating that this phenomenon is present in the original data sets. Results show that density reductions up to 40% can occur 1–3 days post‐storm depending on the location and strength of the storm.
more »
« less
Uncertainty quantification techniques for data-driven space weather modeling: thermospheric density application
Abstract Machine learning (ML) has been applied to space weather problems with increasing frequency in recent years, driven by an influx of in-situ measurements and a desire to improve modeling and forecasting capabilities throughout the field. Space weather originates from solar perturbations and is comprised of the resulting complex variations they cause within the numerous systems between the Sun and Earth. These systems are often tightly coupled and not well understood. This creates a need for skillful models with knowledge about the confidence of their predictions. One example of such a dynamical system highly impacted by space weather is the thermosphere, the neutral region of Earth’s upper atmosphere. Our inability to forecast it has severe repercussions in the context of satellite drag and computation of probability of collision between two space objects in low Earth orbit (LEO) for decision making in space operations. Even with (assumed) perfect forecast of model drivers, our incomplete knowledge of the system results in often inaccurate thermospheric neutral mass density predictions. Continuing efforts are being made to improve model accuracy, but density models rarely provide estimates of confidence in predictions. In this work, we propose two techniques to develop nonlinear ML regression models to predict thermospheric density while providing robust and reliable uncertainty estimates: Monte Carlo (MC) dropout and direct prediction of the probability distribution, both using the negative logarithm of predictive density (NLPD) loss function. We show the performance capabilities for models trained on both local and global datasets. We show that the NLPD loss provides similar results for both techniques but the direct probability distribution prediction method has a much lower computational cost. For the global model regressed on the Space Environment Technologies High Accuracy Satellite Drag Model (HASDM) density database, we achieve errors of approximately 11% on independent test data with well-calibrated uncertainty estimates. Using an in-situ CHAllenging Minisatellite Payload (CHAMP) density dataset, models developed using both techniques provide test error on the order of 13%. The CHAMP models—on validation and test data—are within 2% of perfect calibration for the twenty prediction intervals tested. We show that this model can also be used to obtain global density predictions with uncertainties at a given epoch.
more »
« less
- Award ID(s):
- 2140204
- PAR ID:
- 10381835
- Publisher / Repository:
- Nature Publishing Group
- Date Published:
- Journal Name:
- Scientific Reports
- Volume:
- 12
- Issue:
- 1
- ISSN:
- 2045-2322
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract Thermospheric density influences the atmospheric drag and is crucial for space missions. This paper introduces a global thermospheric density prediction framework based on a deep evidential method. The proposed framework predicts thermospheric density at the required time and geographic position with given geomagnetic and solar indices. It is called global to differentiate it from existing research that only predicts density along a satellite orbit. Through the deep evidential method, we assimilate data from various sources including solar and geomagnetic conditions, accelerometer‐derived density data, and empirical models including the Jacchia‐Bowman model (JB‐2008) and the Naval Research Laboratory Mass Spectrometer and Incoherent Scatter Radar Extended (NRLMSISE‐00) model. The framework is investigated on five test cases along various satellites from 2003 to 2015 involving geomagnetic storms with Disturbance Storm Time (Dst) values smaller than −50 . Results show that the proposed framework can generate density with higher accuracy than the two empirical models. It can also obtain reliable uncertainty estimations. Global density estimations at altitudes from 200 to 550 km are also presented and compared with empirical models on both quiet and storm conditions.more » « less
-
The rapidly increasing congestion in the low Earth environment makes the modeling of uncertainty in atmospheric drag force a critical task, affecting space situational awareness (SSA) activities like the probability of collision estimation. A key element in atmospheric drag modeling is the assessment of uncertainty in the atmospheric drag coefficient estimate. While atmospheric drag coefficients for space objects with known characteristics can be computed numerically, they suffer from large computational costs for practical applications. In this work, we use cost-effective data-driven stochastic methods for modeling the drag coefficients of objects in the low Earth orbit (LEO) region. The training data is generated using the numerical Test Particle Monte Carlo (TPMC) method. TPMC is simulated with Cercignani–Lampis–Lord (CLL) gas-surface interaction (GSI) model. Mehta et al. [1] use a Gaussian process regression (GPR) model to predict satellite drag coefficient, but the authors did not estimate the predictive uncertainty. The first part of this research extends the work by Mehta et al. [1] by fitting a GPR model to the training data and performing predictive uncertainty estimation. The results of the Gaussian fit are then compared against a deep neural network (DNN) model aided by the Monte Carlo dropout approach. To the best of our knowledge, this is the first study to use the aforementioned stochastic deep learning algorithm to perform predictive uncertainty estimation of the estimated satellite drag coefficient. Apart from the accuracy of the models, we also undertake the task of calibrating the models. Simulations are carried out for a spherical satellite followed by the Champ satellite. Finally, quantification of the effect of drag coefficient uncertainty on orbit prediction is carried out for different solar activity and geomagnetic activity levels.more » « less
-
null (Ed.)The specification and prediction of density fluctuations in the thermosphere, especially during geomagnetic storms, is a key challenge for space weather observations and modeling. It is of great operational importance for tracking objects orbiting in near-Earth space. For low-Earth orbit, variations in neutral density represent the most important uncertainty for propagation and prediction of satellite orbits. An international conference in 2018 conducted under the auspices of the NASA Community Coordinated Modeling Center (CCMC) included a workshop on neutral density modeling, using both empirical and numerical methods, and resulted in the organization of an initial effort of model comparison and evaluation. Here, we present an updated metric for model assessment under geomagnetic storm conditions by dividing a storm in four phases with respect to the time of minimum Dst and then calculating the mean density ratios and standard deviations and correlations. Comparisons between three empirical (NRLMSISE-00, JB2008 and DTM2013) and two first-principles models (TIE-GCM and CTIPe) and neutral density data sets that include measurements by the CHAMP, GRACE, and GOCE satellites for 13 storms are presented. The models all show reduced performance during storms, notably much increased standard deviations, but DTM2013, JB2008 and CTIPe did not on average reveal a significant bias in the four phases of our metric. DTM2013 and TIE-GCM driven with the Weimer model achieved the best results taking the entire storm event into account, while NRLMSISE-00 systematically and significantly underestimates the storm densities. Numerical models are still catching up to empirical methods on a statistical basis, but as their drivers become more accurate and they become available at higher resolutions, they will surpass them in the foreseeable future.more » « less
-
To improve Thermosphere–Ionosphere modeling during disturbed conditions, data assimilation schemes that can account for the large and fast-moving gradients moving through the modeled domain are necessary. We argue that this requires a physics based background model with a non-stationary covariance. An added benefit of using physics-based models would be improved forecasting capability over largely persistence-based forecasts of empirical models. As a reference implementation, we have developed an ensemble Kalman Filter (enKF) software called Thermosphere Ionosphere Data Assimilation (TIDA) using the physics-based Coupled Thermosphere Ionosphere Plasmasphere electrodynamics (CTIPe) model as the background. In this paper, we present detailed results from experiments during the 2003 Halloween Storm, 27–31 October 2003, under very disturbed ( K p = 9) conditions while assimilating GRACE-A and B, and CHAMP neutral density measurements. TIDA simulates this disturbed period without using the L1 solar wind measurements, which were contaminated by solar energetic protons, by estimating the model drivers from the density measurements. We also briefly present statistical results for two additional storms: September 27 – October 2, 2002, and July 26 – 30, 2004, to show that the improvement in assimilated neutral density specification is not an artifact of the corrupted forcing observations during the 2003 Halloween Storm. By showing statistical results from assimilating one satellite at a time, we show that TIDA produces a coherent global specification for neutral density throughout the storm – a critical capability in calculating satellite drag and debris collision avoidance for space traffic management.more » « less
