skip to main content


Title: Short-term solar irradiance forecasting using convolutional neural networks and cloud imagery
Abstract

Access to accurate, generalizable and scalable solar irradiance prediction is critical for smooth solar-grid integration, especially in the light of the accelerated global adoption of solar energy production. Both physical and statistical prediction models of solar irradiance have been proposed in the literature. Physical models require meteorological forecasts—generated by computationally expensive models—to predict solar irradiance, with limited accuracy in sub-daily predictions. Statistical models leveragein-situmeasurements which require expensive equipment and do not account for meso-scale atmospheric dynamics. We address these fundamental gaps by developing a convolutional global horizontal irradiance prediction model, using convolutional neural networks and publicly accessible satellite cloud images. Our proposed model predicts solar irradiance in 12 different locations in the US for various prediction time horizons. Our model yields up to 24% improvement in an hour-ahead predictions and 26% in a day-ahead predictions compared to a persistence forecast. Moreover, using saliency maps and target-location-focused cropping, we demonstrate the benefits of incorporating meso-scale atmospheric dynamics for prediction performance. Our results are critical for energy systems planners, utility managers and electricity market participants to ensure efficient harvesting of the solar energy and reliable operation of the grid.

 
more » « less
NSF-PAR ID:
10362276
Author(s) / Creator(s):
; ;
Publisher / Repository:
IOP Publishing
Date Published:
Journal Name:
Environmental Research Letters
Volume:
16
Issue:
4
ISSN:
1748-9326
Page Range / eLocation ID:
Article No. 044045
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract

    Electrons with energies ≥40 keV can be found at low density in many different regions of Earth's magnetosphere. A litany of fundamental questions in space physics have focused on the acceleration mechanism of these particles, given that the sources of plasma are the relatively cool ionosphere and solar wind (∼1–100s eV). Upgraded global solar wind‐magnetosphere simulations which can resolve mesoscale dynamics have the ability to enhance our understanding of these high energy particles. This is because the energization of particles often takes the form of a sequence of discrete steps, potentially occurring in different regions of the magnetosphere and due to both meso‐ and global‐scale processes. First, brief results are presented from the Grid Agnostic MHD for Extended Research Applications (GAMERA) global simulation on the structure of the cusp diamagnetic cavity for northward and southward IMF. Then, the Conservative Hamiltonian Integrator for Magnetospheric Particles (CHIMP) framework, with both guiding center and full Lorentz integrators, evolves necessary parameters such as the energy and pitch angle of electron test particles to investigate particle acceleration inside the cavity, as well as the ultimate fate of electrons accelerated inside the cavity. The simulation shows that particles can gain ≥ 10 keV inside the cavity and subsequently leak into the magnetosheath or onto dipolar field lines where they execute different types of bounce motion. The distribution of test particles initialized inside the cavity is compared with Magnetospheric Multi‐Scale (MMS) observations.

     
    more » « less
  2. Power grid operators rely on solar irradiance forecasts to manage uncertainty and variability associated with solar power. Meteorological factors such as cloud cover, wind direction, and wind speed affect irradiance and are associated with a high degree of variability and uncertainty. Statistical models fail to accurately capture the dependence between these factors and irradiance. In this paper, we introduce the idea of applying multivariate Gated Recurrent Units (GRU) to forecast Direct Normal Irradiance (DNI) hourly. The proposed GRU-based forecasting method is evaluated against traditional Long Short-Term Memory (LSTM) using historical irradiance data (i.e., weather variables that include cloud cover, wind direction, and wind speed) to forecast irradiance forecasting over intra-hour and inter-hour intervals. Our evaluation on one of the sites from Measurement and Instrumentation Data Center indicate that both GRU and LSTM improved DNI forecasting performance when evaluated under different conditions. Moreover, including wind direction and wind speed can have substantial improvement in the accuracy of DNI forecasts. Besides, the forecasting model can accurately forecast irradiance values over multiple forecasting horizons. 
    more » « less
  3. Abstract

    A field campaign at Siple Dome in West Antarctica during the austral summer 2019/20 offers an opportunity to evaluate climate model performance, particularly cloud microphysical simulation. Over Antarctic ice sheets and ice shelves, clouds are a major regulator of the surface energy balance, and in the warm season their presence occasionally induces surface melt that can gradually weaken an ice shelf structure. This dataset from Siple Dome, obtained using transportable and solar-powered equipment, includes surface energy balance measurements, meteorology, and cloud remote sensing. To demonstrate how these data can be used to evaluate model performance, comparisons are made with meteorological reanalysis known to give generally good performance over Antarctica (ERA5). Surface albedo measurements show expected variability with observed cloud amount, and can be used to evaluate a model’s snowpack parameterization. One case study discussed involves a squall with northerly winds, during which ERA5 fails to produce cloud cover throughout one of the days. A second case study illustrates how shortwave spectroradiometer measurements that encompass the 1.6-μm atmospheric window reveal cloud phase transitions associated with cloud life cycle. Here, continuously precipitating mixed-phase clouds become mainly liquid water clouds from local morning through the afternoon, not reproduced by ERA5. We challenge researchers to run their various regional or global models in a manner that has the large-scale meteorology follow the conditions of this field campaign, compare cloud and radiation simulations with this Siple Dome dataset, and potentially investigate why cloud microphysical simulations or other model components might produce discrepancies with these observations.

    significance statement

    Antarctica is a critical region for understanding climate change and sea level rise, as the great ice sheets and the ice shelves are subject to increasing risk as global climate warms. Climate models have difficulties over Antarctica, particularly with simulation of cloud properties that regulate snow surface melting or refreezing. Atmospheric and climate-related field work has significant challenges in the Antarctic, due to the small number of research stations that can support state-of-the-art equipment. Here we present new data from a suite of transportable and solar-powered instruments that can be deployed to remote Antarctic sites, including regions where ice shelves are most at risk, and we demonstrate how key components of climate model simulations can be evaluated against these data.

     
    more » « less
  4. Abstract

    “Supermodeling” climate by allowing different models to assimilate data from one another in run time has been shown to give results superior to those of any one model and superior to any weighted average of model outputs. The only free parameters, connection strengths between corresponding variables in each pair of models, are determined using some form of machine learning. It is demonstrated that supermodeling succeeds because near critical states, interscale interactions are important but unresolved processes cannot be effectively represented diagnostically in any single parameterization scheme. In two examples, a pair of toy quasigeostrophic (QG) channel models of the midlatitudes and a pair of ECHAM5 models of the tropical Pacific atmosphere with a common ocean, supermodels dynamically combine parameterization schemes so as to capture criticality, associated critical structures, and the supporting scale interactions. The QG supermodeling scheme extends a previous configuration in which two such models synchronize with intermodel connections only between medium-scale components of the flow; here the connections are trained against a third “real” model. Intermittent blocking patterns characterize the critical behavior thus obtained, even where such patterns are missing in the constituent models. In the ECHAM-based climate supermodel, the corresponding critical structure is the single ITCZ pattern, a pattern that occurs in neither of the constituent models. For supermodels of both types, power spectra indicate enhanced interscale interactions in frequency or energy ranges of physical interest, in agreement with observed data, and supporting a generalized form of the self-organized criticality hypothesis.

    Significance Statement

    In a “supermodel” of Earth’s climate, alternative models (climate simulations), which differ in the way they represent processes on the smallest scales, are trained to exchange information as they run, adjusting to one another much as weather prediction models adjust to new observations. They form a consensus, capturing atmospheric behaviors that have eluded all the separate models. We demonstrate that simplified supermodels succeed, where no single approach can, by correctly representingcritical phenomenainvolving sudden qualitative transitions, such as occur in El Niño events, that depend on interactions among atmospheric processes on many different scales in space and time. The correct reproduction of critical phenomena is vital both for predicting weather and for projecting the effects of climate change.

     
    more » « less
  5. Abstract

    Numerical weather prediction models and high-performance computing have significantly improved our ability to model near-surface variables, but their uncertainty quantification still remains a challenging task. Ensembles are usually produced to depict a series of possible future states of the atmosphere, as a means to quantify the prediction uncertainty, but this requires multiple instantiation of the model, leading to an increased computational cost. Weather analogs, alternatively, can be used to generate ensembles without repeated model runs. The analog ensemble (AnEn) is a technique to identify similar weather patterns for near-surface variables and quantify forecast uncertainty. Analogs are chosen based on a similarity metric that calculates the weighted multivariate Euclidean distance. However, identifying optimal weights for similarity metric becomes a bottleneck because it involves performing a constrained exhaustive search. As a result, only a few predictors were selected and optimized in previous AnEn studies. A new machine learning similarity metric is proposed to improve the theoretical framework on how weather analogs are identified. First, a deep learning network is trained to generate latent features using all the temporal multivariate input predictors. Analogs are then selected in this latent space, rather than the original predictor space. The proposed method does not require prior predictor selection and an exhaustive search, thus presenting a significant computational benefit and scalability. It is tested for surface wind speed and solar irradiance forecasts in Pennsylvania from 2017 to 2019. Results show that the proposed method is capable of handling a large number of predictors, and it outperforms the original similarity metric in RMSE, bias, and CRPS. Since the data-driven transformation network is trained using the historical record, the proposed method has been found to be more flexible for searching through a longer record.

     
    more » « less