An official website of the United States government
Abstract. Machine learning is quickly becoming a commonly used technique for wind speed and power forecasting. Many machine learning methods utilize exogenous variables as input features, but there remains the question of which atmospheric variables are most beneficial for forecasting, especially in handling non-linearities that lead to forecasting error. This question is addressed via creation of a hybrid model that utilizes an autoregressive integrated moving-average (ARIMA) model to make an initial wind speed forecast followed by a random forest model that attempts to predict the ARIMA forecasting error using knowledge of exogenous atmospheric variables.Variables conveying information about atmospheric stability and turbulence as well as inertial forcing are found to be useful in dealing with non-linear error prediction. Streamwise wind speed, time of day, turbulence intensity, turbulent heat flux, vertical velocity, and wind direction are found to be particularly useful when used in unison for hourly and 3 h timescales. The prediction accuracy of the developed ARIMA–random forest hybrid model is compared to that of the persistence and bias-corrected ARIMA models. The ARIMA–random forest model is shown to improve upon the latter commonly employed modeling methods, reducing hourly forecasting error by up to 5 % below that of the bias-corrected ARIMA model and achieving an R2 value of 0.84 with true wind speed.
more »
« less
Novel methods for adaptive time-series forecasting and prediction-interval construction
Abstract We propose novel methods for adaptive series forecasting and prediction-interval construction, illustrated with COVID-19 case and death counts. Our framework applies an automated transformation to reduce heteroscedasticity, then imposes a constrained smoothing near the forecast edge via robust quadratic regression, emphasizing recent data. A Long Short-Term Memory (LSTM) model combined with ARIMA-based noise correction further refines the forecast. Compared to conventional methods (e.g., ARIMA alone, unprocessed deep learning), this adaptive approach achieves superior metrics and reliable bootstrap-derived confidence and prediction intervals. We also highlight how reinforcement learning (RL) can offer promising avenues for real-time decision-making and further improvements in forecasting adaptability.
more »
« less
- Award ID(s):
- 1662629
- PAR ID:
- 10669719
- Publisher / Repository:
- Springer
- Date Published:
- Journal Name:
- Annals of Operations Research
- ISSN:
- 0254-5330
- Subject(s) / Keyword(s):
- Adaptive learning, Pre-processing, Forecast, Confidence and prediction interval
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Forecasting time series data is an important subject in economics, business, and finance. Traditionally, there are several techniques to effectively forecast the next lag of time series data such as univariate Autoregressive (AR), univariate Moving Average (MA), Simple Exponential Smoothing (SES), and more notably Autoregressive Integrated Moving Average (ARIMA) with its many variations. In particular, ARIMA model has demonstrated its outperformance in precision and accuracy of predicting the next lags of time series. With the recent advancement in computational power of computers and more importantly development of more advanced machine learning algorithms and approaches such as deep learning, new algorithms are developed to analyze and forecast time series data. The research question investigated in this article is that whether and how the newly developed deep learning-based algorithms for forecasting time series data, such as “Long Short-Term Memory (LSTM)”, are superior to the traditional algorithms. The empirical studies conducted and reported in this article show that deep learning-based algorithms such as LSTM outperform traditional-based algorithms such as ARIMA model. More specifically, the average reduction in error rates obtained by LSTM was between 84 - 87 percent when compared to ARIMA indicating the superiority of LSTM to ARIMA. Furthermore, it was noticed that the number of training times, known as “epoch” in deep learning, had no effect on the performance of the trained forecast model and it exhibited a truly random behavior.more » « less
-
Chen, Li-Pang (Ed.)BackgroundMany disciplines, such as public health, rely on statistical time series models for real-time and retrospective forecasting efforts; however, effectively implementing related methods often requires extensive programming knowledge. Therefore, such tools remain largely inaccessible to those with limited programming experience, including students training in modeling, as well as professionals and policymakers seeking to forecast an epidemic’s trajectory. To address the need for accessible and intuitive forecasting applications, we presentStatModPredict, an R-Shiny dashboard for conducting robust forecasting analysis utilizing auto-regressive integrated moving average (ARIMA), generalized linear models (GLM), generalized additive models (GAM), and Meta’s Prophet model. MethodsStatModPredictsupports robust real-time forecasting and retrospective model analysis, including fitting, forecasting, evaluation, visualization, and comparison of results from four popular models. After loading an incident time series data set into the interface, users can easily customize model parameters and forecasting options to obtain the desired output. Additionally,StatModPredictoffers multiple editable figures for, but not limited to, the time series data, the forecasts, and model fit and forecast metrics. Users can also upload external forecasts produced elsewhere and evaluate their performance alongside the dashboard’s built-in models, thereby enabling direct comparisons. We provide a detailed demonstration of the dashboard’s features using publicly available annual HIV case data in the US. A video tutorial is available athttps://www.youtube.com/watch?v=zgZOvqhvqw8. ConclusionsBy eliminating programming barriers,StatModPredictfacilitates exploration and use by students training in forecasting, as well as professionals and policymakers aiming to forecast epidemic trajectories. Additionally, the flexibility in the required input data structure and parameter specification process extends the application ofStatModPredictto any discipline that employs time series data. By offering this open-source interface, we aim to broaden access to forecasting tools, promote hands-on learning, and foster contributions from users across disciplines.more » « less
-
During the 2022–2023 unprecedented mpox epidemic, near real-time short-term forecasts of the epidemic’s trajectory were essential in intervention implementation and guiding policy. However, as case levels have significantly decreased, evaluating model performance is vital to advancing the field of epidemic forecasting. Using laboratory-confirmed mpox case data from the Centers for Disease Control and Prevention and Our World in Data teams, we generated retrospective sequential weekly forecasts for Brazil, Canada, France, Germany, Spain, the United Kingdom, the United States and at the global scale using an auto-regressive integrated moving average (ARIMA) model, generalized additive model, simple linear regression, Facebook’s Prophet model, as well as the sub-epidemic wave andn-sub-epidemic modelling frameworks. We assessed forecast performance using average mean squared error, mean absolute error, weighted interval scores, 95% prediction interval coverage, skill scores and Winkler scores. Overall, then-sub-epidemic modelling framework outcompeted other models across most locations and forecasting horizons, with the unweighted ensemble model performing best most frequently. Then-sub-epidemic and spatial-wave frameworks considerably improved in average forecasting performance relative to the ARIMA model (greater than 10%) for all performance metrics. Findings further support sub-epidemic frameworks for short-term forecasting epidemics of emerging and re-emerging infectious diseases.more » « less
-
ABSTRACT Accurate wind power forecasts are essential for energy management and resource allocation. However, because of complex weather dynamics and other nonlinearities, it is exceedingly difficult to forecast wind power on the multisite level for dozens of wind farms at once. This paper proposes a hybridized approach that leverages deep learning to predict future forecast errors from physics‐based numerical weather prediction (NWP) model estimates. Utilizing errors from NWP forecasts allows integration of critical atmospheric and meteorological dynamics into the forecasting model, and we demonstrate the importance of post‐calibration based on the physics versus pure data‐driven wind power prediction. This post‐calibration approach is enabled by the inverted transformer architecture, which efficiently and effectively learns meaningful wind farm variate representations, resulting in accurate spatiotemporal corrections to the forecasts. We also investigate modifying the iTransformer with a new embedding approach, named SpaceEmbed, that explicitly encodes spatial distance information into the network. The proposed approach is validated with a case study using real‐world data and forecasts from the Electric Reliability Council of Texas (ERCOT) in 2015 for 74 wind farms in Texas at different time scales. Using the high sustained limit as the metric for power generation, the iTransformer outperforms other state‐of‐the‐art deep learning forecasting methods, succeeding at the post‐calibration task by reducing NWP forecast error by up to 33% on average.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript
- Journal Article:
- https://doi.org/10.1007/s10479-025-06795-2
