An official website of the United States government
Few-shot machine learning attempts to predict outputs given only a very small number of training examples. The key idea behind most few-shot learning approaches is to pre-train the model with a large number of instances from a different but related class of data, classes for which a large number of instances are available for training. Few-shot learning has been most successfully demonstrated for classification problems using Siamese deep learning neural networks. Few-shot learning is less extensively applied to time-series forecasting. Few-shot forecasting is the task of predicting future values of a time-series even when only a small set of historic time-series is available. Few-shot forecasting has applications in domains where a long history of data is not available. This work describes deep neural network architectures for few-shot forecasting. All the architectures use a Siamese twin network approach to learn a difference function between pairs of time-series, rather than directly forecasting based on historical data as seen in traditional forecasting models. The networks are built using Long short-term memory units (LSTM). During forecasting, a model is able to forecast time-series types that were never seen in the training data by using the few available instances of the new time-series type as reference inputs. The proposed architectures are evaluated on Vehicular traffic data collected in California from the Caltrans Performance Measurement System (PeMS). The models were trained with traffic flow data collected at specific locations and then are evaluated by predicting traffic at different locations at different time horizons (0 to 12 hours). The Mean Absolute Error (MAE) was used as the evaluation metric and also as the loss function for training. The proposed architectures show lower prediction error than a baseline nearest neighbor forecast model. The prediction error increases at longer time horizons.
more »
« less
This content will become publicly available on August 6, 2026
zSHIFT: A Siamese Hierarchical Transformer Network for Zero Shot Time Series Forecasting
Zero shot time series forecasting is the challenge of forecasting future values of a time dependent sequence without having access to any historical data from the target series during model training. This setting differs from the traditional domain of time series forecasting, where models are typically trained using large volumes of historical data, from the same distribution. Zero shot time series forecasting models are designed to generalize to unseen time series by leveraging their knowledge learned from other, similar series during training. This work proposes two architectures designed for zero shot time series forecasting: zSiFT and zSHiFT. Both architectures use transformer models arranged in a Siamese network configuration. The zSHiFT architecture differs from the zSiFT by the introduction of a hierarchical transformer component to the Siamese network. These architectures are evaluated on vehicular traffic data in California available from the Caltrans Performance Measurement System (PeMS). The models were trained with traffic flow data collected in one region of California and then are evaluated by forecasting traffic in other regions. Forecast accuracy was evaluated at different time horizons (4 to 48 hours). The zSiFT model achieves a Mean Absolute Error (MAE) that is 8.3% lower than the baseline LSTM with attention mechanism model. The zSiFT model achieves an MAE which is 6.6% lower than zSHiFT’s MAE.
more »
« less
- Award ID(s):
- 2125654
- PAR ID:
- 10647108
- Publisher / Repository:
- IEEE
- Date Published:
- Page Range / eLocation ID:
- 331 to 336
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Slope displacement forecasting is crucial for geotechnical early warning. Conventional deterministic models cannot adapt easily to real-time monitoring, and large deep-learning models demand extensive data and compute. We leverage IBM’s Tiny Time Mixer (TTM), a compact, MLP-Mixer inspired time-series model with specialized temporal and channel mixing that scales linearly with sequence length. TTM was applied to daily aggregated data from six instrumented highway embankment slopes, including soil moisture, matric suction and temperature at 1.5–5 m depths, plus rainfall. These inputs yielded 16-channel time series of roughly 950–998 points per slope, while the displacement target is available only at sparse measurement times, comprising approximately 36–38 inclinometer surveys per slope over ~ 2.75 years. A transfer-learning protocol was used; the model was sequentially fine-tuned on four slopes and evaluated zero-shot on two held-out slopes. On trained slopes, TTM forecasts proved highly accurate. For example, on the active Terry Road slope the fine-tuned model gave MAE ≈ 0.394 mm (R2=0.99), and on the I-220 N Ramp MAE ≈ 0.257 mm (R2=0.97). Even on more stable slopes, absolute errors were sub-millimetric (e.g. MAE ≈ 0.005–0.023 mm, with R2=0.84 and 0.72). Crucially, in zero-shot tests the model maintained high accuracy on unseen sites: the Sowell Road slope forecast had MAE = 0.132 mm (R2=0.99) and McRaven Road MAE = 0.203 mm (R2=0.96). These results indicate that learned precipitation–displacement relationships transfer robustly across slopes. Overall, this study shows that lightweight, efficient time-series models like TTM can deliver accurate, real-time slope forecasts for early warning in resource-constrained geotechnical monitoring applications.more » « less
-
Time-series forecasting is a challenging problem that traditionally requires specialized models custom-trained for the specific task at hand. Recently, inspired by the success of large language models, foundation models pre-trained on vast amounts of time-series data from diverse domains have emerged as a promising candidate for general-purpose time-series forecasting. The defining characteristic of these foundation models is their ability to perform zero-shot learning, that is, forecasting a new system from limited context data without explicit re-training or fine-tuning. Here, we evaluate whether the zero-shot learning paradigm extends to the challenging task of forecasting chaotic systems. Across 135 distinct chaotic dynamical systems and 108 timepoints, we find that foundation models produce competitive forecasts compared to custom-trained models (including NBEATS, TiDE, etc.), particularly when training data is limited. Interestingly, even after point forecasts fail, large foundation models are able to preserve the geometric and statistical properties of the chaotic attractors. We attribute this success to foundation models’ ability to perform in-context learning and identify context parroting as a simple mechanism used by these models to capture the long-term behavior of chaotic dynamical systems. Our results highlight the potential of foundation models as a tool for probing nonlinear and complex systems.more » « less
-
na (Ed.)Considering the difficulty of financial time series forecasting in financial aid, much of the current research focuses on leveraging big data analytics in financial services. One modern approach is to utilize ”predictive analysis”, analogous to forecasting financial trends. However, many of these time series data in Financial Aid (FA) pose unique challenges due to limited historical datasets and high dimensional financial information, which hinder the development of effective predictive models that balance accuracy with efficient runtime and memory usage. Pre-trained foundation models are employed to address these challenging tasks. We use state-of-the-art time series models including pre-trained LLMs (GPT-2 as the backbone), transformers, and linear models to demonstrate their ability to outperform traditional approaches, even with minimal (”few-shot”) or no fine-tuning (”zero-shot”). Our benchmark study, which includes financial aid with seven other time series tasks, shows the potential of using LLMs for scarce financial datasets.more » « less
-
Wind energy technologies, including advanced turbine management and scheduling, rely on accurate wind power forecasting (WPF) for optimal operation. Enhancing forecast precision is crucial for reducing volatility in wind power integration and improving forecasting reliability. While time series forecasting methods estimate future values from historical data, traditional approaches often struggle with computational efficiency and model complexity. To address these challenges, we propose a hybrid forecasting model that integrates multi- variate estimation (MVE) and pure time series prediction (TSP) using a bidirectional long-short-term memory network (Bi-LSTM) optimized with Tree-structured Parzen Estimator (TPE)- based Bayesian optimization. The model incorporates numerical weather prediction (NWP) data for real-time forecasting, addressing a key limitation in existing methods. MVE utilizes meteorological features such as wind speed, direction, temperature, and pressure, while TSP captures historical power generation patterns. The TPE-optimized Bi-LSTM architecture effectively captures bidirectional temporal dependencies, improving accuracy in both short-term and long-term forecasting. The model is evaluated using a six-year historical wind energy dataset from NREL, with performance assessed through RMSE, MAE, and R2 score. It outperforms traditional LSTM variants (Vanilla LSTM, Stacked LSTM, Bi-LSTM) and state-of-the-art models such as Transformers and GRUs, achieving R2 scores of 0.976 for MVE and 0.932, 0.928, and 0.864 for TSP across short-term, day- ahead, and long-term forecasting, respectively. Additionally, TPE based Bayesian optimization reduces computational time around 8%–10%, enhancing hyperparameter tuning efficiency. The study further analyzes the model’s computational burden, scalability, and practical implementation, offering a robust and efficient approach for improving wind power forecasting accuracy. While the model achieves high accuracy on the NREL dataset, future work will validate its performance across diverse geographic regions (e.g., Europe, Asia) and turbine configurations.more » « less
