An official website of the United States government
Machine and deep learning-based algorithms are the emerging approaches in addressing prediction problems in time series. These techniques have been shown to produce more accurate results than conventional regression-based modeling. It has been reported that artificial Recurrent Neural Networks (RNN) with memory, such as Long Short-Term Memory (LSTM), are superior compared to Autoregressive Integrated Moving Average (ARIMA) with a large margin. The LSTM-based models incorporate additional “gates” for the purpose of memorizing longer sequences of input data. The major question is that whether the gates incorporated in the LSTM architecture already offers a good prediction and whether additional training of data would be necessary to further improve the prediction. Bidirectional LSTMs (BiLSTMs) enable additional training by traversing the input data twice (i.e., 1) left-to-right, and 2) right-to-left). The research question of interest is then whether BiLSTM, with additional training capability, outperforms regular unidirectional LSTM. This paper reports a behavioral analysis and comparison of BiLSTM and LSTM models. The objective is to explore to what extend additional layers of training of data would be beneficial to tune the involved parameters. The results show that additional training of data and thus BiLSTM-based modeling offers better predictions than regular LSTM-based models. More specifically, it was observed that BiLSTM models provide better predictions compared to ARIMA and LSTM models. It was also observed that BiLSTM models reach the equilibrium much slower than LSTM-based models.
more »
« less
Comparative Analysis of Machine Learning and Statistical Methods for Aircraft Phase of Flight Prediction
Phase of flight (POF) prediction estimates the future state of aircraft along planned trajectories, allowing the prediction of potential conflicts as well as optimization of air space, controlled by the Federal Aviation Administration. In this paper, we present a study conducted to develop three different POF forecasting machine learning models and a statistical regression model using four-dimensional GPS and RADAR Track data from 57 flights provided by an En Route Computer System. The investigated machine learning models include Long Short-Term Memory Recurrent Neural Network (LSTM-RNN), Support Vector Machine (SVM), and Neural Ordinary Differential Equations (NODE). These were developed to forecast the horizontal and vertical POF of the current aircraft for the next time step. The results in this study indicate that LSTM-RNN models are more suitable for POF prediction than SVM and statistical regression models, with NODE being a promising model for future trajectory prediction research.
more »
« less
- Award ID(s):
- 2008690
- PAR ID:
- 10323039
- Date Published:
- Journal Name:
- International Conference on Research in Air Transportation 2020, 9th International Conference
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
null (Ed.)As the COVID-19 pandemic evolves, reliable prediction plays an important role in policymaking. The classical infectious disease model SEIR (susceptible-exposed-infectious-recovered) is a compact yet simplistic temporal model. The data-driven machine learning models such as RNN (recurrent neural networks) can suffer in case of limited time series data such as COVID-19. In this paper, we combine SEIR and RNN on a graph structure to develop a hybrid spatiotemporal model to achieve both accuracy and efficiency in training and forecasting. We introduce two features on the graph structure: node feature (local temporal infection trend) and edge feature (geographic neighbor effect). For node feature, we derive a discrete recursion (called I-equation) from SEIR so that gradient descend method applies readily to its optimization. For edge feature, we design an RNN model to capture the neighboring effect and regularize the landscape of loss function so that local minima are effective and robust for prediction. The resulting hybrid model (called IeRNN) improves the prediction accuracy on state-level COVID-19 new case data from the US, out-performing standard temporal models (RNN, SEIR, and ARIMA) in 1-day and 7-day ahead forecasting. Our model accommodates various degrees of reopening and provides potential outcomes for policymakers.more » « less
-
Accurate prediction of the uptake and translocation of emerging contaminants in plants has serious implications for assessing impacts on ecosystems and human health. However, traditional modeling approaches are not reliable in the prediction of transpiration stream concentration factor (TSCF) and root concentration factor (RCF). This study applied deep neural networks (DNN), recurrent neural networks (RNN), and long short-term memory (LSTM) to enhance the accuracy of predictive models for TSCF and RCF. The predictions and feature importance analysis were based on nine chemical properties and two plant root macromolecular compositions. The results indicated that deep learning models predict TSCF and RCF with improved accuracy compared to mechanistic models. The coefficient of determination (R^2) for the DNN, RNN, and LSTM models in predicting TSCF was 0.62, 0.67, and 0.56, respectively. The corresponding mean squared error (MSE) on the test set for the models was 0.055, 0.035, and 0.06, respectively. The R^2 for the DNN, RNN, and LSTM models in predicting RCF was 0.90, 0.91, and 0.84, respectively. The corresponding MSE for the models was 0.124, 0.071, and 0.126, respectively. The results of feature extraction using extreme gradient boosting underlined the importance of lipophilicity and root lipid fraction.more » « less
-
null (Ed.)Predicting workload behavior during execution is essential for dynamic resource optimization of processor systems. Early studies used simple prediction algorithms such as a history tables. More recently, researchers have applied advanced machine learning regression techniques. Workload prediction can be cast as a time series forecasting problem. Time series forecasting is an active research area with recent advances that have not been studied in the context of workload prediction. In this paper, we first perform a comparative study of representative time series forecasting techniques to predict the dynamic workload of applications running on a CPU. We adapt state-of-the-art matrix profile and dynamic linear models (DLMs) not previously applied to workload prediction and compare them against traditional SVM and LSTM models that have been popular for handling non-stationary data. We find that all time series forecasting models struggle to predict abrupt workload changes. These changes occur because workloads go through phases, where prior work has studied workload phase detection, classification and prediction. We propose a novel approach that combines time series forecasting with phase prediction. We process each phase as a separate time series and train one forecasting model per phase. At runtime, forecasts from phase-specific models are selected and combined based on the predicted phase behavior. We apply our approach to forecasting of SPEC workloads running on a state-of-the-art Intel machine. Our results show that an LSTM-based phase-aware predictor can forecast workload CPI with less than 8% mean absolute error while reducing CPI error by more than 12% on average compared to a non-phase-aware approach.more » « less
-
Digital twin is a vital enabling technology for smart manufacturing in the era of Industry 4.0. Digital twin effectively replicates its physical asset enabling easy visualization, smart decision-making and cognitive capability in the system. In this paper, a framework of dynamic data driven digital twin for complex engineering products was proposed. To illustrate the proposed framework, an example of health management on aircraft engines was studied. This framework models the digital twin by extracting information from the various sensors and Industry Internet of Things (IIoT) monitoring the remaining useful life (RUL) of an engine in both cyber and physical domains. Then, with sensor measurements selected from linear degradation models, a long short-term memory (LSTM) neural network is proposed to dynamically update the digital twin, which can estimate the most up-to-date RUL of the physical aircraft engine. Through comparison with other machine learning algorithms, including similarity based linear regression and feed forward neural network, on RUL modelling, this LSTM based dynamical data driven digital twin provides a promising tool to accurately replicate the health status of aircraft engines. This digital twin based RUL technique can also be extended for health management and remote operation of manufacturing systems.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
