An official website of the United States government
Battery lifetime and reliability depend on accurate state-of-health (SOH) estimation, while complex degradation mechanisms and varying operating conditions strengthen this challenge. This study presents two physics-informed neural network (PINN) configurations, PINN-Parallel, and PINN-Series, designed to improve SOH prediction by combining an equivalent circuit model (ECM) with a long short-term memory (LSTM) network. PINN-Parallel process input data through parallel ECM and LSTM modules and combine their outputs for SOH estimation. On the other hand, the PINN-Series uses a sequential approach that feeds ECM-derived parameters into the LSTM network to supplement temporal data analysis with physics information. Both models utilize easily accessible voltage, current, and temperature data that match realistic battery monitoring constraints. Experimental evaluations show that PINN-Series outperforms the PINN-Parallel and the baseline LSTM model in accuracy and robustness. It also adapts well to different input conditions. This demonstrates that the simulated battery dynamic states from ECM increase the LSTM's ability to capture degradation patterns and improve the model's ability to explain complex battery behavior. However, the trade-off between the robustness and training efficiency of PINNs is also discussed. The research findings show the potential of PINN models (particularly the PINN-Series) in advancing battery management systems, but the required computational resources need to be considered.
more »
« less
Data-driven learning of chaotic dynamical systems using Discrete-Temporal Sobolev Networks
We introduce the Discrete-Temporal Sobolev Network (DTSN), a neural network loss function that assists dynamical system forecasting by minimizing variational differences between the network output and the training data via a temporal Sobolev norm. This approach is entirely data-driven, architecture agnostic, and does not require derivative information from the estimated system. The DTSN is particularly well suited to chaotic dynamical systems as it minimizes noise in the network output which is crucial for such sensitive systems. For our test cases we consider discrete approximations of the Lorenz-63 system and the Chua circuit. For the network architectures we use the Long Short-Term Memory (LSTM) and the Transformer. The performance of the DTSN is compared with the standard MSE loss for both architectures, as well as with the Physics Informed Neural Network (PINN) loss for the LSTM. The DTSN loss is shown to substantially improve accuracy for both architectures, while requiring less information than the PINN and without noticeably increasing computational time, thereby demonstrating its potential to improve neural network forecasting of dynamical systems.
more »
« less
- Award ID(s):
- 2220211
- PAR ID:
- 10527597
- Publisher / Repository:
- Elsevier Ltd
- Date Published:
- Journal Name:
- Neural networks
- ISSN:
- 1879-2782
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Few-shot machine learning attempts to predict outputs given only a very small number of training examples. The key idea behind most few-shot learning approaches is to pre-train the model with a large number of instances from a different but related class of data, classes for which a large number of instances are available for training. Few-shot learning has been most successfully demonstrated for classification problems using Siamese deep learning neural networks. Few-shot learning is less extensively applied to time-series forecasting. Few-shot forecasting is the task of predicting future values of a time-series even when only a small set of historic time-series is available. Few-shot forecasting has applications in domains where a long history of data is not available. This work describes deep neural network architectures for few-shot forecasting. All the architectures use a Siamese twin network approach to learn a difference function between pairs of time-series, rather than directly forecasting based on historical data as seen in traditional forecasting models. The networks are built using Long short-term memory units (LSTM). During forecasting, a model is able to forecast time-series types that were never seen in the training data by using the few available instances of the new time-series type as reference inputs. The proposed architectures are evaluated on Vehicular traffic data collected in California from the Caltrans Performance Measurement System (PeMS). The models were trained with traffic flow data collected at specific locations and then are evaluated by predicting traffic at different locations at different time horizons (0 to 12 hours). The Mean Absolute Error (MAE) was used as the evaluation metric and also as the loss function for training. The proposed architectures show lower prediction error than a baseline nearest neighbor forecast model. The prediction error increases at longer time horizons.more » « less
-
Accurately forecasting well-being may enable people to make desirable behavioral changes that could improve their future well-being. In this paper, we evaluate how well an automated model can forecast the next-day’s well-being (specifically focusing on stress, health, and happiness) from static models (support vector machine and logistic regression) and time-series models (long short-term memory neural network models (LSTM)) using the previous seven days of physiological, mobile phone, and behavioral survey data. We especially examine how using only a portion of the day’s data (e.g. just night-time, or just daytime) influences the forecasting accuracy. The results show that accuracy is improved, across every condition tested, by using an LSTM instead of using static models. We find that daytime-only physiology data from wearable sensors, using an LSTM, can provide an accurate forecast of tomorrow’s well-being using students’ daily life data (stress: 80.4%, health: 86.0%, and happiness: 79.1%), achieving the same accuracy as using data collected from around the clock. These findings are valuable steps toward developing a practical and convenient well-being forecasting system.more » « less
-
Abstract Predicting failure in solids has broad applications including earthquake prediction which remains an unattainable goal. However, recent machine learning work shows that laboratory earthquakes can be predicted using micro-failure events and temporal evolution of fault zone elastic properties. Remarkably, these results come from purely data-driven models trained with large datasets. Such data are equivalent to centuries of fault motion rendering application to tectonic faulting unclear. In addition, the underlying physics of such predictions is poorly understood. Here, we address scalability using a novel Physics-Informed Neural Network (PINN). Our model encodes fault physics in the deep learning loss function using time-lapse ultrasonic data. PINN models outperform data-driven models and significantly improve transfer learning for small training datasets and conditions outside those used in training. Our work suggests that PINN offers a promising path for machine learning-based failure prediction and, ultimately for improving our understanding of earthquake physics and prediction.more » « less
-
Abstract Long‐lead forecasting for spatio‐temporal systems can entail complex nonlinear dynamics that are difficult to specify a priori. Current statistical methodologies for modeling these processes are often highly parameterized and, thus, challenging to implement from a computational perspective. One potential parsimonious solution to this problem is a method from the dynamical systems and engineering literature referred to as an echo state network (ESN). ESN models usereservoir computingto efficiently compute recurrent neural network forecasts. Moreover, multilevel (deep) hierarchical models have recently been shown to be successful at predicting high‐dimensional complex nonlinear processes, particularly those with multiple spatial and temporal scales of variability (such as those we often find in spatio‐temporal environmental data). Here, we introduce a deep ensemble ESN (D‐EESN) model. Despite the incorporation of a deep structure, the presented model is computationally efficient. We present two versions of this model for spatio‐temporal processes that produce forecasts and associated measures of uncertainty. The first approach utilizes a bootstrap ensemble framework, and the second is developed within a hierarchical Bayesian framework (BD‐EESN). This more general hierarchical Bayesian framework naturally accommodates non‐Gaussian data types and multiple levels of uncertainties. The methodology is first applied to a data set simulated from a novel non‐Gaussian multiscale Lorenz‐96 dynamical system simulation model and, then, to a long‐lead United States (U.S.) soil moisture forecasting application. Across both applications, the proposed methodology improves upon existing methods in terms of both forecast accuracy and quantifying uncertainty.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- The DOI is not currently available.
