skip to main content

Title: Physics guided RNNs for modeling dynamical systems: A case study in simulating lake temperature profiles
This paper proposes a physics-guided recurrent neural network model (PGRNN) that combines RNNs and physics-based models to leverage their complementary strengths and improve the modeling of physical processes. Specifically, we show that a PGRNN can improve prediction accuracy over that of physical models, while generating outputs consistent with physical laws, and achieving good generalizability. Standard RNNs, even when producing superior prediction accuracy, often produce physically inconsistent results and lack generalizability. We further enhance this approach by using a pre-training method that leverages the simulated data from a physics-based model to address the scarcity of observed data. Although we present and evaluate this methodology in the context of modeling the dynamics of temperature in lakes, it is applicable more widely to a range of scientific and engineering disciplines where mechanistic (also known as process-based) models are used, e.g., power engineering, climate science, materials science, computational chemistry, and biomedicine.
; ; ; ; ; ;
Award ID(s):
Publication Date:
Journal Name:
Proceedings of the 2019 SIAM International Conference on Data Mining
Page Range or eLocation-ID:
Sponsoring Org:
National Science Foundation
More Like this
  1. Physics-based models are often used to study engineering and environmental systems. The ability to model these systems is the key to achieving our future environmental sustainability and improving the quality of human life. This article focuses on simulating lake water temperature, which is critical for understanding the impact of changing climate on aquatic ecosystems and assisting in aquatic resource management decisions. General Lake Model (GLM) is a state-of-the-art physics-based model used for addressing such problems. However, like other physics-based models used for studying scientific and engineering systems, it has several well-known limitations due to simplified representations of the physical processesmore »being modeled or challenges in selecting appropriate parameters. While state-of-the-art machine learning models can sometimes outperform physics-based models given ample amount of training data, they can produce results that are physically inconsistent. This article proposes a physics-guided recurrent neural network model (PGRNN) that combines RNNs and physics-based models to leverage their complementary strengths and improves the modeling of physical processes. Specifically, we show that a PGRNN can improve prediction accuracy over that of physics-based models (by over 20% even with very little training data), while generating outputs consistent with physical laws. An important aspect of our PGRNN approach lies in its ability to incorporate the knowledge encoded in physics-based models. This allows training the PGRNN model using very few true observed data while also ensuring high prediction accuracy. Although we present and evaluate this methodology in the context of modeling the dynamics of temperature in lakes, it is applicable more widely to a range of scientific and engineering disciplines where physics-based (also known as mechanistic) models are used.« less
  2. The prevalence of mobile phones and wearable devices enables the passive capturing and modeling of human behavior at an unprecedented resolution and scale. Past research has demonstrated the capability of mobile sensing to model aspects of physical health, mental health, education, and work performance, etc. However, most of the algorithms and models proposed in previous work follow a one-size-fits-all (i.e., population modeling) approach that looks for common behaviors amongst all users, disregarding the fact that individuals can behave very differently, resulting in reduced model performance. Further, black-box models are often used that do not allow for interpretability and human behaviormore »understanding. We present a new method to address the problems of personalized behavior classification and interpretability, and apply it to depression detection among college students. Inspired by the idea of collaborative-filtering, our method is a type of memory-based learning algorithm. It leverages the relevance of mobile-sensed behavior features among individuals to calculate personalized relevance weights, which are used to impute missing data and select features according to a specific modeling goal (e.g., whether the student has depressive symptoms) in different time epochs, i.e., times of the day and days of the week. It then compiles features from epochs using majority voting to obtain the final prediction. We apply our algorithm on a depression detection dataset collected from first-year college students with low data-missing rates and show that our method outperforms the state-of-the-art machine learning model by 5.1% in accuracy and 5.5% in F1 score. We further verify the pipeline-level generalizability of our approach by achieving similar results on a second dataset, with an average improvement of 3.4% across performance metrics. Beyond achieving better classification performance, our novel approach is further able to generate personalized interpretations of the models for each individual. These interpretations are supported by existing depression-related literature and can potentially inspire automated and personalized depression intervention design in the future« less
  3. Process-based numerical simulation, includ- ing for climate modeling applications, is compute- and resource-intensive, requiring extensive customization and hand-engineering for encoding governing equations and other domain knowledge. On the other hand, modern deep learning employs a much simplified and efficient computational workflow, and has been showing impres- sive results across myriad applications in computational sciences. In this work, we investigate the potential of deep generative learning models, specifically conditional Gen- erative Adversarial Networks (cGANs), to simulate the output of a physics-based model of the spatial distribution of the water content of mountain snowpack, or snow water equivalent (SWE). We show preliminarymore »results indicating that the cGANs model is able to learn map- pings between meteorological forcing (e.g., minimum and maximum temperature, wind speed, net radiation, and precipitation) and SWE output. Moreover, informing the model with simple domain-inspired physical constraints results in higher model accuracy, and lower training time. Thus Physics-Informed cGANs provide a means for fast and accurate SWE modeling that can have significant impact in a variety of applications (e.g., hydropower forecasting, agriculture, and water supply management).« less
  4. Demeniconi, Carlotta ; Davidson, Ian (Ed.)
    This paper proposes a physics-guided machine learning approach that combines machine learning models and physics-based models to improve the prediction of water flow and temperature in river networks. We first build a recurrent graph network model to capture the interactions among multiple segments in the river network. Then we transfer knowledge from physics-based models to guide the learning of the machine learning model. We also propose a new loss function that balances the performance over different river segments. We demonstrate the effectiveness of the proposed method in predicting temperature and streamflow in a subset of the Delaware River Basin. Inmore »particular, the proposed method has brought a 33%/14% accuracy improvement over the state-of-the-art physics-based model and 24%/14% over traditional machine learning models (e.g., LSTM) in temperature/streamflow prediction using very sparse (0.1%) training data. The proposed method has also been shown to produce better performance when generalized to different seasons or river segments with different streamflow ranges.« less
  5. Abstract Modeling and simulation for additive manufacturing (AM) are critical enablers for understanding process physics, conducting process planning and optimization, and streamlining qualification and certification. It is often the case that a suite of hierarchically linked (or coupled) simulation models is needed to achieve the above tasks, as the entirety of the complex physical phenomena relevant to the understanding of process-structure-property-performance relationships in the context of AM precludes the use of a single simulation framework. In this study using a Bayesian network approach, we address the important problem of conducting uncertainty quantification (UQ) analysis for multiple hierarchical models to establishmore »process-microstructure relationships in laser powder bed fusion (LPBF) AM. More significantly, we present the framework to calibrate and analyze simulation models that have experimentally unmeasurable variables, which are quantities of interest predicted by an upstream model and deemed necessary for the downstream model in the chain. We validate the framework using a case study on predicting the microstructure of a binary nickel-niobium alloy processed using LPBF as a function of processing parameters. Our framework is shown to be able to predict segregation of niobium with up to 94.3% prediction accuracy on test data.« less