skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: Symbolic regression via neural networks
Identifying governing equations for a dynamical system is a topic of critical interest across an array of disciplines, from mathematics to engineering to biology. Machine learning—specifically deep learning—techniques have shown their capabilities in approximating dynamics from data, but a shortcoming of traditional deep learning is that there is little insight into the underlying mapping beyond its numerical output for a given input. This limits their utility in analysis beyond simple prediction. Simultaneously, a number of strategies exist which identify models based on a fixed dictionary of basis functions, but most either require some intuition or insight about the system, or are susceptible to overfitting or a lack of parsimony. Here, we present a novel approach that combines the flexibility and accuracy of deep learning approaches with the utility of symbolic solutions: a deep neural network that generates a symbolic expression for the governing equations. We first describe the architecture for our model and then show the accuracy of our algorithm across a range of classical dynamical systems.  more » « less
Award ID(s):
2016004
PAR ID:
10585170
Author(s) / Creator(s):
; ;
Publisher / Repository:
American Institute of Physics
Date Published:
Journal Name:
Chaos: An Interdisciplinary Journal of Nonlinear Science
Volume:
33
Issue:
8
ISSN:
1054-1500
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; (, Communications Physics)
    Abstract Identifying the governing equations of a nonlinear dynamical system is key to both understanding the physical features of the system and constructing an accurate model of the dynamics that generalizes well beyond the available data. Achieving this kind of interpretable system identification is even more difficult for partially observed systems. We propose a machine learning framework for discovering the governing equations of a dynamical system using only partial observations, combining an encoder for state reconstruction with a sparse symbolic model. The entire architecture is trained end-to-end by matching the higher-order symbolic time derivatives of the sparse symbolic model with finite difference estimates from the data. Our tests show that this method can successfully reconstruct the full system state and identify the equations of motion governing the underlying dynamics for a variety of ordinary differential equation (ODE) and partial differential equation (PDE) systems. 
    more » « less
  2. ; ; ; ; (, Communications Physics)
    Abstract Traditional data-driven deep learning models often struggle with high training costs, error accumulation, and poor generalizability in complex physical processes. Physics-informed deep learning (PiDL) addresses these challenges by incorporating physical principles into the model. Most PiDL approaches regularize training by embedding governing equations into the loss function, yet this depends heavily on extensive hyperparameter tuning to weigh each loss term. To this end, we propose to leverage physics prior knowledge by “baking” the discretized governing equations into the neural network architecture via the connection between the partial differential equations (PDE) operators and network structures, resulting in a PDE-preserved neural network (PPNN). This method, embedding discretized PDEs through convolutional residual networks in a multi-resolution setting, largely improves the generalizability and long-term prediction accuracy, outperforming conventional black-box models. The effectiveness and merit of the proposed methods have been demonstrated across various spatiotemporal dynamical systems governed by spatiotemporal PDEs, including reaction-diffusion, Burgers’, and Navier-Stokes equations. 
    more » « less
  3. ; (, Renewable and Sustainable Energy Reviews)
    As power systems evolve with the increasing integration of renewable energy sources and smart grid technologies, there is a growing demand for flexible and scalable modeling approaches capable of capturing the complex dynamics of modern grids. This review focuses on symbolic regression, a powerful methodology for deriving parsimonious and interpretable mathematical models directly from data. Symbolic regression is particularly valuable for power systems due to its ability to uncover governing equations without prior structural assumptions, enabling transparent and data-driven insights into nonlinear system behavior. The paper presents a comprehensive overview of symbolic regression methods, including sparse identification of nonlinear dynamics, automatic regression for governing equations, and deep symbolic regression, highlighting their applications in power systems. Through comparative case studies of the single machine infinite bus system, grid-following, and grid-forming inverters, we analyze the strengths, limitations, and suitability of each symbolic regression method in modeling nonlinear power system dynamics. Additionally, we identify critical research gaps and discuss future directions for leveraging symbolic regression in the optimization, control, and operation of modern power grids. This review aims to provide a valuable resource for researchers and engineers seeking innovative, data-driven solutions for modeling in the context of evolving power system infrastructure. 
    more » « less
  4. ; ; ; ; (, Proceedings of Machine Learning Research)
    Pappas, George; Ravikumar, Pradeep; Seshia, Sanjit A (Ed.)
    We study the problem of learning neural network models for Ordinary Differential Equations (ODEs) with parametric uncertainties. Such neural network models capture the solution to the ODE over a given set of parameters, initial conditions, and range of times. Physics-Informed Neural Networks (PINNs) have emerged as a promising approach for learning such models that combine data-driven deep learning with symbolic physics models in a principled manner. However, the accuracy of PINNs degrade when they are used to solve an entire family of initial value problems characterized by varying parameters and initial conditions. In this paper, we combine symbolic differentiation and Taylor series methods to propose a class of higher-order models for capturing the solutions to ODEs. These models combine neural networks and symbolic terms: they use higher order Lie derivatives and a Taylor series expansion obtained symbolically, with the remainder term modeled as a neural network. The key insight is that the remainder term can itself be modeled as a solution to a first-order ODE. We show how the use of these higher order PINNs can improve accuracy using interesting, but challenging ODE benchmarks. We also show that the resulting model can be quite useful for situations such as controlling uncertain physical systems modeled as ODEs. 
    more » « less
  5. ; ; (, Advanced Theory and Simulations)
    Abstract The fast simulation of dynamical systems is a key challenge in many scientific and engineering applications, such as weather forecasting, disease control, and drug discovery. With the recent success of deep learning, there is increasing interest in using neural networks to solve differential equations in a data‐driven manner. However, existing methods are either limited to specific types of differential equations or require large amounts of data for training. This restricts their practicality in many real‐world applications, where data is often scarce or expensive to obtain. To address this, a novel multi‐modal foundation model, namedFMint(FoundationModel based onInitialization) is proposed, to bridge the gap between human‐designed and data‐driven models for the fast simulation of dynamical systems. Built on a decoder‐only transformer architecture with in‐context learning, FMint utilizes both numerical and textual data to learn a universal error correction scheme for dynamical systems, using prompted sequences of coarse solutions from traditional solvers. The model is pre‐trained on a corpus of 400K ordinary differential equations (ODEs), and extensive experiments are performed on challenging ODEs that exhibit chaotic behavior and of high dimensionality. The results demonstrate the effectiveness of the proposed model in terms of both accuracy and efficiency compared to classical numerical solvers, highlighting FMint's potential as a general‐purpose solver for dynamical systems. This approach achieves an accuracy improvement of 1 to 2 orders of magnitude over state‐of‐the‐art dynamical system simulators, and delivers a 5X speedup compared to traditional numerical algorithms. The code for FMint is available athttps://github.com/margotyjx/FMint. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email