More Like this

1. Discovering Nonlinear Dynamics Through Scientific Machine Learning

   https://doi.org/10.1007/978-3-030-82193-7 17  

   Huang, Lei; Vrinceanu, Daniel; Wang, Yunjiao; Kulathunga, Nalinda; Ranasinghe, Nishath (October 2021, Lecture notes in networks and systems)

   null (Ed.)

   Scientific Machine Learning (SciML) is a new multidisciplinary methodology that combines the data-driven machine learning models and the principle-based computational models to improve the simulations of scientific phenomenon and uncover new scientific rules from existing measurements. This article reveals the experience of using the SciML method to discover the nonlinear dynamics that may be hard to model or be unknown in the real-world scenario. The SciML method solves the traditional principle-based differential equations by integrating a neural network to accurately model the nonlinear dynamics while respecting the scientific constraints and principles. The paper discusses the latest SciML models and apply them to the oscillator simulations and experiment. Besides better capacity to simulate, and match with the observation, the results also demonstrate a successful discovery of the hidden physics in the pendulum dynamics using SciML. 

   more » « less
2. Symbolic-numeric integration of univariate expressions based on sparse regression

   https://doi.org/10.1145/3572867.3572882  

   Iravanian, Shahriar; Martensen, Carl Julius; Cheli, Alessandro; Gowda, Shashi; Jain, Anand; Ma, Yingbo; Rackauckas, Chris (June 2022, ACM Communications in Computer Algebra)

   The majority of computer algebra systems (CAS) support symbolic integration using a combination of heuristic algebraic and rule-based (integration table) methods. In this paper, we present a hybrid (symbolic-numeric) method to calculate the indefinite integrals of univariate expressions. Our method is broadly similar to the Risch-Norman algorithm. The primary motivation for this work is to add symbolic integration functionality to a modern CAS (the symbolic manipulation packages of SciML, the Scientific Machine Learning ecosystem of the Julia programming language), which is designed for numerical and machine learning applications. The symbolic part of our method is based on the combination of candidate terms generation (ansatz generation using a methodology borrowed from the Homotopy operators theory) combined with rule-based expression transformations provided by the underlying CAS. The numeric part uses sparse regression, a component of the Sparse Identification of Nonlinear Dynamics (SINDy) technique, to find the coefficients of the candidate terms. We show that this system can solve a large variety of common integration problems using only a few dozen basic integration rules. 

   more » « less
3. An attention-based spatio-temporal neural operator for evolving physics

   https://doi.org/10.1088/2632-2153/ae1277  

   Karkaria, Vispi; Lee, Doksoo; Chen, Yi-Ping; Yu, Yue; Chen, Wei (November 2025, Machine Learning: Science and Technology)

   Abstract In scientific machine learning (SciML), a key challenge is learning unknown, evolving physical processes and making predictions across spatio-temporal scales. For example, in real-world manufacturing problems like additive manufacturing, users adjust known machine settings while unknown environmental parameters simultaneously fluctuate. To make reliable predictions, it is desired for a model to not only capture long-range spatio-temporal interactions from data but also adapt to new and unknown environments; traditional machine learning models excel at the first task but often lack physical interpretability and struggle to generalize under varying environmental conditions. To tackle these challenges, we propose the attention-based spatio-temporal neural operator (ASNO), a novel architecture that combines separable attention mechanisms for spatial and temporal interactions and adapts to unseen physical parameters. Inspired by the backward differentiation formula, ASNO learns a transformer for temporal prediction and extrapolation and an attention-based neural operator for handling varying external loads, enhancing interpretability by isolating historical state contributions and external forces, enabling the discovery of underlying physical laws and generalizability to unseen physical environments. Empirical results on SciML benchmarks demonstrate that ASNO outperforms existing models, establishing its potential for engineering applications, physics discovery, and interpretable machine learning. 

   more » « less
4. Accelerating Simulation of Stiff Nonlinear Systems using Continuous-Time Echo State Networks

   Anantharaman, Ranjan; Ma, Yingbo; Gowda, Shashi; Laughman, Chris; Shah, Viral; Edelman, Alan; Rackauckas, Chris (January 2021, Proceedings of the AAAI 2021 Spring Symposium on Combining Artificial Intelligence and Machine Learning with Physical Sciences)

   Lee, Jonghyun; Darve, Eric F.; Kitanidis, Peter K.; Mahoney, Michael W.; Karpatne, Anuj; Farthing, Matthew W.; Hesser, Tyler (Ed.)

   Modern design, control, and optimization often require multiple expensive simulations of highly nonlinear stiff models. These costs can be amortized by training a cheap surrogate of the full model, which can then be used repeatedly. Here we present a general data-driven method, the continuous time echo state network (CTESN), for generating surrogates of nonlinear ordinary differential equations with dynamics at widely separated timescales. We empirically demonstrate the ability to accelerate a physically motivated scalable model of a heating system by 98x while maintaining relative error of within 0.2 %. We showcase the ability for this surrogate to accurately handle highly stiff systems which have been shown to cause training failures with common surrogate methods such as Physics-Informed Neural Networks (PINNs), Long Short Term Memory (LSTM) networks, and discrete echo state networks (ESN). We show that our model captures fast transients as well as slow dynamics, while demonstrating that fixed time step machine learning techniques are unable to adequately capture the multi-rate behavior. Together this provides compelling evidence for the ability of CTESN surrogates to predict and accelerate highly stiff dynamical systems which are unable to be directly handled by previous scientific machine learning techniques. 

   more » « less
5. Panoramic Mapping of Phonon Transport from Ultrafast Electron Diffraction and Scientific Machine Learning

   https://doi.org/10.1002/adma.202206997  

   Chen, Zhantao; Shen, Xiaozhe; Andrejevic, Nina; Liu, Tongtong; Luo, Duan; Nguyen, Thanh; Drucker, Nathan_C; Kozina, Michael_E; Song, Qichen; Hua, Chengyun; et al (November 2022, Advanced Materials)

   Abstract One central challenge in understanding phonon thermal transport is a lack of experimental tools to investigate frequency‐resolved phonon transport. Although recent advances in computation lead to frequency‐resolved information, it is hindered by unknown defects in bulk regions and at interfaces. Here, a framework that can uncover microscopic phonon transport information in heterostructures is presented, integrating state‐of‐the‐art ultrafast electron diffraction (UED) with advanced scientific machine learning (SciML). Taking advantage of the dual temporal and reciprocal‐space resolution in UED, and the ability of SciML to solve inverse problems involving coupled Boltzmann transport equations, the frequency‐dependent interfacial transmittance and frequency‐dependent relaxation times of the heterostructure from the diffraction patterns are reliably recovered. The framework is applied to experimental Au/Si UED data, and a transport pattern beyond the diffuse mismatch model is revealed, which further enables a direct reconstruction of real‐space, real‐time, frequency‐resolved phonon dynamics across the interface. The work provides a new pathway to probe interfacial phonon transport mechanisms with unprecedented details. 

   more » « less