More Like this

1. Differential Equations for Continuous-Time Deep Learning

   https://doi.org/10.1090/noti2930  

   Ruthotto, Lars (May 2024, Notices of the American Mathematical Society)

   Malek-Madani, Reza (Ed.)

   This short, self-contained article seeks to introduce and survey continuous-time deep learning approaches that are based on neural ordinary differential equations (neural ODEs). It primarily targets readers familiar with ordinary and partial differential equations and their analysis who are curious to see their role in machine learning. Using three examples from machine learning and applied mathematics, we will see how neural ODEs can provide new insights into deep learning and a foundation for more efficient algorithms. 

   more » « less
2. Diffeomorphic Image Registration using Lipschitz Continuous Residual Networks

   Joshi, Ankita; Hong, Yi (October 2022, Proceedings of Machine Learning Research)

   Image registration is an essential task in medical image analysis. We propose two novel unsupervised diffeomorphic image registration networks, which use deep Residual Networks (ResNets) as numerical approximations of the underlying continuous diffeomorphic setting governed by ordinary differential equations (ODEs), viewed as a Eulerian discretization scheme. While considering the ODE-based parameterizations of diffeomorphisms, we consider both stationary and non-stationary (time varying) velocity fields as the driving velocities to solve the ODEs, which give rise to our two proposed architectures for diffeomorphic registration. We also employ Lipschitz-continuity on the Residual Networks in both architectures to define the admissible Hilbert space of velocity fields as a Reproducing Kernel Hilbert Spaces (RKHS) and regularize the smoothness of the velocity fields. We apply both registration networks to align and segment the OASIS brain MRI dataset. Experimental results demonstrate that our models are computational efficient and achieve comparable registration results with a smoother deformation field. 

   more » « less
3. On a positive-preserving, energy-stable numerical scheme to mass-action kinetics with detailed balance

   https://doi.org/10.4310/CMS.250517004551  

   Liu, Chun; Wang, Cheng; Wang, Yiwei (January 2025, Communications in Mathematical Sciences)

   In this paper, we provide a detailed theoretical analysis of the numerical scheme introduced in [C. Liu, C. Wang, and Y. Wang, J. Comput. Phys., 436:110253, 2021] for the reaction kinetics of a class of chemical reaction networks that satisfies detailed balance condition. In contrast to conventional numerical approximations, which are typically constructed based on ordinary differential equations (ODEs) for the concentrations of all involved species, the scheme is developed using the equations of reaction trajectories, which can be viewed as a generalized gradient flow of a physically relevant free energy. The unique solvability, positivity-preserving, and energy-stable properties are proved for the general case involving multiple reactions, under a mild condition on the stoichiometric matrix. 

   more » « less
4. Two-Stage Approach to Parameter Estimation of Differential Equations Using Neural ODEs

   https://doi.org/10.1021/acs.iecr.1c00552  

   Bradley, William; Boukouvala, Fani (November 2021, Industrial engineering chemistry research)

   Modeling physiochemical relationships using dynamic data is a common task in fields throughout science and engineering. A common step in developing generalizable, mechanistic models is to fit unmeasured parameters to measured data. However, fitting differential equation-based models can be computation-intensive and uncertain due to the presence of nonlinearity, noise, and sparsity in the data, which in turn causes convergence to local minima and divergence issues. This work proposes a merger of machine learning (ML) and mechanistic approaches by employing ML models as a means to fit nonlinear mechanistic ordinary differential equations (ODEs). Using a two-stage indirect approach, neural ODEs are used to estimate state derivatives, which are then used to estimate the parameters of a more interpretable mechanistic ODE model. In addition to its computational efficiency, the proposed method demonstrates the ability of neural ODEs to better estimate derivative information than interpolating methods based on algebraic data-driven models. Most notably, the proposed method is shown to yield accurate predictions even when little information is known about the parameters of the ODEs. The proposed parameter estimation approach is believed to be most advantageous when the ODE to be fit is strongly nonlinear with respect to its unknown parameters. 

   more » « less
5. Using SOS and Sublevel Set Volume Minimization for Estimation of Forward Reachable Sets

   Jones, M.; Peet, M. (January 2019, IFAC proceedings series)

   In this paper we propose a convex Sum-of-Squares optimization problem for finding outer approximations of forward reachable sets for nonlinear uncertain Ordinary Differential Equations (ODE’s) with either (or both) L2 or point-wise bounded input disturbances. To make our approximations tight we seek to minimize the volume of our approximation set. Our approach to volume minimization is based on the use of a convex determinant-like objective function.We provide several numerical examples including the Lorenz system and the Van der Pol oscillator. 

   more » « less