More Like this

1. Neural Manifold Ordinary Differential Equations

   Lou, Aaron; Lim, Derek; Katsman, Isay; Huang, Leo; Jiang, Qingxuan; Lim, Ser Nam (January 2021, 2020 Advances in Neural Information Processing Systems (NeurIPS 2020))

   Larochelle, Hugo; Ranzato, Marc'Aurelio; Hadsell, Raia; Balcan, Maria-Florina; Lin, Hsuan-Tien (Ed.)

   To better conform to data geometry, recent deep generative modelling techniques adapt Euclidean constructions to non-Euclidean spaces. In this paper, we study normalizing flows on manifolds. Previous work has developed flow models for specific cases; however, these advancements hand craft layers on a manifold-by-manifold basis, restricting generality and inducing cumbersome design constraints. We overcome these issues by introducing Neural Manifold Ordinary Differential Equations, a manifold generalization of Neural ODEs, which enables the construction of Manifold Continuous Normalizing Flows (MCNFs). MCNFs require only local geometry (therefore generalizing to arbitrary manifolds) and compute probabilities with continuous change of variables (allowing for a simple and expressive flow construction). We find that leveraging continuous manifold dynamics produces a marked improvement for both density estimation and downstream tasks. 

   more » « less
2. Survival Analysis via Ordinary Differential Equations

   https://doi.org/10.1080/01621459.2022.2051519  

   Tang, Weijing; He, Kevin; Xu, Gongjun; Zhu, Ji (January 2022, Journal of the American Statistical Association)
3. Heavy Ball Neural Ordinary Differential Equations

   Xia, Hedi; Suliafu, Vai; Ji, Hangjie; Nguyen, Tan; Bertozzi, Andrea; Osher, Stanley; Wang, Bao. (January 2021, Advances in neural information processing systems)

   We propose heavy ball neural ordinary differential equations (HBNODEs), leveraging the continuous limit of the classical momentum accelerated gradient descent, to improve neural ODEs (NODEs) training and inference. HBNODEs have two properties that imply practical advantages over NODEs: (i) The adjoint state of an HBNODE also satisfies an HBNODE, accelerating both forward and backward ODE solvers, thus significantly reducing the number of function evaluations (NFEs) and improving the utility of the trained models. (ii) The spectrum of HBNODEs is well structured, enabling effective learning of long-term dependencies from complex sequential data. We verify the advantages of HBNODEs over NODEs on benchmark tasks, including image classification, learning complex dynamics, and sequential modeling. Our method requires remarkably fewer forward and backward NFEs, is more accurate, and learns long-term dependencies more effectively than the other ODE-based neural network models. Code is available at https://github.com/hedixia/HeavyBallNODE. 

   more » « less
4. Stochastic physics-informed neural ordinary differential equations

   https://doi.org/10.1016/j.jcp.2022.111466  

   O'Leary, Jared; Paulson, Joel A.; Mesbah, Ali (November 2022, Journal of Computational Physics)
5. Principal blocks with six ordinary irreducible characters

   https://doi.org/10.1016/j.jalgebra.2023.08.006  

   Hung, Nguyen N; Schaeffer_Fry, AA; Vallejo, Carolina (October 2024, Journal of Algebra)