More Like this

1. Enhancing Urban Flow Maps via Neural ODEs

   https://doi.org/10.24963/ijcai.2020/180  

   Zhou, Fan; Li, Liang; Zhong, Ting; Trajcevski, Goce; Zhang, Kunpeng; Wang, Jiahao (July 2020, Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, {IJCAI} 2020)

   null (Ed.)

   Flow super-resolution (FSR) enables inferring fine-grained urban flows with coarse-grained observations and plays an important role in traffic monitoring and prediction. The existing FSR solutions rely on deep CNN models (e.g., ResNet) for learning spatial correlation, incurring excessive memory cost and numerous parameter updates. We propose to tackle the urban flows inference using dynamic systems paradigm and present a new method FODE -- FSR with Ordinary Differential Equations (ODEs). FODE extends neural ODEs by introducing an affine coupling layer to overcome the problem of numerically unstable gradient computation, which allows more accurate and efficient spatial correlation estimation, without extra memory cost. In addition, FODE provides a flexible balance between flow inference accuracy and computational efficiency. A FODE-based augmented normalization mechanism is further introduced to constrain the flow distribution with the influence of external factors. Experimental evaluations on two real-world datasets demonstrate that FODE significantly outperforms several baseline approaches. 

   more » « less
2. Multiwavelet-based operator learning for differential equations

   Gaurav Gupta, Xiongye Xiao (December 2021, Advances in neural information processing systems)

   The solution of a partial differential equation can be obtained by computing the inverse operator map between the input and the solution space. Towards this end, we introduce a multiwavelet-based neural operator learning scheme that compresses the associated operator's kernel using fine-grained wavelets. By explicitly embedding the inverse multiwavelet filters, we learn the projection of the kernel onto fixed multiwavelet polynomial bases. The projected kernel is trained at multiple scales derived from using repeated computation of multiwavelet transform. This allows learning the complex dependencies at various scales and results in a resolution-independent scheme. Compare to the prior works, we exploit the fundamental properties of the operator's kernel which enable numerically efficient representation. We perform experiments on the Korteweg-de Vries (KdV) equation, Burgers' equation, Darcy Flow, and Navier-Stokes equation. Compared with the existing neural operator approaches, our model shows significantly higher accuracy and achieves state-of-the-art in a range of datasets. For the time-varying equations, the proposed method exhibits a (2X−10X) improvement (0.0018 (0.0033) relative L2 error for Burgers' (KdV) equation). By learning the mappings between function spaces, the proposed method has the ability to find the solution of a high-resolution input after learning from lower-resolution data. 

   more » « less
3. Multiwavelet-based Operator Learning for Differential Equations

   Gaurav Gupta, Xiongye Xiao (December 2021, Advances in neural information processing systems)

   The solution of a partial differential equation can be obtained by computing the inverse operator map between the input and the solution space. Towards this end, we introduce a multiwavelet-based neural operator learning scheme that compresses the associated operator's kernel using fine-grained wavelets. By explicitly embedding the inverse multiwavelet filters, we learn the projection of the kernel onto fixed multiwavelet polynomial bases. The projected kernel is trained at multiple scales derived from using repeated computation of multiwavelet transform. This allows learning the complex dependencies at various scales and results in a resolution-independent scheme. Compare to the prior works, we exploit the fundamental properties of the operator's kernel which enable numerically efficient representation. We perform experiments on the Korteweg-de Vries (KdV) equation, Burgers' equation, Darcy Flow, and Navier-Stokes equation. Compared with the existing neural operator approaches, our model shows significantly higher accuracy and achieves state-of-the-art in a range of datasets. For the time-varying equations, the proposed method exhibits a ( 2 X − 10 X ) improvement ( 0.0018 ( 0.0033 ) relative L 2 error for Burgers' (KdV) equation). By learning the mappings between function spaces, the proposed method has the ability to find the solution of a high-resolution input after learning from lower-resolution data. 

   more » « less
4. Spectral-Refiner: Accurate Fine-Tuning of Spatiotemporal Fourier Neural Operator for Turbulent Flows

   Cao, Shuhao; Brarda, Francesco; Li, Rui_Peng; Xi, Yuanzhe (April 2025, ICLR 2025)

   Recent advancements in operator-type neural networks have shown promising results in approximating the solutions of spatiotemporal Partial Differential Equations (PDEs). However, these neural networks often entail considerable training expenses, and may not always achieve the desired accuracy required in many scientific and engineering disciplines. In this paper, we propose a new learning framework to address these issues. A new spatiotemporal adaptation is proposed to generalize any Fourier Neural Operator (FNO) variant to learn maps between Bochner spaces, which can perform an arbitrary-length temporal super-resolution for the first time. To better exploit this capacity, a new paradigm is proposed to refine the commonly adopted end-to-end neural operator training and evaluations with the help from the wisdom from traditional numerical PDE theory and techniques. Specifically, in the learning problems for the turbulent flow modeled by the Navier-Stokes Equations (NSE), the proposed paradigm trains an FNO only for a few epochs. Then, only the newly proposed spatiotemporal spectral convolution layer is fine-tuned without the frequency truncation. The spectral fine-tuning loss function uses a negative Sobolev norm for the first time in operator learning, defined through a reliable functional-type a posteriori error estimator whose evaluation is exact thanks to the Parseval identity. Moreover, unlike the difficult nonconvex optimization problems in the end-to-end training, this fine-tuning loss is convex. Numerical experiments on commonly used NSE benchmarks demonstrate significant improvements in both computational efficiency and accuracy, compared to end-to-end evaluation and traditional numerical PDE solvers under certain conditions. The source code is publicly available at https://github.com/scaomath/torch-cfd. 

   more » « less
5. Finding Closure Models to Match the Time Evolution of Coarse Grained 2D Turbulence Flows Using Machine Learning

   https://doi.org/10.3390/fluids7050154  

   Chen, Xianyang; Lu, Jiacai; Tryggvason, Grétar (May 2022, Fluids)

   Machine learning is used to develop closure terms for coarse grained model of two-dimensional turbulent flow directly from the coarse grained data by ensuring that the coarse-grained flow evolves in the correct way, with no need for the exact form of the filters or an explicit expression of the subgrid terms. The closure terms are calculated to match the time evolution of the coarse field and related to the average flow using a Neural Network with a relatively simple structure. The time dependent coarse grained flow field is generated by filtering fully resolved results and the predicted coarse field evolution agrees well with the filtered results in terms of instantaneous vorticity field in the short term and statistical quantities (energy spectrum, structure function and enstropy) in the long term, both for the flow used to learn the closure terms and for flows not used for the learning. This work shows the potential of using data-driven method to predict the time evolution of the large scales, in a complex situation where the closure terms may not have an explicit expression and the original fully resolved field is not available. 

   more » « less