Reduced-order models (ROMs) have achieved a lot of success in reducing the computational cost of traditional numerical methods across many disciplines. In fluid dynamics, ROMs have been successful in providing efficient and relatively accurate solutions for the numerical simulation of laminar flows. For convection-dominated (e.g., turbulent) flows, however, standard ROMs generally yield inaccurate results, usually affected by spurious oscillations. Thus, ROMs are usually equipped with numerical stabilization or closure models in order to account for the effect of the discarded modes. The literature on ROM closures and stabilizations is large and growing fast. In this paper, instead of reviewing all the ROM closures and stabilizations, we took a more modest step and focused on one particular type of ROM closure and stabilization that is inspired by large eddy simulation (LES), a classical strategy in computational fluid dynamics (CFD). These ROMs, which we call LES-ROMs, are extremely easy to implement, very efficient, and accurate. Indeed, LES-ROMs are modular and generally require minimal modifications to standard (“legacy”) ROM formulations. Furthermore, the computational overhead of these modifications is minimal. Finally, carefully tuned LES-ROMs can accurately capture the average physical quantities of interest in challenging convection-dominated flows in science and engineering applications. LES-ROMs are constructed by leveraging spatial filtering, which is the same principle used to build classical LES models. This ensures a modeling consistency between LES-ROMs and the approaches that generated the data used to train them. It also “bridges” two distinct research fields (LES and ROMs) that have been disconnected until now. This paper is a review of LES-ROMs, with a particular focus on the LES concepts and models that enable the construction of LES-inspired ROMs and the bridging of LES and reduced-order modeling. This paper starts with a description of a versatile LES strategy called evolve–filter–relax (EFR) that has been successfully used as a full-order method for both incompressible and compressible convection-dominated flows. We present evidence of this success. We then show how the EFR strategy, and spatial filtering in general, can be leveraged to construct LES-ROMs (e.g., EFR-ROM). Several applications of LES-ROMs to the numerical simulation of incompressible and compressible convection-dominated flows are presented. Finally, we draw conclusions and outline several research directions and open questions in LES-ROM development. While we do not claim this review to be comprehensive, we certainly hope it serves as a brief and friendly introduction to this exciting research area, which we believe has a lot of potential in the practical numerical simulation of convection-dominated flows in science, engineering, and medicine.
Reduced-order modeling of fluid flows with transformers
Reduced-order modeling (ROM) of fluid flows has been an active area of research for several decades. The huge computational cost of direct numerical simulations has motivated researchers to develop more efficient alternative methods, such as ROMs and other surrogate models. Similar to many application areas, such as computer vision and language modeling, machine learning and data-driven methods have played an important role in the development of novel models for fluid dynamics. The transformer is one of the state-of-the-art deep learning architectures that has made several breakthroughs in many application areas of artificial intelligence in recent years, including but not limited to natural language processing, image processing, and video processing. In this work, we investigate the capability of this architecture in learning the dynamics of fluid flows in a ROM framework. We use a convolutional autoencoder as a dimensionality reduction mechanism and train a transformer model to learn the system's dynamics in the encoded state space. The model shows competitive results even for turbulent datasets.
more »
« less
- Award ID(s):
- 1953222
- PAR ID:
- 10464608
- Date Published:
- Journal Name:
- Physics of Fluids
- Volume:
- 35
- Issue:
- 5
- ISSN:
- 1070-6631
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
more » « less
-
null (Ed.)Reduced order models (ROMs) are computational models whose dimension is significantly lower than those obtained through classical numerical discretizations (e.g., finite element, finite difference, finite volume, or spectral methods). Thus, ROMs have been used to accelerate numerical simulations of many query problems, e.g., uncertainty quantification, control, and shape optimization. Projection-based ROMs have been particularly successful in the numerical simulation of fluid flows. In this brief survey, we summarize some recent ROM developments for the quasi-geostrophic equations (QGE) (also known as the barotropic vorticity equations), which are a simplified model for geophysical flows in which rotation plays a central role, such as wind-driven ocean circulation in mid-latitude ocean basins. Since the QGE represent a practical compromise between efficient numerical simulations of ocean flows and accurate representations of large scale ocean dynamics, these equations have often been used in the testing of new numerical methods for ocean flows. ROMs have also been tested on the QGE for various settings in order to understand their potential in efficient numerical simulations of ocean flows. In this paper, we survey the ROMs developed for the QGE in order to understand their potential in efficient numerical simulations of more complex ocean flows: We explain how classical numerical methods for the QGE are used to generate the ROM basis functions, we outline the main steps in the construction of projection-based ROMs (with a particular focus on the under-resolved regime, when the closure problem needs to be addressed), we illustrate the ROMs in the numerical simulation of the QGE for various settings, and we present several potential future research avenues in the ROM exploration of the QGE and more complex models of geophysical flows.more » « less
-
Many sign languages are bona fide natural languages with grammatical rules and lexicons hence can benefit from machine translation methods. Similarly, since sign language is a visual-spatial language, it can also benefit from computer vision methods for encoding it. With the advent of deep learning methods in recent years, significant advances have been made in natural language processing (specifically neural machine translation) and in computer vision methods (specifically image and video captioning). Researchers have therefore begun expanding these learning methods to sign language understanding. Sign language interpretation is especially challenging, because it involves a continuous visual-spatial modality where meaning is often derived based on context. The focus of this article, therefore, is to examine various deep learning–based methods for encoding sign language as inputs, and to analyze the efficacy of several machine translation methods, over three different sign language datasets. The goal is to determine which combinations are sufficiently robust for sign language translation without any gloss-based information. To understand the role of the different input features, we perform ablation studies over the model architectures (input features + neural translation models) for improved continuous sign language translation. These input features include body and finger joints, facial points, as well as vector representations/embeddings from convolutional neural networks. The machine translation models explored include several baseline sequence-to-sequence approaches, more complex and challenging networks using attention, reinforcement learning, and the transformer model. We implement the translation methods over multiple sign languages—German (GSL), American (ASL), and Chinese sign languages (CSL). From our analysis, the transformer model combined with input embeddings from ResNet50 or pose-based landmark features outperformed all the other sequence-to-sequence models by achieving higher BLEU2-BLEU4 scores when applied to the controlled and constrained GSL benchmark dataset. These combinations also showed significant promise on the other less controlled ASL and CSL datasets.more » « less
-
null (Ed.)With the widespread adoption of the Next Generation Science Standards (NGSS), science teachers and online learning environments face the challenge of evaluating students’ integration of different dimensions of science learning. Recent advances in representation learning in natural language processing have proven effective across many natural language processing tasks, but a rigorous evaluation of the relative merits of these methods for scoring complex constructed response formative assessments has not previously been carried out. We present a detailed empirical investigation of feature-based, recurrent neural network, and pre-trained transformer models on scoring content in real-world formative assessment data. We demonstrate that recent neural methods can rival or exceed the performance of feature-based methods. We also provide evidence that different classes of neural models take advantage of different learning cues, and pre-trained transformer models may be more robust to spurious, dataset-specific learning cues, better reflecting scoring rubrics.more » « less
-
With the widespread adoption of the Next Generation Science Standards (NGSS), science teachers and online learning environments face the challenge of evaluating students' integration of different dimensions of science learning. Recent advances in representation learning in natural language processing have proven effective across many natural language processing tasks, but a rigorous evaluation of the relative merits of these methods for scoring complex constructed response formative assessments has not previously been carried out. We present a detailed empirical investigation of feature-based, recurrent neural network, and pre-trained transformer models on scoring content in real-world formative assessment data. We demonstrate that recent neural methods can rival or exceed the performance of feature-based methods. We also provide evidence that different classes of neural models take advantage of different learning cues, and pre-trained transformer models may be more robust to spurious, dataset-specific learning cues, better reflecting scoring rubrics.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1063/5.0151515
