skip to main content


Title: Supersonic turbulent flow simulation using a scalable parallel modal discontinuous Galerkin numerical method
Abstract

The scalability and efficiency of numerical methods on parallel computer architectures is of prime importance as we march towards exascale computing. Classical methods like finite difference schemes and finite volume methods have inherent roadblocks in their mathematical construction to achieve good scalability. These methods are popularly used to solve the Navier-Stokes equations for fluid flow simulations. The discontinuous Galerkin family of methods for solving continuum partial differential equations has shown promise in realizing parallel efficiency and scalability when approaching petascale computations. In this paper an explicit modal discontinuous Galerkin (DG) method utilizing Implicit Large Eddy Simulation (ILES) is proposed for unsteady turbulent flow simulations involving the three-dimensional Navier-Stokes equations. A study of the method was performed for the Taylor-Green vortex case at a Reynolds number ranging from 100 to 1600. The polynomial orderP = 2 (third order accurate) was found to closely match the Direct Navier-Stokes (DNS) results for all Reynolds numbers tested outside of Re = 1600, which had a normalized RMS error of 3.43 × 10−4in the dissipation rate for a 603element mesh. The scalability and performance study of the method was then conducted for a Reynolds number of 1600 for polynomials orders fromP = 2 toP = 6. The highest order polynomial that was tested (P = 6) was found to have the most efficient scalability using both the MPI and OpenMP implementations.

 
more » « less
NSF-PAR ID:
10153753
Author(s) / Creator(s):
; ; ; ;
Publisher / Repository:
Nature Publishing Group
Date Published:
Journal Name:
Scientific Reports
Volume:
9
Issue:
1
ISSN:
2045-2322
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract

    As the use of spectral/hpelement methods, and high-order finite element methods in general, continues to spread, community efforts to create efficient, optimized algorithms associated with fundamental high-order operations have grown. Core tasks such as solution expansion evaluation at quadrature points, stiffness and mass matrix generation, and matrix assembly have received tremendous attention. With the expansion of the types of problems to which high-order methods are applied, and correspondingly the growth in types of numerical tasks accomplished through high-order methods, the number and types of these core operations broaden. This work focuses on solution expansion evaluation at arbitrary points within an element. This operation is core to many postprocessing applications such as evaluation of streamlines and pathlines, as well as to field projection techniques such as mortaring. We expand barycentric interpolation techniques developed on an interval to 2D (triangles and quadrilaterals) and 3D (tetrahedra, prisms, pyramids, and hexahedra) spectral/hpelement methods. We provide efficient algorithms for their implementations, and demonstrate their effectiveness using the spectral/hpelement libraryNektar++by running a series of baseline evaluations against the ‘standard’ Lagrangian method, where an interpolation matrix is generated and matrix-multiplication applied to evaluate a point at a given location. We present results from a rigorous series of benchmarking tests for a variety of element shapes, polynomial orders and dimensions. We show that when the point of interest is to be repeatedly evaluated, the barycentric method performs at worst$$50\%$$50%slower, when compared to a cached matrix evaluation. However, when the point of interest changes repeatedly so that the interpolation matrix must be regenerated in the ‘standard’ approach, the barycentric method yields far greater performance, with a minimum speedup factor of$$7\times $$7×. Furthermore, when derivatives of the solution evaluation are also required, the barycentric method in general slightly outperforms the cached interpolation matrix method across all elements and orders, with an up to$$30\%$$30%speedup. Finally we investigate a real-world example of scalar transport using a non-conformal discontinuous Galerkin simulation, in which we observe around$$6\times $$6×speedup in computational time for the barycentric method compared to the matrix-based approach. We also explore the complexity of both interpolation methods and show that the barycentric interpolation method requires$${\mathcal {O}}(k)$$O(k)storage compared to a best case space complexity of$${\mathcal {O}}(k^2)$$O(k2)for the Lagrangian interpolation matrix method.

     
    more » « less
  2. We present a fully-coupled, implicit-in-time framework for solving a thermodynamically-consistent Cahn-Hilliard Navier-Stokes system that models two-phase flows. In this work, we extend the block iterative method presented in Khanwale et al. [Simulating two-phase flows with thermodynamically consistent energy stable Cahn-Hilliard Navier-Stokes equations on parallel adaptive octree based meshes, J. Comput. Phys. (2020)], to a fully-coupled, provably second-order accurate scheme in time, while maintaining energy-stability. The new method requires fewer matrix assemblies in each Newton iteration resulting in faster solution time. The method is based on a fully-implicit Crank-Nicolson scheme in time and a pressure stabilization for an equal order Galerkin formulation. That is, we use a conforming continuous Galerkin (cG) finite element method in space equipped with a residual-based variational multiscale (RBVMS) procedure to stabilize the pressure. We deploy this approach on a massively parallel numerical implementation using parallel octree-based adaptive meshes. We present comprehensive numerical experiments showing detailed comparisons with results from the literature for canonical cases, including the single bubble rise, Rayleigh-Taylor instability, and lid-driven cavity flow problems. We analyze in detail the scaling of our numerical implementation. 
    more » « less
  3. Abstract

    A new discontinuous Galerkin finite element method for the Stokes equations is developed in the primary velocity‐pressure formulation. This method employs discontinuous polynomials for both velocity and pressure on general polygonal/polyhedral meshes. Most finite element methods with discontinuous approximation have one or more stabilizing terms for velocity and for pressure to guarantee stability and convergence. This new finite element method has the standard conforming finite element formulation, without any velocity or pressure stabilizers. Optimal‐order error estimates are established for the corresponding numerical approximation in various norms. The numerical examples are tested for low and high order elements up to the degree four in 2D and 3D spaces.

     
    more » « less
  4. This work discusses the development, verification and performance assessment of a discontinuous Galerkin solver for the compressible Navier-Stokes equations using the Legion programming system. This is motivated by (i) the potential of this family of high-order numer- ical methods to accurately and efficiently realize scale-resolving simulations on unstructured grids and (ii) the desire to accommodate the utilization of emerging compute platforms that exhibit increased parallelism and heterogeneity. As a task-based programming model specifically designed for performance portability across distributed heterogeneous architectures, Legion represents an interesting lternative to the traditional approach of using Message Passing Interface for massively parallel computational physics solvers. Following detailed discussion of the implementation, the high-order convergence of the solver is demonstrated by a suite of canonical test cases and good strong scaling behavior is obtained. This work constitutes a first step towards a research platform that is able to be deployed and efficiently run on modern supercomputers. 
    more » « less
  5. We perform direct numerical simulations of a gas bubble dissolving in a surrounding liquid. The bubble volume is reduced due to dissolution of the gas, with the numerical implementation of an immersed boundary method, coupling the gas diffusion and the Navier–Stokes equations. The methods are validated against planar and spherical geometries’ analytical moving boundary problems, including the classic Epstein–Plesset problem. Considering a bubble rising in a quiescent liquid, we show that the mass transfer coefficient $k_L$ can be described by the classic Levich formula $k_L = (2/\sqrt {{\rm \pi} })\sqrt {\mathscr {D}_l\,U(t)/d(t)}$ , with $d(t)$ and $U(t)$ the time-varying bubble size and rise velocity, and $\mathscr {D}_l$ the gas diffusivity in the liquid. Next, we investigate the dissolution and gas transfer of a bubble in homogeneous and isotropic turbulence flow, extending Farsoiya et al. ( J. Fluid Mech. , vol. 920, 2021, A34). We show that with a bubble size initially within the turbulent inertial subrange, the mass transfer coefficient in turbulence $k_L$ is controlled by the smallest scales of the flow, the Kolmogorov $\eta$ and Batchelor $\eta _B$ microscales, and is independent of the bubble size. This leads to the non-dimensional transfer rate ${Sh}=k_L L^\star /\mathscr {D}_l$ scaling as ${Sh}/{Sc}^{1/2} \propto {Re}^{3/4}$ , where ${Re}$ is the macroscale Reynolds number ${Re} = u_{rms}L^\star /\nu _l$ , with $u_{rms}$ the velocity fluctuations, $L^*$ the integral length scale, $\nu _l$ the liquid viscosity, and ${Sc}=\nu _l/\mathscr {D}_l$ the Schmidt number. This scaling can be expressed in terms of the turbulence dissipation rate $\epsilon$ as ${k_L}\propto {Sc}^{-1/2} (\epsilon \nu _l)^{1/4}$ , in agreement with the model proposed by Lamont & Scott ( AIChE J. , vol. 16, issue 4, 1970, pp. 513–519) and corresponding to the high $Re$ regime from Theofanous et al. ( Intl J. Heat Mass Transfer , vol. 19, issue 6, 1976, pp. 613–624). 
    more » « less