More Like this

1. Curl Constraint-Preserving Reconstruction and the Guidance it Gives for Mimetic Scheme Design

   https://doi.org/10.1007/s42967-021-00160-3  

   Balsara, Dinshaw S. ; Käppeli, Roger ; Boscheri, Walter ; Dumbser, Michael ( December 2021 , Communications on Applied Mathematics and Computation)

   Several important PDE systems, like magnetohydrodynamics and computational electrodynamics, are known to support involutions where the divergence of a vector field evolves in divergence-free or divergence constraint-preserving fashion. Recently, new classes of PDE systems have emerged for hyperelasticity, compressible multiphase flows, so-called first-order reductions of the Einstein field equations, or a novel first-order hyperbolic reformulation of Schrödinger’s equation, to name a few, where the involution in the PDE supports curl-free or curl constraint-preserving evolution of a vector field. We study the problem of curl constraint-preserving reconstruction as it pertains to the design of mimetic finite volume (FV) WENO-like schemes for PDEs that support a curl-preserving involution. (Some insights into discontinuous Galerkin (DG) schemes are also drawn, though that is not the prime focus of this paper.) This is done for two- and three-dimensional structured mesh problems where we deliver closed form expressions for the reconstruction. The importance of multidimensional Riemann solvers in facilitating the design of such schemes is also documented. In two dimensions, a von Neumann analysis of structure-preserving WENO-like schemes that mimetically satisfy the curl constraints, is also presented. It shows the tremendous value of higher order WENO-like schemes in minimizing dissipation and dispersion for this class of problems. Numerical results are also presented to show that the edge-centered curl-preserving (ECCP) schemes meet their design accuracy. This paper is the first paper that invents non-linearly hybridized curl-preserving reconstruction and integrates it with higher order Godunov philosophy. By its very design, this paper is, therefore, intended to be forward-looking and to set the stage for future work on curl involution-constrained PDEs.

    

   more » « less
2. Von Neumann Stability Analysis of DG-Like and PNPM-Like Schemes for PDEs with Globally Curl-Preserving Evolution of Vector Fields

   https://doi.org/10.1007/s42967-021-00166-x  

   Balsara, Dinshaw S. ; Käppeli, Roger ( January 2022 , Communications on Applied Mathematics and Computation)

   This paper examines a class of involution-constrained PDEs where some part of the PDE system evolves a vector field whose curl remains zero or grows in proportion to specified source terms. Such PDEs are referred to as curl-free or curl-preserving, respectively. They arise very frequently in equations for hyperelasticity and compressible multiphase flow, in certain formulations of general relativity and in the numerical solution of Schrödinger’s equation. Experience has shown that if nothing special is done to account for the curl-preserving vector field, it can blow up in a finite amount of simulation time. In this paper, we catalogue a class of DG-like schemes for such PDEs. To retain the globally curl-free or curl-preserving constraints, the components of the vector field, as well as their higher moments, must be collocated at the edges of the mesh. They are updated using potentials collocated at the vertices of the mesh. The resulting schemes: (i) do not blow up even after very long integration times, (ii) do not need any special cleaning treatment, (iii) can operate with large explicit timesteps, (iv) do not require the solution of an elliptic system and (v) can be extended to higher orders using DG-like methods. The methods rely on a special curl-preserving reconstruction and they also rely on multidimensional upwinding. The Galerkin projection, highly crucial to the design of a DG method, is now conducted at the edges of the mesh and yields a weak form update that uses potentials obtained at the vertices of the mesh with the help of a multidimensional Riemann solver. A von Neumann stability analysis of the curl-preserving methods is conducted and the limiting CFL numbers of this entire family of methods are catalogued in this work. The stability analysis confirms that with the increasing order of accuracy, our novel curl-free methods have superlative phase accuracy while substantially reducing dissipation. We also show that PNPM-like methods, which only evolve the lower moments while reconstructing the higher moments, retain much of the excellent wave propagation characteristics of the DG-like methods while offering a much larger CFL number and lower computational complexity. The quadratic energy preservation of these methods is also shown to be excellent, especially at higher orders. The methods are also shown to be curl-preserving over long integration times.

    

   more » « less
3. Techniques, Tricks, and Algorithms for Efficient GPU-Based Processing of Higher Order Hyperbolic PDEs

   https://doi.org/10.1007/s42967-022-00235-9  

   Subramanian, Sethupathy ; Balsara, Dinshaw S. ; Bhoriya, Deepak ; Kumar, Harish ( March 2023 , Communications on Applied Mathematics and Computation)

   Chi-Wang Shu (Ed.)

   GPU computing is expected to play an integral part in all modern Exascale supercomputers. It is also expected that higher order Godunov schemes will make up about a significant fraction of the application mix on such supercomputers. It is, therefore, very important to prepare the community of users of higher order schemes for hyperbolic PDEs for this emerging opportunity. Not every algorithm that is used in the space-time update of the solution of hyperbolic PDEs will take well to GPUs. However, we identify a small core of algorithms that take exceptionally well to GPU computing. Based on an analysis of available options, we have been able to identify weighted essentially non-oscillatory (WENO) algorithms for spatial reconstruction along with arbitrary derivative (ADER) algorithms for time extension followed by a corrector step as the winning three-part algorithmic combination. Even when a winning subset of algorithms has been identified, it is not clear that they will port seamlessly to GPUs. The low data throughput between CPU and GPU, as well as the very small cache sizes on modern GPUs, implies that we have to think through all aspects of the task of porting an application to GPUs. For that reason, this paper identifies the techniques and tricks needed for making a successful port of this very useful class of higher order algorithms to GPUs. Application codes face a further challenge—the GPU results need to be practically indistinguishable from the CPU results—in order for the legacy knowledge bases embedded in these applications codes to be preserved during the port of GPUs. This requirement often makes a complete code rewrite impossible. For that reason, it is safest to use an approach based on OpenACC directives, so that most of the code remains intact (as long as it was originally well-written). This paper is intended to be a one-stop shop for anyone seeking to make an OpenACC-based port of a higher order Godunov scheme to GPUs. We focus on three broad and high-impact areas where higher order Godunov schemes are used. The first area is computational fluid dynamics (CFD). The second is computational magnetohydrodynamics (MHD) which has an involution constraint that has to be mimetically preserved. The third is computational electrodynamics (CED) which has involution constraints and also extremely stiff source terms. Together, these three diverse uses of higher order Godunov methodology, cover many of the most important applications areas. In all three cases, we show that the optimal use of algorithms, techniques, and tricks, along with the use of OpenACC, yields superlative speedups on GPUs. As a bonus, we find a most remarkable and desirable result: some higher order schemes, with their larger operations count per zone, show better speedup than lower order schemes on GPUs. In other words, the GPU is an optimal stratagem for overcoming the higher computational complexities of higher order schemes. Several avenues for future improvement have also been identified. A scalability study is presented for a real-world application using GPUs and comparable numbers of high-end multicore CPUs. It is found that GPUs offer a substantial performance benefit over comparable number of CPUs, especially when all the methods designed in this paper are used. 

   more » « less
4. GRaM-X: a new GPU-accelerated dynamical spacetime GRMHD code for Exascale computing with the Einstein Toolkit

   https://doi.org/10.1088/1361-6382/acf2d9  

   Shankar, Swapnil ; Mösta, Philipp ; Brandt, Steven R. ; Haas, Roland ; Schnetter, Erik ; de Graaf, Yannick ( September 2023 , Classical and Quantum Gravity)

   We presentGRaM-X(GeneralRelativisticacceleratedMagnetohydrodynamics on AMReX), a new GPU-accelerated dynamical-spacetime general relativistic magnetohydrodynamics (GRMHD) code which extends the GRMHD capability of Einstein Toolkit to GPU-based exascale systems.GRaM-Xsupports 3D adaptive mesh refinement (AMR) on GPUs via a new AMR driver for the Einstein Toolkit calledCarpetXwhich in turn leveragesAMReX, an AMR library developed for use by the United States DOE’s Exascale Computing Project. We use the Z4c formalism to evolve the Einstein equations and the Valencia formulation to evolve the equations of GRMHD.GRaM-Xsupports both analytic as well as tabulated equations of state. We implement TVD and WENO reconstruction methods as well as the HLLE Riemann solver. We test the accuracy of the code using a range of tests on static spacetime, e.g. 1D magnetohydrodynamics shocktubes, the 2D magnetic rotor and a cylindrical explosion, as well as on dynamical spacetimes, i.e. the oscillations of a 3D Tolman-Oppenheimer-Volkhof star. We find excellent agreement with analytic results and results of other codes reported in literature. We also perform scaling tests and find thatGRaM-Xshows a weak scaling efficiency of ∼40%–50% on 2304 nodes (13824 NVIDIA V100 GPUs) with respect to single-node performance on OLCF’s supercomputer Summit.

    

   more » « less
5. A Fixed-Point Fast Sweeping WENO Method with Inverse Lax-Wendroff Boundary Treatment for Steady State of Hyperbolic Conservation Laws

   https://doi.org/10.1007/s42967-021-00179-6  

   Li, Liang ; Zhu, Jun ; Shu, Chi-Wang ; Zhang, Yong-Tao ( March 2022 , Communications on Applied Mathematics and Computation)

   Fixed-point fast sweeping WENO methods are a class of efficient high-order numerical methods to solve steady-state solutions of hyperbolic partial differential equations (PDEs). The Gauss-Seidel iterations and alternating sweeping strategy are used to cover characteristics of hyperbolic PDEs in each sweeping order to achieve fast convergence rate to steady-state solutions. A nice property of fixed-point fast sweeping WENO methods which distinguishes them from other fast sweeping methods is that they are explicit and do not require inverse operation of nonlinear local systems. Hence, they are easy to be applied to a general hyperbolic system. To deal with the difficulties associated with numerical boundary treatment when high-order finite difference methods on a Cartesian mesh are used to solve hyperbolic PDEs on complex domains, inverse Lax-Wendroff (ILW) procedures were developed as a very effective approach in the literature. In this paper, we combine a fifth-order fixed-point fast sweeping WENO method with an ILW procedure to solve steady-state solution of hyperbolic conservation laws on complex computing regions. Numerical experiments are performed to test the method in solving various problems including the cases with the physical boundary not aligned with the grids. Numerical results show high-order accuracy and good performance of the method. Furthermore, the method is compared with the popular third-order total variation diminishing Runge-Kutta (TVD-RK3) time-marching method for steady-state computations. Numerical examples show that for most of examples, the fixed-point fast sweeping method saves more than half CPU time costs than TVD-RK3 to converge to steady-state solutions.

    

   more » « less