Search for: All records

GPU computing is expected to play an integral part in all modern Exascale supercomputers. It is also expected that higher order Godunov schemes will make up about a significant fraction of the application mix on such supercomputers. It is, therefore, very important to prepare the community of users of higher order schemes for hyperbolic PDEs for this emerging opportunity. Not every algorithm that is used in the space-time update of the solution of hyperbolic PDEs will take well to GPUs. However, we identify a small core of algorithms that take exceptionally well to GPU computing. Based on an analysis of available options, we have been able to identify weighted essentially non-oscillatory (WENO) algorithms for spatial reconstruction along with arbitrary derivative (ADER) algorithms for time extension followed by a corrector step as the winning three-part algorithmic combination. Even when a winning subset of algorithms has been identified, it is not clear that they will port seamlessly to GPUs. The low data throughput between CPU and GPU, as well as the very small cache sizes on modern GPUs, implies that we have to think through all aspects of the task of porting an application to GPUs. For that reason, this paper identifies the techniques and tricks needed for making a successful port of this very useful class of higher order algorithms to GPUs. Application codes face a further challenge—the GPU results need to be practically indistinguishable from the CPU results—in order for the legacy knowledge bases embedded in these applications codes to be preserved during the port of GPUs. This requirement often makes a complete code rewrite impossible. For that reason, it is safest to use an approach based on OpenACC directives, so that most of the code remains intact (as long as it was originally well-written). This paper is intended to be a one-stop shop for anyone seeking to make an OpenACC-based port of a higher order Godunov scheme to GPUs. We focus on three broad and high-impact areas where higher order Godunov schemes are used. The first area is computational fluid dynamics (CFD). The second is computational magnetohydrodynamics (MHD) which has an involution constraint that has to be mimetically preserved. The third is computational electrodynamics (CED) which has involution constraints and also extremely stiff source terms. Together, these three diverse uses of higher order Godunov methodology, cover many of the most important applications areas. In all three cases, we show that the optimal use of algorithms, techniques, and tricks, along with the use of OpenACC, yields superlative speedups on GPUs. As a bonus, we find a most remarkable and desirable result: some higher order schemes, with their larger operations count per zone, show better speedup than lower order schemes on GPUs. In other words, the GPU is an optimal stratagem for overcoming the higher computational complexities of higher order schemes. Several avenues for future improvement have also been identified. A scalability study is presented for a real-world application using GPUs and comparable numbers of high-end multicore CPUs. It is found that GPUs offer a substantial performance benefit over comparable number of CPUs, especially when all the methods designed in this paper are used. 

Abstract Adaptive mesh refinement (AMR) is the art of solving PDEs on a mesh hierarchy with increasing mesh refinement at each level of the hierarchy. Accurate treatment on AMR hierarchies requires accurate prolongation of the solution from a coarse mesh to a newly defined finer mesh. For scalar variables, suitably high-order finite volume WENO methods can carry out such a prolongation. However, classes of PDEs, such as computational electrodynamics (CED) and magnetohydrodynamics (MHD), require that vector fields preserve a divergence constraint. The primal variables in such schemes consist of normal components of the vector field that are collocated at the faces of the mesh. As a result, the reconstruction and prolongation strategies for divergence constraint-preserving vector fields are necessarily more intricate. In this paper we present a fourth-order divergence constraint-preserving prolongation strategy that is analytically exact. Extension to higher orders using analytically exact methods is very challenging. To overcome that challenge, a novel WENO-like reconstruction strategy is invented that matches the moments of the vector field in the faces, where the vector field components are collocated. This approach is almost divergence constraint-preserving, therefore, we call it WENO-ADP. To make it exactly divergence constraint-preserving, a touch-up procedure is developed that is based on a constrained least squares (CLSQ) method for restoring the divergence constraint up to machine accuracy. With the touch-up, it is called WENO-ADPT. It is shown that refinement ratios of two and higher can be accommodated. An item of broader interest in this work is that we have also been able to invent very efficient finite volume WENO methods, where the coefficients are very easily obtained and the multidimensional smoothness indicators can be expressed as perfect squares. We demonstrate that the divergence constraint-preserving strategy works at several high orders for divergence-free vector fields as well as vector fields, where the divergence of the vector field has to match a charge density and its higher moments. We also show that our methods overcome the late time instability that has been known to plague adaptive computations in CED. 

In this paper we present the first set of 3D magnetohydrodynamic (MHD) simulations performed with the riemann geomesh code. We study the dynamics of the magnetically channeled winds of magnetic massive stars in full three dimensions using a code that is uniquely suited to spherical problems. Specifically, we perform isothermal simulations of a smooth wind on a rotating star with a tilted, initially dipolar field. We compare the mass-loss, angular momentum loss, and magnetospheric dynamics of a template star (with the properties that are reminiscent of the O4 supergiant ζ Pup) over a range of rotation rates, magnetic field strengths, and magnetic tilt angles. The simulations are run up to a quasi-steady state and the results are observed to be consistent with the existing literature, showing the episodic centrifugal breakout events of the mass outflow, confined by the magnetic field loops that form the closed magnetosphere of the star. The catalogued results provide perspective on how angular-momentum loss varies for different configurations of rotation rate, magnetic field strength, and large magnetic tilt angles. In agreement with previous 2D MHD studies, we find that high magnetic confinement reduces the overall mass-loss rate, and higher rotation increases the mass-loss rate. This and future studies will be used to estimate the angular-momentum evolution, spin-down time, and mass-loss evolution of magnetic massive stars as a function of magnetic field strength, rotation rate, and dipole tilt.

 

This paper examines a class of involution-constrained PDEs where some part of the PDE system evolves a vector field whose curl remains zero or grows in proportion to specified source terms. Such PDEs are referred to as curl-free or curl-preserving, respectively. They arise very frequently in equations for hyperelasticity and compressible multiphase flow, in certain formulations of general relativity and in the numerical solution of Schrödinger’s equation. Experience has shown that if nothing special is done to account for the curl-preserving vector field, it can blow up in a finite amount of simulation time. In this paper, we catalogue a class of DG-like schemes for such PDEs. To retain the globally curl-free or curl-preserving constraints, the components of the vector field, as well as their higher moments, must be collocated at the edges of the mesh. They are updated using potentials collocated at the vertices of the mesh. The resulting schemes: (i) do not blow up even after very long integration times, (ii) do not need any special cleaning treatment, (iii) can operate with large explicit timesteps, (iv) do not require the solution of an elliptic system and (v) can be extended to higher orders using DG-like methods. The methods rely on a special curl-preserving reconstruction and they also rely on multidimensional upwinding. The Galerkin projection, highly crucial to the design of a DG method, is now conducted at the edges of the mesh and yields a weak form update that uses potentials obtained at the vertices of the mesh with the help of a multidimensional Riemann solver. A von Neumann stability analysis of the curl-preserving methods is conducted and the limiting CFL numbers of this entire family of methods are catalogued in this work. The stability analysis confirms that with the increasing order of accuracy, our novel curl-free methods have superlative phase accuracy while substantially reducing dissipation. We also show that PNPM-like methods, which only evolve the lower moments while reconstructing the higher moments, retain much of the excellent wave propagation characteristics of the DG-like methods while offering a much larger CFL number and lower computational complexity. The quadratic energy preservation of these methods is also shown to be excellent, especially at higher orders. The methods are also shown to be curl-preserving over long integration times.

 

Several important PDE systems, like magnetohydrodynamics and computational electrodynamics, are known to support involutions where the divergence of a vector field evolves in divergence-free or divergence constraint-preserving fashion. Recently, new classes of PDE systems have emerged for hyperelasticity, compressible multiphase flows, so-called first-order reductions of the Einstein field equations, or a novel first-order hyperbolic reformulation of Schrödinger’s equation, to name a few, where the involution in the PDE supports curl-free or curl constraint-preserving evolution of a vector field. We study the problem of curl constraint-preserving reconstruction as it pertains to the design of mimetic finite volume (FV) WENO-like schemes for PDEs that support a curl-preserving involution. (Some insights into discontinuous Galerkin (DG) schemes are also drawn, though that is not the prime focus of this paper.) This is done for two- and three-dimensional structured mesh problems where we deliver closed form expressions for the reconstruction. The importance of multidimensional Riemann solvers in facilitating the design of such schemes is also documented. In two dimensions, a von Neumann analysis of structure-preserving WENO-like schemes that mimetically satisfy the curl constraints, is also presented. It shows the tremendous value of higher order WENO-like schemes in minimizing dissipation and dispersion for this class of problems. Numerical results are also presented to show that the edge-centered curl-preserving (ECCP) schemes meet their design accuracy. This paper is the first paper that invents non-linearly hybridized curl-preserving reconstruction and integrates it with higher order Godunov philosophy. By its very design, this paper is, therefore, intended to be forward-looking and to set the stage for future work on curl involution-constrained PDEs.