More Like this

1. Techniques, Tricks, and Algorithms for Efficient GPU-Based Processing of Higher Order Hyperbolic PDEs

   https://doi.org/10.1007/s42967-022-00235-9  

   Subramanian, Sethupathy ; Balsara, Dinshaw S. ; Bhoriya, Deepak ; Kumar, Harish ( March 2023 , Communications on Applied Mathematics and Computation)

   Chi-Wang Shu (Ed.)

   GPU computing is expected to play an integral part in all modern Exascale supercomputers. It is also expected that higher order Godunov schemes will make up about a significant fraction of the application mix on such supercomputers. It is, therefore, very important to prepare the community of users of higher order schemes for hyperbolic PDEs for this emerging opportunity. Not every algorithm that is used in the space-time update of the solution of hyperbolic PDEs will take well to GPUs. However, we identify a small core of algorithms that take exceptionally well to GPU computing. Based on an analysis of available options, we have been able to identify weighted essentially non-oscillatory (WENO) algorithms for spatial reconstruction along with arbitrary derivative (ADER) algorithms for time extension followed by a corrector step as the winning three-part algorithmic combination. Even when a winning subset of algorithms has been identified, it is not clear that they will port seamlessly to GPUs. The low data throughput between CPU and GPU, as well as the very small cache sizes on modern GPUs, implies that we have to think through all aspects of the task of porting an application to GPUs. For that reason, this paper identifies the techniques and tricks needed for making a successful port of this very useful class of higher order algorithms to GPUs. Application codes face a further challenge—the GPU results need to be practically indistinguishable from the CPU results—in order for the legacy knowledge bases embedded in these applications codes to be preserved during the port of GPUs. This requirement often makes a complete code rewrite impossible. For that reason, it is safest to use an approach based on OpenACC directives, so that most of the code remains intact (as long as it was originally well-written). This paper is intended to be a one-stop shop for anyone seeking to make an OpenACC-based port of a higher order Godunov scheme to GPUs. We focus on three broad and high-impact areas where higher order Godunov schemes are used. The first area is computational fluid dynamics (CFD). The second is computational magnetohydrodynamics (MHD) which has an involution constraint that has to be mimetically preserved. The third is computational electrodynamics (CED) which has involution constraints and also extremely stiff source terms. Together, these three diverse uses of higher order Godunov methodology, cover many of the most important applications areas. In all three cases, we show that the optimal use of algorithms, techniques, and tricks, along with the use of OpenACC, yields superlative speedups on GPUs. As a bonus, we find a most remarkable and desirable result: some higher order schemes, with their larger operations count per zone, show better speedup than lower order schemes on GPUs. In other words, the GPU is an optimal stratagem for overcoming the higher computational complexities of higher order schemes. Several avenues for future improvement have also been identified. A scalability study is presented for a real-world application using GPUs and comparable numbers of high-end multicore CPUs. It is found that GPUs offer a substantial performance benefit over comparable number of CPUs, especially when all the methods designed in this paper are used. 

   more » « less
2. Curl Constraint-Preserving Reconstruction and the Guidance it Gives for Mimetic Scheme Design

   https://doi.org/10.1007/s42967-021-00160-3  

   Balsara, Dinshaw S. ; Käppeli, Roger ; Boscheri, Walter ; Dumbser, Michael ( December 2021 , Communications on Applied Mathematics and Computation)

   Several important PDE systems, like magnetohydrodynamics and computational electrodynamics, are known to support involutions where the divergence of a vector field evolves in divergence-free or divergence constraint-preserving fashion. Recently, new classes of PDE systems have emerged for hyperelasticity, compressible multiphase flows, so-called first-order reductions of the Einstein field equations, or a novel first-order hyperbolic reformulation of Schrödinger’s equation, to name a few, where the involution in the PDE supports curl-free or curl constraint-preserving evolution of a vector field. We study the problem of curl constraint-preserving reconstruction as it pertains to the design of mimetic finite volume (FV) WENO-like schemes for PDEs that support a curl-preserving involution. (Some insights into discontinuous Galerkin (DG) schemes are also drawn, though that is not the prime focus of this paper.) This is done for two- and three-dimensional structured mesh problems where we deliver closed form expressions for the reconstruction. The importance of multidimensional Riemann solvers in facilitating the design of such schemes is also documented. In two dimensions, a von Neumann analysis of structure-preserving WENO-like schemes that mimetically satisfy the curl constraints, is also presented. It shows the tremendous value of higher order WENO-like schemes in minimizing dissipation and dispersion for this class of problems. Numerical results are also presented to show that the edge-centered curl-preserving (ECCP) schemes meet their design accuracy. This paper is the first paper that invents non-linearly hybridized curl-preserving reconstruction and integrates it with higher order Godunov philosophy. By its very design, this paper is, therefore, intended to be forward-looking and to set the stage for future work on curl involution-constrained PDEs.

    

   more » « less
3. Multidimensional WENO-AO Reconstructions Using a Simplified Smoothness Indicator and Applications to Conservation Laws

   https://doi.org/10.1007/s10915-023-02319-x  

   Huang, Chieh-Sen ; Arbogast, Todd ; Tian, Chenyu ( August 2023 , Journal of Scientific Computing)

   Finite volume, weighted essentially non-oscillatory (WENO) schemes require the computation of a smoothness indicator. This can be expensive, especially in multiple space dimensions. We consider the use of the simple smoothness indicator$$\sigma ^{\textrm{S}}= \frac{1}{N_{\textrm{S}}-1}\sum _{j} ({\bar{u}}_{j} - {\bar{u}}_{m})^2$$${\sigma }^{\text{S}}=\frac{1}{N_{\text{S}}-1}{\sum }_j{({\overline{u}}_j-{\overline{u}}_m)}^2$, where$$N_{\textrm{S}}$$$N_{\text{S}}$is the number of mesh elements in the stencil,$${\bar{u}}_j$$${\overline{u}}_j$is the local function average over mesh elementj, and indexmgives the target element. Reconstructions utilizing standard WENO weighting fail with this smoothness indicator. We develop a modification of WENO-Z weighting that gives a reliable and accurate reconstruction of adaptive order, which we denote as SWENOZ-AO. We prove that it attains the order of accuracy of the large stencil polynomial approximation when the solution is smooth, and drops to the order of the small stencil polynomial approximations when there is a jump discontinuity in the solution. Numerical examples in one and two space dimensions on general meshes verify the approximation properties of the reconstruction. They also show it to be about 10 times faster in two space dimensions than reconstructions using the classic smoothness indicator. The new reconstruction is applied to define finite volume schemes to approximate the solution of hyperbolic conservation laws. Numerical tests show results of the same quality as standard WENO schemes using the classic smoothness indicator, but with an overall speedup in the computation time of about 3.5–5 times in 2D tests. Moreover, the computational efficiency (CPU time versus error) is noticeably improved.

    

   more » « less
4. The Influence of WENO Schemes on Large-Eddy Simulations of a Neutral Atmospheric Boundary Layer

   https://doi.org/10.1175/JAS-D-21-0033.1  

   Wang, Aaron ; Pan, Ying ; Markowski, Paul M. ( September 2021 , Journal of the Atmospheric Sciences)

   Abstract This work explores the influence of Weighted Essentially Non-Oscillatory (WENO) schemes on Cloud Model 1 (CM1) large-eddy simulations (LES) of a quasi-steady, horizontally homogeneous, fully developed, neutral atmospheric boundary layer (ABL). An advantage of applying WENO schemes to scalar advection in compressible models is the elimination of acoustic waves and associated oscillations of domain-total vertical velocity. Applying WENO schemes to momentum advection in addition to scalar advection yields no further advantage, but has an adverse effect on resolved turbulence within LES. As a tool designed to reduce numerically generated spurious oscillations, WENO schemes also suppress physically realistic instability development in turbulence-resolving simulations. Thus, applying WENO schemes to momentum advection reduces vortex stretching, suppresses the energy cascade, reduces shear-production of resolved Reynolds stress, and eventually amplifies the differences between the surface-layer mean wind profiles in the LES and the mean wind profiles expected in accordance with the filtered law of the wall (LOTW). The role of WENO schemes in adversely influencing surface-layer turbulence has inspired a concept of anti-WENO (AWENO) schemes to enhance instability development in regions where energy-containing turbulent motions are inadequately resolved by LES grids. The success in reproducing the filtered LOTW via AWENO schemes suggests that improving advection schemes is a critical component toward faithfully simulating near-surface turbulence and dealing with other "Terra Incognita" problems. 

   more » « less
5. A Fixed-Point Fast Sweeping WENO Method with Inverse Lax-Wendroff Boundary Treatment for Steady State of Hyperbolic Conservation Laws

   https://doi.org/10.1007/s42967-021-00179-6  

   Li, Liang ; Zhu, Jun ; Shu, Chi-Wang ; Zhang, Yong-Tao ( March 2022 , Communications on Applied Mathematics and Computation)

   Fixed-point fast sweeping WENO methods are a class of efficient high-order numerical methods to solve steady-state solutions of hyperbolic partial differential equations (PDEs). The Gauss-Seidel iterations and alternating sweeping strategy are used to cover characteristics of hyperbolic PDEs in each sweeping order to achieve fast convergence rate to steady-state solutions. A nice property of fixed-point fast sweeping WENO methods which distinguishes them from other fast sweeping methods is that they are explicit and do not require inverse operation of nonlinear local systems. Hence, they are easy to be applied to a general hyperbolic system. To deal with the difficulties associated with numerical boundary treatment when high-order finite difference methods on a Cartesian mesh are used to solve hyperbolic PDEs on complex domains, inverse Lax-Wendroff (ILW) procedures were developed as a very effective approach in the literature. In this paper, we combine a fifth-order fixed-point fast sweeping WENO method with an ILW procedure to solve steady-state solution of hyperbolic conservation laws on complex computing regions. Numerical experiments are performed to test the method in solving various problems including the cases with the physical boundary not aligned with the grids. Numerical results show high-order accuracy and good performance of the method. Furthermore, the method is compared with the popular third-order total variation diminishing Runge-Kutta (TVD-RK3) time-marching method for steady-state computations. Numerical examples show that for most of examples, the fixed-point fast sweeping method saves more than half CPU time costs than TVD-RK3 to converge to steady-state solutions.

    

   more » « less