skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: Structured Adaptive Mesh Refinement Adaptations to Retain Performance Portability With Increasing Heterogeneity
Adaptive mesh refinement (AMR) is an important method that enables many mesh-based applications to run at effectively higher resolution within limited computing resources by allowing high resolution only where really needed. This advantage comes at a cost, however: greater complexity in the mesh management machinery and challenges with load distribution. With the current trend of increasing heterogeneity in hardware architecture, AMR presents an orthogonal axis of complexity. The usual techniques, such as asynchronous communication and hierarchy management for parallelism and memory that are necessary to obtain reasonable performance are very challenging to reason about with AMR. Different groups working with AMR are bringing different approaches to this challenge. Here, we examine the design choices of several AMR codes and also the degree to which demands placed on them by their users influence these choices.  more » « less
Award ID(s):
1835402
PAR ID:
10479680
Author(s) / Creator(s):
; ; ; ; ;
Editor(s):
Hinsen, Konrad; Dubey, Anshu
Publisher / Repository:
IEEE
Date Published:
Journal Name:
Computing in Science & Engineering
Edition / Version:
Computing in Science and Engineering
Volume:
23
Issue:
5
ISSN:
1521-9615
Page Range / eLocation ID:
62 to 66
Subject(s) / Keyword(s):
adaptive mesh refinement, parallel computing
Format(s):
Medium: X Size: 225 KB Other: pdf
Size(s):
225 KB
Sponsoring Org:
National Science Foundation
More Like this
  1. ; (, ICLASS 2018, 14th Triennial International Conference on Liquid Atomization and Spray Systems)
    Adaptive mesh refinement (AMR) has been introduced as an attractive means of significantly improving computational efficiency for a variety of two-phase flow problems. In the current study, the benefits of AMR are investigated for the case of liquid jet atomization. The evaluation consists of a systematic analysis of results from the interDymFoam (AMR octree) and interFoam (static octree) codes, both of which form part of the family of solvers distributed within the open source OpenFOAM C++ Toolbox. The two-phase flow treatment is based on an algebraic VoF methodology. As a preliminary set of exercises, cases for pure advection, stationary wave dynamics, and Rayleigh-Plateau breakup of a cylindrical liquid element are considered. The results from these exercises confirm the expected trend of higher numerical efficiency in AMR, while still retaining essentially the same level of accuracy as the fixed embedded mesh solutions. However, for the liquid jet atomization, the behavior is a bit more complicated. First, at lower levels of Weber number, we observe a similar trend as the preliminary exercises. At higher Weber numbers, due to a noticeable increase in interfacial area density, substantial inhomogeneities are formed in the underlying grids yielding slower solutions of pressure Poisson equation, thereby potentially offsetting the benefits of this approach. In fact, at much higher Weber numbers, for instance, those pertaining to Diesel injection, the results suggest that a fixed embedded mesh would provide better computational efficiency. However, this conclusion depends on the target lowest level of numerical resolution, Δxmin. The current work shows how the efficiency of AMR suffers from increasing interfacial area density, and how this can be alleviated via a decrease in Δxmin. Various test cases are presented to illustrate this effect. 
    more » « less
  2. ; ; ; (, arXiv but under review in Monthly Weather Review)
    This paper examines the application of adaptive mesh refinement (AMR) in the field of numerical weather prediction (NWP). We implement and assess two distinct AMR approaches and evaluate their performance through standard NWP benchmarks. In both cases, we solve the fully compressible Euler equations, fundamental to many non-hydrostatic weather models. The first approach utilizes oct-tree cell-based mesh refinement coupled with a high-order discontinuous Galerkin method for spatial discretization. In the second approach, we employ level-based AMR with the finite difference method. Our study provides insights into the accuracy and benefits of employing these AMR methodologies for the multi-scale problem of NWP. Additionally, we explore essential properties including their impact on mass and energy conservation. Moreover, we present and evaluate an AMR solution transfer strategy for the tree-based AMR approach that is simple to implement, memory-efficient, and ensures conservation for both flow in the box and sphere. Furthermore, we discuss scalability, performance portability, and the practical utility of the AMR methodology within an NWP framework -- crucial considerations in selecting an AMR approach. The current de facto standard for mesh refinement in NWP employs a relatively simplistic approach of static nested grids, either within a general circulation model or a separately operated regional model with loose one-way synchronization. It is our hope that this study will stimulate further interest in the adoption of AMR frameworks like AMReX in NWP. These frameworks offer a triple advantage: a robust dynamic AMR for tracking localized and consequential features such as tropical cyclones, extreme scalability, and performance portability. 
    more » « less
  3. ; ; ; (, Proceedings of the International Conference on Computational Fluid Dynamics)
    The solution of partial differential equations (PDEs) on modern high performance computing (HPC) platforms is essential to the continued success of groundwater flow and transport modeling in Pacific islands where complex regional groundwater flow is governed by highly heterogeneous volcanic rocks and dynamic interaction between freshwater and seawater. For accurate simulations of complex groundwater flow processes in the Hawaiian islands, the PISALE (Pacific Island Structured-AMR with ALE) software has been developed to offer an innovative combination of advanced mathematical techniques such as arbitrary Lagrangian-Eulerian method (ALE) and Adaptive Mesh Refinement (AMR). The software uses parallel programming models to accelerate the time to solution and dynamically adapt the grids using AMR. This allows for the solution of equations that can reproduce the sharp freshwater-seawater interface in large-scale coast aquifers. In this work, we summarize our ongoing efforts to create a publicly available sustainable branch of the software focused on the groundwater problem. The island-scale numerical groundwater flow modeling will play an important role in predicting the sustainable yields and potential contaminant transport for the volcanic aquifer systems and planning groundwater resources management. 
    more » « less
  4. ; ; (, Thermal and Fluids Engineering Conference)
    In this study, we examine the performance of a multi-scale model for large-eddy simulation (LES) of tur- bulent combustion. The model referred to as RRLES performs the closure of the filtered reaction-rate term in the species transport equation while performing LES by using the linear eddy mixing (LEM) model. The RRLES model uses a multi-scale strategy to obtain the filtered reaction rate of the species and has been shown to address some of the challenges associated with the well-established LEMLES approach. The orig- inally proposed RRLES strategy used a multilevel adaptive mesh refinement (AMR) framework, which was extended to use a single grid-based strategy to enable the application to complex geometries. Additionally, a local dual-resolution grid strategy has also been developed and can potentially be used with different grid topologies, without the need for the AMR. We assess the accuracy and efficiency of the single and dual-grid RRLES approaches by considering a freely propagating turbulent premixed flame under two different initial conditions corresponding to the thin reaction zone (TRZ) and the broken/distributed reaction zone (B/DRZ) regimes. 
    more » « less
  5. ; ; ; ; ; ; ; ; ; (, Classical and Quantum Gravity)
    Abstract We demonstrate the flexibility and utility of the Berger–Rigoutsos adaptive mesh refinement (AMR) algorithm used in the open-source numerical relativity (NR) code GRC hombo for generating gravitational waveforms from binary black-hole (BH) inspirals, and for studying other problems involving non-trivial matter configurations. We show that GRC hombo can produce high quality binary BH waveforms through a code comparison with the established NR code L ean . We also discuss some of the technical challenges involved in making use of full AMR (as opposed to, e.g. moving box mesh refinement), including the numerical effects caused by using various refinement criteria when regridding. We suggest several ‘rules of thumb’ for when to use different tagging criteria for simulating a variety of physical phenomena. We demonstrate the use of these different criteria through example evolutions of a scalar field theory. Finally, we also review the current status and general capabilities of GRC hombo . 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email