skip to main content

Title: Simulating Hydrodynamics in Cosmology with CRK-HACC

We introduce CRK-HACC, an extension of the Hardware/Hybrid Accelerated Cosmology Code (HACC), to resolve gas hydrodynamics in large-scale structure formation simulations of the universe. The new framework couples the HACC gravitationalN-body solver with a modern smoothed-particle hydrodynamics (SPH) approach called conservative reproducing kernel SPH (CRKSPH). CRKSPH utilizes smoothing functions that exactly interpolate linear fields while manifestly preserving conservation laws (momentum, mass, and energy). The CRKSPH method has been incorporated to accurately model baryonic effects in cosmology simulations—an important addition targeting the generation of precise synthetic sky predictions for upcoming observational surveys. CRK-HACC inherits the codesign strategies of the HACC solver and is built to run on modern GPU-accelerated supercomputers. In this work, we summarize the primary solver components and present a number of standard validation tests to demonstrate code accuracy, including idealized hydrodynamic and cosmological setups, as well as self-similarity measurements.

more » « less
Award ID(s):
Author(s) / Creator(s):
; ; ; ; ; ;
Publisher / Repository:
DOI PREFIX: 10.3847
Date Published:
Journal Name:
The Astrophysical Journal Supplement Series
Medium: X Size: Article No. 34
["Article No. 34"]
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract. Plume-SPH provides the first particle-based simulation ofvolcanic plumes. Smoothed particle hydrodynamics (SPH) has several advantagesover currently used mesh-based methods in modeling of multiphase freeboundary flows like volcanic plumes. This tool will provide more accurateeruption source terms to users of volcanic ash transport anddispersion models (VATDs), greatly improving volcanic ash forecasts. The accuracy ofthese terms is crucial for forecasts from VATDs, and the 3-D SPH modelpresented here will provide better numerical accuracy. As an initial effortto exploit the feasibility and advantages of SPH in volcanic plume modeling,we adopt a relatively simple physics model (3-D dusty-gas dynamic modelassuming well-mixed eruption material, dynamic equilibrium and thermodynamicequilibrium between erupted material and air that entrained into the plume,and minimal effect of winds) targeted at capturing the salient features of avolcanic plume. The documented open-source code is easily obtained andextended to incorporate other models of physics of interest to the largecommunity of researchers investigating multiphase free boundary flows ofvolcanic or other origins.

    The Plume-SPH code ( also incorporates several newly developed techniques inSPH needed to address numerical challenges in simulating multiphasecompressible turbulent flow. The code should thus be also of general interestto the much larger community of researchers using and developing SPH-basedtools. In particular, the SPHε turbulence model is used to capturemixing at unresolved scales. Heat exchange due to turbulence is calculated bya Reynolds analogy, and a corrected SPH is used to handle tensile instabilityand deficiency of particle distribution near the boundaries. We alsodeveloped methodology to impose velocity inlet and pressure outlet boundaryconditions, both of which are scarce in traditional implementations of SPH.

    The core solver of our model is parallelized with the message passinginterface (MPI) obtaining good weak and strong scalability using novel techniquesfor data management using space-filling curves (SFCs), object creationtime-based indexing and hash-table-based storage schemes. These techniques areof interest to researchers engaged in developing particles in cell-typemethods. The code is first verified by 1-D shock tube tests, then bycomparing velocity and concentration distribution along the central axis andon the transverse cross with experimental results of JPUE (jet or plume thatis ejected from a nozzle into a uniform environment). Profiles of severalintegrated variables are compared with those calculated by existing 3-D plumemodels for an eruption with the same mass eruption rate (MER) estimated forthe Mt. Pinatubo eruption of 15 June 1991. Our results are consistent withexisting 3-D plume models. Analysis of the plume evolution processdemonstrates that this model is able to reproduce the physics of plumedevelopment.

    more » « less
  2. We present a multi-scale mathematical model and a novel numerical solver to study blood plasma flow and oxygen concentration in a prototype model of an implantable Bioartificial Pancreas (iBAP) that operates under arteriovenous pressure differential without the need for immunosuppressive therapy. The iBAP design consists of a poroelastic cell scaffold containing the healthy transplanted cells, encapsulated between two semi-permeable nano-pore size membranes to prevent the patient’s own immune cells from attacking the transplant. The device is connected to the patient’s vascular system via an anastomosis graft bringing oxygen and nutrients to the transplanted cells of which oxygen is the limiting factor for long-term viability. Mathematically, we propose a (nolinear) fluid–poroelastic structure interaction model to describe the flow of blood plasma through the scaffold containing the cells, and a set of (nonlinear) advection–reaction–diffusion equations defined on moving domains to study oxygen supply to the cells. These macro-scale models are solved using finite element method based solvers. One of the novelties of this work is the design of a novel second-order accurate fluid–poroelastic structure interaction solver, for which we prove that it is unconditionally stable. At the micro/nano-scale, Smoothed Particle Hydrodynamics (SPH) simulations are used to capture the micro/nano-structure (architecture) of cell scaffolds and obtain macro-scale parameters, such as hydraulic conductivity/permeability, from the micro-scale scaffold-specific architecture. To avoid expensive micro-scale simulations based on SPH simulations for every new scaffold architecture, we use Encoder–Decoder Convolution Neural Networks. Based on our numerical simulations, we propose improvements in the current prototype design. For example, we show that highly elastic scaffolds have a higher capacity for oxygen transfer, which is an important finding considering that scaffold elasticity can be controlled during their fabrication, and that elastic scaffolds improve cell viability. The mathematical and computational approaches developed in this work provide a benchmark tool for computational analysis of not only iBAP, but also, more generally, of cell encapsulation strategies used in the design of devices for cell therapy and bio-artificial organs. 
    more » « less
  3. Abstract

    The amount of vapor in the impact-generated protolunar disk carries implications for the dynamics, devolatilization, and moderately volatile element isotope fractionation during lunar formation. The equation of state (EoS) used in simulations of the giant impact is required to calculate the vapor mass fraction (VMF) of the modeled protolunar disk. Recently, a new version of M-ANEOS (Stewart M-ANEOS) was released with an improved treatment of heat capacity and expanded experimental Hugoniot. Here, we compare this new M-ANEOS version with a previous version (N-SPH M-ANEOS) and assess the resulting differences in smoothed particle hydrodynamics (SPH) simulations. We find that Stewart M-ANEOS results in cooler disks with smaller values of VMF and in differences in disk mass that are dependent on the initial impact angle. We also assess the implications of the minimum “cutoff” density (ρc), similar to a maximum smoothing length, that is set as a fast-computing alternative to an iteratively calculated smoothing length. We find that the low particle resolution of the disk typically results in >40% of disk particles falling toρc, influencing the dynamical evolution and VMF of the disk. Our results show that the choice of EoS,ρc, and particle resolution can cause the VMF and disk mass to vary by tens of percent. Moreover, small values ofρcproduce disks that are prone to numerical instability and artificial shocks. We recommend that future giant impact SPH studies review smoothing methods and ensure the thermodynamic stability of the disk over simulated time.

    more » « less
  4. null (Ed.)
    ABSTRACT The morphology of bipolar planetary nebulae (PNe) can be attributed to interactions between a fast wind from the central engine and the dense toroidal-shaped ejecta left over from common envelope (CE) evolution. Here we use the 3D hydrodynamic adaptive mesh refinement (AMR) code AstroBEAR to study the possibility that bipolar PN outflows can emerge collimated even from an uncollimated spherical wind in the aftermath of a CE event. The output of a single CE simulation via the smoothed particle hydrodynamics (SPH) code phantom serves as the initial conditions. Four cases of winds, all with high enough momenta to account for observed high momenta pre-PN outflows, are injected spherically from the region of the CE binary remnant into the ejecta. We compare cases with two different momenta and cases with no radiative cooling versus application of optically thin emission via a cooling curve to the outflow. Our simulations show that in all cases highly collimated bipolar outflows result from deflection of the spherical wind via the interaction with the CE ejecta. Significant asymmetries between the top and bottom lobes are seen in all cases. The asymmetry is strongest for the lower momentum case with radiative cooling. While real post-CE winds may be aspherical, our models show that collimation via ‘inertial confinement’ will be strong enough to create jet-like outflows even beginning with maximally uncollimated drivers. Our simulations reveal detailed shock structures in the shock-focused inertial confinement (SFIC) model and develop a lens-shaped inner shock that is a new feature of SFIC-driven bipolar lobes. 
    more » « less
  5. Abstract

    Solar Dynamics Observatory (SDO)/Helioseismic and Magnetic Imager (HMI) observations reveal a class of solar flares with substantial energy and momentum impacts in the photosphere, concurrent with white-light emission and helioseismic responses, known as sunquakes. Previous radiative hydrodynamic modeling has demonstrated the challenges of explaining sunquakes in the framework of the standard flare model of “electron beam” heating. One of the possibilities to explain the sunquakes and other signatures of the photospheric impact is to consider additional heating mechanisms involved in solar flares, for example via flare-accelerated protons. In this work, we analyze a set of single-loop Fokker–Planck and radiative hydrodynamics RADYN+FP simulations where the atmosphere is heated by nonthermal power-law-distributed proton beams which can penetrate deeper than the electron beams into the low atmospheric layers. Using the output of the RADYN models, we calculate synthetic Fei6173 Å line Stokes profiles and from those the line-of-sight observables of the SDO/HMI instrument, as well as the 3D helioseismic response, and compare them with the corresponding observational characteristics. These initial results show that the models with proton beam heating can produce the enhancement of the HMI continuum observable and explain qualitatively the generation of sunquakes. The continuum observable enhancement is evident in all models but is more prominent in ones withEc≥ 500 keV. In contrast, the models withEc≤ 100 keV provide a stronger sunquake-like helioseismic impact according to the 3D acoustic modeling, suggesting that low-energy (deka- and hecto-keV) protons have an important role in the generation of sunquakes.

    more » « less