An official website of the United States government
Abstract We presentGRaM-X(GeneralRelativisticacceleratedMagnetohydrodynamics on AMReX), a new GPU-accelerated dynamical-spacetime general relativistic magnetohydrodynamics (GRMHD) code which extends the GRMHD capability of Einstein Toolkit to GPU-based exascale systems.GRaM-Xsupports 3D adaptive mesh refinement (AMR) on GPUs via a new AMR driver for the Einstein Toolkit calledCarpetXwhich in turn leveragesAMReX, an AMR library developed for use by the United States DOE’s Exascale Computing Project. We use the Z4c formalism to evolve the Einstein equations and the Valencia formulation to evolve the equations of GRMHD.GRaM-Xsupports both analytic as well as tabulated equations of state. We implement TVD and WENO reconstruction methods as well as the HLLE Riemann solver. We test the accuracy of the code using a range of tests on static spacetime, e.g. 1D magnetohydrodynamics shocktubes, the 2D magnetic rotor and a cylindrical explosion, as well as on dynamical spacetimes, i.e. the oscillations of a 3D Tolman-Oppenheimer-Volkhof star. We find excellent agreement with analytic results and results of other codes reported in literature. We also perform scaling tests and find thatGRaM-Xshows a weak scaling efficiency of ∼40%–50% on 2304 nodes (13824 NVIDIA V100 GPUs) with respect to single-node performance on OLCF’s supercomputer Summit.
more »
« less
This content will become publicly available on December 27, 2025
AsterX: a new open-source GPU-accelerated GRMHD code for dynamical spacetimes
Abstract We presentAsterX, a novel open-source, modular, GPU-accelerated, fully general relativistic magnetohydrodynamic (GRMHD) code designed for dynamic spacetimes in 3D Cartesian coordinates, and tailored for exascale computing. We utilize block-structured adaptive mesh refinement (AMR) throughCarpetX, the new driver for theEinstein Toolkit, which is built onAMReX, a software framework for massively parallel applications.AsterXemploys the Valencia formulation for GRMHD, coupled with the ‘Z4c’ formalism for spacetime evolution, while incorporating high resolution shock capturing schemes to accurately handle the hydrodynamics.AsterXhas undergone rigorous testing in both static and dynamic spacetime, demonstrating remarkable accuracy and agreement with other codes in literature. Using subcycling in time, we find an overall performance gain of factor 2.5–4.5. Benchmarking the code through scaling tests on OLCF’s Frontier supercomputer, we demonstrate a weak scaling efficiency of about 67%–77% on 4096 nodes compared to an 8-node performance.
more »
« less
- PAR ID:
- 10563328
- Publisher / Repository:
- Classical and Quantum Gravity
- Date Published:
- Journal Name:
- Classical and Quantum Gravity
- Volume:
- 42
- Issue:
- 2
- ISSN:
- 0264-9381
- Page Range / eLocation ID:
- 025016
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
We present GRaM-X (General Relativistic accelerated Magnetohydrodynamics on AMReX), a new GPU-accelerated dynamical-spacetime general relativistic magnetohydrodynamics (GRMHD) code which extends the GRMHD capability of Einstein Toolkit to GPU-based exascale systems. GRaM-X supports 3D adaptive mesh refinement (AMR) on GPUs via a new AMR driver for the Einstein Toolkit called CarpetX which in turn leverages AMReX, an AMR library developed for use by the United States DOE's Exascale Computing Project. We use the Z4c formalism to evolve the Einstein equations and the Valencia formulation to evolve the equations of GRMHD. GRaM-X supports both analytic as well as tabulated equations of state. We implement TVD and WENO reconstruction methods as well as the HLLE Riemann solver. We test the accuracy of the code using a range of tests on static spacetime, e.g. 1D magnetohydrodynamics shocktubes, the 2D magnetic rotor and a cylindrical explosion, as well as on dynamical spacetimes, i.e. the oscillations of a 3D Tolman-Oppenheimer-Volkhof star. We find excellent agreement with analytic results and results of other codes reported in literature. We also perform scaling tests and find that GRaM-X shows a weak scaling efficiency of ∼40%–50% on 2304 nodes (13824 NVIDIA V100 GPUs) with respect to single-node performance on OLCF's supercomputer Summit.more » « less
-
Abstract Accretion of magnetized gas on compact astrophysical objects such as black holes (BHs) has been successfully modeled using general relativistic magnetohydrodynamic (GRMHD) simulations. These simulations have largely been performed in the Kerr metric, which describes the spacetime of a vacuum and stationary spinning BH in general relativity (GR). The simulations have revealed important clues to the physics of accretion flows and jets near the BH event horizon and have been used to interpret recent Event Horizon Telescope images of the supermassive BHs M87* and Sgr A*. The GRMHD simulations require the spacetime metric to be given in horizon-penetrating coordinates such that all metric coefficients are regular at the event horizon. Only a few metrics, notably the Kerr metric and its electrically charged spinning analog, the Kerr–Newman metric, are currently available in such coordinates. We report here horizon-penetrating forms of a large class of stationary, axisymmetric, spinning metrics. These can be used to carry out GRMHD simulations of accretion on spinning, nonvacuum BHs and non-BHs within GR, as well as accretion on spinning objects described by non-GR metric theories of gravity.more » « less
-
Abstract In this paper, we introduceSuperLite, an open-source Monte Carlo radiation transport code designed to produce synthetic spectra for astrophysical transient phenomena affected by circumstellar interaction.SuperLiteutilizes Monte Carlo methods for semi-implicit, semirelativistic radiation transport in high-velocity shocked outflows, employing multigroup structured opacity calculations. The code enables rapid post-processing of hydrodynamic profiles to generate high-quality spectra that can be compared with observations of transient events, including superluminous supernovae, pulsational pair-instability supernovae, and other peculiar transients. We present the methods employed inSuperLiteand compare the code’s performance to that of other radiative transport codes, such asSuperNuand CMFGEN. We show thatSuperLitehas successfully passed standard Monte Carlo radiation transport tests and can reproduce spectra of typical supernovae of Type Ia, Type IIP, and Type IIn.more » « less
-
Abstract Multiwavelength observations are now the norm for studying blazars’ various states of activity, classifying them, and determining the possible underlying physical processes driving their emission. Broadband emission models became unavoidable tools for testing emission scenarios and setting the values of physical quantities such as the magnetic field strength, Doppler factor, or shape of the particle distribution of the emission zone(s). We announce here the first public release of a new tool,Bjet_MCMC, that can automatically fit the broadband spectral energy distributions (SEDs) of blazars. The complete code is available on GitHub and allows for testing leptonic synchrotron self-Compton models with or without external inverse-Compton processes from the thermal environment of supermassive black holes (accretion disk and broad-line region). The code is designed to be user-friendly and computationally efficient. It contains a core written in C++ and a fully parallelized SED fitting method. The original multi-SSC zone model ofBjetis also available on GitHub but is not included in the Markov Chain Monte Carlo fitting process at the moment. We present the features, performance, and results ofBjet_MCMC, as well as user advice.more » « less
