Search for: All records

Abstract We presentAsterX, a novel open-source, modular, GPU-accelerated, fully general relativistic magnetohydrodynamic (GRMHD) code designed for dynamic spacetimes in 3D Cartesian coordinates, and tailored for exascale computing. We utilize block-structured adaptive mesh refinement (AMR) throughCarpetX, the new driver for theEinstein Toolkit, which is built onAMReX, a software framework for massively parallel applications.AsterXemploys the Valencia formulation for GRMHD, coupled with the ‘Z4c’ formalism for spacetime evolution, while incorporating high resolution shock capturing schemes to accurately handle the hydrodynamics.AsterXhas undergone rigorous testing in both static and dynamic spacetime, demonstrating remarkable accuracy and agreement with other codes in literature. Using subcycling in time, we find an overall performance gain of factor 2.5–4.5. Benchmarking the code through scaling tests on OLCF’s Frontier supercomputer, we demonstrate a weak scaling efficiency of about 67%–77% on 4096 nodes compared to an 8-node performance. 

Black holes offer a unique laboratory for fundamental physics and are crucial for understanding theories beyond Einstein’s theory of general relativity. In this paper, we focus on 4D effective field theories and string-theory inspired models that include scalar fields. We focus on one such model, axi-dilaton gravity, a quadratic gravity theory with two kinetically coupled scalar fields, an axion and a dilaton. To study the evolution and structure of these fields around black holes, we introduce canuda–axidil, the first open-source, parametrized numerical relativity code for quadratic and biscalar gravity. Using this code, we perform single black hole simulations to show the dynamical formation of axion and dilaton hairs and quantify the effect of higher-order terms in coupling and spin. Through these simulations, we measure the impact of black hole spin and curvature coupling strength on the profiles of the axion and dilaton and show that including kinetic coupling between the fields increases the observed deviations from general relativity. Furthermore, we simulate the axion and dilaton fields around a binary black hole coalescence demonstrating the growth of axion hair during the inspiral and the production of radiative modes for both fields. 

As gravitational wave detectors improve in sensitivity, signal-to-noise ratios of compact binary coalescences will dramatically increase, reaching values in the hundreds and potentially thousands. Such strong signals offer both exciting scientific opportunities and pose formidable challenges to the template waveforms used for interpretation. Current waveform models are informed by calibrating or fitting to numerical relativity waveforms and such strong signals may unveil computational errors in generating these waveforms. In this paper, we isolate a single source of computational error, that of the finite grid resolution, and investigate its impact on parameter estimation for aLIGO and Cosmic Explorer. We demonstrate that increasing the inclination angle or decreasing the mass ratio q (q≤1) raises the resolution required for unbiased parameter estimation. We quantify the error associated with the highest-resolution waveform utilized in our study using an extrapolation procedure on the median of recovered posteriors and confirm the accuracy of current waveforms for the synthetic sources. We introduce a measure to predict the necessary numerical resolution for unbiased parameter estimation and use it to predict that current waveforms are suitable for equal and moderately unequal mass binaries for both detectors. However, current waveforms fail to meet accuracy requirements for high signal-to-noise ratio signals from highly unequal mass ratio binaries (q≲1/6), for all inclinations in Cosmic Explorer, and for high inclinations in future updates to LIGO. Given that the resolution requirement becomes more stringent with more unequal mass ratios, current waveforms may lack the necessary accuracy, even at median signal-to-noise ratios for future detectors. 

Triaxial neutron stars can be sources of continuous gravitational radiation detectable by ground-based interferometers. The amplitude of the emitted gravitational wave can be greatly affected by the state of the hydrodynamical fluid flow inside the neutron star. In this work, we examine the most triaxial models along two sequences of constant rest mass, confirming their dynamical stability. We also study the response of a triaxial figure of quasiequilibrium under a variety of perturbations that lead to different fluid flows. Starting from the general relativistic compressible analog of the Newtonian Jacobi ellipsoid, we perform simulations of Dedekind-type flows. We find that in some cases the triaxial neutron star resembles a Riemann-S-type ellipsoid with minor rotation and gravitational wave emission as it evolves towards axisymmetry. The present results highlight the importance of understanding the fluid flow in the interior of a neutron star in terms of its gravitational wave content. 

We present a 3D general-relativistic magnetohydrodynamic simulation of a short-lived neutron star remnant formed in the aftermath of a binary neutron star merger. The simulation uses an M1 neutrino transport scheme to track neutrino–matter interactions and is well suited to studying the resulting nucleosynthesis and kilonova emission. A magnetized wind is driven from the remnant and ejects neutron-rich material at a quasi-steady-state rate of 0.8 × 10^−1M⊙ ​s^−1. We find that the ejecta in our simulations underproduce r-process abundances beyond the second r-process peak. For sufficiently long-lived remnants, these outflows alone can produce blue kilonovae, including the blue kilonova component observed for AT2017gfo. 

Reproducibility of results is a cornerstone of the scientific method. Scientific computing encounters two challenges when aiming for this goal. Firstly, reproducibility should not depend on details of the runtime environment, such as the compiler version or computing environment, so results are verifiable by third-parties. Secondly, different versions of software code executed in the same runtime environment should produce consistent numerical results for physical quantities. In this manuscript, we test the feasibility of reproducing scientific results obtained using the IllinoisGRMHD code that is part of an open-source community software for simulation in relativistic astrophysics, the Einstein Toolkit. We verify that numerical results of simulating a single isolated neutron star with IllinoisGRMHD can be reproduced, and compare them to results reported by the code authors in 2015. We use two different supercomputers: Expanse at SDSC, and Stampede2 at TACC. By compiling the source code archived along with the paper on both Expanse and Stampede2, we find that IllinoisGRMHD reproduces results published in its announcement paper up to errors comparable to round-off level changes in initial data parameters. We also verify that a current version of IllinoisGRMHD reproduces these results once we account for bug fixes which have occurred since the original publication. 

We present GRaM-X (General Relativistic accelerated Magnetohydrodynamics on AMReX), a new GPU-accelerated dynamical-spacetime general relativistic magnetohydrodynamics (GRMHD) code which extends the GRMHD capability of Einstein Toolkit to GPU-based exascale systems. GRaM-X supports 3D adaptive mesh refinement (AMR) on GPUs via a new AMR driver for the Einstein Toolkit called CarpetX which in turn leverages AMReX, an AMR library developed for use by the United States DOE's Exascale Computing Project. We use the Z4c formalism to evolve the Einstein equations and the Valencia formulation to evolve the equations of GRMHD. GRaM-X supports both analytic as well as tabulated equations of state. We implement TVD and WENO reconstruction methods as well as the HLLE Riemann solver. We test the accuracy of the code using a range of tests on static spacetime, e.g. 1D magnetohydrodynamics shocktubes, the 2D magnetic rotor and a cylindrical explosion, as well as on dynamical spacetimes, i.e. the oscillations of a 3D Tolman-Oppenheimer-Volkhof star. We find excellent agreement with analytic results and results of other codes reported in literature. We also perform scaling tests and find that GRaM-X shows a weak scaling efficiency of ∼40%–50% on 2304 nodes (13824 NVIDIA V100 GPUs) with respect to single-node performance on OLCF's supercomputer Summit. 

Black holes have a unique sensitivity to the presence of ultralight matter fields or modifications of the underlying theory of gravity. In the present paper we combine both features by studying an ultralight, dynamical scalar field that is nonminimally coupled to the gravitational Chern-Simons term. In particular, we numerically simulate the evolution of such a scalar field around a rotating black hole in the decoupling approximation and find a new kind of massive scalar hair anchored around the black hole. In the proximity of the black hole, the scalar exhibits the typical dipolar structure of hairy solutions in (massless) dynamical Chern-Simons gravity. At larger distances, the field transitions to an oscillating scalar cloud that is induced by the mass term. Finally, we complement the time-domain results with a spectral analysis of the scalar field characteristic frequencies.