Search for: All records

Abstract We presentAsterX, a novel open-source, modular, GPU-accelerated, fully general relativistic magnetohydrodynamic (GRMHD) code designed for dynamic spacetimes in 3D Cartesian coordinates, and tailored for exascale computing. We utilize block-structured adaptive mesh refinement (AMR) throughCarpetX, the new driver for theEinstein Toolkit, which is built onAMReX, a software framework for massively parallel applications.AsterXemploys the Valencia formulation for GRMHD, coupled with the ‘Z4c’ formalism for spacetime evolution, while incorporating high resolution shock capturing schemes to accurately handle the hydrodynamics.AsterXhas undergone rigorous testing in both static and dynamic spacetime, demonstrating remarkable accuracy and agreement with other codes in literature. Using subcycling in time, we find an overall performance gain of factor 2.5–4.5. Benchmarking the code through scaling tests on OLCF’s Frontier supercomputer, we demonstrate a weak scaling efficiency of about 67%–77% on 4096 nodes compared to an 8-node performance. 

Abstract Systems consisting of spheres rolling on elastic membranes have been used to introduce a core conceptual idea of General Relativity: how curvature guides the movement of matter. However, such schemes cannot accurately represent relativistic dynamics in the laboratory because of the dominance of dissipation and external gravitational fields. Here we demonstrate that an “active” object (a wheeled robot), which moves in a straight line on level ground and can alter its speed depending on the curvature of the deformable terrain it moves on, can exactly capture dynamics in curved relativistic spacetimes. Via the systematic study of the robot’s dynamics in the radial and orbital directions, we develop a mapping of the emergent trajectories of a wheeled vehicle on a spandex membrane to the motion in a curved spacetime. Our mapping demonstrates how the driven robot’s dynamics mix space and time in a metric, and shows how active particles do not necessarily follow geodesics in the real space but instead follow geodesics in a fiducial spacetime. The mapping further reveals how parameters such as the membrane elasticity and instantaneous speed allow the programming of a desired spacetime, such as the Schwarzschild metric near a non-rotating blackhole. Our mapping and framework facilitate creation of a robophysical analog to a general relativistic system in the laboratory at low cost that can provide insights into active matter in deformable environments and robot exploration in complex landscapes. 

Modern gravitational-wave science demands increasingly accurate and computationally intensive numerical relativity (NR) simulations. The Python-based, open-sourceNRPyframework generates optimized C/C++ code for NR, including the complete NR codeBlackHoles@Home(BH@H), which leverages curvilinear coordinates well-suited to many astrophysical scenarios. Historically,BH@Hwas limited to single-nodeOpenMPCPU parallelism. To address this, we introducesuperB, an open-source extension toNRPythat enables automatic generation of scalable, task-based, distributed-memoryCharm++code from existingBH@Hmodules. The generated code partitions the structured grids used byNRPy/BH@H, managing communication between them. Its correctness is validated through bit-identical results with the standardOpenMPversion on a single node and via a head-on binary black hole simulation in cylindrical-like coordinates, accurately reproducing quasi-normal modes (up to $\ell =8$). ThesuperB/NRPy-generated code demonstrates excellent strong scaling, achieving an ≈45× speedup on 64 nodes (7168 cores) compared to the original single-nodeOpenMPcode for a large 3D vacuum test. This scalable infrastructure benefits demanding simulations and lays the groundwork for future multi-patch grid support, targeting long inspirals, extreme parameter studies, and rapid follow-ups. This infrastructure readily integrates with otherNRPy/BH@H-based projects, enabling performant scaling for the general relativistic hydrodynamics codeGRoovy, and facilitating future coupling with GPU acceleration via theNRPy-CUDAproject. 

Black holes offer a unique laboratory for fundamental physics and are crucial for understanding theories beyond Einstein’s theory of general relativity. In this paper, we focus on 4D effective field theories and string-theory inspired models that include scalar fields. We focus on one such model, axi-dilaton gravity, a quadratic gravity theory with two kinetically coupled scalar fields, an axion and a dilaton. To study the evolution and structure of these fields around black holes, we introduce canuda–axidil, the first open-source, parametrized numerical relativity code for quadratic and biscalar gravity. Using this code, we perform single black hole simulations to show the dynamical formation of axion and dilaton hairs and quantify the effect of higher-order terms in coupling and spin. Through these simulations, we measure the impact of black hole spin and curvature coupling strength on the profiles of the axion and dilaton and show that including kinetic coupling between the fields increases the observed deviations from general relativity. Furthermore, we simulate the axion and dilaton fields around a binary black hole coalescence demonstrating the growth of axion hair during the inspiral and the production of radiative modes for both fields. 

Next-generation gravitational wave (GW) detectors such as Cosmic Explorer, the Einstein Telescope, and LISA, demand highly accurate and extensive GW catalogs to faithfully extract physical parameters from observed signals. However, numerical relativity (NR) faces significant challenges in generating these catalogs at the required scale and accuracy on modern computers, as NR codes do not fully exploit modern GPU capabilities. In response, we extend NRPy, a Python-based NR code-generation framework, to develop NRPyEllipticGPU—a CUDA-optimized elliptic solver tailored for the binary black hole initial data problem. NRPyEllipticGPU is the first GPU-enabled elliptic solver in the NR community, supporting a variety of coordinate systems and demonstrating substantial performance improvements on both consumer-grade and HPC-grade GPUs. We show that, when compared to a high-end CPU, NRPyEllipticGPU achieves on a high-end GPU up to a sixteenfold speedup in single precision while increasing double-precision performance by a factor of 2–4. This performance boost leverages the GPU’s superior parallelism and memory bandwidth to achieve a compute-bound application and enhancing the overall simulation efficiency. As NRPyEllipticGPU shares the core infrastructure common to NR codes, this work serves as a practical guide for developing full, CUDA-optimized NR codes. 

Two-dimensional models assuming axisymmetry are an economical way to explore the long-term evolution of black hole accretion disks, but they are only realistic if the feedback of the nonaxisymmetric turbulence on the mean momentum and magnetic fields is incorporated. Dynamo terms added to the 2D induction equation should be calibrated to 3D magnetohydrodynamics simulations. For generality, the dynamo tensors should be calibrated as functions of local variables rather than explicit functions of spatial coordinates in a particular basis. In this paper, we study the feedback of nonaxisymmetric features on the 2D mean fields using a global 3D, relativistic, Cartesian simulation from the illinoisgrmhd code. We introduce new methods for estimating overall dynamo alpha and turbulent diffusivity effects, as well as measures of the dominance of nonaxisymmetric components of energies and fluxes within the disk interior. We attempt closure models of the dynamo electromotive force using least-squares fitting, considering both models where coefficient tensors are functions of space and more global, covariant models. None of these models are judged satisfactory, but we are able to draw conclusions on what sorts of generalizations are and are not promising.