In this work, we introduce a scalable and efficient GPU-accelerated methodology for volumetric particle advection and finite-time Lyapunov exponent (FTLE) calculation, focusing on the analysis of Lagrangian coherent structures (LCS) in large-scale direct numerical simulation (DNS) datasets across incompressible, supersonic, and hypersonic flow regimes. LCS play a significant role in turbulent boundary layer analysis, and our proposed methodology offers valuable insights into their behavior in various flow conditions. Our novel owning-cell locator method enables efficient constant-time cell search, and the algorithm draws inspiration from classical search algorithms and modern multi-level approaches in numerical linear algebra. The proposed method is implemented for both multi-core CPUs and Nvidia GPUs, demonstrating strong scaling up to 32,768 CPU cores and up to 62 Nvidia V100 GPUs. By decoupling particle advection from other problems, we achieve modularity and extensibility, resulting in consistent parallel efficiency across different architectures. Our methodology was applied to calculate and visualize the FTLE on four turbulent boundary layers at different Reynolds and Mach numbers, revealing that coherent structures grow more isotropic proportional to the Mach number, and their inclination angle varies along the streamwise direction. We also observed increased anisotropy and FTLE organization at lower Reynolds numbers, with structures retaining coherency along both spanwise and streamwise directions. Additionally, we demonstrated the impact of lower temporal frequency sampling by upscaling with an efficient linear upsampler, preserving general trends with only 10% of the required storage. In summary, we present a particle search scheme for particle advection workloads in the context of visualizing LCS via FTLE that exhibits strong scaling performance and efficiency at scale. Our proposed algorithm is applicable across various domains, requiring efficient search algorithms in large, structured domains. While this article focuses on the methodology and its application to LCS, an in-depth study of the physics and compressibility effects in LCS candidates will be explored in a future publication.
more »
« less
Aquila-LCS: GPU/CPU-Accelerated Particle Advection Schemes for Large-Scale Simulations
We introduce Aquila-LCS, GPU and CPU optimized object-oriented, in-house codes for volumetric particle advection and 3D Finite-Time Lyapunov Exponent (FTLE) and Finite-Size Lyapunov Exponent (FSLE) computations. The purpose is to analyze 3D Lagrangian Coherent Structures (LCS) in large Direct Numerical Simulation (DNS) data. Our technique uses advanced search strategies for quick cell identification and efficient storage techniques. This solver scales effectively on both GPUs (up to 62 Nvidia V100 GPUs) and multi-core CPUs (up to 32,768 CPU cores), tracking up to 8-billion particles. We apply our approach to four turbulent boundary layers at different flow regimes and Reynolds numbers.
more »
« less
- PAR ID:
- 10500004
- Publisher / Repository:
- SSRN
- Date Published:
- Journal Name:
- Software X
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
In this work, we introduce a scalable and efficient GPU-accelerated methodology for volumetric particle advection and finite-time Lyapunov exponent (FTLE) calculation, focusing on the analysis of Lagrangian Coherent Structures (LCS) in large-scale Direct Numerical Simulation (DNS) datasets across incompressible, supersonic, and hypersonic flow regimes. LCS play a significant role in turbulent boundary layer analysis, and our proposed methodology offers valuable insights into their behavior in various flow conditions. Our novel owning-cell locator method enables efficient, constant-time cell search, and the algorithm draws inspiration from classical search algorithms and modern multi-level approaches in numerical linear algebra. The proposed method is implemented for both multi-core CPUs and Nvidia GPUs, demonstrating strong scaling up to 32,768 CPU cores and up to 62 Nvidia V100 GPUs. By decoupling particle advection from other problems, we achieve modularity and extensibility, resulting in consistent parallel efficiency across different architectures. Our methodology was applied to calculate and visualize the FTLE on four turbulent boundary layers at different Reynolds and Mach numbers, revealing that coherent structures grow more isotropic proportional to the Mach number, and their inclination angle varies along the streamwise direction. We also observed increased anisotropy and FTLE organization at lower Reynolds numbers, with structures retaining coherency along both spanwise and streamwise directions. Additionally, we demonstrated the impact of lower temporal frequency sampling by upscaling with an efficient linear upsampler, preserving general trends with only 10% of the required storage. In summary, we present a particle search scheme for particle advection workloads in the context of visualizing LCS via FTLE that exhibits strong scaling performance and efficiency at scale. Our proposed algorithm is applicable across various domains requiring efficient search algorithms in large structured domains. While this manuscript focuses on the methodology and its application to LCS, an in-depth study of the physics and compressibility effects in LCS candidates will be explored in a future publication.more » « less
-
Abstract During the 2019/2020 Australian bushfire season, intense wildfires generated a rising plume with a record concentration of smoke in the lower stratosphere. Motivated by this event, we use the atmospheric wind reanalysis model ERA5 to characterize the three dimensional atmospheric transport in the general region of the plume following a dynamical system approach in the Lagrangian framework. Aided by the Finite Time Lyapunov Exponent tool (FTLE), we identify Lagrangian Coherent Structures (LCS) which simplify the three‐dimensional transport description. Different reduced FTLE formulations are compared to study the impact of the vertical velocity and the vertical shear on the movement of the plume. We then consider in detail some of the uncovered LCS that are directly relevant for the evolution of the plume, as well as other LCS that are less relevant for the plume but have interesting geometries, and we show the presence of 3D lobe dynamics at play. Also, we unveil the qualitatively different dynamical fates of the smoke parcels trajectories depending on the region in which they originated. One feature that had a pronounced influence on the evolution of the smoke plume is a synoptic‐scale anticyclone that was formed near the same time as, and close to the region of, intense wildfires. We analyze this anticyclone in detail, including its formation, the entrainment of the smoke plume, and how it maintained coherence for a long time. Transport paths obtained with the inclusion of the buoyancy effects are compared with those obtained considering only the reanalysis velocity.more » « less
-
In this study, we delve into the intricate relation between Lagrangian Coherent Structures (LCS), primarily represented by the finite-time Lyapunov exponent (FTLE), and instantaneous temperature in turbulent wall-bounded flow scenarios. Turbulence, despite its chaotic facade, houses coherent structures vital to understanding the dynamical behavior of fluid flows. Recognizing this, we leverage high-fidelity Direct Numerical Simulation (DNS) to investigate compressible flows, focusing on the attracting manifolds in FTLE and their correlation with instantaneous temperature. The consequent insights into the coupling between fluid dynamics and thermodynamics reveal the profound influence of vortex stretching, shearing, and compression on local thermodynamic characteristics. Notably, the interplay of instantaneous static temperature and fluid properties, along with the cascading nature of energy in turbulent flows, underpins the observed correlation. Furthermore, we leveraged a high-performance, scalable volumetric particle advection scheme for LCS determination in subsonic (M∞ = 0.8) and supersonic (M∞ = 1.6) turbulent boundary layers over adiabatic flat plates.more » « less
-
We prove a conditional theorem on the positivity of the Lyapunov exponent for a Schrödinger cocycle over a skew-shift base with a cosine potential and the golden ratio as frequency. For coupling below $$1$$ , which is the threshold for Herman’s subharmonicity trick, we formulate three conditions on the Lyapunov exponent in a finite but large volume and on the associated large-deviation estimates at that scale. Our main results demonstrate that these finite-size conditions imply the positivity of the infinite-volume Lyapunov exponent. This paper shows that it is possible to make the techniques developed for the study of Schrödinger operators with deterministic potentials, based on large-deviation estimates and the avalanche principle, effective.more » « less