In this work, we introduce a scalable and efficient GPU-accelerated methodology for volumetric particle advection and finite-time Lyapunov exponent (FTLE) calculation, focusing on the analysis of Lagrangian Coherent Structures (LCS) in large-scale Direct Numerical Simulation (DNS) datasets across incompressible, supersonic, and hypersonic flow regimes. LCS play a significant role in turbulent boundary layer analysis, and our proposed methodology offers valuable insights into their behavior in various flow conditions. Our novel owning-cell locator method enables efficient, constant-time cell search, and the algorithm draws inspiration from classical search algorithms and modern multi-level approaches in numerical linear algebra. The proposed method is implemented for both multi-core CPUs and Nvidia GPUs, demonstrating strong scaling up to 32,768 CPU cores and up to 62 Nvidia V100 GPUs. By decoupling particle advection from other problems, we achieve modularity and extensibility, resulting in consistent parallel efficiency across different architectures. Our methodology was applied to calculate and visualize the FTLE on four turbulent boundary layers at different Reynolds and Mach numbers, revealing that coherent structures grow more isotropic proportional to the Mach number, and their inclination angle varies along the streamwise direction.
We also observed increased anisotropy and FTLE organization at lower Reynolds numbers, with structures retaining coherency along both spanwise and streamwise directions. Additionally, we demonstrated the impact of lower temporal frequency sampling by upscaling with an efficient linear upsampler, preserving general trends with only 10% of the required storage. In summary, we present a particle search scheme for particle advection workloads in the context of visualizing LCS via FTLE
that exhibits strong scaling performance and efficiency at scale. Our proposed algorithm is applicable across various domains requiring efficient search algorithms in large structured domains. While this manuscript focuses on the methodology and its application to LCS, an in-depth study of the physics and compressibility effects in LCS candidates will be explored in a future publication.
more »
« less
This content will become publicly available on March 1, 2025
Aquila-LCS: GPU/CPU-Accelerated Particle Advection Schemes for Large-Scale Simulations
We introduce Aquila-LCS, GPU and CPU optimized object-oriented, in-house codes for volumetric particle advection and 3D Finite-Time Lyapunov Exponent (FTLE) and Finite-Size Lyapunov Exponent (FSLE) computations. The purpose is to analyze 3D Lagrangian Coherent Structures (LCS) in large Direct Numerical Simulation (DNS) data. Our technique uses advanced search strategies for quick cell identification and efficient storage techniques. This solver scales effectively on both GPUs (up to 62 Nvidia V100 GPUs) and multi-core CPUs (up to 32,768 CPU cores), tracking up to 8-billion particles. We apply our approach to four turbulent boundary layers at different flow regimes and Reynolds numbers.
more »
« less
- NSF-PAR ID:
- 10500004
- Publisher / Repository:
- SSRN
- Date Published:
- Journal Name:
- Software X
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
more » « less
In this work, we introduce a scalable and efficient GPU-accelerated methodology for volumetric particle advection and finite-time Lyapunov exponent (FTLE) calculation, focusing on the analysis of Lagrangian coherent structures (LCS) in large-scale direct numerical simulation (DNS) datasets across incompressible, supersonic, and hypersonic flow regimes. LCS play a significant role in turbulent boundary layer analysis, and our proposed methodology offers valuable insights into their behavior in various flow conditions. Our novel owning-cell locator method enables efficient constant-time cell search, and the algorithm draws inspiration from classical search algorithms and modern multi-level approaches in numerical linear algebra. The proposed method is implemented for both multi-core CPUs and Nvidia GPUs, demonstrating strong scaling up to 32,768 CPU cores and up to 62 Nvidia V100 GPUs. By decoupling particle advection from other problems, we achieve modularity and extensibility, resulting in consistent parallel efficiency across different architectures. Our methodology was applied to calculate and visualize the FTLE on four turbulent boundary layers at different Reynolds and Mach numbers, revealing that coherent structures grow more isotropic proportional to the Mach number, and their inclination angle varies along the streamwise direction. We also observed increased anisotropy and FTLE organization at lower Reynolds numbers, with structures retaining coherency along both spanwise and streamwise directions. Additionally, we demonstrated the impact of lower temporal frequency sampling by upscaling with an efficient linear upsampler, preserving general trends with only 10% of the required storage. In summary, we present a particle search scheme for particle advection workloads in the context of visualizing LCS via FTLE that exhibits strong scaling performance and efficiency at scale. Our proposed algorithm is applicable across various domains, requiring efficient search algorithms in large, structured domains. While this article focuses on the methodology and its application to LCS, an in-depth study of the physics and compressibility effects in LCS candidates will be explored in a future publication.
-
more » « less
Abstract During the 2019/2020 Australian bushfire season, intense wildfires generated a rising plume with a record concentration of smoke in the lower stratosphere. Motivated by this event, we use the atmospheric wind reanalysis model ERA5 to characterize the three dimensional atmospheric transport in the general region of the plume following a dynamical system approach in the Lagrangian framework. Aided by the Finite Time Lyapunov Exponent tool (FTLE), we identify Lagrangian Coherent Structures (LCS) which simplify the three‐dimensional transport description. Different reduced FTLE formulations are compared to study the impact of the vertical velocity and the vertical shear on the movement of the plume. We then consider in detail some of the uncovered LCS that are directly relevant for the evolution of the plume, as well as other LCS that are less relevant for the plume but have interesting geometries, and we show the presence of 3D lobe dynamics at play. Also, we unveil the qualitatively different dynamical fates of the smoke parcels trajectories depending on the region in which they originated. One feature that had a pronounced influence on the evolution of the smoke plume is a synoptic‐scale anticyclone that was formed near the same time as, and close to the region of, intense wildfires. We analyze this anticyclone in detail, including its formation, the entrainment of the smoke plume, and how it maintained coherence for a long time. Transport paths obtained with the inclusion of the buoyancy effects are compared with those obtained considering only the reanalysis velocity.
-
High-speed, spatially-evolving turbulent boundary layers are of great importance across civilian and military applications. Furthermore, compressible boundary layers present additional challenges for energy and active scalar transport. Understanding transport phenomena is critical to efficient high-speed vehicle designs. Although at any instantaneous point in time a flow field may seem random, regions within the flow can exhibit coherency across space and time. These coherent structures play a key role in momentum and energy transport within the boundary layer. The two main categories for coherent structure identification are Eulerian and Lagrangian approaches. In this video, we focus on 4D (3D+Time) Lagrangian Coherent Structure (LCS), and the effect of wall curvature/temperature on these structures. We present the finite-time Lyapunov exponent (FTLE) for three wall thermal conditions (cooling, quasi-adiabatic and heating) for a concave wall curvature that builds on the experimental study by Donovan et al. (J. Fluid Mech., 259, 1-24, 1994). The flow is subject to a strong concave curvature (δ/R ~ -0.083, R is the curvature radius) followed by a very strong convex curvature (δ/R = 0.17). A GPU-accelerated particle simulation forms the basis for the 3-D FTLE where particles are advected over flow fields obtained via Direct Numerical Simulation (DNS) with high spatial/temporal resolution. We also show the cross-correlation between Q2 events (ejections) and the FTLE. The video is available at: https://gfm.aps.org/meetings/dfd-2022/63122e0e199e4c2da9a946a0more » « less
-
In this study, we delve into the intricate relation between Lagrangian Coherent Structures (LCS), primarily represented by the finite-time Lyapunov exponent (FTLE), and instantaneous temperature in turbulent wall-bounded flow scenarios. Turbulence, despite its chaotic facade, houses coherent structures vital to understanding the dynamical behavior of fluid flows. Recognizing this, we leverage high-fidelity Direct Numerical Simulation (DNS) to investigate compressible flows, focusing on the attracting manifolds in FTLE and their correlation with instantaneous temperature. The consequent insights into the coupling between fluid dynamics and thermodynamics reveal the profound influence of vortex stretching, shearing, and compression on local thermodynamic characteristics. Notably, the interplay of instantaneous static temperature and fluid properties, along with the cascading nature of energy in turbulent flows, underpins the observed correlation. Furthermore, we leveraged a high-performance, scalable volumetric particle advection scheme for LCS determination in subsonic (M∞ = 0.8) and supersonic (M∞ = 1.6) turbulent boundary layers over adiabatic flat plates.more » « less
