An official website of the United States government
Scientific simulations and observations are generating massive volumes of time-varying vector field data, posing significant challenges for long-term storage and data transmission. Lossy compression is widely regarded as a promising approach for reducing data volume, as lossless methods typically achieve only modest compression ratios and therefore provide limited reduction. However, directly applying existing lossy compression techniques to time-varying vector fields can introduce undesirable distortions in critical-point trajectories, which encode essential structural properties of the underlying field. In this work, we present an efficient lossy compression framework that exactly preserves all critical-point trajectories in time-varying vector fields. Our contributions are threefold. First, we extend the theory of critical-point preservation from the spatial domain to space-time and develop a corresponding compression framework to guarantee trajectory preservation. Second, we introduce a semi-Lagrangian predictor to more effectively exploit spatiotemporal correlations in advection-dominated regions, and integrate it with the classical Lorenzo predictor to further improve compression efficiency. Third, we evaluate the proposed approach against state-of-the-art lossy and lossless compressors on four real-world scientific datasets. Experimental results show that our method achieves compression ratios of up to 124.48X while effectively preserving all critical-point trajectories. This compression ratio is up to 56.07X higher than the best-performing lossless compressors. In contrast, none of the existing lossy compressors can preserve all critical-point trajectories at comparable compression ratios.
more »
« less
This content will become publicly available on May 4, 2027
Time-Varying Vector Field Compression with Preserved Critical Point Trajectories
Scientific simulations and observations are generating massive volumes of time-varying vector field data, posing significant challenges for long-term storage and data transmission. Lossy compression is widely regarded as a promising approach for reducing data volume, as lossless methods typically achieve only modest compression ratios and therefore provide limited reduction. However, directly applying existing lossy compression techniques to time-varying vector fields can introduce undesirable distortions in critical-point trajectories, which encode essential structural properties of the underlying field. In this work, we present an efficient lossy compression framework that exactly preserves all critical-point trajectories in time-varying vector fields. Our contributions are threefold. First, we extend the theory of critical-point preservation from the spatial domain to space-time and develop a corresponding compression framework to guarantee trajectory preservation. Second, we introduce a semi-Lagrangian predictor to more effectively exploit spatiotemporal correlations in advection-dominated regions, and integrate it with the classical Lorenzo predictor to further improve compression efficiency. Third, we evaluate the proposed approach against state-of-the-art lossy and lossless compressors on four real-world scientific datasets. Experimental results show that our method achieves compression ratios of up to 124.48X while effectively preserving all critical-point trajectories. This compression ratio is up to 56.07X higher than the best-performing lossless compressors. In contrast, none of the existing lossy compressors can preserve all critical-point trajectories at comparable compression ratios.
more »
« less
- PAR ID:
- 10683914
- Publisher / Repository:
- IEEE
- Date Published:
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Lossy compression has been employed to reduce the unprecedented amount of data produced by today's large-scale scientific simulations and high-resolution instruments. To avoid loss of critical information, state-of-the-art scientific lossy compressors provide error controls on relatively simple metrics such as absolute error bound. However, preserving these metrics does not translate to the preservation of topological features, such as critical points in vector fields. To address this problem, we investigate how to effectively preserve the sign of determinant in error-controlled lossy compression, as it is an important quantity of interest used for the robust detection of many topological features. Our contribution is three-fold. (1) We develop a generic theory to derive the allowable perturbation for one row of a matrix while preserving its sign of the determinant. As a practical use-case, we apply this theory to preserve critical points in vector fields because critical point detection can be reduced to the result of the point-in-simplex test that purely relies on the sign of determinants. (2) We optimize this algorithm with a speculative compression scheme to allow for high compression ratios and efficiently parallelize it in distributed environments. (3) We perform solid experiments with real-world datasets, demonstrating that our method achieves up to 440% improvements in compression ratios over state-of-the-art lossy compressors when all critical points need to be preserved. Using the parallelization strategies, our method delivers up to 1.25X and 4.38X performance speedup in data writing and reading compared with the vanilla approach without compression.more » « less
-
The objective of this work is to develop error-bounded lossy compression methods to preserve topological features in 2D and 3D vector fields. Specifically, we explore the preservation of critical points in piecewise linear and bilinear vector fields. We define the preservation of critical points as, without any false positive, false negative, or false type in the decompressed data, (1) keeping each critical point in its original cell and (2) retaining the type of each critical point (e.g., saddle and attracting node). The key to our method is to adapt a vertex-wise error bound for each grid point and to compress input data together with the error bound field using a modified lossy compressor. Our compression algorithm can be also embarrassingly parallelized for large data handling and in situ processing. We benchmark our method by comparing it with existing lossy compressors in terms of false positive/negative/type rates, compression ratio, and various vector field visualizations with several scientific applications.more » « less
-
With ever-increasing volumes of scientific floating-point data being produced by high-performance computing applications, significantly reducing scientific floating-point data size is critical, and error-controlled lossy compressors have been developed for years. None of the existing scientific floating-point lossy data compressors, however, support effective fixed-ratio lossy compression. Yet fixed-ratio lossy compression for scientific floating-point data not only compresses to the requested ratio but also respects a user-specified error bound with higher fidelity. In this paper, we present FRaZ: a generic fixed-ratio lossy compression framework respecting user-specified error constraints. The contribution is twofold. (1) We develop an efficient iterative approach to accurately determine the appropriate error settings for different lossy compressors based on target compression ratios. (2) We perform a thorough performance and accuracy evaluation for our proposed fixed-ratio compression framework with multiple state-of-the-art error-controlled lossy compressors, using several real-world scientific floating-point datasets from different domains. Experiments show that FRaZ effectively identifies the optimum error setting in the entire error setting space of any given lossy compressor. While fixed-ratio lossy compression is slower than fixed-error compression, it provides an important new lossy compression technique for users of very large scientific floating-point datasets.more » « less
-
null (Ed.)With ever-increasing volumes of scientific floating-point data being produced by high-performance computing applications, significantly reducing scientific floating-point data size is critical, and error-controlled lossy compressors have been developed for years. None of the existing scientific floating-point lossy data compressors, however, support effective fixed-ratio lossy compression. Yet fixed-ratio lossy compression for scientific floating-point data not only compresses to the requested ratio but also respects a user-specified error bound with higher fidelity. In this paper, we present FRaZ: a generic fixed-ratio lossy compression framework respecting user-specified error constraints. The contribution is twofold. (1) We develop an efficient iterative approach to accurately determine the appropriate error settings for different lossy compressors based on target compression ratios. (2) We perform a thorough performancemore » « less
