1 of 1

Region-adaptive, Error-controlled Scientific Data Compression for Scientific Data

Significance and Achievement

Built an adaptive data compression pipeline upon the multilevel compressor MGARD with automatic critical region detection and spatially varying error bounds and demonstrated that data generated by E3SM can be greatly reduced while the quantities of interest (QoI) by post-analysis are preserved.

Technical Approach

Developed a region-adaptive compression method which can detect critical regions and compress data using spatially varying error bounds
Derived mathematical equations and compression errors bounds for region-adaptive accuracy control
Evaluated on two climate use cases showing improved data compression ratios and lower errors in post-analysis compared to single-error-bounded approaches

Significance and Impact

Comparing to existing single-error-bounded lossy compressors, we derive theories of point-wise error preservation and develop a pipeline capable of delivering larger compression ratios and preserving the accuracy of post-processing by applying spatially changed error bound for data inside and outside regions of interest (RoI)

"Region-adaptive, Error-controlled Scientific Data Compression using Multilevel Decomposition." Gong, Qian, et al. SSDBM 2022.

Work was performed at Oak Ridge National Laboratory

Fig 1. Region-adaptive compression pipeline

Fig 2. TC detection: adaptive vs uniform compression

Fig 2. AR detection: adaptive vs uniform compression

TALKING POINTS:

As recent advancements in computational power continue to outpace those in storage and I/O systems, scientific discoveries are increasingly constrained by the amount of data that can be stored and efficiently retrieved.
Data compression has been regarded as one of the most effective ways to address data challenges. However, the conventional uniform-error-bounded compression cannot provide sufficient compression ratios needed by scientific applications.
Because not all data is equally important for science, we develop a novel multilevel compressor which integrates a critical region detection to capture variational features in space-time along with error preservation theories to preserve varied precision for different regions during data compression.
We evaluated the proposed work with a real-world application, DOE’s flagship climate model – E3SM -- on two climate extreme event studies. The results show that our method can substantially reduce the storage size and improves the quality of climate analysis, comparing to uniform-error-bounded compression.