FZ: �A fine-tunable cyberinfrastructure framework to streamline specialized lossy compression development �

Franck Cappello, Sheng Di, Robert Underwood

Dingwen Tao

Hanqi Guo

Kai Zhao

Motivations

Trend

Many research domains (beyond CS) are exploring the use of data reduction techniques to address these challenges.

​

However, existing generic lossy compressors (e.g. ZFP) often do not correspond to user-specific applications, use cases, and requirements

​

Hence, many users (NASA, HEP) develop their own specialized lossy compression software 🡪

• requires tremendous collaborations between compressor experts and domain scientists,
• demands extensive coding to optimize performance on multiple platforms, and
• often leads to redundant research and development efforts.

FZ Objectives

-intuitively research, compose, implement, and test specialized lossy compressors

​

-from a library of predeveloped, high-performance data reduction modules optimized for heterogeneous platforms

Provide a comprehensive ecosystem to enable users to

Thrusts

1. Develop programming interfaces and a compressor generator to create new compressors from high-level languages (e.g., Python) and optimize their execution;

​

(2) Refactor the SZ lossy compressors infrastructure to enable fine-grained composability of a large diversity of data transformation modules and integrate nonuniform compression capabilities (e.g., feature-based differentiated compression), new preprocessing

(e.g., preconditioning), decorrelation (e.g., prediction, transform, decomposition), approximation

(e.g., quantization), and entropy coding data transformation modules to produce specialized lossy compressors;

​

(3) Provide interactive visualization, quality assessment, and graphical user interface

(GUI) tools that adapt and extend existing capabilities:

• OptZConfig to automatically search optimized lossy compression module compositions,
• Z-checker/QCAT compression quality assessment tools with advanced Paraview/VTK visualization support to provide users with a powerful tool to identify relevant compression ratio, speed, and quality trade-o s for their use cases

Framework Architecture

3 main components:

1. Expression/compilation interface for the composition

​

• Library of compressor modules

​

• Visualization/optimization/Web repo.

​

Framework Architecture

1. Users use LibPressio as the input interface

(2) Users describe the compression composition using

either the FZ-DSL1 description language to concisely

express compositions (Python)

(3) and/or the Z-Vis graphical interface for interactive exploration of designs

(4) (5) assemble modules from the FZ module library,

(6) potentially browse compression modules and compositions on Z-Hub

(7) The compressor generator produces compressors (8)

(9) Z-opt optimizes composition and parameters

(10) until the specialized lossy compressor is finalized

8 Applications

(1) Cosmology (NYX)

(2) Climate (CESM)

(3) Seismology (SCOPED)

(4) Plasma Physics (SGTC, XGC)

(5) Molecular Dynamics (LAMMPS)

(6) Materials Science (LCLS, APS)

(7) Quantum Simulation (Q-Tensor)

(8) Combustion (BLASTNET)

Tasks

Relation to SZ4

Today’s focus:

1. Programming interfaces and a compressor generator

Robert Underwood, Dingwen Tao

​

​

(2) SZ lossy compressors infrastructure (compression module library)

Kai Zhao, Martin Burtscher, Jon Calhoun, Xin Liang

​

​

(3) Interactive visualization, quality assessment, and optimization

Hanqi Guo, Xiaodong Liu, Robert Underwood, Sheng Di, and more