1 of 1

Understanding HPC I/O Performance Models�

Scientific Achievement

Developed approaches to better characterize and model I/O performance of applications run on HPC systems. We propose a taxonomy comprised of five categories of I/O modeling errors: poor models of application behavior, global system disturbances, inadequate training set coverage, external disturbances such as I/O contention, and inherent system noise.

Significance and Impact

I/O efficiency is crucial to productivity in scientific computing, but rising system complexity and application complexity make it difficult for practitioners to understand and optimize I/O behavior at scale. The developed taxonomy and litmus tests can help I/O researchers to assess the impact of each class of errors, enhance I/O throughput models, and improve future generations of HPC logging and analysis tools.

Technical Approach

We analyze four years of application, scheduler, and storage systems logs on two production leadership-class HPC platforms – ALCF Theta and NERSC Cori.
Application Modeling Errors: Characterizes the quality of mapping between application and ideal HPC system; System Modelling Errors: Characterizes the effects of non-stationarity in data; Generalization Errors: Characterize the ability of the model to generalize beyond the trained scenarios/ systems; Contention Errors: Characterize the effect of lack of visibility into job interactions; Inherent Noise Errors: Characterize the effect of random behavior by the system
Our test show that Theta and Cori jobs have overall I/O throughput standard deviation of 5.7% and 7.2%, respectively.�

Fig. 1: Taxonomy of I/O throughput modeling errors, with examples of the effects of each class of error

Fig. 1: Results of errors breakdown from ALCF Theta and NERSC Cori systems

Isakov, Mi, M. Currier, E. Rosario, S. Madireddy, P. Balaprakash, P. Carns, R. Ross, G. Lockwood, and M. Kinsy (2022). “A Taxonomy of Error Sources in HPC I/O Machine Learning Models”. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis. (Super Computing 2022)

TALKING POINTS:

I/O efficiency is crucial to productivity in scientific computing,
Increasing complexity of the system and the applications makes it difficult for

practitioners to understand and optimize I/O behavior at scale.

Data-driven machine learning-based I/O throughput models offer a solution: they can be used

to identify bottlenecks, automate I/O tuning, or optimize job scheduling with

minimal human intervention.

Unfortunately, current state-of-the-art I/O models are not robust enough for production use and underperform after being deployed.
We analyze four years of application, scheduler, and storage system logs on two

leadership-class HPC platforms to understand why I/O models underperform in

practice.

We propose a taxonomy consisting of five categories of I/O modeling

errors: poor application and system modeling, inadequate dataset coverage, I/O

contention, and I/O noise.

We develop litmus tests to quantify each category,

allowing researchers to narrow down failure modes, enhance I/O throughput

models, and improve future generations of HPC logging and analysis tools.

METADATA:

Name of the associated awarded project: RAPIDS

PI name(s): Robert Ross (ANL), Prasanna Balaprakash(ANL), Sandeep Madireddy (ANL

Name of the program manager: ASCR

CITATIONS:

AWARDS:

N/A

REPRODUCIBILITY:

Yes

BACKGROUND: