1 of 1

How to Achieve High I/O Throughput on Perlmutter

ECP WarpX

Scientific Achievement

We explored a massive number of options to study the I/O performance of the all-flash file system on Perlmutter at NERSC and showed that the Progressive File Layout is a convenient option to achieve the best I/O performance

Significance and Impact

Our extensive tests demonstrated that the dynamic Lustre configuration option, known as the Progressive File Layout, allows user code to achieve good performance in both benchmarks and science applications

Research Details

Benchmark: openPMD-api, VPIC-IO
Application: WarpX
Weak scale, nodes range from 64 to 512 nodes
I/O library: ADIOS, Aggregation by node or rank
File system: Lustre
Perlmutter all-flash storage system, PFL.

This plot shows a weak-scaling of study of write performance using openPMD-api benchmark, each node generates 30GB. Total file sizes range from 1.88 TB to 15 TB. Based on past experiences with Lustre system, the stripe size and stripe count combination is a leading impact factor of the I/O rates. However in this study, the Progressive File Layout (PFL) offered by the Lustre file system is able to reliably provide solid write performance.

Gu, Eisenhauer, Klasky, Podhorszki, Wang, and Wu. SSDBM 2022. DOI: 10.1145/3538712.3538734

PFL

TALKING POINTS:

Perlmutter offers a brand-new file system consisting of all flash storage devices, how to arrange the user I/O operation to achieve the best performance is of great interest to the users
Extensive tests during the Phase I deployment of Perlmutter indicates that the dynamic scheme offered by the Lustre file system (named Progressive File Layout, PFL in short) offers the better I/O rate (throughput) than nearly all combinations of parameters we have tested
Note: Perlmutter is going through a period of system testing and the availability of PFL is being affected by this testing.

METADATA:

PI name(s): R Ross (ANL), L Oliker (LBNL) SciDAC RAPIDS2

Name of the program manager: Lali Chatterjee

CITATIONS:

Junmin Gu, Greg Eisenhauer, Scott Klasky, Norbert Podhorszki, Ruonan Wang, and Kesheng Wu. 2022. Exploring Large All-Flash Storage System with Scientific Simulation. In Proceedings of the 34th International Conference on Scientific and Statistical Database Management (SSDBM '22). Association for Computing Machinery, New York, NY, USA, Article 23, 1–4. https://doi.org/10.1145/3538712.3538734

AWARDS:

None

REPRODUCIBILITY:

BACKGROUND:

Solid state storage systems have been very effectively used in small devices; however, their effectiveness for large systems such as supercomputers is not yet proven. Recently, for the first time, a new supercomputer is being deployed with an all-flash storage as its main file system. In this work, we report our preliminary study of the I/O performance on this supercomputer named Perlmutter. We are able to achieve 1.4 TB/s with the dynamic strategy named Progressive Lile Layout (PFL) for controlling the crucial file system parameters during 2022 when the Perlmutter system was going through its Phase I installation. This dynamic configuration outperforms dozens of other choices tested, which indicates that the Lustre file system has an effective way of utilizing the all-flash storage system.