1 of 1

ML-based Performance Portability for Density Functional Theory in HPC Environments (DECODE SciDAC-5)

Adrian P. Dieguez∗, Min Choi, Xinran Zhu, Bryan M. Wong and Khaled Z Ibrahim “ML-based Performance Portability for Time-Dependent Density Functional Theory in HPC Environments" 2022 International Workshop on Performance Modeling, Benchmarking and Simulation of High Performance Computer Systems (PMBS), Supercomputing 2022.

3D FFT mapping in rt-TDDFT

The internal process for searching the optimal space using Bayesian Optimization. Transfer learning leverages configuration on one platform to find optimal configuration on another.

ASCR-BES DECODE SciDAC-5

The RT-DFT Tuning stack for optimizing the performance on HPC machines.

Scientific Achievement

Developed a framework for optimizing the search of runtime configurations and code variants to improve the performance portability on HPC platforms for RT-TDDFT codes.

Significance and Impact

Improving the search speed for optimal performance configuration is critical for performance portability for applications with large space of algorithmic variants and runtime configurations. The developed methodology, despite focusing on RT-TDDFT computation, can be used with a wide set of applications.

Research Details

Optimized search of the optimization space can leverage a wide set of techniques including Bayesian optimization and model-based estimation.
Developed a framework to do transfer-learning, where the search on an HPC platform is used to reduce the number of search attempts on another platform.
Focused on finding optimized configurations for the calculation involved in the conjugate gradient calculation part of rt-TDDFT.
Improved the performance by up to 9x, reducing the search attempts by up to 86%.

Optimizing the performance of scientific computing on DOE HPC machines is a daunting process due to the diverse architectures and number of algorithmic variants and runtime configurations.

This work develops a framework that leverages transfer learning techniques to optimize the search for optimal configurations.

Our efforts focused in real-time time-dependent density functional theory, where the computation is dominated by many small 3D FFT computation.

Orchestrating the computation is associated with a large number of tuning parameters that impact performance significantly by up to 9x.

Leveraging expensive exploration on one platform on the tuning in another is critical for efficient use of DOE resources.

We improved the search speed by up to 86% using the developed transfer learning technique finding a configuration that is 9x faster than the baseline configuration.

Figure legend: The top figures show the infrastructure we developed to do transfer learning to leverage the search on a platform for tuning another. The rt-TDDFT computation is dominated by many samll 3D FFT, as shown in the Figure on the bottom left. The detailed steps for tuning based on Bayesian optimization is shown in the bottom right Figure.

ACKNOWLEDGMENTS: This material is based upon work supported by the Advanced Scientific Computing Research Program in the U.S. Department of Energy, Office of Science, under Award Number DE-AC02-05CH11231 and used resources of the National Energy Research Scientific Computing Center (NERSC) which is supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC02-05CH11231. The TDDFT developments on QBox were supported by the U.S. Department of Energy, Office of Science, Office of Advanced Scientific Computing Research, Scientific Discovery through the Advanced Computing (SciDAC) program under Award Number DE-SC0022209.

The work was done at Lawrence Berkeley National Laboratory, University of California Riverside, and University of California Berkeley.