1 of 1

ytopt: A ML-based Autotuning for Energy Efficiency�With the NP Partnership (Femtoscale)

Scientific Achievement

As the complexity of HPC ecosystems continues to rise, achieving optimal performance and energy becomes a challenge. Large system and application parameter space requires few shots for automatic exploration of the space. ytopt is our autotuning effort to tune scientific applications for energy efficiency.

Significance and Impact

Our low-overhead ytopt framework can be used to identify the best combination of application and system parameters to result in the best performance. This helps not only improve application performance portability on different architectures but also explore performance tradeoffs for efficiently utilizing underlying HPC systems.

This table presents the energy improvement percentage for using ytopt to autotune four ECP proxy apps in energy and EDP (Energy Delay Product) as the performance metrics on up to 4096 nodes on ALCF Theta.

Technical Approach

Utilize Bayesian Optimization for search and Random Forests for model
Use few shots strategy to identify the best combination of parameters
Explore tradeoffs between application runtime and power/energy

X. Wu, M. Kruse, P. Balaprakash, H. Finkel, P. Hovland, V. Taylor, and M. Hall, "Autotuning PolyBench benchmarks with LLVM Clang/Polly loop optimization pragmas using Bayesian optimization" Concurrency and Computation. Practice and Experience, Volume 34, Issue 20, 2022. ISSN 1532-0626 DOI: 10.1002/cpe.6683

PI(s)/Facility Lead(s): Robert Ross; Local Lab POC: Xingfu Wu, ANL

Collaborating Institutions: ANL, LBNL, ORNL,

ASCR Program: [SciDAC RAPIDS2 and OASIS, ECP]

ASCR PM: Lali Chatterji, Kalyan Perumalla

Publication(s) for this work: X. Wu, et al., “ytopt: Autotuning Scientific Applications for Energy Efficiency at Large Scales”, Proceeding of Cray User Group Conference 2023 (CUG’23) doi:10.48550/arXiv.2303.16245.

and/or Code Developed or Datasets: https://github.com/ytopt-team/ytopt

Metric	XSBench	SWFFT	AMG	SW4lite
Energy	8.6%	2.1%	20.9%	21.2%
EDP	37.8%	5.2%	24.1%	23.7%

This diagram shows the ytopt framework using Bayesian Optimization for the search and Random Forests for the surrogate model to identify the best combination of application and system parameters for energy efficiency

LOCAL LAB POC: Xingfu Wu, ANL

TALKING POINTS:

improve application performance portability on different architectures
explore performance tradeoffs for efficiently utilizing underlying HPC systems
Explore tradeoffs between application runtime and power/energy
Utilize Bayesian Optimization for search and Random Forests for model
Use few shots strategy to identify the best combination of parameters

METADATA:

Name of the associated awarded project: SciDAC RAPIDS2 and OASIS

PI name(s): Robert Ross

Name of the program manager: Lali Chatterji, Kalyan Perumalla

AWARDS:

This work has been selected including in Journal CCPE special issue in 2024.

REPRODUCIBILITY:

We setup the GitHub repo for this work: https://github.com/ytopt-team/ytopt

BACKGROUND AND CONTEXT INFORMATION:

As the complexity of high performance computing (HPC) ecosystems continues to rise, achieving optimal performance becomes a challenge.

The number of tunable parameters an HPC user can configure has increased significantly, resulting in a dramatically increased parameter space.

Exhaustively evaluating all parameter combinations becomes very time-consuming.

Solution: autotuning for automatic exploration of parameter space is necessary.

Current large-scale HPC systems such as Polaris, Sunspot and Aurora at ANL and Crusher and Frontier at ORNL have complex system architectures and software stacks with many tunable parameters that may affect the system performance and energy.

Application developers and users often rely on these systems with the default configurations setup by the vendors to run their applications. How efficiently are these applications executed?

How can we identify the best combination of these parameters for the best system performance or the lowest system energy consumption?

Can we develop a framework to autotune large-scale applications for performance or energy efficiency on large-scale HPC production systems such as Frontier and Aurora?

This work is to address these questions.