Distributed and Dependable SDS �Control Plane for HPC

5^th April 2022

16th EuroSys Doctoral Workshop (EuroDW 22)

Supervised by

João Tiago Paulo and José Orlando Pereira

HASLab, INESCTEC & University of Minho

Mariana Miranda

mariana.m.miranda@inesctec.pt

HASLab, INESCTEC & University of Minho

​

​

High-Performance Computing

​

• Essential tool in many scientific and industrial advancements

​

• Large magnitude of computing power

​

• Comprise hundreds to thousands of compute and storage nodes

​

​

Challenges:

High I/O concurrency:

I/O Interference and performance degradation

​

Complexity:

Hinders global I/O optimizations and the guarantees of the Quality-of-Service requirements

​

​

​

​

​

​

​

2

Distributed and Dependable SDS Control Plane for HPC – Mariana Miranda

High-Performance Computing

​

• Essential tool in many scientific and industrial advancements

​

• Large magnitude of computing power

​

• Comprise hundreds to thousands of compute and storage nodes

​

​

• Challenges:
• High I/O concurrency:

I/O Interference and performance degradation

​

Complexity:

Hinders global I/O optimizations and the guarantees of the Quality-of-Service requirements

​

​

​

​

​

​

​

3

Distributed and Dependable SDS Control Plane for HPC – Mariana Miranda

High-Performance Computing

​

• Essential tool in many scientific and industrial advancements

​

• Large magnitude of computing power

​

• Comprise hundreds to thousands of compute and storage nodes

​

​

• Challenges:
• High I/O concurrency:

I/O Interference and performance degradation

​

• Complexity:

Hinders global I/O optimizations and the Quality-of-Service (QoS) requirements

​

​

​

​

​

​

​

4

Distributed and Dependable SDS Control Plane for HPC – Mariana Miranda

High-Performance Computing

The storage issues that HPC systems are currently facing go hand-in-hand with the improvements that Software-Defined Storage (SDS) aims to deliver.

5

Challenges

Distributed and Dependable SDS Control Plane for HPC – Mariana Miranda

Software-Defined Storage

​

• Handles the control logic

​

• Global visibility over the system

​

• Ability to holistically orchestrate the storage services

​

6

Control Plane

Data Plane

Control Application

<<<

Stage

Controller

Controller

Controller

Overview

Control Application

<<<

Stage

<<<

Stage

​

• Implements and executes the policies defined by the control plane

Distributed and Dependable SDS Control Plane for HPC – Mariana Miranda

Software-Defined Storage

​

• Handles the control logic

​

• Global visibility over the system

​

• Ability to holistically orchestrate the storage services

​

7

Control Plane

Data Plane

Control Application

<<<

Stage

Controller

Controller

Controller

Overview

Control Application

<<<

Stage

<<<

Stage

​

• Implements and executes the policies defined by the control plane

Distributed and Dependable SDS Control Plane for HPC – Mariana Miranda

Software-Defined Storage

​

• Handles the control logic

​

• Global visibility over the system

​

• Ability to holistically orchestrate the storage services

​

8

Control Plane

Data Plane

Control Application

<<<

Stage

Controller

Controller

Controller

Overview

Control Application

<<<

Stage

<<<

Stage

​

• Implements and executes the policies defined by the control plane

Distributed and Dependable SDS Control Plane for HPC – Mariana Miranda

Software-Defined Storage

​

• The design of the control plane is usually overlooked

​

​

• SDS systems provide a very superficial solution
• Present themselves as “logically centralized”
• Do not discuss its implementation and components
• Generally overlook scalability and dependability properties

​

​

​

​

​

​

9

Control Plane

Data Plane

Control Application

<<<

Stage

Controller

Controller

Controller

Overview

Control Application

<<<

Stage

<<<

Stage

Distributed and Dependable SDS Control Plane for HPC – Mariana Miranda

Ph.D. Goals

​

​

​

10

​

​

• Propose a control plane suitable for HPC systems

​

• Explore scalability and dependability mechanisms

​

• Apply control algorithms

(e.g., I/O prioritization, bandwidth guarantees, latency control, among others)

​

• Establish a high-level language to specify storage policies

​

​

Distributed and Dependable SDS Control Plane for HPC – Mariana Miranda

Distributed and Dependable SDS �Control Plane for HPC

5^th April 2022

16th EuroSys Doctoral Workshop (EuroDW 22)

Supervised by

João Tiago Paulo and José Orlando Pereira

HASLab, INESCTEC & University of Minho

Mariana Miranda

mariana.m.miranda@inesctec.pt

HASLab, INESCTEC & University of Minho

​

​