1 of 24

Centre for High Performance Computing

An Untapped Opportunity for

Humanities and Social Sciences?

DH-IGNITE – WC Region

SA Infrastructure for Digital/Computational Research

​

​

​

​

​

Dr Werner Janse van Rensburg

(CHPC Research Manager)

wjvrensburg@csir.co.za

9 March 2023

​

www.chpc.ac.za

2 of 24

2

Overview

  • Introduction
  • Background
  • Compute Infrastructure
  • Users and Usage
  • Training and Development
  • Practical Access to the CHPC

3 of 24

3

CHPC Background

4 of 24

4

  • CHPC is national HPC facility funded by the SA Government
  • Department of Science and Innovation (DSI)
  • National Integrated Cyber-Infrastructure System (NICIS)
  • Administered by the CSIR
  • Started operations in June 2007 and based in Cape Town
  • Until recently hosted the largest HPC system in Africa
  • CHPC has total number of ~35 employees
  • CHPC has Research, Technical and Operational divisions

Introduction to CHPC

5 of 24

5

Introduction to CHPC

6 of 24

6

SANReN

(Networking Services)

CHPC

(Computing Services)

DIRISA

(Data Services)

The Cyber-Infrastructure

Triangle…

NATIONAL INTEGRATED CYBERINFRASTRUCTURE SYSTEM

Introduction to CHPC

www.nicis.ac.za

​

www.sanren.ac.za

​

www.dirisa.ac.za

​

NICIS: National Integrated

Cyber-Infrastructure System…

www.chpc.ac.za

​

7 of 24

7

  • Research Enabler Role via HPC
  • Provisioning of Computational Resources to:
    • South African Academic Research Community
    • African SKA and and SADC Partner Countries
    • Non-Academic Public and Private Sector Users (in SA)
  • Research enabling role of the CHPC fourfold:
    • Access to HPC hardware infrastructure
    • Code/Software support
    • Research domain area support
    • Training and education initiatives (Human Capital Development)

CHPC Mandate

8 of 24

8

CHPC Research Compute Infrastructure

9 of 24

9

  • Lengau cluster since 7 March 2017 (1st phase June 2016)
  • Africa’s 1st Petascale system ⇒ 1.029 PFlops = 1029 TFlops

(121st on Top500* – Jun 2016)

(127th on Top 500* – Jun 2017)

Current Resources at the CHPC:

Introduction to CHPC

10 of 24

10

CHPC Lengau Compute Cluster

​

​

Lengau: Setswana for ‘Cheetah’

​

​

Emphasis on ‘Speed’ and ‘The Fastest’…

​

​

Fastest HPC Cluster on African Continent

​

​

1.029 PFlops = 1029 TFlops

​

Slide #10

___________________

CHPC

11 of 24

11

System Configuration

Phase 1

Phase 2

Dell PowerEdge C6320 Servers:

​

​

Standard Compute nodes 128GB (64GB) / node

1 008

1 368

2 x Intel Xeon E5-2690 v3 (Haswell) processors

(12 Cores Each ⇒ 24 cores / node)

24 192

32 832

Dell PowerEdge R930 servers:

​

​

Large Memory Compute Nodes 1024GB / node (FAT nodes)

5

5

4 x Intel Xeon E7-4850 v3 processors

(14 Cores Each ⇒ 56 cores / node)

280

280

Infiniband FDR 2:1 Blocking (56 Gbps)

​

​

Parallel Storage (Useable) PB

4

4

Total Number of Racks (including Compute, Login, Management and Storage Nodes)

19

24

Centos 7.1 with Bright Cluster Manager and Altair PBS Pro

​

​

Total Linpack Performance (Tflop/s)

783

1029

April 2016

June 2017

Lengau CPU Cluster

Resources at the CHPC:

Introduction to CHPC

Slide #11

___________________

CHPC

12 of 24

12

  • Graphical Processor Unit (GPU) Cluster:

30 NVIDIA V100’s

​

  • Since September 2018

​

  • Usage Demand:
    • Transfer of Chemistry Users (MD)
    • Resources for Machine Learning (ML)

Resources at the CHPC: GPU Cluster

Introduction to CHPC

13 of 24

13

Since March 2020…

​

  • Openstack Cloud with Ceph storage
  • 320 cores with memory of 4GB/core 
  • 500 TB of Ceph storage (2 copies)
  • 100 Gbps speed form switch split in 4

🡺 each node gets 25GB network speed.

NICIS Production Cloud named: SEBOWA

Resources at the CHPC: Cloud

Introduction to CHPC

14 of 24

14

CHPC Users and Usage

15 of 24

15

  • Total Active* Research Programmes 520

​

  • Total Core Hours Used (Million) 1175

  • Total Active Users 2018

Usage of CHPC Lengau Cluster in Numbers

ALL TIME

* Active refers to usage of more >1000 core hours

From Apr 2016…

[Status: 1 Mar 2023]

16 of 24

16

CHPC Cluster Usage: Past 6 Years Trend

* Active refers to usage of more >1000 core hours

17 of 24

CHPC User Categories Total Active* Programmes

*Active refers to at least 1000 compute hours used over the relevant period.

2016/17 – 2021/22

Past 6 Years

# Active Programmes

Total: 458

18 of 24

CHPC Use: SA Universities Total Active* Programmes

*Active refers to at least 1000 compute hours used over the relevant period.

2016/17 – 2021/22

Past 6 Years

# Active Programmes

Total: 323

19 of 24

CHPC Use: SA Universities Total Active* Programmes

*Active refers to at least 1000 compute hours used over the relevant period.

2016/17 – 2021/22

Past 6 Years

# Active Programmes

Total: 323

20 of 24

CHPC Domain Areas: Total Active* Programmes

*Active refers to at least 1000 compute hours used over the relevant period.

2016/17 – 2021/22

Past 6 Years

# Active Programmes

Total: 458

21 of 24

21

CHPC Training and Development

…if time allows…

22 of 24

22

CHPC Training and Development Initiatives

  • Dedicated CHPC Training events
    • Introductory Programming School (Summer School) +
    • Winter School in Parallel Programming
    • HPC Ecosystems Project*
    • Student Cluster Competition (SCC)*
    • Domain specific workshops / Ad Hoc Training
  • CHPC National Conference*
  • SKA Readiness program
  • Women in HPC (WHPC) Program
  • Outreach Programmes

23 of 24

23

Practical Access to CHPC

  • Research Programmes
  • Faculty members at Academic Institutions (PI’s)
  • (Post-Graduate) Students

​

Practical Aspects*:

  • Registration Procedures
  • Principal Investigator vs User
  • Research Programme Allocations
  • Scheduling (Queues, Walltimes, Limits)
  • Storage (Lustre and Home Limits)
  • Allocation Evaluation
  • Helpdesk

https://users.chpc.ac.za

www.chpc.ac.za

wjvrensburg@csir.co.za

helpdesk@chpc.ac.za

24 of 24

24

Thank You!