Introduction to ISAAC Resources

Victor Hazlewood

Associate CIO & Director

High Performance and Scientific Computing (HPSC)

Office of Information Technology

https://oit.tennessee.edu/hpsc/

• Website & Documentation
• HPSC and ISAAC system overview
• Access and Login
• File systems and data storage
• Data Transfer
• Programming Environment

Agenda for today’s section

OIT HPSC Resources and Services

• High Performance & Scientific Computing (HPSC) group is a branch of Office of Information Technology at the University of Tennessee, Knoxville

​

• Provides research computing resources and services to the University community

https://oit.tennessee.edu/hpsc/

Website & Documentation

• Documentation: https://oit.tennessee.edu/hpsc/

Claiming an Account

• Need further help?

Go to Request an ISAAC Account.

Click on I have a UT NetID and

Follow the prompts.

​

A form will be filled in with your

information. You need to add

missing fields – citizenship, etc.

​

ISAAC Legacy account created

immediately.

How to Request Help (a service request)

Zoom office hours (3 times/week for this semester, will be posted on the website soon)

Call OIT HelpDesk at 865-974-9900 (available 24/7)

• Need further help?

News Items

Latest News Items

for HPSC

Workshop Schedule (next week) & Office Hours

https://oit.utk.edu/news/spring-2022-hpsc-workshops/

Workshops will be recorded and put

On HPSC website under

-> Overview

-> Workshops and Training

​

​

Office Hours are posted on Website

with days/times and Zoom link

which requires UTK authentication

to access

ISAAC Clusters Overview

Open research with Torque/Moab

New cluster for open research using SLURM

• ISAAC

Infrastructure for Scientific Applications and Advanced Computing�(formerly Advanced Computing Facility - ACF)

ISAAC Legacy System Overview

• ISAAC Legacy
• 303 nodes, 8,662 cores
• Memory 32GB to 1.5TB
• 2.7 PB Lustre storage
• 4 Data Transfer Nodes�with 10Gb/s networking
• 6 types of compute nodes
• 2 types of large memory nodes
• 2 types of GPU nodes

ISAAC Legacy Overview

• ISAAC Legacy cluster node types

https://oit.utk.edu/hpsc/isaac-open/open-enclave-system-overview/

Four nodes for debug: 2x beacon, 2x rho

ISAAC Next Generation Cluster

• ISAA-NG
• 73 nodes, 3,504 cores
• Memory 192 GB
• 2.9 PB Lustre storage
• 1 Data Transfer Node�with 40 Gb/s networking
• 1 compute node type
• 1 type of GPU nodes

ISAAC-NG Overview

• ISAAC-NG cluster node types

https://oit.utk.edu/hpsc/isaac-open-enclave-new-kpb/system-overview-cluster-at-kpb/

New equipment on site and to be installed this quarter

• 3 new Intel Ice Lake 2 TB large memory nodes
• 7 more 1x NVIDIA V100S GPU node with Intel Gold 6248R CPUs
• 1 more 2x NVIDIA V100S GPU nodes with Intel Gold 6248R CPUs
• 1 XL GPU node with 4x NVIDIA A40 with Intel Platinum 8358 CPUs

Methods of Access

• Access to ISAAC Legacy and ISAAC-NG
• Login with SSH client (command line access)
• Torque or SLURM batch job
• Torque or SLURM interactive batch job
• Examples in /lustre/haven/example/jobs, /lustre/Isaac/examples/jobs
• Browser-based Access (Open OnDemand)
• Torque or SLURM batch job
• Jupyter Notebook access
• Note: Do not run computational work on login nodes

https://oit.utk.edu/hpsc/isaac-open/access-and-login/

Batch Jobs

Access and Login

• Access to ISAAC through terminal in Mac or Linux
• Open a terminal and type

Legacy ssh {yourNetID}@acf-login.acf.tennessee.edu �NG ssh {yourNetID}@login.Isaac.utk.edu�

• Access requires UT Netid + password + Duo
• Hardware token or NetID associated apps on cell phone (https://oit.utk.edu/accounts/2fa/login-options/)

​

Open OnDemand

• ISAAC Legacy�https://acf-login1.acf.utk.edu/
• ISAAC-NG�https://login.Isaac.utk.edu/
• Quick Demo

File systems and data storage on ISAAC

• Two types of file system (NFS and Lustre) with three user directories for data storage

• It is important to back up your important data

https://oit.utk.edu/hpsc/isaac-open/file-systems/

Data Transfer

• Data transfer between ISAAC and your local machine is important for data back up
• Requires UT NetID and DUO authentication
• Data transfer tools:
• Globus Online: Web interface; Good performance
• SCP/SFTP: Secure; Slow performance
• Third party clients: Winscp (Windows only); FileZilla; etc.

recommended

https://oit.utk.edu/hpsc/isaac-open/data-transfer/

Data Transfer: ISAAC Legacy only

https://oit.utk.edu/hpsc/isaac-open/data-transfer/

• Available Data transfer nodes (DTNs) Legacy :
• Hostname: datamover1.acf.tennessee.edu
• Globus endpoint: nics#datamover1
• Same for datamover2, datamover3, datamover4
• Globus also requires you configure a CILogon certificate in the portal. Only you can get the Distinguished Name and enter it into the portal. �See documentation or come to ISAAC office hours

Data Transfer ISAAC-NG

https://oit.utk.edu/hpsc/isaac-open/data-transfer/

• Available Data transfer node on ISAAC-NG :
• Hostname: dtn1.Isaac.utk.edu
• Globus endpoint: UTK ISAAC-NG DTN1
• Uses regular CAS authentication

Data Transfer with Globus

https://oit.utk.edu/hpsc/isaac-open/data-transfer/

• Demo

Data Transfer with SCP

• Data transfer using SCP:
• Local machine to ISAAC:

scp <source-dir> <NetID>@datamover1.acf.tennessee.edu:<dest-dir>

• ISAAC to local machine

scp <NetID>@datamover1.acf.tennessee.edu:<source-dir> <dest-dir>

• When copying a directory, replace “scp” by “scp -r”

Programming Environment

• Both ISAAC Legacy and ISAAC-NG:
• Use modules
• Have Intel and gcc compilers and libraries
• The latest versions are on ISAAC-NG

Finding software

• Module commands to check and use a software packages

​

​

​

​

​

​

Questions?