1 of 33

Introduction to the Minnesota Supercomputing Institute

Benjamin Lynch, Ph.D.

Director, MSI

Minnesota Supercomputing Institute

2 of 33

Overview

Mission
Organization
Resources and Services
Eligibility, Access, and Accounts

Minnesota Supercomputing Institute

3 of 33

Minnesota Supercomputing Institute

Founded in 1984 to provide advanced research computing infrastructure and expertise to the State of Minnesota in order to advance and accelerate research and foster innovation and discoveries through advanced computing technologies, scientific computing, and informatics.

Minnesota Supercomputing Institute

4 of 33

Are you a new faculty member?

z.umn.edu/msiaccess

Minnesota Supercomputing Institute

5 of 33

Research Groups

Social Science

Physics

Other

Mathematical Sciences

Informatics

Health Sciences

Veterinary Medicine

Agriculture

Astronomy

Biochemistry

Computer

Science

Earth Sciences

Economics

Chemistry

Biology

Engineering

Genetics

1041 Groups

122 Departments

5409 Users

Minnesota Supercomputing Institute

6 of 33

Minnesota Supercomputing Institute

(Ben Lynch)

U-Spatial

(Len Kne)

Research Computing�Administration

(Brian Carlton)

Research Computing

(Jim Wilgenbusch)

Research Informatics

Outreach and Training

(Laura Fingerson)

Data Science Initiative

(Haley Borck)

International Institute for Biosensing

(Patricia Khashayar)

Operations and Development

Research Computing

Minnesota Supercomputing Institute

7 of 33

Minnesota Supercomputing Institute

58 full time employees and 4 students
> 1000 active Faculty groups using MSI resources
~5000 active users
Most resources and expertise are provided as a common good to University of Minnesota faculty and their groups.

Minnesota Supercomputing Institute

8 of 33

High-performance Hardware and Software

>80,000 processor cores
448 GPUs
30 Petabytes of high-performance storage

523 Software Packages
Automated testing

Minnesota Supercomputing Institute

9 of 33

Remote Interfaces

Minnesota Supercomputing Institute

10 of 33

Research Informatics

Advanced Support

Collaborations

Partnerships

Services

Minnesota Supercomputing Institute

11 of 33

High Performance Computing (HPC)

Highly parallel computing environment with high-speed network connections, graphics processing units, high memory, and high-performance storage
batch HPC and interactive HPC
HPC access controlled by the SLURM Fairshare scheduler

Minnesota Supercomputing Institute

12 of 33

Linux

Unix was developed at AT&T Bell Labs in the 1970’s.

Linux was developed by Linus Torvalds in the 1990’s, and is now developed by a community as a free operating system.

Linux is: free, stable, extensible, and fast.

Unix/Linux are the dominant OSes for large HPC systems.

Minnesota Supercomputing Institute

13 of 33

Batch HPC

Using batch scheduling is the primary way to make use of MSI’s shared supercomputing resources.
You will submit jobs to the cluster scheduler, and your analysis eventually runs on the computer without any further intervention.

Requires jobs to be written as shell scripts and submitted via the command line.

Jobs are queued and priority is based on a combination of the size of the request and how many jobs your group has run recently.
Jobs will run when resources are available.

Minnesota Supercomputing Institute

14 of 33

Example batch SLURM script

#!/bin/bash -l

#SBATCH --time=8:00:00

#SBATCH --ntasks=4

#SBATCH --mem=20g

#SBATCH --gres=gpu:a100:2

#SBATCH --partition=msigpu,some-other-partition

#SBATCH --mail-type=ALL

#SBATCH --mail-user=sample_email@umn.edu

#

cd ~/program_directory

module load (...software modules you need….)

Command_to_run_your_application (cmd line options)

Example for GPU usage

Minnesota Supercomputing Institute

15 of 33

Rightsize Your Resource Request

Time until execution

Minnesota Supercomputing Institute

16 of 33

Rightsize Your Resource Request

Does it going to use these resources?

Time until execution

Minnesota Supercomputing Institute

17 of 33

Rightsize Your Resource Request

If so, it’s going to have to wait.

Time until execution

Minnesota Supercomputing Institute

18 of 33

Rightsize Your Resource Request

If it’s needs fewer resources, it will start faster!

Time until execution

Minnesota Supercomputing Institute

19 of 33

Interactive HPC

Types of interactive HPC offered:

srun: SLURM batch method to get an interactive shell on HPC. Partition to use is called “interactive”
Open OnDemand: Web browser interface with access to local and cluster files, custom applications, Jupyter, or a terminal window. ondemand.msi.umn.edu

See https://msi.umn.edu/about-msi-services/interactive-hpc

Minnesota Supercomputing Institute

20 of 33

Open OnDemand

Minnesota Supercomputing Institute

21 of 33

Software Resources

Scientific and numerical software packages are built into “modules” and available to be loaded as you need them.
Common software tools used in many fields are available on our systems

Genetics
Proteomics
Image processing
Computational chemistry
… and many more

500+ software applications msi.umn.edu/software
We can work with your research groups to install software that you need

Contact the help desk!

Minnesota Supercomputing Institute

22 of 33

Data Storage

Primary Storage Tier-2 Tape

Minnesota Supercomputing Institute

23 of 33

Primary Storage

Primary storage is where you will do most of your work
Watch your disk usage:

If you are up against file number or disk usage quota, your entire group cannot make new files

Important directories in your primary storage space:��/users/<some number>/Your_userID�Your home directory. Store scripts and small files here��/home/groupname/shared�PI group shared directory. Store shared software, common datasets, reference databases, etc, here.

Minnesota Supercomputing Institute

24 of 33

Scratch Storage

Use it for intermediate/temporary files in analytical workflows
Files are kept for 30 days, so be sure to copy important files to your group directory
Scratch space is not backed up

Location:��/scratch.global/

Minnesota Supercomputing Institute

25 of 33

Second Tier Storage

Second tier (Tier-2) storage is separate from primary storage - you must use special tools to access it.
From the command line, you can use the Amazon “s3cmd” software package to interact with your storage:

https://msi.umn.edu/our-resources/knowledge-base/data-storage-faqs/how-do-i-use-second-tier-storage-command-line

For a graphical interface, you can use Globus

https://msi.umn.edu/our-resources/knowledge-base/file-transfer-faqs/how-do-i-use-globus-transfer-data-msi

Not backed up, so if you delete a file, it will be gone for good

Quotas are 120 TB per research group, typically under the PI’s userID

Minnesota Supercomputing Institute

26 of 33

Data Storage Allocation

Default storage allocation: 150GB/group primary storage
Requests of up to 5TB:

No justification or review required

For requests of 5 TB–20 TB:

Justification
Review by MSI staff

ANNUAL renewals for all non-default storage allocations

If you haven’t made active use of the storage you’ve been allocated, you’ll be asked to cut down on your primary storage allocation.

Minnesota Supercomputing Institute

27 of 33

Service Costs

Free for UMN faculty and their associates
Free for other MN academic institutions
Free for expert consultation
Fee for data storage over allocations limits
Fee for extended expert collaboration
Fee for service for external organizations (any organization that is not MN higher-ed)

Minnesota Supercomputing Institute

28 of 33

PI Group Structure

PI: Controls everything

Group Administrators: Users within the group designated to do everything a PI can do except add other Group Administrators�
Users: Added by PI, usually lab members (postdocs, techs, undergrads, grad students)

Users may be part of multiple accounts if they perform analysis for multiple PIs
The PI is a user, too, and their SU and disk usage counts the same as their users’

Minnesota Supercomputing Institute

29 of 33

Accounts and Renewals

UMN faculty members, department staff, and researchers at other MN institutions for higher education are eligible for MSI accounts

PI accounts are renewed annually

Renewal period starts in late October
Extra storage allocations all must be checked and justified

Minnesota Supercomputing Institute

30 of 33

Research Support�Services

MSI has ~33 consulting staff
The majority of consultants have an advanced degree
We provide >20 workshops and tutorials annually
Informational sessions for proposal development
Expertise including:

Research Computing; Informatics
Big Data Analysis
Parallel Algorithm Optimization and Development
Custom Analysis Pipeline Development
Genomics, Chemistry
Application Development

Minnesota Supercomputing Institute

31 of 33

Training and Tutorials

We host tutorials on a range of topics�https://msi.umn.edu/tutorials
Introductory tutorials

Intro to MSI
Intro to Linux
Intro to Job Submission

Bioinformatics tutorials

Single Cell Genomics
RNAseq data analysis

Advanced tutorials

Parallelization
Python for scientific computing

https://www.youtube.com/@UofMMSI

Minnesota Supercomputing Institute

32 of 33

Summary

High Performance Computing and Consulting
Free access for researchers
Online access through ood.msi.umn.edu
Additional dedicated resources if you have a budget
help@msi.umn.edu if you have any questions

Minnesota Supercomputing Institute

33 of 33

Questions?

MSI Help Desk

Monday-Tuesday-Thursday in person Walk-in help is available in Walter Library 587. 9:00 AM – 4:00 PM
Wednesday & Friday, schedule a virtual appointment to meet with helpdesk staff by visiting the page below and selecting a time block
https://www.msi.umn.edu/content/helpdesk-video-consultations

Call 6-0802 (1-612-626-0802)
Email (staffed 9am - 4pm weekdays) help@msi.umn.edu
Scheduled Zoom consults with our Specialized Staff: https://www.msi.umn.edu/content/helpdesk-video-consultations

Minnesota Supercomputing Institute