1 of 46

Introduction to Interactive HPC

Michael Milligan

Fall 2025

1

MN Supercomputing Institute

for Advanced Computational Research

2 of 46

What is Interactive Computing?

Software GUIs
Prototyping workflows

Design your workflow for a single node (multi-core) or small set of nodes
Discover and test new tools/concepts
Profile, optimize and debug

Data Visualization

2

Interactive Computing is about immediacy. When the human requests an action, the computer immediately responds. When the computer finishes a task, the human immediately perceives the result.

MN Supercomputing Institute

for Advanced Computational Research

3 of 46

Why Interactive HPC?

HPC is part of a larger research workflow

Dr. so-and-so needs to explore a huge dataset� Needs: interactivity, analytics tools, large memory/storage

Dr. such-and-such is preparing to create a cutting-edge simulation visualization� Needs: interactivity, remote visualization, compute, bandwidth

A grad student is prototyping a new algorithm� Needs: interactivity, dev and debug tools, time to iterate

MN Supercomputing Institute

for Advanced Computational Research

4 of 46

Tutorial Outline

Batch vs. Interactive Computing @ MSI
Getting Started with Interactive Resources

Connecting to MSI
Conscientious Computing
Prototyping Workflows

Hands-on

Remote desktops
Jupyter Notebooks

4

MN Supercomputing Institute

for Advanced Computational Research

5 of 46

MSI Access Overview

5

MN Supercomputing Institute

for Advanced Computational Research

6 of 46

Resource Allocations

ALL groups will have access to these basic resources:

High Performance Computing cluster access
High Performance Storage

200 GB user home directory quota
Project storage accessible from most MSI systems

Access to interactive gateways
Access to technical support via helpdesk

access to expert consultants by referral

6

MN Supercomputing Institute

for Advanced Computational Research

7 of 46

Recent Changes

There have been some significant changes recently that may affect your usage:

Mesabi retirement

The mesabi cluster was decommissioned
Use agate for all your cluster computing needs
The mesabi login node is gone, connect to agate now
The mangi nodes were moved to agate, find them there

User home directories now separate from projects

Your $HOME is now in a separate /users/ space
Your $HOME quota is fixed and doesn’t affect project space
Find your project area (and old home directory/old files) in the usual location at /home/PROJECT/

7

MN Supercomputing Institute

for Advanced Computational Research

8 of 46

Survey of Options

8

MN Supercomputing Institute

for Advanced Computational Research

9 of 46

Batch Jobs

When should you use Batch Jobs?

Whenever possible! This is the traditional way to work in HPC
Share resources and be considerate of other researchers

What are the benefits of Batch Jobs?

Headless execution of automated processes
Long runtimes
Large core counts
A scheduler packs jobs in hardware to maximize utilization, reduce latency, etc.

9

MN Supercomputing Institute

for Advanced Computational Research

10 of 46

Interactive Options

OnDemand Portal
SSH terminal
Interactive batch (srun via the command line)
Galaxy portal (graphically compose workflows)
Specialized solutions

CITRIX for Windows apps

10

MN Supercomputing Institute

for Advanced Computational Research

11 of 46

OnDemand Portal

https://ondemand.msi.umn.edu

OnDemand Web Portal is:

One-stop access portal for interactive HPC services

Remote desktop access
Command line terminal
File browser
Launchers for common GUI scientific applications
JupyterLab server access
RStudio server access

11

MN Supercomputing Institute

for Advanced Computational Research

12 of 46

Interactive Command Line

ssh <username>@<resource>.msi.umn.edu

See: https://www.msi.umn.edu/content/connecting-hpc-resources

12

SSH is the traditional first point of access to MSI resources
Access agate login head node
Uses ssh program available on most personal computers

These “login nodes” are shared by many MSI users. They are NOT for heavy computational tasks. Appropriate uses include:

Browsing and editing files
Submitting jobs
Querying job status and other informational commands
Browsing software modules
Light testing or software compilation

MN Supercomputing Institute

for Advanced Computational Research

13 of 46

Interactive Batch with ‘srun’

Dedicated compute resources via the scheduler
Command line terminal running on the cluster

similar to a SSH session

Supports all usual Slurm options and commands
Best used with interactive partitions

have priority access for small interactive job requests

Job terminates when shell exits or connection broken

srun <resource requests> --pty bash

See: https://www.msi.umn.edu/content/interactive-queue-use-srun

13

Example:

srun -N 1 --ntasks-per-node=4 --mem-per-cpu=1gb -t 1:00:00 -p interactive --pty bash

MN Supercomputing Institute

for Advanced Computational Research

14 of 46

Web Portals: Galaxy

http://galaxy.msi.umn.edu

Galaxy is a web-based workflow manager, principally for bioinformatics/’omics research

14

Compose data analysis workflows in a unified data flow view
Workflows execute when requested on MSI compute clusters
Wide variety of bioinformatics and ‘omics applications available as tools

See https://www.msi.umn.edu/content/galaxy for additional details, especially regarding linking data into Galaxy

MN Supercomputing Institute

for Advanced Computational Research

15 of 46

Need Windows?

MSI no longer provides Windows application servers.

Select HPC applications that require Windows are available through partnership with OIT via CITRIX

See: �https://msi.umn.edu/about-msi-services/interactive-hpc/citrix for access to Apps.umn.edu

15

MN Supercomputing Institute

for Advanced Computational Research

16 of 46

Hands-On: Getting Connected

16

MN Supercomputing Institute

for Advanced Computational Research

17 of 46

Pre-requisites to connect

Campus VPN

https://it.umn.edu/services-technologies/virtual-private-network-vpn

DUO Authentication

https://it.umn.edu/services-technologies/self-help-guides/duo-set-use-duo-security

17

MSI services are only available to on-campus network addresses
Especially during the pandemic, many users are working off-campus
VPN clients and instructions are available from OIT for most operating systems and mobile devices

DUO 2-factor authentication is required for most MSI services
DUO is also required to connect to the campus VPN

MN Supercomputing Institute

for Advanced Computational Research

18 of 46

Sidebar: Windows�SSH and File Transfer

Current Windows 10+ computers will have ssh/scp available from the Windows terminal.

For users who desire a graphical ssh or file transfer client:

https://winscp.net/eng/downloads.php

Download and install both WinSCP and PuTTY*

*MSI recommends WinSCP and PuTTY for Windows users, but does not offer support for these programs. Contact your local IT department for assistance.

18

MN Supercomputing Institute

for Advanced Computational Research

19 of 46

Choosing a Cluster/Partition

See: https://msi.umn.edu/our-resources/slurm-scheduler/choosing-a-partition

Long running jobs and batch jobs run on Agate
MSI compute clusters are heterogeneous (e.g., up to 2TB Memory, GPUs, SSDs, etc.)

Think: match your needs to available configurations
Do: reserve the (CPU/RAM/GPUs/etc) that you need
Avoid: reserving much more (waste, long queue times) or �less than required (job doesn’t finish before walltime expires)

For interactive use, begin with the interactive partition and experiment from there.

19

MN Supercomputing Institute

for Advanced Computational Research

20 of 46

Interactive Services OnDemand

20

MN Supercomputing Institute

for Advanced Computational Research

21 of 46

OnDemand Walkthrough

21

https://ondemand.msi.umn.edu/

MN Supercomputing Institute

for Advanced Computational Research

22 of 46

OnDemand Walkthrough

22

Services accessed through menu bar

MN Supercomputing Institute

for Advanced Computational Research

23 of 46

OnDemand Walkthrough

23

Return to landing page

Dashboard of running and completed interactive activities

Job list and composer app�(web interface panels to squeue and sbatch)

MN Supercomputing Institute

for Advanced Computational Research

24 of 46

OnDemand Walkthrough

24

File browser access

File browser:

browse/navigate/manage files
upload/download files up to few GB size
access different starting dirs if you have multiple groups/projects

MN Supercomputing Institute

for Advanced Computational Research

25 of 46

OnDemand Walkthrough

25

Command line terminal access

Command line terminal:

“SSH” in a web browser (almost)
Launch terminal on cluster headnode
Roughly equivalent to e.g.�ssh <username>@agate.msi.umn.edu
Also launch in specific directory via file browser pane

MN Supercomputing Institute

for Advanced Computational Research

26 of 46

OnDemand Walkthrough

26

Interactive Apps menu lists available application launchers

Each launcher leads to a session options page

Each session runs as an interactive job on a cluster resource

Launchers also available via the Interactive Sessions dashboard

MN Supercomputing Institute

for Advanced Computational Research

27 of 46

OnDemand Walkthrough

27

My Interactive Sessions

List job status for interactive sessions: queued, running, finished

Launch into a running session (usually opens new browser tab)

Access full menu of interactive launchers

MN Supercomputing Institute

for Advanced Computational Research

28 of 46

OnDemand Walkthrough

28

My Interactive Sessions

Delete a running or queued session to cancel the job and release compute resources early

Deleting a completed job removes it from this dashboard; no other effect

MN Supercomputing Institute

for Advanced Computational Research

29 of 46

OnDemand Walkthrough

29

Remote desktop sessions

VNC embedded in a web browser

Runs on compute node in selected cluster resource

These launchers start a remote desktop session and launch corresponding application

Jupyter and RStudio servers run on compute node

Web interface proxied through OnDemand

MN Supercomputing Institute

for Advanced Computational Research

30 of 46

Interactive Linux Desktops

30

Interactive desktop: noVNC session proxied through OnDemand server

Graphical fidelity will suffer on low-bandwidth connections but quality can be adjusted to maintain usability

Good connections allow graphically intensive apps, GPU accelerated graphics available on GPU nodes

Session persists until logged out on desktop - don’t forget to log out to release resources when done!

MN Supercomputing Institute

for Advanced Computational Research

31 of 46

OnDemand Walkthrough

31

Example launcher:

Jupyter server

Options to customize most aspects of the interactive job

cluster / partition
group (if multiple available)
resource requests
launcher-specific parameters (e.g. Python environment for Jupyter servers)

MN Supercomputing Institute

for Advanced Computational Research

32 of 46

OnDemand Walkthrough

32

Logout and server restart options

Session cleanup

OnDemand sessions run as scheduled jobs: �cancel jobs or use desktop “logout” to end jobs and release resources early, or allow them to close automatically when walltime expires

Logging out from this page clears browser login but otherwise has no effect on active OnDemand sessions

Rarely may need to use “restart server” option to recover from failures

MN Supercomputing Institute

for Advanced Computational Research

33 of 46

Session Clean-up and Resume

What is lost when the connection drops?

Session: if not expired, reconnect to the same session/node
Data: no loss

What if you End Job/Logout Desktop?

Session: Terminated; Reconnect and start fresh
Data: Scratch data is deleted periodically* (clean up after yourself to be sure)
No loss for data in group home

How do you reconnect?

Login again through OnDemand Interactive Sessions dashboard

33

MN Supercomputing Institute

for Advanced Computational Research

34 of 46

Interactive Jobs with �srun

34

MN Supercomputing Institute

for Advanced Computational Research

35 of 46

Selecting Resources

SSH to the cluster headnode:

ssh <username>@agate.msi.umn.edu
Launch interactive terminal in OnDemand

Queue an Interactive job on the cluster:

srun <job options> --pty bash

35

MN Supercomputing Institute

for Advanced Computational Research

36 of 46

Selecting Resources

Options to srun specify job requirements:

srun [options]

--time=8:00:00 job can run for up to 8 hours
-N 1 --cpus-per-task=4 job requests 4 CPUs on one node
--mem=2500m job requests 2500 MB (2.5GB) of RAM
-p interactive
--pty bash job must specify shell program to run

Enable graphics via X-tunneling (--x11)

not available in web terminals (OnDemand, Jupyter, etc)

36

MN Supercomputing Institute

for Advanced Computational Research

37 of 46

Selecting Resources

Refer to SLURM partitions reference page:�https://msi.umn.edu/our-resources/slurm-scheduler/slurm-partitions

Example - basic interactive session with 4 CPU cores, �8 GB RAM, 4 hours runtime:

srun -N 1 --ntasks-per-node=4 --mem-per-cpu=1gb -t 1:00:00 -p interactive --pty bash

Example - interactive session for machine learning with 1 A40 GPU, 12 CPU cores, 1 hour runtime:

srun -n 12 -t 1:00:00 -p interactive-gpu --gres=gpu:a40:1 --pty bash

37

MN Supercomputing Institute

for Advanced Computational Research

38 of 46

Navigating Home and Project Spaces

Start location will be $HOME (something like /users/7/YOUR_ID)
Most work will happen in project space

cd $MSIPROJECT�or simply type msiproject

If you are attached to multiple project groups you may need to select the correct one

newgrp PROJECTNAME�srun -A PROJECTNAME <more options>

Similar effect to selecting group from dropdown in OnDemand

38

MN Supercomputing Institute

for Advanced Computational Research

39 of 46

Software Modules

See all modules available

module avail

Load a module (adds commands to your shell)

module load python3

Run the software from the module

ipython

39

https://msi.umn.edu/our-resources/knowledge-base/accessing-software-resources

MN Supercomputing Institute

for Advanced Computational Research

40 of 46

Clean up your session

Delete the session (choose one)

Exit out of all connections
scancel <job ID>

What happens on disconnect/delete?

All computation stops

How to resume?

Start with fresh session
Load intermediate files (if you created them)

40

MN Supercomputing Institute

for Advanced Computational Research

41 of 46

Questions?

These slides and the hands-on guide will be posted to http://msi.umn.edu (see the “MSI Tutorials” section)�

41

MN Supercomputing Institute

for Advanced Computational Research

42 of 46

Extra Material

42

MN Supercomputing Institute

for Advanced Computational Research

43 of 46

Optional: SSH Keys

SSH keys allow terminal logins without typing a password repeatedly
Some applications may automatically make SSH connections and may require SSH keys
See MSI knowledge base page: https://msi.umn.edu/our-resources/knowledge-base/interactive-connections-faqs/how-do-i-setup-ssh-keys

43

MN Supercomputing Institute

for Advanced Computational Research

44 of 46

Hands-On: Interactive Development and Data Interrogation with Jupyter

44

MN Supercomputing Institute

for Advanced Computational Research

45 of 46

JupyterLab Workspace

See: https://jupyterlab.readthedocs.io/en/stable/user/interface.html

45

Rich tabbed multi-document workspace
File navigation able to access home and group directories
Reproducible, sharable computational stories in Jupyter Notebooks
View and edit text and other files directly in JupyterLab
Command line terminal available

MN Supercomputing Institute

for Advanced Computational Research

46 of 46

Additional Documentation

46

MN Supercomputing Institute

for Advanced Computational Research