1 of 13

User Workflows on SubMIT

Jan Eysermans on behalf of the SubMIT team

Annual Review

June 3 2026

2 of 13

Overview

​

​

  • Account creation and login
  • Accessing SubMIT resources
  • Storage resources
  • Computing resources
  • Software resources
  • Other tools

2

3 of 13

Creation of new accounts

Streamlined account creation based on a request by the new user

​

​

​

​

​

Typical information we ask to include in the request

  • Short motivation why and project context
  • Role (student, PhD, postdoc, researcher, professor)

After approval, the welcome email directs the user to setup the necessary login via TouchStone

  • Gives access to setup ssh keys to login into SubMIT system

For non-MIT users, a MIT guest account has to be created beforehand

  • Full name, email address, date of birth, institution

3

4 of 13

How to access SubMIT?

Several ways of accessing SubMIT

  • Classic access via terminal – requires to setup the keys
  • A more modern way via a web-browser using JupyterHub
  • VSCode terminal like
  • Interactive GUI via X2Go (Linux) or X-Win32 (X11 server on Windows)

Secured authentication via Touchstone and ssh keys

All options provide full access to the SubMIT resources and functionalities

4

Terminal

Web-browser

X-Win32

5 of 13

What can users get on SubMIT?

5

See Jacob’s talk

6 of 13

Storage resources

What do users get on SubMIT by default?

​

​

​

​

​

​

​

​

​

​

​

Other resources and tools

  • Fast mount space (/scratch), can go up to 120 Gb/s (storage space upon request)
  • MIT Tier-2 site storage (storage upon request)
  • XrootD available for easy and efficient file transfers (internally and externally)
    • Requires authentication via grid certificate

6

Doubled the quota

7 of 13

Computing resources

What do users get on SubMIT?

​

​

​

7

Type

Purpose

Available resources

​

SubMIT login nodes

Development and debug workflows interactively

6 machines

​

SubMIT worker nodes dedicated for compute within SubMIT computing environment

​

​

~1500 CPUs

~ 66 GPUs

Access: submit, submit-gpu, submit-gpu-express

GPU types: 4 NVidia GTX 1080, 12 V100, 8 A30, 2 RTX6000

​

Large-scale computing, access to external resources

​

Several thousands CPUs

(MIT T2, T3, OSG, CMS)

Increasing computational need and complexity

Interactive nodes

8 of 13

Computing resources

Monitoring in place for both users and admins

  • Fair-share on interactive nodes maintained by users
  • Scheduling on Condor and Slurm resources

Good usage and pressure of Condor and Slurm resources

Diverse types of workflows continuously being executed at SubMIT

8

Load on interactive nodes

Condor schedd

Slurm

Up to 500 jobs running, often in saturation

Up to 5-10k jobs running, often in saturation, mainly CMS

Well balanced load

9 of 13

Computing resources – user job monitoring

New monitoring in place for memory and CPU efficiency

  • From admin/schedd point of view, need to optimize the resource requests: the request should match the demand
  • Monitoring for all finished Condor jobs per user in intervals of 7 days, 1 month and 3 months
  • Users are notified manually if the job resource request largely exceeds the job demands

Similar monitoring being set up for Slurm jobs

9

Condor CPU efficiency

Condor memory efficiency

10 of 13

Software resources

Users have many tools available to easy their workflows

  • Try to maintain common tools centrally
  • Otherwise, users make their own environment via package managers and containers

10

(new!)

11 of 13

Others tools

Many other tools available at SubMIT for the users to get their work done efficiently

​

11

Personal and group websites to share files and display plots

Globus endpoint for data transfer and sharing

Parallel computing software

Dask Gateway and OpenMPI integrated with Slurm

Support and guidance for VSCode and AI-integrated applications (Claude, Codex)

12 of 13

Future plans

SubMIT keeps being a mature and performance computing cluster

  • OSG authentication via tokens ongoing
  • XrootD authentication via tokens, so members without a certificate (typically non-LHC users) can also use XrootD for larger data transfer
  • Support of Slurm user monitoring, similarly as implemented for Condor

​

12

13 of 13

Summary

SubMIT keeps being a mature and performance computing cluster

  • Easy to join for members of the Department of Physics or external users
  • Straightforward to interact with via terminal or web-browser (among others)
  • Large, fast storage space available with a modern and efficient filesystem
  • Suite of software tools centrally maintained or supported for users to access any software they need
  • Access to large compute pools via batch-submission using Condor and Slurm

We continuously maintain the core tools, improve monitoring and expand based on the user’s needs

​

13