LIP Computing training basic level

​

​

​

1

Outline

• Introduction
• LIP
• LIP Computing infrastructure
• Computing farm structure
• Authentication mechanism
• Documentation
• How to access the infrastructure
• What is ssh & why use it
• Generate ssh keys

​

• Useful information
• Linux commands
• Best practices

2

LIP

LIP- Laboratório de Instrumentação e Física Experimental de Partículas

• Created in 1986
• Three polos: Lisbon, Coimbra and Braga
• More than 202 people: staff, researchers, students, collaborations
• Four main research areas:
• Astroparticle Physics
• Particle Physics
• Detectors and Applications
• Computing

3

LIP - IT infrastructure

• general IT services – Core institutional services, network services, security, and also LIP-Lisbon specific services
• Data Intensive Computing Infrastructure @INCD

4

Introduction �LIP Computing infrastructure

​

​

​

IT infrastructure context

ISEC - Laced

Minerva Cluster

20x Dell R720

Infiniband

Miguel Couceiro

U. Coimbra

Old Datacenter

Physics Dep.

Small LIP facility

Point-of-Presence

​

NCG @ LNEC

Sala-Grid Datacenter

New INCD equipment

Old INGRID equipment

LIP specific equipment

​

LIP users

LIP Tier-2

INCD users

​

​

U. Lisboa @ reitoria

Mail, web, VMs

Home directories

​

LIP @ 3Is

Tape library

Local network+services

​

​

BOB @ REN Riba-de-Ave

HPC cluster from FCT

Offered by TACC

20% is managed by INCD+LIP

LIP helping in the storage

​

IT network infrastructure

U. Coimbra

Old Datacenter

Physics Dep.

​

NCG @ LNEC

Sala-Grid Datacenter

New L2/L3 switches

48x 10/25 GbE

18x 40/100 GbE

​

LIP @ 3Is

Tape library

Local network

​

​

BOB @ REN Riba-de-Ave

HPC cluster from FCT

​

U. Lisboa @ reitoria

Mail, web, VMs

Home directories

​

LIP @ U. Coimbra

​

LIP @ U. Minho

​

INTERNET

LNEC

Internal network

​

INCD Private Net

Introduction �INCD Computing infrastructure

​

​

​

INCD - Infraestrutura Nacional de Computação Distribuída

• INCD is a digital infrastructure:
• LIP Technical coordination
• Goals:
• Provide computing and data services for the research community.
• Computing Services:
• Cloud.
• HTC and HPC (farm)

9

Computing Farm:

Applications run in a compute node:

• Linux (Centos 7)
• You do not have any direct access to the compute nodes
• Access trough scheduler (slurm)
• Storage accessible in both submission node and compute nodes
• ​

10

Compute nodes

Job submission�(pauli.ncg.ingrid.pt)

Job scheduling

Scheduler (slurm)

Storage

Computing Farm Advantages:

• Applications with limited execution time (minutes-weeks).
• “Almost” static environment:
• Operating System, software, compilers and libraries are deployed by the system administrator. Users request the sys admins to deployed specific software, libraries, compilers to their needs.
• Applications are scheduled into a queue for execution.
• Resources are shared with other others users
• ​

11

How to access �the infrastructure�SSH

​

​

​

SSH

How to access - SSH

Allow us to connect to local or remote servers �(inside safe/isolated networks)

​

​

​

​

​

HOME

​

​

​

​

​

LABORATORY

​

​

​

​

​

It is the most secure way to establish a �connection between two machines

​

​

​

​

​

​

13

Secure network

Generate an ssh key pair

• Generate a new key pair in a terminal with the next command��

The key generator will ask for location and file name to which the key is saved to. The recommended is the default by pressing enter with nothing. If you have already a key named id_rsa, choose another name.

• Create a passphrase for the key when prompted

This is a password that will protect your private key from someone that not supposed to get their hands on it. Enter the password you wish or continue without a password. The recommended is to choose a strong password.

​

14

ssh-keygen -b 4096 -t rsa

More details in https://wiki.incd.pt/books/how-to-access/page/create-a-ssh-key

Generate an ssh key pair

• See your key pair generated with the next command��

You will see the keys stored in your computer:� id_rsa → private key.� id_rsa.pub → public key.

• Send your public key (id_rsa.pub) to LIP administrators via email

Once the key pair was created, you must send to LIP administrators the public key to associate it to your account. (continue in next slide)

​

​

15

ls ~/.ssh/

Generate an ssh key pair

(continuation) Send your public key (id_rsa.pub)

• Show the public key on terminal

​

• Copy the public key and paste it into an email�Just underline the key with your mouse, right click (shift-right-click on MobaXTerm) and copy. Then paste the key into an email.���
• Send the email with public key to:�To: helpdesk@lip.pt

​

16

cat ~/.ssh/id_rsa.pub

Connect to server

• Connect to remote server in a terminal with the next command��

username → replace username that will be set to you by email.

pauli.ncg.ingrid.pt → the name of the server you going to connect to.

If your private key was protected by a passphrase (2.), you going to be asked to insert it.

​

17

ssh username@pauli.a.incd.pt

How to access - using Windows

• Install MobaXTerm https://mobaxterm.mobatek.net

​

​

​

​

18

To start a new terminal, click on “Start local terminal”

How to access - using Windows

• Click right-click on the terminal and choose “Paste”
• This way, to paste something into the terminal you just need to right-click.
• To copy something from the terminal, you must shift-right-click and then click copy.

​

19

macOS

Required for export Display:

• install: https://www.xquartz.org/

20

Useful information�Linux commands �&�Best practices

​

​

​

Useful Linux commands

• cd: Change directory.
• rm: Delete files.
• mkdir: Create directory.
• ls: List files.
• pwd: Print current directory name.
• cat: View files.
• Ctrl-c: press “Ctrl” and “c” keys to terminate what your are writing in terminal.

For more commands, check the Quick Reference guide: http://www.macs.hw.ac.uk/~hwloidl/Courses/LinuxIntro/linux-quickref.pdf

22

Best practices 1/x

@LIP with Personal Laptops

• Torrents
• No licensed Software
• All software needed in the course is free
• Windows activate the Anti-Virus

​

​

​

​

​

23

Best practices 2/x

Secrets

• PRIVATE SSH key should always be encrypted with strong password
• Don’t share the password or the private key.
• Accounts are private

​

​

​

​

​

24

Best practices 3/x

Phishing/Ransomware

• Emails (From: LIP vs From: @lip.pt)
• Attachs
• Links
• Report anything suspicious

​

​

​

​

​

25

Documentation

&

Helpdesk

​

​

​

Documentation

• Lip Wiki: https://wiki-lip.lip.pt/
• Configure email
• Eduroam access
• LIP internal groups documentation
• LIP computing FARM access and commands

​

• More generic FARM: https://wiki.incd.pt/

​

27

��Helpdesk ( helpdesk@lip.pt)

​

​

Good practices:

• Always use the email that’s in the LIP DB
• Try to explain clearly the problem on the email
• Required information:
• login name (name provided to your by LIP IT team to access LIP services)
• Source machine (personal laptop, desktop, etc)
• Destination machine: (example: pauli.ncg.ingrid.pt)
• Problem with job submission:
• My submission code is here: /home/userxxx/submit_to_farm.sh
• job ID: try to get the job ID of you job (Check FARM wiki)

28

Docker preparation:

For those who require the use Root Software please follow the following guidelines:

https://docs.google.com/presentation/d/1F2AJrJu5wUSf04cbXL_RBAkUEXXQJBsigDc69yTAEfk/edit#slide=id.p

All questions regarding the installation/running should be done in SLack for the internship

29

�Q&A

​

Nice Internship for everyone!!�Um Bom Estágio para todos!!