GDM 2025 Workshop - Unix

Presented by Lara Frcej and Victor Olet

Overview

• Will be going through a basic overview of the setonix unix environment
• Working with a shell to run commands on setonix using training accounts
• Learning to work within a filesystem, read files, modify files, check permissions, and do some simple scripting.

Operating systems

• An operating system (OS) is a software which interfaces you with your computer hardware
• Unix and Linux are OSs
• Linux was based on Unix
• A shell is a command line interface (CLI) which gives you access to the OS resources
• We will be running commands here
• Bash (Borne Again SHell) is a type of shell with its own defined commands that we’ll be using today

Files

• Storing information
• Have a name and extension
• Avoid spaces

DNA_SEQUENCE.txt

File systems

• Contain:
• Files (names and extensions)
• file relationships (pathways between files)
• file attributes (size, owner, time stamps)
• Directories which contain files
• Directories are separated by slashes (/)
• Unix file systems are like tree roots
• Starting ‘root’ directory is represented by a slash

File systems

​

• bin - short for binaries, this is the directory where many commonly used executable commands reside
• dev - contains device specific files
• etc - contains system configuration files
• home - contains user directories and files
• lib - contains all library files
• mnt - contains device files related to mounted devices
• proc - contains files related to system processes
• root - the root users' home directory (note this is different than /)
• sbin - system binary files reside here. If there is no sbin directory on your system, these files most likely reside in etc
• tmp - storage for temporary files which are periodically removed from the filesystem

File systems

Graphical

​

Text based

​

​

Setonix file structure

• Bin
• Dev
• Home
• Lib64
• Opt
• Proc
• Run
• Scratch
• Software
• sys

• Usr
• Boot
• Etc
• Lib
• Mnt
• Pe
• Root
• Sbin
• selinux
• srv

• Tmp
• var

Setonix file structure

• Home
• Store small amount of important files
• cd /home/(username)
• Scratch
• Storing files you are currently working on
• High storage but 21 day deletion policy
• cd /scratch/(groupID)/(username)
• cd $MYSCRATCH
• Software
• For storing software you download

Secure SHell (SSH) protocol

• Encrypting our messages while they are being sent to the remote machine
• Can be done by generating keys or using a password
• Keys are generally safer but more complicated

​

Log into setonix

• Will be utilising training accounts : [link]
• Log in using “ ssh username@setonix.pawsey.org.au”
• Enter your password

​

Download the data

• Download the zip file from link:
• Will use scp to upload the zip file from your local computer to setonix
• scp location/to/file username@setonix.pawsey.org.au:/location/on/setonix
• Software with GUI is doing this, just more elaborately
• unzip (filename)

1. Navigating files - commands

​

mkdir (make directory)

• Makes a directory
• Mkdir (name)

touch

• Creates a file
• touch (name)

man (manual)

• Gives you a manual for a command
• Man (command)
• Q to exit

​

• Overview: working with the file system. Moving around and creating files and directories

pwd (print working directory)

• Shows you the path to your current directory

cd (change directory)

• cd (name)
• . specifies current directory
• .. specifies directory behind your current
• ~ points to home directory
• - back to the directory you were in last

ls (list)

• List files and directories in your current directory
• ls -a (lists hidden files)
• ls -l (lists “long” informative version of ls)
• ls -la (combines flags a and l)

1. Navigating files - module 1

• Overview:
• Module 1 contains directories of years. In each year, we’ve highlighted a notable individuals name and birth date.

1. Create a directory with your year of birth which includes two files including your name and date of birth.

Useful shortcuts

• Overview: shortcuts to make working with the terminal more efficient

• Tab autocompletion for finding files and directories
• ctrl a = start of line
• ctrl e = end of line
• alt/ctrl left or right arrow = start /end of word
• ctrl c = cancel the running command/bring me back to empty line
• clear = clears your terminal

2. Working with files - commands 1/2

• mv (move)
• Same conventions as cp, will just move the file rather than duplicate it
• mv (filename1) (filename2)
• The same as renaming a file**
• rm (remove) !!! DANGEROUS !!!
• rm (filename)
• Removes your file ** WITH NO WARNING
• rm -d (dirname)
• Removes empty directories only
• rm -r (dirname)
• Recursively removes directories and any files OR directories within them
• Rm -i (filename)
• Confirms what you want to delete before deleting

• Overview: moving, duplicating, removing files precisely in the command line

cp (copy)

• cp (filename) /path/to/file/(filename)
• Copy your file to a different location (the new file with have the same filename as the original)
• cp (filename) (filename2)
• Copy your file into the same directory, with a different name
• cp file/path/(filename) /different/path/(name2)
• Copy a file (in a specified directory) into a different directory (plus under a different name)
• cp (file1) (file2) (file3) destination_folder
• cp {file1,file2,file3} destination_folder
• Ways to copy numerous files at once

2. Working with files - commands 2/2

Examples:

• *.txt
• Any file with extensions .txt
• T*_man*
• Any file starting with T, with _man somewhere in the middle and ending with anything
• Matches:
• Tiny_man.txt
• Tall_man.png
• Tough_manners
• Th?n
• Matches Thin, Than, Thln, Thqn, ect

• Overview: moving, duplicating, removing files precisely in the command line

Globs (Wild cards which match patterns)

• *
• Matches any string
• ?
• Matches any single character
• [rna]
• match any single character/s r n or a
• [a-f]
• Match any single character from a to f (lower case)
• [A-Da-z0-7]
• Match any single character from A to D (uppercase) a to z (lowercase) and 0 to 7

​

​

2. Working with files - module 2

​

• Overview:
• Module 2 contains famous places around the world as text and image files

1. Please create a copy of your tajmahal and babylon text files into the backup directory, so you’ll never lose them!
2. Unfortunately, Stonehenge.txt has been named incorrectly, can you please rename the file without duplicating the file?
3. How many places start with the letter G ?
4. Now, we’d like to move all the text files into the text directory, and all the images into the image directory. Since there's a lot of these files, it would be a good idea to use globs.
5. From the image folder, type a command to create a copy of all our image files into the backup folder so we don't lose them.

​

3. Reading files - commands

• ctrl+Y page up
• ctrl+V page down
• alt+G go to specified line
• ctrl+W search for a string
• alt+W for next occurrence
• alt+Q for previous occurrence
• ctrl+Y to start
• ctrl+V to end

• Overview: opening files to see whats inside

• cat (filename)
• Prints all lines of the file into the terminal
• Nano (filename)
• Edit a file using text editor nano
• If the file doesn’t exist, it will create one
• Within nano:
• ctrl+S saves a file
• ctrl+O saves and lets you specify a name (like save as)
• ctrl+X exit nano
• If you haven’t saved yet, it will primpt you to save

3. Reading files - module 3

​

​

• Overview:
• Module 3 contains files with names of different breeds of dogs, with files containing information about the breeds

1. How many times is exercise used in French_Bulldog?
2. How many times is exercise used in Siberian_Husky?
3. What is the last word in Labrador_Retriever?
4. Create a file with a description for a different kind of breed of dog and give it a short description.

4. Finding and replacing - commands

sed (manipulate files)

• sed 's/string1/string2/' (filename)
• Substitute string1 to string2 on the first occurrence of each line. This will output to the command line by default
• sed -i ‘s/string1/string2/g’ (filename)
• Substitute within a file string1 to string2, globally (all occurrences)
• sed -i '2i This is Line 2' (filename)
• Inserts a line into the file
• sed ‘3d;11d;20d’ (filename)
• Deletes specific lines
• sed ‘2,10d’ (filename)
• Deletes a range of lines
• sed '2c new_line‘ (filename)
• Replaces existing line

​

• Overview: file manipulation as well as locating information from files

grep (search a file for input string)

• grep “string” (filename)
• Search for a string in a file, outputting lines that contain that string
• Grep -n “string” (filename)
• Outputs matched lines with their line numbers
• grep -v “string” (filename)
• Outputs the reverse of what you’ve specified (ony lines without string will be displayed)
• grep -x “string” (filename)
• Outputs only lines that match our string
• grep -c “string” (filename)
• Prints just the count of times the string was found in the file

4. Finding things - module 4

​

​

​

• Which lines contain the characters ra in French Bulldogs
• How many times does exercise come up for all of the breeds?
• Up the lifespan of the German_Shepard
• Change every instance of well to unwell in Siberian_Husky
• Delete everything after Lifespan from Labrador_retriever
• Add a line under Lifespan of French_Bulldog describing their shedding level (low)

• Overview:
• Module 4 is identical to module 3. It contains files with names of different breeds of dogs, with files containing information about the breeds

5. Variables and alias - commands

• echo “$name”
• Double quotes evaluates special characters
• Evaluates the variable $name to Alex
• echo ‘$name’
• Single quotes is literal
• Evaluates the text ‘$name” to $name

Setting bash alias

• nano ~/.bashrc
• .bashrc is a script that is run immediately after you log in
• You can set your environment with it
• alias (name)=(command)

• Overview: setting our own variables and alias that we can use whenever we’ve logged in

$variable (assign values to strings)

• name=Alex
• Sets a variable named “name” to Alex
• To call the variable, you must use a dollar sign (eg. $name)

echo (display text)

• echo name
• No evaluation (missing $), prints “name” into terminal
• echo $name
• Evaluates the variable $name to print Alex

​

5. Variables and alias - module 5

​

​

​

​

• Overview:
• Module 4 contains a_file.txt

• Create a bash alias “lsl” which represents ls -l to quickly be able to see file permissions

6. File permissions - commands 1/2

• Overview: showing you the numerous ways to alter file permissions

Showing permissions

• ls -l (filename)
• Files begin with a -
• Directories begin with a d

3 categories

• u (user), g (group), o (others)
• a (all)

3 alterations

• - (remove), + (add), = (equal to)

3 main kinds of permissions

• r (read), w (write), x (execute)

Files on setonix are created with -rw-r--r--

Directories are created with drwxr-sr-x/drwxr-xr-x

• The s here sets any file created within the directory to belong to the group where the original directory was created

​

6. File permissions - commands 2/2

Using numerical representations

Examples:

• chmod 320
• -wx-w----
• Chmod 414
• r----xr--

• Overview: showing you the numerous ways to alter file permissions

6. File permissions - understanding

​

1. Directory 1 only has dr-------- permissions. What can you as the user do there?
2. Directory2 only has chmod 200 permissions. What can you as the user do there?
3. Directory3 has been set with u=x,g=,o= permissions. What can you as the user do there?
4. A directory4 has execute permissions for your user. Can your user execute a file in this directory if the file itself is missing the execute permissions for the user?

​

6. File permissions - understanding

​

• Directory 1 only has dr-------- permissions. What can you as the user do there?
1. A: you can use ls on the directory to find out what files are inside
• Directory2 only has chmod 200 permissions. What can you as the user do there?
• A: essentially nothing
• Directory3 has been set with u=x,g=,o= permissions. What can you as the user do there?
• A: you can move into the directory and execute files
• A directory4 has execute permissions for your user. Can your user execute a file in this directory if the file itself is missing the execute permissions for the user?
• A: No, you have to set the file permissions specifically

6. File permissions - module 6

• Find out what permissions log.log currently has, and update them so that you are able to add a line at the bottom of the file.
• Check the permissions of script.sh. Would you be able to execute the script ?
• Update script.sh so that you can run it with the shorthand ./script.sh

• Overview:
• Module 6 contains a log.log file and a script.sh file which we will try to edit and run

7. Scripting - commands 1/3

Examples:

• Overview: maintaining useful commands you’re likely to run frequently

#!/bin/bash (shebang)

• Executable that should be at line 1 of your script

# (hash)

• Used for comments

Bash (filename)

• Run your script

$1, $2, $3

• Command line arguments starting with the first, second, third

$@

• All the command line arguments

read (variable)

• Waits for the user to input a response

​

​

​

7. Scripting - commands 2/3

Examples:

• Overview: maintaining useful commands you’re likely to run frequently

$1

$1 not defined

#!/bin/bash (shebang)

• Executable that should be at line 1 of your script

# (hash)

• Used for comments

Bash (filename)

• Run your script

$1, $2, $3

• Command line arguments starting with the first, second, third

$@

• All the command line arguments

read (variable)

• Waits for the user to input a response

​

​

​

7. Scripting - commands 3/3

• Overview: maintaining useful commands you’re likely to run frequently

If statements (checking if things are true before proceeding)

General structure:

if [ test ]

then

[ commands]

elif [ test ]

then

[commands]

else

[commands]

fi

​

​

Check if file exists:� If [ -f (filename) ]� then� [commands] � Fi

Check if file does not exist:� If [ ! -f (filename) ]� then� [commands] � Fi

​

​

​

7. Scripting - module 7

​

1. Create a script which takes two command line arguments which greets the user by their name and age
2. Create a script which creates as many files as the user would like (hint: use touch)
3. Create a script which creates a back-up file for a specified file
4. Ask the user for an input file. If the file exists, create a backup of the file
5. Create a script which changes all instances of colour and attribute from rainbow.txt based on a user's input

• Overview:
• Module 7 is mostly empty (for now), containing only rainbow.txt