Introducion to Shell Scripting

By AECC-UPRRP

Get down & dirty with the Terminal

• Everyday use shell commands and their use:
• man: the most important command. Displays a manual telling you information about what a command does and how to use it e.g. “man ls”
• pwd: print working directory (tells you where you are in the filesystem path)
• ls: list files and directories in current directory
• cd: change directory. Changes the current directory to the one given as an argument
• cat: used to read files sequentially i.e. displays the contents of a file, or concatenate files
• less/more: allows you to scroll through long files
• mv: used to move files around the file system. Also used to rename files
• rm: used to remove files (or directories if used with proper arguments)
• clear: clear the terminal screen
• cp: copies a given file to a given path
• history: displays the history of the commands a user has run in a shell

That’s All!

Thanks for Coming

​

Now you can be this guy too!

THE SHELL

• Command line interface for running commands and scripts to control an OS.
• Usually interacted with via a terminal emulator(use of a shell in a gui) or command line

Usual file system structure and usage:

Table of Contents

1. Permissions & Execution
2. stdin, stdout, stderr, stdwtf??
3. Parsing
4. Path
5. There’s more than 1 shell!?
6. Scripting

Permissions:

• File ownership (user : group)
• File permissions (octal, rwx)

​

• check: ls -l
• change: chown, chmod

Shebang ( #! ) :

• Tells the OS (*nix) what program to use to run the program
• Program needs to be executable
• Arguments can be used
• Portability matters:
• #! /usr/bin/env <program_on_path>

Standard streams

• Data Stream: ordered sequence of bytes read until EOF

​

​

stdin

​

​

​

​

stdout

stderr

stdin ( ) :

• Stream data (often text) going into a program
• Usually expected from keyboard

stdout ( ) :

• Stream where a program writes its output data
• Usually output to the terminal or terminal emulator

stderr ( ) :

• Another output stream typically used by programs to output error messages or diagnostics
• Usually output to the terminal or terminal emulator

These can all be redirected!

< > << >> |

Piping and Redirecting

Piping:

• Piping “pipes” the output of the first command into the input of the next command
• Example:
• `wget -q -O - http://ccom.uprrp.edu/2016/wordpress/ | grep -o "computer" | wc -l`

Redirecting:

• Redirects the output of a command into a file or redirect the input from a file
• Example: `curl icanhazip.com > publicIP.txt`
• Example: `grep -E "^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])$" < publicIP.txt`
• Explained: https://regex101.com/r/w3OJvc/1
• Combining both: `last | cut -d ' ' -f1 | uniq -u > users.txt`

• All input and output in the usual *nix shells are text streams
• Most commands take input from from standard input which usually comes from a keyboard
• Most commands output to standard output which defaults to outputting to the display

* we are defining these concepts in the context of bash

$PATH

$PATH: an enviromental variable. Contains the directories where executables are stored

• Enviromental Variable:
• Type of variables that control the behaviour and information on the shell

Check Enviromental Vars: `env`

Check $PATH: `echo $PATH`

$PATH cont.

​

• $PATH per user
• Modify $PATH:
• For every login: in .bashrc or .bash_profile:

`export PATH=$PATH:<directory>`

• For current session:

`PATH=$PATH:<directory>`

​

Parsing

• Extracting useful information from some text or string of symbols
• Usually done by describing the text’s format or some pattern in it

Side note:�Extremely important for compilers

Regular expressions (regex)

• Text strings that describe patterns to find text or positions within a body of text
• Examples:
• [A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,4}
• <([A-Z][A-Z0-9]*)\b[^>]*>(.*?)</\1>

Regular expressions (regex) cont.

• Good tutorial site:
• https://regexone.com
• Good for testing your regexes out:
• https://regex101.com

Variety of Shells

Shells :

​

• bash
• tcsh
• ksh
• zsh
• fish
• powershell^*

​

*: Not *nix. Similar-ish, but EVERYTHING is an Object (I/O).

Scripting:

What’s the point?

• automation
• eliminate repetitive tasks
• some companies care
• “unlimited” power
• system administration
• backup
• mail
• security monitoring
• parsing / data processing
• logging

​

Scripting cont. :

Choose your favorite text editor. It’s probably better if you’re comfortable with a shell text editor such as vi/vim, nano, emacs (we’ll cover this more in depth in another workshop :) )

• Remember to check permissions and make your script executable
• We will be talking about bash from now on

Bash...tricky sintax:

​

• Variables:
• var=<value>
• var=$((<math>))
• var=$<other_var>
• let var=<math>
• var=$(<command>)
• var=`<command>`
• User input:
• read var
• Access return value:
• $?
• Access parameters:
• $0,$1,$2,…,$n
• Functions:
• function f {}
• f()
• Number of arguments:
• $#

B

Bash...tricky sintax pt 2:

​

Conditionals:

if [ <condition> ]; then

<stuff>

elif (( <arithmetic condition> )); then

<stuff>

else

<stuff>

fi

** use [ ] or test

​

A

Bash...tricky sintax pt 3:

• Comparison operators:
• -eq, -ne
• -lt, -gt, -ge, -le
• -z, -n (null)

- Arithmetic operators:

• python style
• let, or ((<math>))

​

• Bitwise operators
• C style

​

​

• Misc. :
• -e
• -f
• -d
• -h
• -r
• -w
• -x

- Logical operators:

• C style

S

Bash...tricky sintax pt 4:

• For loop:

for <iterations>

do

<stuff>

done

** can be:

in {i..n}

in <dir> or in <command_out>

C style

• While loop:

while <condition>

do

<stuff>

done

H

Workshop :

Go to: http://bit.ly/aecc-shell

• Can you guess what the script does? Each part?
• Is this useful to you?
• Do you have an idea for a script we can help you write?

​

Helpful list of more advanced shell commands:

• ssh
• grep
• rsync
• chmod
• find
• tail / head / tac
• ps
• kill
• awk
• eval
• alias
• ln

• !! (sudo !!)
• mount
• dd
• whoami / who / last
• reset
• chroot
• history
• curl
• uniq
• sort
• readlink
• top

• tar
• strings
• time
• date
• which
• env
• sed
• ifconfig
• shasum
• nmap
• traceroute
• chown

AECC’s Favorite/Most used Commands or packages and favorite shell:

• Alejandro (shell: zsh):
• atool (aunpack)
• curl
• say
• nmap
• open
• José (shell: fish):
• find
• grep
• rename
• youtube-dl
• open

• Israel (shell: zsh):
• su
• tar
• ssh
• John Wilson
• aircrack-ng

Some more resources:

• More about the shell:
• http://hyperpolyglot.org/unix-shells
• http://linuxcommand.org/lc3_learning_the_shell.php
• More about bash scripting:
• Basic: http://tldp.org/HOWTO/Bash-Prog-Intro-HOWTO.html#toc1
• Advanced: http://tldp.org/LDP/abs/html/index.html
• Pitfalls: http://mywiki.wooledge.org/BashPitfalls
• Thorough: https://www.dartmouth.edu/~rc/classes/ksh/welcome.html#top
• More about permissions:
• https://www.tutorialspoint.com/unix/unix-file-permission.htm
• Check out other shells:
• http://www.tecmint.com/different-types-of-linux-shells/
• https://www.howtogeek.com/163127/how-powershell-differs-from-the-windows-command-prompt/