LFCE

Network-Services

Packet Filtering

Shell Scripting

Positional Parameters

  1. $?
  1. result of previous command
  1. $#
  1. number of parameters
  1. $1
  1. first positional parameter similarly $2, $3, …

Exceptional Handling

  1. -e mode
  1. In this mode any command your script runs which returns a non-zero exitcode - an error in the world of shell - will cause your script to itself terminate immediately with an error.
  2. Two ways to implement
  1. #!/bin/sh -e
  2. set -e
  1. trap is a snippet of code that the shell executes when it exits or receives a signal. ex:,
  1. pressing CTRL-C in the terminal where the script is running generates the INT signal.
  2. killing the process by default generates a TERM (I.e., terminate) signal.

#!/bin/sh -e

TMPFILE=$(tempfile)

trap 'echo "removing $TMPFILE"; rm -f $TMPFILE' INT TERM EXIT

echo TMPFILE=$TMPFILE

…..

…..

Debug

  1. -x mode to debug a shell script:

#!/bin/bash

echo 'Hi'

echo 'I will now fetch you a list of connected users:'

…..

Now, running a shell script with -x option enables the display of commands and its arguments when they are executed.

nikesh@poison:~$ bash -x script.sh

+ echo Hi

Hi

+ echo 'I will now fetch you a list of connected users:'

I will now fetch you a list of connected users:

……..

Reference

  1. http://linuxcommand.org/wss0150.php
  2. http://www.turnkeylinux.org/blog/shell-error-handling
  3. http://linuxpoison.blogspot.com/2010/11/how-to-debug-shell-scripts.html

Standard File Descriptors

The numbers are file descriptors and only the first three (starting with zero) have a standardized meaning:

0 - stdin
1 - stdout
2 - stderr

You can either redirect a file descriptor to a file with > or redirect it to another file descriptor with >&

Descriptive example

3>&1 1>&2 2>&3

So basically STDOUT and STDERR are switched as follows, these are the steps:

If the program now prints something to the file descriptor 1 it will be printed to the file descriptor 2 et vice versa.

Reference

  1. 3>&1 1>&2 2>&3 - http://unix.stackexchange.com/questions/42728/what-does-31-12-23-do-in-a-script
  2. Using custom file descriptors to allow control/configurability over stdout and std err https://coderwall.com/p/xcoraw

Programming Constructs

Functions

Commands

  1. who am i : Displays the user name, terminal id and time and date
  1. sandeep-kunkunuru pts/0        2012-08-11 13:52 (:0.0)
  1. mkdir  ~/dev/shell_scripting/
  2. cd  ~/dev/shell_scripting/
  3. pwd: Tells the directory in which the shell is
  1. /home/sandeep-kunkunuru/dev/shell_scripting
  1. touch test1
  1. creates an empty file by the name test1
  2. can be used to create as many empty files as you can simultaneously just  a sequence of file names followed by Touch command to be created.
  1. cat test
  1. display contents of file test 1
  1. cat > test
  1. opens a file and blinks the cursor indicating that I can type in content.
  2. type as much as you want and press ctrl + d i.e. ctrl and and d keys together indicating end of document.
  1. cat < test
  1. displays contents of file test
  1. cat test1 test2 > test
  1. concatenates contents of test1 and test2 files in that order and writes them to the file test.
  1. mv test1 test4
  1. renames the files or directories test1 to test4
  1. rm new
  1. if new is a directory -r option is required.e.g. “rm -r new” or “rmdir new”
  2. otherwise deletes the file
  1. cp cptest destination/
  1. copies the file “cptest” to the destination folder
  1. ln lntest hard_link
  1. create a hard link file hard_link for the file lntest
  2. replicates the file means creating a duplicate file physically in same directory and named by the given name (second name).
  1. ln -s lntest soft_link
  1. create a soft link (shortcut) for the file lntest
  1. ls
  1. lists all files along with permissions given to the owner, owner group and all users.
  1. ls -l
  1. list all files along with permission in long listing format and total blocks (1024 bytes)
  2. d -directory, r - 4, w -2, x -1
  1. ls -a
  1. list all files along with hidden files
  1. umask
  1. lists default permissions as an octal number - 0002
  2. this number is used to derive the default permissions
  1. for files it is 666 - umask for the files
  2. for folder it is 777 - umask for the folders
  1. touch .hidden_file
  1. creates a hidden file
  1. On a directory you need x previliges to be able to double click and open it, move to it and list files etc. i.e. essentially execute commands against a directory.
  2. uname
  1. display kernel for example
  2. Linux
  1. uname -a
  1. display all information for example
  2. Linux sandeepkunkunuru-Latitude-E6520 3.2.0-29-generic #46-Ubuntu SMP Fri Jul 27 17:03:23 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux
  1. file *
  1. displays all files/folders along with their type. for example
  2. cptest:      ASCII text

destination: directory

  1. wc jazzy
  1. number of lines, words, alphabets for example
  2. 4 19 96 jazzy
  3. use -l for just lines and use -w for words
  1. sort animals
  1. sort all words in the document in the alphabetical order
  1. sort
  1. blinks the cursor so that we can type in content . Use ctrl + d to sort it.
  1. cut -d "-" -f 1,3 players
  1. use - as the delimiter display 1, 3 fields in the file players. for example
  2. Name-Age

Sourav-30

Sachin-35

Sandeep-30

michael Phelps-27

Kobe bryant-34

  1. dd if=test of=out conv=ucase
  1. convert an input file test into an output file out after converting content as upper case.
  2. there are a lot of other applications of dd
  1. format drives
  2. extract files from iso images
  1. dd if-test of=test2 conv=ebcdic
  1. convert encoding type to european
  1. man cat
  1. use q to exit
  1. sudo apt-get install ncompress
  2. compress Statement.txt
  1. this creates the file Statement.txt.Z
  1. uncompress Statement.txt.Z
  1. this decompresses the file back to Statement.txt
  1. zcat Statement.txt.Z
  1. this command prints the contents of the file Statement.txt from within its compressed form.
  1. ifconfig
  1. displays ip address of the system
  1. which gedit
  1. prints the location of the current gedit
  2. result is /usr/bin/gedit
  1. whereis gedit
  1. prints the location details of gedit
  2. result is “gedit: /usr/bin/gedit /usr/lib/gedit /usr/bin/X11/gedit /usr/share/gedit /usr/share/man/man1/gedit.1.gz”
  1. xdg_open Statement.txt
  1. directly opens the file using the corresponding default software application e.g. gedit for txt files
  1. gksu gedit /usr/bin/openproj
  1. Open a software application with root privileges.
  2. result is open the file openproj with the software application gedit as root
  1. dpkg --list
  1. list all packages installed on the system
  1. strings ~/Downloads/AMEX_to_VIQ_Q3ActualizedPivot.xls
  1. For each file given, GNU strings prints the printable character sequences that are at least 4 characters long. strings is mainly useful for determining the contents of non-text files.
  1. history
  1. Lists all the commands that have been executed thus far
  1. top 
  1. list of top processes by the amount of resources that they consume
  1. htop 
  1. It is designed to replace the Unix program top

Special Notations

  1. /dev/null

In Unix-like operating systems, /dev/null or the null device is a special file that discards all data written to it (but reports that the write operation succeeded) and provides no data to any process that reads from it (yielding EOF immediately).[1]

In programmer jargon, especially Unix jargon, it may also be called the bit bucket[2] or black hole.

Assignments

  1. Recursively change extension of files in a folder ex: rename all .so files under the current folder to .dylib
  1. find -L . -type f -name "*.so" -print0 | while IFS= read -r -d '' FNAME; do
       mv -- "$FNAME" "${FNAME%.so}.dylib"
    done
  2. http://stackoverflow.com/questions/2171966/bash-rename-extension-recursive
  1. Display all files and folders under the current folder

        Solution

Use find command to list all files and folders under a specified directory

find ./

  1. Find a command from the command history

Solution

Use history command to list all commands executed thus far and then pipe the list to grep to search for the command  of your interest.

history | grep ps

  1. Search for all files of a specific type, at any depth within a given directory, which have a specific string irrespective of the case.

        Solution

        grep -i -r -n  --include=\*.java UPDATE_CONSTRAINT ./

        

        Reference

  1. http://stackoverflow.com/questions/4121803/how-can-i-use-grep-to-find-a-word-inside-a-folder
  2. http://stackoverflow.com/questions/221921/grep-exclude-include-syntax-do-not-grep-through-certain-files

  1. Find out if a excel sheet is readable without opening it.

        Solution

Use “strings” command to check if the excel sheet contains strings. If you see adequate number of strings in the excel sheet then most probably excel sheet is readable.

strings ~/Downloads/AMEX_to_VIQ_Q3ActualizedPivot.xls

Reference

  1. Find all running processes containing a string and do something with each of the processes e.g. find ‘postgres’ and kill them
  1. kill -9 `ps aux | grep postgres | grep -v grep | awk ‘{print $2}’`
  2. http://www.howtogeek.com/howto/ubuntu/kill-a-process-by-process-name-from-ubuntu-command-line/
  1. Find working directory of a process
  1. ps aux | grep {process-name} # to get pid
  2. pwdx 13456 # use pid
  3. http://www.cyberciti.biz/tips/linux-report-current-working-directory-of-process.html
  1. Thunderbird is linked to bing how can we change that to google


Appendix

POSIX

POSIX :  "Portable Operating System Interface",[1] is a family of standards specified by the IEEE for maintaining compatibility between operating systems. POSIX defines the application programming interface (API), along with command line shells and utility interfaces, for software compatibility with variants of Unix and other operating systems.

File Descriptors

The file descriptors for input, output, and error.

A file descriptor (FD) is an abstract indicator for accessing a file.

In POSIX, a file descriptor is an integer, specifically of the C type int. It is an index for an entry in a kernel-resident array data structure called a file descriptor table containing the details of open files. Each process has its own file descriptor table. The process itself cannot read or write the file descriptor table directly it passes the file descriptor to the kernel through a system call, and the kernel will access the file on behalf of the process.

On Linux, the set of file descriptors open in a process can be accessed under the path /proc/PID/fd/, where PID is the process identifier.