(B)itovi (A)nd (SH)

Easily writing predictable shell scripts

tl;dr / TOC

1. Which shell should I use?
2. High-leverage tooling (use a linter!)
3. Shell scripting basics
4. Demo

Which shell should I use?

• bash
• commonly used
• widely available, but may need to be installed (e.g. on Alpine Linux or BSDs)
• Out-of-date on OS X (two major versions behind), may affect some scripts run there
• Run brew install bash and change your shell to /usr/local/bin/bash to get the latest
• sh
• Follows the POSIX standard[3],
• Most portable (especially between musl/BSD systems)
• Missing some features (e.g. arrays), but should be sufficient for most use cases

Use a linter! Rationale

As with any programming language-- you should be using a linter!

Linters:

• Point out errors or potential mistakes
• Teach you things you didn’t know
• Catch typos (humans are bad at repetitive tasks)

Use a linter! shellcheck

​

https://www.shellcheck.net/ is an excellent linter that covers:

• bash vs sh (POSIX compliance for script portability)
• Common mistakes (see Gallery of bad code on their README)
• Long-form explanations on their wiki that explain:
• Why you’re getting this error/warning
• Alternatives to fix it
• How to ignore it, if you know your usage is safe

Use a linter! Installing shellcheck

​

Shellcheck is available for all common platforms, and can even be run from docker. Install instructions in shellcheck's README.

Editor plugins that can automatically show shellcheck errors in your editor:

• Vim: https://github.com/vim-syntastic/syntastic
• Emacs: https://www.flycheck.org
• VSCode: https://github.com/timonwong/vscode-shellcheck
• Atom: https://atom.io/packages/linter-shellcheck
• More in shellcheck's README

Bonus: format your scripts

• You can use mvdan/sh (shfmt) to format your shell scripts.
• Consistency helps other devs read your code and helps prevent typos and errors
• goto fail, anyone?

​

Basics - The anatomy of a script

#!/bin/bash

echo “hello world”

“hashbang” or “shebang” line -> specifies which interpreter to use[1][2]

one or more commands to run

Basics - Exit statuses

​

• Commands return a number after finishing
• 0 => success
• Nonzero => some kind of error[4]
• Usually “1” is used for failure, but there are other reserved/suggested numbers for specific cases (e.g. could not write to disk)
• Use echo $? to see the exit status of the last command
• cat does-not-exist.txt; echo $? (will print an error, then ‘1’)
• These are used in conditionals, loops, and boolean expressions!

Basics - Conditionals

​

if command; then

echo ‘command exited 0!’

elif other-command; then

echo ‘other-command exited 0, but command exited nonzero!’

else

echo ‘both commands exited nonzero’

fi

​

# takeaway: if statements use exit statuses, not output

​

Basics - Conditionals with `test`

The `test` command is the same as `[` (see `man test` for details)

​

if [ “$var” = some-string ]; then

echo ‘$var equals “some-string”’

elif [ $(grep -c str file.txt) -gt 10 ]; then

echo ‘“str” shows up more than 10x in file.txt’

fi

​

test “$var” = some-string && echo ‘Same as the first `if`’

# takeaway: see `man test` for help with `[` (most comparisons)

​

​

Basics - Loops

# print files in a directory

for file in $(ls directory/); do

echo “directory/$file”

done

# count to 5

for num in 1 2 3 4 5; do

echo “$num”

done

# echo statements or quit on ‘q’

statement=’’

while [ “$statement” != ‘q’ ]; do

echo “Text? (q to quit): “

read statement

echo “You entered: $statement”

done

# wait for file to exist

name=file.txt

while [ -f “$name” ]; do

sleep 1

done

echo “file $name should exist now”

Basics - AND / OR

&& || both deal with exit statuses:

test (prog1 && prog2) && echo “prog1 and prog2 both returned 0”

​

if grep something file.txt || [ $var -gt 2 ]; then

echo “Found ‘something’ in file.txt”

echo or

echo ‘$var is a number greater than 2!’

fi

​

Basics - output

• Programs emit output over stdout (standard out) and stderr (standard error)
• You can redirect stdout with >
• You can redirect stderr with 2>

# ignore error output

echo ‘test’ 2> /dev/null # prints ‘test’

# ignore standard output

cat file-that-does-not-exist.txt > /dev/null # prints an error

Summary

• Shell scripting is a useful, iterative, high-leverage way to automate systems tasks.
• Using a linter will help you:
• Learn new things
• Catch common errors
• Avoid typos
• You can (and should!) format your shell scripts with shfmt
• Specify which shell you’re script uses with #!/path/to/shell at the top
• Exit statuses are used for boolean logic and flow control

References

1. https://mywiki.wooledge.org/BashGuide/CommandsAndArguments#Scripts
2. https://stackoverflow.com/questions/10376206/what-is-the-preferred-bash-shebang/10383546#10383546
3. https://en.wikipedia.org/wiki/POSIX
4. https://wiki.bash-hackers.org/dict/terms/exit_status

Demo

Questions?