Linux Summer Training 23

By Ahmed Khaled, 29/Aug/23

Text processing

yees?

yes please?�

Outcomes

• Increase productivity (By automating tasks)
• Manipulate text (Search, Filter, Replace, Transform etc.)
• Write advanced bash scripts
• Validate user input in a smarter way
• Understand new concepts

​

​

Agenda

Simple commands

​

​

Regex

​

02

Awk

05

Grep

03

Summary

06

01

04

Sed

Simple commands

Sort

Options

• -n -> numerical
• -r -> reverse

Rev

Uniq

Options

• -c -> count
• -d -> duplicates
• -u -> unique

Cut

Options

• -c -> characters
• -f -> fields
• -d -> delimiter

Example

Tr

Options

• -d -> delete
• -s -> replace sequence with char

Hands on

Sort�Rev

Uniq

Cut (-c, -f, -d)

Tr

Hands on�Solution!

Paste

Options

• -s -> single line
• -d -> delimiter

Recap

Regex

Example

Regex symbols

1 or more of the previous char

+

0 or 1 of the previous (optional)

0 or more of the previous (‘+’ and ‘?’)

Anything

Characters, digits, spaces, word breaks

Beginning , end of line

?

*

.

\w, \d, \s, \b

^, $

Regex symbols

Character sets (^ -> except)

[ ]

Ranges {4} , {4, } , {4,5}

Grouping

Reference

{ }

( )

\1, \2

OR operator

|

Regex applications

• Programming languages, IDEs
• Finding patterns
• User input validation
• Cleaner code
• Smart search (Solution)

​

​

BREAK

Grep

Options

• -B, -E ({ }, (), ?, |, +), -P (\d) -> (Regex types)
• -i -> Case insensitive
• -n -> Line numbers
• -c (Count)
• -A, -B, -C -> (After, Before, Combination)
• -r -> Recursive

​

​

Hands on

• Grep email addresses

Ex: ahmed@gmail.com

​

• Grep old discord usernames

Ex: Ahmed#1010

• Grep credit cards that contain 2 similar consecutive digits

Sed

Syntax

sed “s/pattern/replace/flags” file.txt

Sed options

Detect ‘/’s

Detect pattern in a line first (/ /)

Delete (d)

Multiple sed commands

Insert (&)

Hands on

Hands on�Solution!

Awk

How it works

Syntax

awk ‘/pattern/ {action}’ file.txt

Example

Awk features

Arithmetic

If conditions + &&, ||

Relational expressions (~, !~, >)

Range patterns (,)

Begin, end

C = (F – 32) * 5/9

Hands on�Solution!

Summary

Outcomes

• Increase productivity (By automating tasks)
• Manipulate text (Search, Filter, Replace, Transform etc.)
• Write advanced bash scripts
• Validate user input in a smarter way
• Understand new concepts

​

​

Thanks!