Python Programming

UNIT-3

Syllabus

Regular Expressions:

Introduction, Special Symbols and Characters, Res and Python

​

Multithreaded Programming:

Introduction, Threads and Processes, Python, Threads, and the Global

Interpreter Lock, Thread Module, Threading Module, Related Modules

Regular Expressions: �

• A regular expression (or RE) specifies a set of strings that matches it.
• A regular expression is a string that contains special symbols and characters to find and extract the information needed by us from the given data.
• A regular expression helps us to search information, match, find and split information.
• Regular Expression is also called simple regex .
• Regular Expression (RegEx) is a sequence of characters that defines a search pattern.

​

any five letter string starting with a and ending with s.

Python provides re module that stands for regular expressions.

^a...s$

Python has a module named re to work with regular expression

re.match function to grab pattern within test_string., the The method returns a match object if the search is successful. If not, it returns None

import re

pattern = '^a...s$'

test_string = 'abyss'

result = re.match(pattern, test_string)

if result:

print("Search successful.")

else:

print("Search unsuccessful.")

Output:

Search successful.

Metacharacters

Metacharacters are characters with a special meaning

[] Square Brackets�

• The [] square brackets represent the set of the characters. 
•  to match the words that are inside the square bracket to the target string.

Ex: We can use the [abc] to match such pattern. The [abc] will match contains any of a, b or c.

• [0-5] - It is same as the [012345].
• [A-E] - It is same as the [ABCDE].
• [a-d] - It is same as the [abcd].

#Find all lower case characters between "a" and “j"

import re

txt = "Python is a most popular programming language.“

x = re.findall("[a-j]", txt)

print(x)

['h', 'i', 'a', 'a', 'g', 'a', 'i', 'g', 'a', 'g', 'a', 'g', 'e']

['h', 'i', 'a', 'a', 'g', 'a', 'i', 'g', 'a', 'g', 'a', 'g', 'e']

Output:

['h', 'i', 'a', 'a', 'g', 'a', 'i', 'g', 'a', 'g', 'a', 'g', 'e']

import re

​

txt = "hello planet“

x = re.findall("he..o", txt)

print(x)

#Search for a sequence that starts with "he", followed by two (any) characters, and an "o":

Output: ['hello']

import re

​

txt = "hello planet"

​

x = re.findall("^hello", txt)

if x:

print("Yes, the string starts with 'hello'")

else:

print("No match")

Output: Yes, the string starts with 'hello'

import re

​

txt = "hello planet"

​

x = re.findall("planet$", txt)

if x:

print("Yes, the string ends with 'planet'")

else:

print("No match")

Output:

Yes, the string ends with 'planet'

#Search for a sequence that starts with "he", followed by 0 or more (any) characters, and an "o":

import re

txt = "hello planet“

​

x = re.findall("he.*o", txt)

​

print(x)

​

Output:

['hello']

#Search for a sequence that starts with "he", followed by 1 or more (any) characters, and an "o":

​

import re

​

txt = "hello planet"

​

x = re.findall("he.+o", txt)

​

print(x)

Output:

​

['hello']

#Search for a sequence that starts with "he", followed by 0 or 1 (any) character, and an "o":

import re

​

txt = "hello planet"

​

x = re.findall("he.?o", txt)

​

print(x)

Output:

​

[ ]

#Search for a sequence that starts with "he", followed exactly 2 (any) characters, and an "o":

​

import re

​

txt = "hello planet"

​

x = re.findall("he.{2}o", txt)

​

print(x)

Output:

​

['hello']

#Check if the string contains either "falls" or "stays":

import re

​

txt = "The rain in Spain falls mainly in the plain!"

​

x = re.findall("falls|stays", txt)

print(x)

if x:

print("Yes, there is at least one match!")

else:

print("No match")

�

Output:

['falls']�Yes, there is at least one match!

A special sequence is a \ followed by one of the characters in the list below, and has a special meaning

SPECIAL SEQUENCES

#Check if the string starts with "The":

import re

​

txt = "The rain in Spain"

​

x = re.findall("\AThe", txt)

​

print(x)

​

if x:

print("Yes, there is a match!")

else:

print("No match")

Output:

​

['The']�Yes, there is a match!

#Check if "ain" is present at the beginning of a WORD:

import re

​

txt = "The rain in Spain“

x = re.findall("\bain", txt)

print(x)

if x:

print("Yes, there is at least one match!")

else:

print("No match")

[] No match

Output:

[ ]

No match

#Check if "ain" is present, but NOT at the beginning of a word:

import re

​

txt = "The rain in Spain”

x = re.findall("\Bain", txt)

​

print(x)

if x: Output:

print("Yes, there is at least one match!") ['ain', 'ain']

Yes, there is at least one match!

else:

print("No match")

['ain', 'ain'] Yes, there is at least one match!

#Check if the string contains any digits (numbers from 0-9):

​

import re

​

txt = "The rain in Spain"

​

x = re.findall("\d", txt)

​

print(x)

​

if x:

print("Yes, there is at least one match!")

else:

print("No match

Output:

[ ]�No match

#Return a match at every no-digit character:

​

import re

​

txt = "The rain in Spain“

x = re.findall("\D", txt)

print(x)

​

if x:

print("Yes, there is at least one match!")

else:

print("No match")

Output:

​

['T', 'h', 'e', ' ', 'r', 'a', 'i', 'n', ' ', 'i', 'n', ' ', 'S', 'p', 'a', 'i', 'n']�Yes, there is at least one match!

#Return a match at every white-space character:

​

import re

​

txt = "The rain in Spain"

​

x = re.findall("\s", txt)

​

print(x)

​

if x:

print("Yes, there is at least one match!")

else:

print("No match")

Output:

​

[' ', ' ', ' ']�Yes, there is at least one match!

#Return a match at every NON white-space character:

import re

​

txt = "The rain in Spain"

​

x = re.findall("\S", txt)

​

print(x)

​

if x:

print("Yes, there is at least one match!")

else:

print("No match")

Output:

​

['T', 'h', 'e', 'r', 'a', 'i', 'n', 'i', 'n', 'S', 'p', 'a', 'i', 'n']�Yes, there is at least one match!

#Return a match at every word character (characters from a to Z, digits from 0-9, and the underscore _ character):

​

import re

txt = "The rain in Spain“

x = re.findall("\w", txt)

print(x)

​

if x:

print("Yes, there is at least one match!")

else:

print("No match")

Output:

​

['T', 'h', 'e', 'r', 'a', 'i', 'n', 'i', 'n', 'S', 'p', 'a', 'i', 'n']�Yes, there is at least one match!

​

#Return a match at every NON word character (characters NOT between a and Z. Like "!", "?" white-space etc.):

​

import re

​

txt = "The rain in Spain“

x = re.findall("\W", txt)

print(x)

​

if x:

print("Yes, there is at least one match!")

else:

print("No match")

Output:

​

[' ', ' ', ' ']�Yes, there is at least one match!

• Python has a module named re to work with regular expressions. To use it, we need to import the module.

​

• The module defines several functions and constants to work with RegEx.

RegEx Functions: The re module offers a set of functions that allows us to search a string for a match

findall() Function�

The findall() function returns a list containing all matches.

#Return a list containing every occurrence of "ai“

import re

​

txt = "The rain in Spain"

x = re.findall("ai", txt)

print(x)

Output:

​

['ai', 'ai']

#Check if "Portugal" is in the string:

​

import re

txt = "The rain in Spain“

x = re.findall("Portugal", txt)

print(x)

if (x):

print("Yes, there is at least one match!")

else:

print("No match")

Output:

[]�No match

search() Function�

• The search() function searches the string for a match, and returns a Match object if there is a match.
• If there is more than one match, only the first occurrence of the match will be returned:

​

import re

​

txt = "The rain in Spain"

x = re.search("\s", txt)

​

print("The first white-space character is located in position:", x.start())

Output:

​

The first white-space character is located in position: 3

split() Function�

• The split() function returns a list where the string has been split at each match.
• You can control the number of occurrences by specifying the maxsplit parameter

#Split the string at every white-space character:

import re

​

txt = "The rain in Spain"

x = re.split("\s", txt)

print(x)

Output:�

['The', 'rain', 'in', 'Spain']

#Split the string at the first white-space character:

​

import re

​

txt = "The rain in Spain"

x = re.split("\s", txt, 1)

print(x)

Output:�

['The', 'rain in Spain']

sub() Function�

The sub() function replaces the matches with the text of your choice.

You can control the number of replacements by specifying the count parameter

#Replace all white-space characters with the digit "9":

import re

​

txt = "The rain in Spain"

x = re.sub("\s", "9", txt)

print(x)

Output:

The9rain9in9Spain

​

#Replace the first two occurrences of a white-space character with the digit 9:

​

import re

​

txt = "The rain in Spain"

x = re.sub("\s", "9", txt, 2)

print(x)

Output:�

The9rain9in Spain

Match Object�

• A Match Object is an object containing information about the search and the result.
• If there is no match, the value None will be returned, instead of the Match Object.

#The search() function returns a Match object:

​

import re

​

txt = "The rain in Spain"

x = re.search("ai", txt)

print(x)

​

Output:

<re.Match object; span=(5, 7), match='ai'>

<re.Match object; span=(5, 7), match='ai'>

<re.Match object; span=(5, 7), match='ai'>

#Search for an upper case "S" character in the beginning of a word, and print its position:

​

import re

​

txt = "The rain in Spain"

x = re.search(r"\bS\w+", txt)

print(x.span())

​

Output:�(12, 17)

​

Difference between process and thread�

What is Multiprocessing?

• A multiprocessing system has more than two processors. The CPUs are added to the system that helps to increase the computing speed of the system. Every CPU has its own set of registers and main memory.

Characteristics of Multiprocessing

• Here are the essential features of Multiprocessing:
• Multiprocessing are classified according to the way their memory is organized.
• Multiprocessing improves the reliability of the system
• Multiprocessing can improve performance by decomposing a program into parallel executable tasks.

What is Multithreading?

• Multithreading is a program execution technique that allows a single process to have multiple code segments (like threads). It also runs concurrently within the “context” of that process. Multi-threaded applications are applications that have two or more threads that run concurrently. Therefore, it is also known as concurrency.

​

Thread Life Cycle

New: This is the initial state of a thread. When a thread is created but not yet started, it is in the “new” state. At this point, the thread’s resources have been allocated, but it has not begun executing.

​

Runnable/Running: Once a thread is started using the start() method, it transitions to the “runnable” state. In this state, the thread is eligible to run, but it may not be actively executing if the Python interpreter scheduler has not yet selected it for execution. When the thread’s turn comes, it moves to the “running” state, where it executes its code.

​

Blocked/Waiting: Threads can transition to the “blocked” or “waiting” state when they are waiting for some event or condition to be met. This could include waiting for I/O operations, acquiring locks or semaphores, or waiting on a condition variable. Threads in these states are not actively executing and are not eligible to run until the event they are waiting for occurs.

​

Terminated/Dead: When a thread finishes its execution or is explicitly terminated using the join() method or other termination mechanisms, it enters the “terminated” or “dead” state. In this state, the thread’s resources are released, and it cannot be restarted. A terminated thread cannot return to any other state in its life cycle.

How to achieve multithreading in Python?- Creation of Thread��

• There are two main modules of multithreading used to handle threads in Python.
• The thread module
• The threading module

​

Note: Thread module

• Part Of Python's Standard Library Since Version 2.
• It Is A Low-level Api For Thread Management

Thread modules�

• can only be accessed with _thread that supports backward compatibility.
• To implement threads in Python, we need to import  thread module and then define a function that performs some action by setting the target with a variable.

Syntax: thread.start_new_thread ( function_name, args[, kwargs] )  

import _thread

def MyThread1():

print("This is thread1")

def MyThread2():

print("This is thread2")

_thread.start_new_thread(MyThread1, ())

_thread.start_new_thread(MyThread2, ())

Output:

This is thread1

This is thread2

Syntax:

threading.Thread(target, name, args, kwarg, daemon)

Parameters

target − function to be invoked when a new thread starts. Defaults to None, meaning nothing is called.

name − is the thread name. By default, a unique name is constructed such as "Thread-N".

daemon − If set to True, the new thread runs in the background.

args and kwargs − optional arguments to be passed to target function.

2. Threading Module: The newer threading module provides much more powerful, high-level support for thread management.

from threading import *

def show():

print("This is a thread")

t = Thread(target=show())

t.start()

print("Thread Execution Completed")

Output:

This is a thread

Thread Execution Completed

Basic Python threading program- Example 1

Basic Python threading program- Example 2

import threading  

def print_hello(n):  

    Print("Hello, how old are you? ", n)  

​

T1 = threading.Thread( target = print_hello, args = (20, ))  

​

T1.start()  

​

Print("Thank you")    

Output:

Hello, how old are you? 20

Thank you

from threading import *

def MyThread1():

print("I am in thread1.", "Current Thread in Execution is", current_thread().getName())

def MyThread2():

print("I am in thread2.", "Current Thread in Execution is", current_thread().getName())

t1 = Thread(target=MyThread1, args=[])

t2 = Thread(target=MyThread2, args=[])

t1.start()

t2.start()

Deadlocks and Race conditions��

Critical Section: It is a fragment of code that accesses or modifies shared

variables and must be performed as an atomic transaction.

​

Context Switch: It is the process that a CPU follows to store the state of a

thread before changing from one task to another so that it can be resumed

from the same point later.

Race Conditions��

A race condition is an unwanted state of a program which occurs when a system performs two or more operations simultaneously. 

i=0

for x in range(100):

print(i)

i+=1

• If you create n number of threads which run this code at once, you cannot determine the value of i (which is shared by the threads) when the program finishes execution.

​

• This is because in a real multithreading environment, the threads can overlap, and the value of i which was retrieved and modified by a thread can change in between when some other thread accesses it.

Deadlocks

• Deadlocks Is By Using The Classic Computer Science Example Problem Known As The Dining Philosophers Problem.

• a deadlock occurs when different threads or processes (philosophers) try to acquire the shared system resources (forks) at the same time.
• As a result, none of the processes get a chance to execute as they are waiting for another resource held by some other process.

Multithreading in Python��

• Python virtual machine is not a thread-safe interpreter, meaning that the interpreter can execute only one thread at any given moment.
• This limitation is enforced by the Python Global Interpreter Lock (GIL), which essentially limits one Python thread to run at a time.
• GIL ensures that only one thread runs within the same process at the same time on a single processor. 

Synchronizing threads��

The idea is that when a thread wants access to a specific resource, it acquires a lock for that resource.

Once a thread locks a particular resource, no other thread can access it until the lock is released.

​

As a result, the changes to the resource will be atomic, and race conditions will be averted.

A lock is a low-level synchronization primitive implemented by the __thread module. At any given time, a lock can be in one of 2 states: locked or unlocked. 

It supports two methods:

​

acquire()When the lock-state is unlocked, calling the acquire() method will change the state to locked and return. However, If the state is locked, the call to acquire() is blocked until the release() method is called by some other thread.

​

release()The release() method is used to set the state to unlocked, i.e., to release a lock. It can be called by any thread, not necessarily the one that acquired the lock.

Synchronization in Python- Python Global Interpreter Lock �

trouble arises when different threads try to work on the same data at same time. To avoid such troubles and process the threads safely without any problems, the code synchronization should be implemented which restricts multiple threads to work on the same code at the same time.

• We can solve these inconsistency problems by synchronizing the threads such that they will be executed one by one.

• Locks are the most fundamental synchronization mechanism provided by the threading module. We can create Lock object as follows,

l=Lock()

The Lock object can be held by only one thread at a time. If any other thread wants the same lock then it will have to wait until the other one releases it.

acquire() method: A Thread can acquire the lock by using acquire() method�Ex: l.acquire()�release() method: A Thread can release the lock by using release() method.� Ex: l.release()

Note: Only the thread currently holding the lock is allowed to call the release() method thread, otherwise we will get Runtime Error saying, RuntimeError: release unlocked lock

Example:1

from threading import *

l=Lock()

l.acquire()

print("lock acquired")

l.release()

print("lock released")

​

Output:

EXAMPLE

Producer- Consumer Problem

Producer-Consumer Problem consists of 3 components:

1. Bounded Buffer

• A buffer is temporary storage that is accessible by different threads.
• A simple example of a buffer is an array.
• Multiple threads can read the data from the buffer as well as can write the data to the buffer concurrently
• A bounded buffer is one that has a limited capacity and can’t store the data beyond its capacity.

2. Producer Thread: A Producer Thread is one that generates some data, puts it into the buffer, and starts again until all the data needed is not produced.

Example: a thread that downloads some data over the network and stores it temporarily into the buffer.

3. Consumer Thread: A Consumer Thread is one that consumes the data present inside the buffer, uses it for some task, and starts again until the task assigned to the thread is not completed.

Example: a thread that reads the data that is downloaded over the internet and stores it in the database.

Issues In Producer-consumer Problem

• If the Producer Thread is trying to generate the data into the buffer and found that the buffer is already full, the Producer Thread can neither add more data inside the buffer nor it can overwrite the existing data that has not been consumed by the consumer yet. Therefore, the Producer Thread should stop itself until some data is not consumed from the buffer. This scenario might be possible if the Producer Thread is fast.
• If the Consumer Thread is trying to consume the data from the buffer but found that the buffer is empty, the Consumer Thread can’t take the data and it should stop itself until some data is not added into the buffer. This scenario might be possible if the Consumer Thread is fast.
• Since the buffer is shared among different threads that can access the data from the buffer simultaneously, race conditions are possible and both threads should not access the shared buffer at the same time. Either the Producer Thread should add the data to the buffer and the Consumer Thread should wait or the Producer Thread should wait while the Consumer Thread is working on shared buffer to read the data.

Solution to the problem using Semaphore

• We can solve this problem with the help of Semaphores, which is a tool for synchronization between threads.
• We maintain 3 Semaphores in order to tackle 3 issues defined in our problem statement of the Producer-Consumer problem.

empty: This semaphore stores the number of slots that are empty in our buffer. The initial value of this semaphore is the size of our bounded buffer. Before adding any data in the buffer, the Producer thread will try to acquire this semaphore and will decrease its value by 1. If the value of this semaphore is already 0, this means that the buffer is full and our empty semaphore will block the Producer Thread until the value of the empty semaphore becomes greater than 0. Similarly, after the Consumer Thread has consumed the data from the buffer, it will release this semaphore, increasing the value of the semaphore by 1.

full: This semaphore stores the number of slots that are full in our buffer. The initial value of this semaphore is 0. Before consuming the data from the buffer, the Consumer Thread will try to acquire this semaphore. If the value of this semaphore is already 0, this means that the buffer is already empty and our full semaphore will block the Consumer Thread until the value of the full semaphore becomes greater than 0. Similarly, the Producer Thread will release this semaphore after it has added one item in it.

​

mutex: This semaphore will handle the race condition by allowing only one semaphore to operate on the shared buffer at a time. The initial value of this semaphore is 1. Before operating on the shared buffer, both threads will try to acquire this semaphore. If any thread found the value of this semaphore as 0, this means that the other thread is operating on the buffer and it will be blocked by the semaphore. After operating on the buffer, the working thread will release this semaphore so that the other thread can operate on the buffer.

We also maintain 2 pointer to help our threads where to add or take the data.

in pointer: This pointer will tell our Producer Thread where to add the next data in the buffer generated by the producer. After adding, the pointer is incremented by 1.

out pointer: This pointer will tell our Consumer Thread where to read the next data from the buffer. After reading, the pointer is incremented by 1.

import threading

import time

# Shared Memory variables

CAPACITY = 10

buffer = [-1 for i in range(CAPACITY)]

in_index = 0

out_index = 0

# Declaring Semaphores

mutex = threading.Semaphore()

empty = threading.Semaphore(CAPACITY)

full = threading.Semaphore(0)

# Producer Thread Class

class Producer(threading.Thread):

  def run(self):

    global CAPACITY, buffer, in_index, out_index

    global mutex, empty, full

    items_produced = 0

    counter = 0

    while items_produced < 20:

      empty.acquire()

      mutex.acquire()

      counter += 1

      buffer[in_index] = counter

      in_index = (in_index + 1)%CAPACITY

      print("Producer produced : ", counter)

      mutex.release()

      full.release()

      time.sleep(1)

      items_produced += 1

# Consumer Thread Class

class Consumer(threading.Thread):

  def run(self):

    global CAPACITY, buffer, in_index, out_index, counter

    global mutex, empty, full

    items_consumed = 0

    while items_consumed < 20:

      full.acquire()

      mutex.acquire()

      item = buffer[out_index]

      out_index = (out_index + 1)%CAPACITY

      print("Consumer consumed item : ", item)

      mutex.release()

      empty.release()     

      time.sleep(2.5)

      items_consumed += 1

# Creating Threads

producer = Producer()

consumer = Consumer()

# Starting Threads

consumer.start()

producer.start()

# Waiting for threads to complete

producer.join()

consumer.join()

Output:

Producer produced :  1

Consumer consumed item :  1

Producer produced :  2

Producer produced :  3

Consumer consumed item :  2

Producer produced :  4

Producer produced :  5

Consumer consumed item :  3

Producer produced :  6

Producer produced :  7

Producer produced :  8

Consumer consumed item :  4

Producer produced :  9

Producer produced :  10

Consumer consumed item :  5

Producer produced :  11

Producer produced :  12

Producer produced :  13

Consumer consumed item :  6

Producer produced :  14

Producer produced :  15

Consumer consumed item :  7

Producer produced :  16

Producer produced :  17

Consumer consumed item :  8

Producer produced :  18

Consumer consumed item :  9

Producer produced :  19

Consumer consumed item :  10

Producer produced :  20

Consumer consumed item :  11

Consumer consumed item :  12

Consumer consumed item :  13

Consumer consumed item :  14

Consumer consumed item :  15

Consumer consumed item :  16

Consumer consumed item :  17

Consumer consumed item :  18

Consumer consumed item :  19

Consumer consumed item :  20

Benefits of Using Python for Multithreading��

• It guarantees powerful usage of PC framework assets.
• Applications with multiple threads respond faster.
• It is more cost-effective because it shares resources and its state with sub-threads (child).
• It makes the multiprocessor engineering more viable because of closeness.
• By running multiple threads simultaneously, it cuts down on time.
• To store multiple threads, the system does not require a lot of memory.