1 of 44

PHÂN TÍCH VÀ THIẾT KẾ THUẬT TOÁN

BÀI 1

THUẬT TOÁN VÀ CÁC VẤN ĐỀ LIÊN QUAN

Khoa Công nghệ Thông tin - UTEHY

1

10/31/2022

Bộ môn Công nghệ Phần mềm,�Khoa Công nghệ Thông tin,�Trường Đại học Sư phạm Kỹ thuật Hưng Yên

2 of 44

Nội dung

Khoa Công nghệ Thông tin - UTEHY

2

10/31/2022

Khái niệm thuật toán

2

Mục tiêu bài học

1

Tại sao phải phân tích thuật toán

3

Những vấn đề phát sinh trong phân tích và thiết kế thuật toán �

4

Tổng kết bài học

5

3 of 44

1. Mục tiêu bài học

  • Trình bày được các khái niệm về thuật toán
  • Giải thích được tại sao phải phân tích thuật toán
  • Trình bày được các đặc trưng của thuật toán
  • Các vấn đề phát sinh trong quá trình phân tích và thiết kế thuật toán

Khoa Công nghệ Thông tin - UTEHY

3

10/31/2022

4 of 44

4

10/31/2022

WHY STUDY: Design & Analysis of Algorithms

5 of 44

5

10/31/2022

  • Efficient algorithms lead to efficient programs.
  • Efficient programs sell better.
  • Efficient programs make better use of hardware.
  • Programmers who write efficient programs are preferred.

WHY STUDY: Design & Analysis of Algorithms

6 of 44

Design and Analysis of Algorithms

L1.6

  • Analysis: predict the cost of an algorithm in terms of resources and performance

  • Design: design algorithms which minimize the cost

7 of 44

7

10/31/2022

7

Objectives

  • To gain experiences in fundamental techniques used for algorithm analysis and the main methodologies used for the design of efficient algorithms.
  • To study the most important computer algorithms of current practical use.

8 of 44

8

10/31/2022

8

Course Outcomes

After completing the course, students should be able to:

  1. Determine the time and space complexity of simple algorithms.
  2. Use big O, omega, and theta notation to give asymptotic upper, lower, and tight bounds on time and space complexity of algorithms.
  3. Recognize the difference between mathematical modeling and empirical analysis of algorithms, and the difference between deterministic and randomized algorithms.
  4. Deduce recurrence relations that describe the time complexity of recursively defined algorithms and work out their particular and general solutions.

9 of 44

9

10/31/2022

9

Course Outcomes (Contd…)

5. Practice the main algorithm design strategies of Brute Force, Divide & Conquer, Greedy methods, Dynamic Programming, Backtracking and Branch & Bound and implement examples of each.

6. Implement the most common sorting and searching algorithms and perform their complexity analysis.

7. Solve problems using the fundamental graph algorithms including DFS, BFS, SSSP and APSP, transitive closure, topological sort, and the minimum spanning tree algorithms.

8. Evaluate, select and implement algorithms in programming context.

10 of 44

10

10/31/2022

10

NỘI DUNG MÔN HỌC

  • Phương pháp Phân tích thuật toán
  • Chiến lược Thiết kế thuật toán:
    • Thuật toán Chia để trị
    • Thuật toán Quy hoạch động
    • Thuật toán Tham lam

11 of 44

What is an algorithm?

An algorithm is a list of steps (sequence of unambiguous instructions ) for solving a problem that transforms the input into the output.

11

10/31/2022

“computer”

problem

algorithm

input

output

12 of 44

Difference between Algorithm and Program

12

10/31/2022

S.No

Algorithm

Program

1

Algorithm is finite

Program need not to be finite

2

Algorithm is written using natural language or algorithmic language

Programs are written using a specific programming language

13 of 44

Fundamentals of Algorithm and Problem Solving

13

10/31/2022

14 of 44

Problem Solving Techniques

  1. Understand the problem or Review the Specifications.
  2. Plan the logic
  3. a) (Informal Design)

i. List major tasks

ii. List subtasks,sub-subtasks & so on

b) (Formal Design)

i. Create formal design from task lists

ii. Desk check design

  1. Writing an algorithm 5. Flowcharting 6. Coding

7. Translate the program into machine language

8. Test the program

i. If necessary debug the program

  1. Documentation
  2. Put the program into production. If necessary maintain the program.

14

10/31/2022

15 of 44

Basic Issues Related to Algorithms

  • How to design algorithms

  • How to express algorithms

  • Proving correctness (CM tính đúng đắn)

  • Efficiency (or complexity) analysis
    • Theoretical analysis (Phân tích thực nghiệm)

    • Empirical analysis (Phân tích lý thuyết)

  • Optimality

15

10/31/2022

16 of 44

PROPERTIES OF AN ALGORITHM

  1. An algorithm takes zero or more inputs
  2. An algorithm results in one or more outputs
  3. All operations can be carried out in a finite amount of time
  4. An algorithm should be efficient and flexible
  5. It should use less memory space as much as possible
  6. An algorithm must terminate after a finite number of steps.
  7. Each step in the algorithm must be easily understood for some reading it
  8. An algorithm should be concise and compact to facilitate verification of their correctness.

DAA - Unit - I Presentation Slides

16

10/31/2022

17 of 44

DAA - Unit - I Presentation Slides

17

10/31/2022

Some Important Problem Types

  • Sorting
    • a set of items
  • Searching
    • among a set of items
  • String processing
    • text, bit strings, gene sequences
  • Graphs
    • model objects and their relationships
  • Combinatorial
    • find desired permutation, combination or subset
  • Geometric
    • graphics, imaging, robotics
  • Numerical
    • continuous math: solving equations, evaluating functions

17

18 of 44

Why study algorithms?

  • Theoretical importance

    • the core of computer science

  • Practical importance

    • A practitioner’s toolkit of known algorithms

    • Framework for designing and analyzing algorithms for new problems

18

Example: Google’s PageRank Technology

19 of 44

Euclid’s Algorithm

Problem: Find gcd(m,n), the greatest common divisor of two nonnegative, not both zero integers m and n

Examples: gcd(60,24) = 12, gcd(60,0) = 60, gcd(0,0) = ?

Euclid’s algorithm is based on repeated application of equality

gcd(m,n) = gcd(n, m mod n)

until the second number becomes 0, which makes the problem

trivial.

Example: gcd(60,24) = gcd(24,12) = gcd(12,0) = 12

19

20 of 44

Two descriptions of Euclid’s algorithm

Step 1 If n = 0, return m and stop; otherwise go to Step 2

Step 2 Divide m by n and assign the value of the remainder to r

Step 3 Assign the value of n to m and the value of r to n. Go to� Step 1.�

while n ≠ 0 do

r ← m mod n

m← n

n ← r

return m

20

21 of 44

Other methods for computing gcd(m,n)

Consecutive integer checking algorithm

Step 1 Assign the value of min{m,n} to t

Step 2 Divide m by t. If the remainder is 0, go to Step 3;� otherwise, go to Step 4

Step 3 Divide n by t. If the remainder is 0, return t and stop;� otherwise, go to Step 4

Step 4 Decrease t by 1 and go to Step 2

21

Is this slower than Euclid’s algorithm? How much slower?

O(n), if n <= m , vs O(log n)

22 of 44

Other methods for gcd(m,n) [cont.]

Middle-school procedure

Step 1 Find the prime factorization of m

Step 2 Find the prime factorization of n

Step 3 Find all the common prime factors

Step 4 Compute the product of all the common prime factors� and return it as gcd(m,n)

Is this an algorithm?

How efficient is it?

22

Time complexity: O(sqrt(n))

23 of 44

Sieve of Eratosthenes

Input: Integer n 2

Output: List of primes less than or equal to n

for p ← 2 to n do A[p] ← p

for p ← 2 to n do

if A[p] ≠ 0 //p hasn’t been previously eliminated from the list� j p* p

while j n do

A[j] ← 0 //mark element as eliminated

j j + p

Example: 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

23

Time complexity: O(n)

24 of 44

Two main issues related to algorithms

  • How to design algorithms

  • How to analyze algorithm efficiency

24

25 of 44

Analysis of algorithms

  • How good is the algorithm?
    • time efficiency
    • space efficiency
    • correctness ignored in this course

  • Does there exist a better algorithm?
    • lower bounds
    • optimality

25

26 of 44

Algorithm Efficiency

  • The efficiency of an algorithm is usually measured by its CPU time and Storage space.
  • The time is measured by counting the number of key operations, that is how much time does it take to run the algorithm. For example, in sorting and searching algorithms, number of comparisons.
  • The space is measured by counting the maximum of memory needed by the algorithm. That is, the amount of memory required by an algorithm to run to completion

26

10/31/2022

27 of 44

27

10/31/2022

  • Time and space complexity
    • This is generally a function of the input size
      • E.g., sorting, multiplication
    • How we characterize input size depends:
      • Sorting: number of input items
      • Multiplication: total number of bits
      • Graph algorithms: number of nodes & edges
      • Etc

28 of 44

Algorithm Analysis

  • We only analyze correct algorithms
    • An algorithm is correct
    • If, for every input instance, it halts with the correct output
  • Incorrect algorithms
    • Might not halt at all on some input instances
    • Might halt with other than the desired answer
  • Analyzing an algorithm
    • Predicting the resources that the algorithm requires
    • Resources include
      • Memory, Communication bandwidth, Computational time (usually most important)
  • Factors affecting the running time
    • computer , compiler, algorithm used
    • input to the algorithm
      • The content of the input affects the running time
      • typically, the input size (number of items in the input) is the main consideration
        • E.g. sorting problem ⇒ the number of items to be sorted
        • E.g. multiply two matrices together ⇒ the total number of elements in the two matrices
  • Machine model assumed
    • Instructions are executed one after another, with no concurrent operations ⇒ Not parallel computers

28

10/31/2022

29 of 44

Algorithm Analysis (Contd…)

  • Many criteria affect the running time of an algorithm, including
    • speed of CPU, bus and peripheral hardware
    • design think time, programming time and debugging time
    • language used and coding efficiency of the programmer
    • quality of input (good, bad or average)

29

10/31/2022

30 of 44

Algorithm Analysis (Contd…)

  • Programs derived from two algorithms for solving the same problem should both be
    • Machine independent
    • Language independent
    • Environment independent (load on the system,...)
    • Amenable to mathematical study
    • Realistic

30

10/31/2022

31 of 44

31

10/31/2022

Faster Algorithm vs. Faster CPU

  • A faster algorithm running on a slower machine will always win for large enough instances
  • Suppose algorithm S1 sorts n keys in 2n2 instructions
  • Suppose computer C1 executes 1 billion instruc/sec
    • When n = 1 million, takes 2000 sec
  • Suppose algorithm S2 sorts n keys in 50nlog2n instructions
  • Suppose computer C2 executes 10 million instruc/sec
    • When n = 1 million, takes 100 sec

31

32 of 44

32

10/31/2022

Algorithm Design Techniques

  • Brute Force & Exhaustive Search
    • follow definition / try all possibilities
  • Divide & Conquer
    • break problem into distinct subproblems
  • Transformation
    • convert problem to another one
  • Dynamic Programming
    • break problem into overlapping subproblems
  • Greedy
    • repeatedly do what is best now
  • Iterative Improvement
    • repeatedly improve current solution
  • Randomization
    • use random numbers
  • Backtracking
  • Branch and bound

32

33 of 44

Algorithm Classification

  • There are various ways to classify algorithms:
    • 1. Classification by implementation :
      • Recursion or iteration:
      • Logical:
      • Serial or parallel or distributed:
      • Deterministic or non-deterministic:
      • Exact or approximate:
    • 2.Classification by Design Paradigm :
      • Divide and conquer.
      • Dynamic programming.
      • The greedy method.
      • Linear programming.
      • Reduction.
      • Search and enumeration.
      • The probabilistic and heuristic paradigm.

33

34 of 44

Sorting (I)

  • Rearrange the items of a given list in ascending order.
    • Input: A sequence of n numbers <a1, a2, …, an>
    • Output: A reordering <a´1, a´2, …, a´n> of the input sequence such that a´1≤ a´2 a´n.
  • Why sorting?
    • Help searching
    • Algorithms often use sorting as a key subroutine.
  • Sorting key
    • A specially chosen piece of information used to guide sorting. E.g., sort student records by names.

34

35 of 44

Sorting (II)

  • Examples of sorting algorithms
    • Selection sort
    • Bubble sort
    • Insertion sort
    • Merge sort
    • Heap sort …
  • Evaluate sorting algorithm complexity: the number of key comparisons.
  • Two properties
    • Stability: A sorting algorithm is called stable if it preserves the relative order of any two equal elements in its input.
    • In place : A sorting algorithm is in place if it does not require extra memory, except, possibly for a few memory units.

35

36 of 44

Selection Sort

Algorithm SelectionSort(A[0..n-1])

//The algorithm sorts a given array by selection sort

//Input: An array A[0..n-1] of orderable elements

//Output: Array A[0..n-1] sorted in ascending order

for i 🡨 0 to n – 2 do

min 🡨 i

for j 🡨 i + 1 to n – 1 do

if A[j] < A[min]

min 🡨 j

swap A[i] and A[min]

36

37 of 44

Searching

  • Find a given value, called a search key, in a given set.
  • Examples of searching algorithms
    • Sequential search
    • Binary search …

37

Input: sorted array a_i < … < a_j and key x;

m 🡨(i+j)/2;

while i < j and x != a_m do

if x < a_m then j 🡨 m-1

else i 🡨 m+1;

if x = a_m then output a_m;

Time: O(log n)

38 of 44

Example of computational problem: sorting

  • Statement of problem
    • Input: A sequence of n numbers <a1, a2, …, an>
    • Output: A reordering of the input sequence <a´1, a´2, …, a´n> so that a´i a´j whenever i < j
  • Instance: The sequence <5, 3, 2, 8, 3>
  • Internal Sorting

Here, all data are held in primary memory during the sorting process.

  •  External Sorting

Here, it uses primary memory for the data currently being sorted and uses secondary storage for string data.

38

10/31/2022

39 of 44

Types of Sorting

  • Internal Sorting
    • Insertion (Insertion sort, Address Calculation sort, Shell sort)
    • Selection (Selection sort, Heap sort)
    • Exchange (Bubble sort, Quick sort, Radix sort)
  • External Sorting
    • Natural sort
    • Merge sort
    • Multi-way merge sort
    • Balanced sort
    • Polyphase sort
  •  

39

10/31/2022

40 of 44

Selection Sort

Suppose A is an array which consists of ‘n’ elements namely A[l], A[2], . . . , A[N]. The selection sort algorithm will works as follows.

  1. Step 1: a. First find the location LOC of the smallest element in the list A[l], A[2], . . . , A[N] and put it in the first position.

b. Interchange A[LOC] and A[1].

c. Now, A[1] is sorted.

2. Step 2: a. Find the location of the second smallest element in the list A[2], . . . , A[N] and put it in the second position.

b. Interchange A[LOC] and A[2].

c. Now, A[1] and A[2] is sorted. Hence, A[l] ≤ A[2].

3. Step 3: a. Find the location of the third smallest element in the list A[3], . . . , A[N] and put it in the third position.

b. Interchange A[LOC] and A[3].

c. Now, A[1], A[2] and A[3] is sorted. Hence, A[l] ≤ A[2] ≤ A[3].

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . .

N-1. Step N–1 : a. Find the location of the smallest element in the list A[A–1] and A[N].

b. Interchange A[LOC] and A[N–1] & put into the second last position.

c. Now, A[1], A[2], …..,A[N] is sorted. Hence, A[l] ≤ … ≤ A[N–1] ≤A[N].

40

10/31/2022

41 of 44

DAA - Unit - I Presentation Slides

41

10/31/2022

42 of 44

Some Well-known Computational Problems

  • Sorting
  • Searching
  • Shortest paths in a graph
  • Minimum spanning tree
  • Primality testing
  • Traveling salesman problem
  • Knapsack problem
  • Chess
  • Towers of Hanoi
  • Program termination

42

10/31/2022

Some of these problems don’t have efficient algorithms, or algorithms at all!

43 of 44

BÀI TẬP

  1. Nhập một mảng số nguyên ngẫu nhiên gồm n phần tử. Hiển thị mảng đó ra màn hình.
  2. Sắp xếp một mảng gồm n phần tử (trong bài 1) rồi hiển thị ra màn hình mảng đó trước khi và sau khi sắp xếp theo chiều tang/giảm.
  3. Cài đặt giải thuật Euclid tìm ước chung lớn nhât của 2 số (được nhập nhẫu nhiên).
  4. Sử dụng sàng Eratosthenes để hiện thị các số nguyên tố từ 2 tới n (n là số nguyên và n > 2).

43

10/31/2022

Some of these problems don’t have efficient algorithms, or algorithms at all!

44 of 44

DAA - Unit - I Presentation Slides

44

10/31/2022