1 of 47

(2024-25 EVEN)

UTA027

Artificial Intelligence

Machine Learning

(Introduction)

Thapar Institute of Engineering and Technology

(Deemed to be University)

2 of 47

Machine Learning

Introduction

Raghav B. Venkataramaiyer

Thapar Institute of Engineering and Technology

(Deemed to be University)

3 of 47

Ref

Artificial Intelligence: Structures and Strategies for Complex Problem Solving

By: Luger & Stubblefield

[Download URL]

PRML

[Download URL]

Notations

Linear Algebra
Set Notations

World of discourse
Problem Formulation�e.g. MNIST Classification

4 of 47

Notations

Concepts

Set Notation:�{a,b,c,…} (e.g. set of vertices)�{a,b} ≡ {b,a}�{a∈ℕ : a even.}

Vectors:�Row Vectors: (w₁,…,w_M) OR [w₁,…,w_M]�Column Vectors: w = [w₁,…,w_M]^T�Closed/Open Intervals: [a,b],(a,b),[a,b)

Matrices: M (uppercase bold letters)�M×M Identity (Unit) Matrix: I_M� I_M ⊢ I_ij = 1 if i=j; I_ij = 0 if i≠j

Probability:�Expectation: 𝔼[X], Variance: Var(X)�Conditionals: 𝔼_x[f(x)|z], Var_x(f(x)|z)

Set Partition:

Given set S≡{a,b,c,…}

Partitions of S:�S₁,S₂,S₃,…⊆S, ⋃_iS_i=S ⊢�∀i,j i≠j → S_i∩S_j=∅�(pairwise disjoint subsets that span the space)

PS:

⋃_iS_i=S �enforces the spanning property;
∀i,j i≠j → S_i∩S_j=∅�defines the pairwise disjoint condition.

5 of 47

World of discourse

6 of 47

Pattern Recognition

What is this?

7 of 47

Pattern Recognition

What is this?

8 of 47

Supervision

Settings (or Context)

Annotated vs Non-(or weakly)-annotated Setting
Supervised vs Unsupervised (or self-supervised) Setting
Weakly Supervised (or Zero-shot/ One-shot/ Few-shot) Setting

Why are annotations important?
Why bother about non-or-weakly annotated setup?
Are there examples/ use-cases from your domain?

9 of 47

Curve Fitting

TreadWill.Org

10 of 47

Curve Fitting

11 of 47

Partition

12 of 47

Formulation

Example: MNIST Classification

13 of 47

Formulation�MNIST Classification

Inputs

m rows

n columns

14 of 47

Formulation�MNIST Classification

Inputs

m rows

n columns

15 of 47

Formulation�MNIST Classification

Inputs

m rows

n columns

16 of 47

Formulation�MNIST Classification

Inputs

m rows

n columns

17 of 47

Formulation�MNIST Classification

Inputs

Bound and Continuous:

H × W

18 of 47

Formulation�MNIST Classification

Outputs

0

1

2

3

4

5

6

7

8

9

Input

X∈ℝ^m×n

y=6

19 of 47

Formulation�MNIST Classification

Outputs

0

1

2

3

4

5

6

7

8

9

Input

X∈ℝ^m×n

y=6

0

1

2

3

4

5

6

7

8

9

y=5

X∈ℝ^m×n

20 of 47

Formulation�MNIST Classification

Outputs

0

1

0

1

2

3

4

5

6

7

8

9

y=6

Input

X∈ℝ^m×n

0

1

2

3

4

5

6

7

8

9

y=5

X∈ℝ^m×n

21 of 47

Formulation�MNIST Classification

Outputs

0

1

0

1

2

3

4

5

6

7

8

9

y=6

Input

X∈ℝ^m×n

0

1

0

1

2

3

4

5

6

7

8

9

y=5

X∈ℝ^m×n

22 of 47

Formulation�MNIST Classification

Outputs

0

1

0

1

2

3

4

5

6

7

8

9

y=6

0

1

0

1

2

3

4

5

6

7

8

9

y=5

Input

X∈ℝ^m×n

23 of 47

Formulation�MNIST Classification

Outputs

0

1

0

1

2

3

4

5

6

7

8

9

y=6

0

1

0

1

2

3

4

5

6

7

8

9

y=5

Input

X∈ℝ^m×n

24 of 47

Formulation�MNIST Classification

Outputs

=0 everywhere except x=0

The value at zero satisfies:

Area under the curve is 1.

x discrete

25 of 47

Formulation�MNIST Classification

Dirac-delta Indicator Equivalence

26 of 47

Formulation�MNIST Classification

Model (x∈ℝ^m×n, y∈ℤ)

27 of 47

Formulation�MNIST Classification

Model (x∈ℝ^m×n, y∈ℤ)

28 of 47

Formulation�MNIST Classification

Model (x∈ℝ^m×n, y∈ℤ)

29 of 47

Formulation�MNIST Classification

Model (x∈ℝ^m×n, y∈ℤ)

30 of 47

Formulation�MNIST Classification

Model

x∈ℝ^m×n

y∈ℤ

Data

(aka. evidence)

31 of 47

Formulation�MNIST Classification

Model

x∈ℝ^m×n

y∈ℤ

Data

Sample x (from data)

32 of 47

Formulation�MNIST Classification

Model

x∈ℝ^m×n

y∈ℤ

Data

Sample x and y (from data)

33 of 47

Formulation�MNIST Classification

Model

x∈ℝ^m×n

y∈ℤ

Data

Conditional distribution of y given x.

34 of 47

Formulation�MNIST Classification

Model

x∈ℝ^m×n

y∈ℤ

Data

Using the model, estimate y given x.

35 of 47

Formulation�MNIST Classification

Model

x∈ℝ^m×n

y∈ℤ

Data

Optimal case.

36 of 47

Formulation�MNIST Classification

Model

0

1

0

1

2

3

4

5

6

7

8

9

y=6

x∈ℝ^m×n

37 of 47

Formulation�MNIST Classification

Model

0

1

0

x∈ℝ^m×n

0

1

2

3

4

5

6

7

8

9

as per evidence

38 of 47

Formulation�MNIST Classification

Model

0

1

0

x∈ℝ^m×n

0

1

2

3

4

5

6

7

8

9

as per evidence

as per estimate

39 of 47

Formulation�MNIST Classification

Model

0

1

0

x∈ℝ^m×n

0

1

2

3

4

5

6

7

8

9

as per evidence

Optimally.

40 of 47

Formulation�MNIST Classification

Model

0

1

0

x∈ℝ^m×n

0

1

2

3

4

5

6

7

8

9

as per evidence

41 of 47

Formulation�MNIST Classification

Objective

x∈ℝ^m×n

If the two values are approximately the same,

Then the difference between them may be expected to be miniscule

42 of 47

Formulation�MNIST Classification

Objective

x∈ℝ^m×n

If the two values are approximately the same,

Then the DISTANCE between them may be EXPECTED to be MINIMUM

43 of 47

Formulation�MNIST Classification

Objective

x∈ℝ^m×n

If the two values are approximately the same,

Then the DISTANCE between them may be EXPECTED to be MINIMUM

44 of 47

Formulation�MNIST Classification

Objective

x∈ℝ^m×n

If the two values are approximately the same,

Then the EXPECTED VALUE of the DISTANCE between them shall be MINIMUM

45 of 47

Formulation�MNIST Classification

Objective

x∈ℝ^m×n

If the two values are approximately the same,

Then MINIMISE the EXPECTED VALUE of the DISTANCE between them.

46 of 47

Formulation�MNIST Classification

Objective

x∈ℝ^m×n

If the two values are approximately the same,

Then the optimal params shall MINIMISE the EXPECTED VALUE of the DISTANCE between them.

47 of 47

Thank you!