1 of 64

Linear Algebra 1

Some materials from linear algebra review by Prof. Zico Kolter from CMU

2 of 64

Linear Equations

Set of linear equations (two equations, two unknowns)

2

3 of 64

Linear Equations

3

4 of 64

Linear Equations

4

5 of 64

Linear Equations

5

6 of 64

Linear Equations in Python

6

7 of 64

System of Linear Equations

7

8 of 64

Elements of a Matrix

8

9 of 64

Vector-Vector Products

9

10 of 64

Matrix-Vector Products

10

11 of 64

Matrix-Vector Products

11

12 of 64

Symmetric Matrices

12

13 of 64

Norms (Strength or Distance in Linear Space)

13

14 of 64

Norms in Python

14

15 of 64

Orthogonality

15

and

16 of 64

Angle between Vectors

16

17 of 64

Half Space

17

18 of 64

Linear Algebra 2

19 of 64

Vector

Vector

19

20 of 64

Matrix and (Linear) Transformation

20

21 of 64

Rotation

21

22 of 64

Rotation

22

23 of 64

Rotation

23

24 of 64

Rotation

24

25 of 64

Stretch/Compress

Stretch/Compress

keep the direction

Still represented by a matrix

25

26 of 64

Stretch/Compress: Example

26

27 of 64

Projection

27

28 of 64

Multiple Transformations

28

29 of 64

29

30 of 64

Another way to find this projection matrix�

30

31 of 64

Linear Transformation

See if the given transformation is linear

A linear system makes our life much easier

Superposition
Homogeneity

31

32 of 64

Linear Transformation

Superposition

32

33 of 64

Linear Transformation

Homogeneity

33

34 of 64

Linear Transformation

Linear vs. Non-linear

34

35 of 64

Linear Transformation

35

36 of 64

Eigenvalue and Eigenvector

36

37 of 64

Linear Transformation

37

38 of 64

How to Compute Eigenvalue and Eigenvector

38

39 of 64

39

40 of 64

Find eigenvalues and eigenvectors.

40

41 of 64

Eigen Analysis in Python

41

42 of 64

Example: Eigen Analysis of Projection

42

43 of 64

What kind of a linear transformation?

43

44 of 64

Example: Eigen Analysis of Mirror

44

45 of 64

Example: Eigen Analysis of Mirror

45

46 of 64

What kind of a linear transformation?

46

47 of 64

What kind of a linear transformation?

Multiple transformations

47

48 of 64

Example: Eigen Analysis of Rotation

What kind of a linear transformation?

Eigenvalues: complex numbers

What is the physical meaning?

48

49 of 64

Linear Algebra 3

50 of 64

System of Linear Equations

Well-determined linear systems
Under-determined linear systems
Over-determined linear systems

50

51 of 64

Well-Determined Linear Systems

System of linear equations

Geometric point of view

51

52 of 64

Well-Determined Linear Systems

System of linear equations

Matrix form

52

53 of 64

Under-Determined Linear Systems

System of linear equations

Geometric point of view

53

54 of 64

Under-Determined Linear Systems

System of linear equations

Matrix form

54

55 of 64

Over-Determined Linear Systems

System of linear equations

Geometric point of view

55

56 of 64

Over-Determined Linear Systems

System of linear equations

Matrix form

56

57 of 64

Summary of Linear Systems

Square: Well-determined

Fat: Under-determined

Skinny: Over-determined

57

58 of 64

Least-Norm Solution

58

59 of 64

Least-Norm Solution

Optimization problem

Geometric interpretation

Select one solution among many solutions
Often control problem

59

60 of 64

Least-Square Solution

60

61 of 64

Least-Square Solution

Optimization problem

Geometric interpretation

Often estimation problem

61

62 of 64

Vector Projection onto Y

62

63 of 64

Vector Projection onto Y

63

64 of 64

Orthogonal Projection onto a Subspace

64