1 of 67

Linear Algebra 1

Some materials from linear algebra review by Prof. Zico Kolter from CMU

2 of 67

Linear Equations

Set of linear equations (two equations, two unknowns)

2

3 of 67

Linear Equations

3

4 of 67

Linear Equations

4

5 of 67

Linear Equations

5

6 of 67

Linear Equations in Python

6

7 of 67

System of Linear Equations

7

8 of 67

Elements of a Matrix

8

9 of 67

Vector-Vector Products

9

10 of 67

Matrix-Vector Products

10

11 of 67

Matrix-Vector Products

11

12 of 67

Norms (Strength or Distance in Linear Space)

12

13 of 67

Orthogonality

13

and

14 of 67

Angle between Vectors

14

15 of 67

Half Space

15

16 of 67

Linear Algebra 2

17 of 67

Matrix and (Linear) Transformation

A matrix is not just an array of numbers but a powerful tool for encoding and understanding linear transformations

Matrix represents a linear transformation.

In the equation

17

18 of 67

Linear Transformation

See if the given transformation is linear

A linear system makes our life much easier

Superposition
Homogeneity

18

19 of 67

Linear Transformation: Superposition

Superposition

19

20 of 67

Linear Transformation: Homogeneity

Homogeneity

20

21 of 67

Linear Transformation

Linear vs. Non-linear

21

22 of 67

Rotation

22

23 of 67

Rotation

23

24 of 67

Rotation

24

25 of 67

Rotation

25

26 of 67

Stretch/Compress

Stretch/Compress

keep the direction

Still represented by a matrix

26

27 of 67

Stretch/Compress: Example

27

28 of 67

Projection

28

29 of 67

Multiple Transformations

29

30 of 67

30

31 of 67

Another way to find this projection matrix�

31

32 of 67

Linear Transformation and Basis Representation

32

33 of 67

Eigenvalue and Eigenvector

33

34 of 67

Basis and Eigenvector Representation in Linear Transformation

34

35 of 67

How to Compute Eigenvalue and Eigenvector

35

36 of 67

36

37 of 67

Find eigenvalues and eigenvectors

37

38 of 67

Example: Eigen Analysis of Projection

38

39 of 67

What kind of a linear transformation?

39

mirror

40 of 67

Example: Eigen Analysis of Mirror

Eigenvalues and eigenvectors?

40

41 of 67

Example: Eigen Analysis of Mirror

41

42 of 67

What kind of a linear transformation?

42

43 of 67

Example: Eigen Analysis of Rotation

What kind of a linear transformation?

Eigenvalues: complex numbers

What is the physical meaning?

43

44 of 67

Rotation Matrix in 2D

Rotation matrix in 2D

Compute eigen-analysis

Since these eigenvectors are not real, it indicates that a pure rotation in 2D has no real eigenvectors, meaning no real direction remains invariant

44

45 of 67

Rotation Matrix in 3D

45

46 of 67

Linear Algebra 3

47 of 67

System of Linear Equations

Well-determined linear systems
Under-determined linear systems
Over-determined linear systems

47

48 of 67

Well-Determined Linear Systems

System of linear equations

Geometric point of view

48

49 of 67

Well-Determined Linear Systems

System of linear equations

Matrix form

49

50 of 67

Under-Determined Linear Systems

System of linear equations

Geometric point of view

50

51 of 67

Under-Determined Linear Systems

51

52 of 67

Over-Determined Linear Systems

System of linear equations

Geometric point of view

52

53 of 67

Over-Determined Linear Systems

53

54 of 67

Summary of Linear Systems

Square: Well-determined

Fat: Under-determined

Skinny: Over-determined

54

55 of 67

Least-Norm Solution

55

56 of 67

Least-Norm Solution

Optimization problem

Geometric interpretation

Often control problem where one often seeks a control input with minimum energy or least actuator effort that still achieves the desired output

56

57 of 67

Least-Squares Solution

57

58 of 67

58

59 of 67

Matrix Interpretation

This projection can also be written in the form:

59

60 of 67

Alternative Derivation via Orthogonality Condition

60

61 of 67

Back to Least-Squares Solution

61

62 of 67

Back to Least-Squares Solution

62

63 of 67

Minimizing Error

63

64 of 67

Minimizing Error

64

65 of 67

Minimizing Error

65

66 of 67

Orthogonal Projection onto a Subspace

66

67 of 67

Orthogonal Projection onto a Subspace

67