1 of 33

ME 5990�Machine Learning for ME

�

Linear Regression

2 of 33

Outline

  • 1-Dimension least-square fitting
    • Linear fitting
    • Polynomial fitting
  • Multivariate least-square fitting

3 of 33

Regression is supervised learning

  • Label and feature vector

4 of 33

Regression

  • The label is not necessarily binary
    • In a course, textbook is optional; students not always buy the textbook
    • Historical data show that the number of students of the course and textbooks purchased as follows:
    • How many textbook we should pre-order for next�semester with 35 students in the course?

​

5 of 33

Maths of fitting

  •  

6 of 33

Linear regression

  •  

7 of 33

For example…

  • For demo purpose, we reduce the data previously

​

Sstudent(x)

Book(y)

28

29

35

34

39

35

30

29

 

8 of 33

Training (fitting)

  •  

9 of 33

Training (fitting)

  •  

10 of 33

Training (fitting)

  •  

11 of 33

Training (fitting)

  •  

12 of 33

It is first order equations!

  •  

13 of 33

It is first order equations!

  •  

14 of 33

Outline

  • 1 dimension least-square fitting
    • Linear fitting
    • Polynomial fitting
  • Multivariate least-square fitting

15 of 33

Poly fitting

  •  

16 of 33

Poly fitting

  •  

17 of 33

Poly fitting

  •  

18 of 33

Outline

  • 1 dimension least-square fitting
    • Linear fitting
    • Poly fitting
  • Multivariate least-square fitting

Slide based on Katherine Keith, University of Massachusetts, Amherst

(https://kakeith.github.io/classes/cs335/spring2020_ml.html)

​

​

19 of 33

Multivariate Linear Fitting

  •  

20 of 33

Linear Model

  •  

21 of 33

Linear Model

  •  

22 of 33

Linear model

  •  

23 of 33

Linear Model

  •  

24 of 33

Linear Model

  •  

25 of 33

 

  • When x is one-dimension, we fit a line.
  • When x is two-dimension, we fit a plane.

26 of 33

Residual

  •  

27 of 33

Residual

  •  

28 of 33

Cost function

  •  

29 of 33

Training (fitting) thoughts

  •  

30 of 33

Solution (1)

  •  

31 of 33

Solution (1)

  •  

From : https://en.wikipedia.org/wiki/Computational_complexity_of_mathematical_operations

32 of 33

Solution (2)

  • Gradient Descent …

​

Not quite relevant video:

https://youtu.be/4FAuNcWSxuY

​

33 of 33

Summary

  •