1 of 56

LINEAR REGRESSION

Atul Nag

Associate Professor

2 of 56

LINEAR REGRESSION WITH ONE VARIABLE

3 of 56

Linear Regression with one variable

  • Supervised Learning
    • Data has right numbers
    • Regression model predicts numbers
      • Infinitely any number of outputs
    • Classification model predicts categories
      • Small number of possible outputs.

​

4 of 56

DATA TABLE

5 of 56

Terminology

  •  

6 of 56

7 of 56

Quiz

  •  
  •  

y is the true value for that training example, referred to as the output variable, or “target”.

8 of 56

COST FUNCTION

9 of 56

10 of 56

11 of 56

12 of 56

Quiz

w and b are parameters of the model, adjusted as the model learns from the data. They’re also referred to as “coefficients” or “weights”

13 of 56

14 of 56

15 of 56

16 of 56

17 of 56

18 of 56

19 of 56

20 of 56

21 of 56

Quiz

  •  

When the cost is relatively small, closer to zero, it means the model fits the data better compared to other choices for w and b.

22 of 56

VISUALIZATION OF COST FUNCTION

23 of 56

24 of 56

25 of 56

26 of 56

27 of 56

28 of 56

Quiz

  •  
  •  

 

29 of 56

Quiz

  •  
  • The selected values of the parameters w and b cause the algorithm to fit the training set really well.
  • The selected values of the parameters w and b cause the algorithm to fit the training set really poorly.
  • This is never possible -- there must be a bug in the code.

​

​

​

​

​

​

When the cost is small, this means that the model fits the training set well.

30 of 56

GRADIENT DESCENT

31 of 56

32 of 56

33 of 56

34 of 56

35 of 56

36 of 56

37 of 56

Quiz

  •  
  •  

 

38 of 56

39 of 56

40 of 56

Quiz

  •  
  • w stays the same
  • w decreases
  • It is not possible to tell if w will increase or decrease.
  • w increases

 

41 of 56

LEARNING RATE

42 of 56

43 of 56

44 of 56

45 of 56

GRADIENT DESCENT WITH LINEAR REGRESSION

46 of 56

47 of 56

48 of 56

49 of 56

50 of 56

51 of 56

52 of 56

53 of 56

54 of 56

55 of 56

Quiz

  •  
  •  

The learning rate is always a positive number, so if you take W minus a negative number, you end up with a new value for W that is larger (more positive).

56 of 56

Quiz

  • For linear regression, what is the update step for parameter b?