Applied Statistics for Business (MAS202)

Lecturer: Thi Minh Phuong Vu

phuongvtm11@fe.edu.vn

Website: https://sites.google.com/site/thiminhphuongvu/home?authuser=0

​

CHAPTER XIII: Simple linear regression

1. Simple linear regression model

Example: Which data pairs are dependent/ independent ?

1. Air temperature- Altitude
2. Age of people- Age of dogs
3. Height of students - MAS202 final scores
4. Look at the scatter plot

(biểu đồ phân tán)

​

​

CHAPTER XIII: Simple linear regression

• Simple linear regression model

Relationship between two variables

​

​

—> Linear relation is the simplest,

efficient model in explaining

the dependence of Y on X

​

Tương quan

tuyến tính

Tương quan

phi tuyến tính

Không xác định

CHAPTER XIII: Simple linear regression

• Simple linear regression model

AIM:

We use simple linear regression analysis to:

• Establish a linear relationship between 2 random variables (data pair)
• Predict the values of a dependent variable based on a given independent variable

CHAPTER XIII: Simple linear regression

• Simple linear regression model

SAMPLE LINEAR REGRESSION LINE (PREDICTION LINE)

X: independent variable Y: dependent variable

The linear regression line

where

​

• b₁: the regression slope
• b₀: the regression intercept
• : the estimated (predicted) Y value for observation Xi

​

​

CHAPTER XIII: Simple linear regression

• Simple linear regression model

How to compute b₀, b₁?

Assume: (X₁, Y₁), (X₂, Y₂),...., (Xn, Yn) be a sample pairs of data , sample means

the sample linear regression line is

where:

CHAPTER XIII: Simple linear regression

• Simple linear regression model

​

Example 1: Does studying hard implies high grades?

The following table show the grades of 5 students and their number of study hours per week

1. What is dependent variable, independent

variable?

• Compute the estimated linear regression line

for the sample

• predict the grade of a student spending

20 hours/week studying

​

​

CHAPTER XIII: Simple linear regression

• Simple linear regression model

Exercise:

CHAPTER XIII: Simple linear regression

• Simple linear regression model

NOTE:

• The intercept b₀ represents the estimated value of Y when X=0
• The slope b₁ represents the expected change in Y per unit change in X.
• The predicted values has the same mean as Y:

​

​

Example: Re-consider Example 1:

​

CHAPTER XIII: Simple linear regression

• Simple linear regression model

EXCEL package:

b₀

b₁

CHAPTER XIII: Simple linear regression

• Simple linear regression model

​

​

QUESTION: Are the predicted values correct ? How to estimate the errors ?

—--> Consider the MEASURES OF VARIATION !

CHAPTER XIII: Simple linear regression

2. Measures of Variation SUMS OF SQUARES

Assume: (X₁, Y₁), (X₂, Y₂),...., (Xn, Yn) be a sample pairs of data , sample means

Error sum of squares (unexplained variation)

​

​

Regression sum of squares (explained variation):

​

​

Total sum of squares (total variation):

Measures the variation of the Yi values around their mean Y.

Variation attributed to the relationship between X and Y.

Variation in Y attributed to factors other than X.

CHAPTER XIII: Simple linear regression

2. Measures of Variation SUMS OF SQUARES

​

​

​

​

​

​

​

​

CHAPTER XIII: Simple linear regression

2. Measures of Variation SUMS OF SQUARES

EXCEL package: SST= 17,2; SSR= 2,28; SSE= 14,9

​

​

​

​

​

​

​

Question: Find the percentage of total variation that can

be explained by the variable X? Answer: SSR/SST *100%= 13,31%

CHAPTER XIII: Simple linear regression

2. Measures of Variation COEFFICIENT OF DETERMINATION & CORRELATION

• The regression line gives good predictions when SSE is small !
• To test whether the regression model is weak or strong, use:

​

​

NOTE:

- 0 ≤ r² ≤ 1

- r² measures the proportion of the total variation of Y which can be

explained by the factor X

Coefficient of determination:

​

CHAPTER XIII: Simple linear regression

2. Measures of Variation COEFFICIENT OF DETERMINATION & CORRELATION

• If r² is close to 1 —---> strong linear relationships between X and Y
• If r² is close to 0 —---> weak linear relationships between X and Y

​

​

CHAPTER XIII: Simple linear regression

2. Measures of Variation COEFFICIENT OF DETERMINATION & CORRELATION

​

Example: Re-consider Example 1, find the coefficient of determination. Explain the result.

​

​

​

CHAPTER XIII: Simple linear regression

2. Measures of Variation COEFFICIENT OF DETERMINATION & CORRELATION

EXCEL package:

CHAPTER XIII: Simple linear regression

2. Measures of Variation COEFFICIENT OF DETERMINATION & CORRELATION

EXCEL package:

CHAPTER XIII: Simple linear regression

3. The residual analysis

​

​

​

Assume that (X₁, Y₁), (X₂, Y₂),...., (Xn, Yn) is a sample of n- observations in pairs.

—--> The residuals are e₁, e₂, …, en

NOTE:

​

CHAPTER XIII: Simple linear regression

3. The residual analysis

EXCEL package:

​

​

​

​

CHAPTER XIII: Simple linear regression

3. The residual analysis

4 ASSUMPTIONS ON THE RESIDUALS: L.I.N.E

1. Linearity: Only consider the linear relationship between variables. Relationships that are not linear are not discussed here!
2. Independence of Errors: The random residuals e, e, …, e, must be independent of each other.
3. Normality of Error: Each random residual e_i must be normally distributed, for all i
4. Equal Variance: The random residuals e must have equal constant variance

​

CHAPTER XIII: Simple linear regression

3. The residual analysis

NOTE: EXCEL can help to create a residual plot/normal probability plot for the residuals of a data set

• When the residual plot has

horizontal straight line shape

—-> reasonable to use the linear model!

​

• When using a normal probability plot,

normal errors will approximately display in

a straight line.

​

CHAPTER XIII: Simple linear regression

4 . Hypothesis Testing for the slope & correlation

​

The linear regression line for the population is unknown:

​

​

The random error is

​

​

CHAPTER XIII: Simple linear regression

4 . Hypothesis Testing for the slope & correlation

​

Note: The random error 𝜀 has the estimated (sample) standard deviation

​

​

​

​

called standard error of the estimate

CHAPTER XIII: Simple linear regression

4 . Hypothesis Testing for the slope & correlation

EXCEL package:

​

​

​

​

​

CHAPTER XIII: Simple linear regression

4 . Hypothesis Testing for the slope (hệ số góc) & correlation (hệ số tương quan)

​

​

2 different ways to test for the slope 𝛽₁

​

T- test with df= n-2

F- test (only for zero slope)

CHAPTER XIII: Simple linear regression

4 . Hypothesis Testing for the slope & correlation

First way: t- test, df= n-2

The estimated standard

deviation of 𝛽₁ is:

​

​

where

SYX: standard error of the estimate

​

​

CHAPTER XIII: Simple linear regression

4 . Hypothesis Testing for the slope & correlation

First way: t- test, df= n-2

​

CHAPTER XIII: Simple linear regression

4 . Hypothesis Testing for the slope & correlation

Example: Test whether 𝛽₁= 0.2 at 5% significance level

​

CHAPTER XIII: Simple linear regression

4 . Hypothesis Testing for the slope & correlation

Second way: F- test for zero slope

​

​

Reject H₀ when FSTAT > F𝛼

If Reject H₀—-> significant linear relationship

If not reject H₀—> the linear relationship

is not significant

CHAPTER XIII: Simple linear regression

4 . Hypothesis Testing for the slope & correlation

EXCEL package

​

​

CHAPTER XIII: Simple linear regression

4 . Hypothesis Testing for the slope & correlation

Example: Use F-test to test for significant linear relationship. Use 𝛼=0.05

​

​

CHAPTER XIII: Simple linear regression

4 . Hypothesis Testing for the slope & correlation

T- test for zero correlation Coefficient (or significant regression model)

​

𝜌: population correlation coefficient

r: estimated (sample) correlation for 𝜌

CHAPTER XIII: Simple linear regression

4 . Hypothesis Testing for the slope & correlation

T- test for zero correlation Coefficient (or significant regression model)

​

Example 1: You want to explore the relationship between the grades students receive on their first two exams. For a sample of 25 students, you find a correlation of 0.45.

What is your conclusion in testing 𝐻₀:𝜌=0 versus 𝐻₁:𝜌≠0 at significant level α=0.05.

​

​

CHAPTER XIII: Simple linear regression

QUIZ

1.

CHAPTER XIII: Simple linear regression

2.

​

CHAPTER XIII: Simple linear regression

3.

​

CHAPTER XIII: Simple linear regression

4.

​

CHAPTER XIII: Simple linear regression

5.

​

CHAPTER XIII: Simple linear regression

6.

​

CHAPTER XIII: Simple linear regression

CHAPTER XIII: Simple linear regression

7.

​

CHAPTER XIII: Simple linear regression

8.

​

CHAPTER XIII: Simple linear regression

​

​

CHAPTER XIII: Simple linear regression

9.

​

CHAPTER XIII: Simple linear regression

10.

​

CHAPTER XIII: Simple linear regression

​

​

CHAPTER XIII: Simple linear regression

11.

​

​

CHAPTER XIII: Simple linear regression

12.

​

​

CHAPTER XIII: Simple linear regression

13.

​

​

CHAPTER XIII: Simple linear regression

14.

​

​

​