MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

(A Unit of Rajalaxmi Education Trust ®, Mangalore)

Autonomous Institute affiliated to VTU, Belgavi, Approved by AICTE, New Delhi

Accredited by NAAC with A+ Grade and ISO 9001:2015 Certified Institution

CONTENT

• Correlation: Introduction
• Types of Correlation
• Correlation and Causation
• Scatter Diagram Method
• Karl Pearson’s Coefficient of Correlation
• Regression: Introduction
• Linear and Non-Linear Regression
• Lines of Regression
• Coefficient of Regression
• Mean values from the two lines of Regression
• Regression coefficients and the Correlation Coefficient from the lines of Regression
• Correlation Analysis Vs. Regression Analysis.

​

​

2

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

MODULE -1:

CORRELATION & REGRESSION

3

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

LEARNING OBJECTIVES:

Provide strong foundation in Correlation and Regression techniques to solve engineering problems.

​

​

​

RBT LEVEL: L1, L2, L3

4

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

Response and Explanatory Variables:

Response variable (Dependent Variable)

• The outcome variable on which comparisons are made.

​

Example : College GPA

​

Explanatory variable (Independent variable)

• When the explanatory variable is categorical, it defines the groups to be compared with respect to values on the response variable.
• When the explanatory variable is quantitative, it defines the change in different numerical values to be compared with respect to the values for the response variable.

​

Example : Number of hours a week spent studying.

​

​

​

Correlation and Regression-scatter plot :

MODULE - 1:

Statistics

5

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

​

• The main purpose of data analysis with two variables is to investigate whether there is an association and to describe that association.

​

​

• An association exists between two variables if a particular value for one variable is more likely to occur with certain values of the other variable.

​

​

6

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

Correlation:

Correlation is a statistical technique to ascertain the association or relationship between two or more variables.

variables may be:

​

​

​

​

​

If two quantities vary in such a way that movements in one are accompanied by movements in the other, then these quantities are said to be correlated.

​

​

​

​

​

​

​

​

7

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

​

A positive correlation indicates the extent to which those variables increase or decrease in parallel

​

​

​

A negative correlation indicates the extent to which one variable increases as the other decreases.

​

​

8

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

​

Example:

​

​

​

​

AMOUNT OF TELEVISION CHILDREN WATCH

​

​

​

THE LIKELIHOOD THAT THEY WILL BECOME BULLIES

AND

The studies report a correlation, a lack of parental supervision – may be the influential factor (an unknown factor that influences both variables) .

9

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

Statisticians have developed measures for describing the correlation between two variables

​

​

​

​

​

​

​

​

​

COEFFICIENT OF CORRELATION

The sample correlation coefficient (r) measures the degree of linearity in the relationship between “X” and “Y”

-1 ≤ r ≤ + 1

-1 is a strong negative relationship and +1 has a strong positive relationship

r = 0 indicates no linear relationship

10

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

Types of Correlation

Types of Correlation

Positive and Negative Correlation

Simple, multiple and Partial Correlation

Linear and Non-Linear Correlation

11

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

​

Positive, Negative and Zero Correlation:

​

Positive Correlation: If both the variables vary in the same direction, correlation is said to be positive. It means if one variable is increasing, the other on an average is also increasing or if one variable is decreasing, the other on an average is also deceasing, then the correlation is said to be positive correlation.

​

Example: The correlation between heights and weights of a group of persons is a positive correlation.

​

​

​

​

​

​

​

​

​

​

​

12

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

Negative Correlation: If both the variables vary in opposite direction, the correlation is said to be negative. If means if one variable increases, but the other variable decreases or if one variable decreases, but the other variable increases, then the correlation is said to be negative correlation.

​

Example: the correlation between the price of a product and its demand is a negative correlation.

​

​

​

​

​

​

​

13

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

Zero Correlation: Actually, it is not a type of correlation but still it is called as zero or no correlation. When we don’t find any relationship between the variables then, it is said to be zero correlation. It means a change in value of one variable doesn’t influence or change the value of another variable.

Example: the correlation between weight of person and intelligence is a zero or no correlation.

​

Multiple Correlation: When three or more variables are studied, it is a case of multiple correlation. For example, in above example if study covers the relationship between student marks, attendance of students, effectiveness of teacher, use of teaching aids etc, it is a case of multiple correlation.

​

Partial Correlation: In case of partial correlation, one studies three or more variables but considers only two variables to be influencing each other and the effect of other influencing variables being held constant.

Example : In above example of relationship between student marks and attendance, the other variable influencing such as effective teaching of teacher, use of teaching aid like computer, smart board etc are assumed to be constant.

​

​

​

​

​

​

​

​

14

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

Methods of measurement of correlation:

Quantification of the relationship between variables is very essential to take the benefit of study of correlation. For this, we find there are various methods of measurement of correlation, which can be represented as given below:

Methods of Measurement of Correlation

​

​

Graphic Method Algebraic Method

​

​

Scatter Diagram Karl Pearson’s Coefficient of Correlation

​

​

​

15

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

Scatter Diagram: Graphical display of relationship between two quantitative variables:

​

Horizontal Axis: Explanatory variable, x

Vertical Axis: Response variable, y

​

​

​

MINITAB Scatterplot for Internet use and Facebook use for 33 Countries.The point for Japan is labeled and has coordinates x = 74 and y = 2 .

​

Question: Is there any point that you would identify as standing out in some way? Which country does it represent, and how is it unusual in the context of these variables?

16

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

Solution:

​

Yes, the point representing Japan (74, 2) stands out significantly in the scatterplot. Here's why:

​

• The x-coordinate (74) represents the percentage of internet users in Japan, which is relatively high.

​

• The y-coordinate (2) represents Facebook use, which is extremely low compared to other countries with similar internet penetration rates.

​

By inspecting the scatterplot, we observe - Nations with high internet usage tend to have higher Facebook usage. For the countries with relatively low usage of Internet (below 20%) there is little variability in FB use.

Why is Japan Unusual?

However, Japan deviates from this trend by having very low Facebook engagement despite widespread internet access. This suggests that other social media platforms (such as LINE or Twitter) are more popular in Japan, making it an outlier in the dataset.

​

​

​

​

17

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

How to Examine a Scatterplot :

We examine a scatterplot to study association. How do values on the response variable change as values of the explanatory variable change?

​

You can describe the overall pattern of a scatterplot by the trend, direction, and strength of the relationship between the two variables.

​

• Trend: Linear, Curved, Clusters, No pattern.

​

• Direction: Positive, Negative, No direction.

​

• Strength: How closely the points fit the trend.

​

Also look for outliers from the overall trend.

​

​

18

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

Interpreting Scatterplots: Two quantitative variables x and y are

​

Positively associated : when high values of x tend to occur with high values of y.

low values of x tend to occur with low values of y.

​

​

Negatively associated : when high values of one variable tend to pair with low values of the other variable.

​

​

​

​

​

​

​

19

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

A scatter diagram reveals whether the movements in one series are associated with those in the other series.

​

​

​

​

​

​

​

​

​

​

​

20

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

Example: Given the following pairs of values

​

​

​

​

​

(a)Draw a scatter diagram

​

(b) Do you think that there is any correlation between profits and capital employed?

​

Is it positive or negative? Is it high or low?

​

21

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

​

Solution:

From the observation of scatter diagram we can say that the variables are positively correlated. In the diagram the points trend toward upward rising from the lower right-hand corner to the upper right-hand corner, hence it is positive correlation.

​

Plotted points are in narrow band which indicates that it is a case of high degree of positive correlation.

22

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

Example: 100 cars on the lot of a used-car dealership. Would you expect a positive association, a negative association or no association between the age of the car and the reading on the odometer?

Solution :

A positive association between the age of the car and the reading on the odometer.

​

This means that as the age of a car increases, the odometer reading (mileage) generally increases as well. Older cars have typically been driven for more years, accumulating more miles.

While there are exceptions (such as classic cars with low mileage),

the overall trend in a used car lot would likely show that older cars tend to have

higher mileage.

23

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

Practice problem

​

State in each case whether there is

(i)Positive Correlation

(ii)Negative Correlation

(iii)No Correlation

24

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

​

​

​

​

​

​

​

​

Karl Pearson’s Coefficient of Correlation

​

Karl Pearson’s method of calculating coefficient of correlation is based on the covariance of the two variables in a series. This method is widely used in practice and the coefficient of correlation is denoted by the symbol “r”. Pearson's correlation coefficient between two variables is defined as the ”covariance of the two variables divided by the product of their standard deviations”

​

​

Direct Method:This method used when given variables are small in magnitude

25

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

Interpretation of Correlation Coefficient (r) : 

​

• The Correlation measures the strength and direction of the linear association between x and y.
• The Sign of r denotes the nature of the association while the value of r denotes the strength of association

26

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

27

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

Calculation of Karl Pearson’s coefficient of correlation

28

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

Interpretation: From the above calculation it is very clear that there is high degree of positive correlation i.e. r = 0.7866, between the two variables. i.e. Increase in advertisement expenses leads to increased sales volume.

29

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

Example:

30

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

31

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

Example: Find the correlation coefficient between age and playing habits of the following students using Karl Pearson’s coefficient of correlation method.

​

​

​

​

​

Solution:

To find the correlation between age and playing habits of the students, we need to compute the percentages of students who are having the playing habit.

Percentage of playing habits = (No. of Regular Players / Total No. of Students) * 100

Now, let us assume that ages of the students are variable X and percentages of playing habits are variable Y.

​

​

32

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

33

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

Interpretation: From the above calculation it is very clear that there is

high degree of negative correlation

​

i.e. r = -0.9912, between the two variables of age and playing habits.

Playing habits among students decreases when their age increases.

​

34

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

Practice Problems

​

1. Calculate Karl Pearson’s coefficient of correlation between the age and weight of the children:

​

2. Coefficient of correlation between X and Y is 0.3. Their covariance is 9. The variance of X is 16. Find the standard devotion of Y .

​

35

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

36

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

Regression:

A study of measuring the relationship between associated variables, wherein one variable is dependent on another independent variable, called Regression.

Regression analysis is a statistical tool to study the nature and extent of functional relationship between two or more variables and to estimate (or predict) the unknown values of dependent variable from the known values of independent variable.

The variable that forms the basis for predicting another variable is known as the Independent Variable and the variable that is predicted is known as dependent variable.

Example: if we know that the two variables price (X) and demand (Y) are closely related we can find out the most probable value of X for a given value of Y or the most probable value of Y for a given value of X.

​

37

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

It was Sir Francis Galton who first used the term regression as a statistical concept in 1877 to measure the relationship of height between parents and their children.

• He made a statistical study that showed that the height of children born to tall parents tends to ‘regress’ (go back) towards the mean height of population

� �� ��

38

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

​

​

• He used the term regression as a statistical technique to predict one variable (the height of children) from another variable (the height of parents).

​

​

​

• This is called ‘regression’ or ‘simple regression’ confined to bivariate data.

� �� � The Regression Technique is Primarily Used to��

39

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

​

​

Estimate the relationship that exists, on the average, between the dependent variable and the explanatory variable

Determine the effect of each of the explanatory variables on the dependent variable

Controlling the effects of all other explanatory variables

Predict the value of dependent variable for a given value of the explanatory variable

� ��

40

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

​

​

Difference between Regression and Correlation

• Regression

• Correlation

• Regression is concerned with bringing out the nature of relationship and using it to know the best approximate value of one variable corresponding to a known value of another variable

​

• Correlation on the other hand is concerned with quantifying the closeness of such relationship

​

41

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

If the relationship is best explained by a straight line, it is said to be Linear

On the contrary, if it is described more appropriately by a curve, the relationship is said to be Non-Linear

42

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

What is linear regression?

Suppose you are thinking of selling your home. Different sized homes around you have sold for different amounts:

Your home is 3000 square feet. How much should you sell it for? You have to look at the existing data and predict a price for your home. This is called linear regression.

​

Here's an easy way to do it. Look at the data you have so far:

43

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

Each point represents one home. Now you can eyeball it and roughly draw a line that gets pretty close to all of these points:

Then look at the price shown by the line, where the square footage is 3000:

44

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

Your home should sell for $260,000.

Plot your data, eyeball a line, and use the line to make predictions

45

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

46

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

Nonlinear Regression

A visual representation of a non-linear regression model. The blue points represent the data, while the red curve is the fitted regression model, capturing the non-linear relationship between the independent and dependent variables.

47

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

48

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

49

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

50

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

​

Properties of Regression Coefficients

:

​

51

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

​

Example: The following data gives the age and blood pressure (BP) of 10 sports persons.

​

​

​

​

​

(a) Find regression equation of Y on X and X on Y (Use the method of deviation from arithmetic mean)

​

(b)Find the correlation coefficient (r) using the regression coefficients.

​

(c) Estimate the blood pressure of a sports person whose age is 45.

​

​

52

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

53

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

54

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

55

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

Computation of coefficient of correlation using regression coefficient:

​

56

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

57

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

​

​

Example :

​

Following data about the sales and advertisement expenditure of a firm is given below

​

​

​

​

​

​

Correlation coefficient is 0.9.

(i) Estimate the likely sale for a proposed advertisement expenditure of Rs.10.crores .

(ii) What should be the advertisement expenditure if the firm proposes a sales target of Rs.60 crores.

​

​

58

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

​

​

​

59

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

​

​

60

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

Curve Fitting using method of least squares:

Curve fitting is a statistical technique used to create a curve that best represents a set of data points.

• It involves adjusting a mathematical function to approximate the relationship between variables in a dataset. The goal is to find a function that minimizes the difference between the observed data points and the values predicted by the function.

​

​

​

​

​

• Curve fitting is a powerful tool that enables the analysis and interpretation of data across various disciplines . This process is essential in various fields, including engineering, economics, biology, and machine learning, where understanding trends and making predictions are crucial.

Least Squares Method: This is the most common approach for curve fitting. It minimizes the sum of the squares of the residuals (the differences between observed and predicted values).

61

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

Methods of Fitting a Straight Line:

Least Square Method:

• The least square method of fitting a straight line is the one that statistically minimizes the sum of squares of the deviations
• It is by virtue of this property that the resultant straight line is indeed the line of best fit

62

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

Following are the two methods to form the two regression equations that is, equation for:

• Y on X
• X on Y

The above equations can be done by 2 methods:

Regression equations through normal equations

63

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

For Y on X, the equation is, For X on Y, the equation is

Y= a + b X X= a + b Y

​

For determining the values of ‘ a’ and ‘b’ we determine two normal equations which can be solved simultaneously

Two normal equations:

​

​

​

​

​

​

​

​

​

​

​

​

​

Regression Equations Through Normal Equations

64

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

The equation for a straight line is

Where

• Y is the dependent variable
• X is the independent variable
• a is the Y-intercept, which is the point at which the regression line crosses the Y-axis (the vertical axis) and
• b is the slope of the regression line.

It should be noted that the values of both a and b will remain constant for any given straight line.

Y= a + bX

65

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

The two normal equations are:

ƸY = na + b ƸX

ƸXY = a ƸX + b ƸX²

  where:

ƸY = the total of Y series

n = number of observations

ƸX = the total of X series

ƸX² = the total of squares of individual items in X series

ƸXY = the sum of XY column

​

a and b are the Y-intercept and the slope of the regression line, respectively.

​

66

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

Example :

Given the bivariate data:

​

​

• Fit a straight line on Y on X and hence predict Y , if X=20
• Fit the regression line on X on Y and hence predict X, if Y= 5

Solution:

In this case both the regression lines are needed. It is necessary to find ‘a’ and ‘b’. Given X, Y can be estimated and vice versa

67

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

68

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

Y= -1.8375

69

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

FITTING OF PARABOLA

70

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

T

71

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

72

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

73

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

Ans: [ a= 1.04; b= -0.198 ; c =0.244 ]

Ans: [ a= 1.33 ; b= 0.72

74

MANGALORE INSTITUTE OF TECHNOLOGY & ENGINEERING

2. Using the method of least squares, fit a linear relation of the form P = a+ bW to the given data and estimate P when W is150

​

Ans: [ a= 0.1329 ; b= -0.00393 ; c 0.4975]

Ans: [ a= 2.2785 ; b= 0.1879 and P= 30.4635 when w=150]