1 of 13

ניתוח נתונים ולמידת מכונה

אמית רננים – חלופה ליחידה 3

מוטי בזר

mottibz@gmail.com

https://www.commridge.com

בואו נתחיל עם האלגוריתם הראשון:

Linear Regression

Some of the materials are with permission from

Jose Portilla, Ariel Bar Itzhak, Koby Mike

2 of 13

Introduction to Linear Regression = חיזוי משתנה רציף

2

אימון

דוגמאות

מודל

חיזוי מחיר בית לא ידוע

מאפיינים

מטרה

חיזוי מחיר בית

https://www.kaggle.com/code/burhanykiyakoglu/predicting-house-prices

3 of 13

Introduction to Linear Regression

  • A linear relationship implies some constant straight-line relationship. The simplest possible being y=x
  • Below graph gives for each x=[1,2,3], the result of y=[1,2,3] (#1)
  • We could then build the y=x relationship as our “fitted” line (#2)
  • This means that for a new x value, we can predict its related y (#3)

3

#1

#2

#3

4 of 13

Introduction to Linear Regression

  • But what happens with real data?
  • Where we draw this line?
  • Is the red line the best?
  • We want to minimize the overall distance from the points to the line

4

סה"כ השקעה בשיווק

מכירות

5 of 13

Introduction to Linear Regression

  • הנוסחה לייצוג רגרסיה לינארית עם משתנה יחיד:�(הסימבול y-hat מדגיש שזאת פרדיקציה,� ולא חישוב מדויק של y)

​

  • חיזוי ערך דירה בהתבסס על מספר החדרים:

​

​

  • חיזוי ערך דירה עם שני משתנים:

​

5

6 of 13

Introduction to Linear Regression

בואו נחזה את רוחב העלה של אירוס על בסיס אורכו, לפי הפרמטרים הבאים:

w=0.416, b=-0.366

בעזרת נוסחת הייצוג:

​

​

6

Petal length

Petal width

1.5

​

2.5

​

6.5

​

0.5

​

0

​

What do you think about the values 0.5 and 0?

7 of 13

Introduction to Linear Regression

  • Multiple lines could be drawn. Some will better fit than others
  • We can measure the error from the real data points to the line, known as the residual error
  • The residual error can be positive or negative

7

8 of 13

Introduction to Linear Regression

  • Ordinary Least Squares (OLS) work by minimizing the sum of the squares of the differences between the observed value and the linear function’s predicted value
  • Squaring the distance does two things:
    • All numbers are positive
    • Punish high distances

​

​

8

9 of 13

Introduction to Linear Regression

  • Let’s explore how to translate a real data set into its mathematical notation for linear regression
  • There is a relationship between multiple features to the target

​

​

9

Features (X) Target (y)

x1 x2 x3 y

10 of 13

Introduction to Linear Regression

  • Let’s build the linear relationship between X (features) and y (target/label):

​

  • Each feature has an associated beta coefficient

​

​

​

  • It can be expressed as a sum:

10

The y-hat symbol means that this is a prediction (not a perfect fit to y)

= Intercept ( = 1)

11 of 13

Introduction to Linear Regression

  • There are pitfalls in relying on pure calculations
  • The Anscombe’s Quartet illustrates this:
    • Each graph results the same calculated regression line
    • Only the upper-left example makes sense

11

12 of 13

Introduction to Linear Regression

We define “best fit” as minimizing the squared error:

  • The residual error for some row j is:
  • Its Squared Error is:
  • The sum of all squared errors for m rows is:
  • The average squared error for m rows is: �(we will later discuss this further)

12

13 of 13

Introduction to Linear Regression

Let’s start with a simple Linear Regression (introduction notebook)

13