1 of 26

Week 3

Logistik Regression as Shallow Neural Network

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com

2 of 26

Outline

  • Konsep Logistic Regression
  • Logistic Regression as Shallow Neural Network

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com

3 of 26

Logistic Regression

  • Supervised machine learning algorithm untuk klasifikasi.
  • Tujuan:
    • Memprediksi probabilitas suatu instance termasuk ke dalam kelas tertentu (categorical dependent variable), berdasarkan sekumpulan independent variables tertentu.
    • Target berupa kategori/nilai diskret (misal: 0 atau 1, yes/no).
  • Cara kerja:
    • Disebut "regresi" karena mengambil output fungsi regresi linier sebagai input, lalu menggunakan fungsi sigmoid untuk mengubahnya menjadi probabilitas.
    • LR menyesuaikan fungsi logistik berbentuk "S", yang membatasi output di antara dua nilai (0 dan 1), bukan menyesuaikan garis lurus (seperti regresi linier),
  • Output:
    • Bukan nilai eksak 0 atau 1, melainkan nilai probabilitas antara 0–1 yang kemudian diklasifikasikan ke kelas tertentu berdasarkan threshold (umumnya 0.5).

​

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com

4 of 26

Logistic Function (Sigmoid Function)

  • Fungsi yang memetakan nilai prediksi menjadi probabilitas.
  • Input: sembarang nilai real (−∞,+∞)
  • Output: nilai probabilitas (0,1).
  • Untuk menentukan klasifikasi akhir digunakan threshold .
    • Umumnya threshold = 0.5
    • Nilai ≥ threshold → cenderung ke kelas 1
    • Nilai < threshold → cenderung ke kelas 0

​

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com

5 of 26

Type of Logistic Regression

  • Binomial
    • Variabel dependen hanya memiliki 2 kemungkinan kategori.
    • Contoh: 0 atau 1, Lulus atau Tidak Lulus.
  • Multinomial
    • Variabel dependen memiliki 3 atau lebih kemungkinan kategori yang tidak berurutan (unordered).
    • Contoh: "kucing", "anjing", atau "domba".
  • Ordinal
    • Variabel dependen memiliki 3 atau lebih kemungkinan kategori yang berurutan (ordered).
    • Contoh: "rendah", "sedang", atau "tinggi".

​

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com

6 of 26

Linear Regression vs. Logistic Regression

Aspek

Linear Regression

Logistic Regression

Kegunaan

Menyelesaikan masalah regresi (memprediksi variabel dependen kontinu)

Menyelesaikan masalah klasifikasi (memprediksi variabel dependen kategorikal)

Output

Nilai kontinu (contoh: harga, usia)

Nilai kategorikal (contoh: 0/1, Ya/Tidak)

Pendekatan

Mencari garis terbaik (best fit line)

Mencari kurva-S (S-curve)

Metode estimasi

Least Square Estimation

Maximum Likelihood Estimation

Hubungan variabel

Membutuhkan hubungan linear antara variabel dependen dan independen

Tidak membutuhkan hubungan linear

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com

7 of 26

How does Logistic Regression Works?

  • Independent Variables (Variabel Independen)
    • Karakteristik input atau faktor prediktor yang digunakan untuk memprediksi variabel dependen.

​

​

​

  • Dependent Variable (Variabel Dependen)
    • Variabel target dalam model logistic regression yang ingin diprediksi.

​

 

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com

8 of 26

How does Logistic Regression Works?

  • Coefficient (Koefisien)
    • Parameter yang diestimasi dalam model logistic regression, menunjukkan bagaimana hubungan antara variabel independen dan variabel dependen.

​

  • Intercept / Bias
    • Konstanta dalam model logistic regression.

​

​

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com

9 of 26

How does Logistic Regression Works?

Nilai z berada pada rentang (−∞,+∞)

​

​

​

​

Bagaimana mengubah output z menjadi nilai probabilitas (0,1) ???

​

 

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com

10 of 26

How does Logistic Regression Works?

  • Odds
    • Rasio antara suatu peristiwa terjadi dengan peristiwa tidak terjadi.

​

  • Log-odds (logit)
    • LR mendefinisikan log-odds sebagai fungsi linear:

​

​

​

​

  • Remove the log by exponentiating

 

 

 

 

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com

11 of 26

How does Logistic Regression Works?

  •  

 

Fungsi Sigmoid

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com

12 of 26

How does Logistic Regression Works?

 

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com

13 of 26

How does Logistic Regression Works?

 

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com

14 of 26

How does Logistic Regression Works?

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com

15 of 26

How does Logistic Regression Works?

 

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com

16 of 26

Predict the Output of LR

 

 

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com

17 of 26

Predict the Output of LR

 

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com

18 of 26

Predict the Output of LR

 

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com

19 of 26

Outline

  • Konsep Logistic Regression
  • Logistic Regression as Shallow Neural Network

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com

20 of 26

Logistic Regression is An Artificial Neuron

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com

21 of 26

Log Likelihood and Binary Cross Entropy Loss

  • We then saw that maximization of the log likelihood

​

​

​

  • is identical to minimization of the loss (Binary Cross Entropy-loss)

 

 

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com

22 of 26

Training Logistic Regression

  • When considered as a deep learning model or a machine learning model, one sometimes ignores the maximum likelihood interpretation of logistic regression and starts with the cross entropy loss as an engineered loss function which requires minimization.

​

​

​

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com

23 of 26

Training Logistic Regression

  •  

 

 

 

 

 

 

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com

24 of 26

Training Logistic Regression

  •  

 

 

 

 

 

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com

25 of 26

Assumptions of Logistic Regression

  1. Independent observations: Each observation is independent of the other. meaning there is no correlation between any input variables.
  2. Binary dependent variables: It takes the assumption that the dependent variable must be binary or dichotomous, meaning it can take only two values. For more than two categories SoftMax functions are used.
  3. Linearity relationship between independent variables and log odds: The relationship between the independent variables and the log odds of the dependent variable should be linear.
  4. No outliers: There should be no outliers in the dataset.
  5. Large sample size: The sample size is sufficiently large

​

​

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com

26 of 26

Latihan

  •  

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com