Bayesian Reasoning

In Data Science

18/10/2022 - Lectures 13

Outline

2

• Bayesian Logistic Regression — notebook mod2_part2_Bayesian_Logistic_Regression
• Integrated notes on Logistic Regression

Logistic Regression

3

Credits: University of Toronto

Credits: references [1], [2], [3]

Likely, you are familiar with logistic regression, a Machine Learning classification algorithm used to assign observations to a discrete set of classes.

Example

Linear VS Logistic Regression

4

Credits: references [1], [2], [3]

Logistic (Sigmoid) Function

5

Credits: references [1], [2], [3]

Hypothesis Representation:

Sigmoid:

features

You can decide a threshold to make a decision (e.g., everything above 0.5 is dog, anything below is cat)

Cost Function and Gradient Descent

6

Credits: references [1], [2], [3]

The cost function represents and optimization objective. i.e., we create a cost function and minimize:

Cost Function and Gradient Descent

7

to minimize our cost function we need to run the gradient descent function on each parameter, i.e.:

Gradient Descent Simplified | Image: Andrew Ng Course

Credits: references [1], [2], [3]

Now, let’s take a look at Bayesian Logistic Regression

8

Credits: references [1], [2], [3]

https://cfteach.github.io/brds/mod2_part2_Bayesian_Logistic_Regression.html

​

Useful References

9

• These lectures largely based on O. Martin, Chap 4
• https://cfteach.github.io/brds/referencesmd.html
• https://cfteach.github.io/brds/
• “Ordinary” Logistic Regression
• David J. MacKay, Chap. 39 — single neuron as classifier [link]
• https://towardsdatascience.com/introduction-to-logistic-regression-66248243c148
• https://towardsdatascience.com/logistic-regression-b0af09cdb8ad
• https://www.analyticsvidhya.com/blog/2021/08/conceptual-understanding-of-logistic-regression-for-data-science-beginners/

​

References of our course

10

https://cfteach.github.io/brds/referencesmd.html

Backup