Logistic Regression

Learning Objectives

• Use logistic regression for binary classification
• Implement logistic regression for binary classification
• Address overfitting using regularization, to improve model performance

Linear Regression for Classification

Classification

Classification

Classification

Classification

Classification

Quiz

• Which of the following is an example of a classification task?

• Decide if an animal is a cat or not a cat.
• Estimate the weight of a cat based on its height.

Correct: This is an example of binary classification where there are two possible classes (True/False or Yes/No or 1/0).

Classification with Logistic Regression

Logistic Regression

Logistic Regression

Interpretation of Logistic Regression Output

Quiz

• g(z) is near zero
• g(z) is near negative one (-1)

Decision Boundary

Non-Linear Decision Boundaries

Non-Linear Decision Boundaries

Quiz

• Let’s say you are creating a tumor detection algorithm. Your algorithm will be used to flag potential tumors for future inspection by a specialist. What value should you use for a threshold?

• High, say a threshold of 0.9?
• Low, say a threshold of 0.2?

Correct: You would not want to miss a potential tumor, so you will want a low threshold. A specialist will review the output of the algorithm which reduces the possibility of a ‘false positive’. The key point of this question is to note that the threshold value does not need to be 0.5.

Quiz

• Which is an example of a classification task?

• Based on a patient's age and blood pressure, determine how much blood pressure medication (measured in milligrams) the patient should be prescribed.
• Based on the size of each tumor, determine if each tumor is malignant (cancerous) or not.
• Based on a patient's blood pressure, determine how much blood pressure medication (a dosage measured in milligrams) the patient should be prescribed.

​

​

This task predicts one of two classes, malignant or not malignant.

Quiz

​

• g(z) is near negative one (-1)
• g(z) is near zero(0)
• g(z) is near one (1)
• g(z) will be near 0.5

​

​

​

​

Quiz

• Predict it is a cat if g(z) >= 0.5
• Predict it is a cat if g(z) < 0.5
• Predict it is a cat if g(z) < 0.7
• Predict it is a cat if g(z) = 0.5

​

Think of g(z) as the probability that the photo is of a cat. When this number is at or above the threshold of 0.5, predict that it is a cat.

Quiz

• True/False? No matter what features you use (including if you use polynomial features), the decision boundary learned by logistic regression will be a linear decision boundary.

• True
• False

The decision boundary can also be non-linear, as described in the lectures.

Cost Function for Logistic Regression

Training Set

Squared error cost

Logistic Loss Function

Logistic Loss Function

Cost

Quiz

• Why is the squared error cost not used in logistic regression?

• The non-linear nature of the model results in a “wiggly”, non-convex cost function with many potential local minima.
• The mean squared error is used for logistic regression.

If using the mean squared error for logistic regression, the cost function is "non-convex", so it's more difficult for gradient descent to find an optimal value for the parameters w and b.

Simplified loss function

Simplified Cost function

Quiz

The second term of the expression is reduced to zero when the target equals 1.

Quiz

• In this lecture series, "cost" and "loss" have distinct meanings. Which one applies to a single training example?

In these lectures, loss is calculated on a single training example. It is worth noting that this definition is not universal. Other lecture series may have a different definition.

Quiz

Gradient Descent for Logistic Regression

Training Logistic Regression

Gradient Descent

Gradient Descent for Logistic Regression

Quiz

• Which of the following two statements is a more accurate statement about gradient descent for logistic regression?

Regularization to overfitting

Regression Example

Classification

�Quiz

• Our goal when creating a model is to be able to use the model to predict outcomes correctly for new examples. A model which does this is said to generalize well.
• When a model fits the training data well but does not work well with new examples that are not in the training set, this is an example of:

• Underfitting (high bias)
• None of these
• A model that generalizes well (neither high variance nor high bias)
• Overfitting (high variance)

Addressing Overfitting

Collect more training examples

Select features to include or exclude

Regularization

Quiz

• Applying regularization, increasing the number of training examples, or selecting a subset of the most relevant features are methods for…

• Addressing underfitting (high bias)
• Addressing overfitting (high variance)

These methods can help the model generalize better to new examples that are not in the training set.

Intution

Intution

Regularization

Regularization

Quiz

Regularised Linear Regression

Implementing gradient descent

Implementing gradient descent

Implementing gradient descent

Quiz

Regularised Logistic Regression

�Regularised Logistic Function

Quiz

• For regularized logistic regression, how do the gradient descent update steps compare to the steps for linear regression?

Quiz

• Which of the following can address overfitting?

• Select a subset of the more relevant features.
• Remove a random set of training examples
• Collect more training data
• Apply regularization

​

Quiz

• You fit logistic regression with polynomial features to a dataset, and your model looks like this.

​

• What would you conclude? (Pick one)
• The model has high bias (underfit). Thus, adding data is likely to help
• The model has high bias (underfit). Thus, adding data is, by itself, unlikely to help much.
• The model has high variance (overfit). Thus, adding data is, by itself, unlikely to help much.
• The model has high variance (overfit). Thus, adding data is likely to help

​

​