Logistic Regression 2
Chris Gregg
CS109, Stanford University
Autumn, 2026
Logistic Regression: A Refresher
1
Predict a probability
Example: Is this email spam?
Features x: links, words, etc. Label y: 1 = spam, 0 = not spam
2
Compute a weighted score
z = θᵀx
The weights θ determine how features affect the score.
3
Use the sigmoid to get a probability
p = σ(z) = 1 / (1 + e⁻ᶻ)
Large negative score: p near 0. Zero: p = 0.5.
Large positive score: p near 1.
We can turn the probability into a class using a threshold.
Learning the Weights
1
Learning means choosing θ
Choose weights that assign high probability
to the labels we actually observed.
2
Score the probability of the observed label
Suppose the model predicts p = 0.9 for spam.
If y = 1: observed-label probability is 0.9.
If y = 0: observed-label probability is 0.1.
3
Today’s math
Write the likelihood of all observed labels.
Take its log, then differentiate to improve θ.
Chapter 2: How Come?
5
Logistic Regression
1
2
3
Make logistic regression assumption
Calculate the log probability for all data
Get derivative of log probability with respect to thetas
Often call this
1
6
How did we get that LL function?
Probability mass function:
7
Recall: PMF of Bernoulli
0
1
p
1 - p
PMF of Bernoulli
PMF of Bernoulli (p = 0.2)
8
Implies
For IID data
Take the log
1
Recall:
9
How did we get that gradient?
10
Sigmoid has a Beautiful Slope
True fact about sigmoid functions
11
Sigmoid has a Beautiful Slope
Sigmoid, you should be a ski hill
Chain rule!
Plug and chug
12
Sigmoid has a Beautiful Slope
ARE YOU READY???
14
I think I’m Ready…
Where
15
Think About Only One Training Instance
We only need to calculate the gradient for one training example!
We will pretend we only have one example
We can sum up the gradients of each example to get the correct answer
16
First, imagine only one example
Where
CHAIN RULE
Already did that one
Derive this one
Simplify
17
Now, all the data
See last slide
Derivative of sum…
Some people don’t like hats…
18
Logistic Regression
1
2
3
Make logistic regression assumption
Calculate the log probability for all data
Get derivative of log probability with respect to thetas
1
19
The Hard Way
Phew!
Chapter 3: Philosophy
(if time)
Neuron
Neuron
Neuron
Neuron
Neuron
Some inputs are more important
Artificial Neurons
x1
x2
x3
x4
θ1
θ2
θ3
θ4
y
x1
x2
x3
x4
Biological Basis for Neural Networks
Deep learning is (at its core) many logistic regression pieces stacked on top of each other.
(aka Neural Networks)
Computer Vision
Alpha GO
Revolution in AI
Computers Making Art
35
Basically just many logistic regression cells
And lots of chain rule…