COSI 115b - Lab 4

2/7/2025

Review

• What is Bayes rule?
• What is Naive Bayes?
• What is Logistic Regression?
• What is a perceptron?
• What is a structured perceptron?
• What is an activation function?
• What is softmax?
• What is a loss function?
• What is regularization?
• What is an optimizer?

Perceptron

• Simpler than logistic regression
• Activation function is {1,0}

Perceptron

• Learning Algorithm
• If ŷ = y, then ŷ - y = 0
• No update

Perceptron

• Learning Algorithm
• If ŷ = y, then ŷ - y = 0
• No update
• If ŷ = 0 and y = 1, then ŷ - y = - 1
• ∇ L = -x
• θ_t+1 = θ_t + ηx
• Increment the weights

Perceptron

• Learning Algorithm
• If ŷ = y, then ŷ - y = 0
• No update
• If ŷ = 0 and y = 1, then ŷ - y = - 1
• ∇ L = -x
• θ_t+1 = θ_t + ηx
• Increment the weights
• If ŷ = 1 and y = 0, then ŷ - y = 1
• ∇ L = x
• θ_t+1 = θ_t - ηx
• Decrement the weights

Perceptron Loss

L(ŷ,y) = (ŷ-y)z

​

Perceptron Example

Suppose pos = 1 and neg = 0.

We have features weights:

“dog”: -0.5

“like”: 0.2

“tacos”: -0.3

“not”: -0.7

“coffee”: 0.4

bias: 0.1

We get neg example, “The dog does not like fish”

What is the gradient?

(y_hat - y) x what is y_hat?

y_hat = sign(-0.5 * 0.2 * -0.7 * 0.1)

y_hat = 1

(1 - 0) x = (1)[1, 1, 0, 1, 0, 1] = [1, 1, 0, 1, 0, 1]

How do we update the current weights?

[-0.5, -0.2, -0.3, -0.7, 0.4, 0.1] - (lr *[1, 1, 0, 1, 0, 1])

What would happen if the gold label (true class) was positive?

We would get it right so no update (yhat - y) = 0

POS Tagging:

​

​

​

​

NER:

​

​

​

​