2 of 22

Announcements

Office hours

Is W 12-3PM (Zoom) OK? Would you prefer in-person options?

Lab 3 reminders

Part 3.2 check-in due 11:59pm, April 24
Final report due 11:59pm, May 8

Reference guides

smrichardson.github.io/stat214

Lab 3 Reference Guide (pipeline, notation, PCS)
Lab 3.2 LoRA Fine-Tuning Guide

3 of 22

Where we are in Lab 3

	What	Due	Status
3.1	BoW, Word2Vec, GloVe → ridge → CC	Apr 10	Done!
3.2	BERT + LoRA → ridge → compare all	Apr 24	← you are here
3.3	SHAP/LIME interpretation + final report	May 8

Today's focus: everything you need for 3.2

The good news: your pipeline from 3.1 carries over. The only new part is the embedding step.

Remaining discussions:

Apr 24 — SHAP & LIME interpretation (for 3.3)
May 1 — topic of your choice (last class!)

4 of 22

Outline

About me

~ 5 min

3.1 debrief: what went well, what was hard?

~ 5 min

LoRA and parameter-efficient fine-tuning

~ 15 min

Practical tips for 3.2

~ 10 min

Questions + lab work time

~ 15 min

5 of 22

About me

Sean Richardson

he/him · seanrichardson@berkeley.edu

1st year PhD student in Statistics
MS in Statistics from UChicago (causal inference, interpretability)
BA in Philosophy from UCLA, 1 year of law school

Research interests:

AI evaluation, causal inference, mechanistic interpretability

Taking over from Sequoia for the rest of the semester

6 of 22

3.1 Debrief

What was your experience with 3.1?

What worked well?

What was the hardest part?

Memory? Storage? Bridges-2?

What correlation coefficients did you get?

Most voxels near zero is expected!

Any questions before we move to 3.2?

7 of 22

Why fine-tune BERT?

Pretrained BERT already produces contextual embeddings
But it was trained on Wikipedia + BookCorpus, not podcast stories

Fine-tuning adapts the representations to our domain:

Informal, conversational narrative language
Specific vocabulary and phrasing from The Moth Radio Hour

Question: BERT has ~110 million parameters. Our story data has ~24,000 words. Does anyone see a problem?

8 of 22

Why fine-tune BERT?

Pretrained BERT already produces contextual embeddings
But it was trained on Wikipedia + BookCorpus, not podcast stories

Fine-tuning adapts the representations to our domain:

Informal, conversational narrative language
Specific vocabulary and phrasing from The Moth Radio Hour

Question: BERT has ~110 million parameters. Our story data has ~24,000 words. Does anyone see a problem?

Massive overfitting risk! We need a way to fine-tune with far fewer parameters → “parameter-efficient finetuning.” Also offers big compute savings!

9 of 22

LoRA: Low-Rank Adaptation

Key idea

We can probably find a good task-specific weight update within a low-dimensional subspace

Linear algebra review:�

Q: What’s a subspace?
Q: What does it mean for it to be low-dimensional?

10 of 22

LoRA: Low-Rank Adaptation

Key idea

We can probably find a good task-specific weight update within a low-dimensional subspace – which corresponds to the span of some low-rank matrix

11 of 22

LoRA: Low-Rank Adaptation

Key idea

We can probably find a good task-specific weight update within a low-dimensional subspace – which corresponds to the span of some low-rank matrix�

Q: How can we constrain the rank of a matrix when its parameters are being optimized by SGD?

12 of 22

LoRA: Low-Rank Adaptation

Key idea

We can probably find a good task-specific weight update within a low-dimensional subspace – which corresponds to the span of some low-rank matrix�

Q: How can we constrain the rank of a matrix when its parameters are being optimized by SGD?

13 of 22

LoRA: Low-Rank Adaptation

Steps

Instead of updating the full weight matrix W:

1. Freeze W (e.g., all original BERT parameters)

2. Add small trainable matrices A and B

3. The update is: ΔW = (γ/r) · A · Bᵀ

We could add this to W, but because matrix multiplication distributes over addition, can also keep W fixed and just route the inputs through both W and the LoRA update (you might see both formulations)

where r is the rank (e.g. 4 or 8), and γ is a scaling hyperparameter

Example dimensions

Original W: 768 × 768

= 590K parameters

LoRA (rank 8):

A: 768 × 8

B: 768 × 8

= 12K parameters

98% reduction!

14 of 22

LoRA intuition: PCA for weight updates

Think of it like PCA:

In PCA, a few principal components capture most of the variance in your data

In LoRA, a few "principal directions" capture most of the needed adaptation

The rank r controls how many directions you allow

Where do we apply LoRA?

BERT's attention layers have query, key, and value matrices
Standard approach: apply LoRA to query and value matrices

15 of 22

Hyperparameters to explore

Parameter	What it controls	Start with	Try
Rank (r)	Capacity of adaptation	8	4, 8, 16
lora_alpha (γ)	Update magnitude	8	r to 2r
Target modules	Which attention weights	Q + V	Add K
Learning rate	Optimizer step size	1e-4	1e-4 to 5e-4
Epochs	Passes over data	3	3-5

A full sweep of all hyperparameters is not expected. More systematic approaches exist (e.g. Bayesian optimization), but grid search over a few values is fine for this lab.

16 of 22

Practical: extracting BERT embeddings

Quick question: how many r's are in "strawberry"?

Why do LLMs famously get this wrong?

17 of 22

Practical: extracting BERT embeddings

LLMs don't see letters — they see tokens (subwords).

The subword problem

BERT tokenizes into subwords, not words:

"playing" → ["play", "##ing"] (2 vectors, but we need 1)

Solution: mean-pool subtokens belonging to the same word
Use word_ids() to get the subtoken-to-word mapping
Sanity check: len(embeddings) must equal len(words)

The 512-token limit

Most stories > 512 subtokens. Process in overlapping windows.
Average embeddings for words seen in multiple windows.

18 of 22

Important: two different BERT classes

BertModel

(for extracting embeddings)

Output: last_hidden_state

Shape: (seq_len, 768)

These are the contextual embeddings you want for the pipeline

BertForMaskedLM

(for training with MLM loss)

Output: logits

Shape: (seq_len, 30522)

These are vocabulary predictions, NOT embeddings!

Train with BertForMaskedLM (need the MLM loss head)
Extract embeddings from BertModel + LoRA adapter (need hidden states)
The LoRA adapter modifies the attention layers, which are shared between both classes

19 of 22

The 3.2 pipeline (what's new vs 3.1)

Same as 3.1:

Downsample → Delays → Trim → Ridge → CC evaluation

New for 3.2 (the embedding step only):

1. Extract embeddings from pretrained BERT (no training)

→ Handle subword aggregation and 512-token limit

2. Fine-tune BERT with LoRA on MLM objective

→ Tokenize stories into overlapping chunks

→ Train with DataCollatorForLanguageModeling (handles masking)

→ Save the LoRA adapter

3. Extract embeddings from fine-tuned BERT

→ Load adapter onto BertModel, use same extraction code

4. Compare all methods: BoW, W2V, GloVe, BERT, BERT+LoRA

20 of 22

Questions to address in your report

Does fine-tuning actually help?

Compare pretrained vs LoRA-finetuned BERT. If it doesn't help, why?

Do contextual embeddings predict different voxels?

Jaccard similarity of top voxel sets across embedding methods

Data leakage?

You fine-tune on story text, then use the same text for ridge. The MLM never sees fMRI data. Discuss.

Stability of LoRA hyperparameters

If changing rank flips which voxels are well-predicted, the conclusions are fragile (PCS!)

Weight interpretability

Why can't we just look at the ridge weights? (Answer: embedding dimensions aren't interpretable. SHAP/LIME in 3.3.)

21 of 22

Upcoming sessions

Next week (4/24): SHAP & LIME interpretation

You'll need this for Lab 3.3. I'll cover the wrapper pattern, how SHAP and LIME work, and practical tips for running them on your models.

Last session (5/1): your choice!

Causal inference

Potential outcomes, DAGs, common identification strategies

RL and RLHF/RLVR/RLAIF for LLMs

Reward modeling, policy optimization, DeepSeek-style reasoning RL

Both (overview of each)

Show of hands at the end?

1 of 22

2 of 22

3 of 22

4 of 22

5 of 22

6 of 22

7 of 22

8 of 22

9 of 22

10 of 22

11 of 22

12 of 22

13 of 22

14 of 22

15 of 22

16 of 22

17 of 22

18 of 22

19 of 22

20 of 22

21 of 22

22 of 22