1 of 75

Inference

Chris Gregg

CS109, Stanford University

Autumn 2026

2 of 75

Today: Stanford Eye Test

Chris Piech, CS109

3 of 75

  1. Combine Bayes Theorem and Random Variables

Learning Goals

Chris Piech, CS109

4 of 75

Four Prototypical Trajectories

Review

Chris Piech, CS109

5 of 75

Probabilistic Models

​

Roommates

2RoomDbl

Shared Partner

Single

​

Frosh

0.30

0.07

0.00

0.00

0.37

Soph

0.12

0.17

0.00

0.03

0.32

Junior

0.04

0.01

0.00

0.10

0.15

Senior

0.01

0.01

0.02

0.01

0.05

5+

0.02

0.00

0.05

0.04

0.11

​

0.49

0.26

0.07

0.18

1.00

Probability

Joint

Marginals

Chris Piech, CS109

6 of 75

Today: Inference

Inference noun

​

Updating one’s belief about a random variable (or multiple) based on conditional knowledge regarding another random variable (or multiple) in a probabilistic model.

​

TLDR: conditional probability with random variables.

Chris Piech, CS109

7 of 75

7

Chris Piech, CS109

8 of 75

8

Baby Delivery

Its 17 days before the due date and still no baby

Chris Piech, CS109

9 of 75

9

Baby Delivery

Its 17 days before the due date and still no baby

Chris Piech, CS109

10 of 75

10

Baby Delivery

Its 17 days before the due date and still no baby

Probability baby is born on due date increases by 12%

Chris Piech, CS109

11 of 75

Big idea for re-calculating PMF

11

We have a dictionary of probabilities mapped to days, based on the due date being day 0. We have probability values from -50 days before the due date to 30 days after the due date. The peak is after the due date because the due date is an estimate.

Day,Probability

-50,0.00009390

-49,0.00011268

-48,0.00014085

...

-3,0.02529675

-2,0.02633363

-1,0.02708620

0,0.02769864

1,0.02845020

2,0.02910833

3,0.02967256

...

28,0.00093990

29,0.00056337

30,0.00028168

Chris Piech, CS109

12 of 75

Big idea for re-calculating PMF

12

Once we reach a day (say NB17, or “no baby 17 days prior to the due date”), we have the information that we have not yet seen a baby 17 days prior (and, of course, all the days before that).

Day,Probability

-50,0.00009390

-49,0.00011268

-48,0.00014085

...

-3,0.02529675

-2,0.02633363

-1,0.02708620

0,0.02769864

1,0.02845020

2,0.02910833

3,0.02967256

...

28,0.00093990

29,0.00056337

30,0.00028168

Chris Piech, CS109

13 of 75

Big idea for re-calculating PMF

13

We need to apply Bayes’ Theorem for all the values (-50, 30) to get an updated PMF. We’ll do this in a loop. D is the delivery day relative to the due date with d being the day the baby is actually born in our model, and NB is the event that the baby is not born nb days before the due date.)

Chris Piech, CS109

14 of 75

Big idea for re-calculating PMF

14

Notably: is 0 for any day, d, up to and including nb, and 1 for any day after nb. That is, the probability that we have not yet seen the baby by day nb is zero if the baby was born on or before nb, and one if the baby is born after nb.

Chris Piech, CS109

15 of 75

To the code!

15

Chris Piech, CS109

16 of 75

Update Belief PMF

Before observation

After

Observation

Chris Piech, CS109

17 of 75

Bayesian Carbon Dating: Inference Overview

17

Let A be how many years old the sample is (A = 100 means the sample is 100 years old)

Let M be the observed amount of C14 left in the sample

Such that

Chris Piech, CS109

18 of 75

Probability of Having 900 Remain

18

There were originally 1000 C14 molecules.

Each molecule remains independently with equal probability pi

What is the probability that 900 remain?

Let T be the time to decay for any one molecule

Each molecules’ time to live is exponential with λ = 1/8267

Chris Piech, CS109

19 of 75

Inference as Code

19

def update_belief(m = 900):

"""

Returns a dictionary A, where A[i] contains the

corresponding probability, P(A = i| M = 900).

m is the number of C14 molecules remaining and i

is age in years. i is in the range 100 to 10000

"""

pr_A = {}

n_years = 9901

for i in range(100,10000+1):

prior = 1 / n_years # P(A = i)

likelihood = calc_likelihood(m, i) # P(M=m | A=i)

pr_A[i] = prior * likelihood

# implicitly computes the normalization constant

normalize(pr_A)

return pr_A

Chris Piech, CS109

20 of 75

Probability of Having 900 Remain

20

def calc_likelihood(m = 900, age):

"""

Computes P(M = m | A = age), the probability of

having m molecules left given the sample is age

years old. Uses the exponential decay of C14

"""

n_original = 1000

p_remain = math.exp(-age/C14_MEAN_LIFE)

return stats.binom.pmf(m, n_original, p_remain)

Chris Piech, CS109

21 of 75

Probability of Having 900 Remain

21

def calc_likelihood(m = 900, age):

"""

Computes P(M = m | A = age), the probability of

having m molecules left given the sample is age

years old. Uses the exponential decay of C14

"""

n_original = 1000 + delta_start(age)

p_remain = math.exp(-age/C14_MEAN_LIFE)

return stats.binom.pmf(m, n_original, p_remain)

Chris Piech, CS109

22 of 75

Four Prototypical Trajectories

Why is inference hard?

Bayes with variables

Can mix discrete and continuous

Normalization term is trippy at first

Likelihood term can be complex

Chris Piech, CS109

23 of 75

Bayes + Continuous Random Variables

Let X be a continuous random variable

Let N be a discrete random variable

Chris Piech, CS109

24 of 75

Four Prototypical Trajectories

End Review

Chris Piech, CS109

25 of 75

Today: Compare and Contrast Many Examples

25

Age from C14

Updated Delivery Prob

Age from Name

Hidden Chambers

Stanford Eye Test

Updating Lidar Belief

Chris Piech, CS109

26 of 75

Name 2 Age

26

Birth year

Name

Size of Dataset

Number of people with this name born this year

Chris Piech, CS109

27 of 75

Name 2 Age

27

Birth year

Name

Size of Dataset

Number of people with this name born this year

How can you use the joint to compute this?

Chris Piech, CS109

28 of 75

Name 2 Age

28

Number of people with this name born this year

How can you use the joint to compute this?

I.e., loop over all the possible birth years, and what is the probability they were born in that year and their name was Michael?

Chris Piech, CS109

29 of 75

Name 2 Age

29

How would you compute the entire PMF using code?

Chris Piech, CS109

30 of 75

30

Chris Piech, CS109

31 of 75

Compare and Contrast Code: (1) Bayesian Carbon Dating

31

After

Observation

Chris Piech, CS109

32 of 75

Compare and Contrast Code: (2) Baby Delivery

32

For each value d:

Chris Piech, CS109

33 of 75

Compare and Contrast Code: (2) Baby Delivery

33

Its 19 days until the due date and no baby

For each value d:

Chris Piech, CS109

34 of 75

Compare and Contrast Code: (3) Name To Age

34

Chris Piech, CS109

35 of 75

35

def update_belief_carbon_dating(m = 900):

# pr_A[i] is P(Age = i| m = 900).

pr_A = {}

for i in range(100,10000+1):

prior = 1 / n_years # P(A = i)

likelihood = calc_likelihood(m, i) #P(M=m|A=i)

pr_A[i] = likelihood * prior

# implicitly computes the normalization constant

normalize(pr_A)

return pr_A

def update_belief_baby(prior, today = 10):

# pr_D[i] is P(D = i| No Baby Yet).

pr_D = {}

for i in range(-50,25):

# P(NoBaby | D = i)

likelihood = 0 if i < today else 1

pr_D[i] = likelihood * prior[i]

# implicitly computes the LOTP

normalize(pr_D)

return pr_D

def update_belief_name_to_age(name = 'Laura'):

# pr_age[i] is P(Age = i| name).

# prob_name_and_age is just a counting from the US

# Social Security database.

pr_age = {}

for i in range(10,110):

pr_age[i] = calc_prob_name_and_age(name, i)

# implicitly computes the normalization constant

normalize(pr_age)

return pr_age

What do you notice is the same. What is different?

Chris Piech, CS109

36 of 75

Normalize in Python

36

# list normalization

def normalize_list(data_list):

total_sum = np.sum(data_list)

return np.array(data_list) / total_sum

​

>>> norm = normalize_list([10, 20, 30, 40])

>>> np.sum(norm) # 1.0, always (within floating point error)

​

​

# dictionary normalization

def normalize_dict(data_dict):

total_sum = sum(data_dict.values())

normalized = {}

for key, value in data_dict.items():

normalized[key] = value / total_sum

return normalized

​

>>> norm = normalize_dict({'a': 100, 'b': 200, 'c': 300})

>>> np.sum(norm.values()) # 1.0, always (within floating point error)

​

​

return

[0.1 0.2 0.3 0.4]

return

{'a': 0.166, 'b': 0.333, 'c': 0.5}

Chris Piech, CS109

37 of 75

37

Chris Piech, CS109

38 of 75

Four Prototypical Trajectories

Are you ready

For Hidden Chambers???

Chris Piech, CS109

39 of 75

Hidden Pyramid Chambers with Poisson + Bayes

Chris Piech, CS109

40 of 75

Basics of Muography

40

Beer Lambert Law

Rate of muons depends on x, amount of limestone

Chris Piech, CS109

41 of 75

41

Beer Lambert Law

Rate of muons depends on x, amount of limestone

Chris Piech, CS109

42 of 75

42

Chris Piech, CS109

43 of 75

43

Beer Lambert Law

Rate of muons depends on x, amount of limestone

Chris Piech, CS109

44 of 75

44

Chris Piech, CS109

45 of 75

45

Chris Piech, CS109

46 of 75

Four Prototypical Trajectories

Beyond Inference:

What hidden chambers make all the muon readings most likely?

Chris Piech, CS109

47 of 75

Four Prototypical Trajectories

Are you ready

For Stanford Acuity Test???

Chris Piech, CS109

48 of 75

A Better Eye Test

48

https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(21)02149-8/fulltext

https://www.science.org/content/article/eye-robot-artificial-intelligence-dramatically-improves-accuracy-classic-eye-exam

https://ojs.aaai.org/index.php/AAAI/article/view/5384/5240

Chris Piech, CS109

49 of 75

Prior Belief in Ability to See (Random Var A)

49

Can’t see

Standard vision

Probability Mass

Chris Piech, CS109

50 of 75

PMF is Actually Stored as a Dictionary

50

a

P(A=a)

​

a

P(A=a)

​

​

​

a

P(A=a)

0.00

0.00198

​

0.20

0.00372

​

​

​

0.80

0.01684

0.01

0.00205

​

0.21

0.00384

​

​

​

0.81

0.01708

0.02

0.00211

​

0.22

0.00396

​

​

​

0.82

0.01731

0.03

0.00218

​

0.23

0.00408

​

​

​

0.83

0.01753

0.04

0.00225

​

0.24

0.00421

​

​

​

0.84

0.01774

0.05

0.00233

​

0.25

0.00434

​

​

​

0.85

0.01795

0.06

0.0024

​

0.26

0.00447

​

​

​

0.86

0.01814

0.07

0.00248

​

0.27

0.00461

​

​

​

0.87

0.01832

0.08

0.00256

​

0.28

0.00475

​

…

​

0.88

0.01848

0.09

0.00264

​

0.29

0.00489

​

​

​

0.89

0.01864

0.10

0.00273

​

0.30

0.00504

​

​

​

0.90

0.01877

0.11

0.00281

​

0.31

0.00519

​

​

​

0.91

0.0189

0.12

0.0029

​

0.32

0.00535

​

​

​

0.92

0.019

0.13

0.00299

​

0.33

0.00551

​

​

​

0.93

0.01909

0.14

0.00309

​

0.34

0.00567

​

​

​

0.94

0.01916

0.15

0.00319

​

0.35

0.00584

​

​

​

0.95

0.01921

0.16

0.00329

​

0.36

0.00601

​

​

​

0.96

0.01924

0.17

0.00339

​

0.37

0.00619

​

​

​

0.97

0.01925

0.18

0.0035

​

0.38

0.00637

​

​

​

0.98

0.01924

0.19

0.00361

​

0.39

0.00655

​

​

​

0.99

0.01921

def main():

belief = get_prior_belief()

…

Chris Piech, CS109

51 of 75

Prior Belief in Ability to See (Random Var A)

51

belief = get_prior_belief()

As a graph

As a dictionary

Chris Piech, CS109

52 of 75

The Patient is Shown One Letter and They Get it Wrong

52

User: wrong

Observation Y = 0

Font size 0.7

Chris Piech, CS109

53 of 75

Number or Dictionary?

53

Chris Piech, CS109

54 of 75

Number or Dictionary?

54

belief[a] = 0.001

belief

Chris Piech, CS109

55 of 75

Today: I am going to simplify the units of vision

55

Normally doctors measure ability to see in logarithmic units. To make today’s demo easier to understand

​

I have translated both onto a

[0, 1] scale.

Where 0 means can’t see and 1.0 is standard vision

Chris Piech, CS109

56 of 75

Posterior Belief in Ability to See (Random Var A)

56

Can’t see

Standard vision

Probability Mass

Chris Piech, CS109

57 of 75

Bayes with Random Variables

57

User: wrong

1

2

3

Observation Y = 0

(At font size s1)

Chris Piech, CS109

58 of 75

Inference on a non-bernoulli random variable

58

# RV bayes as code

def update(belief, obs):

for a in support:

prior_a = belief[a]

likelihood = calc_likelihood(a, obs)

belief[a] = prior_a * likelihood

normalize(belief)

likelihood

In plain English: run bayes for each value of a

Chris Piech, CS109

59 of 75

Normalize???

59

# RV bayes as code

def update(belief, obs):

for a in support:

prior_a = belief[a]

likelihood = calc_likelihood(a, obs)

belief[a] = prior_a * likelihood

normalize(belief)

In plain English: this is the numerator, summed over all values of A

Chris Piech, CS109

60 of 75

Inference

60

In general Bayes theorem with a random variable is like the cellphone problem: multiple possible assignments to keep track of

Still true when some variables are continuous

Chris Piech, CS109

61 of 75

Random Variables

61

Not all beliefs can be represented as a function. Dictionary / table is a great way to represent a random variable belief.

This is formally called non-parametric

Chris Piech, CS109

62 of 75

Representing Continuous Variables

62

Dictionary can also be used to represent a discretization of a continuous random var

We do it all the time! Yay compute!

Chris Piech, CS109

63 of 75

Four Prototypical Trajectories

Multiple observations??

Chris Piech, CS109

64 of 75

Multiple Observations

User: wrong

Obs 1

Observation Y = 0

(At font size 0.7)

Observation Y = 0

(At font size 0.8)

User: correct

Obs 2

Chris Piech, CS109

65 of 75

Multiple Observations

65

Single Observation:

Multiple Observations:

2

2

Chris Piech, CS109

66 of 75

Multiple Observations

66

Single Observation:

Multiple Observations:

2

2

Chris Piech, CS109

67 of 75

Posterior becomes new prior

67

User: correct

1

2

3

Observation Y = 1

(At font size 0.8)

Chris Piech, CS109

68 of 75

In the Code

Chris Piech, CS109

69 of 75

Four Prototypical Trajectories

Beyond Inference:

How do you select the next size to show?

Chris Piech, CS109

70 of 75

Today: Five New Real + Exciting Problems

70

Age from C14

Updated Delivery Prob

Age from Name

Hidden Chambers

Stanford Eye Test

Cellphone Tracking Cont.

Chris Piech, CS109

71 of 75

Chris Piech, CS109

72 of 75

Likelihood function f(X=x | T=t)? (soln)

Chris Piech, CS109

73 of 75

Observe LiDAR measurement of 4m. f(X=4 | T=t)=?

Chris Piech, CS109

74 of 75

We want a new belief in the true dist., f(T=t | X=4)

Chris Piech, CS109

75 of 75

Today: Compare and Contrast Many Examples

75

Age from C14

Updated Delivery Prob

Age from Name

Hidden Chambers

Stanford Eye Test

Updating Lidar Belief

Chris Piech, CS109