Lecture 5: Base Rates and Reference Classes

Jacob Steinhardt

Stat 165, Spring 2024

Announcements

• HW3 released
• due Feb 6
• You can suggest questions for the forecasting competition!
• Opportunity to earn BP if your question gets used
• See Ed megathread

Warm-up Question

“What is the probability that Joe � Biden is President of the United � States on Nov. 1st, 2024?”

�[Forecast made as of Jan. 2022]

​

What information would you look at to answer this question?

Information I used

• What fraction of presidential terms are fully completed (last all 4 years)? 84%
• What fraction of presidents completed their term, conditional on making it through at least 288 days? 92%
• What is the death rate per year for 78-year-olds, according to actuarial tables? 15% (or 85% survival rate)

�These are all examples of using base rates.

• Think of this as “zeroth-order approximation + analogies”.

Decomposing the Problem

Just like with Fermi estimates, can improve accuracy �by decomposing the problem.

Reasons Biden wouldn’t complete his term:

• Death by natural causes
• Death by assassination
• Impeachment / resignation
• The U.S. does not have a president (coup, invasion, etc.)

Death by Natural Causes

• Previously gave this 15% chance (actuarial death rate for 78-year-olds)
• Do you think this overestimates or underestimates Biden in particular?�

• Biden is rich and seems reasonably �healthy. This should push the �chance of death downwards.
• I put this chance at 9%--⅗ of the�15% estimate.
• Adjusted to 34 months ≈ 25%

Death by Assassination

• 4 out of 58 presidents (7%) have been assassinated
• Most (3 of 4) were early in the term. Only 1 president made it as far as Biden and got assassinated (1.7%).���
• I decided to go with a middle �ground of 3.5%.

Impeachment / Resignation / Coup

• Only 1 out of 58 presidents have left office due to resignation.
• None have left due to impeachment�
• I would put the probability of a coup �or invasion that ends the presidency �at <1%.�
• Collectively, this gives ~2%.

Adding it all up

• We got 25% (natural causes), 3.5% (assassination), 2% (other).
• Adding this up gives 30.5% that Biden doesn’t complete his term, �or 69.5% that he does�
• Now it’s your turn (for Jan. 2024)

Other Examples

• Probability that I get Covid in the next month�
• What job will I have after I graduate Berkeley?�
• How much Series A funding will this startup get?�
• Will someone break Usain Bolt’s 100-meter dash world record (9.58s) by the end of 2024?�

Brainstorming Exercise

Pick one of these and list 2-3 base rates you would look up.�

Then come up with one additional forecasting question you are personally interested in, and list 2-3 base rates you could use for that.

Dangers of Base Rates

The reference class they used: �“Candidates that are good with the media and give them something to write about but, let’s be real, could never be president”

Base Rates for Events that Have Never Happened

Here are some examples where a “naive” base rate would give zero:

• Probability of fully self-driving cars by 2030
• Probability of a cure for diabetes by 2025
• Probability of a magnitude 8 earthquake in CA by 2025
• Probability that a comet hits earth and wipes out humanity

Laplace’s Rule of Thumb

If an event has has n opportunities to occur but has never happened, we assign probability 1/(n+2) to it happening the next time.

• Example: If someone is late to their first two meetings with me, I assign 25% probability to them being on time the next time.

​

Question. What are other places where this is reasonable? Any places where it’s obviously unreasonable?

Revisiting our examples

For each of the following examples, decide on a reference class and determine the value of n. Is the 1/(n+2) estimate reasonable, or would you give a different number?

1. Probability of fully self-driving cars by 2030
2. Probability of a magnitude 8 earthquake in CA by 2030

Laplace’s Rule with a Prior

• One problem with Laplace’s rule: when n = 0, assigns 50% probability to event happening.
• Should we have assigned 50% probability to George Washington resigning?�
• Seems too high to me: I would’ve maybe given 25%.�
• Generalization of Laplace’s rule: 1/(n + (1/p)), �where p is the “probability when n = 0”.�
• Washington example: p = 0.25, rule is 1/(n+4).
• 25% for Washington
• 20% for Adams (after seeing Washington didn’t resign)

Brainstorming Exercise

For the George Washington example, suppose we wanted to set p using a base rate (instead of just guessing 25%). What reference class(es) could we use?