1 of 8

Aggregating & inferring from x-risk estimates

2 of 8

Common relevant problems

  1. Being dismissive/alarmist without being quantitative
  2. Attending to only one or a few sources of estimates
    • E.g., just 2008 GCR survey or just Precipice
    • Ignores other evidence & disagreement
  3. Collections miss many things & aren't updated
  4. Aggregating things that are actually quite different
    • E.g., different time horizons
    • E.g., extinction vs x-risk vs collapse vs GCR
    • E.g., state nuclear detonation vs nuclear war kills 1m
  5. Giving lower-quality sources equal weight
  6. Only arithmetic means of probabilities

3 of 8

Proposed solution

  • Collect all relevant estimates together
  • Living resource (not static)
  • Crowdsource suggestions
  • Organise into categories
  • Extrapolate?
    • To end up with many estimates on each of a small number of key things
      • E.g., per year P of any nuclear conflict
    • Use transparent process & assumptions
  • Give weights?
  • Show multiple central tendency and spread stats?
  • Accompany with a static paper/post

4 of 8

(Screen share nuclear database as example)

5 of 8

Things I’d be keen to discuss

(We only have 10 mins, so please also reach out after the talk if you want to discuss further!)

  1. Is this a good idea? Should this be done for other x-risks?
  2. How can I make this maximally useful to decision-makers and increase the chance that they actually do use it? (Including non-EA stakeholders, who I think in theory should find this really useful.)
  3. Any easy way to automate additions and updates?
  4. Is the sort of extrapolation I’m doing sensible and useful?
  5. Maybe I should multiply distributions, not point estimates?
  6. What summary stats should I show? What should I emphasise?
  7. How comprehensive should the database be?
  8. Anything else related to these topics

6 of 8

Some arguments that ~this might be harmful

  1. Garbage in, garbage out?
    • Probably noisy
    • Maybe biased
  2. Overly privileges one specific type of evidence?
  3. Anchoring and information cascades?
  4. Overstates agreement/disagreement?
    • Some estimates aren’t independent
    • Some estimates are low-quality
  5. Reputational risk? E.g.:
    • “arrogantly & naively trying to quantify the unknowable / sci-fi scenarios”
    • “dangerously implying nuclear war is unlikely”
    • “taking seriously alarmist/off-the-cuff forecasts from random bloggers and cranks”
  6. Infohazards?

7 of 8

Further arguments that ~this might just not be worthwhile

  • Time-consuming to make?
  • Difficult/unappealing to navigate & use?
  • Risk levels often not key crux?
    • Though note that we can also solicit/aggregate estimates on mechanisms, interventions, etc.

8 of 8

See also