EA Failure Modes

Roxanne Heston

​

Thanks to Matthew Gentzel, Daniel Dewey, Owen Cotton-Barratt, and Alyssa Vance for helping me put this together.

Assumptions

• Social movements are susceptible to a host of failure modes.
• Effective altruism is a social movement, and unremarkable in this regard.
• We have spent insufficient time mapping and addressing failure modes.
• CEA and other organizations should collaborate on attending to these.

State of the discussion

• Kerry’s movement-building talk
• Geoff Anders’ movement-building talk
• Owen Cotton-Barratt in “Movement building” doc
• Post on Leverage blog
• Ben Kuhn’s argument against EA
• Jacob Steinhardt’s argument against EA
• Former EA Outreach team discussions
• “Old Guard” EAs and rationalists (Michael Vassar, Jonah Sinick, Mark Lee)
• Other individuals: Matthew Gentzel, Lawrence Chan
• The Craft and the Community series

How this will work

Questions:

• Does this overview seem right? Does it have the important problems? Does it overemphasize non-problems?
• Which of these seem the most I/N/T? Which should we prioritize?
• How can we make sure we take steps in the future to address the more I/N/T ones?

Types of failure modes

Effective altruism...

Exists

Disappears or Collapses

Grows

Stagnates or Shrinks

++ Good

<+ Good

Effective altruism disappears or collapses.

The EA movement as we know it no longer continues to exist. Not included as a failure mode is no longer existing because it becomes mainstream and is no longer necessary.

Effective altruism...

Exists

Disappears or Collapses because of...

• Infighting
• Insufficient organizational capacity

Effective altruism disappears or collapses.

• Infighting: Failure to converge on public-good priorities, such as the EA brand or new-member recruitment strategy. See Scott Alexander’s piece about outgroups.
• Failure to converge
• Failure to coordinate
• Insufficient organizational capacity: This is a problem when it seems unsolvable in the long-run and is preventative of our long-term goals, i.e. when it is structural rather than frictional.

Effective altruism stagnates or shrinks.

Assuming that the movement does not disappear or collapse for bad reasons, there are many reasons that it may fail to grow in value, missing or losing resources or ideas.

Stagnates or Shrinks because of…

People/Resources

• Missed value
• Dissuasion
• Stagnation

Ideas/Updates

• Ossification
• Well poisoning
• Locking in

Exists

Grows

Effective altruism stagnates or shrinks.

People/Resources

• Missed value
• Failure to maintain engagement of current members (e.g. burnout)
• Failure to engage new potential members (i.e. no clearly good functions for them to serve)
• Dissuasion
• Discourage otherwise-good people
• Stagnation
• Stick to the same set of things

Ideas/Updates

• Well poisoning
• No longer able to get people, resources, etc. on board because the stigma of joining is too strong
• Can’t work with external partners, or lose a bunch of previous EAs
• This is what Holden mentioned as the first point in his talk today
• Locking in
• Narrow range of considerations

Effective altruism does less than lots of good.

EAs are doing only marginally better than their counterfactuals, as much good as they would have without the community, or are actually making the problem worse.

<+ Good because of...

• Diffusion / Dilution
• Problem Exacerbation
• Inoculation / Subversion
• Lack of Feedback
• Locking In
• Opportunity Costs

++ Good

Grows

Effective altruism does less than lots of good.

• Diffusion / Dilution
• Includes hijacking of the brand by an outsider; EA prime
• EA comes to mean effective giving or earning to give only, for instance
• Problem Exacerbation
• Make problems worse by trying to make them better, e.g. accelerating AI timelines or bad policies
• Inoculation / Subversion
• ‘Throwing baby out with bathwater’
• Perhaps also includes just using a narrow set of potential approaches
• Lack of Feedback
• Groupthink, and therefore wrong priorities
• Locking In
• Growth in one particular group
• Opportunity Costs
• Doing worse things than we would otherwise do
• Ossification
• Stagnation of ideas
• Failure to attract good people

Discussion?