P(AGI by year X | no AGI yet)

Number of failed trials observed by 2020

First-trial probability

Number of virtual successes

Regime start-time

Initial distribution over P(AGI next year | no AGI yet)

2020 distribution over P(AGI next year | no AGI yet)

Number of trials between 2020 and year X

Update for observed failures

First-trial probability�What are the odds of successfully building AGI on the first “trial”?

Ambitious but feasible technology that a serious STEM field is explicitly trying to develop

Looks at how long it took to get from “start” to “finish” for around 10 successful technologies

Selection biases

R&D failures not included

Difficulty of building AGI

Reference classes

High impact technology that a serious STEM field is trying to build in 2020�Compiles a list of technologies that could be transformative with >=10% probability

Technological development that has a transformative effect on the nature of work and society�E.g. Industrial Revolution. Accounts for population, GWP, and technological progress

Notable mathematical conjectures�(Based on previous work by AI Impacts) Looks at how long notable mathematical conjectures take to be solved, and fits the data with an exponential function

Most informative

Weakly informative

Somewhat informative

Least informative

Most items on the list are not central to the field

Arbitrary inclusion criteria

Limited data points

Primarily looks at impacts rather than feasibility of development

Selection biases�Focuses on remembered conjectures

Number of virtual successes�Affects how quickly P(AGI next year | no AGI yet) is updated. Fewer virtual successes implies more uncertainty about how hard it is to build AGI.

Plausibility of the prior�Looks at prior distributions given by the number of virtual successes and the first-time probability

Plausible updates�How large an update should be, given X years of failure to build AGI

Pragmatism�The mathematical interpretation of first trial probability is simpler with 1 virtual success

Regime start-time�“Time such that the failure to develop AGI before that time tells us very little about the probability of success after that time”

Historical events�Considers several possible starting events, e.g. the 1956 Dartmouth workshop

Downweighting very early start-times�Since the world is changing much faster now than during ancient times; downweight by three factors

Global population�Weight each year by the population in the year; 2x people → expect roughly 2x technological progress

GWP�Weight each year by the % annual increase in GWP

Technological progress�Weight each year by the amount of technological progress in frontier countries

Number of observed failed trials

(since the regime start-time)

If start time is very early

Trial definition�How much a R&D input increases based on a particular “trial”

R&D-based models of economic growth�Specifically the model proposed by Jones (1995); strongly defines the preferred trial definition

(Empirically) Increasing the number of AI researchers does not increase growth rate

Each 1% increase in the level of AI technological development has a constant probability of leading to the development of AGI

Biological anchors

R&D Growth rate

Researcher-year trials�E.g. “a 1% increase in the total researcher-years so far”

Compute trials�E.g. “a 1% increase in the largest amount of compute used to develop an AI system to date”

AI and efficiency

Relative importance of compute vs research

Price of compute from 1956 to 2020

Projected compute spending in 2036

Lifetime anchor

Evolution anchor