A | B | C | D | E | F | G | H | I | J | K | L | M | N | O | P | Q | R | S | T | U | V | W | X | Y | Z | AA | ||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1 | ORIGINAL REVIEW QUESTIONS | SUPPLEMENTAL FORECAST QUESTIONS | ||||||||||||||||||||||||||
2 | REVIEWER (COLLECTED IN SEPTEMBER 2022) | TIMELINES: By 2070, it will become possible and financially feasible to build APS systems. | INCENTIVES: By 2070, and conditional on TIMELINES above, there will be strong incentives to build APS systems. | ALIGNMENT DIFFICULTY: By 2070, and conditional on TIMELINES and INCENTIVES above, it will be much harder to develop APS systems that would be practically PS-aligned if deployed, than to develop APS systems that would be practically PS-misaligned if deployed (even if relevant decision-makers don’t know this), but which are at least superficially attractive to deploy anyway. | HIGH-IMPACT FAILURES: By 2070, and conditional on TIMELINES, INCENTIVES, and ALIGNMENT DIFFICULTY above, some deployed APS systems will be exposed to inputs where they seek power in misaligned and high-impact ways (say, collectively causing >$1 trillion 2021-dollars of damage). | DISEMPOWERMENT: By 2070, and conditional on TIMELINES, INCENTIVES, ALIGNMENT DIFFICULTY, and HIGH-IMPACT FAILURES above, some of this misaligned power-seeking will scale (in aggregate) to the point of permanently disempowering ~all of humanity. | CATASTROPHE: By 2070, and conditional on TIMELINES, INCENTIVES, ALIGNMENT DIFFICULTY, HIGH-IMPACT FAILURES, and DISEMPOWERMENT above, this disempowerment would constitute an existential catastrophe? | EXISTENTIAL CATASTROPHE | Link to review | AGI BY 2043: Will Artificial General Intelligence (AGI) exist by 1 January 2043? | AGI BY 2070: Will Artificial General Intelligence (AGI) exist by 1 January 2070? | AGI BY 2100: Will Artificial General Intelligence (AGI) exist by 1 January 2100? | EXTINCTION: Assuming that Artificial General Intelligence (AGI) exists by 2070, will humanity either go extinct or have had its future potential drastically curtailed due to loss of control of AGI by 2200? | |||||||||||||||
3 | Superforecaster AI Expert #1 | 80.0% | 90.0% | 37.5% | 37.5% | 0.1% | 0.1% | 0.000009% | https://goodjudgment.io/AI/Review_1_Superforecaster_AI_Expert_1.pdf | |||||||||||||||||||
4 | Superforecaster AI Expert #2 | 10.0% | 100.0% | 60.0% | 1.0% | 0.0% | 5.0% | 0.0% | https://goodjudgment.io/AI/Review_2_Superforecaster_AI_Expert_2.pdf | |||||||||||||||||||
5 | Superforecaster AI Expert #3 | 90.0% | 60.0% | 70.0% | 30.0% | 70.0% | 98.0% | 7.7% | Declined to make public | |||||||||||||||||||
6 | Superforecaster AI Expert #4 | 67.5% | 72.0% | 70.0% | 7.0% | 5.0% | 6.0% | 5.0% | Declined to make public | |||||||||||||||||||
7 | Superforecaster AI Expert #5 | 70.0% | 30.0% | 40.0% | 80.0% | 12.0% | 20.0% | 0.2% | https://goodjudgment.io/AI/Review_3_Superforecaster_AI_Expert_5.pdf | |||||||||||||||||||
8 | Superforecaster AI Expert #6 | 9.0% | 95.0% | 50.0% | 2.0% | 50.0% | 95.0% | 0.04% | Declined to make public | |||||||||||||||||||
9 | Superforecaster AI Expert #7 | 80.0% | 95.0% | 80.0% | 20.0% | 33.0% | 20.0% | 0.8% | https://goodjudgment.io/AI/Review_4_Superforecaster_AI_Expert_7.pdf | |||||||||||||||||||
10 | Superforecaster AI Expert #8 | 35.0% | 100.0% | 100.0% | 0.001% | 1.0% | 50.0% | 0.000002% | Declined to make public | |||||||||||||||||||
11 | Superforecaster AI Expert #9 | 90.0% | 80.0% | 95.0% | 40.0% | 15.0% | 10.0% | 15.0% | https://goodjudgment.io/AI/Review_5_Superforecaster_AI_Expert_9.pdf | |||||||||||||||||||
12 | Superforecaster AI Expert #10 (note that this forecaster provided two different forecasts for each question, for two different definitions of "strategic awareness" -- I've here averaged them together) | 91.5% | 89.0% | 49.0% | 0.054% | 29.0% | 78.0% | 0.1% | https://goodjudgment.io/AI/Review_6_Superforecaster_AI_Expert_10.pdf | |||||||||||||||||||
13 | Superforecaster AI Non-expert #1 | 60.0% | 95.0% | 50.0% | 50.0% | 20.0% | 98.0% | 5.0% | https://goodjudgment.io/AI/Review_7_Superforecaster_AI_Non-expert_1.pdf | |||||||||||||||||||
14 | Superforecaster AI Non-expert #2 | 5.0% | 50.0% | 50.0% | 25.0% | 40.0% | 95.0% | 0.1% | https://goodjudgment.io/AI/Review_8_Superforecaster_AI_Non-expert_2.pdf | |||||||||||||||||||
15 | Superforecaster AI Non-expert #3 | 60.0% | 70.0% | 5.0% | 60.0% | 1.0% | 95.0% | "Less than 1%" | https://goodjudgment.io/AI/Review_9_Superforecaster_AI_Non-expert_3.pdf | |||||||||||||||||||
16 | Superforecaster AI Non-expert #4 | 60.0% | 70.0% | 10.0% | 60.0% | 30.0% | 50.0% | 0.4% | https://goodjudgment.io/AI/Review_10_Superforecaster_AI_Non-expert_4.pdf | |||||||||||||||||||
17 | Superforecaster AI Non-expert #5 | 90.0% | 40.0% | 40.0% | 40.0% | 4.0% | 100.0% | 0.2% | https://goodjudgment.io/AI/Review_11_Superforecaster_AI_Non-expert_5.pdf | |||||||||||||||||||
18 | Superforecaster AI Non-expert #6 | 85.0% | 95.0% | 100.0% | 100.0% | 20.0% | 99.0% | 12.0% | https://goodjudgment.io/AI/Review_12_Superforecaster_AI_Non-expert_6.pdf | |||||||||||||||||||
19 | Superforecaster AI Non-expert #7 | 20.0% | 90.0% | 40.0% | 35.0% | 20.0% | 99.0% | 0.5% | https://goodjudgment.io/AI/Review_13_Superforecaster_AI_Non-expert_7.pdf | |||||||||||||||||||
20 | Superforecaster AI Non-expert #8 | 15.0% | 20.0% | No answer | 5.0% | "Less than .5%" | <.5% or 5% | "Well below .5%" | https://goodjudgment.io/AI/Review_14_Superforecaster_AI_Non-expert_8.pdf | |||||||||||||||||||
21 | Superforecaster AI Non-expert #9 | 90.0% | 99.0% | 50.0% | 15.0% | 1.0% | 75.0% | 0.1% | https://goodjudgment.io/AI/Review_15_Superforecaster_AI_Non-expert_9.pdf | |||||||||||||||||||
22 | Superforecaster AI Non-expert #10 | 90.0% | 82.0% | 10.0% | 3.0% | 0.5% | 20.0% | 20% or around 0% | https://goodjudgment.io/AI/Review_16_Superforecaster_AI_Non-expert_10.pdf | |||||||||||||||||||
23 | Superforecaster AI Non-expert #11 | 35.0% | 85.0% | 20.0% | 50.0% | 10.0% | 99.0% | 0.3% | https://goodjudgment.io/AI/Review_17_Superforecaster_AI_Non-expert_11.pdf | |||||||||||||||||||
24 | Median calculated via this spreadsheet (note that some cells do not have numbers) | 67.5% | 85.0% | 50.0% | 30.0% | 13.5% | 76.5% | 0.3% | ||||||||||||||||||||
25 | ||||||||||||||||||||||||||||
26 | Superforecaster Aggregation May 2023 (note: the numbers above were from Superforecasters doing their work in isolation, after which point all the questions were posted on a platform where superforecasters could act as a team, challenge/question each other's views, and make more updates. Plus other superforecasters added their judgments. The questions were then re-opened in May 2023 for a final updating round, the aggregation here is the median of the last forecast made by everyone at that time.) | 80.0% | 90.0% | 58.0% | 25.0% | 5.0% | 40.0% | 1.0% | 12.0% | 40.0% | 60.0% | 6.0% | ||||||||||||||||
27 | ||||||||||||||||||||||||||||
28 | ||||||||||||||||||||||||||||
29 | ||||||||||||||||||||||||||||
30 | ||||||||||||||||||||||||||||
31 | ||||||||||||||||||||||||||||
32 | ||||||||||||||||||||||||||||
33 | ||||||||||||||||||||||||||||
34 | ||||||||||||||||||||||||||||
35 | ||||||||||||||||||||||||||||
36 | ||||||||||||||||||||||||||||
37 | ||||||||||||||||||||||||||||
38 | ||||||||||||||||||||||||||||
39 | ||||||||||||||||||||||||||||
40 | ||||||||||||||||||||||||||||
41 | ||||||||||||||||||||||||||||
42 | ||||||||||||||||||||||||||||
43 | ||||||||||||||||||||||||||||
44 | ||||||||||||||||||||||||||||
45 | ||||||||||||||||||||||||||||
46 | ||||||||||||||||||||||||||||
47 | ||||||||||||||||||||||||||||
48 | ||||||||||||||||||||||||||||
49 | ||||||||||||||||||||||||||||
50 | ||||||||||||||||||||||||||||
51 | ||||||||||||||||||||||||||||
52 | ||||||||||||||||||||||||||||
53 | ||||||||||||||||||||||||||||
54 | ||||||||||||||||||||||||||||
55 | ||||||||||||||||||||||||||||
56 | ||||||||||||||||||||||||||||
57 | ||||||||||||||||||||||||||||
58 | ||||||||||||||||||||||||||||
59 | ||||||||||||||||||||||||||||
60 | ||||||||||||||||||||||||||||
61 | ||||||||||||||||||||||||||||
62 | ||||||||||||||||||||||||||||
63 | ||||||||||||||||||||||||||||
64 | ||||||||||||||||||||||||||||
65 | ||||||||||||||||||||||||||||
66 | ||||||||||||||||||||||||||||
67 | ||||||||||||||||||||||||||||
68 | ||||||||||||||||||||||||||||
69 | ||||||||||||||||||||||||||||
70 | ||||||||||||||||||||||||||||
71 | ||||||||||||||||||||||||||||
72 | ||||||||||||||||||||||||||||
73 | ||||||||||||||||||||||||||||
74 | ||||||||||||||||||||||||||||
75 | ||||||||||||||||||||||||||||
76 | ||||||||||||||||||||||||||||
77 | ||||||||||||||||||||||||||||
78 | ||||||||||||||||||||||||||||
79 | ||||||||||||||||||||||||||||
80 | ||||||||||||||||||||||||||||
81 | ||||||||||||||||||||||||||||
82 | ||||||||||||||||||||||||||||
83 | ||||||||||||||||||||||||||||
84 | ||||||||||||||||||||||||||||
85 | ||||||||||||||||||||||||||||
86 | ||||||||||||||||||||||||||||
87 | ||||||||||||||||||||||||||||
88 | ||||||||||||||||||||||||||||
89 | ||||||||||||||||||||||||||||
90 | ||||||||||||||||||||||||||||
91 | ||||||||||||||||||||||||||||
92 | ||||||||||||||||||||||||||||
93 | ||||||||||||||||||||||||||||
94 | ||||||||||||||||||||||||||||
95 | ||||||||||||||||||||||||||||
96 | ||||||||||||||||||||||||||||
97 | ||||||||||||||||||||||||||||
98 | ||||||||||||||||||||||||||||
99 | ||||||||||||||||||||||||||||
100 |