Election forecast comparison

	A	B	C	D	E	F	G	H	I	J	K	L	M	N	O	P	R	S	V	W	X	Y	Z	AA	AB	AC	AD	AE
1	See post linked here for details and analysis	Republican	Democrat		Metaculus	538	Manifold	Polymarket	Election Betting Odds	PredictIt	Avg	538 Lite	538 Classic	Metaculus community prediction	Good Judgement Open	Salem Manifold	Outcome	Outcome	PredictIt	Polymarket	Metaculus	Manifold	Good Judgement Open	Salem Manifold	538	R PredictIt	D PredictIt

2	Average score				3.98	3.60	3.09	2.52	2.23	2.12	3.00	4.10	4.09	3.72	3.67	1.20
3	Platform links				https://www.metaculus.com/questions/5632/gop-controls-us-senate-in-2023/	https://projects.fivethirtyeight.com/2022-election-forecast/senate/	https://manifold.markets/midterms	https://polymarket.com/midterms	https://electionbettingodds.com/	https://www.predictit.org/		https://github.com/fivethirtyeight/data/tree/master/election-forecasts-2022	https://github.com/fivethirtyeight/data/tree/master/election-forecasts-2022	https://www.metaculus.com/questions/5632/gop-controls-us-senate-in-2023/	https://www.gjopen.com/challenges/64-2022-us-midterm-elections-challenge?status=resolved	https://salemcenter.manifold.markets/SalemCenter/will-republicans-win-the-senate
4	Platform type				Prediction aggregator	Statistical modeling + poll aggregator	Prediction market (play money)	Prediction market	Prediction market aggregator	Prediction market		Polls-only model (no fundamentals or expert ratings)	Polls and fundamentals model (no expert ratings)	Simple median (no weighting / calibration model)	Prediction aggregator	Prediction market (play-money)
5
6					Predictions (all probabilities are for Republican win, on the evening of Nov 7, the day before Election Day)
7	Senate control				46	59	58	66	67	71	61	50	51	60	57	74	D	0	https://www.predictit.org/markets/detail/6874	https://polymarket.com/market/which-party-will-control-the-us-senate-after-the-2022-election	https://www.metaculus.com/questions/5632/gop-controls-us-senate-in-2023/	https://manifold.markets/BoltonBailey/will-democrats-maintain-control-of-8d067eb38c33	https://www.gjopen.com/questions/2162-which-party-will-win-control-of-the-us-senate-in-the-2022-midterm-elections	https://salemcenter.manifold.markets/SalemCenter/will-republicans-win-the-senate		73	32
8	House control				83	84	89	87	88	89	87	75	82	85	82	88	R	1	https://www.predictit.org/markets/detail/6892/Which-party-will-win-the-House-in-the-2022-election	https://polymarket.com/market/which-party-will-win-the-us-house-in-the-2022-elections	https://www.metaculus.com/questions/5814/2022-us-house-midterm-election/	https://manifold.markets/BoltonBailey/will-democrats-maintain-control-of	https://www.gjopen.com/questions/2161-which-party-will-win-control-of-the-us-house-of-representatives-in-the-2022-midterm-elections	https://salemcenter.manifold.markets/SalemCenter/will-republicans-win-the-house-of-r		90	12
9	Pennsylvania - Senate	Mehmet Oz	John Fetterman		42	57	57	58	61	62	56	44	47	51	48	57	D	0	https://www.predictit.org/markets/detail/7016	https://polymarket.com/market/2022-us-senate-elections-will-a-democrat-or-republican-win-in-pennsylvania	https://www.metaculus.com/questions/11422/oz-to-win-2022-pa-senate-race/	https://manifold.markets/MattP/will-dr-oz-be-elected-to-the-us-sen	https://www.gjopen.com/questions/2275-who-will-win-the-us-senate-election-for-pennsylvania-in-2022	https://salemcenter.manifold.markets/SalemCenter/will-republicans-win-the-senate-in-d755992df1b5		64	41	Pennsylvania - Senate - Mehmet Oz (R) vs John Fetterman (D)
10	Nevada - Senate	Adam Laxalt	Catherine Cortez Masto		65	51	64	63	65	66	62	58	54	59	65	71	D	0	https://www.predictit.org/markets/detail/7131	https://polymarket.com/market/2022-us-senate-elections-will-a-democrat-or-republican-win-in-nevada	https://www.metaculus.com/questions/12577/catherine-cortez-masto-re-elected-22/	https://manifold.markets/NcyRocks/will-a-democrat-win-the-2022-us-sen	https://www.gjopen.com/questions/2274-who-will-win-the-us-senate-election-for-nevada-in-2022	https://salemcenter.manifold.markets/SalemCenter/will-republicans-win-the-senate-in-836641ae8816		68	37	Nevada - Senate - Adam Laxalt (R) vs Catherine Cortez Masto (D)
11	Georgia - Senate	Herschel Walker	Raphael Warnock		63	63	54	64	63	63	62	52	54	51	53	65	D	0	https://www.predictit.org/markets/detail/7107	https://polymarket.com/market/2022-us-senate-elections-will-a-democrat-or-republican-win-in-georgia	https://www.metaculus.com/questions/11427/raphael-warnock-re-elected-to-senate-in-2022/	https://manifold.markets/NcyRocks/will-a-democrat-win-the-2022-us-sen-3d2432ba6d79	https://www.gjopen.com/questions/2315-who-will-win-the-us-senate-election-for-georgia-in-2022	https://salemcenter.manifold.markets/SalemCenter/will-republicans-win-the-senate-in-ef0f57fefe45		65	39	Georgia - Senate - Herschel Walker (R) vs Raphael Warnock (D)
12	Wisconsin - Senate	Ron Johnson	Mandela Barnes		80	81	85	83	87	88	84	75	74	81	87	79	R	1	https://www.predictit.org/markets/detail/7132	https://polymarket.com/market/2022-us-senate-elections-will-a-democrat-or-republican-win-in-wisconsin	https://www.metaculus.com/questions/9311/ron-johnson-wins-2022-wisconsin-us-senate/	https://manifold.markets/BTE/will-ron-johnson-be-reelected-in-th	https://www.gjopen.com/questions/2367-who-will-win-the-us-senate-election-for-wisconsin-in-2022	https://salemcenter.manifold.markets/SalemCenter/will-republicans-win-the-senate-in-617c73e9909d		88	12	Wisconsin - Senate - Ron Johnson (R) vs Mandela Barnes (D)
13	Ohio - Senate	J. D. Vance	Tim Ryan		81	87	84	87	90	91	87	86	83	85	87	85	R	1	https://www.predictit.org/markets/detail/7112	https://polymarket.com/market/2022-us-senate-elections-will-a-democrat-or-republican-win-in-ohio	https://www.metaculus.com/questions/11180/jd-vance-to-win-22-ohio-sen-election/	https://manifold.markets/BTE/will-jd-vance-win-the-ohio-senate-s	no question	https://salemcenter.manifold.markets/SalemCenter/will-republicans-win-the-senate-in-5dd0dba203a5		92	10	Ohio - Senate - J. D. Vance (R) vs Tim Ryan (D)
14	Arizona - Senate	Blake Masters	Mark Kelly		28	34	47	47	52	52	43	31	35	40	37	57	D	0	https://www.predictit.org/markets/detail/7112	https://polymarket.com/market/2022-us-senate-elections-will-a-democrat-or-republican-win-in-arizona	https://www.metaculus.com/questions/11206/blake-masters-to-win-22-az-sen-election/	https://manifold.markets/BTE/will-blake-masters-win-the-arizona	https://www.gjopen.com/questions/2281-who-will-win-the-us-senate-election-for-arizona-in-2022	https://salemcenter.manifold.markets/SalemCenter/will-republicans-win-the-senate-in		54	51	Arizona - Senate - Blake Masters (R) vs Mark Kelly (D)
15	Pennsylvania - Governor	Doug Mastriano	Josh Shapiro		1	3	12	10	12	11	8	2	3	5	12	17	D	0	https://www.predictit.org/markets/detail/7380/Which-party-will-win-the-2022-Pennsylvania-gubernatorial-election	https://polymarket.com/market/2022-us-gubernatorial-elections-will-a-democrat-or-republican-win-in-pennsylvania	https://www.metaculus.com/questions/11553/doug-mastriano-elected-pa-governor-in-2022/	https://manifold.markets/JonathanMast/will-josh-shapiro-win-the-2022-penn	https://www.gjopen.com/questions/2225-who-will-be-elected-governor-of-pennsylvania-in-2022	https://salemcenter.manifold.markets/SalemCenter/republican-governor-in-pa		12	90	Pennsylvania - Governor - Doug Mastriano (R) vs Josh Shapiro (D)
16	Texas - Governor	Greg Abbott	Beto O'Rourke		99	98	94	98	95	95	96	94	94	98	98	94	R	1	https://www.predictit.org/markets/detail/7566/Who-will-win-the-2022-Texas-gubernatorial-election	https://polymarket.com/market/2022-us-gubernatorial-elections-will-a-democrat-or-republican-win-in-texas	https://www.metaculus.com/questions/11301/beto-orourke-elected-texas-governor-in-2022/	https://manifold.markets/LarsDoucet/republicans-will-win-the-2022-texas	no question	No market, but see related one: https://salemcenter.manifold.markets/SalemCenter/democratic-governor-in-texas-or-flo		95	5	Texas - Governor - Greg Abbott (R) vs Beto O'Rourke (D)
17
18	Average prediction				59	62	64	66	68	69	65	57	58	62	63	69									Website: https://projects.fivethirtyeight.com/2022-election-forecast/senate/
19																									Spreadsheet: https://docs.google.com/spreadsheets/d/169Enzpea44VS18gKfkleZO-5vhsNUsv1WJ5_OUvSAxE/edit#gid=1984318916
20					Scores (transformed log score, higher is better, see definition below)
21					Metaculus	538	Manifold	Polymarket	Election Betting Odds	PredictIt	Avg	538 Lite	538 Classic	Metaculus community prediction	Good Judgement Open	Salem Manifold
22	Senate control				1.11	-2.69	-2.52	-5.56	-6.13	-7.61	-3.60	-0.14	-0.42	-3.22	-2.18	-9.43
23	House control				7.31	7.48	8.32	7.99	8.14	8.32	7.93	5.80	7.22	7.66	7.14	8.16
24	Pennsylvania - Senate	Mehmet Oz	John Fetterman		2.14	-2.29	-2.18	-2.52	-3.40	-3.77	-1.86	1.52	0.96	-0.29	0.57	-2.18
25	Nevada - Senate	Adam Laxalt	Catherine Cortez Masto		-5.15	-0.36	-4.74	-4.34	-5.10	-5.35	-4.06	-2.65	-1.25	-2.86	-5.15	-7.86
26	Georgia - Senate	Herschel Walker	Raphael Warnock		-4.34	-4.43	-1.20	-4.74	-4.23	-4.34	-3.83	-0.69	-1.14	-0.29	-0.89	-5.15
27	Wisconsin - Senate	Ron Johnson	Mandela Barnes		6.78	6.98	7.66	7.31	7.92	8.16	7.48	5.81	5.61	6.96	7.99	6.60
28	Ohio - Senate	J. D. Vance	Tim Ryan		6.96	8.03	7.48	7.99	8.42	8.64	7.93	7.81	7.24	7.66	7.99	7.66
29	Arizona - Senate	Blake Masters	Mark Kelly		5.26	4.01	0.84	0.84	-0.56	-0.44	1.83	4.61	3.89	2.63	3.33	-2.18
30	Pennsylvania - Governor	Doug Mastriano	Josh Shapiro		9.86	9.60	8.16	8.48	8.09	8.32	8.77	9.72	9.60	9.26	8.16	7.31
31	Texas - Governor	Greg Abbott	Beto O'Rourke		9.86	9.66	9.11	9.71	9.18	9.26	9.47	9.18	9.17	9.71	9.71	9.11
32
33	Average score				3.98	3.60	3.09	2.52	2.23	2.12	3.00	4.10	4.09	3.72	3.67	1.20
34
35
36			Prediction platform	Avg score		These are transformed log scores: higher (green) means more accurate, lower (red) means less accurate
37			Metaculus	3.98		Max score is 10 and guessing 50% gets score 0. Predicting right gets a positive score, wrong gets a negative score.
38			538	3.60		See my important note in the post - this is really just one data point, because all the races are highly correlated, so you can't infer too much from this one election cycle
39			Manifold	3.09
40			Polymarket	2.52
41			Election Betting Odds	2.23		See the second tab "Head-to-head comparisons" to see more detailed pairwise comparisons
42			PredictIt	2.12		See post linked here for details and analysis
43
44			Including extra forecasts in the comparison:
45			Prediction platform	Avg score
46			538 Lite	4.10
47			538 Classic	4.09
48			Metaculus	3.98
49			Metaculus community prediction	3.72
50			Good Judgement Open	3.67
51			538	3.60
52			Manifold	3.09
53			Avg	3.00
54			Polymarket	2.52
55			Election Betting Odds	2.23
56			PredictIt	2.12
57			Salem Manifold	1.20
58
59
60
61
62
63	Rough estimates for how much to update - based only on the overall Senate control question
64	Prediction for Democrat Senate control				54%	41%	42%	34%	33%	30%	39%	50%	49%	40%	43%	26%
65	Bayesian update / Kelly bet return ratio				1.39	1.07	1.08	0.87	0.84	0.76	1.00	1.27	1.25	1.03	1.10	0.67
66	Kelly bet profits/losses; Bayesian update increase/decrease.		This is how much more/less to believe each forecast after observing the result of the Senate control question.		39%	7%	8%	-13%	-16%	-24%	0%	27%	25%	3%	10%	-33%
67	How many similar updates we'd have to observe to get to roughly p=0.05 significance				55	285	238	132	103	65	0	75	82	674	182	44
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104