ZeroSCROLLS-Benchmark-Scores

	A	B	C	D	E	F	G	H	I	J	K	L	M	N	O	P	Q	R	S	T	U	V	W	X	Y
1	SubmissionDate	Contributors	ModelName	#Parameters	InputLength	Description	ZeroScrollsScore	GovReport_Rouge1	GovReport_Rouge2	GovReport_RougeL	SummScreenFD_Rouge1	SummScreenFD_Rouge2	SummScreenFD_RougeL	QMSUM_Rouge1	QMSUM_Rouge2	QMSUM_RougeL	SQuALITY_Rouge1	SQuALITY_Rouge2	SQuALITY_RougeL	Qasper_f1	NarrativeQA_f1	Quality_acc	MuSiQue_f1	SpaceDigest_ExpSim	BookSumSort_ConcInd
2	2023-05-23T17:07:42.061Z	ZeroSCROLLS team	GPT-4	-	8K		41.66501	41.7662	19.3479	22.5926	34.4243	7.9888	18.8565	32.3128	9.4299	20.7663	39.6929	12.6275	22.977	50.7116	27.5857	89.2	41.1342	62.8097	60.4899
3	2023-05-23T17:05:23.710Z	ZeroSCROLLS team	ChatGPT	-	4K		34.02411	33.4908	15.0795	19.1436	33.234	6.9554	17.8912	27.2552	7.4657	18.6268	35.6361	11.2324	21.0992	49.3094	25.0742	66.6	27.077	49.0874	49.7825
4	2023-05-23T17:04:43.961Z	ZeroSCROLLS team	DaVinci003	-	4K		33.73674	33.5889	15.9089	19.2402	33.2322	7.0026	18.0886	30.0567	8.0141	20.0493	38.5122	12.0063	22.9051	52.7489	24.5841	69	33.5371	31.2693	49.4734
5	2023-05-23T17:02:28.732Z	ZeroSCROLLS team	Flan-UL2	20B	8K		30.62439	23.7019	11.8375	14.8508	22.7339	4.6623	14.1823	21.9859	7.0488	16.3747	8.6734	3.2012	6.594	56.9339	25.5155	75.6	51.3241	35.9617	14.0437
6	2023-05-23T17:01:33.421Z	ZeroSCROLLS team	Flan-T5	11B	8k		29.89511	26.958	12.6343	15.9619	16.285	2.7791	10.4492	18.256	5.3165	13.6798	12.4766	4.33	9.3402	48.2806	19.2724	75.2	46.7896	48.6789	16.4029
7	2023-05-23T17:06:05.100Z	ZeroSCROLLS team	Claude	-	8K		39.07266	43.772	15.7431	20.5217	32.9474	7.2081	17.5913	26.2603	7.1919	16.4583	40.6497	10.8055	21.2336	52.3422	32.6056	84.8	36.1042	61.5765	47.3664
8	2023-05-23T17:16:31.952Z	ZeroSCROLLS team	Naive	-	-		19.64396	45.0786	14.0685	18.147	19.8182	1.5308	10.0241	15.2521	1.9124	10.4418	25.5148	3.3702	13.5912	6.1266	2.0873	26.6	20	45.0346	50.0275
9	2023-05-23T17:00:13.504Z	ZeroSCROLLS team	T0pp	11B	8K		14.34186	10.9925	4.6656	6.9177	20.6231	3.2202	13.3149	12.7997	2.8657	9.9924	6.2021	1.888	5.0249	25.0432	18.7238	21.4	35.2844	15.2437	0
10	2023-08-01T23:45:51.434Z	Meta AI	230801_long_llama_28				34.37697	44.2191	18.7871	22.409	29.765	6.3247	16.6337	30.5514	9.7574	20.4121	40.059	12.363	22.8087	52.121	31.6534	82.4	25.4634	29.2445	41.0602
11	2023-08-01T23:45:20.219Z	Meta AI	230801_long_llama_all_data				39.46678	45.4818	19.2556	22.7729	29.4932	6.0866	16.3351	30.7573	10.1945	20.7751	39.3772	12.0587	22.7146	55.8243	30.9688	82.4	44.5349	51.0677	47.6694
12	2023-08-31T19:22:37.932Z	yuzhenm	Stable Beluga 7B		4K		23.01201	20.2401	8.4454	12.8357	29.2081	5.3157	16.8548	27.1294	6.3731	18.1355	30.8327	9.3973	19.4823	28.309	15.89	51.8	18.547	46.8685	9.489
13	2023-09-01T17:38:08.452Z	yuzhenm	Stable Beluga 13B	13B	4K		16.84367	12.2363	2.0009	8.7662	20.2589	1.5537	12.8384	24.3095	5.0617	17.1546	23.4151	6.3237	15.9287	19.9816	13.4009	47.8	25.0101	14.8089	7.9159
14	2023-10-25T21:33:25.076Z	Tam Doan	GPT4				2.29318	42.3102	14.5388	19.604	0	0	0	0.0237	0	0.0237	0	0	0	0	0	0	0	0	0
15	2024-03-31T17:14:14.622Z	1	1	1	1		14.74492	16.3061	6.9179	10.7582	29.6895	4.9936	16.3359	29.2006	7.3387	18.5862	33.8732	10.2593	20.3588	21.7472	18.7079	39	8.8496	0	0
16	2024-03-31T17:50:48.553Z	1	llama2chat_bestbase_1e-4_7top3_bs8_ratio1_gate_v3_3-pretrain-4	1	1		14.99151	16.1183	6.8099	10.7458	29.3361	4.7136	16.0122	29.3885	7.0239	18.5341	33.643	10.0237	20.0005	23.1276	17.7084	40	10.9456	0	0
17	2024-04-02T05:05:39.559Z	1	por_ms	1	1		20.45421	36.9524	14.628	19.0774	29.3987	5.4627	15.6495	28.426	7.1228	18.2343	39.3537	10.928	21.3831	21.1802	17.7258	47.8	9.2569	33.3641	3.4467
18	2024-04-03T01:33:22.910Z	1	3-4-open	1	4k		19.25568	39.2362	15.0219	19.5847	29.3796	5.635	15.5919	28.4604	6.9933	18.2295	39.4707	11.103	21.3627	24.0965	17.5762	43.8	8.1226	24.4415	1.76
19	2024-04-19T14:42:00.805Z	zwang	llama2_H2O_final				19.40595	27.2319	8.9947	14.8782	28.7071	5.0822	15.9346	26.6351	6.2691	17.6135	33.8126	9.26	19.6661	20.4529	14.9801	43.2	9.4765	40.8278	3.8298
20	2024-04-25T19:47:57.387Z	zwang33	llama2_7B_chat_ours				18.95579	26.0263	9.2259	14.6372	25.4143	4.6072	14.5473	26.1061	6.4161	17.5807	32.2261	9.2334	19.4211	19.7186	15.0753	42.8	9.9236	39.048	3.5671
21	2024-08-03T01:39:04.631Z	1	llama2-7b-chat	1	4k	1	15.46953	28.0961	9.8315	15.8948	16.1115	2.5017	9.789	22.2666	5.1113	15.3148	29.0261	7.6556	17.2797	14.1188	10.2816	22.2	10.9268	44.0329	1.7322
22	2024-09-06T18:30:37.106Z	zecheng	simpo_llama3				29.8533	36.3763	13.8336	18.7749	30.0918	5.2336	15.7156	30.2032	7.9028	19.3434	32.6353	9.3804	19.4922	50.0584	24.8172	52.2	19.8511	46.0582	36.0914
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100