ABCDEFGHIJKLMNOPQRSTUVWXYZAAABACADAEAF
1
SubmissionDateContributorsModelName#ParametersInputLengthDescriptionScrollsScoreGovReport_Rouge1GovReport_Rouge2GovReport_RougeLSummScreenFD_Rouge1SummScreenFD_Rouge2SummScreenFD_RougeLQMSUM_Rouge1QMSUM_Rouge2QMSUM_RougeLNarrativeQA_f1Qasper_f1ContractNLI_EMQuality_EMQualityHard_EM
2
2022-01-01T11:16:17.065Z
SCROLLS team
LED-base162M16K
LED baseline from the original SCROLLS paper, using 16384 input tokens
29.156741156.168826.558128.821324.2474.501315.42325.08226.734818.846918.516726.644971.544725.751925.3831
3
2022-01-01T11:15:26.776Z
SCROLLS team
BART-base139M1K
BART baseline from the original SCROLLS paper, using 1024 input tokens
29.0132653747.882518.624322.689227.20614.926316.707230.15978.659120.743715.441826.342177.427125.986825.8621
4
2022-01-01T11:19:16.357Z
SCROLLS team
BART-base (512)139M512
BART baseline from the original SCROLLS paper, using 512 input tokens
27.5829151245.59816.860321.753226.28875.110816.224129.53098.15120.077314.520424.689171.592526.832727.3946
5
2022-01-01T11:17:58.956Z
SCROLLS team
LED-base (1024)162M1K
LED baseline from the original SCROLLS paper, using 1024 input tokens
27.0594065640.879816.057123.097722.65543.550315.128224.61126.53218.994415.173124.395873.36226.550827.2031
6
2022-01-01T11:17:10.719Z
SCROLLS team
LED-base (4096)162M4K
LED baseline from the original SCROLLS paper, using 4096 input tokens
28.3008109252.452923.302926.808623.01424.128415.105326.57116.941219.943416.296124.950871.544726.597727.2989
7
2022-01-02T14:12:10.899Z
SCROLLS team
BART-base (256)139M256
BART baseline from the original SCROLLS paper, using 256 input tokens
26.3520710541.942214.195120.297924.54653.766915.290629.93378.313520.406113.97823.317469.775226.033825.7663
8
2022-01-01T11:20:26.412Z
SCROLLS team
Naive--
Naive baseline from the original SCROLLS paper
19.3492702145.266317.907720.842819.62691.792410.992114.24582.00459.27741.45193.44565.9525.2326.0536
9
2022-03-14T19:57:44.110Z
Google Research
UL220B2K37.873753.581426.135828.805532.86717.777219.358931.11748.468120.428524.172437.596988.713545.770740.7088
10
2022-08-27T07:02:21.690ZIvgi et al.,BART-large SLED406M16K
BART large SLED (c=256) with beam size 4
37.9957.526.327.435.28.719.434.2112224.146.987.334.834.8
11
2022-08-21T18:18:37.955ZMeta AIBART-LS460M16K39.7659.429.830.837.710.221.535.1112226.248.787.137.834
12
2023-02-28T22:50:01.572Z
Google Research
CoLT5 XL16K43.5161.332.233.836.410.121.736.212.924.231.153.988.448.143.8
13
2023-02-28T22:49:17.974Z
Google Research
CoLT5 Large16K41.0460.731.332.936.710.62234.911.523.127.749.888.739.936.8
14
2023-02-28T22:48:06.808Z
Google Research
CoLT5 Base16K37.6458.729.631.434.59.220.6329.32123.342.186.536.534
15
2023-03-07T04:44:08.834ZLongT5LongT5 Base220M16K
LongT5 Base model SCROLL performance. All tasks have max output length 512 tokens, except for GovRep which has 1024 tokens.
38.657.73031.434.89.621.133.91122.82346.685.637.936.6
16
2023-03-07T04:44:57.857ZLongT5LongT5 Large770M16K
LongT5 Large model SCROLLS performance. All tasks have max output length 512 tokens, except for GovRep which has 1024 tokens.
41.0360.331.132.835.69.221.235.11223.327.252.387.340.638.6
17
2023-03-07T04:42:02.675ZLongT5LongT5 XL3B16K
LongT5 XL model SCROLLS performance. All tasks have max output length 512 tokens except for GovRep which has 1024 tokens.
42.5361.132.333.735.89.621.134.911.823.529.353.188.24642.1
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100