Preprint review metadata for ranks and scores

	A	B	C	D	E
1	Review project	Rank system (create new row for each dimension)	Values	How represented in metadata?	Notes/source

2	Biophysics Colab	Strength of evidence	Inadequate Convincing Exceptional	Not yet in metadata, but could be	This is a working scheme; it has yet to be confirmed. It may also expand over time.
3	eLife	significance of findings	Landmark: findings with profound implications that are expected to have widespread influence Fundamental: findings that substantially advance our understanding of major research questions Important: findings that have theoretical or practical implications beyond a single subfield Valuable: findings that have theoretical or practical implications for a subfield Useful: findings that have focused importance and scope	Not currently in metadata for reviewed preprints but we are considering how best to do this. Captured (solely) as keywords in JATS for VORs.	https://elifesciences.org/inside-elife/db24dd46/elife-s-new-model-what-is-an-elife-assessment
4	eLife	strength of support	Exceptional: exemplary use of existing approaches that establish new standards for a field Compelling: evidence that features methods, data and analyses more rigorous than the current state-of-the-art Convincing: appropriate and validated methodology in line with current state-of-the-art Solid: methods, data and analyses broadly support the claims with only minor weaknesses Incomplete: main claims are only partially supported Inadequate: methods, data and analyses do not support the primary claims	Not currently in metadata for reviewed preprints but we are considering how best to do this. Captured (solely) as keywords in JATS for VORs.	https://elifesciences.org/inside-elife/db24dd46/elife-s-new-model-what-is-an-elife-assessment
5	F1000 Research	Reviewer recommendation	Approved Approved with reservations Not approved	Overall peer review status is in square brackets at the end of the title e.g.: "I am the article title [version 1; peer review: 2 approved]". In the metadata, that overall status is included as a footnote on the title: <fn-group content-type="pub-status"><fn><p>[version 1; peer review: 2 approved]</p></fn></fn-group>. The Full Text of peer review reports is also included in the JATS XML - each report is included as a <sub-article> with the attribute: article-type="ref-report". The status of the report is included in the <sub-article> as a <custom-meta-group>, e.g.: <custom-meta-group><custom-meta><meta-name>recommendation</meta-name><meta-value>approve</meta-value></custom-meta></custom-meta-group>.	We’re also curently looking at capturing more structured information around the prepublication checks we do and would be interested in exploring consistent ways of capturing those in the metadata
6	Peer Community in	Editorial acceptance (recommendation) by a recommender	Accepted = Recommended	By virtue of having public reviews on Peer Community In
7	PreLights	[Recommendation by service/platform]	N/A	By virtue of being covered by PreLights
8	PREreview structured review	Does the introduction explain the objective and match the rest of the preprint?	Likert 1-3	Whole review is tagged as a "Rapid PREreview" in the metadata to aggregate anlyses with previous version called "Rapid PREreviews"
9	PREreview structured review	Are the methods appropriate?	Likert 1-5	Whole review is tagged as a "Rapid PREreview" in the metadata to aggregate anlyses with previous version called "Rapid PREreviews"
10	PREreview structured review	Are the results presented supported by the data?	Likert 1-5	Whole review is tagged as a "Rapid PREreview" in the metadata to aggregate anlyses with previous version called "Rapid PREreviews"
11	PREreview structured review	Are the data presentations, including visualizations, appropriate and clear?	Likert 1-5	Whole review is tagged as a "Rapid PREreview" in the metadata to aggregate anlyses with previous version called "Rapid PREreviews"
12	PREreview structured review	How well do the authors discuss, explain, and interpret their findings and potential next steps for the research?	Likert 1-5	Whole review is tagged as a "Rapid PREreview" in the metadata to aggregate anlyses with previous version called "Rapid PREreviews"
13	PREreview structured review	Are the findings novel?	Likert 1-5	Whole review is tagged as a "Rapid PREreview" in the metadata to aggregate anlyses with previous version called "Rapid PREreviews"
14	PREreview structured review	Would it benefit from language editing?	Yes/No	Whole review is tagged as a "Rapid PREreview" in the metadata to aggregate anlyses with previous version called "Rapid PREreviews"
15	PREreview structured review	Should others read this preprint?	Yes/Yes with improvment/No	Whole review is tagged as a "Rapid PREreview" in the metadata to aggregate anlyses with previous version called "Rapid PREreviews"
16	PREreview structured review	Is it ready for a full and detailed review?	Yes/Yes with improvement/No	Whole review is tagged as a "Rapid PREreview" in the metadata to aggregate anlyses with previous version called "Rapid PREreviews"
17	RR\ID	Strength of Evidence	📕 ◻️◻️◻️◻️ = Misleading 📙📙 ◻️◻️◻️ = Not Informative 📒📒📒 ◻️◻️ = Potentially Informative 📗📗📗📗◻️ = Reliable 📘📘📘📘📘 = Strong	Not currently in metadata, but considering how to implement
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100