1 of 54

ASAPbio Fellows program

Preprints - the landscape

Iratxe Puebla, Associate Director

23 June 2020

@ASAPbio_ | #ASAPbio | @IratxePuebla

2 of 54

Welcome - today we’ll cover

  • Brief history of preprints
  • The current landscape
    • Preprint use - per discipline, per country, COVID-19
    • Preprint servers
    • Building evidence-based conversations around preprints
  • Small group discussion
  • Anything to share regarding preprints?

​

​

​

​

​

​

@ASAPbio_ | #ASAPbio | @IratxePuebla

3 of 54

Information Exchange groups (IEGs)

Started in 1961 as ‘an ‘experiment’

​

3,600 researchers involved, over 2,500 papers circulated

​

Encountered resistance from journals

​

Closed in 1967 after a group of journals indicated they would not consider for publication papers circulated as an IEG memo

Credit: Cold Spring Harbor Laboratory Archive.

Cobb M (2017) The prehistory of biology preprints: A forgotten experiment from the 1960s. PLoS Biol 15(11): e2003995.

​

@ASAPbio_ | #ASAPbio | @IratxePuebla

4 of 54

arXiv

Started in 1991, founded by Paul Ginsparg

​

Covers physics, mathematics, computer science, quantitative biology, quantitative finance, statistics, electrical engineering & systems science, economics

​

Hosts over 1,700,000 preprints

​

Quantitative biology section since 2003 (1.2% of submissions in 2019)

​

​

Monthly submissions to arXiv since August 1991. https://arxiv.org/stats/monthly_submissions

​

​

@ASAPbio_ | #ASAPbio | @IratxePuebla

5 of 54

Initial steps in the life sciences

HAL archives ouvertes 2001

Nature Precedings 2007-2012

PeerJ Preprints 2013-2019

bioRxiv launched in November 2013

​

Potential factors *

Greater familiarity and adoption of digital tools

bioRxiv hosted by Cold Spring Harbor Laboratory

community driven

reputation in publishing

Social media fosters visibility

​

* Penfold NC, Polka JK (2020) Technical and social issues influencing the adoption of preprints in the life sciences. PLoS Genet 16(4): e1008565.

​

​

@ASAPbio_ | #ASAPbio | @IratxePuebla

6 of 54

Preprints in the life sciences

​

​

​

​

​

​

@ASAPbio_ | #ASAPbio | @IratxePuebla

7 of 54

Preprints in the life sciences

​

​

​

​

​

​

Simons Foundation

​

CrossRef indexes preprints

supports bioRxiv

preprints on

preprints on

PeerJ Preprints

bioRxiv

preprints.org

SciELO preprints

OSF preprints

medRxiv

Research Square

preprints on

@ASAPbio_ | #ASAPbio | @IratxePuebla

8 of 54

Preprints in the life sciences

​

​

​

​

​

​

https://asapbio.org/preprint-info/biology-preprints-over-time

@ASAPbio_ | #ASAPbio | @IratxePuebla

9 of 54

Preprint use varies per discipline

​

​

​

bioRxiv content distribution per discipline March 2020

@ASAPbio_ | #ASAPbio | @IratxePuebla

10 of 54

What about the clinical sciences?

​

​

​

medRxiv content distribution per discipline 15 June 2020 (subjects with > 50 preprints)

@ASAPbio_ | #ASAPbio | @IratxePuebla

11 of 54

Preprint use varies per country

Distribution of bioRxiv preprints per country, based on affiliation of corresponding author

Richard J. Abdill, Elizabeth M. Adamowicz, Ran Blekhman. bioRxiv 2020.04.25.060756

@ASAPbio_ | #ASAPbio | @IratxePuebla

12 of 54

Preprint use varies per country

Correlation per country between total citable documents (2014–2018, log scale), and preprints (log scale)

Red line demarcates a “bioRxiv adoption” score of 1.0

​

​

Richard J. Abdill, Elizabeth M. Adamowicz, Ran Blekhman. bioRxiv 2020.04.25.060756

@ASAPbio_ | #ASAPbio | @IratxePuebla

13 of 54

Preprints in COVID-19

Jonny Coates & Gautam Dey

​

jc2216@cam.ac.uk

@JACoates91

jacoates.co.uk

g.dey@ucl.ac.uk

@Dey_Gautam

https://www.embl.de/research/units/cbb/dey/index.html

@ASAPbio_ | #ASAPbio | @IratxePuebla

14 of 54

Preprints were almost poised for a pandemic…

  • Open access
  • No editorial/reviewer bias
  • No journal-specific formatting requirements
  • Screening takes ~1 day

@ASAPbio_ | #ASAPbio | @IratxePuebla

15 of 54

The scientific community has rapidly responded to the pandemic

@ASAPbio_ | #ASAPbio | @IratxePuebla

16 of 54

COVID-19 preprints are downloaded significantly more than non-COVID-19 preprints across servers

~15x more than non-COVID-19 preprints

@ASAPbio_ | #ASAPbio | @IratxePuebla

17 of 54

@ASAPbio_ | #ASAPbio | @IratxePuebla

18 of 54

@ASAPbio_ | #ASAPbio | @IratxePuebla

19 of 54

@ASAPbio_ | #ASAPbio | @IratxePuebla

20 of 54

@ASAPbio_ | #ASAPbio | @IratxePuebla

21 of 54

@ASAPbio_ | #ASAPbio | @IratxePuebla

22 of 54

@ASAPbio_ | #ASAPbio | @IratxePuebla

23 of 54

@ASAPbio_ | #ASAPbio | @IratxePuebla

24 of 54

@ASAPbio_ | #ASAPbio | @IratxePuebla

25 of 54

@ASAPbio_ | #ASAPbio | @IratxePuebla

26 of 54

​

Preprint servers

Image compiled by Jeroen Bosman (@jeroenbosman) via Bianca Kramer (@MsPhelps)

Currently 44 platforms with biomedical or medical scope

@ASAPbio_ | #ASAPbio | @IratxePuebla

27 of 54

Directory of preprint servers

Directory of preprint servers

Information about preprint scope, policies and practice

https://asapbio.org/preprint-servers

​

A systematic examination of preprint platforms for use in the medical and biomedical sciences setting. Jamie J Kirkham, Naomi Penfold, Fiona Murphy, Isabelle Boutron, John PA Ioannidis, Jessica K Polka, David Moher. bioRxiv 2020.04.27.063578

​

​

​

​

​

@ASAPbio_ | #ASAPbio | @IratxePuebla

28 of 54

SciELO PREPRINTS

​

​

​

Alex Mendonça�alex.mendonca@scielo.org

Online Submission & Preprints Coordinator

SciELO Brazil

29 of 54

ABOUT SciELO

SciELO is an international cooperation program for the advancement of research communication implemented via a network of nationally operated collections of peer review journals published in Open Access.

​

Research infrastructure programs at national level

Ex. SciELO / FAPESP Program – SciELO Brazil

​

Specific objectives – to maximize the availability, visibility, use, impact and credibility of independently edited journals and the research they communicate to improve the quality of the journals to complement international bibliographic – bibliometric indexes

​

Multilingual and coverage of all disciplines.

30 of 54

SciELO PREPRINTS, A BRIEF INTRODUCTION

  • Announced in 2017.
  • ASAPBio is the reference.
  • Opportunity to collaborate with Public Knowledge Project (PKP) on the development of Open Preprints Systems (OPS), in 2018.
  • Multiliguism was a key determinant.
  • OPS is a stripped-down version of Open Journal System (OJS) and contains features specifically designed for preprint servers.
  • Pilot version launched in April 7, 2020.
  • Prioritizing preprints related to COVID-19.

​

31 of 54

SciELO PREPRINTS, TWO MONTHS LATER

  • 480.000+ total views
  • Preprint with most views: 16.000+ 
  • 559 submissions received (as of June 23, 2020)
    • 217 rejected (39% rejection rate)
    • 338 preprints posted

​

  • SciELO Preprints becoming an integral part of the SciELO Publishing Workflow.

32 of 54

SciELO PREPRINTS, THE MODERATION APPROACH

​

​

​

When it comes to preprints, building trust is crucial

​

33 of 54

SciELO PREPRINTS, THE MODERATION APPROACH

  • Golden Rule: submissions must be eligible to undergo peer review process by a typical SciELO journal.

​

  • At least one of the authors must have authenticated ORCID ID.
  • Submitting Authors are requested to inform up to three articles of any date or two recent articles of the last two years with active DOIs in the Crossref repository.
  • In other cases, a basic screening will be done to identify if it is a research related manuscript.

​

​

34 of 54

SciELO PREPRINTS, THE MODERATION APPROACH

​

​

Use of custom OPS plugins to provide visual feedback for authors and moderators regarding submission compliance:

​

​

An example of submission that meets all requirements for pre-approval

​

​

An example of submission that needs attention

​

​

35 of 54

SciELO PREPRINTS, THE MODERATION APPROACH

​

​

  • Failing to pass the initial screening does not mean preprint will be rejected;
  • Same way as passing the initial screening alone does not guarantee immediate posting of a preprint.

​

  • The plugin helps moderation but does not determine it.

​

36 of 54

SciELO PREPRINTS, THE MODERATION APPROACH

​

​

  • SciELO Preprints is a multilingual, multi-area server
  • SciELO Journal editors as area moderators
    • Members of the SciELO Brazil Collection’s Advisory Committee
    • Part of SciELO’s strategy of bringing journals close to preprints and seeing preprints as their allies
    • Building trust
  • Golden Rule: submissions must be eligible to undergo peer review process by a typical SciELO journal.

​

​

​

37 of 54

SciELO PREPRINTS, INTERACTIVE

Annotations via Hypothesis now available!

​

​

38 of 54

SciELO PREPRINTS, THE CHALLENGES

  • Metadata curatorship – authors don’t always fill metadata fields correctly, which may result in poor indexing (reduces discoverability);
  • Authors understanding of what is a preprint and server editorial policies;
  • Maintaining a short turnaround time (preprints are supposed to become available much faster) facing the high volume of manuscripts;
  • Limitations of working with a platform that is still in beta (not all functionalities are available yet).

​

  • Consolidation of preprints and automatic validation tools can help overcoming these challenges.

​

39 of 54

SciELO PREPRINTS, THE FUTURE

  • Other plugins and developments currently in the works:
    • PDF Title Page (submission checklist, submission date, DOI, preprint status);
    • Similarity Check (adapt existing iThenticate plugin to allow checking of only selected manuscripts);
    • Content Analysis (indicate presence of Conflict of Interest, Authors’ contributions etc)
    • Custom acceptance criteria according to area of knowledge.
    • Altmetrics
    • Preprint transfer to Open Journal Systems and other journal platforms (via SWORD protocol)
    • and more!

​

40 of 54

THANK YOU!

​

​

​

Alex Mendonça�alex.mendonca@scielo.org

Online Submission & Preprints Coordinator

SciELO Brazil

41 of 54

Building evidence-based conversations around preprint use

(Poll)

@ASAPbio_ | #ASAPbio | @IratxePuebla

42 of 54

‘My work will be scooped’

There is no evidence that preprints increase risk for scooping

​

Paul Ginsparg, founder of arXiv on scooping: “It can’t happen, since arXiv postings are accepted as date-stamped priority claims.”

Resources on scooping available on ASAPbio website: https://asapbio.org/preprint-info/preprint-faq A number of journals operate scooping protection policies: EMBO, eLife, PLOS journals

​

​

42

bioRxiv survey N=3127

‘bioRxiv: the preprint server for biology’ https://doi.org/10.1101/833400

​

@ASAPbio_ | #ASAPbio | @IratxePuebla

43 of 54

‘The journal will not publish my work’

Abdill & Blekhman; eLife 2019;8:e45133

⅔ of preprints are published within two years

​

The study by Addill & Blekhman focused on preprints in bioRxiv, the same statistic has been reported for preprints in arXiv (Larivière et al. Journal of the American Society for Information Science and Technology, 65(6): 1157–1169)

@ASAPbio_ | #ASAPbio | @IratxePuebla

44 of 54

‘The journal will not publish my work’

  • SHERPA/RoMEO lists over 1,200 publishers with policies that accept preprints
  • TRANSPOSE database (https://transpose-publishing.github.io/#/) provides information on preprint policies at journals
  • Some journals give the authors the option to post the paper at a partner preprint platform in parallel to consideration at the journal
  • Some journals have dedicated editors who check preprints to invite submission to the journal (see https://asapbio.org/journal-policies for more info on innovative journal practices)

​

@ASAPbio_ | #ASAPbio | @IratxePuebla

45 of 54

‘Preprints will lead to a deluge of poor papers’

​

​

​

​

​

​

​

Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature. Carneiro et al. bioRxiv 581892; doi: https://doi.org/10.1101/581892

​

​

Comparison of random sample (76) of bioRxiv preprints to peer-reviewed articles from PubMed, and a paired comparison of a sample (43) of bioRxiv preprints to their own peer-reviewed article versions

Peer-reviewed articles have higher quality of reporting than preprints, but the difference is small

​

5.0 % in independent sample

4.7 % in paired sample comparison

@ASAPbio_ | #ASAPbio | @IratxePuebla

46 of 54

‘Without peer review, there is a risk to public health’

Theo Bloom, presentation at FORCE2019 http://bit.ly/preprints-FORCE2019

Risk mitigation framework - medRxiv

​

Is it nonsense?

Is it non-science?

Is it a paper?

Is it research?

Is it plagiarized?

Is it a health threat?

Is there a benefit to sharing now vs. after peer review?

​

@ASAPbio_ | #ASAPbio | @IratxePuebla

47 of 54

‘Without peer review, there is a risk to public health’

medRxiv requires declarations in line with those required for reporting of clinical work in peer-reviewed literature:

  • Competing interests
  • Funding statement
  • Ethical approval/consent
  • Clinical trial registration

​

As well as data statements (beyond what some journals operate)

​

​

​

​

​

@ASAPbio_ | #ASAPbio | @IratxePuebla

48 of 54

‘What’s in it for me?’

A number of funders encourage preprints as evidence of productivity in grant applications & reports

List and links to policies at asapbio.org/funder-policies

@ASAPbio_ | #ASAPbio | @IratxePuebla

49 of 54

‘What’s in it for me?’

​

​

​

​

​

​

​

​

Fu and Hughey. eLife 2019;8:e52646. DOI: https://doi.org/10.7554/eLife.52646

​

​

​

Having a preprint on bioRxiv is associated with a higher Altmetric Attention Score and more citations of the peer-reviewed article

​

@ASAPbio_ | #ASAPbio | @IratxePuebla

50 of 54

‘What’s in it for me?’

​

​

​

​

​

​

​

The effect of bioRxiv preprints on citations and altmetrics. Nicholas Fraser, Fakhri Momeni, Philipp Mayr, Isabella Peters. bioRxiv 673665; doi: https://doi.org/10.1101/673665

​

​

​

Papers posted to bioRxiv receive citations prior to journal publication

​

Preprints can extend the reach of the work

​

@ASAPbio_ | #ASAPbio | @IratxePuebla

51 of 54

Breakout groups

Discuss the information and evidence presented

What do you find particularly useful?

What are the evidence gaps around preprint use & practice?

Any other data, evidence or materials that you are aware of?

​

​

@ASAPbio_ | #ASAPbio | @IratxePuebla

52 of 54

Project recap

​

  • Organize a preprint panel/event
  • Organize a preprint journal club
  • Organize panel or blog/resources for clinical community
  • Create preprint-related infographics
  • Training course - the scientific process and where preprints fit
  • What will be the future of Preprints vs. ‘classical’ publishing

@ASAPbio_ | #ASAPbio | @IratxePuebla

53 of 54

Information/events of interest

  • Survey on perceived benefits and concerns about preprints: https://forms.gle/br2tsYjYw8EJTEfm6

​

​

​

Next session

21 July 2020 ‘Engaging with preprints’

​

​

@ASAPbio_ | #ASAPbio | @IratxePuebla

54 of 54

References

  • Cobb M (2017) The prehistory of biology preprints: A forgotten experiment from the 1960s. PLoS Biol 15(11): e2003995
  • Penfold NC, Polka JK (2020) Technical and social issues influencing the adoption of preprints in the life sciences. PLoS Genet 16(4): e1008565.
  • Richard J. Abdill, Elizabeth M. Adamowicz, Ran Blekhman. International authorship and collaboration in bioRxiv preprints. bioRxiv 2020.04.25.060756; doi: https://doi.org/10.1101/2020.04.25.060756
  • Preprinting a pandemic: the role of preprints in the COVID-19 pandemic. Nicholas Fraser, Liam Brierley, Gautam Dey, Jessica K Polka, Máté Pálfy, Jonathon Alexis Coates. bioRxiv 2020.05.22.111294; doi: https://www.biorxiv.org/content/10.1101/2020.05.22.111294v1
  • A systematic examination of preprint platforms for use in the medical and biomedical sciences setting. Jamie J Kirkham, Naomi Penfold, Fiona Murphy, Isabelle Boutron, John PA Ioannidis, Jessica K Polka, David Moher. bioRxiv 2020.04.27.063578; doi: https://doi.org/10.1101/2020.04.27.063578
  • Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature. Clarissa F. D. Carneiro, Victor G. S. Queiroz, Thiago C. Moulin, Carlos A. M. Carvalho, Clarissa B. Haas, Danielle Rayêe, David E. Henshall, Evandro A. De-Souza, Felippe E.Amorim, Flávia Z. Boos, Gerson D. Guercio, Igor R. Costa, Karina L. Hajdu, Lieve van Egmond, Martin Modrák, Pedro B. Tan, Richard J. Abdill, Steven J. Burgess, Sylvia F. S.Guerra, Vanessa T. Bortoluzzi, Olavo B. Amaral. bioRxiv 581892; doi: https://doi.org/10.1101/581892
  • Fu and Hughey. eLife 2019;8:e52646. DOI: https://doi.org/10.7554/eLife.52646
  • The effect of bioRxiv preprints on citations and altmetrics. Nicholas Fraser, Fakhri Momeni, Philipp Mayr, Isabella Peters. bioRxiv 673665; doi: https://doi.org/10.1101/673665
  • bioRxiv: the preprint server for biology. Richard Sever, Ted Roeder, Samantha Hindle, Linda Sussman, Kevin-John Black, Janet Argentine, Wayne Manos, John R. Inglis. bioRxiv 833400; doi: https://doi.org/10.1101/833400
  • Abdill RJ, Blekhman R. Tracking the popularity and outcomes of all bioRxiv preprints. Elife. 2019;8:e45133. Published 2019 Apr 24. doi:10.7554/eLife.45133
  • Larivière, V., Sugimoto, C.R., Macaluso, B., Milojević, S., Cronin, B., Thelwall, M. (2014). arXiv e-prints and the journal of record: An analysis of roles and relationships. Journal of the American Society for Information Science and Technology, 65(6): 1157–1169
  • Bourne PE, Polka JK, Vale RD, Kiley R (2017) Ten simple rules to consider regarding preprint submission. PLoS Comput Biol 13(5): e1005473. https://doi.org/10.1371/journal.pcbi.100547
  • Serghiou S, Ioannidis JPA. Altmetric Scores, Citations, and Publication of Studies Posted as Preprints. JAMA.2018;319(4):402–404. doi:10.1001/jama.2017.21168

​

​

​

​

​

​

​

​

@ASAPbio_ | #ASAPbio | @IratxePuebla