1 of 20

Accelerate progress with a journal research data policy framework

STM Society workshop, April 2020

Iain Hrynaszkiewicz, Publisher, Open Research, PLOS

2 of 20

Data availability makes research more reliable

  • Studies have found that unavailability of data and insufficient data curation reduce the repeatability / reproducibility (reliability) of research1,2

​

  • Journal policies that promote transparency support publication of more reliable research, and are the first, simple, logical thing a publisher can do to raise awareness of issues3�
  • Journal policies support STM year of research data objective 1: SHARE
    • Increase the number of journals with data policies and articles with Data Availability Statements (DAS)
      • DAS report if and where data supporting the results reported in a published article are available – including, where applicable, hyperlinks to publicly archived datasets analysed or generated during the study.

​

  1. Ioannidis JPA et al (2009). Nat Genet 41:149–155
  2. Hardwicke TE et al (2018) R Soc Open Sci 5:180448
  3. Hrynaszkiewicz I. (2019) In: Handbook of Experimental Pharmacology. https://doi.org/10.1007/164_2019_290

3 of 20

The research data policy landscape is evolving

�

More funding agencies (and institutions) are introducing research data policies, which publishers and journals are obliged to support

85 (~22%) funders mandate or encourage data sharing (up from ~50 funders in 2016).�Over 300 have no stated policy yet.

�Source: Springer Nature (2019)

Key: “OA” means “open access data”, “data sharing” or “data management”

4 of 20

Some research data policy early adopters

  • Requirements in some communities for more than 25 years (genetic sequences, protein structure)
  • Since 2014 PLOS required authors to make all data underlying the findings described in their manuscript available without restriction at the time of publication, with rare exceptions, and provide a “Data Availability Statement” (DAS)
  • Since 2011, some BMC (BioMed Central) journals either required a DAS or encouraged authors to provide a DAS
  • In 2015, all BMC journals (250+ journals) mandated a DAS for all articles

5 of 20

TOP guidelines and data transparency (2015)

Not implemented

Level I

Level II

Level III

Data transparency

Journal encourages data sharing or says nothing

Article states whether data are available and, if so, where to access them

Data must be posted to a trusted repository. Exceptions must be identified at article submission

Data must be posted to a trusted repository, and reported analyses will be reproduced independently prior to publication

Nosek B et al (2014) Transparency and openness promotion (TOP) guidelines. https://osf.io/vj54c/

6 of 20

“The evidence shows that the current research data policy ecosystem is in critical need of standardization and harmonization”

-- Naughton, L. & Kernohan, D., (2016). Making sense of journal research data policies. Insights. 29(1), pp.84–89. DOI: http://doi.org/10.1629/uksg.284

Full Policy

Partial Policy

No Policy

Data source: Linda Naughton, JISC Journal Research Data Policy Bank project presentation (n = 250)

Journal research data policies can be confusing

7 of 20

Since 2016, more stakeholders introduce policies

  • 2016: Springer Nature: 4 policy types
  • 2017: Elsevier: 5 policy options
  • 2017: Wiley: 3, latterly 4 policy types
  • 2018: Taylor & Francis: 4 policy types; BMJ Group: 3 policy tiers
  • Meanwhile, more community/society initiatives such as:
    • Findable, Accessible, Interoperable and Reusable (FAIR) Data principles
    • Coalition for Publishing Data in the Earth and Space Sciences (COPDESS), American Geophysical Union Enabling FAIR data

8 of 20

Challenges and opportunities

  • Multiple similar but non-identical and non-agreed standard policies
  • Mandatory and enforced policies are much more effective in ensuring data accessibility1
  • Mandatory and enforced policies are more costly to implement2
  • Different needs and expectations for data sharing in different research communities
  • Increasing support for data sharing from large publishers
  • Publisher and journal policies motivate researchers to share data3
  • Vines et al (2013) https://doi.org/10.1096/fj.12-218164
  • Grant & Hrynaszkiewicz, IJDC 2018 https://doi.org/10.2218/ijdc.v13i1.614
  • The State of Open Data Report 2019. figshare. Report. https://doi.org/10.6084/m9.figshare.9980783.v2

9 of 20

https://www.rd-alliance.org/groups/data-policy-standardisation-and-implementation

​

​

​

​

Iain Hrynaszkiewicz (PLOS), Natasha Simons (ANDS), Simon Goudie (Wiley), Azhar Hussain (Jisc), Rebecca Grant (Springer Nature)

�Formed in 2017, Group activities build on research carried by Jisc, ongoing activities of Australian Research Data Commons and work of journal publishers on data policy

Research Data Alliance (RDA) helps engage different stakeholders on shared problems

10 of 20

A research data policy framework for all publishers?

11 of 20

Methodology: open development of framework

  • 2017: Community calls to gather requirements from different stakeholders (researchers, publishers, funders, librarians, societies)
  • 2018: First public draft (v1.2) made available for comment
  • More than 30 comments received from more than 20 reviewers
  • 2018: Discussion and review at RDA Plenary meetings
  • Late 2018 - early 2019:
    • Revision of framework
    • Exploration of different presentation formats for tables
    • Draft Implementation requirements
    • Creation of policy templates
  • Jun 2019: Publication of preprint on figshare
  • Feb 2020: Published after peer review, CODATA Data Science Journal

​

12 of 20

Results: 14 policy features, 6 policy types (tiers)

Key:

○ = Information required

● = Information and action required

- = Not applicable

​

​

Hrynaszkiewicz, Iain; Simons, Natasha; Hussain, Azhar; Grant, Rebecca; Goudie, Simon (2020): Developing a research data policy framework for all journals and publishers. CODATA Data Science Journal. http://doi.org/10.5334/dsj-2020-017

13 of 20

Existing policies map to the framework

Key:

○ = Information required

● = Information and action required

- = Not applicable

​

​

E.g. Elsevier policy 4

E.g. Springer Nature policy 4,�Elsevier policy 5

E.g. Wiley policy 3, PLOS policy

E.g. Wiley policy 2, TOP level I

E.g. Springer Nature policy 1

E.g. Wiley policy 1, Taylor & Francis Basic policy

14 of 20

Example feature: Data repositories

Feature definition and evidence for its inclusion

Template text for journal information for authors

15 of 20

Policy summary: honest and clear requirements

Hrynaszkiewicz, Iain; Simons, Natasha; Hussain, Azhar; Goudie, Simon (2019): Developing a research data policy framework for all journals and publishers. figshare. Preprint. https://doi.org/10.6084/m9.figshare.8223365.v1

Makes clear when sharing is optional and when it is not, and when the journal and authors must perform certain actions

16 of 20

Considerations when choosing a policy

  • What are your objectives in implementing a policy?
    • Raising awareness and signalling importance of an issue (policies 1-2)
    • Increasing transparency about data availability and compliance (policy 3)
    • Increasing availability of data (policies 4, 5)
    • Increasing and verifying reusability of data (policy 6)
    • Keeping up with competitors; responding to community (it depends...)
  • What are the needs of the research discipline(s) you serve?
  • Higher policy tiers require greater effort (cost) to implement
    • Implementing a data availability statement for every article will likely increase manuscript processing time
    • Simple, unverified “Available on request” statements will be faster to implement than statements that link to data in a repository (Grant & Hrynaszkiewicz, IJDC 2018 https://doi.org/10.2218/ijdc.v13i1.614)

​

17 of 20

There are benefits to stronger policies

  • Several studies of specific disciplines have found that data sharing is associated with increased citations to papers
  • The largest study, of papers published in PLOS and BMC, found that linking to research data in a repository via the data availability statement was correlated with a 25% increase in citations

Piwowar & Vision (2013) https://doi.org/10.7717/peerj.175

18 of 20

Analysis of PLOS & BMC shows mandates work

Colavizza et al (2020) https://doi.org/10.1371/journal.pone.0230416

Mandatory data availability statements (DAS) introduced at PLOS in 2014, and at BMC in 2015; DAS mostly optional at BMC 2012-2015

19 of 20

Opportunities for review/ audit of established policies

  • Scientific Data journal (Springer Nature; policy 6 in this framework) added a new feature to support Data Management Plans (feature 14 in this framework) in 2019
  • PLOS also added DMP language to its policy in 2019
  • Several large publishers have indicated intention to drive up standards, particularly for data availability statements (DAS; policy 3 and above)

​

https://journals.plos.org/plosone/s/data-availability#loc-data-management-plans

https://www.nature.com/sdata/policies/data-policies

20 of 20

Thanks for listening

ihrynaszkiewicz@plos.org

rda-data-policy-standardisation-ig@rda-groups.org

​

For more information on the STM Research Data Year �https://www.stm-researchdata.org/ please contact Joris van Rossum, STM Research Data Director, at rossum@stm-assoc.org

�

​

​