“Open wide: Open Research Training across the disciplines”
These slides:
Abstract
What I promised:
The UK Reproducibility Network (UKRN) is a national peer-led consortium that aims to ensure the UK retains its place as a centre for world-leading research. The Open Research Programme is a five-year programme supported by Research England, UKRN institutional members and project partners which aims to raise the standard of open research practices across the UK research ecosystem. I will talk about the first of three programme work packages: Training. https://www.ukrn.org/open-research-programme/
What I’ll also speak to:
Responses to open research from different disciplines
These slides:
UKRN & ORP
These slides:
Dominic Cummings, giving evidence at the Health and Social Care Committee and Science and Technology Committee 2021-05-26
“..in this field of behavioural science, there are a lot of charlatans...the whole field is riddled with duff studies and memes that people believe are true but are not true.”
UKRN
“a national peer-led consortium that aims to ensure the UK retains its place as a centre for world-leading research”
Image: UKRN institutional membership 2022,Adam Kenny
These slides:
UKRN/REDF project: ‘Growing and Embedding Open Research in Institutional Practice and Culture’
5 year, £8.5m Project
3 workstreams:
Adam is Open Research Coordinator and Administrator (ORCA) for University of Sheffield. Every institutional partner should have an ORCA!
Adam Partridge,
Open Research Training Lead,
Research Services
These slides:
Open Research Programme scope
“to accelerate the uptake of high-quality open research practices”
Year 1 report : Jim Grange & Hannah De’Bell
These slides:
Findings
- There is a large diversity in the amount of open research training resources available between different UKRN member institutions.
- Support for different aspects of open research practice is also variable, with the “planning” and “analysing” stages of the research lifecycle better supported than the “conducting” or “disseminating” stages.
- The institutions surveyed perceived the majority of open research practice areas they were asked about as valuable, and, where they were available, training resources on open research were perceived to be high quality.
- Only 10.5% of total resources are open-access, raising the costs for each individual institution to create training resources.
- Training provision is usually delivered “in house” by research support or library services.
Components
Schema (Sarah Callaghan)
Endorsement (Joe Cornelli, David Foxcroft)
Accreditation (Steve Boneham)
Delivery (Adam Partridge)
These slides:
Schema (Sarah Callaghan)
These slides:
Endorsement (Joe Cornelli & David Foxcroft)
Evaluation, feedback and a commitment to act on it
These slides:
Accreditation (Steve Boneham)
Badges, “agreed expectations”, communities of practice
These slides:
Delivery (Adam Partridge)
External partners: Including… The Software Sustainability Institute (UK), VIRT2UE (Netherlands) and Griffith University (Australia), TIER, COS
And supporting “home grown” training and use of open resources from other institutions.
Data management
These slides:
Dependencies / vision
The year is 2026, the Open Research Programme is coming to the end of its initial funding. The programme represents a gold standard for training in open research practices, with a model which is internationally admired and emulated. Leaders at UKRN member institutions use the UKRN schema to identify training areas the programme can address and are confident of the standards that the programme holds for its training. External training providers compete to be able to say they are official partners. Within institutions, local contacts - ORCAs - know they can rely on the programme to facilitate training for staff and value the network/community of practice which the UKRN programme provides for them and staff responsible for training in their institution. Staff involved in training within institutions are proud of the accreditation offered by the programme and are able to use it for career progression. The programme fosters locally excellent practice in open research training, with groups who offer the most comprehensive training on particular topics joining the programme to train trainers at other UKRN institutions. The programme records hundreds of trainers who have taken part in the programme and they have reported, in turn, thousands of researchers who have been trained in open research practices. The UKRN and UKRN partners are able to share best-practice training resources across all areas of open research (both via public facing platforms like the UKRN’s living website and informally via community of practice forums). The programme supports the creation of new, openly licensed materials, in areas where there is a gap. Across the UK, standards for open research are elevated, driving greater consistency in research practice, wider adoption of better standards and greater engagement with developing and defining best practice, which continues to evolve and be reflected in the trainings provided by the programme.
These slides:
Aims
Support institutional, academic led training on open research
Generate materials
Raise engagement with open research across UK
Work to develop best/better practice
Reach underrepresented disciplines in open research
Criticisms of Open Research
These slides:
TUoS Open Research Statement
“The University of Sheffield recognises that Open Research is a vital part of our commitment to enabling and promoting research excellence across our community.”
These slides:
Image: University of Sheffield
These slides:
Extra effort
personally costly?
unnecessary?
distracting?
Image: Sisyphus, 1548 by Titian via titian.org
Data positivism
Also, confidentiality risks
Image: AI generated, via stablediffusionweb.com.
Prompt: “a malevolent tower overlooks a barren wasteland”
Exclusive and performative
Alex McLean ‘Performing openness in academic publishing’ https://slab.org/2022/12/08/performing-openness/
Image: AI generated, via stablediffusionweb.com.
Prompt: “smug scientists sit in preening superiority on thrones of diamond”
These slides:
Privilege entrenching
Image: AI generated, via stablediffusionweb.com.
Prompt: “slaves building the pyramids in impressionist style”
Smith, A. C., Merz, L., Borden, J. B., Gulick, C. K., Kshirsagar, A. R., & Bruna, E. M. (2022). Assessing the effect of article processing charges on the geographic diversity of authors using Elsevier’s “Mirror Journal” system. Quantitative Science Studies, 2(4), 1123-1143.
APC (“Gold”)
APC (hybrid)
free (hybrid)
Mirror journals with OA/APC and closed/no APC routes shows more authors from low and middle income countries published by non-OA route
"Our results for Elsevier’s Mirror-Parent system are consistent with the hypothesis that APCs are a barrier to OA publication for scientists from the Global South."
These slides:
Olejniczak, A. J., & Wilson, M. J. (2020). Who’s writing open access (OA) articles? Characteristics of OA authors at Ph. D.-granting institutions in the United States. Quantitative science studies, 1(4), 1429-1450.
Across 182,320 US researchers, publishing APC-funded OA articles was associated with being a man, at a prestigious institution, being in STEM, having more funding, and at a more senior career stage.
"Participation in APC OA publishing appears to be skewed toward scholars with greater access to resources and job security."
Image CC BY-SA 3.0, Wikimedia commons
These slides:
More on critiques of open research
Ross-Hellauer, T. (2022). Open science, done wrong, will compound inequities. Nature, 603(7901), 363-363.
Lancaster, A. (2016). Open science and its discontents. Ronin Institute.
Naomi Truan (2021). Dealing With Being Exposed: Setting Boundaries While Being Open
Mirowski, P. (2018). The future (s) of open science. Social studies of science, 48(2), 171-203.
Callard F. (2022). Replication and reproduction: Crises in psychology and academic labour. Review of General Psychology, 26, 199–211.
Uygun-Tunç, D., Tunç, M. N., & Eper, B. (2022). Is Open Science Neoliberal?. Perspectives on Psychological Science, 20(X). https://doi.org/10.1177/17456916221114835
Levin, N., & Leonelli, S. (2017). How does one “open” science? Questions of value in biological research. Science, Technology, & Human Values, 42(2), 280-305.
Peterson D., Panofsky A. (2021). Arguments against efficiency in science. Social Science Information, 60(3), 350–355. https://doi.org/10.1177/05390184211021383
These slides:
Synthesis
These slides:
Process quality > output quality
Reproducibility …?
More repeatable
More reusable, extendable
More translatable
Better documented
Better archived
More secure
These slides:
Principles vs practices, an example
PRACTICE: preregistration
publicly archived trial design, outcome measures and statistical models
EXCEPTIONS
exploratory research, feasibility/pilot studies,
PRINCIPLE: avoid (self) deception in results narration
formal record of what you intended when you started
alertness to researcher bias
These slides:
Principles vs practices, example II
PRACTICE: open data
public, permanent, archive of study data
EXCEPTIONS:
commercially or clinically sensitive data, DOUS (data of unusual size), lack of consent
PRINCIPLE: reproducibility, audit, reuse
share post-processed data at earlier possible point
share metadata for discovery, synthesis
These slides:
Onwards
These slides:
“Training researchers in research integrity and the need to ensure reproducibility is inconsistent and often absent. We recommend mandating the provision of such training at undergraduate, postgraduate and early career researcher stages.”
https://publications.parliament.uk/pa/cm5803/cmselect/cmsctech/101/summary.html
These slides:
EVENT: 2023-05-18
Recording coming soon:�https://www.youtube.com/channel/UC_gar6y864eLGfX7zJSWd5w
These slides:
Teaching Reproducible Research and Open Science’, 20-22 June 2023, Sheffield.
Dr Aneta Piekut
Organisers: University of Sheffield (Sheffield Methods Institute, Open Research Working Group and University Library) and Project TIER.
This three day event consists of a one-day symposium with conference style presentations, a workshop on teaching transparent methods of empirical research, and individual or small group in-person consultations with Project TIER Directors.
https://www.sheffield.ac.uk/smi/events/teaching-reproducible-research-and-open-science-conference
These slides:
END
These slides:
END
These slides:
Reserve slides follow
Tour of Depts
Image: Stable Diffusion (via huggingface.co),
prompt “ancient foundations building towards a sky of stars”
Open Research Tour (of Departments…and Flagships?)
1. Meet with DRIs
- use open research aspects of REF environment statements / 5 year strategy plans to structure discussion on enhancing open research
2. Provide researchers with updates on TUoS activities and opportunities
3. Hear from researchers about barriers to research improvement in their work
These slides:
Collectively we already know what to do
but this knowledge is
- under-shared
- under-evidenced
- not packaged for dissemination
- not clearly distinguished from suboptimal practices
investigating the factors that contribute to robust research
disseminate
best practice,
support training
evaluate
implement
These slides:
Research Improvement Strategy
This slide stolen from Malcolm MacLeod
A workable strategy
Add one new practice per project
These slides:
“Made in Sheffield”
These slides: