Open Science MOOC

UPDATE: Content development is now happening HERE on GitHub. The newly updated website can be found HERE. Please do not edit this document further, all material has been transferred to GitHub.

  • Status: In preparation (scroll down for module structure/resources - it helps if you select to view the document outline).
  • If you wish to contribute as a volunteer on any aspect of this project, please see the open list of Potential Contributors.
  • Primary website, including details of the Steering Committee, Code of Conduct, and Terms of Service, as well as all resources mentioned here.
  • All development content is currently managed via Google Drive. This includes copies of all papers/reports referenced, which are themselves checked to be Open Access for re-use.
  • GitHub organisation (one repo per module). Module development will primarily be here in the next phase.
  • Twitter: @OpenSci_MOOC.
  • Etherpad for meeting notes.

Internal communications for this project are primarily via Slack. Please use this link to join or contact us for details.

Rationale

Research is getting a global makeover, in part thanks to the power of the internet and the tools it provides for us, and in part due to a growing call for accountability (e.g., reproducibility and data provenance) in research. Global policies are emerging at different levels that include some aspect of ‘Open Research’, ‘Open Scholarship’, or ‘Open Science’, and inclusive of all research disciplines. But our universities are often letting us down, and they are not teaching us the knowledge, tools and skills we need to do research effectively in the 21st century.

Open Science is about increased rigour, accountability, reproducibility for research. It is based on the principles of inclusion, fairness, equity, and sharing. Open Science can be viewed as research simply done properly, and it extends across the Life and Physical Sciences, Engineering, and Mathematics, to Social Science and Humanities.

This MOOC is designed to help equip students and researchers with the skills they need to excel in a modern research environment. It brings together the efforts and resources of hundreds of researchers and practitioners who have all dedicated their time and experience to create a community platform to help propel research forward. 

The content of this MOOC is distilled into 10 core modules that comprise some of the most important current topics in Open Science. Each module will comprise a complete range of resources including videos, research articles, dummy datasets and code, as well as ‘homework’ tasks to complete as individuals. The MOOC will be hosted through OpenEdX, an open source provider. We expect that in the future different systems of certification will be developed, including completion badges. We also intend to build a forum for the open discussion of the MOOC and any relevant topics.

Disclaimer: We, the contributors, are fully aware that there is no magic “one size fits all solution” when it comes to implementing Open Science philosophies and best practices, especially when covering all research disciplines. However, we also believe that we should not limit ourselves from the onset, just because pragmatic solutions may not exist today for certain disciplines. We aim to set a highly inclusive standard, fully accepting the risk that for some disciplines this strategy may not be fully appropriate. Through that failure, we hope that you, the course users, will join us as course contributors and help us co-create bespoke solutions to your discipline based on principles of transparency, provenance, reproducibility and reuse of knowledge.

 Capture.PNG

Conceptual model of the Open Science MOOC (to be redrawn to match final 10 chapters)

Proposed Structure

A 10-module course with each one focussing on making each step or aspects of the research lifecycle more transparent and accountable, collaborative and efficient, and reusable and easier to build upon.

An introduction to open principles as an initial course, with several modules (chapters) to follow with more detailed discussions of different aspects of the open science movement. Each module is a stepping stone to a more transparent and accountable, collaborative and efficient, re-usable and easier research lifecycle.

Rationale        3

Proposed Structure        4

How to contribute to this document        7

How to volunteer for developing the MOOC itself        7

Modules        8

1: Open Principles        8

Key components:        8

Who to involve:        8

Key resources:        9

Tasks:        11

Learning outcomes:        11

2: Open Collaboration        11

Key components:        12

Who to involve:        12

Key resources:        12

Tasks:        14

Learning outcomes:        14

3: Reproducible Research and Data Analysis        14

Key components:        15

Who to involve:        15

Key resources:        15

Tasks:        18

Learning outcomes:        19

4: Open Research Data        19

Key components:        19

Who to involve:        20

Key resources:        20

Tasks:        22

Learning outcomes:        23

5: Open Research Software and Open Source        23

Key components:        24

Who to involve:        24

Key resources:        24

Tasks:        26

Learning outcomes:        26

6: Open Access to Research Papers        26

Key components:        27

Who to involve:        27

Key resources:        27

Tasks:        30

Learning outcomes:        31

7: Open Evaluation        31

Key components:        32

Who to involve:        32

Key resources:        33

Tasks:        35

Learning outcomes:        35

8: Public Engagement with Science        35

Key components:        36

Who to involve:        36

Key resources:        37

Tasks:        38

Learning outcomes:        39

9: Open Educational Resources        39

Key components:        39

Who to involve:        40

Key resources:        40

Tasks:        42

Learning outcomes:        42

10: Open Advocacy        42

Key components:        43

Who to involve:        43

Key resources:        43

Tasks:        45

Learning outcomes:        45

Communications Strategy        46

Messaging        47

How to contribute to this document

  • This document can be freely edited by anyone.
  • Please feel free to add any information or links that you see fit.
  • If you are unsure about something, please leave a comment.
  • Please be courteous, and do not delete the contributions of others.
  • For any images, data, or other source material, please add to the relevant folders on Google Drive.
  • Please do not share any copyrighted content or content which you do not have permission to re-use.
  • Please be as explicit and detailed as possible regarding source material.
  • Please avoid any large-scale changes to the structure at the present.

How to volunteer for developing the MOOC itself

FULL LIST MAINTAINED HERE

Please note that anyone is free to add themselves to this list and contribute as they see fit. Useful data:

  • Name
  • Affiliation (e.g., University, independent, NGO etc.)
  • Twitter handle
  • Email address
  • Country of work

Modules

1: Open Principles

Rationale (3 lines max):

To innovate in a field frequently implies moving against prevailing trends and cultural inertia. Open Science is no different. No matter how convinced you are, you will come across resistance from peers and colleagues, and the best defence is strong personal conviction that what you are doing may not be perfect now, but is the right decision in the long run. This module will introduce the guiding principles of the ‘open movement’, the different actors involved, and the impact that they are having.

Learning Objectives (1-3 specific):

LO1a: Understand the ethical, legal, social, economic, and research impact arguments for and against Open Science (knowledge).

LO1b: Set up a personal profile for defining your impact: measure the social and academic attention on the full range of research processes and outputs (tasks).

Key components:

  • What is “Open Science”, and why should we care. [a][b]
  • History of Open Science and Open Cultures.
  • Differences and commonalities in understanding and interpretation of the term.
  • Communities and diversity, inclusivity, fairness, equity, social behaviour, accountability, ethics and responsibility.
  • Open Science on a global scale.
  • How Open Science influences your career now and the future of research evaluation.
  • Open licensing, copyright, and speaking ‘legalise’.
  • The different dimensions of Open Science (e.g., Open Access, Open Data, Open Peer Review).
  • What are some of the barriers to Open Science, and why.
  • Open science and reproducible research: 2 sides of the same coin?
  • Open science in daily work: design your workflow with sharing in mind and invest time early.

Who to involve:

Key resources:

Tools

  • Open Content - A practical guide to using Creative Commons licenses/the Creative Commons licensing scheme.

Research Articles and Reports

Key posts

Other

Tasks:

  • Get an ORCID account, and fill out your profile[c][d]. This is a unique identifier for you as a researcher.
  • Get a Publons account[e][f][g][h], integrate with your new ORCID, and valorise your reviewer effort!
  • Get an ImpactStory account, and integrate with ORCID, showcase your output (not just publications)!
  • Write a summary about Open Access efforts either on your research discipline and/or in your country. If you have a website or blog, post it there.
  • Were the data for this easy to acquire? Which sources did you use?
  • Look at the status of Open Science in your research group or lab. Make a note of who is doing what. What could be improved?
  • Define clearly what Open Science means to you. Have a conversation about it with a colleague. Then, find someone from a different country, and have another conversation about Open Science.
  • Find out the policies are in your department or institute regarding:
  • Career progression and assessment.
  • Publishing and Open Access.
  • Data sharing.
  • Intellectual Property (IP).
  • Identify any disciplinary repositories either for research articles or data.

Learning outcomes:

  1. The researcher will be able to describe the ethical, legal, social, economic, and research impact arguments for and against Open Science.
  2. After deciding which platforms/tools/services are most useful for themselves and their community, the researcher will develop a personal profile for showcasing their research profile and outputs.
  3. After reflecting on the status of Open Science within their research group or lab, the researcher will devise concrete ways to locally improve open practices.
  4. Using the guidelines published by their research laboratories, departments, or institutes, researchers will identify the policies for career progression and assessment, publishing and open access, data sharing, and intellectual property.
  5. Researchers will collaborate with colleagues and international peers to develop a shared definition of Open Science.

2: Open Collaboration

Rationale (3 lines max):

Research is becoming an increasingly collaborative process, driven by a combination of technical opportunities and more complex research questions. Virtual Research Environments (VRE) are the way of the future in collaboration across continents, time zones and disciplines. While the definition of a VRE may be up for grabs, they provide powerful examples of high-performing modern research tools. In this module you will develop an understanding of collaborative platforms that work today, and how they can greatly enhance your research workflows.

Learning Objectives (specific):

LO2a: Learn what major types of collaborative platforms are available and what the use cases for each might be (knowledge).

LO2b: Be able to use a variety of collaborative research platforms (tasks).

Key components:

  • Principles of collaborative research.
  • Documentation as conversation and collaboration.
  • Version control, organising repositories, and project management (GitHub, Git, Zenodo).
  • Virtual Research Environments (VRE) on the horizon (e.g., EU projects on VREs).
  • Website and content management.
  • Collaborative writing platforms.
  • Collective annotation services.
  • Community spaces and communication tools.

Who to involve:

Key resources:

Tools

  • Google docs (and sheets and slides).
  • CRediT, defining contributor roles in research outputs (CASRAI).
  • Quartzy, a lab management platform.

Research Articles and Reports

Key Posts

Other

Tasks:

  • Create a GitHub account, and create your first repo with a license and readme file. Use your new account to login to Zenodo and preserve any GitHub repos. Connect with your ORCID profile too.
  • Push different data formats in GitHub (e.g. CSV, SVG) and see what happens.
  • Find a GitHub repo that a colleague has created. Make changes or comments collaboratively, and inspect the differences.
  • Tag the releases and backup to Zenodo.
  • Create an OSF collaborative environment from data to publication.
  • Connect your OSF project to GitHub.
  • Upload any raw code, images, data, tables to project.
  • Obtain a DOI and ARK identifier for your project.
  • Use PaperHive, PubPeer, or Hypothes.is to comment on (annotate) any research article of your choice.
  • Consider doing this as part of a regular preprint journal club[i][j].
  • Create a ScienceOpen/Figshare collection on your favourite research topic.
  • Begin a new article draft using an Overleaf template. Clone this article with GitHub.

Learning outcomes:

  1. The researcher will become familiar with the range of options available to you to aid greater collaborative research.
  2. After deciding what works optimally for their workflow, the researcher will be able to use collaborative tools such as GitHub and the Open Science Framework for increased collaboration for the research process, writing/authoring, and sharing your research outputs.
  3. The researcher will be able to collaborate with colleagues to annotate preprints or other published articles, and share this discussion with the original authors and wider research community.

3: Reproducible Research and Data Analysis

Rationale (3 lines max):

Reproducible research is at the heart of science. There has been an increased need and willingness to open and share research from the data collection right through to the interpretations of results. This has come with its own set of challenges, which include designing workflows that can be adopted by collaborators in a way that does not compromise the integrity of their contribution. This module will introduce the necessary tools required for transparent reporting which is reproducible and readable. 

Learning Objectives (specific):

LO3a: Learn about the nature of reproducible research, workflow design, data management and manipulation, dynamic reporting, what the key requirements are, and which resources are available to support these (knowledge).

LO3b: Be able to use available resources to create a workflow for reproducible research (task).

Key components:

  • Factors that affect reproducibility of research.
  • Principles of reproducibility, and integrity and ethics in research.
  • What is the ‘reproducibility crisis’, and meta-analyses of reproducibility.
  • Open materials, reagents and hardware, including resources, repositories and standards.
  • Electronic lab notebooks.
  • Data analysis documentation and open research workflows.
  • Living figures, turning scripts into reproducible documents, and Markdown.
  • Pre-registration and prevention of p-hacking/HARK-ing (Hypothesising After Results are Known).
  • Reproducible analysis environments (virtualization).
  • What are the computing options and environments that allow collaborative and reproducible set up.

Who to involve:

  • Individuals: Andy Byers, Anna Krystalli, Julien Colomb, Rutger Vos, Brian Nosek, Lorena Barba, Karl Broman, Victoria Stodden, John Ioannidis, Chris Chambers.
  • Organisations: FOSTER, Center for Open Science, COPE, Protocols.io, ROpenSci, Addgene, BITSS, Project TIER.
  • Other: GOSH Community, Software and Data Carpentry communities.

Key resources:

Tools

  • Open Science Framework (COS).
  • Existing reproducible research workshops/practical resources:
  • Binder Documentation, for creating custom computing environments that can be shared and used by multiple remote users.
  • Nextflow[k][l], open source tool than enables reproducible and portable computational workflows across cloud and clusters.

Research Articles and Reports

Key Posts

  • Data hygiene and data provenance.

Other

  • SoS, multi-language notebook (based on Jupyter Notebook) and workflow system for cost-effective reproducible analysis.

Tasks:

  • Find a core data set that is used throughout the examples.
  • If possible, the dataset should have a diverse set of formats and styles for different types of analysis.
  • Designing a reproducible research workflow.
  • Create a flowchart of options to help get you started Check if your collaborators, colleagues or supervisors are using the same tools.
  • This can be created as a Google doc and shared for collaboration.
  • Use validated, standardized reagents where possible.
  • Use an electronic lab notebook and best practices for recording protocols and actual steps, reagents used.
  • How well annotated are your code scripts? As a general rule of thumb, try and include one comment for every three lines of code. Bear in mind, the primary audience is future you and other people less familiar with your code.
  • Posting raw and cleaned data files.
  • Post your data (raw and/or treated) online in a non-proprietary format.
  • Make sure it is in a place where you can get a unique identifier for it.
  • Write a study plan or protocol.
  • Set up a reproducible project using an electronic lab notebook to help organise and track your research.
  • Track changes as your research develops using a version control system such as GitHub.
  • Document everything done by creating a README file.
  • Make sure to select an appropriate license for your repo.
  • Convert the notebook into a standard research manuscript.
  • In this manuscript, include all necessary code to reproduce any figures and tables in their respective captions.

Learning outcomes:

  1. Researchers will be able to describe the key factors that affect the reproducibility of research, including workflow design, data management, and reporting.
  2. The researcher will be able to use a range of resources to create and implement a workflow for reproducible research, including using lab notebooks and tools for sharing code and data.

4: Open Research Data

Rationale (3 lines max):

Open research data refers to the publishing the data underpinning scientific research results so that they have no restrictions on their access. Openly sharing data opens it up to inspection and re-use, forms the basis for research verification and reproducibility, and opens up a path to broader collaboration. In this module, you will gain insight into the importance of data sharing for reproducible research and how to curate and share your own research data.

Learning Objectives (specific):

LO4a: Learn the characteristics of open data, understand the advantages and disadvantages (alternatively, arguments for and against) open data (knowledge).

LO4b: Be able to turn a closed data set made for personal use into an open data set made for maximised accessibility, transparency, and re-use (task).

Key components:

  • What is open data.
  • FAIR Principles and data infrastructure.
  • Pros and cons of sharing data openly.
  • Sensitive data and anonymisation.
  • Data management plans.
  • Raw and primary data.
  • Tidy data.
  • Computer and human readability.
  • Interoperability: from vocabulary to ontologies.
  • Metadata:
  • Basic scheme for data publishing.
  • Additional information for data.
  • Folder organisation.
  • Data publishing (discipline-specific and generic databases) and data journals.
  • Sensitive data: privacy, de-identification/anonymization, mediated access.
  • Data citation.
  • Version control and data.

Who to involve:

Key resources:

Tools

Research Articles and Reports

Key posts

Other

Tasks:

  • Find a core data set that is used throughout the examples.
  • If possible, the dataset should have a diverse set of formats and styles for different types of analysis
  • Metadata: add minimal context for data interpretation and re-use.
  • Think about your target audience, the delivery format, file names, and general accessibility.
  • Upload some of your data to a public repository.
  • Make sure it conforms to the FAIR principles.
  • Search for data that might be of use to you in your research.
  • Does it meet FAIR requirements?

Learning outcomes:

  1. The researcher will be able to define the characteristics of open data, the advantages and disadvantages associated with sharing different types of data openly, and the FAIR principles.
  2. Researchers will be able to share their research data openly to a relevant public repository in a way that conforms to the FAIR principles.
  3. The researcher will be able to locate and re-use datasets for their research from relevant disciplinary repositories.

5: Open Research Software and Open Source

Rationale (3 lines max):

Software and technology underpin modern science. There is an increasing demand for more sophisticated open source software, matched by an increasing willingness for researchers to openly collaborate on new tools. These developments come with a specific ethical, legal and economic challenges that impact upon research workflows. This module will introduce the necessary tools required for transforming software into something that can be openly accessed and re-used by others.

Learning Objectives (1-3 specific):

LO5a: Learn the characteristics of open software; understand the ethical, legal, economic, and research impact arguments for and against open software, and further understand the quality requirements of open code (knowledge).

LO5b: Be able to turn code made for personal use into open code which is accessible by others (task).

LO5c: Use software (tools) that utilizes open content (task).

Key components:

  • Principles of open source software.
  • What open, collaborative platforms, with version control, exist.
  • GitHub and Zenodo plug-in for code archiving.
  • How to document and publish code.
  • Open Source licensing.
  • Tools for better open research (e.g., RStudio).
  • Community codes, governance, and contributions.
  • How to access and start working on general computing platforms (e.g., GCE, AWS, OpenStack and more specific - Galaxy, InsideDNA.)
  • Differences in setting up accounts/storage/computing on different platforms.
  • Comparison in terms of collaboration and openness with clusters/in-house servers.

Who to involve:

  • Individuals: Paola Masuzzo, Naomi Penfold, Titus Brown, René Bernard, Daniel Katz, Neil Chue Hong, Heidi Seibold, Anna Kostikova.
  • Organisations: Sustainable Software Institute, COKO Foundation, Free Software Foundation.
  • Other: WSSSPE community, Editors of software peer-reviewed journals (Open Research Software, JOSS), Testimonial of scientists who just published code explaining why they went through the trouble and of scientists who already use cloud computing. Explanations about large initiatives (e.g. TCGA) moving their data into cloud and why it has huge impact.

Key resources:

Tools

Research Articles and Reports

Key Posts

Other

Tasks:

  • Set up a GitHub account, if you haven’t already.
  • Share some of your code in a new repo.
  • Track changes as your research develops using version control.
  • Document everything done by creating a README file.
  • Make sure to select an appropriate license for your repo.
  • Archive your versioned code in Zenodo.
  • Explore minimum requirements to publish research code.
  • Set up accounts in different platforms and see how to upload and share data and software.
  • Create a Docker ID.
  • Learn how to bring Docker containers into the cloud and execute analyses with them.

Learning outcomes:

  1. The researcher will be able to define the characteristics of open source research software, and the ethical, legal, economic and research impact arguments for and against it.
  2. Based on community standards, researchers will be able to describe the quality requirements of sharing and re-using open code.
  3. The researcher will be able to use a range of research tools that utilise open source software.
  4. Individual researchers will be able to transform code designed for their personal use into code that is accessible and re-usable by others.

6: Open Access to Research Papers

Rationale (3 lines max):

Making scholarly research outputs openly available is easy, legal, and has demonstrable benefits to authors, making it a good beginning step for a researcher just beginning to explore the open world. There is a set of knowledge required to navigate the Open Access landscape, involving copyright, article status, repositories, and economics. This module will introduce key concepts and tools that can help a researcher make their work openly available and maximize the benefits to themselves and others.

Learning Objectives (1-3 specific):

LO6a: Understand the allowances for self-archiving in publishing contracts, including issues to do with copyright, licensing, article versions, availability, embargoes, and the types of outlets for self-archiving (knowledge).

LO6b: Gain an understanding of the history of scholarly publishing, and be able to articulate benefits of Open Access in terms of impact on society and our knowledge economy (knowledge).

LO6c: Develop a personal infrastructure for self-archiving (task).

Key components:

  • Sharing research findings with international academic and non-academic communities without paywall and other usage restrictions.
  • Personal academic impact and advantages of Open Access (e.g., increased citation counts, visibility, readership).
  • Global, national, funder, and institutional policies and mandates.
  • Pre-prints, post-prints, and versions of record (VOR).
  • Different ‘types’ of Open Access: gold, green, diamond/platinum, black.
  • The cost and economics of Open Access.
  • Open Access platforms.
  • Institutional and subject repositories .
  • Scholarly Collaboration Networks (e.g., ResearchGate, Academia.edu).
  • Open Access monographs and books.
  • Pre-registration.

Who to involve:

  • Individuals: Lauren Collister, Martin Paul Eve, Chris Chambers, Jessica Polka, Mark Patterson, Pablo Dorta-González, Ahmed Ogunlaja, Ricardo Hartley, Dasapta Erwin Irawan, Bjoern Brembs, Erin McKiernan, Anna Sharman.
  • Organisations: DOAJ, SPARC, Open Library of Humanities, ASAPbio, Open Access advocacy groups, including local initiatives on the country and institute level.
  • Other: SHERPA/RoMEO, Open Access Directory.

Key resources:

Tools

  • SSRN (Social Sciences Research Network).
  • ChemRxiv (Chemistry).
  • ESSOAr (Earth Sciences).
  • Cogprints (Psychology, Neuroscience and Linguistics).
  • Language-specific servers:
  • SHERPA/RoMEO - Publisher Copyright Policies and Self-Archiving.
  • Open Knowledge Maps.
  • PASTEUR4OA, Open Access Policy Alignment Strategies for European Union Research.
  • The Publishing Trap board game, to help researchers understand how money, intellectual property rights, and both open and closed publishing models affect the dissemination and impact of their work (UK Copyright Literacy).
  • Mathoverflow and PhysicsOverflow.
  • PubPub, collaborative community publishing.
  • JSTOR, a portal for open content.
  • Dimensions, for information on grants, publications, citations, clinical trials and patents.
  • Preprint recommendation services:
  • Overlay journals:

Research Articles and Reports

Key posts

Other

  • HRCAK Repository of the Croatian OA journals.

Tasks:[q][r][s][t]

  • Get an overview of the relevant journals and publishing outlets in your research discipline.
  • Which ones have Open Access options.
  • How much do they each charge for Open Access.
  • What funds are available to you to cover these (where relevant).
  • Preferably, find out which diamond/platinum OA journals (i.e., those which do not charge APCs) with high-quality editorial policies exist in your field.
  • Draft a summary statement/report outlining the pros and cons of these outlets (e.g., editorial quality, OA policies).
  • What do your colleagues think about the credibility, advantages, and disadvantages of these outlets?
  • How does this compare to your views?
  • Simple exercises on average “cost of a paper”; for example the average institute budget/publication output, or your last research grant/papers out compared to the average ‘gold Open Access’ cost in that discipline.
  • Find out if you are eligible for funds to pay for article-processing charges APCs.
  • Is the policy from your funder or institute?
  • What are the conditions?
  • Find a way to make all of your research papers legally freely available.
  • Use SHERPA/RoMEO to detangle the legalese in publishing contracts.
  • Check with Dissem.in which of your papers can be made Open Access via self-archiving.
  • Self-archive one paper (can be previously published) or share a pre-print to an archive.
  • Make sure to identify and include all relevant metadata (e.g. publisher requires citation with a URL to the final published version).
  • Check ImpactStory to see the impact of your research outputs.
  • What can be improved?
  • What happens to your Open Access score when you self-archive your papers?
  • Request an article using the OA Button.
  • Look for a local OA journal at your university or in your region.
  • Is there a preprint server for your research discipline?

Learning outcomes:

  1. The researcher will become familiar with the history of scholarly publishing, and development of the present Open Access landscape.
  2. The researcher will gain a multi-stakeholder insight into Open Access, and be able to convey a balanced overview of the perceived advantages and disadvantages associated with Open Access publishing.
  3. The researcher will be able to describe some of the complexities of the current the Open Access landscape, including allowances for self-archiving and embargoes, copyright transfer, and publishing contracts.
  4. Based on community-specific practices, the researcher will be able to use the different types of outlets (repositories) available for self-archiving, as well as the range of Open Access journal types available to them.
  5. Each researcher will able to make all of their own research papers Open Access through a combination of journals and development of a personal self-archiving protocol.
  6. Researchers will be able to describe the current ebb and flow in the debates around preprints, and be able to locate and use relevant disciplinary preprint platforms.
  7. Researchers will be able to use services like ImpactStory to track the proportion of their research that is Open Access.

7: Open Evaluation

Rationale (3 lines max):

Concurrent with broader developments in Open Science and increased transparency in research, Open Peer Review is a complex, and rapidly evolving topic. Alongside this, more diverse criteria of research evaluation beyond traditional methods are emerging, and with these come a range of practical, ethical, and social factors to consider. This module will provide insight into current developments in Open Peer Review and research evaluation.

Learning Objectives (specific):

LO7a: To understand the history of peer review, and place current developments in Open Peer Review in that context (knowledge).

LO7b: To gain insight into the process of responsible research evaluation, and the role that peer review and traditional and next-generation metrics play in this (knowledge).

LO7c: To be able to identify and apply a range of metrics to demonstrate the broader impact of your research outputs (tasks).

Key components:

  • Fundamentals of good peer review.
  • History of peer review and scholarly publishing.
  • Types of open peer review and new models.
  • Pros and cons associated with different types of open peer review, including post-publication peer review, commenting and annotation.
  • Issues with traditional methods of research assessment and evaluation.
  • The San Francisco Declaration on Research Assessment (DORA), Leiden Manifesto, and Metric Tide reports.
  • Next generation metrics (aka altmetrics), responsible metrics use and peer review.
  • Role of metrics in research evaluation, funding, promotion, signalling and reporting.
  • Differentiating between impact and attention.

Who to involve:

  • Individuals: Nikolaus Kriegeskorte, Irene Hames, Tony Ross-Hellauer, Peter Kraker, Michael Markie, Sabina Alam, Elizabeth Gadd, William Gunn.
  • Organisations: OpenAIRE, ScienceOpen, Publons, PubPeer, OpenUP, Altmetric, ImpactStory, BioMed Central, Frontiers, eLife, PEERE.
  • Other: Editorial staff at journals offering traditional peer review.

Key resources:

Tools

Research Articles and Reports

Key posts

Other

Tasks:

  • Perform one open peer review on a paper of your choice at ScienceOpen, and get a DOI for it.
  • Integrate one peer review (pre- or post-publication) experience into Publons.
  • Use Publons journal list to check open peer review policies of journal(s) in your discipline.
  • Sign DORA in either a personal or business-level capacity.
  • Define your impact.
  • Write a personal impact statement about your research (actual or predicted). Avoid using journal titles or the journal impact factor.
  • Discover the Altmetric scores for your published items using their bookmarklet.
  • Track your research impact by integrating your ORCID profile with either ScienceOpen or ImpactStory (or both).
  • Do you have a personal website? If not, now is a good time to design one and make all of the above information part of your digital profile.
  • Find out what your research department or institutes research evaluation criteria are. Have a discussion about them with your research colleagues.
  • Find out who wrote them, and ask them what evidence they used to support the criteria.

Learning outcomes:

  1. The researcher will be able to describe the history of peer review in the context of scholarly publishing, the criticisms levied against ‘traditional’ peer review, and the ongoing developments with Open Peer Review.
  2. The researcher will be able to use a range of post-publication review, commenting, and annotation services.
  3. The researcher will be able to describe the issues associated with the use of ‘traditional’ metrics in research evaluation, and the role that peer evaluation and ‘next-generation’ metrics (or ‘altmetrics’) play in this.
  4. The researcher will be able to use a range of services to build and demonstrate their personal research impact profile, both quantitatively and qualitatively.
  5. The researcher will become familiar with the relevant criteria for research evaluation to them, and be able to have a critical discussion about them with their colleagues and those who drafted them.

8: [u][v][w][x][y][z]Public Engagement with Science

Rationale (3 lines max):

Citizen science describes the method of engaging people outside of academia within the research process itself. Science communication is often seen as a unidirectional process from scientist to non-scientists, but with careful and strategic engagement they can both be so much more. For this, a deeper understanding of the basics of communication and engagement and existing structures are needed, as well as the capabilities of newer channels like social media. This module will teach effective techniques for communicating your research with a wider non-academic audience, as well as engaging them with the process itself.

Learning Objectives (specific):

LO8a: Understand the basic concepts and the viewpoints of different stakeholders in science communication (knowledge).

LO8b: Understand the different target groups/audiences and communication channels, who to involve in what kind of communication, and how to do it strategically and with which tools (knowledge).

LO8c: Develop either a citizen science program to empower non-academics interested in your research field, or a personal communication strategy to bring your research to a wider audience (tasks).  

Key components:

  • Basics and principles of science communication, public outreach and engagement, and their relationship to Open Science.
  • Different stakeholders and audiences in public engagement and science communication, and how to shape messages for each of them.
  • Press releases and interacting with the media.
  • Different forms of social media:
  • How and why to blog about your research.
  • Using Twitter for outreach, conferences and networking.
  • How to use video and audio for outreach.
  • How to connect with citizen science initiatives, public advocacy groups, and  patient organizations in your research area.
  • How to take your research to the stage (e.g., FameLab, Science Slam, Cosy Science).
  • When sh*t hits the fan - basics in crisis communication.

Who to involve:

Key resources:

Tools

Research Articles and Reports

Key posts:

Other:

Tasks:

  • Search and make a short list of your institution’s people involved in outreach, PR etc. Do you know everybody important for what you do?
  • Read a press release from your institution.
  • How does it compare to the research article itself?
  • Write a blog post summarising a selection of your research papers to date.
  • Start a blog and post them!
  • If possible, connect this to your main website.
  • Respond to discussions on (social) media about your topic of research.
  • Use hashtags to find relevant conversations.
  • Who is popular in your field? What do you notice about their style of engagement?
  • Identify relevant citizen science initiatives on social media.
  • Add them to your contacts.
  • Reach out to them and open a conversation on how you could mutually benefit from shared research.
  • How would you communicate with protesters in front of your institute?
  • What policy-level consultations are open at the moment at a national level?
  • Are any of them in a discipline or topic related to yours?
  • If so, draft a short response based on your understanding of the relevant research.

Learning outcomes:

  1. The researcher will be able to identify and describe some of the major different types of audience and stakeholder involved in science communication, what their needs and viewpoints are, and the importance of citizen science and public engagement with science.
  2. By working either individually or their research group, each researcher will be able to use a range of communication channels, including social media, to strategically engage different types of audience with their research.
  3. If there are relevant policy-related issues to their discipline, the researcher will be able to engage with them through available channels and make sure that their research field is appropriately represented.
  4. Each researcher will be able to identify relevant press/communication contacts at their institute, and be able to convey to them why their research is of importance for wider dissemination.
  5. The researcher will be able to write a blog post or non-specialist summary about either their own research or research that they are familiar with, and communicate this to wider non-academic audiences.

9: Open Educational Resources

Rationale (3 lines max):

Open Educational Resources (OERs) are freely accessible, openly licensed materials for teaching and learning, and represent a paradigm shift compared to traditional methods of education. They are intrinsically related to developments in Open Science, due to the wider implications of access to knowledge in education in our global societies. This module will provide an understanding of the motivations behind OERs and how to develop your own.

Learning Objectives (specific):

LO9a: Understand the driving forces and motivation behind the OER movement (knowledge).

LO9b: Be able to openly license your research to enable educational re-use, or create your own educational resources (task).

Key components:

  • Definition and scope of Open Educational Resources (OER), including aspects of resource licensing and re-use.
  • The ‘five Rs’ of OER: Retain, Re-use, Revise, Remix, Redistribute.
  • Motivations behind OER movement, including lower costs and increasing accessibility to education, and the role of institutional/organizational support.
  • OER repositories (national and others), and some of the major OER initiatives.
  • Principles of open pedagogy/andragogy.
  • The impact of OER on sustainable development, economic growth, social inclusion, and environmental conservation.
  • How OER can influence policy development at national and institutional levels through capacity building and social mobility.

Who to involve:

Key resources:

Tools

Research Articles and Reports

Key posts

Other

  • Definition of OER, OpenContent.
  • MERLOT, Multimedia Educational Resource for Learning and Online Teaching.
  • Open textbook initiatives:

Tasks:

  • Create a Wikipedia account.
  • Integrate one or more of your research articles (or someone else’s) into Wikipedia.
  • Make sure to link to an Open Access version if possible.
  • Make some of your research outputs or teaching materials openly available.
  • Remember to choose an appropriate repository.
  • Make sure the content is openly licensed and granted a DOI.

Learning outcomes:

  1. The researcher will be able to convey the motivations behind the OER movement, and the relationship that this has with Open Science.
  2. The researcher will be able to identify and implement the steps to either prepare content for educational re-use purposes, or be able to design their own OER.
  3. The researcher will be able to either identify relevant places where their research can be integrated into Wikipedia, or integrate it themselves if they are a user.

10: Open Advocacy

Rationale (3 lines max):

Now that you are an expert at applying Open Science at each step of your research lifecycle, here are some basics on becoming a pro-active ambassador for open scholarship in any discipline. This module will teach you how to effectively engage researchers and other stakeholders in scholarly communication with the various aspects of Open Science.

Learning Objectives (specific):

LO10a: To understand the needs of different stakeholder groups in scholarly communication, and the impact that Open Science can have on them.

LO10b: To be able to translate your knowledge into an effective program or tool for external engagement.

Key components:

  • What does it mean to be an ‘advocate’ for Open Science.
  • Advocating for your own rights as an author.
  • The basic steps for achieving local culture change (e.g., Kotter’s 8-step change model of management).
  • Advocating to your peers, including writing letters and articles advocating for Open Science.
  • Talking to journal editors - catalysing the Open Access conversation within your field.
  • Talking to policymakers about Open Science.
  • Building or joining an Open Science community.
  • Effective leadership and training in Open Science, and empowering others to make change.

Who to involve:

  • Individuals: Josh Bolick (and colleagues, see the rebuttal article below), Johan Rooryck, April Clyburne-Sherin, Nick Shockey, Joseph McArthur, Heather Joseph, Nicole Allen, Erin McKiernan. 
  • Organisations: R2RC, SPARC, Creative Commons, IGDORE.
  • Other: Country specific advocacy groups (e.g., AOASG, Open Access Nigeria), OpenCon.

Key resources:

Tools

Research Articles and Reports

Key posts

Other

Tasks:

  • Write a letter to your local political representative about why you think research is important.
  • What do politicians care about? Use this context to empathise with them and deliver your message effectively.
  • Send an email to an editor (or editorial board) of a ‘closed access’ journal in your field to start the OA conversation.
  • Is the journal OA already (or ‘hybrid’)? How much are APCs for it? Are there any cheaper alternatives they might not be aware of?
  • Does your research institute have a magazine, forum, or newsletter? Write a letter/post for it in support of Open Science.
  • Draft your own email template reply for requests to peer review about how you only review for OA journals.
  • Re-use/base it on ones out there already. What has worked or not worked so well in the past?
  • Outline opportunity costs for your university administrators on role of Open Science in hiring, tenure and promotion guidelines.
  • Outline concrete solutions and benefits Open Science can deliver for current headaches university administrators may struggle with.
  • Find your local Open Science advocacy group and volunteer for them!
  • Does one not exist yet? Why not start one! Local groups are a great way to meet like-minded individuals and work together.
  • Having a webpage like Meetup can help keep people engaged and aware of meetings.

Learning outcomes:

  1. The researcher will gain an appreciation of, and be able to identify, the diversity of different communities and stakeholders in scholarly communication, and the potential impact that Open Science can have on them.
  2. The researcher will become an effective leader in Open Science, and use their skills and knowledge to empower others.
  3. By working either alone or with like-minded colleagues, the researcher will either join or establish a local open science advocacy group or meetup, and identify concrete action steps that they can take together.
  4. The research will prepare an open science statement to distribute to administrative staff at their research institute, as well as any other relevant local stakeholders.
  5. Together with like-minded colleagues, the researcher will start the ‘open access conversation’ with the editorial board of a relevant journal in their field.

Communications Strategy

Open Science MOOC platform: https://opensciencemooc.eu/

Two-phase release strategy:

(1) Open review of final draft by Graduate Schools and professional societies aiming for a good discipline coverage, and use the review process as part of the promotion;

(2) Full public release and promotion as below.

  • Mailing lists:
  • Open Science.
  • And any regional variants.
  • OpenCon.
  • Open Access.
  • Psci Comm.
  • Social Media
  • Twitter blitz:
  • Thunderclap.
  • Dedicated Facebook page.
  • Dedicated Slack/Gitter channel for all participants (contact to join).
  • Suggestions for a communication plan, and a template from Leiden University.
  • Non-social media
  • University mailing lists.
  • Assets: EURAXESS, EURODOC.
  • National Open Science contact points (OpenAIRE NOADs = National Open Access Desks ).
  • National science communication organizations.
  • Etherpad for collaborative note taking.

Messaging

For students:

  • These skills will get you a job both inside and outside of academia.
  • These skills will save you time during your research.
  • These skills will make your output of better quality.
  • These skills make peer review of your papers more efficient.

For lab-heads and institutions:

  • Make collaboration in the workplace better and faster.
  • Continuing the work of alumni will be easier (information transfer, patenting).
  • Openness often implies more citations and attention for research.
  • Encourages integration of science within society, increases societal impact of research.

For policymakers:

  • Good for researchers as it teaches them core competencies and transferable skills to be used outside of academia.
  • Contributes to innovation and economic growth.
  • Contributes to a healthier society.
  • Helps them meet policy objectives.

For librarians:

  • Can be integrated with graduate school training programs at zero cost to them.
  • Helps train researchers/students in tasks that will ultimately make their jobs easier.
  • Will create a new knowledge pool that they can draw upon if needed.

For publishers:

  • ...

[a]Comments by RRI Tools: when going through the general structure of the document (Rationale + 10 Modules) we somehow miss a "broader" approach. Students & researchers need to understand the broader context.

Of course they need to learn what "open science" is, but also WHY open science. The "whys" are as important as the "whats"

In this sense we miss an introduction that situates "open science" in a broader context (this intro could be part of the rationale of the MOOC, or be module 0, o part of module 1)

This intro would also help students understand why the course is structured in 10 modules (as for now the rationale for having module 8 is a little bit unclear /and its contents a bit weak yet)

We would recommend to talk about the "science crisis" (useful resources are

++ interview with A. Saltelli http://bit.ly/2svgc0c

++ article by A.Saltelli & S. Funtowicz https://www.sciencedirect.com/science/article/pii/S0016328717301969)

Talking about the science crisis helps you explain some of the whys for open science. And also helps to introduce the issue of the replication crisis and the need for reproducible research (module 3).

And it also helps understanding some of the whys for the concept of "responsible research and innovation" and for the need to foster a higher public engagement in research (the need to "foster R&I processes that are collaborative and multi actor: all societal actors work together during the whole process in order to align its outcomes to the values, needs and expectations of society")

Which is probably the kind of issues you want to adress in module 8

You can find an intro to RRI is this link https://www.rri-tools.eu/about-rri

(sorry for this long comment! - meant to be constructive ;)

[b]This is great! Could some of it be integrated into the Comms Strategy section towards the end? https://docs.google.com/document/d/1KuTSECSYHXZmZX15GDjyD65pJ90eRMhHVEZ-1trsw30/edit#heading=h.hh90c9324cg3

[c]You can now include other outputs on top of publications. You've mentioned reviews, but ORCID also lets you add your data, software, etc. (if it has DOIs). You might also want to register and share your other research outputs with OSF (https://osf.io/) etc.

[d]Ah, i missed you included OSF below

[e]Are folks not concerned that Publons is now a Clarivate Analytics company?

Here is an interesting critique of the role of Publons in the “extremely exploitative system” that is modern publishing:

http://journals.sagepub.com/doi/abs/10.1177/1747016117739941

[f]Thanks for this, Lorena. The real question for me is - is it our role as the MOOC developers to advocate for one particular set of services over another? I'm aware of the Clarivate acquisition, and this heavily-worded critique. What do you think..? (and anyone else!)

[g]"Get a Publons account" is a strong advocacy of this service, I'd say!

[h]Ha, you're right! But I meant more commercial vs non-commercial or for-profit versus non-profit.

[i]see also PREreview and preprint.space

[j]Feel free to add these as an issue here to stay updated! :) https://github.com/OpenScienceMOOC/Module-2-Open-Collaboration

[k]If you include NextFlow what about other workflows such as Galaxy, etc. Common Workflow Language is trying to bring the tribes together so should be mentioned in this area

[l]Excellent! Could these possibly be added here? https://github.com/OpenScienceMOOC/Module-3-Reproducible-Research-and-Data-Analysis

[m]Have some nice Knitr examples that can be used for training purposes http://gigasciencejournal.com/blog/carmen-reproducible-research-and-push-button-papers/

[n]Can this be added here too? :) https://github.com/OpenScienceMOOC/Module-3-Reproducible-Research-and-Data-Analysis/blob/master/key_elements.md

[o]And GigaScience [COI: GigaScience]

[p]This one can go here :) https://github.com/OpenScienceMOOC/Module-4-Open-Research-Data

[q]I think the first and most basic task should be to get an overview of the relevant journals/publishing outlets in the respective research fields. Students should check if these major outlets offer OA, see how much they charge and find out what funds are available at their end. In a second step they should check alternative journals that are platinum in the field (or offer significantly lower APCs). In a third step they can compare editorial boards and policiesand write up a quick pro and con paper/statement.

Another task could entail asking colleagues about the credibility, advantages and their knowledge of these OA outlets and formulating the outcome of these discussions.

[r]Awesome ideas! Have integrated these in - thoughts?

[s]the headache with MOOCs is to find the right balance between offering information and involving students. Plastering students with too much of either will result in higher dropout numbers. Hence I am a little wary of the completist approach. Less is often more in MOOCs.

It utterly depends of course on what the pedagogical goal is: is this supposed to be an interactive reference source or an introductory course on all things Open (Open 101 as it were).

[t]Could this be added as a comment here? :) https://github.com/OpenScienceMOOC/Module-6-Open-Access-to-Research-Papers

[u]I'm a bit weirded out by conflating citizen science and scicomm. Involving people outside academia in the scientific process is more than just a subset of doing scicomm. Instead it aims to empower people to do their own science.

This is completely absent in the current structure, not part of the learning objectives and it's also reflected in the key components where this part is also absent, instead cit sci is only listed as connecting to cit sci initiatives.

[v]Excellent, thanks Bastian. My experience with citizen science is almost zero, so glad to have your input at this stage! Lemme try and fix.

[w]Thanks! Ping me whenever you want feedback! :)

[x]Agreed - I only saw Bastian's comment after making my own!

[y]🎉

[z]Citizen Science basically falls into two camps:

(1) scientists involving the general public

(2) the general public doing science

People from each camp tend to ignore the other camp. We once wrote an OA study on how these projects emerge (see below). I agree with Bastian that both camps need to be represented. Simple example of (2) is local history, where people that live in a street research the history of that street. You could hardly get a "professional" historian to publish with you but people that do these projects use academic methods in their inquiry.

http://journals.sagepub.com/doi/full/10.1177/0963662516678514

I'll put it in the reading section.

[aa]If you are including ECSA should also mention CSA, ACSA and nascent networks around the world (CitizenScience.Asia) currently forming a global secretariat

[ab]Can this one be added here? :) https://github.com/OpenScienceMOOC/Module-8-Public-Engagement-with-Science