HIRMEOS

High Integration of Research Monographs in the European Open Science infrastructure

H2020 Grant agreement 731102

WP7 - Workshop
Valutazione della ricerca e servizi di identificazione/citazione:
prospettive nelle scienze umane e sociali
Turin March 21st, 2018

 Report

By Elena Giglia and Alessandro Leccese, University of Turin

The University of Turin organized a workshop on Valutazione della ricerca e servizi di identificazione/citazione: prospettive nelle scienze umane e sociali [Research evaluation and identifiers/citing services: perspectives in SSH] on March 21st, 9.30-13.00.

25 participants attended the workshop; apart from 2 researchers, they were mostly administrative people in charge of supporting research and evaluation practices. Italian was the selected language for presentations and discussion.

The first session was supposed to set the frame, with three talks about the importance of sound and open data for research evaluation, the Italian peculiar situation when considering ORCID, the presentation of the services offered by HIRMEOS and the future developments in OPERAS. The slides shown during the workshop are available on http://www.oa.unito.it/new/valutazione-della-ricerca-e-servizi-di-identificazione-citazione-prospettive-nelle-scienze-umane-e-sociali/ 

During the second session, participants were involved in lively discussions about research evaluation and the issues concerning identification and certification services.

Setting the frame

Paola Galimberti, University of Milan, introduced her talk underlying the importance for evaluation purposes of the unambiguous and secure connection between researchers, their affiliation (universities or research performing organizations), the funder, and the research outputs. Such information should be available in machine readable format, in the publication’s metadata.

She tackled some of the more widespread issues in SSH, all relevant - as it turned out -  for the services HIRMEOS and OPERAS aim to provide:

Identifiers - at any level: authors, funders, outputs - could be really helpful both in the discovery process and in tracking the real impact of a research output through reuse.

The conclusion was that technologies are already there, what is needed is a cultural change in adopting them and make them work.

Paola Gargiulo works the Italian national IT consortium CINECA, which provides almost all Italian Universities IRIS, a solution which is at the same time a CRIS and an Open Access Institutional repository. IRIS is used also a the basis for the national research assessment exercise, VQR [Valutazione della Qualità della Ricerca, Research Quality Evaluation]. In 2015, ANVUR [Agenzia Nazionale Valutazione Università e Ricerca, National Agency for Evaluation of University and Research] decided to adopt ORCID for the incoming VQR.

ORCID is a sort of “enabler” of a series of web-based services, seamlessly activated when an ORCID-ID is correctly used. It could boost visibility. It could allow to find with one search all the infos about the researchers, their publications, their grants, their projects, their reviews (like the interesting new website Reviewer Credits does, https://www.reviewercredits.com/ ). ANVUR’s  underlying idea was to upload data once, and then use them for different purposes and in different frameworks. Moreover, VQR adopted as identifier the fiscal code, which is unique but works just in Italy, whilst ORCID-ID is international and interoperable. So, ANVUR required any researcher to register for it.

An automated tool (ORCID-Hub, available on GitHub, https://github.com/Cineca/OrcidHub) was created to enable researchers to create their ORCID-ID and link it to the publications hosted in IRIS. ORCID integration in IRIS enables the Push-Pull feature by ORCID. This feature is less useful for SSH, due to the lack of coverage in online databases. In the end, ANVUR didn’t use ORCID for the assessment exercise.

In addition to that, ANVUR’s choice resulted to be a boomerang: many researchers saw ORCID deeply connected to the VQR process, which is largely criticized and opposed, and sort of “boycotted” it, as they boycotted the VQR.

Even though Italy is namely the second widest community after the UK, many profiles were never activated. Moreover, due to a misconception in privacy issues - the researcher decides which information are public and which private - many profiles are useless because the affiliation is not provided, which in most cases makes the disambiguation between two homonyms impossible.

In conclusion, ORCID-ID has a huge potential in integrating services around the researcher, but in Italy is not correctly used at the moment.

Elena Giglia, University of Turin and HIRMEOS project partner, presented OPERAS and HIRMEOS with a particular stress on how they try to respond to the issues raised by the two previous speakers.

She introduced the fragmentation of the SSH landscape in which OPERAS and HIRMEOS play, at disciplinary level and hitherto at publishing level. Facing this fragmentation, integration is the keyword in OPERAS/HIRMEOS; they deal mostly with books, and take care of each step from searching to editing to publishing to disseminating, as the book is at the center of the scholarly communication in SSH - “communication is, indeed, the essence of science”, as professor Guédon reminds us.

She then underlined the unique OPERAS/HIRMEOS approach: to nurture players already operating in the field, to align their practices to the best ones, to improve the global quality of the publishing landscape. OPERAS’ goal, to provide a pan-european infrastructure to rethink and reshape publishing, discovery and dissemination and to integrate SSH into the Open Science world, is being made concrete by the first steps in technical implementations in HIRMEOS. HIRMEOS works as a sort of “proof of concept” of OPERAS, implementing services that then any publishing platform could adapt and use, to improve the quality of their products.

Particular attention was drawn on the solutions provided or foreseen in HIRMEOS/OPERAS for identification and certification services, as they perfectly resonates with the concerns Paola Galimberti highlighted.

Particularly, identifiers are the first step to integrate SSH into the EOSC - European Open Science Cloud, where data ought to be FAIR - Findable, Accessible, Interoperable, Reusable.

Certification services are deeply connected to quality assurance, both for authors and evaluators: HIRMEOS will provide the technical specifications and the tools for publishers to certify their work; OPERAS will provide an international list of SSH open access publications that meet minimal quality criteria regarding peer-reviewing and licensing. It is worth highlighting that good and independent certification services enable better evaluation systems, whose lack in SSH is a prevalent issue: when submitted to research assessment exercises, authors struggle to demonstrate the value of their research outputs and their impact, and of the serious editorial workflow behind their work.

Debating

In reason of the scarce number of participants, the floor opted to have a common discussion instead of splitting into two groups.

Some of the issues raised during the debate were technicalities linked to the specific research evaluation process and tool (IRIS) adopted in Italy: however, it is interesting to note down some common controversies, needs or concerns: