1 of 24

Using big data to tackle corruption in public procurement: �an auto-ethnography of a collaboration between academia, institutions, and journalism

Grant Agreement number: 101038790 — CO.R.E — ISFP-2020-AG-CORRUPT�This paper was funded by the European Union’s Internal Security Fund — Police�The content of this paper represents the views of the author only and is his/her sole responsibility. The European Commission does not accept any responsibility for use that may be made of the information it contains.

Alessio Cornia and Dimitri Bettoni

Dublin City University

2023 Global Investigative Journalism Conference, Gothenburg, Sweden, 21 Sept. 2023

2 of 24

CO.R.E. project

The CO.R.E (COrruption Risk indicators in Emergency) project focuses on assessing the risk of corruption in public procurement during emergency times, when the normal public procurement rules do not apply. The main aim is to develop and validate a replicable procedure for computing a Composite Indicator (CI) of corruption risk in public procurement in a time of emergency, such as the COVID-19 pandemic.

The CI is intended at enhancing earlier detection of corruption risk and fostering a stronger evidence base for policy reform, by serving primarily anti-corruption authorities and law enforcement agencies, but also journalists and the general public/citizenship, for accountability objectives.

3 of 24

The Consortium

Partner name	Type	Country	Expertise
University of Perugia (UNIPG)	University	Italy	Social statistic and administrative law (focus on anti-corruption)
Universitat Oberta de Catalunya (UOC)	University	Spain	Interdisciplinary (focus on anti-corruption)
Dublin City University (DCU)	University	Ireland	Journalism (studies and engagement with industry)
Transparency International Portugal �(TI Portugal)	NGO	Portugal	Anti-corruption activism and research
Info.nodes	NGO	Italy	Anti-corruption activism, data journalism
Oficina Antifrau de Catalunya (OAC)	Anti-corruption authority	Spain	Investigating corruption and fraud in PP
Fondazione Centro Studi Villa Montesca (FCSVM)	Research body specialised in support for EU projects	Italy	Technical support

The CO.R.E consortium is strongly based on the cross-field collaboration (btw different actors)

btw partners from four countries and with diverse backgrounds, ranging from:

statisticians (in charge of the development of the CI and the elementary indicators/red flags that constitute it),
anti-corruption authorities (supporting the CI development and focusing on validating it with their internal data),
and NGOs and activists focusing on anti-corruption (developing the technological tools that will enable stakeholders, including journalists, to apply the indicators in their own investigations and on disseminating the project results).

The involvement of the journalistic community is ensured by the participation of media scholars with a track record of strong engagement with journalists and media leaders, who focus on bringing in the journalistic perspective to make sure the indicators and project tools are developed by also taking into consideration journalists’ input and feedback.

Although the direct involvement of news organisations in the project would have been ideal, individual researchers involved in the project have a journalistic background themselves or have previously worked for journalistic think tanks such as the Reuters Institute for the Study of Journalism at the University of Oxford

4 of 24

CO.R.E. project pipeline

Data & Privacy Policies�PP principles

Mathematical/statistical �models�computational �development

Data visualisation

Project

Management

Journalism�& CS feed

5 of 24

Journalists’ perspectives

Two stages of involvement:�

Interviews

Interviews were held with individual journalists, media organisations and NGOs based in Italy, Spain, and Ireland, which focus their reporting activities on corruption and other financial wrongdoings. Aimed to explore how journalists investigate corruption, their use of indicators and data on public procurement, and feed the theoretical work behind the creation of the CI.�

Testing and feedback

A workshop and individual interviews will be the base for collecting thoughts and feedback on the practical outputs of the project: the Coresoi R package and the data visualisation dashboard.

6 of 24

The Composite Indicator

The CI will enable relevant stakeholders such as journalists, activists, NGOs, policy-makers, anti-corruption agencies and Law Enforcement Agencies (LEAs) to conduct analysis, investigations, assessment and studies on corruption that go beyond the reliance on perception-based indicators and that can be applied in emergency situations.

original contribution of our CI: designed to inform and alert about possible cases of corruption by taking into consideration data on public procurement taking place during emergency periods, such as the COVID-19 pandemic, when normal PP rules do not apply.

Established corruption indicators (such as the use of exceptional procedure types or the single bidding)

are valuable and powerful proxies in ordinary circumstances,

but reduce their potentials during emergency periods.

Indeed, when an emergency state is stated at the national level, the vast majority of the bids receive single offers and are adopted within exceptional procedures, in order to rapidly procure essential goods (Cornia and Bettoni, 2022, based on the contribution by Michela Gnaldi).

Therefore, the development of the CO.R.E. CI addresses the need to have new red flags of corruption risk that works in emergency period when exceptional procedures are adopted.

7 of 24

The Elementary Indicators (red flags) 1/3

Elementary indicators	Description – Focus on…	Motivation – Consider at risk…
1. Winning rate across the crisis	Companies that were awarded contracts in the relevant economic sector (CPV) more frequently after the emergency outbreak	Exceptional increases of competitive power (in terms of proportion of awarded contracts) over the emergency
2. Awarded economic value across the crisis	Companies awarded higher value contracts in the relevant economic market than before the emergency	Exceptional increases of competitive power (in terms of economic value) over the emergency
3. Contract economic deviation across the crisis	Deviation of the contract actual execution economic value from its initial awarded value, with reference to standards in the relevant economic sector (CPV)	Contracts that undergo a significant increase of the actual amount paid
4. Contract length deviation across the crisis	Deviation of the contract actual execution duration from its stated/expected duration, with reference to CPV standards	Contracts that undergo a significant increase of their length

Gnaldi, M., & Del Sarto, S. (2023). Measuring corruption risk in public procurement over emergency periods. arXiv preprint

The CI aggregates and summarises a set of elementary indicators (i.e. red flags) into a single metrics.

The red flags are developed by focusing on the time discontinuity introduced by a crisis outbreak

and the possibility to distinguish two time-spans, a pre-crisis and a post-crisis period.

the indicators “compares, when possible, company and/or contracting authority behaviours after the crisis outbreak with respect to their historical behaviour

and assesses the associated risk [of corruption] through statistical testing.

The proposed procedure is extensible to other crisis contexts [beyond the COVID-19 emergency],

replicable to other national contexts [beyond the four countries involved in the project],

Once developed, the indicators are normalised (i.e. individual indicators are taken to the same scale and polarity),

weighted (i.e. the respective importance of individual elementary indicators is established before combining them into a single CI),

and subject to multivariate and sensitivity analysis to assess their robustness.

a brief general description of some of the elementary indicators/red flags that form the CI is provided in this Table

(Common Procurement Vocabulary (CPV) is the European statistical classification of goods and services procured by the public sector)

8 of 24

Elementary indicators	Description – Focus on…	Motivation – Consider at risk…
5. Excess of �concentration in the winners’ distribution	For each issuer, whether the distribution of winners exhibits an increase of concentration on a restricted group of companies	Authorities that award most of their contracts to a restricted group of companies over the emergency outbreak
6. Award communication default across the crisis	Increases in the rate of contracts for which contracting authorities have not communicated the award data to Anticorruption Authorities	Contracting authorities that worsened their communication duties over the emergency outbreak
7. One-shot opportunistic companies over the crisis	Companies that were awarded one or more contracts, without winning in years before the emergency	Companies that show a "one-shot opportunistic behaviour"
8. Lengthy contracts	Post-emergency, contract duration is significantly longer than the overall d contract duration in the same CPV	Post- emergency contracts whose duration is not justified by the nature of the crisis

The Elementary Indicators (red flags) 2/3

Gnaldi, M., & Del Sarto, S. (2023). Measuring corruption risk in public procurement over emergency periods. arXiv preprint

9 of 24

Elementary indicators	Description – Focus on…	Motivation – Consider at risk…
9. Inconsistent companies across the crisis	Awarded companies that radically change their sector of activity after the emergency outbreak	Companies winning contracts after the emergency that are not consistent with their activity sector (CPV)
10. Inconsistent contract objects across the crisis	Contracts with sub-objectives incoherent with the overall stated emergency context	Contracts including elements in their object field (i.e., TVs for hospitals) incoherent with the main contract object identified by the CPV (i.e., supply of screening tests)
11. Pre-existing contracts modified after the crisis	Authorities/companies that have at least one pre-existing contract (called before the emergency) which has been modified after the emergency outbreak	Authorities/companies with existing contracts (before the emergency) that are modified after 6 months from the emergency outbreak

The Elementary Indicators (red flags) 3/3

Gnaldi, M., & Del Sarto, S. (2023). Measuring corruption risk in public procurement over emergency periods. arXiv preprint

10 of 24

11 of 24

How journalism feeds CO.R.E.

Examples of journalism-inspired Red Flags as derived from the data collection:�

Contract related

Multiple contracts with similar value awarded to the same company�

Tender related

Award criteria consistency�

People related

Analysis of the social network�

Other

Geospatial references�

12 of 24

CO.R.E. Data Visualisation Dashboard

A web platform that enables journalists, activists, citizens and any other interested actor, to navigate the public procurement data analysed within the CO.R.E. project through the lenses of the Composite Indicator and its elementary indicators.

Currently under development, the dashboard produces interactive visualisations, and provides for an intuitive and preliminary although limited overview of the project results.

The CO.R.E. project also develops digital tools that enable relevant stakeholders with different levels of technical expertise to apply the indicators on public procurement (PP) data of their interest.

Two main tools are developed for this purpose: the data visualization dashboard and the coresoi R package.

The data visualization dashboard is a web platform that enables journalists, activists, citizens and other interested actors to analyse public procurement data, to apply the statistical procedures at the foundation of the newly developed CIs, and to produce interactive visualisations of the results. By using the tool, the target groups/beneficiaries will be able to monitor public procurement spending across different geographical areas (local, national or transnational) and/or over different periods of time, in order to investigate specific cases and to enrich their news reporting, investigations and analyses through the data visualization features (Cornia and Bettoni, 2022, based on the contribution by Davide Del Monte). The platform is currently under development and runs on PP data that are currently being collected by the project partners. Italian data, which at the moment constitutes the richest dataset used by the platform, have been provided by ANAC, the Italian Anti-Corruption Authority.

13 of 24

14 of 24

15 of 24

16 of 24

CORESOI R package

https://core-forge.github.io/coresoi/index.html https://github.com/CORE-forge/coresoi

The coresoi R package consists in software codes, a user guide, and other tools to enable journalists, researchers, LEAs and anti-corruption authority to apply the Composite Indicator and the elementary indicator to their own datasets.

Salvini N, Del Sarto S, Gnaldi M (2023). Coresoi: measuring Corruption Risk in Emergency. doi:10.5281/zenodo.8013258

The coresoi R package consists in software codes, an user guide, and other tools to enable journalists, researchers, LEAs and anti-corruption authority to apply the CI and the red flats to their own datasets. The package will be soon published in open-access formats and will be made available to interested stakeholders without any charge. Included in the package are: a) the list and description of the functions to compute the elementary indicators (red flags) of corruption risk in emergency; b) user-friendly guidelines; c) the list and description of the functions in coresoi to perform the normalisation, weighting, aggregation procedures, dimensionality assessment and sensitivity analysis for the Composite Indicator (CI) of corruption risk in emergencies; d) a toy dataset based on the Italian Anti-Corruption Authority (ANAC) open data – the BDNCP (Banca Dati Nazionale dei Contratti Pubblici) – which researchers can use to calculate our elementary indicators of corruption risk and the composite indicator.

17 of 24

18 of 24

19 of 24

20 of 24

https://www.core-anticorruption.eu/

21 of 24

CO.R.E.: collaborative, multi-stakeholder, interdisciplinary

A focus group and a survey conducted with the project partners were the methods used to explore the challenges and the opportunities associated with the cross-border and cross-field nature of the collaboration at the basis of the CO.R.E. project.

In particular, we investigated the specific goals and expectations of each partner participating in the project, analysed their level of homogeneity/divergence, and explored how these have possibility changed during the course the project activities and how they have shaped the functioning, and sometimes the malfunctioning, of the project.

22 of 24

23 of 24

The discrepancy between initial expectations and what is actually possible to realise became more evident once the consortium started detailed discussions about how much access to the data could be granted to journalists, how the dashboard could be navigated, and how the feedback of journalists could be included in the design of the indicators…

The inclusion of journalists’ perspective was particularly supported by those partners who were more familiar with the journalistic professional cultures. The integration of journalists' skills, needs and wishes was achieved through specifically designed activities included in certain work packages and translated into formal project deliverables.

24 of 24

Thank You

We need you!

We want to further collect feedback on the indicators and tools developed within the project. To participate in the testing and feedback phase and use the tools in your work, please reach out to us.

Alessio Cornia

alessio.cornia@dcu.ie

Dimitri Bettoni�dimitri.bettoni@dcu.ie