1 of 22

A Just Transition: ��What does it mean �for AI and Evaluation?

Linda Raftree,

Founder, The MERL Tech Initiative

European Evaluation Society

Autumn Online Event

November 16, 2023

All illustrations in this deck were created using ChatGPT-4/DALL-E

Good afternoon and thank you so much for inviting me to talk today. I’m Linda Raftree, founder of the MERL Tech Initiative. The MERL Tech Initiative has been around since 2014, when evaluation was starting to think about its intersection with emerging technologies like mobile phones and crowdsourcing.

MERL Tech is monitoring, evaluation, research, learning (M-E-R-L) + Tech. We explore the intersection of MERL and Tech - both Tech for doing MERL, and MERL on Tech. We have a cross-cutting focus on responsible use of technology and data in programs and in MERL.

We help the MERL community to “meet the moment” through work to

bring together a range of disciplines - from evaluators to researchers to data scientists to sector experts to tech developers
democratize access and understanding of emerging technologies
help MERL Practitioners understand tech+MERL intersections
and situate MERL and Technology across different adjacent sectors

Back in 2014 the moment was mobile phones – today we are thinking about Generative AI.

2 of 22

Generative AI

in the spotlight

Why is AI in the spotlight now? Generative AI hit the world about a year ago with the release of a publicly available chatbot called Chat GPT. (Others are racing to release theirs too). Generative AI does what the name says - it generates new content.

Chat GPT is built on something called a Large Language Model. Very simply, Chat GPT is like a giant encyclopedia - trained on huge troves of data scraped from the Internet and digital books and a few other sources. Natural Language Processing - NLP - allows a LLM to be queried and the LLM will return back ‘answers’. It finds patterns in language and predicts the most likely next word in a sentence.

Other examples of Generative AI would be the tool that I used to create all the illustrations in this presentation - I imagined what I wanted to illustrate and provided DALL-E with a description and within a few minutes, based on a huge database of images from the Internet, it generated options that I used or tweaked. I’ll note that it’s pretty terrible at spelling, so apologies for that.

The possibilities and the challenges of emerging AI are literally keeping me up at night and as a community of evaluators, this should be keeping you up too!

If the purpose of EES and other evaluation societies is to inform, educate, and debate on the one hand, and to influence mindsets, practices, policy, and resource allocation on the other, it’s clearly critical that evaluators have a point of view about AI and how it affects our collective futures.

3 of 22

Narratives �about �our AI future

4 of 22

Utopian

Once upon a time AI replaced us.

We didn’t have to work anymore.

We all lived a life of leisure,

with time to relax and pursue creative endeavors

while AI resolved our health and other issues.

Our jobs no longer defined us.

AI generated so much wealth

that all human needs were fulfilled.

The nice corporations

who captured this wealth from our data

distributed it back to those of us who were deserving

in the form of a universal basic income.

We all lived happily ever after

in peace and harmony.

Sam Altman’s: Moore’s Law for Everything

Marc Andreesen’s Techno-Optimist Manifesto

There’s a very utopian version.

Some of the authors of this utopian vision include OpenAI/ChatGPT’s Sam Altman’s (see Moore’s Law for Everything) and venture capitalist Marc Andreesen (see Techno-Optimist Manifesto). In these narratives, man is full of endless creativity and can bend the planet to his desires through his inventiveness. Wealthy tech entrepreneurs extend their charitable generosity to the rest of us.

This utopian vision stems from Silicon’ Valley’s belief in the glory of colonial days, free market capitalism and, I might add, a strong belief in its own superiority. (See Emile Torres on “TESCREAL” for a longer explanation of how some of these beliefs appear to coalesce into a shared world view)

Techno-optimism is also in opposition to “AI Doomerism”. “Doomerism” is the belief that AI will soon attain “artificial general intelligence” and destroy the world and humans. It’s part of the “Effective Altruism” movement, which believes that carefully created artificial-intelligence systems, designed with the correct human values, will yield a Golden Age—and failure to do so could have apocalyptic consequences. (Updated on Nov 27, 2023: Doomerists are purportedly behind the chaos at OpenAI in late November 2023. More on Effective Altruism here.).

Both Doomerists and Optimists support a narrative that Silicon Valley will create AI that has immense powers.

5 of 22

Dystopian

Once upon a time AI replaced us.

We didn’t have to work anymore.

Corporations captured all the wealth

generated by their AI creations

and decided they no longer needed us.

We lost our say in collective decisions

because we no longer contributed to the economy or the defense of our nations.

�Protest was stifled because we were surveilled

by biometrics, cameras,

robot dogs, and automated weapons.

Inequalities and -isms were cemented

and the uber wealthy lived happily ever after.

See: Aksel Braanen Sterri: AI and the Transition Paradox and Unleashing Possibilities, Ignoring Risks. Why we need tools to manage AI’s impact on jobs.

6 of 22

Realist

AI is both mind blowingly useful and incredibly harmful.

We don’t understand it fully.

The benefits and harms aren’t equally distributed

AI that is entirely driven by capitalism is leading to increased inequality.

We need to get involved in shaping the AI future.

We need to understand the narratives and theories behind the transition to AI.

There’s also a middle way that requires us to see complexity and nuance. These futures will live side by side and will have differential effects depending on who you are. We need to hold complexity and nuance because AI can be both mind-blowingly useful and terribly harmful. We don’t understand it fully. Its benefits and harms aren’t equally distributed. AI that is entirely driven by capitalism is leading to increased inequality.

We can look at what AI has and is doing right now, and think about how we can play a role in addressing past and current harms as a way to better shape the future. AI ethics researchers like Timnit Gebru, Margaret Mitchell, Emily Bender, Cathy O'Neil and Safiya Noble have called out AI futures thinking as a distraction from AI presents. Meaning that the collective fear of sentient robots (e.g., Doomerism) is distracting us from looking at how we might shape AI right now and mitigate the harms it’s doing right now.

I think we need to think about both right now and the future. We need to look at technicalities and practicalities while also looking at the wider systems and their implications.

As evaluators, it’s our imperative to understand the theories and narratives behind the stories that we are told about technology and the future. AI as something that’s both great and terrible. As evaluators, it’s critical that we unpack this.

7 of 22

The �mixed bag of AI utility

8 of 22

AI as “a multiplier of human ingenuity”

-Demis Hassibis

9 of 22

AI as “an amplifier of human intent”

-Kentaro Toyama

Over a decade ago, when the innovation and “ICT for Development” hype was at an all time high, Kentaro Toyama - the author of Geek Heresy - described digital technology as simply “amplifying human intent and behavior” - meaning that technology allows us to scale both good and bad.

Some of the negative aspects of AI are purposeful. Some are fallout, collateral damage of the rapid push to AI in a capitalist, consumerist world. But theories and political analysis are important for understanding human intent. Is it neo liberal? repressive or authoritarian? socialist or structuralist intent? (See Development Transformation as the Goal for Digital Transformation)

Real world harms that are already happening with AI include:

Mis- and disinformation, deep fakes/non consensual AI porn, loss of a shared truth, democracy challenges -

the organisation WITNESS notes that the Liar’s Dividend allows governments to use the existence of fake videos to say that real video footage has been manufactured and skirt responsibilities
disinformation about what is really happening in conflicts through creation of what look like real images is fueling greater violence

Harmful labor markets - OpenAI contracted a firm in Kenya to sort through horrible content at low wages to make things safe for users
Biased automated decision-making and biased responses - AI making decisions about who does and doesn’t receive social benefits or loans for example
Loss artistic creativity, ‘mediocritization’ - the data sets that fuel large language models are largely Western and English speaking, male dominated - they don’t understand different types of English for example - what does that mean - what might it lead to? What is prioritized?
War and violence - we have robot dogs in New York City - and wars are being fought with AI right now, today. They are already fought in a video game style that enables distance from the actual horrors.
Privacy loss, increased surveillance - faulty facial recognition tools have already led to wrongful arrests of Black people in the US and UK

10 of 22

What about the use

of AI for evaluation?

Despite these deep rooted challenges with AI, we’re also seeing that AI can have some level utility for our work. (note - ChatGPT is not good at spelling…. ) Evaluators are interested in these emerging AI tools. And there are plenty of efforts aimed at trying to make them useful for the kinds of tasks we do as evaluators.

At the AEA and NetHope conferences in October, the sessions that included “AI” in the title had people lined up out the door. The 2024 AfrEA conference theme is “Technology and Innovation in Evaluation Practice in Africa: The Last Nail on the Coffin of Participatory Approaches?” Our NLP Community of Practice has grown to 350 people in just a year.

Some of us have been thinking about what evaluation tasks will be affected by AI. We predict that it will be those which do not require face-to-face engagement - ⅔ estimated of tasks and 35% overall reduction in effort for typical evaluation tasks. (See our New Directions for Evaluation Paper: LLMs and Evaluation: opportunities and challenges)

It’s being used in evaluation for:

Theory of change testing
Log frame brainstorming
Translation and transcription
Text summarization, classification, rewriting for different audiences
Sentiment analysis and topic modeling
Writing code for Python and R
Data visualization

11 of 22

Concerns

about using AI

for evaluation

The way that these emerging AI systems work is that they train on large data sets. We know that these data sets are largely based on the Internet and digital books. We know that there is incredible bias there. Some of us are working to try and make these tools fit for purpose, but we are finding a lot of challenges, including:

Bias (racism, sexism, ethnocentrism, gender, others) and use of biased historical data
Data quality, validity, accuracy issues
Lack of theory, reliance on correlation
“Hallucinations”
Technical limitations causing real-world harm
Overall ethical concerns with AI
The theories and worldviews behind AI - is everyone clear about where they come from?
Credibility of evaluation processes and credibility of evidence
Alienation of evaluators from evaluation craft
Automation bias: undue trusting of the output of AI-enabled evaluation more because it was automatically generated by a machine.
Transparency of evaluation process

12 of 22

A just transition to the AI future?

AI is coming fast and hard, and sometimes it feel like it’s out of our hands. But we have to remember that AI is being built, trained, modeled, tagged, and managed by humans.

It’s currently framed in a corporate, capitalist way because that is how the global economy works, for the most part, and it’s the main narrative in Silicon Valley. Many of the main players in Silicon Valley are pushing the techno-optimist narrative. Marc Andreesen, Elon Musk, Sam Bankman Fried, Sam Altman. It’s a narrative that believes in the primacy of technology and man’s ability to manipulate the environment. And when I say “man” I mean “man”. It’s a simplistic narrative about techno-fixes for everything.

In his Techno-Optimist Manifesto, Andreesen says for example: “We have a problem of poverty, so we invent technology to create abundance.” The “Effective Altruism” movement that many of these folks subscribe to – and which by its name would seem to align with the Evaluation community – is criticized for asserting that it has top-down answers to complex, local problems and that “good” can be abstractly quantified with simplistic metrics.

I’ve been wondering a lot about how the collective ‘we’ could change that.

13 of 22

AI is an �extractive economy

We’ve long been hearing that Data is the new Oil. And data is certainly an extractive economy. Now that the earth’s resources are drying up, the only thing left is to extract wealth from people’s minds and bodies - in the form of data. Data on our behaviours, our thoughts, our patterns, our purchases. The AI economy runs on data in the same way that our general economy runs on fossil fuels.

Resource extraction

Our data is extracted without our knowledge or consent by the large tech companies who build these models
There are huge climate impacts caused by the processing and using our data - the “cloud” is not actually a cloud. It’s a giant stretch of servers that use up water and energy and create environmental damage.

Consumerist and colonial worldview

AI is dominated right now by US companies, with China at a close second
AI data sets: primarily English, “Western” - that there is erasure of other cultures, contexts, and realities

Capture of wealth and power

The few, the wealthy, the powerful dominating the data extraction and AI creation process
Low-paid workers create, fuel AI wealth generation - AI is not all automated - there are real people, sitting in sweatshop-like conditions, tagging these models so that AI can work.

Militarism

Use of AI for war and government surveillance - we’re seeing this in the Israel-Hamas conflict right now

14 of 22

AI and exclusion

https://partnershiponai.org/wp-content/uploads/dlm_uploads/2023/06/pai_guidelines_shared_prosperity.pdf

The AI Economy is reinforcing a centering of wealth and power in the hands of big tech, who are reaping the benefits.

Remember the utopian vision I shared earlier? The idea is that these big tech companies will generate immense wealth and then out of the goodness of their hearts they will reallocate it back to us as a basic income. I honestly don’t believe that there is enough ‘goodness of their hearts’ for that to happen.

Rather than enable shared prosperity, the current AI model is poised to diminish it. The Partnership on AI - who have been thinking about the future of work for a long time, say that AI diminishes shared prosperity when it…

Eliminates, moves, centralizes, reallocates jobs
Makes jobs more precarious, increases barriers to entry, worsens conditions
Relies on poorly compensated or treated labor,
Reduces autonomy, and satisfaction or increases surveillance of workers
Reduces mentorship and apprentice opportunities
Operates in discriminatory ways

15 of 22

Who will be

shut out?

https://partnershiponai.org/wp-content/uploads/dlm_uploads/2023/06/pai_guidelines_shared_prosperity.pdf

If we stay on the current path, if we don’t push for a more just transition, the Partnership on AI says that (in the labor and work space) we’ll see:

Wealth consolidation in the hands of a select few companies and countries
AI will affect labor markets and the nature of work
We’ll see reduced wages and reduced worker agency as larger numbers of workers compete for deskilled, lower-wage jobs
Some job categories will shrink or disappear, new types will arise in their place
Tasks will change – some totally automated, some AI-assisted
Most fulfilling tasks in some jobs allocated to algorithms, humans remain with the drudgery
Highly disruptive spikes in unemployment or underemployment as workers start at the bottom rung in new fields, even if permanent mass unemployment does not arise in the medium term
Benefits will accrue to workers with the most control over how AI shows up in their jobs; harms will occur for workers with minimal agency over workplace AI deployments

16 of 22

Could we

shift current

AI rules and values?

But what if we, the people, were able to agree on a different set of rules for AI and for this AI transition. What if we had more control and more of a say in if, when, where, and how AI is used? And on whom it’s used? What if we could change the overall system rather than play around at the edges, focusing on the effects rather than the causes?

What if we had:

Democratized control and benefits of AI
Wide participation in decisions about AI
Shared prosperity and well-being goals
AI used to advance racial, gender, social and environmental justice and equity
Meaningful choice about engaging with AI
Accountability for use of AI

The EU and the US are developing their AI regulations right now - but much of it is focused on AI safety. Data and Society has critiqued the focus on AI Safety Regulation. They warn that “Unable to challenge AI as systems of data extraction and labor expropriation, activists may discover they hold only the administration of safety as their cudgel,” This risks “building a techno-bureaucracy that supports, not calls into question, the logic of ceaseless data expropriation.”

Civil Society data activists at IT for Change in India were just able to get mandatory sharing of anonymized data by big tech companies into a bill – they are working out how the data would be governed and how communities can access and benefit from its economic value. They are pushing for a concept of “community data” and community rights to their data in the same way that communities have a right to water, forests and other natural resources. They are also pushing back against the State claiming this data and potentially using it against the interests of citizens. See Centre considers seeking access to anonymised data of Big Tech firms | India News - The Indian Express

The CARE principles for indigenous data governance are another ideal framework for thinking about this, as is “Data Feminism”.

The point is that there are models of data governance out there that we can learn from!

17 of 22

A Just Transition

What if we adapted the just transition framework (or a similar framework) to help us think through the AI transition?

This is a framework from the Climate Justice Initiative. There are a few different frameworks

they range from more to less transformative
more to less centred on community ownership
but generally they look at how to go from

a capitalist, colonial model that focuses on extraction and militarism to
a model that focuses on wellbeing, purpose, cooperation, democracy, respect for indigenous knowledge and practices, care for people and planet

Some of these frameworks also include concepts of restoration and reparation as part of getting to regeneration. They talk about changing the rules and drawing down power from where it currently sits. About shifting economic control, democratising wealth and the workplace, advancing racial justice and social equity, and retaining and restoring cultures and traditions, including indigenous knowledge and practices.

As Evaluation Associations and Societies, we need to think about how we can help shape the future in ways that work for the majority.

There’s quite a lot of thinking happening on this already in “The Future of Work” circles. The OECD and others have developed AI principles and many are aligned with this vision of a more regenerative economy. OECD AI Principles overview

18 of 22

What role for evaluation

in a more

just transition

to AI?

What is the role of evaluation and evaluators here?

Here’s an image that DALL-E produced for me when I asked for an illustration of NGO workers in Africa using AI to conduct evaluation tasks. It’s an image of what look like robots or automatons, staring into their phones. Their shirts are tagged with the letters “NGO”. All except one lone dissenter who quietly protested, who said he would not be herded into this new AI reality…. Was this an error – or is DALL-E trying to tell us something?!

In this massive push to let corporations tell us what our AI future will be, evaluators and evaluation societies have a role in dissenting and in being thoughtful about how and where AI comes into the picture.

Where, when, how, and why should we use AI ourselves, in our field. Where can it actually help to “amplify positive intent” or “multiply human ingenuity?” How can we use it within a clear framework of values and principles? How can we bring in greater participation and reduce the biases and potential negative outcomes? How do we contribute to good policy and practice that enable a more just transition?

19 of 22

AI for evaluation

When it comes to AI and evaluation, there are a few things that evaluators can and should do:

Learn to work with AI where useful - learn and upskill!

there have been a few examples of this at the EES already, with sessions from Rick Davies, Sylvia Ferreti, James Goh, Bhavana Mishra, Steve Powell, and immediately after this talk there’s a session with Estelle Raimundo, Harsh Anuj and Virginia Ziulu
We need to be documenting, piloting, safely testing and learning, sharing and collaborating on what does and doesn’t work
What tools might be useful? Do we need bespoke tools or will AI become a seamless part of all the main platforms where we already work like Microsoft?

Be aware of and careful not to cause harm

What are the risks of incorporating AI into our practice?

The NLP CoP is working on a guidelines for responsible use of Natural Language Processing and Gen AI in Evaluation

How do we assess what our own adoption of AI might mean now, in 3 months, in 6 months in 5 years….
We need documentation of potential and real harms so that if/as we bring in AI, we’re not causing “unintended consequences”.
What is the research on AI for Evaluation agenda? How can we get it funded?
We need to honestly and openly share our concerns and mistakes/failures with each other too

Assess and prepare for how AI will affect our jobs and the evaluation economy

What does AI mean for young and emerging evaluators - are entry-level jobs going to go away? Or are there new skills that emerging evaluators need (or already have!) that prepare them for AI-enabled evaluation?
Are we going to lose out to the ‘data shops’ or will evaluation continue to be seen as valid and valuable?
What tasks can be confidently shifted to AI and what can’t be? What does that mean for how we plan and budget?
How are we going to explain to evaluation commissioners that we still need humans in the loop and we still need budgets?

Assess impacts of AI on the short- and long-term quality and efficacy of our work

Where is AI truly bringing impact and efficiency? This needs to be documented!
(Shout out to the IEG who is doing this already)

Support a transition to AI for and in Evaluation in just, equitable, participatory, accountable ways

20 of 22

Evaluation of AI

We’ll also need to know how to evaluate AI and its effects….

Do we know how to do this for AI in programs?
What about when governments integrate it?
When it comes to understanding the broad impact of AI on individuals, communities, groups, governments, societies, we have a role.
Could a just transition framework help us to assess or create participatory processes for AI adoption and assessment?

Computer scientists are constantly developing new benchmarks or tests to evaluate the performance of AI systems and models, yet there has been little role for evaluators in this process. Who is evaluating actual AI models before they are released or used?

What does our field know about evaluation and assessing actual AI models?
Should evaluators be involved?
Could we play a role in facilitating participatory involvement of people in designing or testing models for bias and accuracy?
Who do we collaborate with for that? We may not have the skills to do it on our own. How can we collaborate with data scientists?
What guidelines do we know about that could be used to help us do this?
Can frameworks like “a just transition” help us to do that?

We need to stay abreast of AI -

Understand where it’s affecting people, movements, and institutions we evaluate
Look for effects of AI when we are evaluating
Know how to evaluate AI product performance as well as the wider societal impact of AI

We can speak up for a more just and thoughtful transition to AI to reduce harms

We can be that lone voice that says “no” - let’s not be herded, unthinking, into the AI future.

21 of 22

Summing up

To sum up…

AI is here
Like the wizard of Oz, AI is both great and terrible – but don’t be fooled!

There are ordinary humans of all kinds involved - from uber wealthy Big Tech to people who are paid very little to work away behind the scenes to tag content and ensure AI works.

The “Just Transition” or other human-centred frameworks could help us engage
Evaluators have a role to play - internally and externally - and we need to be clear on our theories for what this could and should look like.
We need to engage at a systems level and we need to look at political aspects as well.
And we need to catch up or we’ll be running behind the bus both in terms of using AI in our practice and influencing the overall AI system and economy.

Evaluation societies and evaluators should have a position on this - if we’re not paying attention yet, what are we waiting for?

22 of 22

Let’s collaborate!

linda@merltech.org

The MERL Tech Initiative

http://MERLTech.org/nlp-cop