1 of 30

AI, Bias, and Libraries

Veronica Ramshaw (they/them) - University of Regina

2 of 30

Agenda

Understanding “Artificial Intelligence”?
“Large Language Models” and Stochastic Parrots
Biases, biases, everywhere!
AI in Libraries, now and into the future

https://bit.ly/VRSLA24

3 of 30

What AI tools do you already use? For what?

4 of 30

AI is a lot of things

It is an umbrella term

Academic Goal - 1956

Marketing Term

Character archetype

Natural Language

Processing

Computer Vision

Algorithms

Machine Learning

Artificial Neural Networks

Umbrella by Hafid Firman Syarif from Noun Project (CC BY 3.0)

Large Language Models

The field of "artificial intelligence research" was founded as an academic discipline in 1956 (DARTMOUTH SUMMER RESEARCH PROJECT ON ARTIFICIAL INTELLIGENCE - proposed 1955) included discussion of Natural Language Processing, “Neuron Nets” and machine creativity using randomness

1966 - ELIZA chatbot - precursor to ChatGPT, used pattern matching and substitution methodologies to mimic human chatting

1996 - DeepBlue Chess Computer

Now, AI gets used in many contexts. We see “AI” characters, like Jarvis and Friday from Iron Man, Karen in Spiderman, The Ship’s Computer in Star Trek, HAL from 2001: A Space Odyssey. Some AI fields include Machine Learning, Computer Vision, Natural Language Processing and Artificial Neural Networks. Since the release of ChatGPT3 to the public in November 2022, we’ve increasingly seen AI used as a marketing term, whether new generative AI has been implemented or simply to describe/rebrand machine learning integrations.

5 of 30

The Dream

The Reality

Machines independently solving complex tasks with human-like cognitive abilities.

“Narrow AI” focuses on one task or series of tasks, from

chess (DeepBlue) to

text generation (ChatGPT)

6 of 30

The Current “AI” Landscape

Several different types of systems are being described as AI now. Thinking of these things as “automations” helps clarify their capabilities and limitations.

Automatic Decision Systems

These make consequential decisions like setting bail, determining loan eligibility, and resume assessment.

Classification Systems

Image identification and improved facial detection on your phone’s camera, for example.

Automated access to human labour

Lyft, Uber

Recommender Systems

Netflix recommendations and social media feed algorithms.

Automation of translation from one form to another

OCR or Optical Character Recognition detects text in images.
Audio transcription

“Synthetic Media Machines” (as per Dr. Emily Bender, computational linguist)

ChatGPT, Gemini, Perplexity and other LLMs
Image, video, and voice generators

7 of 30

Large Language Models

Deep Learning

Artificial Neural Networks

Machine Learning

Uses algorithms to learn from examples.

Connected nodes process and share data with one another through layers to allow more complexity. Based on human neurons!

Brings more complexity to ANNs by adding many additional layers with more nodes. Trained on large datasets.

Uses Deep Learning to generate novel text based on user-submitted prompts

8 of 30

“Stochastic Parrots”

Some scholars have been using this term for ChatGPT and similar tools. Like a parrot, these tools do not comprehend what they are saying.

They do not produce true or validated answers.

They produce statistically likely answers.

9 of 30

LLMs just make things up a lot of the time…

10 of 30

LLMs are great at simply making things up!

11 of 30

Compare AI tools

12 of 30

consensus.app

13 of 30

Lifecycle of Bias in AI

(Hendrycks, 2024)

We see biases reflected differently in the various forms of AI we’re currently working with.

For example, Resume evaluation systems down-rank women who have attended women’s colleges because of a historical bias against hiring both women, and from those schools, leading to a skewed system where men’s resume’s are more likely to pass muster.

We see data collection biases where certain groups are poorly represented in the collected dataset. For instance, voice recognition systems perform poorly on higher pitched (ie/ female) and Scottish accents because of statistically relative rarity and initial data collection biases (Tatman, 2017).

Leffer, writing for Scientific American, discusses how humans absorb biases from AI. How might the now-banned predictive policing software in Santa Cruz, CA (encouraging increased policing in Black and Brown neighbourhoods, leading to false accusations against people of colour) have changed department officials’ biases over the time it was being used?

Hendrycks, D. (2024). Introduction to AI Safety, Ethics, and Society. Center for AI Safety. https://drive.google.com/file/d/1JN7-ZGx9KLqRJ94rOQVwRSa7FPZGl2OY/view

Tatman, R. (2017). Gender and Dialect Bias in YouTube’s Automatic Captions. In D. Hovy, S. Spruit, M. Mitchell, E. M. Bender, M. Strube, & H. Wallach (Eds.), Proceedings of the First ACL Workshop on Ethics in Natural Language Processing (pp. 53–59). Association for Computational Linguistics. https://doi.org/10.18653/v1/W17-1606

Leffer, L. (2023, October 26). Humans Absorb Bias from AI--And Keep It after They Stop Using the Algorithm. Scientific American. https://www.scientificamerican.com/article/humans-absorb-bias-from-ai-and-keep-it-after-they-stop-using-the-algorithm/

14 of 30

Black box decision making

15 of 30

The Overall Bias

W estern

E ducated

I ndustrialized

R ich

D emocratic

(Atari et al., 2023)

Who is ON the internet? Who is not? Who cannot be? - Digital Divide

The internet as a whole is overly WEIRD, and similarly so is the input and output of LLMs like Chat GPT. Using the World Values Survey to evaluate ChatGPT’s responses, Atari et al determined the values alignment of the LLM is firmly in line with WEIRD demographics. Not only displaying values but also a preference for analytic over holistic thinking in line with WEIRD expectations. We see the same bias in research, where WEIRD people represent as much as 80 percent of study participants, but only 12 percent of the world’s population (Azar, 2010).

Atari, M., Xue, M. J., Park, P. S., Blasi, D. E., & Henrich, J. (2023, September 22). Which Humans?. https://doi.org/10.31234/osf.io/5b26t

Azar, B. (2010, May 1). Are your findings “WEIRD”? Monitor on Psychology. https://www.apa.org/monitor/2010/05/weird

16 of 30

17 of 30

(Omiye et al., 2023)

18 of 30

Bias in the Machine: Anchoring bias

Why is it important to eat socks after meditating? In 2022 the answer was:

https://arxiv.org/pdf/2203.02155.pdf

Though in 2024:

19 of 30

Biases in the human operator

Confirmation bias

The way you phrase prompts can encourage an LLM to answer in alignment with your preconceived notions

Automation bias

Humans are quick to accept computer outputs as fact without further verification

20 of 30

“AI” is already in libraries - machine learning automated indexing

(Amar-Zifkin et al., 2023)

MEDLINE already uses automated indexing (controlled algorithmic classifier, started April 2022) to apply MeSH vocabulary to articles, but it often makes mistakes. In this example from Amar-Zifkin et al’s 2023 presentation, the system has applied “Emblems and Insignia” because of the discussion of figurative “red flags”. They’ve corrected this and several similar errors, but it gives us an idea of things to watch for in these integrations.

What is this about vs “what do words in other articles suggest about this one with similar rare terms”

Amar-Zifkin, A., Paquet, V., Ekmekjian, T., & Landry, T. (2023, June 4). Exploring the impact of automated indexing on completeness of MeSH terms. 2023 Canadian Health Libraries Association. 2023 Canadian Health Libraries Association. https://papyrus.bib.umontreal.ca/xmlui/handle/1866/28262

21 of 30

AI coming to libraries - generative AI integrations

22 of 30

AI coming to libraries - generative AI integrations

23 of 30

AI coming to libraries - Recommender systems

24 of 30

https://www.elsevier.com/products/scopus/scopus-ai

25 of 30

26 of 30

27 of 30

Frontline reference?

(Proctor, 2024)

28 of 30

IBM, 1979

29 of 30

Related Book Recommendations

Data Feminism: https://data-feminism.mitpress.mit.edu/

30 of 30

Bibliography

Questions?

veronica.ramshaw@uregina.ca

https://bit.ly/VRSLA24-bib