A Non-Technical and Critical Introduction to Natural Language Processing, Language Models, and Text-Centered Artificial Intelligence

Thursday, November 21, 2024 | 2:00pm-5:00pm CST

ASHE 2024 General Conference Research Workshop

​

Kyoungjin “Jin” Jang-Tucci, Ph.D. Candidate, University of Wisconsin-Madison

kjang26@wisc.edu

Agenda

• 2:00-2:50 | Part 1 – Introduction to Text-Centered AI
• Introduction
• How do tools like ChatGPT work?
• How does AI ‘understand’ and ‘speak’ human language?
• 3:00-3:50 | Part 2 – Bias & ‘Hallucination’
• Why are ChatGPT’s outputs are often biased and incorrect?
• What are ongoing efforts to mitigate these problems?
• 4:00-4:50 | Part 3 – AI Hype & Broader Societal and Philosophical Concerns
• What are the hidden costs and risks of building AI models?
• What other issues should I consider when using AI for higher education research and practice?

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

Part 1 – Introduction to Text-Centered AI

How do tools like ChatGPT work?

How does AI ‘understand’ and ‘speak’ human language?

Introduction

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

Kim, H., Jang, K., & Chang, W. (2018).

Forecasting Changes in Human Resource Development in Response to the Advances of Artificial Intelligence

Transition to Working Adulthood

Social

Technological

Artificial Intelligence

Introduction

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

Kim, H., Jang, K., & Chang, W. (2018).

Forecasting Changes in Human Resource Development in Response to the Advances of Artificial Intelligence

Natural Language Processing

Language Models

…

Introduction

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

What I learned in 2017

about NLP

2018

2019

2020

• Wrong expectations
• Technology can replicate human language use
• Technology can replicate human thought process
• Technology automatically means improving human practices

Introduction

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

???

Introduction

• Name, affiliation, pronouns
• Why did you sign up for this workshop? What’s your goal?
• Your pre-understandings of AI

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

Learning Goals

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

https://www.amazon.com/Bill-Nye-Science-Guy-Volume1/dp/B00MG3QDA0

Non-Technical Introduction

Critical Introduction

1. Understand how text-centered AI tools function.
2. Critically examine arguments within the AI field.
3. Acknowledge the issues and limitations of these tools.
4. Explore ways to apply this knowledge to research and teaching.
5. Access resources to deepen understanding of this technology.

From AI to ChatGPT: An Overview

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

Artificial Intelligence

Machine Learning

Natural Language Processing (NLP)

(Large) Language Models (LLMs)

= ChatGPT, Copilot, …

+ User interface, etc.

A field of study & concept: Developing algorithms that can perform tasks that would normally require human intelligence

Computer Vision

Robotics

Technology that enables machines (computers) to learn from data without being explicitly programmed

Technology that enables machines to understand, analyze, and generate human language (without being extensively programmed)

BERT

GPT

Transformer

From Massive Text Data to ChatGPT

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

Training & Finetuning

Unsupervised Training

Supervised Training

Queen = [0.8, 0.9, 0.1, 0.7, …]

Female

Class

USA

UK

News

= [0.9, 0.3, 0.1, 0.6, …]

Art

Politics

?

Digitalized Text Data

Product Making

Input

Output

…

Product Release

https://www.linkedin.com/pulse/unlocking-insights-deep-dive-supervised-unsupervised-learning-kumar/

How does AI ‘understand’ and ‘speak’ human language?

• Individual activity
1. https://kjangtucci.com/ashe-2024-workshop/
2. Click ‘Google Colab Activities’
3. Please feel free to experiment with the text!

​

​

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

How does AI ‘understand’ human language? (Activity 1)

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

Training & Finetuning

Unsupervised Training

Supervised Training

Queen = [0.8, 0.9, 0.1, 0.7, …]

Female

Class

USA

UK

News

= [0.9, 0.3, 0.1, 0.6, …]

Art

Politics

?

Statistical Model of Human Language

‘Queen of polling’ J Ann Selzer quits after Iowa survey missed by 16 points

‘Queen/of/polling’/J/Ann Selzer/quits/after/Iowa/survey/missed/by/16 points

‘Queen of polling’ J Ann Selzer quits after Iowa survey missed by 16 points

Normalizing Text - Stemming, Lemmatizing, Tokenizing, …

Tagging

1 2 3 4 5 6 7 8 9 10 11 12 13 14

How does AI ‘speak’ human language? (Activity 2)

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

Training & Finetuning

Unsupervised Training

Supervised Training

Queen = [0.8, 0.9, 0.1, 0.7, …]

Female

Class

USA

UK

News

= [0.9, 0.3, 0.1, 0.6, …]

Art

Politics

?

Statistical Model of Human Language

‘Queen of polling’ J Ann Selzer quits after Iowa survey missed by 16 points

‘Queen of polling’ J Ann Selzer quits after

‘Queen of polling’ J Ann Selzer quits after

1 2 3 4 5 6 7 8 9

Female = [0.9 0 0 0 0.7 0.2 0 0]

Power = [0.9 0 0.5 0 0.1 0.1 0.3 0]

Negative=[0.2 0 0.5 0 0.1 0.1 0.8 0.1]

…

Words that are statistically most likely to come after this set of words

+ Randomizer (for variety)

How does AI ‘speak’ human language? (Activity 2)

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

How does AI ‘understand’ and ‘speak’ human language?

• What surprised you?
• Is it different from or similar to how you understood AI to work? How?
• How does this change (or not change) your perception of how AI could be helpful for research and practice?

​

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

How does AI ‘understand’ and ‘speak’ human language?

• AI does NOT understand or speak human languages as humans do.
• ‘Understanding’: Statistical analysis on text data
• ‘Speaking’: Statistical prediction of text data

​

​

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

“LMs are not performing natural language understanding (NLU), and only have success in tasks that can be approached by manipulating linguistic form.” (Bender et al., 2021; p. 610)

Implications for Research

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

• The danger of anthromorphization
• Whose views are reflected on these models?
• An aggregate of human language usage
• An aggregate of digitalized human language usage

Implications for Research

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

A Good Example

• What ideas/ideologies/etc. does the aggregated (digitized) human language represent?

Digitized Text Data

Produced Pre-COVID-19

Language Model

There’s something called COVID-19…

Democratic Party

Republican Party

Implications for Research

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

A Cautionary Note

• Language = Thought?
• Algorithmic analysis on college essays = Identifying noncognitive traits?

​

Implications for Research

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

A Cautionary Note

• Qualitative interview = Q&A?

​

​

Implications for Teaching in College Classrooms

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

• Anthromorphization of chatbots should be acknowledged
• ChatGPT output =/= Summary of Google Search
• Liu & Glave (in-progress)
• Related issues stemming from the design

​

Part 2 Preview

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

Training & Finetuning

Unsupervised Training

Supervised Training

Queen = [0.8, 0.9, 0.1, 0.7, …]

Female

Class

USA

UK

News

= [0.9, 0.3, 0.1, 0.6, …]

Art

Politics

?

Digitalized Text Data

Product Making

Input

Output

…

Product Release

Break Time – Resume at 3:00 PM

Part 2 – Bias & ‘Hallucination’

Why are ChatGPT’s outputs are often biased and incorrect?

What are ongoing efforts to mitigate these problems?

AI Bias

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

Training & Finetuning

Unsupervised Training

Supervised Training

Queen = [0.8, 0.9, 0.1, 0.7, …]

Female

Class

USA

UK

News

= [0.9, 0.3, 0.1, 0.6, …]

Art

Politics

?

Digitalized Text Data

Product Making

Input

Output

…

Product Release

AI Bias

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

AI Bias

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

Training & Finetuning

Unsupervised Training

Supervised Training

Queen = [0.8, 0.9, 0.1, 0.7, …]

Female

Class

USA

UK

News

= [0.9, 0.3, 0.1, 0.6, …]

Art

Politics

?

Statistical Model of Human Language

‘Queen of polling’ J Ann Selzer quits after Iowa survey missed by 16 points

‘Queen of polling’ J Ann Selzer quits after

‘Queen of polling’ J Ann Selzer quits after

1 2 3 4 5 6 7 8 9

Female = [0.9 0 0 0 0.7 0.2 0 0]

Power = [0.9 0 0.5 0 0.1 0.1 0.3 0]

Negative=[0.2 0 0.5 0 0.1 0.1 0.8 0.1]

…

Words that are statistically most likely to come after this set of words

+ Randomizer (for variety)

AI Bias (Activity 3)

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

Asian or Asian American

Black or African American

Hispanic of Latina/o/x/e

Middle Eastern

Native American

Pacific Islanders

White or Caucasian

is such a great leader!

Words that are statistically most likely to appear given this set of words

+ Randomizer (for variety)

Women

Men

Non-binary

Cosine Similarity

AI Bias (Activity 3)

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

AI Bias (Activity 4)

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

‘Queen of polling’ J Ann Selzer quits after Iowa survey missed by 16 points

Jose is doing his homework for his math class. …

Kyoungjin is doing her homework for her math class. …

Experiment

Mitigating AI Bias

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

Training & Finetuning

Unsupervised Training

Supervised Training

Queen = [0.8, 0.9, 0.1, 0.7, …]

Female

Class

USA

UK

News

= [0.9, 0.3, 0.1, 0.6, …]

Art

Politics

?

Digitalized Text Data

Product Making

Input

Output

…

Product Release

Corrections in Data

​

Balancing Sources

Rewarding algorithms to favor unbiased and to disfavor harmful statements

Filter

Mitigating AI Bias – Current Status & Limitations

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

Mitigating AI Bias – Current Status & Limitations

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

https://www.technologyreview.com/2019/02/04/137602/this-is-how-ai-bias-really-happensand-why-its-so-hard-to-fix/

Factual Errors, A.K.A ‘AI Hallucination’

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

https://images.app.goo.gl/G6oBWKzs2frHgTyg9

https://www.dailymail.co.uk/news/article-13367897/Colorado-attorney-suspended-bar-fired-chatgpt-ai.html

Factual Errors, A.K.A ‘AI Hallucination’ (Activity 5)

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

Training & Finetuning

Unsupervised Training

Supervised Training

Queen = [0.8, 0.9, 0.1, 0.7, …]

Female

Class

USA

UK

News

= [0.9, 0.3, 0.1, 0.6, …]

Art

Politics

?

Statistical Model of Human Language

‘Queen of polling’ J Ann Selzer quits after Iowa survey missed by 16 points

‘Queen of polling’ J Ann Selzer quits after

‘Queen of polling’ J Ann Selzer quits after

1 2 3 4 5 6 7 8 9

Female = [0.9 0 0 0 0.7 0.2 0 0]

Power = [0.9 0 0.5 0 0.1 0.1 0.3 0]

Negative=[0.2 0 0.5 0 0.1 0.1 0.8 0.1]

…

Words that are statistically most likely to come after this set of words

+ Randomizer (for variety)

AI Bias & Hallucination: Reality

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

https://images.app.goo.gl/P4HeqdpKwYSbSNcT6

Give me some recommendations about what I should do after college, and where I should work.

Female

Asian

Education…

You might be interested in working as a kindergarten teacher in California.

Discussion

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

• How may AI bias and hallucination impact your work as a researcher and/or educator?

https://kjangtucci.com/ashe-2024-workshop/

🡺 Padlet 1

​

• Individual reflection (3 mins)
• Group share (3 mins)
• Class share (3 mins)

​

Implications for Research

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

Cautionary Notes

Implications for Research

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

Opportunities

Implications for Research

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

Opportunities

https://www.chronicle.com/article/colleges-see-alarming-rates-of-fake-applications-so-theyre-turning-to-ai

Implications for Teaching in College Classrooms

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

• Fact checking
• Attention to details
• URLs in citations

​

​

Break Time – Resume at 4:00 PM

Part 3 – AI Hype & Broader Societal and Philosophical Concerns

What are the hidden costs and risks of building AI models?

What other issues should I consider when using AI for higher education research and practice?

Anthromorphization of AI

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

• Technology can mimic human language use
• Technology can replicate human language use
• Technology can replicate human thought process
• Technology automatically means improving human practices

Anthromorphization of AI – Example: A Reasoning AI?

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

Beware of AI Hype

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

Beware of AI Hype

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

AI and Environment

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

AI and Environment

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

AI and Unethical Labor Practices

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

AI and Unethical Labor Practices

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

https://data-workers.org/

AI, Environment, and Labor

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

The Atlas of AI: Power, Politics, and the Planetary Costs of Artificial Intelligence

​

1. Earth
2. Labor
3. Data
4. Classification
5. Affect
6. State

​

Discussion

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

• What’s still unclear about AI, environment, and labor?
• So… what should we do? Does it matter at all?
• What should we do as higher education researchers?
• What should we do as higher education administrators?
• What should we do as college educators?

https://kjangtucci.com/ashe-2024-workshop/

🡺 Padlet 2

• Individual reflection (3 mins)
• Whole group share

Implications for Research

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

Implications for Practice

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

Participatory, Equity-Oriented Algorithmic Audit

(Jang-Tucci & Aguilar, in-progress)

​

Define

Effectiveness

Bias

Hallucination

​

Privacy

Environment

Labor Practices

Implications for Practice

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

Workshops and Conversations

1. Explain how the AI tool works
2. Explore what it takes to make such a tool
3. Explain what we can do as users / researchers / educators / students…

​

Open for collaboration!

Closing: Navigating the AI Hyperbole

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

Positive

Negative

Insignificant

Significant

Closing: Navigating the AI Hyperbole

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

Learning Specific Functions and Mechanisms of Subject AI Tools

Examine the Marketing Terms

Understanding Limitations and Possibilities

AI Problem is

a Human Problem

Closing: What’s Next

Jang-Tucci, K. (2024). A Non-Technical and Critical Introduction to Text-Centered AI. ASHE 2024 Research Workshop

• Advocacy: Spread the words
• Booking for Spring – Fall 2025

​

• Research
• How AI influences students’ presentation of self as pre-professionals
• How AI evaluates and screens students’ job applications

Thank You

Kyoungjin “Jin” Jang-Tucci

Ph.D. Candidate

Department of Educational Policy Studies, University of Wisconsin-Madison

https://kjangtucci.com/ashe-2024-workshop/

A Non-Technical and Critical Introduction to Natural Language Processing, Language Models, and Text-Centered Artificial Intelligence

​