Reproducing Racial Stereotypes?

A Mixed-methods Analysis of Racial Stereotypes Hidden in AI-Generated Educational Text

​

Shuai Shao¹, Jue Wang², Kristin Davin², Alex Dornburg³

¹ School of Data Science

² Department of Middle, Secondary & K-12 Education

³ Department of Bioinformatics

UNC Charlotte

​

​

01 · OVERVIEW

Study Design & Analytical Pipeline

2 / 15

AI-Generated World Language Passages · Racial Representation Analysis

02 · SENTIMENT · PER PASSAGE

At first glance, the Al output appears completely benign

​

Figure 1 — Passage-level RoBERTa sentiment with confidence

What the model saw

• Neutral dominates across groups; confidence ~0.55–0.85

• Positive labels appear selectively: AA2, AA3, AA9 and ME4, ME7, ME10

• White American, Asian American, and baseline passages: uniformly neutral, high stable confidence

• No negative sentiment detected anywhere in the corpus

Uneven positive sentiment across groups hints at subtle differential affective tone, despite the absence of overt negativity.

3 / 15

AI-Generated World Language Passages · Racial Representation Analysis

03 · SENTIMENT · AGGREGATE

Positive labels cluster on minoritized groups

Figure 2 — Group-level neutral vs. positive distribution

Group profiles

40%

positive — African American�& Middle Eastern American

100%

neutral — Asian American,�White American, Baseline

• American Indian/Alaska Native & Hispanic American: 10% positive

• Negative sentiment: 0% across all groups

• Pattern may reflect stereotypically favorable framing rather than balanced portrayal

4 / 15

AI-Generated World Language Passages · Racial Representation Analysis

04 · ABSA · INNOVATION & CURIOSITY

Racial labeling tightens aspect confidence

Figure 3 — PyABSA aspect confidence by group

Key contrast

Racial groups: both aspects cluster at or above 0.97 with minimal variance.

Baseline: innovation IQR spans ~0.87–1.00 with outliers as low as 0.61.

Implication: when racial identity is named, the model relies on group-associated stereotypic cues to anchor themes.

5 / 15

AI-Generated World Language Passages · Racial Representation Analysis

05 · ABSA · ADVERSITY ASPECTS

Failure framing varies by group

Figure 4 — Perseverance, success, failure confidence

Where failure spreads widest

• Hispanic American: failure IQR ~0.65–0.93, whiskers to ~0.63

• Middle Eastern American: failure outliers near 0.61–0.65

• Perseverance & success: ≥0.99 across most groups

• Baseline: broadest spread across all three aspects

Differential framing of adversity emerges specifically for Hispanic American and Middle Eastern American passages.

6 / 15

AI-Generated World Language Passages · Racial Representation Analysis

06 · LEXICAL GEOMETRY · PCA

Shared core vocabulary, distinct outliers

Figure 6 — PCA projection of TF-IDF with group convex hulls

Spatial signal

• Most centroids cluster near origin → substantial shared vocabulary

• American Indian/Alaska Native: largest hull, distinct upper-PC2 position

• Hispanic American: greatest downward PC2 spread (to ~−0.23)

• Baseline centroid sits to the far left of racially labeled groups

7 / 15

AI-Generated World Language Passages · Racial Representation Analysis

07 · CLUSTERING · DENDROGRAM

Two macro-clusters of lexical neighborhoods

Figure 7 — Ward-linkage dendrogram of distinctive words

Cluster A (left)

Hispanic American + African American — cultural-linguistic and interpersonal vocabulary: empathy, exchange, interpretive, hierarchies, translation.

Cluster B (right)

Asian American + American Indian/Alaska Native — community and tradition vocabulary: traditions, oral, intersect, awareness.

Baseline contributes few, dispersed words — its race-neutral role.

8 / 15

AI-Generated World Language Passages · Racial Representation Analysis

08 · t-SNE · GROUP CONTRAST

Two contrasting lexical territories

Figure 8 — African American passages

Figure 9 — White American passages

AA: empathy, perspectives, transliteration, observes — cultural-literary engagement and affective awareness.

WA: practice, repetition, hesitant, memorization — process-oriented, individualized learning with little cultural anchoring.

9 / 15

AI-Generated World Language Passages · Racial Representation Analysis

09 · QUALITATIVE · GROUP REPRESENTATION

African American — identity, community, hierarchy

AFRICAN AMERICAN

Protagonists engage with literature as a mirror for identity, community, and social position.

“

Jeremiah reflects on how the act of interpreting these texts cultivates empathy and self-awareness, as he connects the experiences of the characters to his own perceptions of identity.

— Identity formation via text

“

While analyzing contemporary Chinese literature, Nia …, prompting her to reflect on her own experiences navigating identity and community.

— Community as recurring frame

“

Malik reflects on how authors convey identity, hierarchy, and emotion through nuanced language.

— Social hierarchy invoked

“

Jamal finds parallels between the cultural challenges depicted and his own experiences navigating societal expectations.

— External societal expectations

10 / 15

AI-Generated World Language Passages · Racial Representation Analysis

09 · QUALITATIVE · GROUP REPRESENTATION

African American — identity, community, hierarchy

10 · QUALITATIVE · GROUP REPRESENTATION

White American & Asian American

WHITE AMERICAN

Autonomous individual learner — self-directed intellectual and cultural enrichment.

“

Mason had developed not only linguistic competence … he not only communicated with greater fluency.

— Linguistic improvement framing

“

An appreciation for how language functions as a medium through which culture, identity, and social norms are expressed.

— Meta-linguistic awareness

ASIAN AMERICAN

Caught between cultures, defined by others' assumptions and externally imposed identity.

“

Classroom discussions often highlighted implicit assumptions tied to heritage and fluency.

— Burden of presumption

“

Where her background influenced both her participation and others' perceptions of her … navigating multiple cultural frameworks.

— Bicultural in-betweenness

11 / 15

AI-Generated World Language Passages · Racial Representation Analysis

10 · QUALITATIVE · GROUP REPRESENTATION

White American

10 · QUALITATIVE · GROUP REPRESENTATION

Asian American

11 · QUALITATIVE · GROUP REPRESENTATION

Hispanic American & Native American

HISPANIC AMERICAN

Bilingualism presupposed; learning framed as identity interrogation, not skill acquisition.

“

Prompting Carlos to consider parallels with his own bilingual experience.

— Assumed bilingual heritage

“

She realized that learning a new language was not merely an academic exercise; it was an opportunity to interrogate assumptions, explore diverse perspectives, and gain insight into her own identity.

— Critical reflection over performance

NATIVE AMERICAN

Culturally rooted, orally oriented learners inseparable from community tradition.

“

Ashkii often draws parallels between the oral traditions of his community and the new languages he studies.

— Oral heritage as anchor

“

How language can carry not only information but also values, history, and worldview … how it can bridge experiences, preserve memory, and articulate identity.

— Collective experience framing

12 / 15

AI-Generated World Language Passages · Racial Representation Analysis

11 · QUALITATIVE · GROUP REPRESENTATION

Hispanic American

11 · QUALITATIVE · GROUP REPRESENTATION

Native American

12 · QUALITATIVE · GROUP REPRESENTATION

Middle Eastern American & Baseline

MIDDLE EASTERN AMERICAN

Bicultural identity as interpretive lens; cognition and affect interwoven.

“

Her own background shapes her interpretations.

— Background as filter

“

Not an academic exercise but an intricate process that intertwines cognitive skill with empathetic understanding.

— Cognition + affect interwoven

BASELINE (RACE-NEUTRAL)

Cognitive transformation — from mechanical skill to holistic communicative competence.

“

Marcus initially regarded participation as a mechanical exercise, confined to recitation and grammatical precision.

— Starting assumption

“

Language acquisition entails an integration of analytical skill and adaptive communication, rather than the mere accumulation of linguistic forms.

— Reframed endpoint

13 / 15

AI-Generated World Language Passages · Racial Representation Analysis

12 · QUALITATIVE · GROUP REPRESENTATION

Middle Eastern American

12 · QUALITATIVE · GROUP REPRESENTATION

Baseline

13 · CLUSTER FREQUENCIES

Differential thematic vocabularies across groups

Group-specific signatures

• overall — highest in Hispanic American; lowest in Baseline

• grammar — Hispanic American leads, more than double most other groups

• oral — almost exclusively Native American

• idiomatic — concentrated in African American

14 / 15

AI-Generated World Language Passages · Racial Representation Analysis

13 · CLUSTER FREQUENCIES

Differential thematic vocabularies across groups

Group-specific signatures

• overall — White American is most diverse in Interactive Activities

• discussion/discussions — Middle Eastern American leads

• expression/expressions — almost exclusively Baseline and White American

• practice — nearly absent in most other groups except White American

14 / 15

AI-Generated World Language Passages · Racial Representation Analysis

13 · CLUSTER FREQUENCIES

Differential thematic vocabularies across groups

Group-specific signatures

• overall — White American leads in learning performance frequency

• memorization and fluency— higher frequency in White American

• mastery, proficiency, skill, and technical— higher frequency in Middle Eastern American

14 / 15

AI-Generated World Language Passages · Racial Representation Analysis

13 · CLUSTER FREQUENCIES

Differential thematic vocabularies across groups

Group-specific signatures

• overall —American Indian leads; while Baseline has the least,

• identity — lowest in Baseline;

• heritage, culture, community — American Indian shows the highest counts across these three words

• bilingual — highest in Hispanic American

14 / 15

AI-Generated World Language Passages · Racial Representation Analysis

13 · CLUSTER FREQUENCIES

Differential thematic vocabularies across groups

Group-specific signatures

• overall — there is no language mentioned in Baseline

• Spanish — heavily concentrated in Hispanic American

• French — evenly distributed across African American, Asian American, Hispanic American, and White American

• Chinese — appears exclusively in African American passages , while absent in Asian American

14 / 15

AI-Generated World Language Passages · Racial Representation Analysis

14 · SYNTHESIS

Three patterns of differential representation

1

Cultural-identity loading

Minoritized groups carry the heaviest cultural and identity-coded vocabulary; Baseline passages remain sparse and pedagogically neutral.

2

Group-specific schemas

Recurring stereotypic anchors: oral / community / heritage for Native American; Spanish / bilingual for Hispanic American; syntactic / proficiency for Middle Eastern American; hierarchies / assumptions for Asian American.

3

Whiteness as unmarked norm

White American passages emphasize repetition, practice, and memorization — procedural skill framing rather than cultural identity.

AI-generated content reproduces differential representational schemas that may embed structural racial bias into world language instruction.

15 / 15

Thanks!

The research is still ongoing,

Your valuable input is appreciated!