© dall-e

Why Does ChatGPT “Delve” So Much? Exploring the Sources of Lexical

Overrepresentation in Large Language Models

21 Jan 2025 @ COLING25

T.S. Juzek & Z. B. Ward

Joint work with �Zina Ward

Throughout the project great input by Gordon Erlebacher

Links

Paper: https://arxiv.org/pdf/2412.11385

Repo: https://github.com/tjuzek/delve

Background

Language changes over time

Scientific English changes over time (→ Elke Teich’s Team at Saarland University)

Examples:

Background

There have been rapid changes recently

These changes are hard to explain ‘naturally’

Background

• That this is happening, is well established

Koppenburg, 2024; Nguyen, 2024; Shapira, 2024; Gray, 2024; Kobak et al., 2024; Liang et al., 2024; Liu and Bu, 2024; Matsui, 2024; Juzek and Ward 2025

Background

• That this is happening, is well established

Koppenburg, 2024; Nguyen, 2024; Shapira, 2024; Gray, 2024; Kobak et al., 2024; Liang et al., 2024; Liu and Bu, 2024; Matsui, 2024; Juzek and Ward 2025

Broader impacts:

(virtually) unprecedented language change

Background

And early on, these changes were attributed to the influence of Large Language Models (LLMs) like ChatGPT

Background

However:

• Handpicked items
• Strengthen the link to LLMs
• Critically: not clear WHY LLMs do this
• Informed speculation �→ RLHF: Hern, 2024; Sheikh, 2024

Background

Our work:

• Procedure to systematically identify overused items
• Why are LLMs overusing certain words

The procedure:

List of factors

• Initial training data
• Fine-tuning
• Model architecture
• Choice of algorithms
• Context priming
• Learning from Human Feedback
• Other factors (parameter settings, etc.)

List of factors

• Initial training data
• Fine-tuning
• Model architecture
• Choice of algorithms
• Context priming
• Learning from Human Feedback
• Other factors (parameter settings, etc.)

possible, but no strong starting points

List of factors

• Initial training data
• Fine-tuning
• Model architecture
• Choice of algorithms
• Context priming
• Learning from Human Feedback
• Other factors (parameter settings, etc.)

List of factors

• Initial training data
• Fine-tuning
• Model architecture
• Choice of algorithms
• Context priming
• Learning from Human Feedback
• Other factors (parameter settings, etc.)

language output by Llama Base (-LHF)

vs �Llama Instruct (+LHF)

→ indicator

List of factors

• Initial training data
• Fine-tuning
• Model architecture
• Choice of algorithms
• Context priming
• Learning from Human Feedback
• Other factors (parameter settings, etc.)

language output by Llama Base (-LHF)

vs �Llama Instruct (+LHF)

→ indicator

N.B.: RLHF, DPO, and LHF

illustration from Rafailov et al. 2024

Direct Preference Learning

Rafailov et al. 2024

Roof term

Learning from Human Feedback (LHF)

Because of Direct Preference Learning → Llama 3

LHF

• LHF is a very plausible candidate,
• Others have pointed to it
• Experimental validation is needed

LHF

• typically:
• done in global south
• precarious conditions
• Toxtli et al., 2021; Roberts, 2022; Novick, 2023
• lack of transparency

Experiment: Emulate LHF

• IRB
• LAMP stack for rating website
• Decision log
• virtually everything pre-planned
• pre-designed: coefficient, 2.5 vs 2
• “preliminary” results
• Recruitment Prolific
• Emulate procedure: Demographics
• Global South
• Highest standards, incl. NCP
• Random item order, random item position, etc. etc.

Analysis

• chi-square
• explorative multifactorial regression
• → paper

​

Results

• Exclusion rate
• “Delve” pushback
• We will come back to this

→virtually no chance to get conclusive experimental results

→conjecture → follow-up

Limitations

• Issues with experiment
• For ethical reasons: cannot truly emulate procedure
• Need to explore other factors

Intellectual merit

• procedure to identify overused words, “focal words”
• factors contributing to lexical bias
• stronger, but not fully conclusive evidence → Learning from Human Feedback

Broader impacts

• Technology is strongly affecting language usage
• It was not clear: What do we make of the recent changes
• What do we make of the causes

Broader impacts

• The big unknown:
• Variety
• vs
• Demographics: Age
• It could be just ‘normal’ language change!
• Or just the Task!
• →follow up

Broader impacts

• Critically:
• Insights can be gained
• It is tough, though, partly because:
• Lack of procedure and data transparency slow down progress

Thank you