In Agents We Trust, but Who Do Agents Trust? Latent Source Preferences Steer LLM Generations
Mohammad Aflah Khan, Mahsa Amani, Soumi Das, Bishwamittra Ghosh, Qinyuan Wu, Krishna P. Gummadi, Manish Gupta, Abhilasha Ravichander
LLMs as User Facing Frontends
How do we Measure Preferences?
LLM prefers these sources and hence will surface them more
LLM does not prefer these sources and hence will surface them less
Direct Extraction - Just ask the model
Indirect Inference - Ask the model to pick the article it prefers (both articles are semantically identical) & see how many times was each source preferred
We try all possible combinations to account for position/order effects
Do LLMs Recognize Different Identities?
# Implies the p-value is less than 0.01 and hence the correlation is statistically significant
Are Latent Credential Preferences Rational?
Correlations between a rational ranking of credentials and the direct (lower triangle) or indirect (upper triangle) rankings, across models and source sets.
Latent Source Preference Hypothesis
Validating the Hypothesis
Can Agents Do User’s Bidding?
Takeaways for Stakeholders
Impact of Source Names in Decisions
Buy Products
Search the Web
Book Trips
Real World
Case Study
Real World
Case Study
In recommending news stories,
the source of the information strongly influences which story an LLM will surface, even over the actual content
Prompting alone does not mitigate this bias.
Organizations / Brands - Training data representation affects how often and how favorably brands are surfaced. Organizations must manage digital identity and guard against impersonation.
Users - LLMs may privilege certain sources rather than act neutrally. Users need controls to align outputs with their trust and values.
LLM Developers / Providers - Source preferences require transparency and auditing. Developers should enable mechanisms to diagnose and modulate these effects.
Policymakers / Regulators - Source biases can shape information exposure at scale. This raises concerns about competition, fairness, and accountability.
We find high correlation in rankings across source representations.
Exceptions arise when the representation diverges from the source name e.g. Associated Press Fact Check v/s @apfactcheck
Credentials are valued differently depending on how they’re presented and evaluated
Models tend to view “K years old” differently from “established in year Y,” assigning them different levels of prestige despite conveying the same information. In contrast, the H5-Index stands out as a more consistent credential.
Prompting with targeted instructions can
shift model preferences to better reflect user needs.
Our findings indicate the potential for designing user-centric LLM agents to counter the effects of platform-centric algorithms
The Latent Source Preference Hypothesis suggests that LLMs develop implicit preferences for certain source entities.
These preferences predictably influence the information they choose about or from those sources.
Status Quo
Agentic Era
LLMs differ in the strength of their preferences across different sources.
Larger models, show greater variance, reflecting stronger and more heterogeneous preferences while smaller models consistently exhibit lower deviations.
In Agents We Trust, but Who Do Agents Trust? Latent Source Preferences Steer LLM Generations
Mohammad Aflah Khan, Mahsa Amani, Soumi Das, Bishwamittra Ghosh, Qinyuan Wu, Krishna P. Gummadi, Manish Gupta, Abhilasha Ravichander
1️⃣ LLMs as User Facing Frontends
3️⃣How do we Measure Preferences?
LLM prefers these sources and hence will surface them more
LLM does not prefer these sources and hence will surface them less
Direct Extraction - Just ask the model
Indirect Inference - Ask the model to pick the article it prefers (both articles are semantically identical) & see how many times was each source preferred
We try all possible combinations to account for position/order effects
5️⃣Do LLMs Recognize Different Identities?
# Implies the p-value is less than 0.01 and hence the correlation is statistically significant
6️⃣Are Latent Credential Preferences Rational?
Correlations between a rational ranking of credentials and the direct (lower triangle) or indirect (upper triangle) rankings, across models and source sets.
2️⃣Latent Source Preference Hypothesis
4️⃣Validating the Hypothesis
8️⃣Can Agents Do User’s Bidding?
9️⃣Takeaways for Stakeholders
7️⃣Impact of Source Names in Decisions
Buy Products
Search the Web
Book Trips
Real World
Case Study
Real World
Case Study
In recommending news stories,
the source of the information strongly influences which story an LLM will surface, even over the actual content
Prompting alone does not mitigate this bias.
Organizations / Brands - Training data representation affects how often and how favorably brands are surfaced. Organizations must manage digital identity and guard against impersonation.
Users - LLMs may privilege certain sources rather than act neutrally. Users need controls to align outputs with their trust and values.
LLM Developers / Providers - Source preferences require transparency and auditing. Developers should enable mechanisms to diagnose and modulate these effects.
Policymakers / Regulators - Source biases can shape information exposure at scale. This raises concerns about competition, fairness, and accountability.
We find high correlation in rankings across source representations.
Exceptions arise when the representation diverges from the source name e.g. Associated Press Fact Check v/s @apfactcheck
Credentials are valued differently depending on how they’re presented and evaluated
Models tend to view “K years old” differently from “established in year Y,” assigning them different levels of prestige despite conveying the same information. In contrast, the H5-Index stands out as a more consistent credential.
Prompting with targeted instructions can
shift model preferences to better reflect user needs.
Our findings indicate the potential for designing user-centric LLM agents to counter the effects of platform-centric algorithms
The Latent Source Preference Hypothesis suggests that LLMs develop implicit preferences for certain source entities.
These preferences predictably influence the information they choose about or from those sources.
Status Quo
Agentic Era
LLMs differ in the strength of their preferences across different sources.
Larger models, show greater variance, reflecting stronger and more heterogeneous preferences while smaller models consistently exhibit lower deviations.
In Agents We Trust, but Who Do Agents Trust? Latent Source Preferences Steer LLM Generations
Mohammad Aflah Khan, Mahsa Amani, Soumi Das, Bishwamittra Ghosh, Qinyuan Wu, Krishna P. Gummadi, Manish Gupta, Abhilasha Ravichander
LLMs as User Facing Frontends
How do we Measure Preferences?
LLM prefers these sources and hence will surface them more
LLM does not prefer these sources and hence will surface them less
The Latent Source Preference Hypothesis suggests that LLMs develop implicit preferences for certain source entities.
These preferences predictably influence the information they choose about or from those sources.
Method A: Direct - Just ask the model
Method B: Indirect - Ask the model to pick the article it prefers (both articles are semantically identical) & see how many times was each source preferred
Try all possible combinations to account for position/order effects
Organizations / Brands - Training data representation affects how often and how favorably brands are surfaced. Organizations must manage digital identity and guard against impersonation.
Users - LLMs may privilege certain sources rather than act neutrally. Users need controls to align outputs with their trust and values.
LLM Developers / Providers - Source preferences require transparency and auditing. Developers should enable mechanisms to diagnose and modulate these effects.
Policymakers / Regulators - Source biases can shape information exposure at scale. This raises concerns about competition, fairness, and accountability.
Do LLMs Recognize Different Identities?
We find high correlation in rankings across source representations. Exceptions arise when the representation diverges from the source name
e.g. Associated Press Fact Check v/s @apfactcheck
# Implies the p-value is less than 0.01 and hence the correlation is statistically significant
There are differences in how these credentials are valued in direct evaluations versus indirectly inferred preferences.
Models tend to view “K years old” differently from “established in year Y,” assigning them different levels of prestige despite conveying the same information. In contrast, the H5-Index stands out as a more consistent credential.
Are Latent Credential Preferences Rational?
Correlations between a rational ranking of credentials and the direct (lower triangle) or indirect (upper triangle) rankings, across models and source sets.
Latent Source Preference Hypothesis
Validating the Hypothesis
Can Agents Do User’s Bidding?
Prompting with targeted instructions can shift model preferences to better reflect user needs.
Our findings indicate the potential for designing user-centric LLM agents to counter the effects of platform-centric algorithms like BuyBox.
Takeaways for Stakeholders
Source information strongly influences LLMs, as seen in the gap between Source Hidden & Shown conditions. Hence, even when left- or centrist-aligned outlets present right-leaning perspectives, they are still more likely to be selected.
Additionally, attempts to mitigate bias through prompting are largely ineffective.
Impact of Source Names in Decisions
1️⃣
2️⃣
3️⃣
4️⃣
5️⃣
6️⃣
7️⃣
8️⃣
9️⃣
LLMs differ in the strength of their preferences across different sources. Larger models, show greater variance, reflecting stronger and more heterogeneous preferences across sources. In contrast, smaller models consistently exhibit lower deviations.
Buy Products
Search the Web
Book Trips
Real World
Case Study
Real World
Case Study
In Agents We Trust, but Who Do Agents Trust? Latent Source Preferences Steer LLM Generations
Mohammad Aflah Khan, Mahsa Amani, Soumi Das, Bishwamittra Ghosh, Qinyuan Wu, Krishna P. Gummadi, Manish Gupta, Abhilasha Ravichander
LLMs as User Facing Frontends
How do we Measure Preferences?
LLM prefers these sources and hence will surface them more
LLM does not prefer these sources and hence will surface them less
The Latent Source Preference Hypothesis suggests that LLMs develop implicit preferences for certain source entities.
These preferences predictably influence the information they choose about or from those sources.
Method A: Direct - Just ask the model
Method B: Indirect - Ask the model to pick the article it prefers (both articles are semantically identical) & see how many times was each source preferred
Try all possible combinations to account for position/order effects
Organizations / Brands - Training data representation affects how often and how favorably brands are surfaced. Organizations must manage digital identity and guard against impersonation.
Users - LLMs may privilege certain sources rather than act neutrally. Users need controls to align outputs with their trust and values.
LLM Developers / Providers - Source preferences require transparency and auditing. Developers should enable mechanisms to diagnose and modulate these effects.
Policymakers / Regulators - Source biases can shape information exposure at scale. This raises concerns about competition, fairness, and accountability.
Do LLMs Recognize Different Identities?
We find high correlation in rankings across source representations. Exceptions arise when the representation diverges from the source name
e.g. Associated Press Fact Check v/s @apfactcheck
# Implies the p-value is less than 0.01 and hence the correlation is statistically significant
There are differences in how these credentials are valued in direct evaluations versus indirectly inferred preferences.
Models tend to view “K years old” differently from “established in year Y,” assigning them different levels of prestige despite conveying the same information. In contrast, the H5-Index stands out as a more consistent credential.
Are Latent Credential Preferences Rational?
Correlations between a rational ranking of credentials and the direct (lower triangle) or indirect (upper triangle) rankings, across models and source sets.
Latent Source Preference Hypothesis
Validating the Hypothesis
Can Agents Do User’s Bidding?
Prompting with targeted instructions can shift model preferences to better reflect user needs.
Our findings indicate the potential for designing user-centric LLM agents to counter the effects of platform-centric algorithms like BuyBox.
Takeaways for Stakeholders
Source information strongly influences LLMs, as seen in the gap between Source Hidden & Shown conditions. Hence, even when left- or centrist-aligned outlets present right-leaning perspectives, they are still more likely to be selected.
Additionally, attempts to mitigate bias through prompting are largely ineffective.
Impact of Source Names in Decisions
1️⃣
2️⃣
3️⃣
4️⃣
5️⃣
6️⃣
7️⃣
8️⃣
9️⃣
LLMs differ in the strength of their preferences across different sources. Larger models, show greater variance, reflecting stronger and more heterogeneous preferences across sources. In contrast, smaller models consistently exhibit lower deviations.
Buy Products
Search the Web
Book Trips
Real World
Case Study
Real World
Case Study
In recommending news stories,
the source of the information strongly influences which story an LLM will surface, even over the actual content
Prompting alone does not mitigate this bias.