Manufacturing Confidence
Information Mediation by Strategic Artificial Intelligence
1
Raghu Arghal Dissertation Proposal
Department of Electrical and Systems Engineering University of Pennsylvania
Image generated using Google Nano Banana
LLMs as Information Mediators (IMs)
Hölbling, Maier, and Feuerriegel 2025
How can we understand the behavior and impact of strategic IMs?
Lorè and Heydari 2024; Payne and Alloui-Cros 2025; Zhang et al. 2024 Potter et al. 2024; Carrasco-Farre 2024; Rogiers et al. 2024
Endless use cases
Personal finance chatbots, customer service, coding assistants, even healthcare
Persuasiveness relative to humans
Models can strategize and persuade
Models are capable of strategic reasoning and persuasion tailored to individual user preferences
Even if a model is perfectly aligned with its designer, the model may not be aligned with a given user or society at large.
Their incentives need not be ours
7
Prior Art
LLM Behavior
LLMs can
al. 2025; Meinke et al. 2024; Greenblatt et al. 2024; van der Weij et al. 2024)
Largely focused on empirical evaluations
Existing Theoretical Frameworks
A number of adjacent game- and control-theoretic areas:
11
Without strong analytical frameworks, limited to isolated measurements and capability demonstrations.
LLMs, societal deployment, and agentic workflows exhibit characteristics not captured by existing models
Prior Art
Existing Theoretical Frameworks
A number of adjacent game- and control-theoretic areas:
Users (hopefully) do not interact with LLMs in a vacuum — they also learn from those around them
LLM workflows have unique monitoring structures
The user has little to no control over the LLMs incentive structure
In agentic settings, LLMs act for the user rather than passively recommending
Empirical evaluations lack theoretical underpinnings
Existing models don’t capture some key traits of interest
18
Bridging the Theory-Experiment Gap
Our Program
27
How much power do strategic IMs have over public opinion?
How does the IMs behavior interact with social learning among users?
Can a strategic IM still reliably communicate its own uncertainty?
What happens when the IM is motivated by user engagement?
How can we shape IM incentives to protect social and user welfare?
Can effective mechanisms be internalized pre-deployment?
Controlling Public Opinion
32
The Beginning of ChatGPT Ads
Emma Lurie1, Ro Encarnacio´n1, Sorelle A. Friedler2, Danae´ Metaxa1
1University of Pennsylvania 2Haverford College
ewlurie@seas.upenn.edu, rone@seas.upenn.edu, sorelle@cs.haverford.edu, metaxa@seas.upenn.edu
How much power do strategic IMs have over public opinion and how do they exercise that power?
•
•
Reliance upon LLMs to inform decision-making is increasingly common
Many decisions (tech adoption, vaccine uptake, voting behavior, etc.) are often heavily influenced by social learning as well as private information
(Bauch and Bhattacharyya 2012; Brady et al. 2021; Gillingham and Bollinger 2021; Guilbeault, Becker, and Centola 2018; Rao, Mobius, and Rosenblat 2007; Weber 2012)
•
A strategic IM may have a stake in the user’s decision, e.g., through an advertising partner
Related Work
33
Relatively few works consider both and information-designing planner and social learning
| Setting | Gap |
Wei & Anastasopoulos 2022 | Profit-maximizing coordinator who receives private signals and a fee in exchange for recommendations | Users buy into mechanism and communication is two-way |
Smith et al. 2021 | Benevolent planner directly alters agents’ choice rules | Planner affects decisions directly rather than agents’ information |
Arieli et al. 2022 | Self-interested sender chooses fixed info structure for a sequence of receivers | One-shot setting does not capture dynamic sender strategies |
Wu et al. 2025 | As above; sender maximizes prob. of correct cascade across info structures more informative than the baseline | Sender can only increase informativeness |
Classic Social Learning
Private signal si
Action ai Observation
•
•
•
•
•
Sequence i = 1, 2, … of agents
Binary fixed state
, signals si, and actions ai
Exogenous, public signal precision qi := ℙ(si = ω)
Public history ℋi := (qj, aj)j<i ⟹ public belief bi = 𝔼[ω |ℋi]
Agents use bi and si to pick ai to match
Agents eventually ignore private signals and copy their predecessor w.p. 1
What happens when private signals are mediated by an LLM?
Banerjee 1992; Bikhchandani, Hirshleifer, and Welch 1992; Bauch and Bhattacharyya 2012; Brady et al. 2021; Gillingham and Bollinger 2021; Guilbeault, Becker, and Centola 2018; Rao, Mobius, and Rosenblat 2007; Weber 2012
34
Our Model: Social Learning + Info Design
Biased Planner
Seeks to induce a preferred action, regardless of truth
Altruistic Planner
Seeks to induce correct actions
Planner chooses the precision of private signals
Policy Questions:
How should/would such a planner manage information? How does this depend on public opinion?
Welfare Questions:
Signal precision
i
Private signal si
Action ai Observation
What does society/the planner gain by accounting for social learning?
What can be done to strengthen/weaken the impact of the planner?
39
Our Model: Social Learning + Info Design
Biased Planner
Seeks to induce a preferred action, regardless of truth
Altruistic Planner
Seeks to induce correct actions
Planner chooses the precision of private signals
Full Transparency
Planner’s choices of i are known to all agents
Planner and agents have identical
knowledge
Information Parity
Constrained signal structure prevents lying or cherry-picking
Honesty & Symmetry
What does society/the planner gain by accounting for social learning?
What can be done to strengthen/weaken the impact of the planner?
42
Policy Questions:
How should/would such a planner manage information? How does this depend on public opinion?
Welfare Questions:
Our Model: Social Learning + Info Design
Signal precision
i
Private signal si
Action ai Observation
rB(bi, qi) = ≥ β( | qi ≥ p | ) ≥ C1(ai = 0)
45
Vπ
θ
1
(b ) =
rA(bi, qi) = ≥ β( | qi ≥ p | ) ≥ C1(ai − ω)
≠
∑
i=1
i≥1
δ 𝔼[rθ(π(bi), ai)]
π∗∈
π∞ π
θ (b) ∗ arg sup Vθ (b)
θ ∗ Altruistic, Biased
•
•
•
•
Sequence of agents i = 1,2,…
Public history ℋi := (qj, aj)j<i ⟹ public belief bi = 𝔼[ω |ℋi]
Agents use bi and si to pick ai to match
Binary fixed state
, signals si, and actions ai
Key Technical Result
49
Our approach: Induction and path coupling
Other applications?
May find use in experimental design, active learning, stochastic search, etc.
Technical Challenge
Actions (and, hence, observations and state updates) depend upon the state and are affected by herding
x0
…
y(x0, π(x0))
z(x0, π(x0))
y(x1, π(x1))
z(x1, π(x1))
y(xk−1, π(xk−1))
z(xk−1, π(xk−1))
xk−1 ∈ | | 2k−1 possible states … | π(xi) 1 − π(xi) (∏ y(xi, π(xi)) ) ( ∏ z(xi, π(xi)) ) x0 i∈𝒢 i∈ℬ |
xk ∈ | | 2k possible states xk−1 … | x¯k−1 |
x1 ∈ | 1 − π(x0) x z(x , π(x )) 0 0 0 | | π(x0) x y(x , π(x )) 0 0 0 | |
y( x0, π( x0))
z( x0, π( x0))
x2 ∈ | 1 − π(x0) x | | π(x0) x | | 1 − π(x1) π(x0) x | | π(x1) x | |
z(x , π(x )) 0 0 0 | | y(x , π(x )) 0 0 0 | | z(x , π(x )) y(x , π(x )) 0 1 1 0 0 | | y(x , π(x )) 1 1 1 | |
…
…
…
…
…
…
A
V∞( ⋅ ) is convex with respect to public belief
Theorem: Altruistic Value Function Convexity
⟹
Optimal Altruistic Policies
50
A
π∞(b) =
p
1
max(b,1 ≥ b)
b ∗ [0,dA) ∪ (1 ≥ dA,1] b ∗ (tA,1 ≥ tA) otherwise
Theorem: Optimal Altruistic Policy
∃tA : 0 < dA ≤ tA ≤ tM ≤ 0.5 such that:
A
π0(b) =
1 b ∗ (tM,1 ≥ tM)
{p otherwise
with tM =
β(1 ≥ p)
C
{0.5
β(1 ≥ p) < C(1 ≥ p)
otherwise
Theorem: Myopic Altruistic Policy
0.0 tA
1 °tM
1 °tA 1.0
Public Belief bi
0.5
Optimal Altruistic Myopic Altruistic
tM 0.5
p
1.0
Optimal Biased Policies
51
∞
1 B
A. If b ≤ t or b > p, then π (
b) = p
B. If b ∗ (t ,1 ≥ p]
∞
1 B
, then π (
b) Π p
B
C. If b ∗ (1 ≥ p,0.5), then π∞(b) Π 1 ≥ b
D. If b ∗ [0.5, 2)
t , then π
∞
B(b) Π b
2
E. If b ∗ (t , p
], then π
ϵ∞
B
(b) = b ≥ ϵ
Theorem: Optimal Biased Policy
0.8
∃t1, t2 : 0 ≤ t1 ≤ 1 ≥ p ≤ 0.5 < t2 < p such that:
0.0
0.2
0.4 0.6
Public Belief
0.8
1.0
0.5
0.6
0.7
0.9
Signal Precision
Optimal Biased Myopic Biased
LLM Evaluation
Scenario:
Potential car-buyers (agents) seeking advice
Agent LLM(s):
Given a customer profile (demography, use cases, concerns, etc.)
Planner LLM:
Given an objective (altruistic/biased)
Oracle LLM:
Given a generated set of specifications of the car for sale Generates signals of chosen precision
Agent
Planner
Oracle
History
Precision
Action
Precision
Message
But what do real LLMs do?
53
Emergent Strategies
0.0
0.2
0.4 0.6
Public Belief
0.8
1.0
0.5
0.6
0.7
0.8
0.9
1.0
Precision
Planner Type LLM
Optimal
Planner Objective Altruistic
Biased
0
40
10 20 30
Percentage Policy Deviation
0.0
2.5
5.0
7.5
10.0
12.5
15.0
Percent
Planner Objective
Altruistic Biased
LLM policies are remarkably similar to analytical optimal
56
Impact on Social Welfare
3 Testing Regimes:
LLM planner beats hybrid
Social learning-aware planners win
Even constrained planners are impactful
61
LLM Behavioral Biases
66
0.0
0.2
0.4 0.6
Prior Belief
0.8
1.0
°0.25
°0.50
°0.75
0.25
0.00
0.50
Posterior - Prior
Realization Positive Negative Agent Type LLM
Bayesian
Overreaction to surprising information
Underreaction to expected information
These biases have also been observed empirically in human studies!
(Ba, Bohren, and Imas 2022; Chan et al. 2025)
How do LLM agents update beliefs and make decisions?
Ongoing Work
67
Agent heterogeneity
How do the results extend to settings where agents have different preferences, information sets, levels of influence, etc.?
Key Contributions
68
Thank You!
Any Questions?
102