1 of 72

Stochastic choice �and �noisy introspective beliefs

John Smith

Rutgers-Camden

Economics

2 of 72

Which line is longer?

Finish with 5 minute ‘warning label’

5 of 72

Which is longer? (again)

8 of 72

So how long were they?

First: Right longer
Second: Left longer

Identical lengths!
On my office monitor
Shorter: 10.5 cm
Longer: 11.0 cm

(L,L) (R,R) (L,R)???

9 of 72

An economist and a psychologist walk into a bar

2 kg. vs. 1.5 kg.
Which is heavier?
Prob correct

2 kg. vs 1.9 kg.
Which is heavier?
Lower prob correct

1834! 1860!

Weber

Fechner

10 of 72

Objective reality �is �perceived imperfectly�

(and this can improve economics experiments)

11 of 72

Standard choice experiment

U(A)=V(A)+ε_A

U(B)=V(B)+ε_B

Can only observe choice

And infer the Vs

(or noisy WTP, rankings, etc.)

Or lotteries

12 of 72

Wouldn’t it be nice

To design an economic choice experiment

“True” utility was known by the experimenter

Could repeat without being obvious

Could make arbitrarily small changes in valuations

13 of 72

Experimental Design(s)

Paid an increasing amount in the

length of line selected

We can observe whether optimal choice was made

Incentivize perception experiments

To resemble economic choice

20 of 72

Random utility models

Select L over R if

V(L) + ε_L ≥ V(R) + ε_R

“How confident are you about your choice?”

“Completely”

Naïve Thurstonian

“Less than completely”

Sophisticated Thurstonian

Different responses in identical menus?

Thurstone

21 of 72

Two-stage design

First stage: line judgment
Always a line of 240 pixels (12.348 cm)

4 line differences range among

10 pixels (0.514 cm)
6 pixels (0.308 cm)
3 pixels (0.154 cm)
1 pixel (0.051 cm)

Second stage: incentivized belief elicitation

Each line pair 4 times

24 of 72

Count

Line

Bets

Here

25 of 72

Noisy beliefs

Same response

in all 4 trials

Not the same

response in all

4 trials

11%

89%

Subjects’ 4 line bet responses for each line pair

Calculate summed absolute deviations

Noisy

Beliefs

26 of 72

Responsive beliefs

More line bets
For optimal choices

Yes, potential endogeneity

Many specifications

Same results

27 of 72

Responsive beliefs

Restrict to length differences
Difficult choices

Responsiveness not significant

Easier choices

Responsive beliefs

28 of 72

Naïve/Sophisticated?

Naïve Thurstonians?

60.8%

59.3%

73.5%

84.8%

29 of 72

Naïve/Sophisticated?

Only evidence for 1 Naïve Thurstonian?

30 of 72

Imprecision in introspective beliefs?

Our MPL format assumes point beliefs

Future design to allow responses to

express diffuse beliefs

Respond with as little as two clicks

31 of 72

Indifference

32 of 72

Random utility models

Choice between L and R

But these are perceived with noise

Select L over R if

V(L) + ε_L ≥ V(R) + ε_R

Same menu

different choices

Prob (V(L) + ε_L = V(R) + ε_R ) = 0

Non-coincident

Indifference is thin (or knife edge)

Thurstone

33 of 72

Luce’s coffee (1956, Econometrica)

Indifferent between any adjacent pairs

Not indifferent between ends

Indifference thick

but not random

…

401

1 grain

n+1

400

34 of 72

“Coin flip”

35 of 72

Random Thickness of Indifference (2026, JEBO)�

Indifference is random

Express strict preference on some trials
and indifference on other materially identical trials

Indifference is thick

(not knife-edge)
Changes in differences do not necessarily break indifference

Choices of indifference have longer response times than suboptimal (or optimal)

36 of 72

Objective vs. Subjective?

Perhaps subjects prefer the

objective probability of the coin flip

over the

subjective probability of their choices?

39 of 72

Experimental Design

Always a line of 240 pixels (12.348 cm)

5 line differences range among

10 pixels (0.514 cm)
6 pixels (0.308 cm)
3 pixels (0.154 cm)
1 pixel (0.051 cm)
0 pixels (identical lengths)

Each line pair (5)
in each treatment (3)
4 times

40 of 72

Still

indifference

choices

And still

noisy

“0” not

special

41 of 72

Many never

select

indifference

Few always

select

indifference

Same or nearly same cup of coffee

Many only sometimes

select indifference

42 of 72

Hurray brain sciences, but….

Warning label!

When reading brain science papers:

BE SKEPTICAL AND ASK QUESTIONS!

43 of 72

On Science

There is an objective truth

which can be discovered through
experimentation, observation, and reason

Science is the process of

discovering this objective truth

44 of 72

On Science

Science is only self-correcting if:

Mistakes are pointed out

Mistakes are corrected

So I am not just “being mean”

45 of 72

Bob

Repeated guessing task

Experimenter takes draw from distribution
Bob guesses the trial-specific draw
After guess, trial-specific draw is revealed

While the distribution is not given to Bob

It can be learned by experience

Bob

Experimenter

1.5

46 of 72

Bob

What if Bob guesses

the revealed draw
from previous trial?

0.7

-0.5

1.3

0.7

-0.5

1.3

Trial t-1

Trial t

Trial t+1

Trial t+2

…AND SO ON

47 of 72

Bob Critique

Average of Bob’s responses

converges to average of distribution

But Bob does not become more accurate

Across trials

48 of 72

Bob Critique

What if Bob guesses

the revealed draw
from previous trial?

Who would argue that Bob has learned the distribution through experience?

Bob

49 of 72

Why be skeptical?

There are thousands of neuroscience papers

Including in Nature

That average data across trials

And would declare Bob as Bayesian

Don’t laugh
Now appearing in top econ journals!

Including QJE

52 of 72

assumed to be

learning distribution

assumed to have

learned distribution

53 of 72

Frydman and Jin (2022, QJE)

Only analyze (some) “test” trials

Averaged across trials

Do not even test for differences between

adapt and test trials

Vulnerable to the Bob Critique

54 of 72

Stochastic

distributions

Apparent

implications of

every subject

having learned

distribution

through experience

by some trial

55 of 72

Learning?

Different specifications of learning across time
200 regressions
400 coefficient estimates

119 of these estimates

are significant at 0.05

Every single one has the opposite sign

as predicted by learning across time

56 of 72

Bob-type recency

Treatment

Effects

Bob-type

recency

57 of 72

Nullius in verba

Do not trust me

Read Frydman and Jin (2022, QJE)

and decide for yourself

58 of 72

Conclusions

Objective reality is perceived imperfectly

and this can improve economics experiments

Noisy introspective beliefs
are in
stochastic choice

Be skeptical and ask questions!

My Camden experiments got complicated

John Smith

smithj@camden.rutgers.edu

JohnSmithEcon.com

59 of 72

Correct faster than incorrect?

Drift diffusion models (DDM)
Decide both what AND when to decide

Fudenberg, Strack, and Strzalecki (AER, 2018)
Incorrect decisions will have

longer response times than
correct decisions

60 of 72