1 of 22

Experimental Method

Roman Sheremeta, Ph.D.

Professor, Weatherhead School of Management

Case Western Reserve University

2 of 22

Part 1: Experimental Method

Part 2: Internal and External Validity

Part 3: Application to Collective Action

3 of 22

Part 1: �Experimental Method

4 of 22

Experimental method�

Experimental method: Learning from observing behavior in an environment created or modified by the researcher for that very purpose.

Why do we use experiments (Smith 1994):

To test theories (e.g., weed out bad theories).
To uncover behavioral regularities (e.g., other-regarding preferences, loss aversion, hyperbolic discounting).
To guide new theories (e.g., prospect theory).
To evaluate policy proposals (e.g., schooling curricula, microfinancing, emissions trading).
To design markets (e.g., matching markets for medical doctors, British spectrum auctions, markets for electric power).

5 of 22

Principles of economic experiments�

Main principles (Hertwig and Ortmann 2001):

Random assignment.
Rules are known to the subjects.
Repeated trials.
Financial incentives.
No deception.

6 of 22

Example: cooperation and punishment�

Fehr and Gachter (2000) conducted a laboratory experiment by randomly assigning subjects (students) to either the “no punishment“ or the “punishment“ condition.

In the no-punishment condition: subjects were split into groups of four and each subject had to decide how many of their 20 “points” to contribute to the “common good.” Contributing 1 point yielded 0.4 point to each member of the group.
In the punishment condition: subjects also contributed to the “common good,” but after they observed partners’ contributions, they could punish them (costly) for not contributing.

Main research question:

Does the opportunity for costly punishment increase cooperation?

7 of 22

Example: cooperation and punishment�

1. Random assignment.

Subjects were randomly assigned to different conditions/treatments.

2. Rules are known to the subjects.

Subjects knew the rules of the game.

3. Repeated trials.

Each condition/treatment lasted for 10 periods.

4. Financial incentives.

Experimental “points” were converted to cash.

5. No deception.

There was no deception in the experiment.

8 of 22

Example: cooperation and punishment�

Results of the experiment by Fehr and Gachter (2000):

The opportunity for costly punishment increases cooperation.

What can we make of these results?

All studies are judged by at least two criteria: internal validity and external validity.

9 of 22

Part 2: �Internal and External Validity

10 of 22

Internal validity�

Internal validity: the criterion that a researcher’s conclusions about what happened in the study are well-grounded.

There is a statistical basis for conclusion.
There are no alternative explanations.

The study of Fehr and Gachter (2000) claiming that punishment increases cooperation is internally valid because:

There is a statistical basis for concluding that there is a positive relationship between opportunities for punishment and cooperation.
There are no alternative explanations other than the causal one.

11 of 22

External validity�

External validity: The criterion that a study's conclusions be relevant for other situations.

The sample being studied is similar to the rest of the population.
The situation created in the experiment is not different from a typical relevant situation (“ecological validity”).
Subjects do not know that they are being studied (“demand effect”).

The study of Fehr and Gachter (2000) claiming that punishment increases cooperation has weak external validity because:

Students do not represent all people.
The situation is an imperfect representation of real institutions.
There could be a demand effect to punish.

12 of 22

Internal versus external validity�

There is a tradeoff between internal and external validity:

Experiments have high internal validity since they strip away aspects of the environment that might lead to alternative explanations.
But precisely because experiments strip away much of the richness of a real-life situation, experiments may have lower external validity.

One solution to the problem of external validity is field experiments (List 2011).

13 of 22

Example: recognition and giving�

Samek and Sheremeta (2017) conducted a field experiment by randomly assigning 205 parents who came to pick up their children to four conditions:

No recognition: all individuals could donate to the Red Cross and donation amounts were publicly displayed
Full recognition: all individuals’ names publicly displayed next to their donations
Positive recognition: the names of the highest two givers publicly displayed next to their donations
Negative recognition: the names of the lowest two givers (or non-givers) publicly displayed

Main research question:

What kind of recognition is more effective at increasing giving?

14 of 22

Example: recognition and giving�

Results of the experiment by Samek and Sheremeta (2017):

All forms of recognition examined increase giving relative to the baseline treatment, and recognizing only the highest or only the lowest donors has the strongest and most significant effect.

15 of 22

Laboratory versus field experiments�

Laboratory experiment: an experiment conducted at a facility specifically designed for that purpose.

Field experiment: An experiment conducted within the setting of subjects’ everyday lives, possibly without their knowledge.

Lab and field experiments tend to have different advantages:

Which has greater internal validity?
Which is easier to replicate?
Which has greater external validity?

16 of 22

Part 3: �Application to Collective Action

17 of 22

Prisoners’ dilemma�

Collective action problem:

Both players are better off as a group if they choose (A, A). Then both would receive (4,4).
But each player has an incentive to choose B, resulting in (1,1).
Thus, the collective action problem: conflict between the individual interest and the group interest results in an inferior group outcome.

4 , 4	0 , 5
5 , 0	1 , 1

Player 1

Player 2

18 of 22

Prisoners’ dilemma�

Experiment #1:

Prisoners’ dilemma game

Cooperation

Defection

19 of 22

Public goods�

20 of 22

Public goods�

Experiment #2:

Public goods game

Cooperation

Defection

21 of 22

Public goods�

22 of 22

References�

Fehr, E., & Gachter, S. (2000). Cooperation and punishment in public goods experiments. American Economic Review, 90, 980-994.
Hertwig, R., & Ortmann, A. (2001) Experimental practices in economics: A methodological challenge for psychologists? Behavioral and Brain Sciences, 24, 383-451.
List, J.A. (2011). Why economists should conduct field experiments and 14 tips for pulling one off. Journal of Economic Perspectives, 25, 3-16.
Samek, A., & Sheremeta, R. M. (2017). Selective recognition: How to recognize donors to increase charitable giving. Economic Inquiry, 55, 1489-1496.
Smith, V.L. (1994). Economics in the laboratory. Journal of Economic Perspectives, 8, 113-131.