Mindset. Flex your humility
Method. Let’s experiment! Data is your friend. Write down your assumptions and how you plan to go about testing them. This could range from A/B testing, to workshops to a Randomised Control Trial (RCT). Capture your learnings about what worked and what didn’t, to build a database of experiments others can learn from.
Stating what we want to find out, and testing whether our ideas actually translate to change in the real world.
Check-in guide. How can we ensure we are reaching diverse audiences, not just the ‘easiest’ to reach?
Step 04
Experiment design
BEHAVIOURAL SCIENCE TOOLS
Experiment cards
Use cases: Set and refine your hypothesis
Material: Cards
Recommended time: 1 hours
Participants: Core working team
33
Concept title
Test duration
What is your hypothesis?�[Describe your concept for behaviour change]
How will you test the idea?
[Design an experiment to find out if it works]
What will you measure?
[List the key metrics to evaluate what works and doesn’t work]
EXPERIMENT CARD
5 STEPS to behavioural innovation by Behavioural by Design is licensed under CC BY-SA 4.0
Learn more from www.behaviouralbydesign.com/toolkit
Randomised controlled trials (RCTs). The key feature of an RCT is the use of a random assignment to create at least two groups that closely resemble each other. The only difference is that one group is exposed to the new strategy while the other does not. By comparing identical groups, chosen at random, an RCT enables you to understand which strategies, if any, are working, and eliminate pre-existing or external factors that normally complicate the evaluation process.
But random allocation may not always be logistically, ethically or politically feasible. Examples of valuable quasi-experimental designs include:
Regression discontinuity (RD): where participants are assigned to intervention and control groups based on a cut point of an assignment variable. The discontinuity between the intervention and control trends is then measured.
Propensity score matching (PSM): where participants in the intervention group are paired to participants in the control group based on the similarity of their scores to account for selection bias.
Difference in differences: where the effect of a intervention is estimated by comparing the pre- and post-intervention differences in the outcome in the treatment and control group.
Hansen, P. G. (2019). Tools and Ethics for Applied Behavioural Insights: The BASIC Toolkit. Organisation for Economic Cooporation and Development, OECD.
Statistical techniques to evaluate impact