APPLIED
BEHAVIORAL
SCIENCE
A SIMPLE GUIDE TO
by Matt Wallaert
This is a slide deck;
Use arrows, scroll, or click to navigate!
Behavior as an Outcome,
Science as a Process
1
2
What is Applied Behavioral Science?
At the highest level, it is right there in the name: behavioral science works by centralizing behavior as an outcome and science as a process.
It is an evolution of existing methods that you might already be familar with from design thinking and product development.
It is distinct from academic behavioral science, which prioritizes understanding human behavior in ways that generalize - applied BeSci is explicitly about changing behavior in ways that are situation specific.
3
All organizations exist to change behavior
Companies are a combination of three things:
Business Model
What behaviors we monetize
Behavioral Model How we change those behaviors
Operational Model
How we do it
at scale
Applied behavioral science is the process by which companies innovate on their behavioral model. The goal is to create interventions that support the business model and can be scaled through the operational model, along with the evidence to evaluate the scaling decision.
Visa uses services fees to monetize buying via an exclusive payment network (VisaNet).
Spotify uses advertising/subscription to monetize listening via a distributed streaming platform.
Pepsi uses retail sales to monetize drinking via a manufacturing, distribution, and marketing network.
Uber uses service fees to monetize travelling via a network of freelance drivers.
Netflix uses subscription to monetize watching via a streaming video service.
4
Examples
Charity: Water impacts people living without clean water by increasing well drilling via a network of engineering staff.
Goodwill impacts employment rates by increasing job training via a network of job training centers.
AnitaB impacts women in tech leadership by increasing hiring of women and non-binary technologists via a variety of programs targeted at the workforce and hiring managers.
Doctors Without Borders impacts conflict death rate by increasing crisis medical care via a network of traveling medical staff.
World Wildlife Fund impacts species extinction by increasing land protection purchases via a conservation holding group.
5
This works for nonprofits and government organisations as well.
PROCESS
This guide uses the SIDE method, originally proposed by Matt Wallaert in his book Start At The End.
Applied Behavioral Science Is A Process
There are many models for doing applied behavioral science; all are designed to create probabilistic evidence that supports scaling decisions. It is better poker, not perfect chess.
It is a four-phase model (Strategy, Insights, Design, Evaluation) that is run as a ~12-week process.
Behavioral Science
Operations
Business
Strategy
Insights
Design
Evaluation
7
Science is on your SIDE
STRATEGY:
Specifically define the desired behavioral outcome.
Agree who will be doing what when we’re successful.
DELIVERABLE:
Behavioral Statement
INSIGHTS:
Uncover the pressures that define the behavioral equilibrium.
Understand why people do and don’t do the behavior today.
DELIVERABLE:
Pressure Map
DESIGN:
Create the potential interventions
to change the equilibrium.
Use our understanding of current behavior to find potential ways to change it.
DELIVERABLE:
Potential Pilots
EVALUATION:
Validate that the interventions change behavior enough to scale.
Generate evidence via
real-world testing to enable decision about what interventions are worth scaling.
DELIVERABLE:
Validated Interventions
8
9
WEEKLY PLAN
1-2
3-5
6-7
8-12
The objective of this phase is to understand from the business model precisely what the monetizable behavior is, how it is measured, and how much it is worth.
This is the "North Star" for the project and is the singular success criteria; if the team can replicably create this behavior at a cost/benefit ratio that is attractive for scaling, they will have done their job.
Each project has only one behavioral statement and it helps to understand how valuable it is; if the behavior is only worth $10, you need a different intervention than if it was worth $100.
STRATEGY
10
The deliverable of the strategy phase is the Behavioral Statement: A precise articulation of who will be doing what when you are successful.
When [Target Audience] who [Limitations] want to [Motivation], they will [Behavior] (as measured by [Data]).
BEHAVIORAL STATEMENT
11
The objective of this phase is to understand why people do and don’t do the behavior today.
Insights are generated by quantitative research (using data about populations) and qualitative research (using observations about individuals) working closely together to cross-validate.
The research is focused on discovering promoting pressures (which make a behavior more likely when strengthened) and inhibiting pressures (which make a behavior less likely when strengthened) by looking at the five existing behavioral groups: Always, Never, Sometimes, Started, Stopped.
WEEKLY PLAN
1-2
3-5
6-7
8-12
INSIGHTS
PRESSURE MAP
The deliverable of the Insights phase is the Pressure Map: A detailed diagram of all the promoting and inhibiting pressures that are supported by both qualitative and quantitative evidence.
BEHAVIOR
color, fabric, meaning, intimacy, status, safety, hunger, stress
cost, risk, time, effort, identity, mental, energy, brand, health, taste, luxury
12
The objective of this phase is to design interventions that change the existing pressures, resulting in behavior change.
We never design for behavior change itself, as it often encourages us to leap to "sexy", epiphany-based interventions; focusing on pressures keeps us rooted in the evidence.
The typical design phase results in hundreds of potential interventions; only a few will be selected to pilot, generally those that are optimally distinct - that is, they work on very different pressures and in very different ways.
13
WEEKLY PLAN
1-2
3-5
6-7
8-12
DESIGN
The deliverable of the
Design phase are Potential Interventions: A set of proposals which might change the pressures enough to result in behavior change.
14
DISHEALTH
Make a low/no sugar version
Emphasize the health benefits (chocolate makes you happy and happy is healthy)
Pair it with an activity (M&Ms
and walk)
Add protein
or vitamins
Emphasize the importance of calories for thinking and doing
POTENTIAL INTERVENTIONS
The objective of this phase is to generate evidence that informs the decision of what interventions to scale.
These are tests based on all the previous phases; there should be a clear line from the pilot (Evaluation), to the pressures we believe it will modify (Design), to evidence that those pressures are determinants of behavior (Insight).
These are not operational prototypes; before we determine how to deploy an intervention at scale, we first need to know whether it is worth deploying in the first place.
15
WEEKLY PLAN
1-2
3-5
6-7
8-12
EVALUATION
The deliverable of the Evaluation phase are Outcome Statements: Detailed articulations of the strength of the evidence that supports the interventions and the associated cost/benefit ratio.
OUTCOME STATEMENTS
16
We are [confidence] that [intervention] will [direction] [behavior] (as measured by [data]). Scaling this requires [effort] and will result in [change].
EXAMPLE: FLU SHOTS
When health plan members who don’t have any contraindications want to stay healthy, they will get a flu shot (as measured by % of active health plan members with a flu shot claim this year).
STRATEGY
INSIGHTS
DESIGN
IMPACT EVALUATION
Working with actuaries, we determined that each flu shot was worth ~$100 in medical expense reduction and direct payments from Medicare.
The key insight came from the Started group: many members who had never had a flu shot previously got one when they had a new grandbaby.
Ran a survey to understand members’ key non-health motivations and designed letters that directly targeted them. For example, if they care about travel, reminding them they can’t travel if they’re sick.
Double digital increases in flu shots, putting the health plan in the top 10% of plans nationwide for less than $5 per member.
17
EXAMPLE: GENDER WAGE EQUITY
When women who are currently employed want to be paid fairly, they will ask for and get a raise (as measured by number of women who received a raise).
A key weakness in PFMs is only looking at what happens after someone gets paid, at which point women are already severely disadvantaged because of unequal pay.
The key insight came from comparing the Always and Never categories: the difference was not in motivation to ask (promoting pressure) but in
their comfort and method in doing so (inhibiting pressures).
Created a tool that asked simple questions to generate a letter asking for a raise, including real normalized wage data from the BLS and concrete ties to increases in business value created.
80% of the women who hand in the letter get a raise, the average raise is ~7K, helped women earn almost $4B in raises so far.
STRATEGY
INSIGHTS
DESIGN
IMPACT EVALUATION
18
EXAMPLE: PCP QUALITY IMPROVEMENT
When health plan members who live within 5 miles of a highly-rated PCP want to have easy access to care, they will switch to the PCP we suggest (as measured by number of eligible members who have 2 or more visits with the recommended PCP within six months).
A strong PCP relationship is a key cost reduction mechanism and also directly drives revenue via CAHPS scores and STARS payments.
The key insight came from comparing the Started and Never categories: the main difference was in belief in the existence of good doctors. If you don’t believe good doctors exist, then you won’t switch away from a bad one.
Created materials to introduce high-quality nearby doctors and paid other insurance plan members who used that doctor to help take them to their first appointment.
High rate of change for members we were able to reach, resulting in increased revenue and reduced cost.
STRATEGY
INSIGHTS
DESIGN
IMPACT EVALUATION
19
EXAMPLE: MEMBER REFERRALS
When insurance members who are satisfied with their plans want to be good to their friends, they will tell them about the insurance plan (as measured by % of new members that come via referral).
Given the immense value of every incremental insurance plan member and the high network effect of recommendations, existing members was considered a key growth channel.
The key insight came from the Started category: those who had begun recommending the insurance plan generally had a non-insurance reason to do so (like swag or a noticeable special benefit).
Staged an event at a local bar with a local band and gave out two tickets to each member, so they could bring a friend.
~80% attendance by insurance members and, of those, nearly 50% brought a non-insurance member friend.
STRATEGY
INSIGHTS
DESIGN
IMPACT EVALUATION
20
EXAMPLE: STUDENT SEARCH
When students who go to a public school want to know something, they will use Bing to find out (as measured by number of Bing queries per student coming from school IP ranges).
Google was making significant inroads by converting young students to their products and it was deemed important to show competitive advantage in the search space.
The key insight came from comparing the Always and Never categories: the difference was not in curiosity (promoting pressure) but teacher concerns about privacy and safety (inhibiting pressures).
Obtained the IP range for several pilot districts and modified their queries to be ad-free, safer, and more private, alongside search-specific lesson plans.
Double digit % increases in search volume from students both at home and at school, highly significant during an era where even a tenth of a percent change garnered news coverage from the search media.
STRATEGY
INSIGHTS
DESIGN
IMPACT EVALUATION
21
EXAMPLE: TRANSLATION SYSTEMS ENGAGEMENT
When adults who write at least one language other than English want to express their cultural identity, they will contribute to Bing Translator (as measured by number of unique contributors to the Bing language index).
Google was considered the de facto translation API for the internet, despite Bing having superior capabilities. Paid API access was a significant revenue driver and key differentiator, particularly in government deals.
The key insight came from the Started category: those who used the Bing Translation APIs had recently begun contributing to the index in an identity-relevant language.
In conjunction with the release of the latest Star Trek movie, Bing added Klingon as an option to the translator and engaged the Klingon-speaking community to contribute.
Widespread press coverage and increases in all languages but particularly in those with the smallest populations, due to powerful in-group pressures.
STRATEGY
INSIGHTS
DESIGN
IMPACT EVALUATION
22
If you are specifically interested in the applied behavioral science process, the best next step is to take a detailed class through
PEOPLE
There are a variety of models for how applied behavioral science is embedded in an organization.
Accountable vs Consultative
Full-Cycle vs Partial-Cycle
All Projects vs Selected Projects
Generally, they differ across three important variables:
Thus, in an Accountable team, if the analysis is wrong, the responsibility lies with the team; in a Consultative team, it lies with the partner.
Accountable versus Consultative teams differ in who is actually responsible for the outcome of the work.
For example, an Accountable Quantitative Researcher is doing the data analysis themselves, while a Consultative Quant is advising a partner on the analysis.
For example, an Insights team may be fully accountable for producing insights, but not for the strategy that preceded them or the designs that are developed based on those insights.
Thus, they are doing components of applied behavioral science work, but are not Full-Cycle.
Full-Cycle versus Partial-Cycle teams differ in whether they are working across the entire SIDE process or only in parts.
“All” may be limited to a specific business unit; for example, all Marketing decisions but not all Product. “All” also refers only to large projects; not every decision is worth spending a full behavioral science sprint on.
All Projects vs Selected Projects teams differ in how work is assigned.
All Projects teams are seen as a necessary component in development; there is no way for a substantial initiative to scale except through an applied behavioral science process.
In contrast, on a Selected Projects team, stakeholders are routing only some projects through the team, while making intuitive decisions for the rest.
The best outcomes are produced with
Accountable, Full-Cycle, All Projects teams
using an applied behavioral science process.
When this Accountable, Full-Cycle, All Projects vision is fully embedded, applied behavioral science as a distinct function may cease to exist, because it is now the default way of working and may be simply called Product, Design, Marketing, etc.
To realize that vision, teams should choose one variable and optimize for it, before moving on to the next.
By doing the work, doing all of it, and doing it in an integrated way,
they get all of the benefits of behavior as outcome and science as process.
25
LEADER
STRATEGIST
QUANTITATIVE RESEARCHER
QUALITATIVE
RESEARCHER
DESIGNER
EVALUATION
MANAGER
There are many team models for doing applied behavioral science. This guide uses six key roles that are mapped to the four SIDE phases.
All the roles are assumed to be T-shaped, with one deep area of vertical expertise (the “legs” of the T) that allows them to have accountability within one phase of the process and broad horizontal interests (the “arms”) that allow them to support across the other phases.
We use an accountable/autonomous model; each person is presumed to be the expert in their space and while they may solicit feedback from other team members, they are empowered to fully determine their methods and are solely accountable for the results.
26
The Leader owns the end-to-end process of applied behavioral science; they guide how projects are accepted from the business and transitioned to operations and have direct accountability for the degree to which behavioral science is embedded in the org.
LEADER: All Phases
The Leader could be a manager or an IC. They may coach, hire, and fire for the team itself or simply guide the functional area managers, but must understand all of the individual disciplines well enough to be a thought partner.
Because their core responsibility is managing up and sideways to embed behavioral science in the organization, Leaders are relatively rare and it is a priority of the field to grow new ones.
LEADER
STRATEGIST
QUANTITATIVE RESEARCHER
QUALITATIVE
RESEARCHER
DESIGNER
EVALUATION
MANAGER
27
The Strategist leads the Strategy phase and is accountable for ensuring that projects begin with a behavioral statement that is fully ratified by all stakeholders and, if accomplished, will result in sufficient business impact to justify scaling.
STRATEGIST: Strategy Phase
Strategists are often MBA-like in stance and have strong understandings of both the business and operational model of the organization.
Well begun is half done, so Strategists need to have the fortitude to push back on stakeholders and hold the project until the behavioral statement is ready.
LEADER
STRATEGIST
QUANTITATIVE RESEARCHER
QUALITATIVE
RESEARCHER
DESIGNER
EVALUATION
MANAGER
28
The Quant co-leads the Insights phase and is accountable for generating cross-validated insights about the world we have to help us get to the world we want, using numerical, population-level methods.
QUANTITATIVE RESEARCHER: Insights Phase
Quants are a type of data specialist characterized by both a strong understanding of inferential statistics and a creative approach to using data to generalize about human behavior.
Because they work collaboratively with the Qual, it is important that Quants have at least a working knowledge of mixed methods research.
LEADER
STRATEGIST
QUANTITATIVE RESEARCHER
QUALITATIVE
RESEARCHER
DESIGNER
EVALUATION
MANAGER
29
The Qual co-leads the Insights phase and is accountable for generating cross-validated insights about the world we have to help us get to the world we want, using non-numerical, individual-level methods.
QUALITATIVE RESEARCHER: Insights Phase
Quals use synthesis methods that generalize from individuals to populations, which can introduce bias; they must be aware of and actively working against their predispositions.
Because they work collaboratively with the Quant, it is important that Quals have at least a working knowledge of mixed methods research.
LEADER
STRATEGIST
QUANTITATIVE RESEARCHER
QUALITATIVE
RESEARCHER
DESIGNER
EVALUATION
MANAGER
QUALITATIVE
RESEARCHER
30
The Designer leads the Design phase and is accountable for ensuring that projects create a diverse set of potential interventions that are clearly linked to the cross-validated pressures from the Insights phase.
DESIGNER: Design Phase
Designers are gifted facilitators, adept at drawing ideas from people with a diverse set of perspectives and then combining those ideas into larger and more nuanced proposals.
Because ideas can also come from what has been tried before, Designers are often experts in behavior change theory, with an understanding of what has worked in similar situations both in academic experiments and real-world trials.
LEADER
STRATEGIST
QUANTITATIVE RESEARCHER
QUALITATIVE
RESEARCHER
DESIGNER
EVALUATION
MANAGER
QUALITATIVE
RESEARCHER
31
The EM leads the Evaluation phase and is accountable for using experimentation to generate the evidence necessary to make scaling decisions across all interventions.
EVALUATION MANAGER : Evaluation Phase
EMs are often operator-like in background and have strong understandings of both the business model and the operational model of the organization.
The temptation at the end of the process is always to leap to scale, so EMs need to have the fortitude to push back on stakeholders and hold the project until their is sufficient evidence to justify scaling.
LEADER
STRATEGIST
QUANTITATIVE RESEARCHER
QUALITATIVE
RESEARCHER
DESIGNER
EVALUATION
MANAGER
QUALITATIVE
RESEARCHER
Strategist
Qual Researcher
Quant Researcher
Designer
Evaluation Manager
Leader
The simplest and most powerful team model is a five-person squad with a Leader managing multiple squads. This works because it creates optimally distinct roles that can be clearly aligned against specific phases of work, thus minimizing joint accountabilities.
Each role is accountable for a single phase (except for Insights, where the desire for independent cross-validation is prioritized) and then drops back to support the leader of each phase without overtaking their accountability. While this may seem like a significant investment, most large companies already have talented people who simply need to be reorganized and may only need to hire a Leader.
THE FIVE-PERSON SQUAD
32
Strategy
Who is going to be doing what when we’re successful?
Insights
Why are people doing/not doing that today?
Design
How do we potentially change those pressures?
Evaluation
Does our intervention work well enough how to scale?
Operational
Model
Business
Model
The four-person squad is perhaps the simplest reduction in overall size but it comes with a serious drawback: the loss of independent cross-validation in the Insights phase, by combining the Quantitative and Qualitative Researchers into a single Mixed Methods Researcher.
And yet reducing to one researcher is still the best solution. It maintains a strong process with clear accountabilities and, perhaps more importantly, is better than any other combination of roles.
THE FOUR-PERSON SQUAD
33
Strategist
Mixed Researcher
Designer
Evaluation Manager
Leader
Strategy
Who is going to be doing what when we’re successful?
Insights
Why are people doing/not doing that today?
Design
How do we potentially change those pressures?
Evaluation
Does our intervention work well enough how to scale?
Operational
Model
Business
Model
The three-person squad eliminates the Researchers entirely in favor of an expanded Designer role that gains accountability for the Insights phase.
The reason for this is simple: by nature of their understanding of what has worked elsewhere and the theory of behavior change, Designers often have at least some experience with either qualitative or quantitative research (and ideally both). And because the Designer typically sets the acceptance criteria for the Insights work, it is reasonable to simply make them also accountable for it.
THE THREE-PERSON SQUAD
34
Strategist
Designer
Evaluation Manager
Leader
Strategy
Who is going to be doing what when we’re successful?
Insights
Why are people doing/not doing that today?
Design
How do we potentially change those pressures?
Evaluation
Does our intervention work well enough how to scale?
Operational
Model
Business
Model
The two-person squad creates a tik-tok rhythm to the process, with each role taking accountability for an alternating phase. To preserve the bias-reducing quality of a scientific process, an independent person is validating that the results of that phase are both valid and useful before moving on.
There is still significant risk. Each transition is a potential for a loss in context and momentum, and two-person teams typically require both more senior hires and more close management by a leader who can help mitigate the inherent conflicts. These are also difficult roles to hire for because they require combinations of skills that don’t often occur naturally.
THE TWO-PERSON SQUAD
35
Strategic Designer
Research Manager
Leader
Strategy
Who is going to be doing what when we’re successful?
Business
Model
Insights
Why are people doing/not doing that today?
Design
How do we potentially change those pressures?
Evaluation
Does our intervention work well enough how to scale?
Operational
Model
LEADER
STRATEGIST
QUAL
QUANT
DESIGNER
FIVE-PERSON TEAM EXAMPLE
37
If you are specifically interested in the applied behavioral science team, the best next step is read the detailed guide at
TO GET HELP IMPLEMENTING APPLIED BEHAVIORAL SCIENCE IN YOUR ORG: