GROWTH METRICS 101

Emigna Consulting

em@emilybasileo.com

AGENDA

1 Introduction to growth metrics

2 Growth metrics in experiments

3 Pirate Framework

4 NSM: North Star Metric

5 OMTM: One metric that matters

6 Types of metrics

7 How to choose your OMTM

8 Prioritizing growth experiments

9 Building a KPI Dashboard

Appendix: Project Management Process

But first…

em@emilybasileo.com

EMILY BASILEO

​

• Serial entrepreneur and executive ops
• Emigna consulting practice
• 13+ years strategy and ops experience
• Alpha Batch of 500, mentored 200+, exit

* The growth metrics part of this presentation is borrowed from 500 Global’s Ignition Program, which was co-developed and recorded by Stephan von Perger and me with help from 500 staff and which references a presentation by Derek Ooi

​

em@emilybasileo.com

1: INTRODUCTION TO GROWTH METRICS

em@emilybasileo.com

1// INTRO TO GROWTH METRICS

Growth Metrics are the numbers that provide clarity on what influences your company’s growth

em@emilybasileo.com

1// INTRO TO GROWTH METRICS

Growth metrics

• Are based on business goals and objectives
• Normally cover areas such as acquisition, conversion, retention, engagement, and monetization
• Consist of a group of merics

em@emilybasileo.com

1// INTRO TO GROWTH METRICS

Input metrics

• Also known as performance metrics
• Actions that influence the output metric
• Should know what action to take to change the metric

Types of growth metrics

Output metrics

• Long term goals for growth of business
• A result of input metrics
• Illustrate WHAT the situation is, not what you need to change

Input Metric

Input Metric

Output �Metric

Input Metric

em@emilybasileo.com

1// INTRO TO GROWTH METRICS

Play

Score

Play

Play

Example from: https://www.reforge.com/blog/north-star-metric-growth

em@emilybasileo.com

1// INTRO TO GROWTH METRICS

Input metrics can be broken down into smaller metrics; the result of a change in one metric will impact all the metrics that are dependent on it

em@emilybasileo.com

1// INTRO TO GROWTH METRICS

Identifying output metrics

• What are your long-term business goals
• What metrics will measure the growth of your business
• What will determine the success or failure of your offering
• Main types: retention, engagement, monetization

em@emilybasileo.com

1// INTRO TO GROWTH METRICS

Identifying output metrics

• Step 1: List your business goal
• Step 2: Identify the metric that can measure the success of this goal

em@emilybasileo.com

1// INTRO TO GROWTH METRICS

Step 1: List your business goal

• To increase consumption of our content by 50% in 2021

Step 2: Decide on a metric that can measure the success of this goal

Hours of watch time per month per user

em@emilybasileo.com

1// INTRO TO GROWTH METRICS

Step 1: List your business goal

• Increase the number of users who order food every week

Step 2: Decide on a metric that can measure the success of this goal

Weekly �Active Users

em@emilybasileo.com

1// INTRO TO GROWTH METRICS

Identifying input metrics

• How can you break down your output metrics into a set of actionable input metrics
• What functions of your offering can directly influence your output metrics
• What important milestones does your offering need to achieve to impact the output metrics
• You have control over your input metrics

em@emilybasileo.com

1// INTRO TO GROWTH METRICS

Identifying input metrics

• Step 1: Break down the Output Metric into smaller input metrics that are actionable. These will directly influence the Output Metric
• Step 2: Break down the input metrics into smaller metrics, if needed

em@emilybasileo.com

1// INTRO TO GROWTH METRICS

Step 1: Break down the Output Metric into smaller actionable input metrics

Hours of watch time per month per user

Watch time �per session

Returning �users

em@emilybasileo.com

1// INTRO TO GROWTH METRICS

Step 1: Break down the Output Metric into smaller actionable input metrics

Step 2: Break down the Input Metrics into smaller actionable metrics, if needed

Hours of watch time per month per user

Watch time �per session

Returning �users

Show completion time

Next episode

click rate

Push notification open rate

em@emilybasileo.com

1// INTRO TO GROWTH METRICS

Step 1: Break down the Output Metric into smaller actionable input metrics

​

Weekly �Active Users

New User Activations

Returning �Orders

em@emilybasileo.com

1// INTRO TO GROWTH METRICS

Step 1: Break down the Output Metric into smaller actionable input metrics

Step 2: Break down the Input Metrics into smaller actionable metrics, if needed

Weekly �Active Users

New User Activations

Returning �Orders

View restaurant menu

New restaurant recs

Promo

code

applied

em@emilybasileo.com

1// INTRO TO GROWTH METRICS

Summary

• Growth metrics indicate the health of your business and what you need to scale, change, or fix for growth
• Output (or “outcome”) metrics represent how your business is doing and are the result of input metrics
• Input (or “performance”) metrics measure the activities you can control that impact the output metric

em@emilybasileo.com

[OPTIONAL] BREAKOUT SESSION 1

1// INTRO TO GROWTH METRICS

1. Define at least one output metric�
2. Identify at least three input metrics for each output metric

em@emilybasileo.com

2: GROWTH METRICS IN EXPERIMENTS

em@emilybasileo.com

2// GROWTH METRICS IN EXPERIMENTS

What tests can we run or what tactics can we use to get the results we want?

Hours of watch time per month per user

Watch time �per session

Returning �users

Show completion time

Next episode

click rate

Push notification open rate

em@emilybasileo.com

2// GROWTH METRICS IN EXPERIMENTS

If we can increase our push notification open rate, we can increase the number of our returning users. If we can increase our returning users, we can increase the hours of watch time per user per month.

How can we increase our push notification open rate?

Test 1: Improve messaging of push message

Test 2: Change time of push notification to increase open rate

Hours of watch time per month per user

Returning �users

Push notification open rate

em@emilybasileo.com

2// GROWTH METRICS IN EXPERIMENTS

What tests can we run or what tactics can we use to get the results we want?

Weekly �Active Users

New User Activations

Returning �Orders

View restaurant menu

New restaurant recs

Promo

code

applied

em@emilybasileo.com

2// GROWTH METRICS IN EXPERIMENTS

If we can increase restaurant menu views, we can increase new user activations. If we can increase new user activations, we will have more weekly active users.

How can we increase restaurant menu views?

Weekly �Active Users

New User Activations

View restaurant menu

em@emilybasileo.com

2// GROWTH METRICS IN EXPERIMENTS

Summary

• Data is key. Design growth experiments to test and impact your growth metrics
• The result of a change in an input metric will impact all the metrics that are dependent on it
• Identifying, experimenting, and iterating on your growth metrics will methodically improve your business

em@emilybasileo.com

2// GROWTH METRICS IN EXPERIMENTS

[OPTIONAL] BREAKOUT SESSION 2

• Choose one input metric�
• Design at least 2 experiments to impact that metric

em@emilybasileo.com

3: PIRATE FRAMEWORK

em@emilybasileo.com

3// PIRATE FRAMEWORK

Dave McClure

Pirate Metrics

em@emilybasileo.com

3// PIRATE FRAMEWORK

The Pirate Framework (AAARRR) is used to track metrics across the 6 stages of the user journey

em@emilybasileo.com

3// PIRATE FRAMEWORK

A: Awareness

A: Acquisition

A: Activation

R: Retention

R: Referral

R: Revenue

em@emilybasileo.com

3// PIRATE FRAMEWORK

Awareness: Get on people’s radars

• Pre-users: How are people discovering us? What channels do they use?
• Note: People are aware that we exist but do not intentionally interact with us

​

Examples

• Reach, brand recognition, mentions

(Awareness)

em@emilybasileo.com

3// PIRATE FRAMEWORK

Acquisition: Get users to engage with us

• More specifically, where do our users come from? What information are they actively sharing with us?

​

Examples

• Acquisition: Site visits, email signups, app downloads, conversion rate, cost per acquisition

​

(Awareness)

Acquisition

em@emilybasileo.com

3// PIRATE FRAMEWORK

Activation: Get users to do what we want

• Are our users taking the actions we want? What % of users are taking these actions? How quickly can we get users to the “Aha!” moment?

​

Examples

• First visit actions, account creation, x new connections, y messages sent, z jobs posted

​

(Awareness)

Acquisition

Activation

em@emilybasileo.com

3// PIRATE FRAMEWORK

Retention: Get users to habitually use us

• Do users keep coming back? Are they continuously engaging with our solution? When do customers churn?�

Examples

• Repeat visitors, daily usage, engagement, actions

​

(Awareness)

Acquisition

Activation

Retention

em@emilybasileo.com

3// PIRATE FRAMEWORK

Referral: Get users to refer us

• Do our users like our solution enough to tell others?

​

Examples

• Referrals, referrals per user, activated referrals, monetized referrals, K factor (invites/user)

​

(Awareness)

Acquisition

Activation

Retention

Referral

em@emilybasileo.com

3// PIRATE FRAMEWORK

Revenue: Get users to pay us

• Can we monetize any of our users’ behavior?
• How can we upsell our current users?

​

Examples

• Users generate x revenue, AOV, LTV

​

​

(Awareness)

Acquisition

Activation

Retention

Referral

Revenue

em@emilybasileo.com

3// PIRATE FRAMEWORK

Awareness

Acquisition

Activation

Retention

Referral

Revenue

Note: Not *quite* a funnel; you want to create habit loops and viral loops. General best practice is to grow in this order, but sometimes you grow differently

em@emilybasileo.com

3// PIRATE FRAMEWORK

Summary

• The Pirate Framework (AAARRR) is used to track metrics across the 6 stages of the user journey: awareness, acquisition, activation, retention, referral, and revenue
• By understanding the critical elements of each stage of behavior, you can easily identify and track the metrics that measure your progress in those areas
• Drive users from the awareness stage to the referral and revenue stages
• Customer journeys may look different - adjust your framework accordingly

em@emilybasileo.com

[OPTIONAL] BREAKOUT SESSION 3

3// PIRATE FRAMEWORK

• Using the Pirate Framework, identify the steps within each stage of your consumer journey �
• Identify the metrics you should track for each stage

em@emilybasileo.com

4: NORTH STAR METRIC

em@emilybasileo.com

4// NORTH STAR METRIC

Your North Star Metric (NSM) is the metric that is the most predictive of your company’s long-term success

em@emilybasileo.com

4// NORTH STAR METRIC

Your North Star Metric must:

• Lead to revenue
• Reflect customer value
• Measure growth progress

Sean Ellis

em@emilybasileo.com

4// NORTH STAR METRIC

Lead �to Revenue

Without revenue, �you cannot build a sustainable business

​

Reflect �Customer Value

If revenue outpaces customer value, you will not grow sustainably

​

Measure �Growth Progress

If it is not measurable and does not measure growth, it is useless

​

em@emilybasileo.com

4// NORTH STAR METRIC

As your Pirate metrics improve, your North Star Metric should improve; improvement in any stage (post-awareness) should impact

your North Star Metric

em@emilybasileo.com

4// NORTH STAR METRIC

North Star Metric examples

• Facebook: Monthly active users
• Airbnb: Booked nights
• Netflix: Minutes watched

em@emilybasileo.com

4// NORTH STAR METRIC

North Star Metric examples

• Facebook: Monthly active users
• Airbnb: Booked nights
• Netflix: Minutes watched
• Tutoring app:

​

em@emilybasileo.com

4// NORTH STAR METRIC

North Star Metric examples

• Facebook: Monthly active users
• Airbnb: Booked nights
• Netflix: Minutes watched
• Tutoring app: Sessions per week

​

em@emilybasileo.com

4// NORTH STAR METRIC

North Star Metric examples

• Facebook: Monthly active users
• Airbnb: Booked nights
• Netflix: Minutes watched
• Tutoring app: Sessions per week
• eCommerce site:

​

em@emilybasileo.com

4// NORTH STAR METRIC

North Star Metric examples

• Facebook: Monthly active users
• Airbnb: Booked nights
• Netflix: Minutes watched
• Tutoring app: Sessions per week
• eCommerce site: Purchases per month

​

em@emilybasileo.com

4// NORTH STAR METRIC

North Star Metric examples

• Facebook: Monthly active users
• Airbnb: Booked nights
• Netflix: Minutes watched
• Tutoring app: Sessions per week
• eCommerce site: Purchases per month
• Rideshare app:

​

em@emilybasileo.com

4// NORTH STAR METRIC

North Star Metric examples

• Facebook: Monthly active users
• Airbnb: Booked nights
• Netflix: Minutes watched
• Tutoring app: Sessions per week
• eCommerce site: Purchases per week
• Rideshare app: Rides per week

em@emilybasileo.com

4// NORTH STAR METRIC

North Star Metric examples

• Facebook: Monthly active users
• Airbnb: Booked nights
• Netflix: Minutes watched
• Tutoring app: Sessions per week
• eCommerce site: Purchases per week
• Rideshare app: Rides per week
• Chat app:

​

em@emilybasileo.com

4// NORTH STAR METRIC

North Star Metric examples

• Facebook: Monthly active users
• Airbnb: Booked nights
• Netflix: Minutes watched
• Tutoring app: Sessions per week
• eCommerce site: Purchases per week
• Rideshare app: Rides per week
• Chat app: Messages sent per day

​

em@emilybasileo.com

4// NORTH STAR METRIC

Summary

• Your North Star Metric (NSM) is the metric that is the most predictive of your company’s long-term success
• Your North Star Metric must lead to revenue, reflect customer value, and measure growth progress
• As your Pirate metrics improve, your North Star Metric will also improve

em@emilybasileo.com

[OPTIONAL] BREAKOUT SESSION 4

4// NORTH STAR METRIC

• Identify your North Star Metric�
• Explain why you chose this NSM�
• Explain how your Pirate metrics impact your NSM

​

em@emilybasileo.com

5: ONE METRIC THAT MATTERS

em@emilybasileo.com

5// OMTM

Build - Measure - Learn

Track & Measure

em@emilybasileo.com

5// OMTM

One Metric That Matters (OMTM) is the single metric that is incredibly important for the step you’re currently working through, the one number you’re completely focused on above everything else at your current stage

em@emilybasileo.com

5// OMTM

The Most, Right Now

• Yes, you will track other metrics [KPIs, reporting]
• Yes, you still need to care about other metrics
• Yes, your OMTM will changed based on need and stage
• Yes, it may change quickly [but it may not]

Capture everything, but focus on what’s important

em@emilybasileo.com

5// OMTM

Your OMTM will directly feed into your North Star Metric; an improvement in your OMTM will yield an improvement in your NSM

em@emilybasileo.com

5// OMTM

Reasons to use OMTM

• Answers the most important questions you have
• Challenges you to draw a line in the sand
• Focuses the entire company
• Inspires a culture of experimentation

em@emilybasileo.com

5// OMTM

A good OMTM will

drive the changes

you are looking for

em@emilybasileo.com

5// OMTM

What makes a good metric

• Understandable
• Comparative
• Ratio or rate
• Changes the way you behave

Remember: Not all metrics are created equal

em@emilybasileo.com

5// OMTM

Understandable

• If people can’t remember it and discuss it, it’s much harder to turn a change in the data into a change in the culture

Understandable

em@emilybasileo.com

5// OMTM

Comparative

• Compare a metric to other time periods, groups of users, or competitors
• Helps understand which way things are moving
• Example: “Increased conversion from last week” is more meaningful than “2% conversion”

Understandable

Comparative

em@emilybasileo.com

5// OMTM

Ratio or rate

• Ratios are easier to act on - they tell you about your current state
• Ratios are inherently comparative
• Good for comparing factors that are somehow opposed or for which there is inherent tension

Understandable

Comparative

Ratio or Rate

em@emilybasileo.com

5// OMTM

Ratio or rate examples

Understandable

Comparative

Ratio or Rate

• Distance traveled: 25 miles (informational)
• Speed: 25 miles per hour (actionable)
• Speed vs. driving tickets (opposing factors)

• Number of transactions (informational)
• Transactions per hour (actionable)
• Transactions per hour vs. mistakes (opposing factors)

em@emilybasileo.com

5// OMTM

Changes the way you behave

• “Accounting metrics” will make your predictions more accurate
• “Experimental metrics” help optimize the product, pricing, or market

​

Understandable

Comparative

Ratio or Rate

Changes Behavior

em@emilybasileo.com

5// OMTM

Changes the way you behave

• What will you do differently based on changes in the metric?

​

Understandable

Comparative

Ratio or Rate

Changes Behavior

em@emilybasileo.com

5// OMTM

If you want to change a behavior, your metric must be tied to the behavioral change you want

em@emilybasileo.com

5// OMTM

Summary

• Your One Metric That Matters is your MOST important growth metric, right now
• Your OMTM will directly impact your North Star Metric
• A good OMTM is understandable, comparative, a ratio or rate, and changes the way you behave
• Not all metrics are created equal
• Your metric must be tied to the behavioral change you want

em@emilybasileo.com

6: TYPES OF METRICS

em@emilybasileo.com

6// TYPES OF METRICS

Qualitative vs. Quantitative

• Qualitative metrics are unstructured, anecdotal, revealing, hard to quantify; answer “why”
• Quantitative metrics involve numbers and statistics; provide hard numbers but less insight; answer “what” and “how much”

Score�(Quantitative)

Why the team performed the way they did�(Qualitative)

em@emilybasileo.com

6// TYPES OF METRICS

Vanity vs. Actionable

• Vanity metrics can be identified as pieces of data on which you cannot act
• Actionable metrics change your behavior by helping you pick a course of action

​

Total �Active Users

Total�Signups

Percent of Active Users of Total Signups Per Month

/

em@emilybasileo.com

6// TYPES OF METRICS

Exploratory vs. Reporting

• Exploratory metrics are speculative and try to find unknown insights’ used to discover something new - “you don’t know what you don’t know”
• Reporting metrics keep you abreast of normal, managerial, day-to-day operations; these are numbers you can report on

em@emilybasileo.com

6// TYPES OF METRICS

Reporting Metrics

What are they using the site for? Why?

1000 users on your site

1000 people actively using it

4m 33s average site visit

Exploratory Metrics

em@emilybasileo.com

em@emilybasileo.com

6// TYPES OF METRICS

Leading vs. Lagging

• Leading metrics [or leading indicators] give you a predictive understanding of the future
• Lagging metrics provide performance results and explain the past
• A metric can be both a leading metric for one business case and lagging metric for another business case

em@emilybasileo.com

6// TYPES OF METRICS

Leading vs. Lagging

Leading Metrics

Lagging Metrics

Marketing

Total Signups

Active Users

em@emilybasileo.com

6// TYPES OF METRICS

Correlated vs. Causal

• Correlated metrics change together
• Causal metrics are when one metric causes the other to change

em@emilybasileo.com

6// TYPES OF METRICS

Correlation does not equal causation

Note: Find a causal relationship between something you want [e.g. revenue] and something you can control [e.g. ad] so you can change the future!

em@emilybasileo.com

6// TYPES OF METRICS

Summary

• Some metrics are better than other, depending on the context
• Metrics that are actionable - that can directly change your outcome - and metrics that help you understand why things are they are the way they are - are much more valuable than other metrics

em@emilybasileo.com

7: HOW TO CHOOSE YOUR OMTM

em@emilybasileo.com

7// HOW TO CHOOSE YOUR OMTM

How to choose your OMTM

1. Identify your business model
2. Identify your business stage
3. Identify your business focus
4. Identify the riskiest area of your business
5. Choose a metric that will measure your performance (and hopefully progress) in that area

em@emilybasileo.com

7// HOW TO CHOOSE YOUR OMTM

1 Identify your business model

• You should know your business model
• Your business model should be simple
• You may have components of multiple business models
• Business growth comes from MORE: more stuff, more people, more often, more money, and doing things more efficiently

em@emilybasileo.com

7// HOW TO CHOOSE YOUR OMTM

2 Identify your business model

​

e-commerce

2-sided Marketplace

Media

SaaS

Mobile App

User-generated Content

em@emilybasileo.com

7// HOW TO CHOOSE YOUR OMTM

2 Identify your business stage

• Empathy
• Stickiness
• Virality
• Revenue
• Scale

em@emilybasileo.com

7// HOW TO CHOOSE YOUR OMTM

• Are you solving a problem that people care about in a way someone will pay for?
• Do you have product-market fit?

Empathy

em@emilybasileo.com

7// HOW TO CHOOSE YOUR OMTM

• Do you have a good product or service?
• Will customers keep coming back and using your product or service?
• There’s no point in promoting something if customers will churn quickly

Empathy

Stickiness

em@emilybasileo.com

7// HOW TO CHOOSE YOUR OMTM

• Are people sharing your product or service?
• This acts as a force multiplier for paid advertising and other marketing

Empathy

Stickiness

Virality

em@emilybasileo.com

7// HOW TO CHOOSE YOUR OMTM

• Focus is on maximizing and optimizing revenue
• Note: you may already be earning revenue - this is about how you can earn more revenue (more stuff, more people, more often, more money, and doing things more efficiently)

Empathy

Stickiness

Virality

Revenue

em@emilybasileo.com

7// HOW TO CHOOSE YOUR OMTM

• Focus moves from growing your business to growing your market

Empathy

Stickiness

Virality

Revenue

Scale

em@emilybasileo.com

7// HOW TO CHOOSE YOUR OMTM

3 Identify your business focus

• Acquisition: Getting new customers, getting new revenue, increasing revenue from new customers
• Loyalty: Retaining existing customers, increasing sales from existing customers, increasing engagement from existing customers, increasing revenue opportunities from existing customers

em@emilybasileo.com

7// HOW TO CHOOSE YOUR OMTM

4 Identify the riskiest area of your business

• What do you need to solve immediately?
• Where are your biggest weaknesses?
• What do you need to improve the most?

Address these first!

em@emilybasileo.com

7// HOW TO CHOOSE YOUR OMTM

5 Choose a metric that will measure your performance in that area

• The intersection of your Business Model + Business Stage + Business Focus drives the metric you track

em@emilybasileo.com

7// HOW TO CHOOSE YOUR OMTM

em@emilybasileo.com

7// HOW TO CHOOSE YOUR OMTM

Some key metrics: Inventory, listings, searches, buyer activity, seller activity, average transaction size, buyer lifetime value, seller LTV

The ultimate metric: volume of sales

Example: Airbnb

• Business Model: 2-sided marketplace
• Stage: Stickiness
• Focus: Acquisition
• OMTM: Photoshoots per month
• Tactic: Free professional photography, watermarks
• Result: Increase in bookings and revenue

em@emilybasileo.com

7// HOW TO CHOOSE YOUR OMTM

Some key metrics: Open jobs, candidates on platform, interviews per candidate, interview requests, job offers

The ultimate metric: volume of filled jobs

Example: Recruiting platform

• Business Model: 2-sided platform
• Stage: Stickiness
• Focus: Acquisition
• OMTM: Interview requests
• Tactics: ?

em@emilybasileo.com

7// HOW TO CHOOSE YOUR OMTM

Some key metrics: Open jobs, candidates on platform, interviews per candidate, interview requests, job offers

The ultimate metric: volume of filled jobs

Example: Recruiting platform

• Business Model: 2-sided platform
• Stage: Stickiness
• Focus: Acquisition
• OMTM: Interview requests
• Tactics: Email reminders for employers to request interviews, candidate recommendations, employer incentives to request more interviews through a reward system

em@emilybasileo.com

7// HOW TO CHOOSE YOUR OMTM

Summary

• The intersection of your Business Model + Business Stage + Business Focus will determine the metric you track
• Choose a metric that will drive the change you want
• Design experiments and tactics that will impact your OMTM

em@emilybasileo.com

[OPTIONAL] BREAKOUT SESSION 5

6// HOW TO CHOOSE YOUR OMTM

• Identify your OMTM�
• Explain why you chose your OMTM�
• Explain how your OMTM impacts your NSM

em@emilybasileo.com

1// INTRO TO GROWTH METRICS

[OPTIONAL] BREAKOUT SESSION 6

• Share your OMTM�
• Explain why you chose this metric�
• Suggest at least one experiment for each of the other companies at this bootcamp

em@emilybasileo.com

8: Prioritizing Experiments

em@emilybasileo.com

8// PRIORITIZING EXPERIMENTS

• Ensure the entire team is aligned on objectives and KPIs
• Explain the “why”
• Hint: Use an All Hands meeting to review company mission, short-term goal, objectives, and KPIs

Objective and �KPI Setting

em@emilybasileo.com

8// PRIORITIZING EXPERIMENTS

Objective and �KPI Setting

Tactic �Brainstorm

• Create an experiment spreadsheet or project board
• Each experiment submission should include an experiment ID or name, description, hypothesis, experiment owner, which part of the sales funnel you’re testing (AAARRR), a target metric, and your prediction
• Collaborators can add experiment submissions directly to the sheet or project board or through a form

em@emilybasileo.com

8// PRIORITIZING EXPERIMENTS

Objective and �KPI Setting

Tactic �Brainstorm

em@emilybasileo.com

8// PRIORITIZING EXPERIMENTS

• What this solves/ why this tactic
• Description of tactic
• Expected outcome
• Quantitative KPIs to track
• Resources, thoughts, questions, considerations

Objective and �KPI Setting

Tactic �Brainstorm

em@emilybasileo.com

8// PRIORITIZING EXPERIMENTS

I: IMPACT (1 - 10 high)

• If we are successful in executing this tactic, what will the impact of this project be (on this KPI)

C: Confidence (1 - 10 high)

• How confident are we in our team to be able to execute this tactic

E: Ease (1 - 10 high)

• How easy is it for our team to execute this tactic

ICE �PRIORITIZATION

FRAMEWORK

Objective and �KPI Setting

Tactic �Brainstorm

ICE

Prioritization

em@emilybasileo.com

8// PRIORITIZING EXPERIMENTS

Objective and �KPI Setting

Tactic �Brainstorm

ICE

Prioritization

• In the Experiment backlog sheet, add columns for your Impact, Confidence, Ease, and Total ICE scores
• You can add more variables to score, but try to limit to the most relevant ones

em@emilybasileo.com

8// PRIORITIZING EXPERIMENTS

• Score = I + C + E
• Higher score = higher priority
• Want: Confidence in our ability to achieve the highest impact for the lowest effort

Objective and �KPI Setting

Tactic �Brainstorm

ICE

Prioritization

em@emilybasileo.com

8// PRIORITIZING EXPERIMENTS

• Consider ICE score, timing, bandwidth, resources, priority of KPIs/ objectives, other contextual information
• Need to balance complementary and competitive objectives
• Share roadmap with the team

Objective and �KPI Setting

Tactic �Brainstorm

ICE

Prioritization

Project �Roadmap

em@emilybasileo.com

8// PRIORITIZING EXPERIMENTS

Objective and �KPI Setting

Tactic �Brainstorm

ICE

Prioritization

Project �Roadmap

Management &�Accountability

• Delegate project owners
• Facilitate regular check-ins
• Remove bottlenecks
• Communicate results (celebrate success!)
• Make sure to document what actually happened, the resulting metric, and any other notes and learnings
• Learn and take action

em@emilybasileo.com

[OPTIONAL] BREAKOUT SESSION 7

8// PRIORITIZING EXPERIMENTS

• Identify your OMTM�
• Design at least 7 experiments to impact that metric�
• Use the ICE framework to prioritize experiments

em@emilybasileo.com

9: Building a KPI Dashboard

em@emilybasileo.com

9// KPI DASHBOARD

A KPI dashboard is a simple display of the most important information

Aggregate

Visualize

Organize

em@emilybasileo.com

9// KPI DASHBOARD

Good KPI dashboards give key stakeholders and decision makers the information they need to make actionable recommendations in an easy-to-understand format

em@emilybasileo.com

9// KPI DASHBOARD

KPI dashboards

• Clearly communicate the actual results of each KPI
• Track metrics
• Communicate outcomes of experiments
• Helps stakeholders make decisions based on outcomes
• Help visualize data

em@emilybasileo.com

9// KPI DASHBOARD

KPI Dashboard examples

Spreadsheets �and Tables

Platforms

Graphs� and Charts

em@emilybasileo.com

9// KPI DASHBOARD

Before building your KPI dashboard

• Establish the purpose of the dashboard
• Identify the key stakeholders
• Know the metrics you should track
• Understand the reason for tracking each metric

​

Designing a dashboard with these points in mind will ensure that the dashboard you build will actually be useful

em@emilybasileo.com

Let’s Chat

Emily Basileo�+1 917 647 5177�em@emilybasileo.com

​

em@emilybasileo.com

APPENDIX

APPENDIX: PROJECT MANAGEMENT PROCESS

Project Owners

• Main point person for each tactic
• Responsible for communicating project objectives, expectations, roles, timelines, successful project delivery, reporting on overall project process, and success metrics
• Keep contributors accountable

Project �Management �101

em@emilybasileo.com

APPENDIX: STRATEGIC PLANNING PROCESS

Project Owners Write Project Briefs

• More detailed information about the tactic; includes
• Team, budget, level of effort
• Defined project scope (limits the project)
• Detailed description with deliverables
• KPIs and expected outcomes
• Proposed timelines

Project �Management �101

em@emilybasileo.com

APPENDIX: STRATEGIC PLANNING PROCESS

Project Approval + Kick-off

• Project owner runs kick-off meeting with contributors and key stakeholders
• Introduce project
• Create/ share project board
• Assign responsibilities
• Set clear expectations

Project �Management �101

em@emilybasileo.com

APPENDIX: STRATEGIC PLANNING PROCESS

Ongoing Project Management + Communication

• Suggest recurring meetings at pre-specified intervals
• Updates, blockers, action items
• Record and communicate action items
• Contributors keep project management board updated and proactively communicate status
• Project owner communicates results and recommendations to stakeholders

Project �Management �101

em@emilybasileo.com