1 of 24

What Type of Lean Startup Experiment Should I Run?

V 10/10/18

Sourced from Grasshopperherder.com Lean Startup Blog

2 of 24

In class assignment: (only if I’ve listed it as an activity on the course website; otherwise ignore)

You will focus on conducting generative research on your market (customer segment), answering and refining answers to these questions:

Who is our customer?
What are their pains?
What job do they need done?
Is our customer segment too broad?
How do we find them?

You will develop customer development questions in class, and each team member will conduct customer development questions between 10/11 and 10/16.

3 of 24

Experiment or Research? Start with your assumptions and learning priorities by asking

Which type of information are we gathering: Generative or Evaluative?
What are you learning about: Customer or Product?

4 of 24

Locate the question

To simplify our search for the right method, we’ll ask two questions:

Do we need to learn about the market or the product?
Do we have a falsifiable hypothesis to evaluate, or do we need to generate a clear idea?

Mapping the intersection of these two questions gives us a 2x2 matrix:

5 of 24

Locate the question with 2x2 categories:

What type of information are we gathering: Generative or Evaluative?
What are we learning about: Market (customer) or Product?

6 of 24

Research and experiments

Generative – Research techniques which don’t necessarily start with a hypothesis, but result in many new ideas. e.g. Customer Discovery Interviews
Evaluative – Testing a specific hypothesis to get a clear yes or no result. e.g. Landing Page a.k.a Smoke tests

7 of 24

Research and experiments

If we don’t have a clear idea of who our customer is, we can do Generative Market Research, such as data mining.
Similarly, if we have a clear hypothesis of which features will solve the customer’s problems, we can do an evaluative product experiment such as Wizard of Oz testing.
If we do not know which features will lead to an acceptable solution, we can do Generative Product Research such as a Concierge Product to try and come up with new ideas.

8 of 24

Defining good hypotheses

Simple and unambiguous
Measurable
Describes a relationship
Cause and effect
Achievable
Falsifiable

9 of 24

Market and product

Some methods tell us a lot about customers, their problems, and how to reach them. For example, we can talk listen to our customers and this will help us understand their situation and what their day to day problems are.
Other methods tell us about the product or solution that will help solve that problem. We can do usability testing on a set of wireframes and see if our interface is usable. However, this won’t tell us anything about whether or not anyone will buy it in the first place.

These methods generally don’t overlap.

10 of 24

Market and product

Who is our customer?
What are their pains?
What job needs to be done?
How are they doing this job today?
Does the customer segment already have a solution to this pain?
Is this customer segment really willing to pay for a better solution for this job?
Is our customer segment too broad?
How do we find them?
How much will this customer segment pay?
How do we convince this customer segment to buy?
What is the cost to acquire a customer in this customer segment?

How can we solve this problem?
What form should this take?
How important is the design?
What’s the quickest solution?
What is the minimum feature set?
How should we prioritize?
Is this solution working?
Are people using it?
Which solution is better?
How should we optimize this?
What do people like/dislike?
Why do they do that?
Why do prospects buy from us?
Why do prospects not buy from us?

11 of 24

Index of questions

	Market	Product
Generative	Who is our customer? What are their pains? What job do they need done? Is our customer segment too broad? How do we find them?	How can we solve this problem? What form should it take? How important is the design? What’s the quickest solution? What is the minimum feature set? How should we prioritize?
Evaluative	Are they willing to pay? How much will they pay? How do we convince them to buy? How much will it cost to sell? Can we scale marketing?	Is this solution working? Are people using it? Which solution is better? How should we optimize this? What do people like? Why do they do that with my product/service?

12 of 24

Tristan Kromer: The Unofficial Startup Real Book Explained in 7 Minutes

13 of 24

A good MVP

The value proposition (not the “product”)
A defined customer, not random people getting thrown to a page
Having a channel to deliver the VP to customers (“how do we get people to the landing page?”)
Relationship: do you have the means to collect the feedback from the customer?

14 of 24

“What don’t we know?” What are you trying to learn?

Do you want to learn more about the market or the product?
Generative research: Talk to humans, do data mining. I don’t know who my customer is
Evaluative experiment: You have a solid hypothesis that you can disprove or find support for

15 of 24

2x2 categories:

What type of information are we gathering: Generative or Evaluative?
What are we learning about: Market (customer) or Product?

17 of 24

Evaluative experiments for the market

20 of 24

Picnic in the Graveyard? “Go find the 10 people who failed at this and see what went wrong”

22 of 24

“Learn ONE thing”

23 of 24