Surveying for Startups

A guide to getting actionable feedback.

​

UW Business School

Gaurav Oberoi, goberoi.com

About me

• Founded and sold 2 startups
• Founded Lexion.ai from AI2 Incubator; raised $4.2M from Madrona, Wilson Sonsini
• CS undergrad, Amazon engineer, Product Mgr
• 6+ years in market research:
• Last startup made phone polling easy (DNC)
• Built a $20M revenue product at SurveyMonkey

SurveyMonkey Audience + Contribute

What I’ll talk about:

​

1. Practical tips on designing surveys
2. Advice on finding respondents

​

But first, some examples

As Seen on TV → What will sell?

They test dozens of product ideas per month using surveys.

• What is your reaction?
• Is it good value?
• Will you buy it?
• Would you recommend it?

Redfin → What to put in our ads?

Royal Bank of Canada → Buy Netflix stock?

Contract Management?

Contract Management?

Contract Management?

Contract Management?

Getting feedback for your startup idea

3 go-to approaches

1. Pick up the phone and talk
2. Test it, with a landing page
3. Run a survey

1. Talk to customers

• Write an Amazon PR-FAQ or similar
• Create a plan: intro, explanation, 5-10 questions
• Aim to talk to 20+ people (50% will be duds)
• Ask:
• More about their problems, than your solution
• Are ready to sign up for $X
• Can make more intros

2. Test, with a landing page

3. Run a survey

• Great direct feedback
• Need to have a good answer for respondents
• Not a panacea, but surprisingly valuable

Practical tips on running surveys

Steps

1. Make sure you can find/afford respondents
2. Write your conclusion first (<-- this is key!)
3. Write a draft survey
4. Test thoroughly, and revise
5. Run and analyze�

Expect to spend 8-20 hours, or 1-2 weeks.

1. Ensure you can afford/find respondents

​

This is a bigger topic, let’s revisit in a few minutes.

2. Write your conclusion first

Good: write bulleted goals and review.

​

Better: write your final report with blanks.

​

3. Draft your survey

Keep it short

• < 20 q
• 2-5 minute limit
• Set expectations

3. Draft your survey (cont.)

Keep it simple

• Avoid matrix
• Use multichoice
• Only 1-2 open ended
• Minimal text
• No industry lingo

3. Draft your survey (cont.)

​

Some best practices:

• Use 1-5 rating not Y/N
• Rotate answer choices
• Demographics at the end
• Inclusive answer choices
• Re-ask targeting questions

3. Draft your survey (cont.)

​

Some expectations:

• 5-20% dropout
• 5-10% garbage responses
• Incorrectly targeted respondents

4. Test and revise

• Get 10+ responses from friends.
• Can you fill out the conclusion?
• Understandable & answerable?

4. Test and revise (cont.)

• Mobile!
• Time limit?
• Skip-logic errors?

5. Run and analyze

• Look for age, gender, income skews
• Categorize open-ended
• For ratings: top2/bottom2
• Fill out conclusion, present

Getting the right respondents

How big a sample?

• Start with 50 answers
• Aim for 400-500

How to find the right people?

• Buy from a panel or ad targeting on FB
• Find through Reddit/LinkedIn/forums/email lists/friends
• Ask your customers if you have a relationship
• Offer incentives where possible.

Buying respondents from a panel

• Costs $1-$4 per response on average
• Broad targeting works, but narrow fails, e.g.:
• Lives in Seattle
• Is a dentist
• Uses Zune�

In conclusion...

Conclusion

• Start simple: talk to 20 customers
• Research is expensive: budget $ and time wisely
• But not as expensive as building the wrong thing
• Be objective, you can tell any story with data
• Take a leap of faith, you won’t get perfect answers.

Questions?

Gaurav Oberoi, goberoi.com