Experimentation Roadmap Template
 Share
The version of the browser you are using is no longer supported. Please upgrade to a supported browser.Dismiss

View only
 
 
ABCDEFGHIJKLMNOPQRSTUVWXYZ
1
Test PlanningResults
2
TitleDescriptionHypothesisSupporting Data / InsightsTarget Metric
Engineering Effort (t-shirt sizing)
Current Conversion Rate %
Goal Conversion Rate %
Relative Change
# of Variants
Sample Size Per Variant Per Day
Minimum Sample Size
Estimated Test Duration (Days)
Test ResultsActual Test Duration
Net Impact on Business
Next Steps
3
Tiering TestTest anchoring new customers towards annual subscriptions instead of monthly subscriptions to understand the net effect on revenueIf we emphasize annual subscription over monthly subscription on the subscription landing page, a higher percent of our customers will choose the annual subscription, while some will choose not to subscribe at all, yet still lead to a net gain in revenue80% of current subscribers opt for monthly yet 80% of them unsubscribe after 3 months already. However 20% of subscribers that choose annual only have a 5% 1 year churn rate. Annualized Revenue1 week web app eng, 1 day design, 1 day iOS eng, 1 day Android eng20% annual subscription25% annual subscription25%21006707What was the actual relative change in the key metric?Useful for tracking whether or not tests take longer than planned for future planning purposesCalculate overall impact on top level business metricsTurn it on? Deprecate it? Try new variants?
4
Use the sample size calculator from Optimizely to calculate the minimum sample size needed for high confidence in the statistical detection: https://www.optimizely.com/sample-size-calculator/?conversion=20&effect=25&significance=95
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
Loading...