Experimentation Program ROI Calculator

	A	B	C	D	G	H	I	K	L	M	N	O	P
1					Settings >	Haircut		False Discovery Rate
2						Power	Haircut	Significance (two-tailed) / Win Rate	30%	25%	20%	15%	10%
3		Experimentation Program ROI Calculator				99%	1.00%	80.00%	23.8%	35.0%	42.9%	61.9%	100.0%
4						95%	3%	90.00%	11.1%	15.0%	20.0%	28.9%	46.7%
5		https://jcgiusto.com				90%	5%	95.00%	5.4%	7.5%	9.7%	14.0%	22.6%
6						80%	13%	98.00%	3.0%	4.0%	5.0%	9.0%	12.0%
7		Pogram Metrics				70%	20%	99.00%	1.0%	1.5%	1.9%	2.7%	4.4%
8	N	# of Experiments per Month (N)	30			Taken from Kohavi's Calculations		Based on Kohavi's paper
9	W	Win Rate (W)	15%	< The higher, the less False Positive Rate
10						Forecasting
11		Statistical Parameters				Conservative	100%
12		Average MDE	2%			Neutral	75%
13		Significance	95%	< The higher, the less False Positive Rate		Optimistic	50%
14		Power	90%	< The higher, the less the shortcut
15		Test Type	Two-tailed
16
17		Forecast	Conservative
18
19		Business Metrics
20		Avg Monthly Revenue	€ 95,000.00
21		Gross Margin	60%
22
23		Business Impact
24		Yearly Uplift in CR	139.67%	< (1 + (MDE * (1-haircut)) ^ (N * W * ( 1 - FDR)) − 1
25		Yearly Revenue Uplift	€ 1,592,209.25	< Uplift % * Monthly Revenue * 12
26		Yearly Gross Profit Uplift	€ 955,325.55	< Yearly Revenue Uplift * Gross Margin
27
28		Experimentation Cost
29		Monthly Retainer	€ 10,000.00
30		Salaries	€ 5,000.00
31		A/B Testing Tool	€ 6,000.00
32
33		Exp Program ROI
34		Yearly Gross Profit - Retainer	€ 703,325.55
35		ROI	7.96
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100