| A | B | C | D | E | F | G | H | I | J | K | L | M | N | O | P | Q | R | S | T | U | V | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1 |  | Emails A/B Test Significance Test | ||||||||||||||||||||
2 | ||||||||||||||||||||||
3 | ||||||||||||||||||||||
4 | la hutte - growth studio - lesensdelahutte.com | |||||||||||||||||||||
5 | ||||||||||||||||||||||
6 | This significance test will let you know which version of your email A/B test is the most performant - and on which metrics. | |||||||||||||||||||||
7 | ||||||||||||||||||||||
8 | ||||||||||||||||||||||
9 | This sheet is locked - copy it in your Drive to make changes, into the blank cells - the grey ones update themselves ;) | |||||||||||||||||||||
10 | ||||||||||||||||||||||
11 | ||||||||||||||||||||||
12 | KPI | Unit | Value | Results (A vs. B) | ||||||||||||||||||
13 | VERSION A | Sent volume | # | 10,000 | Uplift in Open Rate | -63% | 95% significance | YES | 99% significance | YES | p-value | 0.0000 | Z-score | 82.20 | Std. Err. of difference | 0.006 | ||||||
14 | Email opens | # | 3,000 | Uplift in Click Rate | 0% | 95% significance | NO | 99% significance | NO | p-value | 0.5000 | Z-score | 0.00 | Std. Err. of difference | 0.004 | |||||||
15 | Open rate | % | 30.00% | Uplift in Open to Click Rate | 167% | 95% significance | YES | 99% significance | YES | p-value | 1.0000 | Z-score | -22.24 | Std. Err. of difference | 0.009 | |||||||
16 | Email clicks | # | 1,000 | Uplift in Open to Conversion Rate | 167% | 95% significance | YES | 99% significance | YES | p-value | 1.0000 | Z-score | -5.94 | Std. Err. of difference | 0.004 | |||||||
17 | Click rate | % | 10.00% | |||||||||||||||||||
18 | Open to click rate | % | 33.33% | |||||||||||||||||||
19 | Conversion | # | 100 | OPEN RATE (95% significance) | ||||||||||||||||||
20 | Conversion rate | % | 1.00% | Version B | is better! | Version B has a higher probability of being opened by the recipient. | ||||||||||||||||
21 | Open to conversion rate | % | 3.33% | |||||||||||||||||||
22 | Click to conversion rate | % | 10.00% | OPEN TO CLICKS RATE (95% significance) | ||||||||||||||||||
23 | Version A | is better! | Version A has a higher probability of being clicked by the recipients who open the email. | |||||||||||||||||||
24 | VERSION B | Sent volume | # | 10,000 | ||||||||||||||||||
25 | Email opens | # | 8,000 | OPEN TO CONVERSIONS RATE (95% significance) | ||||||||||||||||||
26 | Open rate | % | 80.00% | Version A | is better! | Version A has a higher probability of converting opening recipients. | ||||||||||||||||
27 | Email clicks | # | 1,000 | |||||||||||||||||||
28 | Click rate | % | 10.00% | |||||||||||||||||||
29 | Open to click rate | % | 12.50% | CHECKLIST | ||||||||||||||||||
30 | Conversion | # | 100 | • Keep subject line from Version B | ||||||||||||||||||
31 | Conversion rate | % | 1.00% | • Keep sender identity from Version B | ||||||||||||||||||
32 | Open to conversion rate | % | 1.25% | • Keep time sending period from Version B | ||||||||||||||||||
33 | Click to conversion rate | % | 10.00% | • Keep CTA from Version A | ||||||||||||||||||
34 | • Keep messaging & approach from Version A | |||||||||||||||||||||
35 | ||||||||||||||||||||||
36 | ||||||||||||||||||||||
37 | Do you want more ideas to improve further your rates? We can help! | |||||||||||||||||||||
38 | ||||||||||||||||||||||
39 | ||||||||||||||||||||||