| A | B | C | D | E | F | G | |
|---|---|---|---|---|---|---|---|
1 | Claimed Confidence Level: 95% | True Confidence Levels Are Reported To Within 0.7% Accuracy | |||||
2 | 35,000 simulations were run for each scenario to estimate the true confidence level. In each scenario (e.g. 1-sample, n=5, ChiSq(1)), the four confidence interval methods were applied across the same simulated data. For the bootstrap methods, 10,000 bootstrap samples were used. | ||||||
3 | The 2*SE bootstrap method is only currently available for a couple of cases and so far has performed about on par with that of the percentile bootstrap. | ||||||
4 | |||||||
5 | >93% Capture | 91-93% Capture | 88-90.9% Capture | 84-87.9% Capture | 80-83.9% Capture | 75-79.9% Capture | <75% Capture |
6 | |||||||
7 | Population(s) Distribution, i.e. the Truth | ||||||
8 | 1-sample, n=5 | ChiSq(0.5), µ=0.5 | ChiSq(1), µ = 1 | ChiSq(3), µ = 3 | ChiSq(5), µ = 5 | N(0,1), µ = 0 | |
9 | T Confidence Interval | 73.72% | 82.71% | 90.38% | 92.19% | 94.97% | |
10 | T Conf. Int. using Bootstrap SE | 71.78% | 80.71% | 88.23% | 90.28% | 93.12% | |
11 | Z Confidence Interval | 67.65% | 75.73% | 83.09% | 84.83% | 87.67% | |
12 | Bootstrap Conf. Int via 2*SE | 65.89% | 73.93% | 80.24% | 82.08% | 85.29% | |
13 | Bootstrap Conf. Int via Percentile | 66.72% | 73.8% | 80.34% | 81.43% | 83.64% | |
14 | |||||||
15 | 1-sample, n=10 | ChiSq(0.5), µ=0.5 | ChiSq(1), µ = 1 | ChiSq(3), µ = 3 | ChiSq(5), µ = 5 | N(0,1), µ = 0 | |
16 | T Confidence Interval | 79.73% | 86.22% | 91.5% | 92.89% | 95.03% | |
17 | T Conf. Int. using Bootstrap SE | 78.72% | 85.31% | 90.53% | 91.94% | 94.04% | |
18 | Z Confidence Interval | 76.84% | 83.3% | 88.5% | 89.72% | 91.96% | |
19 | Bootstrap Conf. Int via 2*SE | 76.33% | 82.63% | 87.67% | 88.64% | 90.9% | |
20 | Bootstrap Conf. Int via Percentile | 77.19% | 83.16% | 87.56% | 88.53% | 90.22% | |
21 | |||||||
22 | 1-sample, n=30 | ChiSq(0.5), µ=0.5 | ChiSq(1), µ = 1 | ChiSq(3), µ = 3 | ChiSq(5), µ = 5 | N(0,1), µ = 0 | |
23 | T Confidence Interval | 87.67% | 91.13% | 93.58% | 93.93% | 94.81% | |
24 | T Conf. Int. using Bootstrap SE | 87.29% | 90.74% | 93.23% | 93.63% | 94.4% | |
25 | Z Confidence Interval | 86.82% | 90.13% | 92.67% | 93.11% | 93.8% | |
26 | Bootstrap Conf. Int via Percentile | 87.63% | 90.58% | 92.46% | 92.88% | 93.31% | |
27 | |||||||
28 | 2-sample, n1=n2=5 | ChiSq(0.5), µ_diff=0 | ChiSq(1), µ_diff=0 | ChiSq(3), µ_diff=0 | ChiSq(5), µ_diff=0 | N(0,1), µ_diff=0 | |
29 | T Confidence Interval | 97.97% | 97.01% | 95.69% | 95.49% | 95.07% | |
30 | T Conf. Int. using Bootstrap SE | 96.14% | 94.81% | 93.45% | 93.32% | 92.67% | |
31 | Z Confidence Interval | 95.09% | 93.52% | 92.17% | 92.04% | 91.35% | |
32 | Bootstrap Conf. Int via Percentile | 85.45% | 86.43% | 87.51% | 87.79% | 87.9% | |
33 | |||||||
34 | 2-sample, n1=n2=10 | ChiSq(0.5), µ_diff=0 | ChiSq(1), µ_diff=0 | ChiSq(3), µ_diff=0 | ChiSq(5), µ_diff=0 | N(0,1), µ_diff=0 | |
35 | T Confidence Interval | 97.21% | 96.34% | 95.62% | 95.27% | 94.86% | |
36 | T Conf. Int. using Bootstrap SE | 96.19% | 95.16% | 94.5% | 94.14% | 93.65% | |
37 | Z Confidence Interval | 95.81% | 94.8% | 94.09% | 93.75% | 93.23% | |
38 | Bootstrap Conf. Int via Percentile | 89.51% | 90.49% | 91.65% | 91.6% | 91.88% | |
39 | |||||||
40 | 2-sample, n1=n2=30 | ChiSq(0.5), µ_diff=0 | ChiSq(1), µ_diff=0 | ChiSq(3), µ_diff=0 | ChiSq(5), µ_diff=0 | N(0,1), µ_diff=0 | |
41 | T Confidence Interval | 95.97% | 95.34% | 95.13% | 95.17% | 95.07% | |
42 | T Conf. Int. using Bootstrap SE | 95.57% | 94.96% | 94.75% | 94.76% | 94.73% | |
43 | Z Confidence Interval | 95.48% | 94.86% | 94.68% | 94.69% | 94.66% | |
44 | Bootstrap Conf. Int via Percentile | 92.36% | 92.97% | 93.69% | 93.88% | 94.18% | |