GiveWell's Zusha cost-effectiveness analysis (CEA)
 Share
The version of the browser you are using is no longer supported. Please upgrade to a supported browser.Dismiss

 
View only
 
 
Still loading...
ABCDEFGHIJKLMNOPQRSTUVWXY
1
Study 1 InputValueExplanationSourceStudy 2 InputValueExplanationSource
2
Reported USD cost of program per vehicle per year, as reported in study
$7.00
Includes sticker & lottery costs, probably exclude costs of sticker distribution & lottery operation
Pg. 21, "The cost of the intervention was roughly $2 per vehicle for the stickers, and $5 per vehicle per year for the lottery, or a total of $7,000 per 1,000 vehicles per year." [this is the extent of info on cost in the paper]
Reported USD cost of program per vehicle per year, as reported in study
$12.50
Includes sticker & lottery costs, excludes radio costs, may exclude sticker distribution and/or lottery operation costs
E4668, "The costs of printing the stickers and running the lottery amounted to about $100,000." [This is the extent of info on cost in the paper]
3
Budgeted USD cost of program per vehicle per year, from Zusha budget for Kenya
$22.00
Cost estimate based on recent Zusha budget for Kenya, including all overhead
Private budget document
Budgeted USD cost of program per vehicle per year, from Zusha budget for Kenya
$22.00
Cost estimate based on recent Zusha budget for Kenya, including all overhead
Private budget document
4
Total vehicles1155From studyTable 1Total vehicles8000From studyE4668
5
Total cost used in CEA$25,410.00Uses the budgeted cost per vehicleTotal cost used in CEA$176,000.00Uses the budgeted cost per vehicle
6
% reduction of vehicles involved in accidents per driving-year
5.00%From studyPg. 20
% reduction of vehicles involved in accidents
1.70%From studyTable 4
7
Accidents avoided per driving-year57.75CalculationAccidents avoided136Calculation
8
Deaths per accident0.105From 2nd studyTable 2Deaths per accident0.105From studyTable 2
9
Deaths averted6.06CalculationDeaths averted14.28Calculation
10
11
Ratio of serious injuries to deaths3.8
There are two options for estimating # of injuries and DALYs from injuries: 1) Use the ranges provided by the study authors to take a rough best guess, 2) Use some quick, rough analysis I did by looking at a GBD paper on the relative burden of traffic deaths and injuries. I think a semi-conservative version of the first method suggests putting somewhere between 2 to 5 for "Ratio of serious injuries to deaths" and 5 to 10 for "DALYs per serious injury". Roughly, multiplying the bottom end of these ranges (2*5) would give 10 DALYs from injuries per death. The second method suggests 3.8 hospital-requiring injuries per death and 2 DALYs per injury, but I'm not very confident in the analysis. This would suggest 3.8*2=7.6 DALYs from injuries per death.
Relevant quotes for method 1: "We further assume that disability adjustments mean that each injury avoided saves the equivalent of between 5 and 15 y of healthy life and that the ratio of injuries to deaths is between 2 and 10–1...Even these numbers are conservative. The WHO considers the ratio of injuries to deaths to be between 20 and 50 to 1.3 (see above). Our data suggest a ratio of 10 to 1, although a large fraction of injuries are relatively minor soft tissue injuries." Pg. E4668, Study 2. Rough notes from our call where we asked about this: "Ratio of injuries to death - source is WHO, which reports 1.3 M deaths each year and 150 M injuries. We wanted to be on the conservative end of that. Data in this area is woefully incomplete, so it’s hard to tell. Even insurance company data isn’t definitive in this regard. Ranges are wide, but even at most conservative end it still looks cost-effective compared to others. That derives from poor data, generally. How many DALYs an injury is worth - 5 to 15, no one really knows. WHO makes estimates of these things - ‘expert opinion’ = people guessing; we don’t have great data. 2 to 10 DALY range comes from insurance data - best we can figure the range of injuries to death; range of soft-tissue injuries, which some people think of as fraud."
A World Bank/GBD paper gives a ratio of hospital-requiring injuries:deaths in SSA of ~3.8:1. "Road injuries killed 231,000 people in sub-Saharan Africa in 2010, accounting for almost one-fifth of the global road injury death toll. In addition, there were over 8 million non-fatal injuries, of which 885,000 were severe enough to warrant hospital admission if adequate access to medical care were available." I'm not sure what the average DALY burden is for an injury "severe enough to warrant hospital admission." I think this chart may suggest that disability makes up roughly ~12% of the total health burden imposed by road safety issues: https://www.dropbox.com/s/2vy2obu085ikcpw/Screenshot%202016-09-26%2014.34.37.png?dl=0 (from Pg. 61 of http://pubdocs.worldbank.org/en/356861434469785833/Road-Safety-Burden-of-Injuries-in-Africa.pdf ). But, I'm pretty unsure whether i'm reading that chart correctly. http://pubdocs.worldbank.org/en/356861434469785833/Road-Safety-Burden-of-Injuries-in-Africa.pdf
Ratio of serious injuries to deaths3.8
There are two options for estimating # of injuries and DALYs from injuries: 1) Use the ranges provided by the study authors to take a rough best guess, 2) Use some quick, rough analysis I did by looking at a GBD paper on the relative burden of traffic deaths and injuries. I think a semi-conservative version of the first method suggests putting somewhere between 2 to 5 for "Ratio of serious injuries to deaths" and 5 to 10 for "DALYs per serious injury". Roughly, multiplying the bottom end of these ranges (2*5) would give 10 DALYs from injuries per death. The second method suggests 3.8 hospital-requiring injuries per death and 2 DALYs per injury, but I'm not very confident in the analysis. This would suggest 3.8*2=7.6 DALYs from injuries per death.
Relevant quotes for method 1: "We further assume that disability adjustments mean that each injury avoided saves the equivalent of between 5 and 15 y of healthy life and that the ratio of injuries to deaths is between 2 and 10–1...Even these numbers are conservative. The WHO considers the ratio of injuries to deaths to be between 20 and 50 to 1.3 (see above). Our data suggest a ratio of 10 to 1, although a large fraction of injuries are relatively minor soft tissue injuries." Pg. E4668, Study 2. Rough notes from our call where we asked about this: "Ratio of injuries to death - source is WHO, which reports 1.3 M deaths each year and 150 M injuries. We wanted to be on the conservative end of that. Data in this area is woefully incomplete, so it’s hard to tell. Even insurance company data isn’t definitive in this regard. Ranges are wide, but even at most conservative end it still looks cost-effective compared to others. That derives from poor data, generally. How many DALYs an injury is worth - 5 to 15, no one really knows. WHO makes estimates of these things - ‘expert opinion’ = people guessing; we don’t have great data. 2 to 10 DALY range comes from insurance data - best we can figure the range of injuries to death; range of soft-tissue injuries, which some people think of as fraud."
A World Bank/GBD paper gives a ratio of hospital-requiring injuries:deaths in SSA of ~3.8:1. "Road injuries killed 231,000 people in sub-Saharan Africa in 2010, accounting for almost one-fifth of the global road injury death toll. In addition, there were over 8 million non-fatal injuries, of which 885,000 were severe enough to warrant hospital admission if adequate access to medical care were available." I'm not sure what the average DALY burden is for an injury "severe enough to warrant hospital admission." I think this chart may suggest that disability makes up roughly ~12% of the total health burden imposed by road safety issues: https://www.dropbox.com/s/2vy2obu085ikcpw/Screenshot%202016-09-26%2014.34.37.png?dl=0 (from Pg. 61 of http://pubdocs.worldbank.org/en/356861434469785833/Road-Safety-Burden-of-Injuries-in-Africa.pdf ). But, I'm pretty unsure whether i'm reading that chart correctly. http://pubdocs.worldbank.org/en/356861434469785833/Road-Safety-Burden-of-Injuries-in-Africa.pdf
12
DALYs per serious injury2See explanation aboveSee explanation aboveDALYs per serious injury2See explanation aboveSee explanation above
13
DALYs averted from injuries46.0845CalculationDALYs averted from injuries108.528Calculation
14
15
External validity adjustment (including adjustment for long-term behavior change)
75%
Results only from 1 country, but don't have reason to think it's particularly well-suited to Kenya. We'll see results of RCTs from other countries. We don't have a great sense of how effective the intervention remains over time. In the second RCT, it seemed like bus compliance with the intervention fell over time: see Figure 2, Pg. E4666
-
16
Replicability adjustment82%
See sheet "Replicability adjustment" for details. Two RCTs, good proxy measure, some speed data. Smaller effect in second RCT than in first. One consideration here: The studies are too low-powered to directly measure a reduction in accidents involving death, so the cost-effectiveness calculations instead use better-powered estimates of reductions in accidents or in accidents involving injury or death. This seems pretty reasonable to me, so I don’t discount much for this, but it’s a gap in the evidence.
-
17
Total discount61%Calculation
18
19
Note: the below only looks at cost per traffic death averted, excluding injuries
20
Cost per road accident death averted in study 1 (excluding injuries and discounts)
$4,190Calculation
21
Cost per road accident death averted in study 2 (excluding injuries and discounts)
$12,325Calculation
22
Cost per road accident death averted (combining studies) (excluding injuries and discounts)
$9,900Calculation
23
24
Cost per road accident death averted (including discounts, using both studies, excluding injuries)
$16,134Calculation
25
26
Value of a road accident death averted relative to an AMF death averted (i.e., you would trade this number of AMF deaths averted for 1 road accident death averted)
4
Average age of passenger in matatu: ~33 (see 2nd study, Table 2). Std dev of age of passenger: 8.38 (see 2nd study, Table 2). For this value, I looked at staff values in end of 2016 CEA and pulled something in the middle. See Row 64: https://docs.google.com/spreadsheets/d/1KiWfiAGX_QZhRbC9xkzf3I8IqsXC5kkr-nwY_feVlcM/edit#gid=2064365103
-
27
Zusha's cost per equivalent AMF death averted
$4,033
Calculation. This has the same meaning as "Cost per equivalent life saved" in the end of year 2016 CEA.
-
28
29
Note: the below adds injuries to the expected benefits of the intervention
30
DALYs per road accident death averted30
I eyeballed the total number of DALYs for an adult death averted that people seemed to be using in the end of 2016 CEA (people put in these values for the bednets CEA). See Row 64: https://docs.google.com/spreadsheets/d/1KiWfiAGX_QZhRbC9xkzf3I8IqsXC5kkr-nwY_feVlcM/edit#gid=2064365103
-
31
Cost per equivalent road accident death averted (including injuries, combining studies, including discounts)
$12,873Calculation-
32
Zusha's cost per equivalent AMF death averted (including injuries, combining studies, including discounts)
$3,218Calculation-
33
34
AMF: rough median cost per equivalent AMF death averted
$3,200
Rough approximation from the end of year 2016 CEA. See https://docs.google.com/spreadsheets/d/1KiWfiAGX_QZhRbC9xkzf3I8IqsXC5kkr-nwY_feVlcM/edit#gid=1034883018
-
35
GD: rough median cost per equivalent AMF death averted
$11,000
Rough approximation from the end of year 2016 CEA (not simply using medians). See https://docs.google.com/spreadsheets/d/1KiWfiAGX_QZhRbC9xkzf3I8IqsXC5kkr-nwY_feVlcM/edit#gid=1034883019
-
36
37
Zusha is X times as cost-effective as GD3.4Calculation
38
39
Sources tableLink
40
Study 1
http://faculty.georgetown.edu/wgj/papers/Matatu-paper-July2410.pdf
41
Study 2
http://www.pnas.org/content/112/34/E4661.full
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
12%20.21%#DIV/0!
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
Loading...
 
 
 
CEA
Replicability adjustment
Misc
Lottery winning benefits
Data background check