ABCDEFGHIJKLMNOPQRSTUVWXYZAAABACADAEAFAGAHAIAJAKALAMANAOAPAQARASATAU
1
Study, Author, YearGiveWell internal linkLocationEnvironmentPublication YearStudy Start YearDesignTiming of recruitmentBaselineInterventionEndline data collection methodEffectMeasles included in study?Effect on timeliness?PowerBalanced at baseline?Sample size (Control + Experiment)Analysis sample (Control + Experiment)Loss to follow-upReasons for loss to follow-upDeceased infants included in analysis?Implementation: difference between per-protocol and ITT effects?Implementation: at least 1 SMS receivedSent message %Recalled receiving %BlindingTime of Reminder(s) in Days Before AppointmentOther interventions?Message TextCosts Per Message Sent (USD)Other Costs in USD (name)Other notes
2
Kawakatsu et al. 2020https://givewell.box.com/s/cwh9tax2fjykpi2kp8vj0ppnzs6qzcplKenyaUrban20202019RCTAll clients of 33 PHCs with immunization, ANC, or FP appointmentsNot discussedSMS text reminder 2 days before upcoming appointments, and 7 days after missed appointmentReturns to PHC; verified by scanning QR code on mobile app4.8-6% higher for each timing (as scheduled, 7 days late, etc.)Yes, but not directly (see cell note)Yes. 4.8-6% for each time (as scheduled, 7 days late, etc.)2.5% increase in immunization visitsNot assessed directly but may be some imbalance (see cell note)87268160Not discussedNot discussedNot discussedNot discussed.Not measuredYes2 before, 7 afterNo‘‘Dear [first name of the child or the client], your [immunization] appointment has been scheduled for [date]. Please come back to the health facility.”$0.08 (0.01 for message, 0.07 for data)$0.28 is recurrent cost per appt., $7.90 for each return case
We used the data reported in Table 6 to calculate start-up and re-current costs over both groups as $4.35 per client
3
Haji et al. 2016https://givewell.box.com/s/9lic1d3jfdi8hknybi3z7m5cnzmehj03KenyaMixed/unclear; included urban20162014RCT (cluster-randomized, 9 groups, 3 in SMS arm, 3 in sticker arm)Children aged <12 months presenting for their first dose of pentavalent vaccine were enrolled.83% PENTA3SMS reminderIn person at clinic. "Data were collected by study nurse and principal investigator during routine working hours at the maternal child health clinic on a daily basis. Caretakers were interviewed face to face using a pretested standard questionnaire." p. 4. and "Any caretaker who failed to return the child for vaccinations two weeks or more after the expected completion of third pentavalent dose was contacted by the investigator to establish reasons for missed vaccinations." p. 4. This did not capture vaccinations that took place at a different clinic. "If a care giver took the child to another facility for second or third pentavalent dose, the system considered the child unvaccinated,leading to misclassification, however, a sensitivity analysis that assumed that these children were actually vaccinated had no effect on the general observed difference between the inteventions." p. 7.13% point increase in PENTA3 (CSH: "or 8.45% point if take into account people who took child to different clinic")NoOnly 1-2 day significant effect; however this difference in delay is not clinically significant in terms of susceptibility to disease. "The mean delay in receiv- ing second dose of pentavalent vaccine on the sched- uled date in the SMS intervention group was 0 days (standard deviation (SD): 1.2), in the control group the mean delay was one day (SD: 4.3), while in the sticker group, the mean delay was one day (SD: 6.3). There was a significant difference in the mean delay in days between the SMS and Control group (p < 0.001)," and "The mean delay in receiving the third pentavalent dose on the scheduled date in the SMS intervention group was 0 days (SD: 2), in the control group, two days (SD: 7) and in the sticker group, two days (SD 6). There was a significant difference in the mean delay in days between the SMS and Control group (p < 0.001),""Sample size calculation was done using Casagrande et al. [27] formula for comparing two proportions to detect a 15 % decrease in the drop-out vaccination rate for each of the three intervention groups, assuming a dropout rate for the third dose of pentavalent of 15.6 % [28], study power of 80 %, and confidence level of 95 %. The minimum sample size was 372 participants per interven- tion arm."No clear concerns744Not discussedNot discussedNot discussedYes, counted as non-vaccinated (rather than as lost to followup).Not discussedNot discussedUnclear. Probably not. "Data were collected by study nurse and principal investi- gator during routine working hours at the maternal child health clinic on a daily basis."2,0NoN/A0.020.27 (full cost per child reminded up to 6 months)
4
Ceballos et al. 2020GuatemalaRural20202018RCT (cluster with crossover)Household with child under 2 and pregnant women, with cell phone and 1 literate memberNot discussedSMS and voice reminders (separate treatment arms)Valid responses to SMS or voice callsRR 0.96 (from meta-analysis)YesNo effectNot discussedYes1542Not discussedNot discussedNot discussedNot discussedNot discussedNot discussedNo7Phone callsNot discussed$1.12Not discussed
Cultural and linguistic challenges; phone calls more effective than text messages
5
Domek et al. 2019GuatemalaUrban and rural20192016RCTInfants 6w-6m at first imm. visitNot discussedSMS reminders before infants’ 2nd/3rd vaccine visitsEIR11.5pp increase in on-time 2nd visit (see cell note for other findings)NoYes. 11.5pp increase in on-time 2nd visit (see cell note for other timeliness findings)10% differenceYes7206628% (29 in each group)Did not show up to visit 2Not discussedNo difference84.1% for visit 2, 72.6% for visit 384%Yes3, 2, and 1NoYour child [autopopulate child’s name] is due on [autopopulate date] at [autopopulate clinic name] for vaccines.$0.04Not discussed
6
Ekhaguere et al. 2019NigeriaSemi-rural20192016RCTMothers of healthy newborns at health centers, with phone, planned to immunize75% PENTA3Voice and text reminders 2 and 1 days before scheduled Penta-1, 2, 3 and measlesPhone audits (protocol changed b/c of strike)8pp increase in 12m series (p=0.03), but not sig for Penta3 YesYes. 10pp increase in all immunizations received within 1 week of schedule10% differenceYes6006008%Not discussedYesNo difference (for Penta3, not sure about others)86-92%89%Partial2, 1NoReminder from MCH–Your baby's next immunisation visit is in 2 days (or 1 day as appropriate). Immunisation protects your child against killer diseases. Please bring your baby for this visit$0.01Not discussed
7
Seth et al. 2018https://www.researchgate.net/publication/323758724_Mobile_Phone_Incentives_for_Childhood_Immunizations_in_Rural_IndiaIndiaRural20182016RCTChildren <24m33% at enrollmentSMS reminders (and SMS reminders + mobile topup)Immunization visits (biometrically verified)No effect from reminders alone.NoNo (only with incentive)Not describedYes549549Not discussedNot discussedNot discussedNot discussedUnclearPartialNot discussedNoNot discussedNot discussedNot discussed
8
Bangure et al. 2015https://givewell.box.com/s/ygylkjnb8sxv4qpxtcalqw26i90peqdsZimbabweUrban20152013RCTWoman or caregiver was recruited into the study soon after delivery or during the 3rd and 7th day visits after delivery of the baby.75% PENTA3SMS reminder"All the respondents in both intervention and non intervention groups were followed up at 14 weeks." p. 3. Presumably by phone, so data collection relied on caregiver report. None were lost to followup.20% point increase in 3rd visit, p. 3.NoYes. "The respondents who received short message services reminders were 89% less likely to delay in having their children immunized at 6 weeks than those who were in the control group (p < 0.001). The respondents who re- ceived short message services reminders were 81% less likely to delay in having their children immunized at 10 weeks than those who did not receive short message services (p < 0.001). The respondents who received short message reminders were 75% less likely to delay than those who did not receive the messages (p < 0.001)." p. 5. See pp. 3-4: Median delay for visit 1 in days (treatment, control) was 0, 2. Visit 2: 0, 5. Visit 3: 0, 10.Not discussed.No clear concerns3043040%"All the respondents who were enrolled into the study at the beginning of the study were all followed up and none were lost to follow up." p. 5.N/A or Yes. "All the respondents who were enrolled into the study at the beginning of the study were all followed up and none were lost to follow up." p. 5.Not discussedNot discussed. "all [SMS messages] were delivered to the study respondents". p.3. It is unclear how this was measured.Unclear, not discussed.7,3,1NoThe translated messages were as follows; A week before appointment date: − “Immunization protects your child against killer diseases such as polio, whooping cough, diphtheria, measles, pneumonia and tuberculosis. You are reminded that the vaccination appointment will be due in 7 days time from today.” Three days before appointment: − “You are reminded that the vaccination appointment will be due in 3 days from today.” A day before appointment: − “Your vaccination appointment is due tomorrow, visit the nearest clinic”.0.080.99 (full cost per child up to 18 months, with 3 messages per visit, estimate), 0.33 ("Capturing of data per message", estimate)
9
Eze and Adeleye 2015https://givewell.box.com/s/vh5bysoztkx135rswqam2qugnu8nuutxNigeria (South-South)Urban20152010RCT"Selection criteria was bringing child for routine immunisation for the first or second schedules of RI,and consenting to participating in the study. The second schedule was added because some babies get their first shots at their bedside and as such, at their first visits to the RI centre they are given the second schedule of vaccines." p. 5. ... "Each child recruited into the study at its first immunisation session (BCG) was followed-up for 18 weeks while those who were recruited at their second session (DPT1) were followed up for 12 weeks." p. 5.57.9% PENTA3SMS reminder, recall as wellImmunization registers. "Weekly, updates of immunization data were collected by research assis- tants from immunization registers at all 8 health facilities included in the study." p. 5. And phone follow-up with those who did not show up. "When study duration elapsed, SMS messages were sent from a phone with an identifiable number to all study participants who had not shown up for vaccination for greater than 2 cycles of each facility’s schedule. Some clients responded and phone calls were made to those who did not respond to the SMS messages. All those who could not be reached, or followed were excluded from the analysis." p. 5.8.7% point increase, no p-value givenNoYes. "Logistic regression showed that receiving SMS reminders resulted in an earlier receipt of DPT3 in the intervention group, OR 1.47 (95% CI: 1.1 – 2.0) (Table 2)" p. 6.Not discussed.Not discussed.10019059.6%"Ninety-six clients (9.6%) were lost to follow up through infant deaths, family relocation, change of preferred health facility for immunisation uptake, and other unknown reasons." p. 5.No.Not discussedUnknown. Discussed as a limitation of study. p. 6.Yes. "Data for both intervention and control groups were collected together in the same collection forms to prevent bias on the minds of the data collectors. Data collectors could not tell if a client was in the intervention or control group." p. 5.1, with additional messages for those who miss appointments: "Reminder messages were sent a day before clients’ appointments while recall message were sent one day to the next immunization session each time babies in the intervention group defaulted from appointments." p. 5.NoExample message: "Dear client, your child is due for his/her next dose of vaccines tomorrow Tuesday 20/7/10. Kindly bring your child to Hospital X for vaccination at 8am. Please come with immunization card. Thank you."N/A0.15 (full cost per additional child immunized, estimate, originally in Naira 27.47)
10
Domek et al. 2016https://givewell.box.com/s/rdzyreqtgavpybskf9tn5mdrp47ah96hGuatemalaUrban, low-income20162013RCT"Parents of infants between the ages of 8 and 14 weeks presenting for the first dose of the 3-dose infant primary immunization series were eligible if they owned a mobile phone with SMS text messaging capability. At least one parent had to be literate and able to use SMS technology." p. 2438.80.7% PENTA3SMS reminder"Research nurses collected data at visits 2 and 3, including information regarding family mobile phone use, receipt of SMS reminders, and vaccines given. Immunization records were able to be confirmed for children who returned to either of the two study sites." pp. 2438. "We experienced another limitation when we were unable to track childhood vac- cination records outside of the study clinics." p. 2441.Not statistically significant, visit 2 +4.9% at p = .12, visit 3 + 3.7% at p =.69.NoNone. See Table 3, p. 2441.Not discussed."Upon unlocking the database, it was noted that a disproportionate number of participants were randomized into the intervention group during the first month and the usual care group during the second month of the enrollment period." p. 2438. "Parents in the usual care group had significantly higher income with more fathers working; otherwise, there were no significant differences between the baseline demographics of intervention and usual care children and their parents (Table 1)." p. 2439.32126517.4%Not discussed.No."Per protocol analyses were consistent with intention to treat." p. 2439.78.30%97%78%"Participants were allocated to either an intervention or usual care group using a computer-generated randomization scheme with the investigators being blind to the allocation." p. 2438.6,4,2 See p. 2438.No"Your child [autopopulate child's name] is due on [autopopulate date] at [autopopulate clinic name] for vaccines." p. 2438.N/AN/A
11
Dissieka et al. 2019Côte d’IvoireUrban, semi-urban, and rural20192014RCTMothers at BCG immunization visit to health clinicNot discussedSMS or voice reminders for immunization and VAS visitsReal-time immunization data22.6pp increase in completion of 5 immunization visits YesNot discussed10% coverage differenceYes1596Not discussedNot discussedNot discussedNot discussedNot discussedNot discussedYes2, plus follow-upNoNot discussedNot discussedNot discussed
Phone calls preferred to SMS, but no difference in effect size between the two
12
Gibson et al. 2017 (M-SIMU study)https://givewell.box.com/s/tnv83164w5zd147rzvd7p1ys27dz9lfoKenyaRural20172013RCT (cluster-randomized)Newborns prior to PENTA 1 visit.98% PENTA3 (82% Full)SMS reminder"Community interviewers did household follow-up visits when children reached 12 months of age to document the child’s immunisation status with the maternal and child health (MCH) booklet. If the MCH booklet was not available, a verbal report of immunisation history was taken." p. e430-e431.Not statistically significant. Measles +3% at p=.28.Yes, no effectYes. Significant effect of SMS on full immunization within 2 weeks of measles due date: RR 1.18, 95% CI 1.01–1.39, p=0.045Powered to detect a 15% difference in fully immunized children. "The following assumptions were made for sample size calculations: a baseline coverage of fully immunised children of 70% at 12 months, village birth cohort with harmonic mean of 16 newborns, a between-cluster coefficient of variation (k) of 0·25, a 25% loss to follow-up, a type I error (α) of 0·05, and power (1-beta) of 0·80."No clear concerns96574822%Outmigration, death, excluding caregivers who verbally reported their child’s immunisation history (i.e. no immunization booklet at 12mo follow-up). "Approximately 1600 (94%) of 1707 children who were alive and had not outmigrated had an MCH booklet present at 12-month follow-up."No. Infant–caregiver pairs were considered lost to follow-up if the infant died before 12 months of age.No difference. "Per-protocol analyses for delivering SMS reminders found similar estimates of vaccination coverage and associations with study groups as intention-to-treat analyses (appendix)."93%97%93%Interviewers were not masked. "Field staff were not informed of a village’s allocation, but this could be inferred from some enrolment and follow-up survey questions." p. e430.3,1NoN/AN/AN/A
13
Gibson et al. 2017KenyaRural20172013RCT (cluster)Caregiver of infant 0-34 days in study villageAssumption of 70% fully immunized for power calcs, but no actual baseline providedSMS reminders, recruited prior to PENTA1 (add'l arms of SMS + two levels of cash incentives)Written immunization recordsNo effect on completion for SMS only (see cell note)YesYes. Full immuization within 2 weeks of measles due date RR 1.18 for SMS only 15% differenceYes20181600~21%Outmigration (mostly), death, withdrawalNo (considered lost to follow up)No difference97%97%No3, 1SMS + incentive (US$0.88 and $2.35, two different arms)"Tell Mama <BABY’S FIRST NAME> that <VACCINE NAME> vaccine is due this week. <MOTIVATIONAL MESSAGE>" Not discussedNot discussed
14
Schlumberger et al. 2015https://givewell.box.com/s/6huhgi0warn4ewi8jpaysnkgw1hyyveoBurkina FasoUrban20152014RCTRecruited after birth in an urban health clinic. "Les nouveau-nés du CSPS de Colma 1 étaient inclus dans l’étude lors du premier contact PEV à leur naissance (Tableau 1), à partir de février 2014."42.3% PENTA3SMS reminderImmunization records at clinic. "Sur les 523 enfants enregistrés pour l’étude dans le RIV, 77 (14,6 %) enfants n’ont pu être retrouvés sur les registres papier, remplis à la main, du CSPS." p. 352. 101 Intervention group infants not found in immunization records were followed-up with by phone. "Cent un (39,6 %) des 255 enfants du groupe avec envoi des SMS de rappel, et qui étaient en retard pour le PEV sur le RIV, ont fait l’objet d’un interrogatoire téléphonique, grâce au numéro de téléphone portable fourni par la mère." Of these, 19 (19%) could not be reached for follow-up. (See Table 4, p. 352.)18% pointsNoYes, for 1st and 2nd visit, but not for 3rd. See Table 3, p. 352.Not discussed.Not discussed.521Unclear. In the intervention group, of 255 infants (253 mothers), 101 were followed-up via phone because they were late for vaccination, 19 could not be reached for follow-up. (7.4% of intervention infants.)Unclear.Not reached by phone (number deactivated, number does not belong to enrollee). 2 dead infants. p. 352. Unable to find child's record in clinic records: "77 (14,6 %) enfants n’ont pu être retrouvés sur les registres papier, remplis à la main, du CSPS."Unclear.Not discussedNot discussedNot discussed.Not specified.The clinic was provided with a computerized/digital vaccination register, which it did not previously have. See the beginning of the abstract: "L’impact dans le Programme élargi de vaccination (PEV) d’un registre informatisé de vaccination (RIV), avec l’envoi de SMS aux parents avant les sessions vaccinales, n’a jamais été estimé en Afrique sub-saharienne. Un Centre de santé et de promotion sociale (CSPS) urbain prodiguant le PEV a été tiré au sort (Colma 1, district de Do) à Bobo- Dioulasso (Burkina Faso), puis a été doté d’un RIV." p. 349.N/AN/AN/A
15
Kazi et al. 2018PakistanUrban20182013RCTCaregivers of infants <2weeks, with mobile phone"well under 90%"One-way SMS reminders for routine immunization visitsHousehold visit, observation of immunization card9.6pp increase in PP at 6 weeks; not stat. sig at 10 or 14 weeksNoNot statistically significant.20% difference in coverage (?)Yes300300Not applicable for ITTSee cell note (for PP only)Not in PP analysisOnly 1 PP time period had sig. findings; no ITT analyses had sig. findings71-75.9%73%No4 reminders during weekNo‘Child name’ is due for 6-week vaccination immediately take your child to the nearest EPI center.Not discussedNot discussed
16
Chandir et al. 2022PakistanUrban20222017RCTChildren visiting a study immunization clinic for BCG, penta-1, or penta-2 vaccineAssumption of 51.5% fully immunized for power calcs, but no actual baseline providedSMS text reminders (up to three per scheduled appointment)Immunization records at clinic.OR1.16 on full immunization coverage (and similar results for PENTA3, measles1 and measles2)YesSome effect seen on Measles2, but weakly significant for Measles1, and insignificant finding for PENTA3MDE of 5ppt change in full immunization coverageAppears so111971119730DeathYesOnly ITT for primary outcome84%84%Partial1,0, 6 after (if missed)mCCTs (5 treatment arms)"Get [Child Name] vaccinated on time and save them from disease. [Child Name]'s next vaccine is due tomorrow [Vaccination Day]. For details, please call 0315-8225705 -ZindagiMehfooz"Not discussedNot discussed
17
Mekennon et al. 2021EthiopiaUrban20212019RCTMother-infant pairs presenting for Bacillus Calmette-Guérin vaccination of an infant Assumption of 51.5% fully immunized for power calcs, but no actual baseline providedSMS text reminder day before scheduled vaccinationsImmunization records at clinicsIncrease in Penta3, measles, and full vaccination coverageYesYes, increase in timeliness15% difference in coverageYes4344268Relocation (7), death (1)NoNot discussed89.70%90%Partial1NoNot legible in supplementary pngNot discussedNot discussed
18
Kagucia et al. 2021KenyaRural20212016RCTChildren aged 6-8 months identified by CHVs96% (control) and 95% (SMS arm) received PENTA3 vaccinationSMS text reminder day before scheduled vaccinations
Home-based vaccination record or the caregiver’s verbal report
Not significantly higherYes10ppt increase in timely MCV1 coverage15ppt increase in MCV1 timely coverageMaternal age unevenly distributed57345554Out-migrated, died, refusedNoBroadly similar67%74%No3,1SMS + incentive (KSH 150)Tell Mama <BABY’S FIRST NAME> that Measles vaccine is due this week. Most Gem babies get vaccinated, be one of them! Not discussedNot discussed
19
Yunusa et al. 2024https://givewell.app.box.com/file/1488281776722?s=gdmgea08nquboxob1jt6tk6r5j6ri4wuNigeriaUrban20242023RCTWomen who recently gave birth (with a newborn aged <= 3 weeks) were screened and recruited at health facilities.Not discussedSMS reminders 3 days before the scheduled visit and a call the day before a scheduled due date. A weclome SMS message was also sent after recruitment and SMS messages with information on immunization were also sent after at the fourth and fifth week after the babies date of birth.Immunization records at clinicsStatistically significant increases in immunizations at 6, 10 and 14 weeks after birthNoYes, for all visits4 pptNo, in particular:
1) Significant differences in the average monthly income between groups with a larger share of control group participants (45.2%) in the lowest income bracket of <30,000/38.4 (Naira/USD) compared to the intervention group (2.9%)
2) Larger proportion of caregivers reached when children were >=16 days old in control group (42.2%) compared to intervention group (18.1)%
55453618 or ~3.3%Missing recordsN/ANot discussedNot discussedPartial3,1Phone callsNot discussedNot discussedNot discussed
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100