Study Key information
|Study, Author, Year||Primary outcome||Effect on primary outcome||Study Name||Sample size (Control + Reminders)||Experiment type||Country||Rural/ Urban||Baseline/ Control Pentavalent 1 immunization rate||Baseline/ Control Pentavalent 2 immunization rate||Baseline/ Control Pentavalent 3 immunization rate||Baseline/ Control Measles 1 immunization rate||Baseline/Control full immunization rate||Female literacy rate/ education level||Phone access||Phone ownership||Spam texts common in country?||Intervention||Enrollment strategy||Caregivers eligibility criteria||Time of Reminder(s) in Days Before Appointment||Control Group with Health Information||Message Text||Update based on late vaccines?||Time Between Last Vaccination and Final Data Collection||p-value (statistical significance)||Costs Per Message Sent (USD)||Other Costs in USD (name)||Strength (1-3) Katherine||Statistical power||Knowledge ad attitudes towards reminders||Notes|
|Schlumberger et al 2015||Rate of Pentavalent 3||18 (p<0,001)|
Positive impact on the Expanded Program on Immunization when sending call-back SMS through a Computerized Immunization Register, Bobo Dioulasso (Burkina Faso)
|523||RCT||Burkina-Faso||Urban||55.7 (post-control)||53.6 (post-control)||42.3 (post-control)||not reported||not reported||33||not reported||not reported||Unknown||SMS reminder|
Enrolled at birth at the clinic.
Child is a new bord (enrollment at delivery) and mother has to have access to, or own a phone.
Between 2 to 8 months (inferred from sign up dates and data collection date)
Says it is powered but doesn't specify the MDE
96% of mothers were able to understand the messages
In French. Ask Juliette if have questions. It found ~17.5% point increase for 2nd, 3rd, and 4th vaccination. Followed up with those who received SMS but didn't get vaccinated. 66% of them had gone to different clinic, 19% couldn't be contacted, 18% were on a long voyage, 13% forgotten, 3% said message wasn't sent, 2% children passed away. ~20% of numbers were fake or disactivated over time. 96% of women understood messages even though 42% couldn't read it on screen themselves (unclear whether illiteracy or not owning phone themselves)
|Haji et al 2016||Rate of Pentavalent 3||13 (95 % CI: 5.6–21.26)|
Reducing routine vaccination dropout rates: evaluating two interventions in three Kenyan districts, 2014
RCT (cluster-randomized, 9 groups, 3 in SMS arm, 3 in sticker arm, 3 in control)
|Kenya||Rural and Urban|
90 (baseline district level data)
|91 (post-control)||83 (post-control)||not reported||not reported||75||86|
14 (used family phones)
SMS reminder, Sticker reminders
Enrolled at the first Penta vaccine visit at the health facility.
Children 0 to 1 year whose mothers have a telephone number.
Yes, but no follow-up reminder if never received second dose.
0.27 (full cost per child reminded up to 6 months)
Underpowered? Powered to detect 15 % decrease in the drop-out vaccination rate
Compared text messages to stickers to control. This has a CEA; prior vaccination rates (and dropout rates) given for each district in trial; "Dropout was defined as any child who failed to return for the third dose of pentavalent vaccine [which takes place at 14 weeks] two weeks or more after the scheduled date."; majority (77%) unemployed; "If a care giver took the child to another facility for second or third pentavalent dose, the system considered the child unvaccinated,leading to misclassification, however, a sensitivity analysis that assumed that these children were actually vaccinated had no effect on the general observed difference between the inteventions." Only included regions with >10% dropout rates. Women without phones weren't included (0.9%). Followed up with people who didn't show up >2 weeks after appointment to find out why didn't come. Called and asked why didn't go, said child taken to another facility 39 (35 %); travelled out of town 33 (30 %); forgot 17 (15 %); child was sick 16 (15 %); or child died 2 (2 %). Each district, and randomly allocated to send text messages or provide stickers reminding parents to bring their children for second and third dose of pentavalent vaccine, or to the control group
|Eze et al 2015||Rate of Pentavalent 3||8.7 (OR 1.47, 1.1 – 2.0)|
Enhancing Routine Immunization Performance using Innovative Technology in an Urban Area of Nigeria.
|905||RCT||Nigeria||Urban||~85 (post-control)||~80 (post-control)||60.3 (post-control)||not reported||not reported||67||95||5||Yes|
SMS reminder, recall as well
Enrolled at the health facility for either BCG or DTP3.
Any caregiver bringing their child for the first or second visit of the routine immunization schedule. Illiterate were encouraged to seek help if they could not understand the message. Participants who did not own a phone were swaped with participants in the control group matching on characteristics like age and education.
1 (with additional messages 1 day before next appointment for anyone who missed initial appointment)
Example message: "Dear client, your child is due for his/her next dose of vaccines tomorrow Tuesday 20/7/10. Kindly bring your child to Hospital X for vaccination at 8am. Please come with immunization card. Thank you."
1 month pass reccommended DPT3 dosage (14 weeks) or 1.5 (?) months past third recommended vaccination (p6)
0.15 (full cost per additional child immunized, estimate, originally in Naira 27.47)
93% of respondent said they desire sms reminders for immunization. Among those who preferred not to be sent SMS reminders, self-confidence in not forgetting appointments(61%) and the fear of giving out phone numbers (28.8%) were the greatest barriers to mHealth uptake in this study
Includes CEA. "Some respondants" were initially randomized in control group but later swapped with members of the control group who did have phones. No figure is given for how many (but very likely less than 50, since 95% of people had phones). This could obviously result in selection bias.; Data collection and analysis was blinded to treatment/control. Acknowledged limitations: no known delivery success rate, 9.6% dropout rate, inconsistent DPT vaccine supply, no knowledge of literacy rates of participants, no adjustment for those lost to follow-up getting vaccinated anyway.
|Bangure et al 2015||Rate of Pentavalent 3||16.3 (95% CI: 12.5-28.0)|
Effectiveness of short message services reminder on childhood immunization programme in Kadoma, Zimbabwe - a randomized controlled trial, 2013
|304||RCT||Zimbabwe||Urban||82 (post-control)||80 (post-control)||75% (post-control)||not reported||not reported||95||not reported||not reported||Yes||SMS reminder|
Enrolled at the hospital at delivery or first visit.
Woman or caregiver was recruited into the study soon after delivery or during the 3rd and 7th day visits after delivery of the baby. Eligible respondent must have a cell phone and a resident of Kadoma city.
The translated messages were as follows; A week before appointment date: − “Immunization protects your child against killer diseases such as polio, whooping cough, diphtheria, measles, pneumonia and tuberculosis. You are reminded that the vaccination appointment will be due in 7 days time from today.” Three days before appointment: − “You are reminded that the vaccination appointment will be due in 3 days from today.” A day before appointment: − “Your vaccination appointment is due tomorrow, visit the nearest clinic”.
Unclear. Minimum 17 days past day due because quartiles referenced )
0.99 (full cost per child up to 18 months, with 3 messages per visit, estimate), 0.33 ("Capturing of data per message", estimate)
A majority of participants reported preferring to be reminded a day before the due date
Coverage estimated by following up with people over the phone, potential source of over-estimtation, 100% preferred single language which could have kept costs down; majority unemployed in control and intervention group. While the sample size is fairly small, Katherine did a power calculation and it would've need only half of the used sample size to detect the effect it did. 64% prefer SMS 1 day before appointment, 27% prefer 3 days before, practically nobody wants further out reminders. Nobody lost to followup which is pretty impressive.
Rate of full immunization at 12 months (BCG+Penta123+Measles)
|4 (RR: 1.04, CI:0.97-1.12)|
The readiness, need for, and effect of mhealth interventions to improve immunization timeliness and coverage in rural western Kenya
RCT (cluster-randomized, 4 clusters)
|Kenya||Rural||100 (post-control)||99 (post-control)||98 (post-control)||84 (post-control)||82 (post-control)||75||51||49||No|
SMS reminder, SMS reminder and cash transfer
Enrolled direclty in the village.
Infant aged 0-34 days having received no vaccines or only BCG and Polio birth dose, living in the village where the enrollment happened and whose caregiver is not planning to migrate in the next 6 months, and who has access to a phone (not necessarily owner).
No (but intial sign up included sms with "The greatest wealth is health")
"Tell Mama<Baby FName> that Penta-1 vaccine is due this week. <Motivational Message>" (chart of messages for all arms with motivational messages on pg 95-96)
1 month to 3 months, inferred from timeline fig 3.1 and 9 month latest vaccination
underpowered (could detect 15%increase in coverage)
85% of participants had at least 2 reminders and 97 had at least 1 reminder
Underpowered to detect an effect size of 4% at stat sig. Compared SMS only, SMS plus small cash transfer, SMS plus larger cash transfer, and control. Lots of detail about study logistics and specifics available; located in setting with detailed demographic information on births, deaths and pregancies. Some vaccine stock-outs in area (pg 69-70).; in 3 languages based on parental preference (English, Kiswahili or Dholuo) (74); program adjusted timing of reminders based on when child recieved previous vaccination (74); Was looking for full vaccination vs not full vaccination. 152 cluster (village) randomized controlled trial that enrolled caregivers of infants less than 35 days old who had not initiated the pentavalent vaccine series.
Domek 2016 (pilot, not finished yet)
Rate of full immunization at 22 weeks including doses other than penta
SMS text message reminders to improve infant vaccination coverage in Guatemala: A pilot randomized controlled trial
|321||RCT||Guatemala||Urban||not reported||90.1 (post-control)||80.7 (post-control)||not reported||not reported||76||98.7||not reported||Unknown||SMS reminder|
Enrolled at the hospital at the time of penta 1.
Parents of infants between the ages of 8 and 14 weeks presenting for the first dose of the 3-dose infant primary immunization series were eligible if they owned a mobile phone with SMS text messaging capability. At least one parent had to be literate and able to use SMS technology.
"Your child [autopopulate child's name] is due on [autopopulate date] at [autopopulate clinic name] for vaccines."
underpowered (authors mention it but don't specify MDE)
"Not adequately powered to assess efficacy". If baseline vaccination rate is 80% it would have to have a sample size of 150 to detect 15% point change with 80% probability. To detect a 8% point increase, the lowest increase of the good quality studies, it would've had to have a 656 sample size. The baseline rate was at a pretty high starting point, so is likely to have a smaller effect because the people still not getting vaccinated might have something else stopping them. To participate at least one parent had to be "literate and able to use SMS technology", and be 18 or older.; No data on messages being delivered. SMS system problems inlcuding "power outages and delays in recharging the server that either resulted in missed messages or repeat messages being delivered upon reactivation of the delivery platform."; Usual care included a written reminder of next appointment date at each clinic. Parents of infants between the ages of 8 and 14 weeks presenting for the first dose of the 3-dose infant primary immunization series were eligible if they owned a mobile phone with SMS text messaging capability. Last relevant vaccine is at 6 months old. 17.4% people lost to follow-up.
Domek 2019 (follow-up study)
Rate of Timely Pentavalent 3
Text message reminders to improve infant vaccination coverage in rural and urban Guatemala
|720||RCT||Guatemala||Rural and Urban||NA||92 (post-control)||89.2 (post-control)||not reported||not reported||76||95|
93.9 mothers' ownership (among families who have access to a phone)
Families presenting for the first visit of the primary immunization series (penta 1)
Infants ages 6 weeks-6 months whose family own a phone
Waiting to read the full paper.
Waiting to read the full paper
50% of people who received the reminders said they would found reminders very helpful in the future compared to 28.7% in the control group.20% said they'd be very willing to pay for SMS compared to 8.5% in control.
They found an effect on timeliness
|Uddin et al 2016|
Rate of full immunization at 11 months (BCG+Penta123+Measles)
|29.5 (OR:3.6, 1.5-8.9)||Use of mobile phones for improving vaccination coverage among children living in rural hard-to-reach areas and urban streets of Bangladesh||136||Quasi-experimental, pre-post (clustered, 160 clusters)||Bangladesh||Rural|
99 (baseline rates control+ treat mean)
90.5 (baseline rates control+ treat mean)
80 (baseline rates control+ treat mean)
71 (baseline rates control+ treat mean)
63.3 (baseline rates control+ treat mean)
|70||35||47||Unknown||Mobile registration and SMS reminder and remind health workers and provide info to health worker supervisors||Self-registration to an application that sends automatic sms reminders to mothers. Health workers recruited from their lists women in the third trimester of pregnancy and registered mothers. They were assigned a unique code and taught how to send SMS text messages from a regular mobile phone to the program after childbirth.||Women were eligible for participation if they were over 18 years old, gave birth within one year prior to data collection, and were able to give written informed consent in Bengali.The data included children age 0–11 months. Had to have access to a cell phone either owned or shared with family members.||1, 0 (opening of clinic), 0 (2 hours before closing of clinic)||No||NA||Yes||Between 1 to 2 months, data collection at 11 months.||0.0287||N/A||N/A||1|
Powered to detect a 10% increase in full vaccination coverage from a baseline of 56.
|N/A||Differences in differences; very, very weak; used original software for CHW; sampled different street children at endline and baseline and didn't restrict to just those registered with original software; "Upon registration, mothers were assigned a unique code and taught how to send SMS text messages from a regular mobile phone to mTika after childbirth". Included children 0-11 months old at beginning of study. Sent reminders to mothers and health workers, as well as monitoring for supervisors of health workers. Symbols taught to mothers at registration were used in SMS messages for mothers who were illiterate. Did based on maternal recall|
Rate of full immunization at 11 months (BCG+Penta123+Measles)
|27.1 (OR:2.3, 1.1-5.5)||210||Urban|
96.5 (baseline rates control+ treat mean)
85.5 (baseline rates control+ treat mean)
72.5 (baseline rates control+ treat mean)
51 (baseline rates control+ treat mean)
42.6 (baseline rates control+ treat mean)
Powered to detect a 10% increase in full vaccination coverage from a baseline of 56.
|Kazi et al 2018||Rate of Pentavalent 3||5.3 (p=.31)|
Effect of Mobile Phone Text Message Reminders on Routine Immunization Uptake in Pakistan: Randomized Controlled Trial
|300||RCT||Pakistan||Urban||71.3 (post-control)||52.7 (post-control)||26 (post-control)||not reported||not reported||44||94.1||38||Yes||SMS reminder|
Enrolled at the household.
Infant less than 14 days of age, having access to a valid mobile phone and being comfortable using an sms. Family plans to stay in the catchment area for at least 6 months.
4 times in week child was due
One-time standard verbal counseling at the time of initial visit (enrollment) by the study staff regarding the timing for EPI vaccines at 6, 10, and 14 weeks
"Your baby is due for [6, 10, 14] week vaccination [line break?] immediately take your child to the nearest EPI center" (translation from Urdu and Sindhi)
1 month, measured at 18 weeks of age
underpowered (could detect 20%increase in coverage)
Around 75% of respondents at endline report having received the message.
Study was powered to detect an effect size of 20%, higher than any previously done study. For how we came to the effect size of 5.3% look at the sheet "Caclulations for Kazi". Did this because there wasn't a convenient summary statistic for the entire study.
|Seth et al 2018|
Rate of full immunization (calculated for each child as the proportion of the total number of immunizations received divided by the total number of immunizations required at the time of measurement)
-1.6 (RR: 1.01, 95 CI: 093-1.09)
Mobile Phone Incentives for Childhood Immunizations in Rural India
33.3 (baseline for all study participants)
|not reported||not reported||16.4||95||not reported||Yes|
SMS reminders compared to incentives. For reminders group, they sent 20 text messages per participant and signed up multiple phones. They were also moving all groups over to a new tracking system using biometrics at the time of the study, so that likely added alot of noise. Lastly all groups were mixed together for the study, so their could have been spill over effects.
Signed up people by going door to door.
Infant under 24 months old and have mobile phone in household
Sent over 20 reminders per child
Signed up children between ages of 0 and 24 months. On average children were in the study for 292 days.
|N/A||$0.50 for incentive||2|
Mderately underpowered: Study was powered to detect an effect size of about 14.1%, which is within the range of previous studies, but at the higher end.
Was done in rural Haryana. "Immunization coverage was the primary outcome of the study and was calculated for each child as the proportion of the total number of immunizations received divided by the total number of immunizations required at the time of measurement" They sent reminders to multiple caregivers