Baseline Findings Shareback | September 28, 2021

Design and Impact Evaluation of a Digital Reproductive Health Program in Rwanda

​

Overview of CyberRwanda

​

CyberRwanda

Can a digital health intervention improve young people’s access �to HIV testing and contraception?

​

• CyberRwanda is a digital health intervention focused on economic empowerment, sexual and reproductive health, and safety.
• The intervention is delivered through tablets to Rwandan youth aged 12-19 years old.
• CyberRwanda has 3 main parts
• Stories
• Learn
• Shop

​

​

Ippoliti N, Sekamana M, Baringer L, Hope R. Using Human-Centered Design to Develop, Launch, and Evaluate a National Digital Health Platform to Improve Reproductive Health for Rwandan Youth. Glob Health Sci Pract. 2021 Nov 29;9(Suppl 2):S244-S260.

CyberRwanda

Phase I: DISCOVERY

Aug 2016 - Aug 2017

Phase II: TEST & ITERATE

Sept 2017 - Sept 2018

Phase III: PILOT, BASELINE, IMPLEMENTATION

MIDLINE & ENDLINE Sept 2018 - Dec 2023

LAUNCH AT SCALE

Jan 2023 - Sept 2024

Design Research & Ideation

Prototyping I

Prototyping II

Production

Planning &

Permissions

Pilot

Pilot Evaluation

Implementation,

Midline & Endline

Launch at Scale

CR Implementation and Evaluation Timeline

​

​

​

Data collection: HCD ( n= 1074)

Objectives:

Understanding desirability, acceptability, feasibility of early prototypes; mapping existing interventions to avoid duplication; using data to optimize usability of high-fidelity prototypes and final digital products and marketing materials�

Methods:

1. Predominantly qualitative data from a wide range of stakeholders (youth, parents, teachers, nurses, pharmacists etc

2. Small sample quantitative data e.g. knowledge quiz pre- and post-use of prototype

Data collection: Pilot (n=158)

Objectives: Assess feasibility, acceptability, and engagement and inform the impact evaluation design.

​

Methods:

1. Targeted quantitative and qualitative data from youth, parents, teachers, nurses, pharmacists etc.

2. Google Analytics data on engagement with platform.

​

Data collection: Impact Evaluation (n=6082)

Objectives of impact evaluation: Measure the impact and cost-effectiveness of the intervention in schools and youth centers.

​

Methods:

1. Three-arm non-inferiority RCT in 60 schools and cross-sectional study in 8 youth centers.

2. Cost effectiveness analysis

3. Qualitative studies at midline and endline

4. M&E: Google Analytics data on platform usage and qualitative data from users to inform ongoing iterations of program design.

​

Baseline

CyberRwanda

Objectives

Improve Knowledge

​

Our youth-designed intervention helps adolescents learn health and economic empowerment information in a private, safe space. Topics range from puberty and voluntary FP/RH care to relationships and goal setting.

• Only 12% of sexually active unmarried women ages �15–19 report contraception
• 14.8% of youth in our baseline agreed that HIV could be caused by mosquito bites

​

​

1. National Institute of Statistics of Rwanda (NISR) [Rwanda], Ministry of Health (MOH) [Rwanda], and ICF International. 2014–15 RDHS Key Findings. Rockville: NISR, MOH, and ICF International; 2015.�2. Baseline and midline impact study findings (unpublished).

CyberRwanda

Objectives

Improve Access

​

​

CyberRwanda improves adolescents’ access to adolescent health information, health products including FP/RH services through online resources and trained public and private sector providers.

• 1 in 5 girls have begun childbearing by age 19
• Girls and young women remain more than twice as likely to acquire HIV than their male counterparts

​

​

1. National Institute of Statistics of Rwanda (NISR) [Rwanda], Ministry of Health (MOH) [Rwanda], and ICF International. 2014–15 RDHS Key Findings. Rockville: NISR, MOH, and ICF International; 2015.�2. Baseline and midline impact study findings (unpublished).

CyberRwanda

Objectives

Improve Quality

​

​

​

CyberRwanda Provider training and tools improve service quality �by increasing provider knowledge on FP/RH and providing adolescent-friendly care to increase service quality for youth.

• Of youth who sought FP services in the past year, only 28.2% said the providers were “friendly and helpful”

​

​

​

​

1. National Institute of Statistics of Rwanda (NISR) [Rwanda], Ministry of Health (MOH) [Rwanda], and ICF International. 2014–15 RDHS Key Findings. Rockville: NISR, MOH, and ICF International; 2015.�2. Baseline and midline impact study findings (unpublished).

CyberRwanda

Impact evaluation

(Randomized Controlled Trial)

Intervention schools

Control schools

CyberRwanda Facilitated schools

CyberRwanda Self-service schools

CyberRwanda Impact

Evaluation Study

CyberRwanda

Hybrid “Type 2” Effectiveness

-Implementation Study

​

Study districts

​

• 3-arm, cluster randomized non-inferiority trial
• Evaluate the effectiveness and cost-effectiveness of two implementation models (self-service and facilitated).

​

• 8 districts: Gatsibo, Nyagatare, Kayonza, Huye, Gasabo, Bugesera, Nyarugenge, Rwamagana 
• 60 schools: urban/rural, public/day
• 6,082 students in levels S1 and S2 ages 12-19
• Three survey rounds: baseline, midline (12 months), �endline (24 months)
• Trial pre-registered at ClinicalTrials.gov; �Analysis plan pre-registered at Open Science �Framework (osf)

​

ClinicalTrials.gov Identifier: NCT04198272; Nolan, C., Packel, L., Hope, R. et al. Design and impact evaluation of a digital reproductive health program in Rwanda using a cluster randomized design: study protocol. BMC Public Health 20, 1701 (2020).

Randomization

​

• 60 schools randomized in a 1:1:1 ratio, �stratified by district.
• Two stage process was conducted to ensure global balance and to deal with “misfits (districts with schools that are not multiples of 3).
• Community participatory randomization. Events in each district to assign schools to study arms
• School heads selected a colored ball from an opaque bag corresponding to CyberRwanda facilitated, CyberRwanda self-service, or control
• Bags were ‘pre-populated’ with the correct allocation ratio for each district for global balance.
• Allocation concealment, no masking

​

Kopper, S, Sautmann, A. Randomization. Abdul Latif Jameel Poverty Action Lab.

​

Uptake of modern contraceptive method

Initiation of

childbearing

HIV

Testing

Cost-

effectiveness

Outcomes measures

​

​

Baseline Progress & Interim Findings

​

Are youth who are exposed to CR in the intervention group significantly more likely to be currently using a modern contraceptive method compared to youth who are �not exposed?

​

Are female youth who are exposed to CR in the intervention group significantly less likely to have initiated childbearing by the end of the study compared to female youth who are not exposed?

​

​

Are youth who are exposed to CR in the intervention group significantly more likely to be tested for HIV compared to youth who are not exposed?

Is a “self-service”, �less- resource intensive �model as effective as a peer-facilitated model of CyberRwanda in reaching �the desired outcomes?

CyberRwanda

Youth-Led Implementation

​

​

Trained 200 youth across 40 schools to support the rollout of the program

​

​

CyberRwanda

CyberRwanda Implementation Models

​

​

14

CyberRwanda

Stakeholder Engagement

​

Implementation

1. District orientation meetings
1. District mayors, health and education officials, �and head teachers from all 60 schools
2. Quarterly Reflection meetings �with schools
1. Head teachers, teachers, members of the parent-teacher committee, student representatives.
3. Strong Government support from:
2. MoH, RBC, MINEDUC, REB

Baseline and Midline Surveys

• February-May 2021 & 2022, Sat & Sundays
• Enrolled 6,082 students at baseline & over 94% retention after 12 months..
• COVID-19 safety measures in place
• Best practices to reduce participant anxiety, maximize privacy, and improve data quality for sensitive questions:
• Warm-up questions about sexual behavior prior to questions about individual experiences
• Tablets turned to participants for direct, private entry of responses on the tablet
• Sexual activity assessed by multiple direct and indirect questions throughout

Baseline Characteristics, overall and by study group

​

Common Misconceptions

about Contraception

​

• More than half of participants (54.2%) agreed with a statement that some contraceptives could make women unable to have children even after discontinuation
• 64% agreed that condoms can slip off and disappear inside a woman’s body
• 21.8% agreed with a statement that emergency contraception can prevent pregnancy up to one month after unprotected sex and 40.7% of participants were unsure about this statement

​

18

CyberRwanda

Condom Norms Suggest Opportunities for Education

• 33.8% agreed that condoms were suitable for casual relationships
• 41.9% agreed that condoms were suitable for steady, loving relationships
• 34.6% believed that a girl suggesting condoms to her partner means she does not trust him
• Females reported feeling more embarrassed to buy condoms than males (42.4% vs. 31.7%)

CyberRwanda

Partners

​

​

CyberRwanda

Thank You!

CyberRwanda