May 2022 Monthly Meeting

Quality is key: How to ensure the family planning/ reproductive health content of your digital tool is the best it can be

www.globaldigitalhealthnetwork.org

How to ensure the family planning/reproductive health content of your digital tool is the best it can be: �Overview of WHO SMART Guidelines – Digital Adaptation Kits

24 May 2022

Tigest Tamrat

2

Twitter @HRPresearch

Today, WHO guidelines can take years to be fully adopted by Member States,

Develops global guidelines using evidence base

Access quality person-centered care, delivered in accordance with national directives on guideline use

Deliver and document health services in accordance with national guidelines

Health Workforce

Health Service Users

Adapts guidelines into national guidelines comprising procedures, protocols, datasets and indicators

Ministry of Health (MoH)

World Health Organization (WHO)

Implementation & Technology Partners

• Resource intensive to adapt and scale to broader use

Translate national guidelines into digital solutions

• Difficult to update or digitize with fidelity

• Not integrated quickly or fully into practice

Today’s guidelines are

and accuracy can be compromised during adaptation

• Infrequently digitized with interoperability and indicator standards

This is further complicated with digital leading to questionable results

4

Today’s digitized guidelines are

• Not always accurate
• Not fully trusted
• Not cost-efficient
• Uncoordinated
• Not transparent
• Not scalable
• Not interoperable
• Opaquely & inadequately represented in digital solutions

MOH and their implementation partners need detailed specifications for digitizing guidelines so that they are not misinterpreted

Challenges with current state of digital tools

• Translating narrative guideline recommendations into digital systems may result in misinterpretations by software developers and limit quality assurance of the health content

​

• Governments and partners are faced with different competing digital products with unknown health content, possible misinterpretation of recommendations, and vendor lock-in

​

• Governments need access to trusted minimum health content that is in line with evidence-based recommendations and best practices

​

• Moving from paper to perfect: either ad-hoc validation projects, or unattainable interoperable perfection

5

Launch of WHO SMART Guidelines

WHO SMART guidelines comprise documentation, procedures, and digital health components to steer guideline localization and implementation through digital systems…content is, by design, software-neutral, formulated for adaptation into whichever software platforms a country has elected to use, within an exchanged digital health enterprise architecture

​

​

​

​

​

​

Available at: https://www.thelancet.com/action/showPdf?pii=S2589-7500%2821%2900038-8

Each “Layer” offers Smart Guideline components

L2: Operational

L5: Dynamic

L4: Executable

L3: Machine Readable

L1: Narrative

• Living guidelines approach
• Digital curation of recommendations
• Panels include medical informatics experts

• Human-readable operational components
• Structured workflows
• Decision-support documentation
• Data dictionaries
• Specificity of user(s), beneficiaries

• Software specifications
• Standard terminologies, value sets, data formats, calculations for decision-support and indicator population and data exchange

• Fully executable software tools
• Mechanism for real-time updates

• Advanced analytics for greater local relevance and precision
• AI-based decision support
• Continuous improvement of software code

EXISTING MODEL WITH ENHANCEMENTS

PREPARING TO GO DIGITAL

INTEROPERABLE DIGITAL COMPONENTS

CUSTOMIZABLE SOFTWARE

ADVANCED ANALYTICS FOR PRECISION HEALTH

Digital Adaptation Kits (DAKs)

8

Intelligent digital systems

Paper-based and legacy systems

Narrative Recommendations

Preparing to Go Digital

Interoperable Digital Components

Customizable Software

Advanced Analytics for Precision Health

Operational

Machine-readable

Executable

Dynamic

Existing model with enhancements

Progressive Digital Pathway

GUIDELINE LAYER

L1

L2

L3

L4

L5

Smart Guidelines provide a pathway to advance guideline use, even if a country is not yet fully digital

What are digital adaptation kits

• Software-neutral, operational, and structured documentation based on WHO clinical, health system and data use recommendations to more systematically and transparently inform the design of digital systems

​

• A generic starting point for the content that should be within digital systems from which countries can adapt to local content

​

• A bridging resource for digital health, M&E, HIS and programme teams to ensure digital systems are aligned to WHO recommendations, including defining opportunities for integrating services

9

Twitter @HRPresearch

Ensure fidelity of the health content – independent of �the software systems

Digital Adaptation Kits

Focus

10

Twitter @HRPresearch

Components of a digital adaptation kit

Generic Personas

Roles, responsibilities, Competencies and essential interventions performed by targeted personas

​

​

User Scenarios

Brief narrative description of how the targeted personas may engage with the digital system

Business Processes & Workflows

Generic workflows representing clinical and non-clinical processes

Data elements, used for clinical decision-making, indicators, and other data needs

Decision tables representing counselling and treatment algorithms, scheduling logic

Indicators & Monitoring

Indicators for reporting and monitoring with numerator, denominator of data elements based on existing guidance

​

Health Interventions & Recommendations

Core Data Elements

Decision Support Logic

Functional & Non-functional Requirements

A non-exhaustive list of key functions and non-functional requirements for a digital tracking and decision support system

Relevant health interventions and recommendations from the WHO guideline and guidance

1

2

3

4

5

6

7

8

11

Twitter @HRPresearch

Smart Guidelines within broader planning process

12

Twitter @HRPresearch

Status of DAKs

13

Available at: https://www.who.int/publications/i/item/9789240020306

Available at: https://www.who.int/publications/i/item/9789240029743

FP DAK DEEP DIVE

| Title of the presentation

14

23/05/2022

14

Twitter @HRPresearch

FP DAK

DEEP DIVE

15

Twitter @HRPresearch

Compilation of multiple guidelines and guidance �documents including WHO’s UHC Menu

Health Interventions & Recommendations

+ many other resources

16

Twitter @HRPresearch

Generic personas to be further refined in country

Generic Personas

17

Twitter @HRPresearch

User scenarios —setting context for how personas interact and illustrate how a digital system can be used

User Scenarios

18

Twitter @HRPresearch

Overview of business processes

Business Processes & Workflows

19

Twitter @HRPresearch

Generic workflows for each process

Business Processes & Workflows

20

Twitter @HRPresearch

Core Data Elements refencing the workflow and mapped �to standards for interoperability & indicators

Linked to a full data dictionary that details:

​

• ICD/SNOMED/LOINC/ HL7 resources and other relevant codes
• Response options and underlying calculations for data elements
• Skip logics & validation rules

Core Data Elements

Available as a downloadable spreadsheet annexed within the DAK

21

Twitter @HRPresearch

Data dictionary refencing the workflow and mapped �to standards for interoperability & indicators

Core Data Elements

22

Twitter @HRPresearch

Recommendations embedded in decision support logic

Decision Support Logic

Reason for visit

​

Risk assessment

​

Medical eligibility

​

Where services can be provided

​

Pregnancy checklist

​

When to start

​

Dual protection

​

Exams and tests

​

Follow-up requirements & scheduling

23

Twitter @HRPresearch

Decision support logic aligned to workflow

�

​

�

​

�

​

Decision support logic

24

Twitter @HRPresearch

Aggregate indicators & metrics based on data dictionary

Indicators & Monitoring

25

Twitter @HRPresearch

Functional and non-functional requirements linked to processes

Functional & Non-functional Requirements

26

Twitter @HRPresearch

NEXT STEPS: COUNTRY USE AND IMPLEMENTATION RESEARCH

| Title of the presentation

27

23/05/2022

27

Twitter @HRPresearch

Objectives of the implementation research

• Define a replicable process needed for adapting and incorporating the DAKs within countries’ existing digital health systems

​

• Identify the training and support requirements, including costs, to supplement the use of DAKs in digitizing PHC and optimizing national digital systems?

​

• Assess the use of DAKs impact service delivery, data use, and client health outcomes:
• Integration/linkages across services (e.g. reduced missed opportunities among clients/patients)
• Health workers’ adherence to recommendations (fidelity measures)
• Valued added for data/reporting (monitoring of programme effectiveness, HMIS, etc)
• Possible qualitative outcomes – client and provider/user perspectives?

​

28

The WHO envisions a future where everyone in the world benefits fully and immediately from clinical, public health, and data use guidelines

�Lianne Gonsalves, PhD�World Health Organization

2 global workshops

45

interviews

Development process

The Framework

33

Bonus annexes

Considerations for funders

​

Youth on youth engagement

35

How do we want people to use this?

Want to learn more?

Lianne Gonsalves: gonsalvesl@who.int

​

​

​

So Many Tools; How to Choose?

Findings from R4S’ Review of Content Digital Tools for Family Planning

Agenda

Overview of content analysis of digital tools​ to highlight illustrative findings​ and their application

Introduction of tool which will be piloted to enable widespread application of findings

Background

• Accelerated proliferation of digital tools for FP in the past decade with varied objectives and formats

​

• Resources available to share what interventions exist—such as the Digital Health Atlas and the Digital Health Compendium—and which interventions have evidence of impact—such as the three digitally-focused High Impact Practice Briefs

​

• Gap: No resource that points to which tools have high-quality content

Process for content analysis

Landscape

Landscape existing provider- and client-facing digital tools and partner with tool owners to gain insights into tool and receive content

Develop Rubric

Select evaluation criteria in consultation with global guidance and develop rubric for content assessment

Disseminate

Disseminate findings and recommendations of top tools for introduction or scale up

Analyze

Analyze tool content for comprehensiveness and accuracy

Provide feedback

Suggest components to strengthen tools with content gaps

Rubric

• Complete list of modern methods
• Instructions for use
• Effectiveness
• Dual method use
• Duration of protection
• Return to fertility
• Mechanism of action
• Discreetness
• Side benefits
• Side effects
• Side-effect management

Core FP Content Areas

Recommended tools

• 9ja Girls Life, Love and Health Curriculum
• 9ja Girls Big Sista Chatbot
• askNivi
• Choice Contraception Counsellor
• Counseling for Choice Chatbot
• CyberRwanda
• Love Matters
• M4RH
• Tune Me

Content analysis

Here's what we found

The following content areas are ordered from most to least accurate and comprehensive:

Complete List of Modern Methods: Findings

• Most tools provided users with a complete list of methods available in their context. ​
• Only two tools included more than one omission or inaccuracy. 
• Some tools were inconsistent in the level of detail of the information provided for each method.

Content from Tune Me

Complete List of Modern Methods: Application of Findings

Content considerations

• All modern methods that are available to the targeted population in the context in which the tool is deployed should be named. To verify what methods are available, check local guidelines emitted by the MoH.
• Inconsistency in the level of detail of the information provided for each method has the potential to bias users toward the methods with more comprehensive information.

​

Tip

• It is important to differentiate between Combined Oral Contraceptives and Progestin-Only Pills, especially for adolescent-facing tools.

Return to Fertility: Findings

• Five out of eleven tools provided comprehensive/accurate information about return to fertility.​
• Two tools included three or more omissions or inaccuracies when describing the return to fertility following the use of hormonal methods.​

IMPLANTS

Implants are small rods placed under skin of woman’s arm. Highly effective for 3–5 years. For married and singles. May cause light irregular bleeding. When removed, can become pregnant with no delay. No infertility or birth defects. To return to main menu reply 00.

Text message content from m4RH

Return to Fertility: Application of Findings

Content considerations

• Many women have misperceptions and fears about the return to fertility after the use of contraceptive methods, therefore it is important to include accurate information about any delayed return to fertility or lack thereof.
• Only users of injectables may experience a delay in return to fertility (for 4-9 months on average and up to 1 year).

​

Tip

• Instead of using terms such as “immediately reversible,” which may not be clear to all users, we recommend stating “after removal/discontinuation, ability to get pregnant returns without delay.”

Side Effects: Findings

• Only one of eleven tools included correct and complete information about side effects. ​
• Information about the side effects of oral contraceptive pills was inaccurate or incomplete in ten out of eleven tools. 

Content from M4RH (right) and Knowledge SUCCESS / FHI 360​ (left)

IMPLANT • MAIN SIDE EFFECTS

Abdominal pain, acne, breast tenderness, dizziness, headaches, nausea, and changes in

Side Effects: Application of Findings

Key considerations

• We recommended that tools share information about side effects. Users may be overloaded by this information in a counseling session with a provider and may seek reassurance in a digital tool to support contraceptive continuation.

​

Tips

• It is helpful to clarify that common side effects are not harmful.
• Be specific.
• Don’t include rare complications, such as pelvic perforation with the IUD, as a side effect. This may scare possible users unnecessarily.

Many individuals discontinue or decide not to adopt a modern method because of real or perceived side effects. Digital tools have an opportunity to accompany these users after they adopt a method.

Content from FPHandbook.org

Non-contraceptive benefits: Findings

• The real or perceived benefits of using certain contraceptive methods were infrequently included, though these benefits are often appealing both to youth and adult users of FP. ​
• Information about non-contraceptive benefits was inaccurate or incomplete in seven out of eleven tools. 

Content from Choice Contraception Counsellor​ (above)

What are the advantages of the pill?

• It can make your periods regular, lighter, and less painful
• It gives you the choice not to have a monthly bleed or control when you have a bleed
• Your fertility will return to normal immediately after you stop using the pill
• It is not used during sex so will not affect spontaneity
• It helps protect against some forms of cancer (uterus (womb), colorectal, and ovarian)
• It reduces the risk of getting fibroids, ovarian cysts and con-cancerous breast disease
• It may reduce acne and improve your skin

Non-contraceptive benefits: Application

• Many women are drawn to non-contraceptive benefits such as improved menstrual regularity, reduced cramping and pain, or protection from certain types of cancers. These should be included in tool content whenever relevant.

Content from Knowledge SUCCESS / FHI 360​ (above)

Quality Assessment

Checklist

​

• Presentation of findings in user-friendly format to enable tool owners and developers to learn from our review of FP content and apply to their own content of client-facing tools

​

​

Recommendations

• Update tool content regularly to provide users with high-quality, accurate, and comprehensive information.
• Utilize global and local policies and guidelines as resources for content development.
• Incorporate external medical review when possible.
• Don’t reinvent the wheel! There are great tools available for adaptation.

Thank you!

R4Sinfo@fhi360.org

@R4Sproject

https://www.fhi360.org/projects/research-scalable-solutions-r4s

https://research4scalablesolutions.com/

This presentation is made possible by the support of the American People through the United States Agency for International Development (USAID). Research for Scalable Solutions (R4S) is a global project funded by USAID and led by FHI 360 in partnership with Evidence for Sustainable Human Development Systems in Africa (EVIHDAF), Makerere University School of Public Health in Uganda (MakSPH), Population Services International (PSI), and Save the Children (STC). The contents of this website are the sole responsibility of FHI 360 and do not necessarily reflect the views of USAID or the United States Government.

Join the GDHN!

https://bit.ly/GDHNSignUp