1 of 61

Bhupendra Sheoran

MD MBA

Oscar San Roman Orozco, MD, MPH

Hi, my name is sheoran. I know some of you but there are many of you who are new to me. So let me give you a brief introduction. Started as medical doctor, MBA, global health, HIV/AIDS, SHE and then got into technology in 2009. worked as ED of YTH and now I work at a health tech startup.

What is your expectation from today?

We have come a long way from the time that I started using technology. In mid to late 2000’s, using SMS was considered really high tech. We created the first chat bot, banner ads, interactive websites and use social media to see if we can do behavior change.

For several decades, the work of digital health was like the wild west. We did not have frameworks or standardizations or high quality processes or protocols. Everyone was working in silos and there were a gazillion pilots all over the world. But we soon realized that there was a need for defining the approaches and providing high quality frameworks to guide digital health work, otherwise, we would keep producing digital solutions that were of varying degrees. In order to increase efficiency and quality, lead by WHO, we implemented a rigorous process to develop this framework in collaboration with known experts and youth leaders in the field. But a framework is not very useful, if we are not applying it to our strategies and programs. So last year, I adapted the framework into a training so that folks who are working in youth health can apply this framework to develop high quality digital health programs. Also, there is a lot of practical experience I our organizations and its important to leverage that to build the digital programs. They should not be in isolation but rather build on existing successful programs and approaches.

Before we start, I want to acknowledge that this is the first time, I am doing this training in-person. You are my pilot. I want to request your patience and understanding. Also, we have a diverse audience with varying exposures to using digital technology for health projects for youth. For some of you, you have probably done a lot of work in the digital youth health space, Hence I want to ask for your patience and ask that you share your experiences and advise/perspective so that we can all learn from each other. For those folks new to digital youth health, I want to ask that you ask questions and provide you field perspective of working with youth.

Finally, in the beginning I will be talking a lot and I apologize in advance. This is usually not how I plan workshops. However, there is a lot of background and theoretical aspects that I have to share with you before we begin the actual application of the framework.

2 of 61

Icebreaker

3 of 61

Agenda

Welcome and Ice-breaker

Close-out

Stage 3: Implementation

Stage 2: Development

Stage I: Planning

Framework Introduction

01

02

03

04

05

06

4 of 61

Caribbean Statistics

Estimate population: 43,860,773 (2022)

Internet Users: 29,418,573

Percent penetration: 67.1 %

On Facebook: 18,632,515

https://www.internetworldstats.com/stats2.htm

5 of 61

Number of mobile internet users

6 of 61

https://apps.who.int/iris/bitstream/handle/10665/336223/9789240011717-eng.pdf

This document offers guidance on planning, developing and implementing digital interventions to promote better health among adolescents and youth. It includes the key steps in each stage and the unique aspects of designing such interventions with and for young people.

It draws on the experience of many organizations and individuals working in the field—including young people themselves—and builds on existing knowledge and tools.

The guidance presented in this document is intended for digital health intervention designers, developers, implementers, researchers and funders. Newcomers to digital health can use it as a

start-to-finish primer on how to collaboratively and responsibly develop youth-centred digital health interventions. Those already engaged in this work can jump directly to the chapters and sections with the ideas and resources they need.

7 of 61

The Guidance

Visually, this is a snapshot of one section of the document, specifically the ‘Carry out a phased launch’ section within the Implementing chapter.
Each section of the guidance is short – between 3-5 pages in length, which was done purposefully in order to make this a user-friendly resource for readers.
Each section follows a similar content structure, starting with an overview of that step and brief definitions, moving to current practices, showcasing key tips and warnings – or points readers should be aware of during this step and a list of additional resources which contain further explanation or insight on how to carry out that step.
Within each section and sprinkled throughout the guidance there are case studies which demonstrate how key ideas within each section have been put into practice.

8 of 61

The Framework

The Introduction briefly covers the history and landscape of digital health interventions for young people and how this document was developed. Chapters 1 through 3 lay out an overarching

framework, fundamental principles and cross-cutting actions for successful planning, development and implementation, respectively. Chapters 4 through 6 each focus on one key stage in the process, including examples of current practices, tips and warnings, and case studies from leaders in the field.

Importantly, each step within a chapter is accompanied by a list of additional resources. The above figure provides and overarching framework, fundamental principles and cross-cutting actions for successful planning, development and implementation We are going to go through each stage of the framework but first we will review the principles and cross cutting actions that influence and guide the framework.

9 of 61

The Framework

10 of 61

Iteration

Iterative process
Incrementally and progressively improve each “draft version.”
This iterative process may ultimately lead to pivoting away from the original notions.

Agile process.
Having an iterative process means being open to change and factoring in time to modify and adapt the intervention.
At the start of the process, it is easy to have a preconceived idea of what the intervention will look like. But current practices and field experiences indicate the need to include many points throughout thHaving an iterative process means being open to change and factoring in time to modify and adapt the intervention.
At the start of the process, it is easy to have a preconceived idea of what the intervention will look like. But current practices and field experiences indicate the need to include many points throughout the development and implementation process for collecting data and feedback, prototyping, testing, development and implementation process for collecting data and feedback, prototyping, testing, learning and updating the intervention to incrementally and progressively improve each “draft version.”
This iterative process may ultimately lead to pivoting away from the original notions.

11 of 61

Equity

Equity means ensuring fair and impartial access to the digital health intervention.
This has proven to be a challenge on both the local and global levels because access to mobile phones can vary between groups.

For example, women in low- and middle-income countries are 10% less likely to own a mobile phone and 26% less likely to use mobile internet than men. Rural populations in those countries are 40% less likely to use mobile internet than urban populations.
The digital divide also mirrors prevailing economic gaps, amplifying the advantages of wealthier people and failing to deliver opportunities to the poorest and most disadvantaged.

12 of 61

Sustainability

How the intervention will be able to stand on its own, beyond the initial development and implementation and beyond initial funding.
How to operate at a larger scale (with more users, more functionality and/or more features) in the long term.
“Beginning with the end in mind”
Paying attention at the earliest stages to developing a financial model and enlisting supportive partners.

13 of 61

The Framework

14 of 61

Engage young people

Actively and intentionally involving youth as co-creators, collaborators, problem solvers, champions and change agents.
Meaningful engagement

Inclusive and mutually respectful partnership between young people and adults
Power is shared,
Respective contributions are valued and acknowledged, and
Young people’s ideas, perspectives, skills and strengths are integrated into the design and delivery of intervention.

Young people are not just the beneficiaries of the intervention but also drivers of change, especially when it comes to their own health.
Half-hearted efforts should be avoided at all costs, such as those in which young people are seemingly given a voice but in fact have little or no say in the content or the style of communicating it and little or no decision-making authority.

Engaging young people can be challenging – refer to Annex 2 of the guidance for do’s and don’ts on youth engagement.

How do you currently reach and engage youth?
Engaging adolescents and youth means actively and intentionally involving them as co-creators, collaborators, problem solvers, champions and change agents.
Meaningful engagement requires an inclusive and mutually respectful partnership between young people and adults in which power is shared, respective contributions are valued and acknowledged, and young people’s ideas, perspectives, skills and strengths are integrated into the design and delivery of intervention.
As a result, young people are not just the beneficiaries of the intervention but also drivers of change, especially when it comes to their own health. They are the experts on what health information young people need and what technology young people are using.
Half-hearted efforts should be avoided at all costs, such as those in which young people are seemingly given a voice by in fact have little or no say in the content or the style of communicating it and little or no decision-making authority.

**How to engage young people – the do’s and don’ts

This is in the annex of the guidance

15 of 61

Involve the right people at the right time

Each part of the process requires different expertise, input and buy-in. Involving the right people at the right time means knowing who should be at the table, for what purpose and at what point in the process.
Some people or organizations may be involved throughout the entire process, but at different levels. The core team members should know their own roles and identify the expertise they will need at particular times—for example, expertise in digital technologies or promotion and marketing.
Team members should perform only the tasks they are trained for and capable of doing, while allowing others on the team or other participants to contribute their skills when and where needed.

16 of 61

Science Based:

Make decisions based on data and evidence

Every activity and decision should be supported by quantitative and qualitative data and evidence, which will serve as a check on intuition and ensure that the intervention is aligned with actual user needs and preferences.
Each stage requires collecting data and evidence and factoring it into the decision-making process.

17 of 61

Promote safety, privacy and ethical standards

Youth should be protected from potential abuse, incompetence and violations of privacy.
Information and data users provide should be kept confidential.
Specifics depend on the delivery channel of the intervention, the context in which the intervention is being implemented and the laws and regulations of the country.
This will affect:

what data are collected;
how data are collected, used, stored and shared;
and how confidential information and the identities of young people are protected.

Standard practice: Acknowledge the sensitivity of the data being collected, to be transparent about how the data will be collected and used, and to develop, adopt and enforce security policies to protect the data.

Young collaborators and future users should be protected from potential abuse, incompetence and violations of privacy. The information and data they provide should be kept confidential, and they should be able to decide how much information they want to reveal and to whom.
The specifics of keeping young people safe, ensuring their privacy and maintaining ethical standards will depend on the delivery channel of the intervention, the context in which the intervention is being implemented and the laws and regulations of the country.
This will affect what data are collected; how data are collected, used, stored and shared; and how confidential information and the identities of young people are protected.
However, it should be standard practice to acknowledge the sensitivity of the data being collected, to be transparent about how the data will be collected and used, and to develop, adopt and enforce security policies to protect the data.

18 of 61

18

Q/A

Now available in Spanish!

19 of 61

20 of 61

Stage I: Planning the intervention

Conduct a landscape analysis and a needs assessment
Determine whether digital is optimal
Develop a theory-driven approach

21 of 61

Conduct a landscape analysis and a needs assessment

A landscape analysis and a needs assessment collect information needed to generate an understanding of the needs and preferences of young people and the implementing environment, including existing services, interventions, stakeholders and available resources.

Gathers information on the mobile technology infrastructure, existing relevant digital and nondigital health interventions, and relevant organizations and community groups working to improve the health of young people.

Landscape analysis

Determines who needs the intervention, how great their need is and what activities will best address those needs.

Needs assessment

22 of 61

Literature review

A comprehensive summary of previously published research and documents on the topic of interest.
Traditional sources include peer-reviewed published literature as well as grey literature.
Nontraditional sources include reports or data from telecom providers or the app store and social media metrics.

Community consultations

The initial form of outreach to the community in which the intervention will be developed and implemented; they serve to engage people in the process, build relationships and partnerships, and help the project team gain an understanding of the context, sometimes using formative research methods.
It is important to talk to experts, regulatory bodies and other industry leaders early in the process, to map the existing platforms and products as well as the regulatory landscape.

Formative research

The use of research methods (often qualitative) to inform the development of an intervention.
Formative research focuses on factors that influence health outcomes, including the behaviors, attitudes and needs of young people, in order to develop digital health interventions that are appropriate to the particular context.
Can take the form of individual interviews, focus group discussions (in a “safe space” in which young people are not influenced to respond in certain ways by other people in the room) or observation of young people in their everyday environments.

23 of 61

LOGIC MODEL

24 of 61

25 of 61

Group Work

Answer the following questions to begin building

your logic model:

Define the problem that you want to address through a digital solution?

Who? (age, gender, location, other demographic characteristics)
What health issue do you want to target? (Unwanted teen pregnancies, mental health, STIs, GBV etc)
Collect data to back your statements. Do online research to find statistics on the health issues.
What are your expected outcomes?

26 of 61

Determine whether digital is optimal

Digital health interventions may not always be the preferred or most effective way to reach a young population. Therefore, after assessing the needs of the population and the existing implementing environment, the project team must decide whether to move forward with a digital health intervention.

Carefully cost and map out a timeline for the development of the digital intervention, building in time for rounds of user testing and refinement and exploring whether the team has the capacity, time and budget to build the intervention.

TIP

27 of 61

Questions to consider when deciding whether digital is optimal include:

Is a digital health intervention an appropriate strategy to address the health needs of the target group?
More specifically, is it the right fit for the intended users as well as for the implementing environment? (For example, it may be the right fit for young people but not for implementation in a school.)
How does the intervention fit with other existing health interventions or services? How will it be aligned with or integrated into them?
Will access to the intervention be equitable, or will it exacerbate existing inequities in access to information?

28 of 61

Group work: Rapid Digital Landscape Analysis

Conduct rapid technology landscape analysis to provide evidence and statistics to guide the strategy and answer the following questions for your team:

Youth technology access data (what digital devices, platforms and how youth are using tech).
What platform/s would you want to use to reach youth and

why? (social media, messengers, mobile app, website,

online game….)

What devices do you want your users to access the intervention?

(feature phone, smartphone, laptop, tablet, computer…)

Tools:

www.statista.com
www.datareportal.com
Google searches
Local country and regional reports

29 of 61

Develop a theory-driven approach

A theory-driven approach provides a conceptual model outlining what activities are needed as part of the digital health intervention; this underpins the entire intervention and informs the roles that various components of the intervention will play in achieving outcomes.

A theory-driven approach produces a causal theory that connects the planned activities in the intervention with the intended outcomes.

30 of 61

�Develop a Theory of Change

Interventions that are developed and evaluated using a theory-driven approach use the understanding gained in the landscape analysis and the needs assessment to identify how the intervention can effect change and what activities will result in those changes.. The TOC is a description or vision of how and why a program will work. It explains the reasoning behind the actions you're doing and the results you expect the actions to have. Usually, you base the theory of change on past experience, or best practices.

E.g. Youth are more likely to engage with health information that is made available on platforms where they already are.

31 of 61

Development Stage

32 of 61

33 of 61

The Framework

34 of 61

Stage 2�Developing the intervention

Develop a content and delivery strategy
Create a series of prototypes

35 of 61

Develop a content and delivery strategy

Content and delivery are at the core of the intervention. Both should be informed by the theory-driven approach and developed based on information collected during the landscape analysis and the needs assessment.
The content and/or delivery mechanisms may be adopted or adapted from other interventions, or they may need to be developed for the specific health issue and intended users. In either case, a content and delivery strategy is needed to guide the creation and curation of content and provide clarity in the development and implementation stages.

The evidence-based health education information—aligned with recommended health practices and international, regional and national standards—that will be shared with young people through the digital health intervention. Content can be in various media formats, including text, images, video and other multimedia. Content must be accurate and appealing in order to ensure that users understand it and find it interesting and relevant to their needs and circumstances.

Content

Refers to the use of communications mechanisms and software systems to reach young people with health content. The content will not reach the intended users without a thoughtful selection of delivery mechanisms.

Delivery

36 of 61

What are some key tips for developing a content and delivery strategy?

Determine whether existing content can be adapted or new content is needed.

Think about the appropriate tone, explanations, and examples based on the context and culture.

Consider writing the content in the language it will be delivered in to avoid translation challenges.

Factor in time for editing and uploading the content to the delivery channel.

Consider the market and landscape, such as smartphone penetration, the channels young people are currently using and what they trust or don’t trust.

Understand the data security of the platform, including how data will be stored, used and kept secure.

37 of 61

38 of 61

39 of 61

Human-centered design

To better understand and work with young people, they can benefit from using an approach known as human-centered design (or user-centered design), which focuses on users’ needs, wants and limitations. Designing with the intended users in mind is crucial to developing successful interventions. Key human-centered design approaches include:

Personas

Personas are generic descriptions of the types of people involved in or benefiting from the intervention.
Personas help the core team and other participants view the objectives and challenges of the effort from the vantage point of the people who will receive the intervention.
Personas also help align the team and other participants around shared definitions and perceptions.
They provide a common point of reference for those who deliver the intervention, those who monitor or supervise it and, ultimately, those who receive it.

User journeys

User journeys are a way to visualize the user’s experience with the intervention from beginning to end, from the moment of awareness of the intervention to the decision to engage with it, the first interactions, subsequent engagement with it and being affected by it.

Co-design workshops

Co-design workshops convene the intended users, bring them into the design process and build trust.
The goal of such a workshop is to identify what health information the intended users want to know about and how they want the information delivered to them, including the look and feel of the digital solution.
This is an opportunity to collaborate with young people and involve them in design decisions, not just to hear from them.
The workshop should be a safe space where young people can discuss these health topics openly and share what they need most out of the intervention.

40 of 61

41 of 61

42 of 61

43 of 61

44 of 61

45 of 61

46 of 61

47 of 61

48 of 61

49 of 61

50 of 61

51 of 61

52 of 61

52

Q/A

Now available in Spanish!

53 of 61

STAGE 3

54 of 61

The Framework

55 of 61

Implementing the intervention

Carry out a phased launch

A phased release of the digital health intervention into real-world conditions will allow opportunities to refine and improve it before full-scale implementation

Create a promotion and marketing campaign

When the intervention is ready for full implementation, these activities will publicize and disseminate the intervention to increase engagement. They can also help sensitize the broader community about the intervention.

Conduct monitoring and evaluation

M&E efforts involve continuously collecting information to understand whether the intervention is meeting objectives and what effect it is having on broader health outcomes.

56 of 61

57 of 61

Conduct monitoring and evaluation

Monitoring involves the ongoing, routine collection, review and analysis of data generated by digital systems or collected for the purpose of tracking progress toward achieving the intervention’s objectives. It can identify what is and is not working and enable implementers to adjust the intervention and metrics accordingly. Monitoring focuses on answering the question, “Is the intervention working as intended?”

Monitoring

Evaluation is the systematic and objective assessment of an ongoing or completed intervention with the aim of determining its impact on health outcomes, its efficiency and its sustainability. Evaluation focuses on answering the question, “Is the intervention yielding the desired effect?”

Evaluation

58 of 61

M&E Takeaways

Monitoring should be in mind at intervention design/development
M&E data should be available to all members of the intervention team for continuous quality/experience improvement
Digital channels offer potential for ‘creative’ data collection

59 of 61

Group Work

Develop a Goal statement for your digital intervention

The main purpose of _______________ (intervention) is to ______________ (benefit 1) and ____________ (benefit 2) which will be beneficial to _______________ (stakeholder 1) and __________ (stakeholder 2) to achieve the goal of ___________ (health impact such as reduced teen pregnancies).

60 of 61

https://apps.who.int/iris/bitstream/handle/10665/336223/9789240011717-eng.pdf

61 of 61

61

Thank You!

bsheoran@gmail.com