1 of 28

DIGITAL SEGMENTATION FOR SBCC INTERVENTION

SHUJAAZ INC & iMEDIA

​

28 NOVEMBER 2023

2 of 28

IN THIS PRESENTATION

LIGHTNING INTRO TO SHUJAAZ

WHY WE WORK ON DIGITAL SEGMENTATION

​

OUR JOURNEY TO-DATE

​

1

2

3

28.11.2023

NLP + Digital Segmentation

​

4

Discussion

​

5

3 of 28

  • A network of social ventures with the goal of breaking down barriers so young people can take control
  • 9m+ fans in East Africa aged 15-24
  • Authentic, relevant, relatable stories at the heart of SBCC programming
  • Comic book, social media, events, engagements through distributor network
  • Proven impact in financial health, SRH, governance and elections
  • Through leadership and creativity

WHAT IS

SHUJAAZ?

4 of 28

WHY DIGITAL

SEGMENTATION?

  • Segmentation has always been at the center of our work
  • Digital offers new opportunities for 1-2-1 engagement at scale
  • Familiar brand + safe-space to build trust online
  • Traditional + and digital data to create “transcending” segments
  • Fictional characters + real stories to build genuine engagement
  • Targeted/tailored content + community interaction to empower youth to educate and “build” each other

5 of 28

  • More adolescent girls and young women making personal choices on their health and life (including use of contraception, having children, getting married, etc.) free from judgement, stigma, fear, and violence
  • Timely, relevant offers to men and women – by segment, mostly online – e.g., from inspiration to role modeling to counseling to buying
  • Understand what works, sharpen our practice, teach others to do better - globally

WHAT DO WE WANT

TO ACHIEVE?

6 of 28

OUR JOURNEY TO-DATE

Expert consultations

– Preparing written protocols for all stages of the analysis – traditional data and digital data

– Discussing the approaches with experts vis a dedicated session and email

– Refining the approaches

​

Deep-dive into cross-sectional

survey data

– Developing clusters of young people by attitudes towards SRH, gender and social norms expressed in the survey

– Cross-tabulation by demographics, socio-economic status, and reported SRH behaviors to define segments

– Cross-tabulation by access to mobile phones and internet

​

Qualitative verification of emerging segments

– Recruiting groups of people online and offline representative of each segment by demographics, socio-economic status, and reported SRH behaviors

– Testing emerging segments via vignettes and dilemma stories about the segments

– Refining, expanding segment descriptions

Developing “trigger content” for each segment – 4 Facebook posts for each of the 6 segments designed to stimulate discussions, comments, likes and other reactions – to generate digital data for deep-dive and development of the digital segments

7 of 28

NLP & DIGITAL SEGMENTATION

8 of 28

NLP + Digital Segmentation

Natural language processing (NLP) refers to the branch of computer science/AI concerned with giving computers the ability to understand text and spoken words in much the same way human beings can.

NLP combines rule-based modeling of human language with statistical, machine learning, and deep learning models.

Together, these technologies enable computers to process human language ‘understand’ its full meaning, complete with the speaker or writer’s intent and sentiment

​

What is NLP?

9 of 28

NLP + Digital Segmentation

Together, these technologies enable computers to process human language ‘understand’ its full meaning, complete with the speaker or writer’s intent and sentiment

What is NLP?

10 of 28

NLP + Digital Segmentation

Large Language Models (LLMs) are changing what has been previously possible in traditional text analytics approaches.

With ~ 1 trillion parameters, ChatGPT and other LLMs such as Google’s BERT are able to tease out intent and sentiment with greater accuracy that previously possible.

​

​

Trends in NLP: ChatGPT and LLMs

11 of 28

NLP + Digital Segmentation

Large Language Models (LLMs) and generative AI can enable us to monitor, understand and modify how we engage our audience’s needs at scale.

Greater efficacy = higher impact

Understanding our audience through who they are alongside their beliefs and attitudes can guide us into segmenting them for greater impact.

​

​

​

How will NLP help?

12 of 28

NLP + Digital Segmentation

Approach Overview

Who said it?

Data augmentation through comments/ metadata to further understand the person behind the comment.

What are people saying?

Translation of multiple languages/ Sheng used by youth engaging on the platform to a widely understandable format e.g. English for analysis

What did they mean?

Identifying points of discussion/ topics raised by online consumers and sentiment towards topics on platform

13 of 28

NLP + Digital Segmentation

Attitudes and beliefs

Topics + attitudes

Topic modelling

Sentiment analysis

Text translation

Facebook comment

14 of 28

NLP + Digital Segmentation

Demographics

Demographics

Sex

Location*

Education*

Facebook comment

metadata/augmented data

15 of 28

NLP + Digital Segmentation

Demographics + Attitudes and beliefs

Topics + attitudes

Topic modelling

Digital Segments

Demographics

Sentiment analysis

Sex

Location*

Education*

Text translation

Facebook comment

metadata/augmented data

16 of 28

NLP + Digital Segmentation

Translation

Sentiment analysis

Topic Modelling

Data augmentation

​

​

​

​

NLP/ Generative AI in:

17 of 28

NLP + Digital Segmentation

NLP in Translation

Objective

Accurate translation of texts obtained

Approach

  • Use of annotators
  • Use of Google translate
  • Use of ChatGPT via API

Results

Comments translated into English for easier processing

18 of 28

NLP + Digital Segmentation

NLP in Translation

Outcome & Learnings

  • Annotator approach was time and labor intensive
  • Google translate worked with limited success
  • ChatGPT handles translations well, including euphemisms
  • ChatGPT shows inconsistent behaviour/ lack of translation to some texts
  • Handling emojis, gifs to be considered

19 of 28

NLP + Digital Segmentation

NLP in Translation

Text

“Utajua kwanini HIV/AIDS huandikwa na capital letters 😅”

​

​

Translated results

You'll know why HIV/AIDS is written in capital letters (a colloquial way to say the issue is serious).

​

​

​

​

ChatGPT

20 of 28

NLP + Digital Segmentation

NLP in Sentiment Analysis

Objective

Obtain sentiment expressed by posts in relation to topics discussed

Approach

  • Use of static language models i.e. quanteda package in R

​

  • Use of LLM - roBERTa sentiment model

Results

Positive, negative or neutral sentiment classification

21 of 28

NLP + Digital Segmentation

NLP in Sentiment Analysis

Outcome and learnings

roBERTa handles translations more accurately than static models

22 of 28

NLP + Digital Segmentation

NLP in Topic Modelling

Objective

Obtain meaningful topics from user discussions under posts by Shujaaz

Approach

  • Traditional topic modelling methods eg STM,LDA

​

  • Use of Google’s BERT model: English and multilingual

Results

Subtopics + topics relating to aspects of society, sexual and reproductive health

23 of 28

NLP + Digital Segmentation

NLP in Topic Modelling

Outcome and learnings

  • BERT performed best in topic and subtopic extraction, with multilingual model proving better than english model
  • Multilingual model provides safety chek against ChatGPT erroneous behavior
  • Topics obtained require qualitative analysis as a sense-check
  • LDA, STM showed limited efficacy owing to data size

24 of 28

NLP + Digital Segmentation

NLP in Data Augmentation

Objective

Use publicly available data to infer demographic data

Approach

  • Use names to infer gender through annotators
  • Test efficacy of ChatGPT

Results

Gender description for all individuals under segmentation analysis

25 of 28

NLP + Digital Segmentation

NLP in Data Augmentation

Outcome and learnings

  • ChatGPT classified 90% of genders selected correctly when it predicted as male or female

​

  • 30% of all results were classified under unknown, reducing efficacy to ~68%

26 of 28

NLP + Digital Segmentation

Coming together

“Wacha umalaya”

​

​

negative

​

Stop promiscuity.

​

multiple

sex partners

​

Segmentation

The Traditional Moralist?

The Concerned Observer?

​

​

Translation Engine

Data augmentation

Sentiment analysis

Demographics (urban male)*

​

Topic modelling

27 of 28

NLP + Digital Segmentation

Conclusions

  • Opportunity exists to scale digital segmentation with NLP
  • Need to sense check/ conduct qualitative analysis
  • Handling varied results from ChatGPT
  • How to make more of other data formats + text?

​

28 of 28

@ShujaazInc

Discussion and questions