ESTEEM: Measuring Socio-emotional Skills

Clara Delavallade

​

www.poverty-action.org/socio-emotional-skills

Which skills matter for economic empowerment?�TOC: Skills key to goal achievement, but they depend on the goal

Developing Preferences,

Setting “Achievable” Internalized Goals

Emotional awareness

Self awareness

Listening

Empathy

Action Planning

Personal Initiative

Problem Solving

​

​

​

Taking Action

Self Control

Emotional Regulation

Perseverance

Expressiveness

Negotiation

Interpersonal Relatedness

Influence

Collaboration

​

​

​

Key Behaviors for Economic Empowerment

Developing Networks/ Obtaining support

Overcoming obstacles, binding constraints, social norms

Finding and utilizing opportunities/solutions: jobs/business/programs

​

Why ESTEEM? Addressing Measurement Challenges in LMICs

​

​

Countries -

Introducing ESTEEM

Development

• Lit on Employment & Gender Differences
• Map to other Frameworks
• Focus Group Discussions
• Spanning Range of SES

​

Self Report Scale (Empathy: Cognitive & Affective)

​

Example Measures

​

​

​

Situational Judgement Tests (Emotional Regulation)

​

Tasks: Counting triangles, CPTX, Listening comprehension/behaviors, SJT-embedded tasks

Measure Development�Iterative Psychometric Analysis�

Psychometrics in each context

Content Validity

Key Learnings

• Importance of each content validity step
• Tasks often time consuming and focus on sub-skill
• 2 skills had 2 dimensions: Listening & Relatedness
• Behaviors depend on spheres -> scenario-development was key
• Method differentiates factors more than skill
• Reversed items could not be included on the same scale

​

Key take-aways from validation papers

• Which measure:
• Behavioral measures outperform self-reports in predicting labor market outcomes
• Self-reported measures better predict mental health outcomes

​

• Which skills? Relationship management is only consistent predictor of income: Collaboration, negotiation, influence and empathy are the strongest predictors of income across multiple countries.

​

• Mixed results on gender: For women, SES are a better predictor of income if norms are less restrictive. Seems like a promising avenue for further research.

​

• Implications
• Curricula and measures need to consider relationship management skills, which is often not included
• In development process, we learned about how skill “levels” depend on the “sphere” of usage
• Is it levels of skills that matter, how much you value them, or whether others value you having them?

Pillar 3: RCTs testing various SES trainings�1. BRAC Tanzania IE�Sample: 4800 NEET Youth

Baseline, Follow-up 3 and 12 months after intervention: May 2021-Oct 2022

Baseline Results

• Female disadvantage in widely used self-reported SES measures, which disappears with behavioral measures.
• Gender differences in self-reports are strongly correlated with social desirability and internalized gender norms.
• There is a larger gap between self-reported and behavioral measures among men, also more strongly correlated with SD and regressive gender beliefs on women’s ability to problem-solve among men.
• Suggests that men over-report their skills more than women underestimate theirs.

​

Preliminary IE Results

• Limited impacts on behavioral measures of SES but significantly positive impacts on most self-reported measures at ML, which are not sustained after one year.
• Significantly positive impact on employment and income only for men seeking jobs at baseline.
• No significant difference between the types of SES training (awareness, management, or both)
• These results suggest that providing SES training alone may not be sufficient to improve young women’s employability and income.

​

BRAC Tanzania IE�Sample: 4800 NEET Youth

2. Nigeria APPEALS IE: Intrapersonal v. Interpersonal skills

Sample: APPEALS (Agri-business training + grant) beneficiaries

Design: 4 days interpersonal v. 4 days intra+inter v. Control

​

Preliminary Results from follow-up 2.5 years after intervention:

​

• Different impacts of SES trainings for men and women
• Women specialize in one business, hire more female workers, and increase profits by 50%+, but see no change in decision-making or empowerment.
• Men diversify income sources, increase household income, and reduce spending on temptation goods.

​

• Male employers hire fewer women, female employers hire more, redistributing female workers but reinforcing occupational segregation.

​

• Gendered skill development reinforces social norms
• Women gain interpersonal (communal) skills (e.g., collaboration, empathy).
• Men develop intrapersonal (agentic) skills (e.g., decision-making, problem-solving).
• SRs increase for women, only where gender norms less restrictive

​

• Policy implications
• Tailored training: Interpersonal skills benefit women most; combined training is more effective for men.
• SES trainings improve skills and business outcomes, especially for women, but do not help them deviate from norms.

Lessons and next steps

Lessons learnt:

• Importance of interpersonal skills (relationship management) for women’s labor market outcomes
• Importance of going beyond self-reports 🡪 behavioral measures less subject to biases
• Self-reports matter too
• if interested in wellbeing/mental health outcomes
• Proxy for perception of skills/self-confidence, a predictor of econ outcomes too

​

ESTEEM measure applications:

• Measuring impact of SES or other interventions on SES
• Initial assessment of SES, to inform intervention scoping:
• To highlight skills to focus on
• To capture potential heterogeneity along dimensions such as gender, age, initial goals, etc…

​

Next steps:

• Dissemination: GIL research uptake (incl. webinars) + JPAL-IPA research agenda
• Digitizing SES measures and incorporating them into SkillCraft platform
• Designing short version
• Adapting and testing in other contexts
• Which skills to signal to employers?
• Examining returns to SES when relaxing gender norms (Nigeria for Women)

Website: www.poverty-action.org/ses

Thank you!