1 of 21

Improving Teaching Practices in a FLN Program in Senegal with Applied Behavioral Science

mEducation Alliance Symposium 2024

Carolina Better, Managing Director

1

2 of 21

First, let me introduce …

applies insights from behavioral science to design solutions to some of the world’s most persistent social problems.

focuses on educational innovations and action research to promote quality education in Senegal.

2

3 of 21

ARED’s Ndaw Wune program

Tutors follow tutor guide for relearning group; autonomous groups work independently

Program aims to improve FLN outcomes for 2^nd and 3^rd graders in Senegal

Tutors (public school teachers) teach after school 3x per week

~20 learners are grouped into 2-3 groups based on their level of reading and math

One “relearning” group and 1-2 “autonomous” group(s) per lesson

Tutors should rotate between relearning and autonomous groups throughout the class

3

Specifically, ARED runs a program called Ndaw Wune that aims to improve foundational literacy and numeracy outcomes for 2^nd and 3^rd graders who are behind grade level in reading and math

[CLICK]

Ndaw Wune classes happen 3x a week, for two hours after school, and they are taught by “tutors”, which are predominantly public school teachers

[CLICK]

Each class has about 20 students and these are grouped into 2-3 smaller groups based on their level of reading and math

[CLICK]

Each class, one of these groups is designated as the relearning group, while the other groups are designated as the autonomous groups

[CLICK]

For the relearning group, the tutor follows specific lesson plans in their tutor guide, while the autonomous groups complete reading and math exercises independently in their student booklets

[CLICK]

As such, the tutor needs to spend time rotating among the groups each class to check the work of each group, address any questions, provide positive feedback, etc.

4 of 21

Background on Senegalese schooling

At the end of primary school,13% of learners reach the minimum proficiency level in math and 34% in reading.¹
Focus is on teaching to the curriculum and rote memorization.
Classes are often large and meeting the learning needs of different levels of students is not a priority.

1. UNESO Institute for Statistics (2022)

4

5 of 21

Our behavioral design approach

Align on a behavior of focus

Study the context to identify key barriers

Create and refine solutions to address barriers

Test solutions and learn from the process

DEFINE

DIAGNOSE

DESIGN

TEST

SCALE

Adapt proven solutions for impact

5

6 of 21

Our objective throughout the project

To help tutors increase the amount and quality of support provided to autonomous groups during Ndaw Wune classes.

DEFINE

6

7 of 21

Why is this important?

Learners spend at least half of their time in autonomous groups, equating to 3+ (out of 6) hours of instruction per week

Learners are more likely to being off-task or not engaged in learning when they don’t receive enough support from tutors

DEFINE

7

8 of 21

Behavioral barriers preventing teachers �from engaging with autonomous groups

DIAGNOSE

Tutors prioritize the upskilling of the relearning �group to such an extent that the needs of the autonomous groups are overlooked.

Tutors assume if students are quiet, they are on task.

Tutors don’t have an explicit or implicit rule of thumb for when to rotate and provide support to autonomous groups.

Tutors face obstacles that impede their ability to provide support to autonomous groups.

‘But they are quiet...’

‘No prompts’

‘It’s not easy’

‘I’m focusing on improving the skills of the relearning group’

8

And also through these interviews with tutors and observations of their classes, we identified 4 key behavioral barriers that were preventing tutors from providing enough support to the autonomous groups:

[CLICK]

The first is that tutors prioritized improving the foundational skills of the relearning group to such an extent that the needs of the autonomous groups were overlooked [psychologies: goal conflict, loss aversion, scarcity, & tunneling]

[CLICK]

The second is that tutors assumed that if students were quiet, they were on task … meaning that if they weren’t making noise or running around the classroom, this meant they were doing their exercises (which we often saw was not true) [psychologies: social norms, mental models]

[CLICK]

The third is that tutors didn’t have any rules of thumb to guide them as to when to rotate and provide support to the autonomous groups [psychologies: habit, social norms, salience, and heuristics]

[CLICK]

Finally, tutors face obstacles that impeded their ability to provide support to the autonomous groups -- such as classroom setups that contribute to students being distracted, the timing of exercises not being well aligned, etc. [psychologies: hassles, switching costs, status quo bias]

_________

More details on diagnosis:

Audited all Ndaw Wune materials: tutor guide, student booklet, tutor weekly summaries, supervisor observation grid, training material
Conducted 10 classroom observations
Held 30 qualitative interviews with tutors, supervisors and inspectors

9 of 21

We designed a suite of interventions to address these barriers

DESIGN

Heuristics checklist & reminders

Tutor training + commitment

Ndaw Wune Promise

DESIGN

9

10 of 21

A short guide with a checklist that outlines what tutors should do before, at the start of, and during class. This is placed at the beginning of the tutor guide.

It is accompanied by visual reminders in the lesson plans in the tutor guide to rotate to autonomous groups, and guidance on the ideal classroom setup.�

icon to rotate

DESIGN

Barriers addressed:

DESIGN

Heuristics checklist & reminders

10

The first solution we designed was a short guide with a checklist that provides simple heuristics or rules of thumb for what tutors should do before, at the start of, and during class. We incorporated this short guide into the beginning of the tutor guide that they follow during class.

We also added visual reminders in the lesson plans that tutors follow in the tutor guide to remind them when to rotate to the autonomous groups.

__________________

More details on how this solution addresses the behavioral barriers identified:

‘Focused on skills of the relearning group’: Emphasizes the importance of providing support to autonomous groups and provides a simple list of steps on how to do so.
‘But they are quiet…’: Emphasizes students in autonomous groups should be completing their work (not just quiet) and provides rules of thumb for how to identify off-task students.
‘No prompts:’ Icons in the lesson plan prompt tutors when to rotate to autonomous groups.
‘It’s not easy’: Reduces hassles faced by tutors by providing keys to tutors for how to set up the classroom.

11 of 21

An additional module in tutors’ training that utilizes role modeling and role playing to practice how to set autonomous groups up for success, rotate, and provide support.

This is accompanied by a commitment contract that helps tutors form strategies for how to achieve their goals.

DESIGN

Barriers addressed:

DESIGN

Tutor training + commitment

11

We also created an additional module to be incorporated into the training that tutors receive before the Ndaw Wune classes begin. The goal of this module is to utilize role modeling and role playing to illustrate and practice how to set autonomous groups up for success, rotate and provide support to them.

This is accompanied by a commitment contract that helps tutors form strategies for how to set up all groups for success and provide them with support, and commit to doing so.

__________________

More details on how this solution addresses the behavioral barriers identified:

‘Focused on skills of the relearning group’: Emphasizes the importance of supporting both relearning and autonomous groups.
‘No prompts’: Prompts tutors to outline and commit to how they will support students in autonomous groups.
‘But they are quiet…’: Resets norms around what it means to be a good student.
‘It’s not easy’: Provides tutors with the opportunity to observe and practice classroom management techniques.

12 of 21

A simple, catchy song paired with hand movements to help tutors and students set goals for how they will behave in class and what they will accomplish.

DESIGN

Barriers addressed:

DESIGN

Ndaw Wune Promise

12

13 of 21

We ran a cluster RCT, randomizing at the supervisor level…

CONTROL

83 tutors

Original Ndaw Wune training and materials

TREATMENT

101 tutors

Behavioral interventions

January 2024

Intervention launch

April/May 2024

Endline data collection

Randomized at the supervisor level

TEST

13

14 of 21

…and measured three key outcomes

Number of rotations to autonomous groups

Quality of interactions with autonomous groups

Duration of time spent with autonomous groups

TEST

14

15 of 21

Class arranged in the ideal configuration

Reading: N=138; P=.09*; Math: N=180; P=.026**

+29pp*

+24pp**

Tutors in the treatment group were significantly more likely to set up the desks in the ideal classroom setup

15

16 of 21

Number of rotations to autonomous groups

Reading: N=176; P=.70; Math: N=176; P=.003***; Overall: N=178; P=.29

Tutors in the treatment group had 0.5 more rotations on average (10% increase) to autonomous groups

+0.16

+0.48***

+0.5

ROTATION: any instance where a tutor spends more than 1 minute, 30 seconds with an autonomous group.

16

We also saw that tutors in the treatment group rotated MORE to the autonomous groups than tutors in the control group

We defined a rotation as: any time a tutor went to an autonomous group and spent more than 1 minute, 30 seconds with them

Specifically: [read results]

NOTES:

Reading:

Math:

Reading: rotated 0.16 rotations more

Math: rotated 0.48 rotations more

Average time per rotation (including treatment and control)

Reading: 2 minutes, 57 seconds
Math: 3 minutes, 4 seconds
Overall: 2 minutes, 57 seconds

We also analyzed the number of rotations by number of autonomous groups in reading, as we had 39 observations in reading that 1 autonomous group. The results were not statistically significant, but directionally, we saw a decrease in number of rotations among tutors in the treatment group when they had 1 autonomous group and an increase in the number of rotations among tutors in the treatment group with 2 autonomous groups when compared with the control. The changes were very small: 0.22 rotation decrease and a 0.13 rotation increase

17 of 21

Time spent with autonomous groups

Reading: N=176, P=.27; Math: N=176, P=.017**; Overall: N=181; P=0.13

+2,47

+1,28**

+1,38

Tutors in the treatment group spent an additional 2 min, 47 seconds (14% increase) with autonomous groups

17

18 of 21

Quality of rotations to autonomous groups

Reading: N=157, P=.51; Math: N=154, P=0.175; Overall: N=168, P=.095*

+0.5*

+0.26

+0.2

QUALITY ROTATION: A tutor performing at least three of these behaviors when rotating to an autonomous group:

1. Ask if there are any questions�2. Check the work of 2+ learners�3. Provide corrective feedback�4. Offer positive reinforcement to 2+ learners�5. Share instructions for next activity

Tutors in the treatment group had 0.5 more quality rotations (11% increase) to autonomous groups.

18

Also, when tutors in the treatment group rotated, they provided better quality support to autonomous groups than tutors in the control group

We defined a quality rotation as: a tutor performing at least three of the following five behaviors during a rotation to an autonomous group: 1) ask if students have any questions; 2) check the work of at least two students; 3) provide corrective feedback to students; 4) offer positive reinforcement (e.g. good job, well done) to at least two students; and 5) share the next set of instructions.

Specifically: [read results]

NOTES:

-It’s hard to say why the quality of rotations is insignificant for math and reading but not overall, but it’s likely due to variance and sample size

-Sample size is smaller for reading and math as it does not include observations where there was 1) no relearning group or 2) tutors that did not rotate during class.

-Sample size is slight larger for overall as it’s a combination of tutors who were included in reading and in math

In general, % of rotations that were considered quality was high (NB, these figures include treatment and control):

Reading: 90%
Math: 86%
Overall: 89%

19 of 21

Key takeaways

The results are promising: behavioral interventions improved tutors’ amount and quality of support to autonomous groups.

Behavioral interventions were low cost and relatively simple to implement, building on existing materials and touchpoints of the Ndaw Wune program.

We recommended that ARED roll out these solutions and continue to emphasize the importance of providing support to all learners (including those in autonomous groups).

Cost: 4 cents USD (25 CFA franc) per student

19

20 of 21

Recommendations for implementing teaching methodologies with multiple groups of learners

Frame groups as equal

Role playing in trainings

Clear guidance on how to support each group

Visuals for important information

Rotation reminders in the right channels

Leverage the ‘middle tier’ for implementation and accountability

20

Frame groups as equal without emphasizing one group over another in training, program materials, or communication to teachers.
Incorporate role playing and role modeling exercises into teacher training, including opportunities for authority figures or coaches to role model and teachers to role play specific actions.
Develop clear guidance about how to support and how much time to spend with each group.
Use visuals to convey important information to help teachers quickly grasp and implement recommended practices.
Create reminders to rotate to different groups through channels that teachers use regularly (e.g., teacher guides, WhatsApp groups, posters in classrooms, etc.).
Leverage the middle tier of education systems, such as coaches and supervisors, to strengthen and support the implementation of managing multiple groups of students within the same classroom. This can be done through ongoing guidance, monitoring, and targeted feedback to teachers.

21 of 21

Want to learn more?

HOT OFF THE PRESS

globaldev@ideas42.org

21