Collaborative Lesson Research - improving student learning �(Results of a large-scale RCT)

Geoff Wake

Marie Joubert

Juliane Collings

Outline��- CLR: Change theories and Theories of change - GW�- The research context - GW�- Research design – the RCT - GW�- CLR – adaptation as a mediating mechanism (boundary objects) - MJ from slide 21�- Implementation and process evaluation - MJ�- Impact evaluation - GW�- Conclusion and discussion – GW and MJ����

CLR: Change theories and Theories of change

Change Theories and Theories of change

Fundamental to CLR is that it is a collective activity in which participants engage because of a desire to change (improve) teacher professional knowledge both as individuals and as part of a community.

​

Change theories - apply generally across different settings and interventions

​

Theories of change

• explicitly identify how a specific intervention is designed to bring about a desired change.
• The moderators of the change process are made explicit

​

(Rheinholz and Andrews, 2020)

EDUCATION & TRAINING FOUNDATION

4

Change Theories and Theories of change

In any particular instantiation of a CLR based ToC there will be a substantial number of moderating factors.

​

For example,

• the cultural context and setting in which the teacher(s) work that will affect a teacher’s expectations in relation to, and notions of, professional learning and collaboration with others
• resources available such as time and funding (if necessary),
• expectations of LS and of classroom-based research at individual, and school levels;
• policy at local and national levels,
• frequency of involvement,
• support for knowledge exchange,
• teacher knowledge related to teaching, curriculum and subject

… and so on.

EDUCATION & TRAINING FOUNDATION

5

Researching teaching for mastery

6

The research context

Research context

Students who fail to achieve a sufficiently high enough grade in the sitting of the national GCSE qualification for students aged 16.

​

One year course – effectively 8 months – in Further Education colleges

By age 19 GCSE pass rates for these students indicate that less than twenty percent attain the desired grade 4 or above (DfE, 2019).

​

Centres for Excellence in Maths https://www.et-foundation.co.uk/professional-development/maths-and-english/cfem/cfem-overview/

21 Centres funded by Department for Education

​

University of Nottingham, Centre for Research in Mathematics Education - research partner.

EDUCATION & TRAINING FOUNDATION

8

Researching teaching for mastery

EDUCATION & TRAINING FOUNDATION

9

2019 – 20

2020 – 21

2021 – 22

2022 – 23

Research design – the RCT

TfM intervention RCT

EDUCATION & TRAINING FOUNDATION

11

TfM RCT: three-armed Randomised Controlled Trial

Researching teaching for mastery

12

Two intervention models:

ALL Group 1(full intervention) and Group 2 (partial intervention) teachers:

• provided with lessons A&B + 1 – 5 (lesson plans, presentation, worksheets etc);
• asked to teach the lessons at the appropriate times (Windows 1 to 5) to ALL their GCSE resit classes;
• asked to adopt the approaches modelled by the lessons in other lessons.

EDUCATION & TRAINING FOUNDATION

13

Group 1(full intervention) teachers:

• additionally took part in a modified lesson study process focused on each of the five lessons 1 – 5. Three planned to be face-to-face and two (lessons 3 & 4) online.

Researching teaching for mastery

EDUCATION & TRAINING FOUNDATION

14

2019 – 20

2020 – 21

2021 – 22

2022 – 23

Impact evaluation

Implementation and process evaluation

Evaluation of the trial

Impact evaluation

GCSE scores of students in all groups (quantitative).

​

Implementation and process evaluation (IPE)

Data about how the intervention was implemented (mostly qualitative).

15

Research activity (IPE): quantitative data collection

ALL teachers and students:

• Pre-intervention teacher survey
• Post-intervention teacher survey
• Pre-intervention student survey
• Post-intervention student survey

​

ALL Group 1 and Group 2 teachers:

• Logs of lessons taught 
• PD evaluations

​

EDUCATION & TRAINING FOUNDATION

16

2 trial teachers for each of these lead teachers.

17

Group 1

(Full intervention)

Group 2

(Partial intervention)

Group 3

(Business as Usual)

4 trial teachers

​

4 trial teachers

​

3 lead teachers

​

​

Research activity (IPE): qualitative data collection

Case studies

Research activity (IPE): case studies

Case study teachers (Groups 1 and 2)

• Observations of lessons 1 to 5 (once or twice per lesson)
• Interviews (after the first PD sessions, lessons 2 and 4)

​

Case study teachers (Group 1 only)

• Student focus groups (twice in the year)
• Observations of cluster meetings (all meetings)

​

​

EDUCATION & TRAINING FOUNDATION

18

CLR: adaptation as a mediating mechanism (boundary objects)

Boundary objects

EDUCATION & TRAINING FOUNDATION

20

Boundary objects

ALL teachers and students:

• Pre-intervention teacher survey
• Post-intervention teacher survey
• Pre-intervention student survey
• Post-intervention student survey

​

ALL Group 1 and Group 2 teachers:

• Logs of lessons taught 
• PD evaluations

​

EDUCATION & TRAINING FOUNDATION

21

Researching teaching for mastery

Initial discussion

document

​

Hackathon

22

Teaching for Mastery in FE: the handbook

23

https://www.et-foundation.co.uk/wp-content/uploads/2020/03/CfEM_Mastery_Handbook.pdf

24

Teaching for Mastery: �Five Key Principles

Teaching for Mastery in FE: about the lessons

Define and inform an approach to TfM

​

Designed in alignment with, and exemplify, the Key Principles

Support teacher engagement with, and reflection on, TfM via two research questions per lesson

EDUCATION & TRAINING FOUNDATION

25

CLR process

EDUCATION & TRAINING FOUNDATION

26

CLR process

EDUCATION & TRAINING FOUNDATION

27

Juliane – add about your experience, your cluster

I’ll look for some video

CLR process

EDUCATION & TRAINING FOUNDATION

28

Marie add some quotes from the IPE report

Implementation and process evaluation

Implementation: Attrition

At data analysis

​

No. of settings: Full intervention 39 (-15)

Partial intervention 25 (-9)

Control 48 (-11)

No. of students: 3390

​

Implementation

Teachers in both intervention groups report that the PD intervention programme and using the exemplary Teaching for Mastery materials

​

• were effective as an introduction to the principles of Teaching for Mastery
• led to their improved understanding of Teaching for Mastery
• led to changes in their teaching practice and high levels of intended change in teaching practice (in subsequent years)
• led to improved student engagement and understanding.

Implementation

Implementation

Implementation

Juliane – how your teaching has changed, and the changes in your cluster group.

I can provide some nice photos.

Or video.

Implementation

Implementation

• Marie add some photos to demonstrate some of these things (e.g. use of representations)

37

Impact evaluation

Impact

The post-16 ‘covid’ cohort 2022

Impact

Our impact analysis detects a small positive effect for each intervention model. �There is most impact on students from the most deprived backgrounds (as measured by free school meals) who were taught by teachers in the full-intervention. 

​

​

The full intervention has greater impact against the business-as-usual control than the partial intervention model leading to the tentative conclusion that the lesson study practices of the full intervention add value to the Teaching for Mastery programme in terms of impact on outcomes.

Impact

Impact

At a more detailed level we have investigated a sub-scale score for GCSE questions that are aligned with the exemplary 'Teaching for Mastery' lessons taught by teachers in both full and partial intervention groups.

​

This analysis confirms the analysis of the partial intervention and detects an even greater (admittedly a small increase to an effect size of 1.13), impact on students taught by teachers in the full intervention.

Impact

43

Impact

44

Impact

In line with Education Endowment Foundation reporting, these effect sizes suggest

Business as usual

Months of teaching

8

Conclusion

Conclusion

The design of the intervention appears to have the desired effect: that is, a change in teachers’ practice leading to improved learning outcomes for students.

​

The intervention that incorporates an element of lesson study appears to be more effective as it is students taught by teachers who have taken part in this element of intervention for whom the effect is greatest.

​

This indicates that this model of intervention provides teachers with a more effective experience of engaging with, and incorporating into their practice, Teaching for Mastery.

Conclusion

Lessons for Lesson Study

​

Our study that researches impact on student learning outcomes highlights the following aspects of collaborative lesson research that appear to best support professional learning of teachers:

• Clarity of important shared principles that underpin the carefully designed intervention based on a clear Theory of change.
• Change mechanisms and boundary objects such as agreed lesson structures that embody and support key principles
• CLR processes with shared and understood roles, ways of working and cross school/college involvement

geoffrey.wake@nottingham.ac.uk�marie.joubert1@nottingham.ac.uk�Juliane.Collings@fareham.ac.uk��