Implementation workshop
Mr Alastair Gittner
(Research Lead; Sheffield Associate Research School)
Please could you put your name, school and phase (if its not obvious) into the chat while people are joining. Could you also add if you have been in the current cohort or a refresher from the first cohort.
Thank you
Housekeeping
Please could you put your name, school and phase (if its not obvious) into the chat while people are joining. Could you also add if you have been in the current cohort or a refresher from the first cohort.
Thank you
Pre reading
Session 1: Role of evidence in meeting the needs of those at risk of underachievement
Wednesday 18th Jan 3-5pm
Live online
Session 2: Analysing existing issues and strategies
SELF STUDY
Emailed during w/b 24th January
Session 3: Understanding and reviewing the tiered approach
Thu 9th Feb
3-5pm
Live online
Session 4: Identifying solutions
SELF STUDY
Emailed during w/b 13th Feb
Session 5: Preparing through implementation planning
28th March
3-5pm
REMOTE DELIVERY
Writing an implementation plan
SELF STUDY
Implementation Workshop
4th May 2023
CORE and 1st phase
CORE PROGRAMME 2nd phase
(available for all but aimed at schools and colleagues who did not attend cohort1)
✓
✓
✓
✓
✓
✓
?
Our protocols
Your “homework” from today’s session�or activities while watching the recording.
The role of good implementation
Making the difference for disadvantaged pupils.
Knowledge of how disadvantage impacts pupils and families - understanding the complexity of the problem.
Understanding of the implementation process and how this affects successful decision making and outcomes. An improved implementation climate is also an outcome of effective implementation - doing this well provides a model and set of procedures for doing other things well.
Knowledge of what quality use of evidence looks like.
Current PP strategy reviewed. Issues around data, including the influence of bias, identified.
Knowledge of the evidence which supports the tiered approach.
Using the Tiered Approach to establish priorities for your context.
Completed PP strategy with the knowledge and tools to complete future strategies clearly established.
Processes for monitoring and self-evaluation embedded.
Improved outcomes for disadvantaged pupils.
Understanding of evidence which will inform current, and future, strategies and practice.
Establishing the active ingredients of your approach.
Making the difference for disadvantaged pupils.
Knowledge of how disadvantage impacts pupils and families - understanding the complexity of the problem.
Knowledge of what quality use of evidence looks like.
Current PP strategy reviewed. Issues around data, including the influence of bias, identified.
Knowledge of the evidence which supports the tiered approach.
Using the Tiered Approach to establish priorities for your context.
Completed PP strategy with the knowledge and tools to complete future strategies clearly established.
Processes for monitoring and self-evaluation embedded.
Improved outcomes for disadvantaged pupils.
Understanding of evidence which will inform current, and future, strategies and practice.
Establishing the active ingredients of your approach.
Understanding of the implementation process and how this affects successful decision making and outcomes. An improved implementation climate is also an outcome of effective implementation - doing this well provides a model and set of procedures for doing other things well.
Knowledge of how disadvantage impacts pupils and families - understanding the complexity of the problem.
Today
Distilled & digestible high-level summary
Detailed overview of PP strategy
Implementation plan for specific aspects of PP strategy
Session 5-6 Activity
What’s the problem?
An overview of the implementation planning process
What’s the problem?
These are the kinds of questions we have been asking while exploring problems:
What’s the problem?
Sample problem/challenge statements:
Some staff view disadvantage as an ‘intervention issue’ separate from the classroom – they don’t understand how our approach to curriculum, pedagogy and adaptive teaching directly support disadvantaged students (a school culture / knowledge issue)
Although modelling is used in many lessons, there isn’t a ‘gradual release of responsibility’ – as a result, students remain too reliant on teacher guidance and don’t ‘own’ the strategies modelled. (a classroom practice / PD issue)
Our analysis of intervention X shows that, although it is evidence-informed, it is not having the desired impact. This appears to be linked to problems with consistent timetabling. (An intervention ‘active ingredients’ issue).
What’s the problem?
In contrast, these problem statements are unhelpfully broad. It would be hard to know what to do about them or where to begin.
e.g. low literacy levels might appear to be the problem, but the root cause could be more specific:
Problems: key points
What needs to happen?
What’s the problem?
Active Ingredients
This is the thing that we think will help to address the problem
Active Ingredients
When and how should it happen?
What needs to happen?
What’s the problem?
Implementation activities
Realistically, what can happen?
How will we know if it’s happening?
What impact?
How well?
When and how should it happen?
What needs to happen?
What’s the problem?
Monitoring and evaluation
Now that we know what the approach is, we can think about how we make it happen
Active ingredients
Implementation activities
Examples:
Implementation activities
Problem
Learners not self-evaluating their work leading to missed marks and lower attainment
E.g. CPD session
Scheme of Learning created
Lesson drop-ins to monitor fidelity to the SOL
Let’s focus on implementation activities in more detail
Launching vs Implementing a strategy
Can
Annual
Training
Alone
Produce
Underpinning
Lasting
Transformation?
Fig.1 The infamous September Inset Day Strategy Launcher
A more in-depth example
Create ‘readiness’
Involve staff in ‘Beliefs and assumptions’ survey around vocabulary
Ensure literacy lead receives training / form literacy Champions Group
PD session 1 (Oct): Motivate teachers
Introduce with credible sources: EPI data & centrality of language development to tackling disadvantage.
Vocabulary as an ‘easy entry’ into wider literacy practices
EEF Literacy guidance – vocabulary recommendation
Set goal: Use RAG tool to get a picture of what’s currently happening
Teacher shares multiple examples of key vocabulary items
Students are prompted to recall the meaning in a subsequent lesson
Students use the key vocabulary in structured talk
Teacher defines key vocabulary in student-friendly terms
Active ingredients of a vocabulary strategy
Literacy Champions Group, led by literacy lead:
Review Beliefs and Assumptions RAG findings. What are the priorities? Where do we have good practice that we could use as exemplification and as a way to create a social norm?
PD Session 2 (Early Nov) :
Revisit prior learning: reiterate rationale.
Share findings from RAG survey and priorities.
Introduce SEEC model
Introduce vocab tiers as a way to action the ‘select’ part of SEEC (model use).
Create ‘readiness’
Involve staff in ‘Beliefs and assumptions’ survey around vocabulary
Ensure literacy lead receives training / form literacy Champions Group
PD session 1 (Oct): Motivate teachers
Introduce with credible sources: EPI data & centrality of language development to tackling disadvantage.
Vocabulary as an ‘easy entry’ into wider literacy practices
EEF Literacy guidance – vocabulary recommendation
Set goal: Use RAG tool to get a picture of what’s currently happening
Teacher shares multiple examples of key vocabulary items
Students are prompted to recall the meaning in a subsequent lesson
Students use the key vocabulary in structured talk
Teacher defines key vocabulary in student-friendly terms
A more in-depth example
Literacy Champions Group, led by literacy lead:
Review Beliefs and Assumptions RAG findings. What are the priorities? Where do we have good practice that we could use as exemplification and as a way to create a social norm?
PD Session 2 (Early Nov) :
Revisit prior learning: reiterate rationale.
Share findings from RAG survey and priorities.
Introduce SEEC model
Introduce vocab tiers as a way to action the ‘select’ part of SEEC (model use).
Create ‘readiness’
Involve staff in ‘Beliefs and assumptions’ survey around vocabulary
Ensure literacy lead receives training / form literacy Champions Group
PD session 1 (Oct): Motivate teachers
Introduce with credible sources: EPI data & centrality of language development to tackling disadvantage.
Vocabulary as an ‘easy entry’ into wider literacy practices
EEF Literacy guidance – vocabulary recommendation
Set goal: Use RAG tool to get a picture of what’s currently happening
Literacy Champions support departments (dept meetings)
Review vocab selection. Monitor for unhelpful practices (e.g. excessive generation of vocab lists) – does identification of ‘keystone’ vocabulary support current scheme?
Capture effective examples
Literacy Lead to provide affirmation in staff meeting – share effective practice
Twilight training (Dec)
(Develop teacher techniques) Literacy Champions model collaborative learning strategies for supporting students to use key vocabulary in structured talk.
Record sessions and extract modelled examples. Distribute as video resource.
Teacher shares multiple examples of key vocabulary items
Students are prompted to recall the meaning in a subsequent lesson
Students use the key vocabulary in structured talk
Teacher defines key vocabulary in student-friendly terms
A more in-depth example
Literacy Champions Group, led by literacy lead:
Review Beliefs and Assumptions RAG findings. What are the priorities? Where do we have good practice that we could use as exemplification and as a way to create a social norm?
Create ‘readiness’
Involve staff in ‘Beliefs and assumptions’ survey around vocabulary
Ensure literacy lead receives training / form literacy Champions Group
PD session 1 (Oct): Motivate teachers
Introduce with credible sources: EPI data & centrality of language development to tackling disadvantage.
Vocabulary as an ‘easy entry’ into wider literacy practices
EEF Literacy guidance – vocabulary recommendation
Set goal: Use RAG tool to get a picture of what’s currently happening
Literacy Champions support departments (dept meetings)
Review vocab selection. Monitor for unhelpful practices (e.g. excessive generation of vocab lists) – does identification of ‘keystone’ vocabulary support current scheme?
Capture effective examples
Literacy Lead to provide affirmation in staff meeting – share effective practice
Twilight training (Dec)
(Develop teacher techniques) Literacy Champions model collaborative learning strategies for supporting students to use key vocabulary in structured talk.
Record sessions and extract modelled examples. Distribute as video resource.
Teacher shares multiple examples of key vocabulary items
Students are prompted to recall the meaning in a subsequent lesson
Students use the key vocabulary in structured talk
Teacher defines key vocabulary in student-friendly terms
Assembly and tutorial follow up
Create readiness for students. Focus: Year 7.
Run assembly on importance of vocabulary & explain this will be a focus in upcoming lessons.
Etc.
A more in-depth example
Based on the example . . .
What implementation activities are used?
How is it different from a ‘train and pray’/ September launch approach to implementation?
Is there anything that could be removed without making much overall difference?
Would the same set of implementation activities work for a targeted intervention led by TAs?
What role does the school’s implementation climate play in all of this?
Pause to read the relevant section of the implementation guidance
Read pages 22 – 23 of the guidance.
There are some additional points about implementation activities and, on page 23, a list of implementation activities based on the ERIC framework (Expert Recommendations for Implementing Change).
Powell, B. et al. (2015). A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implementation Science. 2015; 10: p1–14.
“Typically, the application of a single strategy alone will be insufficient to successfully support the implementation of a new approach.”
Common implementation outcomes might include:
Fidelity-the degree to which staff use an intervention as intended
Acceptability-the degree to which different stakeholders – eg teachers, students, parents – perceive an intervention as agreeable
Reach-how many students are impacted
Feasibility-ease and convenience with which the approach can be used
Costs
Use implementation data….
Think about
Fidelity; how well is it being implemented?
Are the active ingredients being applied?
Reach: who is on board, who is being successful, who needs support, individuals, groups?
Acceptability: what is morale like? Feedback?
What were the two foundational recommendations?
Key points
Consider the school’s implementation climate.
Consider how the school’s implementation climate might enable or seriously inhibit the vocabulary strategy in the example.
INFLUENCES
INFLUENCES
IMPLEMENTATION CLIMATE
Realistically, what can happen?
Capacity? Competing priorities? What to stop?
What impact?
How well?
How will we know if it’s happening?
When and how should it happen?
What needs to happen?
What’s the problem?
capacity and prioritising
Key points
Support all levels of the organisation.
“When selecting implementation strategies, aim for a tailored package that supports change at different levels of the organisation—individual practitioners, departmental teams, school level changes, and so on.”
Reflect on the activities in your PP strategy
Questions about the implementation support required:
Our ongoing questions:
Are you prioritising the classroom?
Are you prioritising strategies that are likely to increase attainment?
Are your strategies guided by accurate diagnosis of need and careful use of evidence?
Signposting the EEF’s Professional Development guidance report
It’s very likely that PD will be an implementation activity.
If you are planning and designing PD (or selecting PD from an external provider), this will be a useful guidance report.
In brief, it focuses on mechanisms that research has shown tend to be good bets for changing behaviour, which is what we are trying to do with PD.
The Professional Development guidance report
You might have noticed some of these mechanisms in the vocabulary example earlier.
Making the difference for disadvantaged pupils.
Knowledge of how disadvantage impacts pupils and families - understanding the complexity of the problem.
Knowledge of what quality use of evidence looks like.
Current PP strategy reviewed. Issues around data, including the influence of bias, identified.
Knowledge of the evidence which supports the tiered approach.
Using the Tiered Approach to establish priorities for your context.
Completed PP strategy with the knowledge and tools to complete future strategies clearly established.
Processes for monitoring and self-evaluation embedded.
Improved outcomes for disadvantaged pupils.
Understanding of evidence which will inform current, and future, strategies and practice.
Establishing the active ingredients of your approach.
Understanding of the implementation process and how this affects successful decision making and outcomes. An improved implementation climate is also an outcome of effective implementation - doing this well provides a model and set of procedures for doing other things well.
Knowledge of how disadvantage impacts pupils and families - understanding the complexity of the problem.
Key points
Plan to sustain.
Typically, a lot of effort goes into launching a strategy, but not as much into sustaining it. Plan to sustain it from the beginning and think about how long a given strategy should run.
How are we doing with the PP strategy?
Caveat
Share something from another school that caught your interest.
Implementation
Share something from another school that you found interesting or useful.
1
Recap, updates from the EEF and RSN
2
Approaches to the PP strategy
Agenda
3
Implementation: successes and challenges
2
Implementation: sustain and scale
What can you say about how well it's working? How do you know? If you’re not sure, what evaluation measures can you use to find out?
Think about what you’ve introduced so far
To be, or not to be
We often introduce programmes or strategies, assuming that they will continue indefinitely.
But there are several options:
Should your approach continue indefinitely or should there be a predefined lifespan? In other words, are there circumstances under which it would no longer be necessary?
Is there a 'phase 2' to the approach? In what ways might it be built on? What's the next step?
Who is it currently reaching? In what context does it currently operate? Should it be scaled up?
To be, or not to be – discussion questions
Expected effect
Time
Programme X
Programme X
Programme X
‘Programme drift’
Unintentional shift
Active ingredients, Sept.
Active ingredients, Jan.
Possible reasons?
Expected effect
Scale
Programme X
Programme X
Programme X
‘Voltage drop’
Threats and barriers
SUPERFICIAL EVIDENCE USE
COMPLEXITY OF THE INTERVENTION
LACK OF COMMUNICATION & BUY-IN
RUSHING TO IMPLEMENT
NOT PLANNING TO SUSTAIN
NOT ESTABLISHING THE PROBLEM WE’RE AIMING TO SOLVE
(leads to resources – including the goodwill of staff – being wasted on the wrong initiatives)
(superficial understanding of the evidence is likely to lead to an ill-conceived, less effective strategy)
(a complex or poorly defined intervention is more challenging to implement)
(staff need to see this is on the leadership agenda if we expect them to get behind it)
(if we don’t plan to sustain and continually support a practice, it will be challenged by staff turnover, new priorities, contextual challenges, etc.)
(rushed implementation is unlikely to lead to improved attainment and is also damaging to a school’s implementation climate)
“. . . we reject the notion that an intervention can be optimized prior to implementation, and explicitly reject the validity of 'program drift’ and 'voltage drop.’ Rather, we suggest that the most compelling evidence on the maximal benefit of any intervention can only be realized through ongoing development, evaluation and refinement in diverse populations and systems. Instead of viewing contextual factors as interfering with the delivery of an effective intervention and needing to be controlled, we see the opportunity to learn about the optimal fit of an intervention to different care settings.”
Chambers, D.A., Glasgow, R.E. & Stange, K.C. The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implementation Sci 8, 117 (2013). https://doi.org/10.1186/1748-5908-8-117
Embrace the complexity - and learn from it
Which is to say . . .
Monitor
Support
Give it the best chance of success you can, then . . .
Plan to Sustain
What methods might you use to find out?
This is the COM-B model
Capability
Opportunity
Motivation
towards Behaviour change
A helpful model
What behaviour are we trying to change?
What influences whether this behaviour is performed or not?
Can we predict where the problems are going to be?
Where should we intervene?
A helpful model
Michie S, Atkins L, West R. (2014) The Behaviour Change Wheel: A Guide to Designing Interventions. London: Silverback Publishing. www.behaviourchangewheel.com
Where can I learn more?
Plan to Sustain – we’re effectively in a new ‘explore phase’
Plan to Sustain – we’re effectively in a new explore phase!
‘Treat scale-up of an innovation as a new implementation process.’
Time to plan!
School culture: Is there a shared responsibility for improving outcomes for disadvantaged students? Is there a shared understanding of ambitions for disadvantaged students? Does
everyone know and buy into their
role in the strategy?
Learners, not labels: Is the Pupil Premium strategy based on accurate diagnosis of pupil need, or are we acting on assumptions? Do our intended
outcomes focus on pupil learning?
Quality evidence use: Are we using evidence to inform our decisions? Do we fully understand the implications of the evidence, or are we risking surface-level compliance?
SIP, SEND, PP aligned: To what extent do our PP, SEND and School Improvement practices align? Does everyone understand this, or are PP and SEND viewed as ‘intervention issues’?
Staged implementation: Are we inhibiting our outcomes by doing too many things at once? Is implementation focused on supporting pupils to be better learners?
Improving, not proving: Do we have an ‘improvement mindset’ when it comes to monitoring and evaluation? Are we trying to ‘prove’ success or actually get better? Are we clear from the outset how we intend to evaluate our strategy?
Diagnostic assessment: Does assessment identify controllable challenges that are most preventing pupils from thriving in the classroom and attaining well?
The tiered approach: Does activity within the tiered model (teaching and learning, academic intervention and wider approaches) specifically address the challenges identified through assessment and observation?