1 of 57

Greetings!

Dr. Matt Turkie

District and School Support Liaison

mturkie@montereycoe.org

1

2 of 57

Learning Goals

To build a common language and process to engage in continuous improvement and Plan, Do, Study, Act (PDSA) cycles
To become familiar with improvement science

3 of 57

Agenda

Defining continuous improvement and improvement science
An example of a school’s improvement journey
Plans for learning and doing for 22-23

4 of 57

Defining continuous improvement and improvement science

1

5 of 57

Improvement science

Data driven decision making

Continuous improvement

Cycles of improvement

Policy Analysis for California Education

6 of 57

Team Sharing Time

5 mins - How have you encountered continuous improvement in your current or prior role?

7 of 57

“Every system is perfectly designed to achieve exactly the results it gets.”

8 of 57

Why are we engaging in Continuous Improvement and PDSA cycles?

Continuous improvement is a methodical, data based approach to get better results.

Situates problems of practice as products of a system
Is worked on regularly and consistently
Is fully infused into the day-to-day of individuals who are closest to the work

9 of 57

Why are we engaging in Continuous Improvement and PDSA cycles?

Some compelling reasons:

To get better at getting better
To improve outcomes for all students, especially those who are disproportionately being underserved in our systems
To ensure that all students graduate college, career and community ready

10 of 57

How do we get to effective PDSA cycles?

10

A (brief) introduction to improvement science

11 of 57

Improvement Science is….

A disciplined approach of continuous improvement, in which teams make use of PDSA cycles to make changes and collectively learn their way into stronger system performance.

12 of 57

Improvement Science Framework

12

Brief overview of 3 questions as pre-requisites to engaging in PDSA cycles.

Very flexible framework for improving anything. This framework works with any content at any level. When we become a true continuous improvement organization, we will be applying this to everything we do.

The more specific we are with our aim, the more likely we are to impact and see success. However, our SPSA goals are often broad by necessity. It can be useful to have 2 types of aims - large and small. E.g. Improve CAASPP scores in math in 8th grade specifically for our EL, SWD, H/FY…What is the root cause of this problem?.....Perhaps our EL, SWD, H/FY students struggle to persist in solving challenging math tasks…….Improve the ability of our EL, SWD, H/FY students to persist in solving challenging math tasks in class.
This is the team’s best thinking around what changes might lead to the most improvement. Grounded in logic and evidence based best practice. For academic improvement, these changes must be focused on instruction. E.g. Increase the use of student to student academic discourse concerning challenging math tasks by using specific instructional routines such as 3 act math tasks or 3 reads.
Different types of measures: PDSA measures are very short term, and could process measures such as student perception data on how much math talk they engaged in, or measures such as an exit ticket or a rubric score on a challenging math task. On interim assessments we could also look at claim data - students ability to explain their reasoning and critique the reasoning of others. Then we can also look at our outcome data for our small aim and big aim. The point is not to wait until the end of the year to know how we are doing and what changes we need to make.

13 of 57

Improvement Science Framework

Experimentation and learning!

13

14 of 57

How does this fit with classroom practice and PLC’s?

14

What do we want our students to know and be able to do?
How will we know if they learn it?
How will we respond when some students don’t learn it?
How will we extend the learning for students who are already proficient?

15 of 57

How does this fit with classroom practice and PLC’s?

15

What do we want our students to know and be able to do?
How will we know if they learn it?

How will we respond when some students don’t learn it?
How will we extend the learning for students who are already proficient?

How will we teach it?

What changes might we make in the classroom?

How will we know the changes are effective?

16 of 57

An example of a school’s improvement journey

2

17 of 57

The school’s problem focus area:

18 of 57

The school’s problem focus area:

Only 33% of 8th grade students met standards in math, and significant disproportionality exists for EL, SWD, H/FY

19 of 57

The school’s SPSA goal

8th grade students will increase CAASPP 3’s and 4’s in math from 33% to 37%

20 of 57

The school’s SPSA goal

8th grade students will increase CAASPP 3’s and 4’s in math from 33% to 37%

EL students will improve from 20% - 26%
SWD will improve from 10% - 17%
Homeless students will improve from 10% - 17%

21 of 57

The improvement team is identified...

Principal
Assistant principal
Math 8 course lead
Math specialist
EL specialist
Ed tech specialist
All math 8 teachers
District math specialist
Instructional coach for SWD

22 of 57

The improvement team is identified...

Principal
Assistant principal
Math 8 course lead
Math specialist
EL specialist
Ed tech specialist
All math 8 teachers
District math specialist
Instructional coach for SWD

ILT is adjusted to include course lead

23 of 57

The improvement journey begins…...

Understanding the problem and the system that produces it

24 of 57

24

Low achievement of students (EL, H/FY, SWD) in 8th grade math

25 of 57

25

Lack of prepared math teachers

Students struggle explaining their thinking

Home environment

Poor culture / mindset

Professional learning isn’t meeting teacher needs

Low achievement of students (EL, H/FY, SWD) in 8th grade math

Students struggle with challenging math problems

Lots of long term math subs

Lack of formative feedback given to math teachers

New teachers don’t seem prepared

High math teacher turnover

“Lecture style” math teaching

Math materials don’t include instructional practice guidance

Lack of PL focused on student discourse

Lack of opportunities for academic discourse

Many students do not get the support they need at home

Multiple family households

Students have a lot of trauma to deal with

Students lack mathematical mindset

Students do not all feel they belong in math classes

Students dislike math

PL is not embedded into the school day

Not connected to student learning

PL not differentiated by needs of individual teachers

No PL on integrated ELD for math teachers

Students give up easily

Materials lack challenging tasks

Students lack practice with rigorous tasks

26 of 57

Conducting empathy interviews

Students
Parents
Teachers
Site administrators
Math specialist

???
???
???

27 of 57

Conducting empathy interviews

Students
Parents
Teachers
Site administrators
Math specialist

Tell me about a time when you felt successful in math
Tell me about a time when math was hard
How do you learn math best?

28 of 57

Consolidating the team’s learning

Some common themes were identified:

Many students struggled with challenging math problems on tests
Math problems in class were easier than on tests
Students learned math best when they could talk about it with their peers
Students had few opportunities to engage in student-student academic discourse

29 of 57

29

Lack of prepared math teachers

Students struggle explaining their thinking

Home environment

Poor culture / mindset

Professional learning isn’t meeting teacher needs

Low achievement of students (EL, H/FY, SWD) in 8th grade math

Students struggle with challenging math problems

Lots of long term math subs

Lack of formative feedback given to math teachers

New teachers don’t seem prepared

High math teacher turnover

“Lecture style” math teaching

Math materials don’t include instructional practice guidance

Lack of PL focused on student discourse

Lack of opportunities for academic discourse

Many students do not get the support they need at home

Multiple family households

Students have a lot of trauma to deal with

Students lack mathematical mindset

Students do not all feel they belong in math classes

Students dislike math

PL is not embedded into the school day

Not connected to student learning

PL not differentiated by needs of individual teachers

No PL on integrated ELD for math teachers

Students give up easily

Materials lack challenging tasks

Students lack practice with rigorous tasks

30 of 57

Example Challenges and Aims

Equity Challenge (broad): Low achievement of SWDs in Math	Big Dot Aim (Summative SPSA goal): By May 2024, increase meet and exceeds on District Math Assessment. All - 33% to 37%; SWDs - 10% to 17%

31 of 57

Example Challenges and Aims

Equity Challenge (broad): Low achievement of SWDs in Math	Big Dot Aim (Summative SPSA goal): By May 2024, increase meet and exceeds on District Math Assessment. All - 33% to 37%; SWDs - 10% to 17%
Problem of Practice (specific): SWDs lack opportunities to meaningfully engage in math and as such struggle with challenging math problems in class

32 of 57

Example Challenges and Aims

Equity Challenge (broad): Low achievement of SWDs in Math	Big Dot Aim (Summative SPSA goal): By May 2024, increase meet and exceeds on District Math Assessment. All - 33% to 37%; SWDs - 10% to 17%
Problem of Practice (specific): SWDs lack opportunities to meaningfully engage in math and as such struggle with challenging math problems in class	Small Dot Aim / Aim Statement (Formative classroom goal): By May 2024, increase the percentage of students who score at least a 2 out of 3 on challenging math tasks in class. All - 40% to 75%; SWDs - 20% to 60%;

33 of 57

Problem statements and aims

Problem focus area (broad): Low achievement of EL, SWD, H/FY in 8th grade math	Big dot aim: CASSPP 3’s and 4’s would increase from 33% to 37%, with respectively higher gains for EL, SWD, H/FY

34 of 57

Problem statements and aims

Problem focus area (broad): Low achievement of EL, SWD, H/FY in 8th grade math	Big dot aim: CASSPP 3’s and 4’s would increase from 33% to 37%, with respectively higher gains for EL, SWD, H/FY
Problem statement (specific): Students struggle attempting and persisting with challenging math problems	Small dot aim: Increase the percentage of students who score at least a 2 out of 3 on challenging math tasks in class from 10% to 60%. Specific focus on EL, SWD, H/FY

35 of 57

Processing Time - 5 mins

What are some things you like about the process so far?

What are some things you wonder about the process so far?

36 of 57

Next step in the improvement journey

2. Creating a theory of improvement and developing measures

37 of 57

Aim

What specifically are we trying to accomplish?

What changes might we make?

Change Idea

38 of 57

Change Idea

Aim

Driver

If I do …

...that will impact...

...which in turn will….

39 of 57

Visualizing our theory of improvement through

a Driver Diagram

Aim

Driver

Change Idea

40 of 57

Zooming in: Theory of Improvement and Measures

Small dot aim:

Increase % of students who can solve challenging math tasks

Big dot aim:

Increase in CAASPP scores

Increase in students being able to explain their reasoning. Data collected during monthly CFA’s.

Give students opportunities to engage in rich tasks and vastly increase opportunities for academic discourse via evidence based instructional routines

41 of 57

Academic Example

Math Talk

Growth mindset

Talk cards

Word walls

Academic discourse protocols

Jo Boaler’s week of inspirational math

By May 2024, increase % of students who score a 2 out of 3 on challenging math tasks in class.

All: 40-75%

SWEN: 20-60%

42 of 57

Classroom Measurement Example

Math Talk

Academic Discourse Protocols & Talk Cards

Aim

If ….We give students rich tasks with academic discourse protocols and talk cards (3 reads protocol and talk cards. At least 2 times a week)

M = Weekly exit ticket / CFA (outcome measure, same as small dot aim)

Then….There will be an increase in student math talk.

M = Student survey

M = Student opinions

M = Teacher opinions

(Process measures)

Which will lead to….

M = Small dot aim:

By May 2024, increase the percentage of students who score at least a 2 out of 3 on challenging math tasks in class.

All - 40% to 75%; SWD -20% to 60%

M = Big dot aim:

By May 2024, increase in district assessment 3 and 4’s

All - 33% to 37%;

SWD - 10% to 17%

M

43 of 57

Processing Time - 5 mins

What are some things you like about this part of the process?

What are some things you wonder about this part of the process?

44 of 57

Next step in the improvement journey

3. Testing changes through Plan Do Study Act (PDSA) inquiry cycles

45 of 57

Effective PDSA Cycles

46 of 57

Specific change idea

Instructional routines to use with rich tasks to give students opportunities to practice mathematical reasoning and academic discourse:

Three act math tasks
Three reads
Which one doesn’t belong?
Open middle math
Math talks

Each week, the team decided which routine(s) and task(s) they would all try with their students and what data they would collect.

47 of 57

PDSA data collection

Student self reported measures on how much ‘math talk’ they engaged in with their peers
Productive groupwork measures from GRR rubric via peer/admin observation
3 point rubric to assess students’ mathematical reasoning for challenging math tasks

48 of 57

Using evidence to learn to improve

49 of 57

Improvement Journey

(What it really feels like)

50 of 57

Processing Time - 5 mins

What are some things you like about this part of the process?

What are some things you wonder about this part of the process?

Processing Time - 5 mins

What are some big takeaways that you have from the example of an improvement journey?

51 of 57

Plans for learning and doing

2022-23

3

52 of 57

52

53 of 57

Main Points for 22-23

Minimum expansion is one more content area teacher team. Aim is for teams to start PDSA cycles by 6th week of school.
Focus of classroom PDSAs is to improve at some part of the instructional model and to improve outcomes for ELs, SWDs and YiT.
Schools can expand the work in different ways. The district will support schools wherever they are at with implementation.
Teacher teams/schools determine their own PoP, and how to improve it.
PoP + change ideas can be shared between teams or across a school.
Professional learning & collaboration time is determined at each school.
Change ideas need to be linked to aspects of the instructional model.

53

54 of 57

Main Points for 22-23

Support will be given to improve how PLCs operate (details to come).
District/principal learning walks will continue to support and inform the continuous improvement work
Professional learning and support for ILTs on continuous improvement will continue throughout the year.
All directors will engage in the process to improve aspects of their departments.
Teams will be asked to keep the details of their improvement projects up to date in the Improvement Project spreadsheet (same as this year)

54

55 of 57

55

Cohorts	Schools	Fall ILT Dates	Spring ILT Dates
Cohort A	Alisal High School Salinas High School Everett Alvarez High School ROP	Sept 6 Oct 11 Nov 29	Jan 20 Feb 21 Mar 20
Cohort B	Rancho San Juan High School North Salinas High School El Puente School Mount Toro High School	Sept 7 Oct 12 Nov 30	Jan 25 Feb 22 Mar 22
Cohort C	Harden Middle School La Paz Middle School El Sausal Middle School Washington Middle School	Sept 8 Oct 13 Dec 1	Jan 27 Feb 23 Mar 23

56 of 57

Processing Time - 5 mins

Questions?

What should your next steps be?

57 of 57

Thanks!

57