1 of 37

MTSS Data-Based Decision Making:

Strategies for Systems-level Problem Solving

�DE-MTSS Technical Assistance Center

2 of 37

DE-MTSS Technical Assistance Center

The Delaware Multi-Tiered System of Support TA Center proudly serves as a technical assistance provider for the

Delaware Department of Education.

Our TA Center provides professional learning and coaching to support the academic and nonacademic development of all children.

3 of 37

Transformational Processes Standard

Transformational Processes Standard: IMPLEMENTATION

Professional learning results in equitable and excellent outcomes for all students when:

Educators understand and apply research on change management,
Educators engage in feedback processes, and
Educators implement and sustain professional learning.

(Learning Forward, 2023)

Learn more about the Professional Learning Standards here and the Delaware Professional Development Standards guidance here.

4 of 37

Cascade of MTSS Implementation to

Support the Whole Child

State

District

School

Classroom

Improved Student Outcomes

Delaware multi-tiered system of support. (2021).

5 of 37

Defining MTSS in Delaware

DE-MTSS is a framework designed to meet the needs of the whole child through an integrated, multilevel prevention system that optimizes team-based leadership and data-driven decision making to meet the academic and nonacademic needs of all students. High quality core academic instruction and non-academic practices are provided as universal supports to all children. Evidence-based intervention and supports are matched to student needs and informed by ongoing progress monitoring and additional formative assessments.

6 of 37

When it comes to using data for solving systems-level problems with your teams, which cat do you identify with?

Introduce yourself (name, district and role)

Identify the number that reflects your feelings about using data to solve systems level problems with your teams.

Using the chat feature….

7 of 37

Learning Invitations & Session Agreements

During this professional learning opportunity, Delaware administrators will:

Apply a six-step process for solving systems-level problems using integrated data and generate innovative solutions in teams

Examine strategies for monitoring fidelity of data-based decision-making processes used by teams.

Be present and engaged in discussions and activities.
Share insights, questions and experiences openly
Maintain confidentiality when discussing specific school/district challenges.
Approach the session with a growth mindset.

8 of 37

Let’s Recap: DE-MTSS Implementation Teams

Delaware multi-tiered system of support. (2021).

9 of 37

Foundational Elements of Data-Based Decisions:

Focused Data Systems vs. Team Focused Decision Making

10 of 37

Part I.

Strategies for Solving Systems-Level Problems

11 of 37

Driving Question #1

What do highly effective leaders do to ensure consistent and effective implementation of the Data-Based Decision Making process to improve student outcomes and school effectiveness?

12 of 37

Grounding Our Work

Reflecting on your own leadership role, what practices do you currently use to prioritize data-based decision making in your building/district to support your students’ academic and social-emotional success?

The Key to Educational Improvement: Data and How We Use It by Jeff Edmondson

(Edmondson, 2010)

13 of 37

Tier 1: Core/Universal

Core classroom curriculum, grade-level content

Tier 2: Targeted

Specific, targeted instruction based on individual or small group needs

Tier 3: Intensive

Specially designed instruction based on individual student need

Tier 1: Core/Universal

Access to healthcare, physicals, screenings, vitamins and supplements

Tier 2: Targeted

Medication, specific diets, specific tests

Tier 3: Intensive

Specialists, more specialized care, surgeons

all

some

few

Continuum of Decision-Making

14 of 37

Common Data-Based Decision-Making Models

15 of 37

The Ladder of Inference & Data-Based Decision Making

Resource Alert (for MTSS Teams)!!!!

The Ladder of Inference: Why we jump to conclusions (and how to avoid it) - Synergy Commons

The Blind Spot Activity

TedEd Rethinking Thinking by Trevor Mader

16 of 37

Assessment data informs two types of team-based problem solving conversations:

Systems Problem Solving

Across all tiers, to guide the selection and implementation of core instruction and evidence-based intervention

Student Problem Solving

Across all tiers, to understand student responsiveness and determine the adaptation of intervention for students requiring additional supports

17 of 37

Driving Question #2

How do leaders effectively guide and support teams’ systems level problem solving efforts within the framework of DE-MTSS to ensure equitable outcomes for all students?

18 of 37

POP & Team-Based Problem-Solving

Discuss:

In collaborative teams, review the behavior/academic POP. Discuss how would you approach a problem solving discussion with members of your school/district team so that you are successful in improving the known system area need.

Consider the following:

What solutions-based strategies you might consider for implementation?
What are the desired outcomes?
What resources and supports need to be implemented?
How will you know if you have achieved your desired outcome(s)?

19 of 37

A Systems Level Problem of Practice

Academic:

Only 41% of 5th grade students earned a reading composite score that placed them at or above benchmark. This trend was observed across all 5th grade classrooms, and not unique to a few. Student performance did not vary significantly based on subgroup membership (e.g, race/ethnicity, english proficiency, and disability status). After looking at subtest data, they noted that only 46% of the 5th grade students scored at or above benchmark on the ORF. This was concerning, because the teachers knew that students’ lack of ready accuracy would adversely impact their ability to comprehend what they are reading. After discussing and reviewing their lesson plans, the teachers determine that reading accuracy is below expectation because sufficient instruction on word attack strategies is not occuring. They also confirm that students do not understand that fluent oral reading includes reading quickly, accurately and with expression (adapted from: Problem Solving/Response to Intervention Project, n.d.).

Behavior:

The data coordinator identifies that black students are overrepresented in ODRs. A total of 73% of black students had 2-3 or more ODR last year, compared to 92% of white students. Before the next problem solving team meeting, the data coordinator assesses ODRs by problem behavior, location, and time of day and finds that black students are more likely to receive ODRs for disrespect and more likely to receive these ODRs in the bus area in the afternoon (adapted from: McIntosh & Goodman, 2016).

20 of 37

Team Based Six Step Problem Solving Process

Step 3: Solution Planning

Determine possible strategies that are feasible for your team to implement
Select effective interventions and develop a clear plan for implementation

Step 1: Problem Identification

Determine if a problem exists
Define the problem as precisely as possible

Step 2: Problem Clarification

Evaluate the problem in greater detail and determine why it occurs
Identify factors that may cause or maintain the problem

Step 6: Evaluation

Determine if the problem persists
Use the information gained from this cycle for future problem solving
Begin this process again and apply new information if the problem persists

Step 4: Goal Setting

Develop a clear goal with a measurable outcome
Determine a date for reaching the goal and a plan for assessing progress along the way

Step 5: Implementation

Provide training and resources to effectively implement the intervention
Monitor the fidelity of implementation

Adapted from McIntosh & Goodman, 2016

21 of 37

Problem Solving Process in Action

Problem: Racial/ethnic disproportionality in ODRs
Step 1: Problem Identification: Determined if a problem exists. Define the problem as precisely as possible Guiding question(s): What is the observed discrepancy between our desired outcome and actual outcomes? The data coordinator identifies that black students are overrepresented in ODRs. A total of 73% of black students had 0 to 1 ODR last year, compared to 92% of white students.
Step 2: Problem Clarification: Analyze the problem to determine why the issue is occurring. Generate a hypothesis (reasons why students are not meeting performance goals). Gather information Guiding question(s): What are the known barriers to the desired outcomes? Before the next meeting, the data coordinator assesses ODRs by problem behavior, location, and time of day and finds that black students are more likely to receive ODRs for disrespect and more likely to receive these ODRs in the bus area in the afternoon.
Step 3: Solution Planning: Determine possible strategies that are feasible for your team to implement. Guiding question(s): What are possible strategies? Are the strategies feasible and likely to be effective? Who will implement specific components, by when, and how will you monitor its effectiveness over time? After brainstorming solutions, the team decides to (a) revisit ODR definitions for disrespect, (b) review instructional approaches to problem behavior, (c) reteach expectations for bus area; (d) increase use of acknowledgement tickets, ensuring equity across racial groups, (e) monitor ODRs for improvement by assessing risk ratios for black students.

Racial/ethnic disproportionality in ODRs
Step 4: Goal Setting: Develop a clear goal with a measurable outcome. Determine a date for reaching the goal and a plan for assessing progress along the way. Guiding Question(s): What does success look like? When do you expect to see the problem resolved? How will you measure progress toward your goal? The team notes a current risk ratio of 2.85 and sets a goal to reduce it to below 1.25 by the end of the school year.
Step 5: Intervention Implementation: Provide training and resources to effectively implement the intervention. Monitor the fidelity of implementation. Guiding Question(s): What is your plan to support staff to implement the solutions? Do the solution implementers have necessary resources?How will you measure the fidelity of staff implementation? The team plans to review ODRs, instructional responses to behavior, and bus expectations at the next staff meeting; reteach expectations to students in the next week; and observe implementation of responses to prosocial and problem behavior at the bus area. The team creates a quick checklist to assess completion.
Step 6: Evaluation: Determine if the problem persists. Use information gained from this cycle for future problem solving. Begin this process again and apply new information if the problem persists. Guiding Question(s): What is your plan to review progress toward your goal? Is the plan having the desired impact? Is the plan being implemented with fidelity? Does the original problem still exist? If the problem still exists, how will you modify your plan with information learned? The team plans to revisit its plan and observe outcomes each month until the goal is met, revising plans if no decrease is seen in the risk ratio each month.

22 of 37

Pause and Reflect: 6 Step Problem Solving Process

After completing the data-based decision making activity, choose one sentence stem below to reflect on your problem of practice discussion.

The 6 step data-based decision making process is different from what we normally do because…

If we engage in this data-based process, this would lead us to improve…

In order to integrate principles like systems thinking, continuous improvement, and equity into our decision-making processes, our team needs…

23 of 37

Putting It All Together

Using problem solving processes fosters collaborative decision making grounded in data.

It empowers teams to address challenges and drive informed actions based on evidence, not assumptions.

Promotes learning and growth.

24 of 37

Walkaway Thought

25 of 37

Processing Point

Temperature of the Virtual Room:�

Share an emoji in the chat to let us know how you are feeling about the content so far.

26 of 37

Part II.

Supporting Fidelity of the DBDM Process in Schools

27 of 37

Driving Question #3

How can leaders ensure consistent and effective implementation of the Data-Based Decision Making process to improve student outcomes and school effectiveness?

28 of 37

DBDM Process Monitoring - Why?

Data from monitoring can be used to :

Highlight what’s going well
Identify model teaming processes
Indicate need for revisiting the process overall or particular steps
Connect to team reflection of efficiency & efficacy

29 of 37

DBDM Process Monitoring - How? Who?

Process monitoring strategies may include:

Set expectations for process use across teams & model
Utilize a checklist/reflection tool
Adapt checklist/tool to align with current DBDM Process steps
Plan a timeline with expectations as LEA or building to utilize tool across teams & how to capture data
Utilize: District coaches, building admin, team leads, team

31 of 37

Walkaway Thought

“That which is measured improves. That which is measured and reported improves exponentially.”

— Karl Pearson

32 of 37

Join our upcoming session!

MTSS Data-Based Decision Making - Strategies for student-level problem solving

May 22nd 9:30-10:30 am

Session will provide administrators with guidance to identify problems, goals for improvement, and solutions with a focus on student-level data.

33 of 37

Thank you for being here!

Please complete this brief evaluation so we can plan for our next session with your feedback in mind.

34 of 37

Thank YOU

for your ongoing engagement in implementing equity-grounded MTSS to support all students, staff, & families

35 of 37

DE-MTSS Resources and Supports

MTSS Implementation Guide
3 Quick Reference Guides – District, Building/School, Classroom
5 Essential Component One Pagers

36 of 37

References & Additional Resources

DE-MTSS TA Center:

DE Access Project - https://www.deaccessproject.org/
DE Positive Behavior Support Project - https://www.delawarepbs.org/

DE-MTSS (DDOE site)
Delaware MTSS regulation 508
Digital DE
High Quality Instructional Materials (DDOE site)
SWiFT Education Center - https://swiftschools.org/
Michigan MTSS TA Center - https://mimtsstac.org/

37 of 37

Session Specific References

Edmondson, J. (2010, November 29). TEDxCincy - The key to educational improvement: Data and how we use it. YouTube.

https://youtu.be/FLqc_9VxfCE?si=lYHi8usM_UhQfvkf

McIntosh, K., & Goodman, S. (2016). Integrated multi-tiered systems of support: Blending RTI and PBIS. Guilford Press.
Problem Solving/Response to Intervention Project. (n.d.) Tier 1 problem solving course case study handout. Florida Department of Education and the University of South Florida.