1 of 50

Embedding SEL Data in Practice

Hawaii State Department of Education + Panorama Education

March 23 & 24, 2022

2 of 50

Welcome!

Please log in to your Panorama Reports

Step 1: Open a web browser and go to

https://secure.panoramaed.com

Step 2: Click “Sign in with Google”

and use your [employeeID]@k12.hi.us

2

Closed captions are available!�Click “Closed Captions” then click “Show Subtitles”

3 of 50

Teaching & Learning

@PanoramaEd

About Panorama Education

Our Mission: Radically improve education for every student
Our Team: Educators, software developers, designers, and researchers
Our Community: 12 million students in 21,000 schools in 1500+ school districts across the country!

Geoffrey Gaurano

He/him

Jason Ernest Feldman

He/him

4 of 50

4

Who We Heard From

7

Topics

~90,000

Students

404

Schools

18 Complex

Areas

5 of 50

5

Hawaii Department of Education

Panorama Education is proud to support educators in the Hawaii Department of Education

Student Voice

Student Perception Surveys

MTSS Data Platform

Student Success

Student Voice

Social Emotional Learning Surveys

PD | Strategies

Panorama’s Playbook

6 of 50

Student Perception (EES) Social Emotional Learning (SEL)

Topics

Pedagogical Effectiveness
Classroom Teacher-Student Relationships
Classroom Climate
Classroom Rigorous Expectations
Classroom Engagement
Valuing of School
School Belonging
School Quality Survey - Safety Dimension
School Safety

Topics

Sense of Belonging
Self-Management
Social Awareness
Grit
Self Efficacy
Growth Mindset
Emotion Regulation

Today’s Workshop

Recordings available on resource hub.

7 of 50

7

Measuring What Matters

Social and Emotional Learning (SEL) Data is...

A tool for reflection
Formative
Confidential
Continuous conversation

And so, as we listen t student voice, and as we go through the data I also just want to remind us that social emotional learning data is meant to be a flashlight and not a hammer. 

In other words, the data is not intended to draw attention to the negative; the data is meant to serve as a resource. We want to shine a light on what’s working while also understanding how we can collectively adjust our systems to support students.

Today, we're going to get a chance to go ahead and look at some our own data

 And so, as a reminder, as we explore topics and go deeper we are going to see comparisons against our complex, state, or nation

We will see data that tells us there are certain small groups in our Community that feel like their needs are not being met as much as others. 

 What I want to remind us is this is not intended to draw attention on the negative

But rather that this valuable info is a resource and evidence to start a conversation with your teams and stakeholders

https://www.panoramaed.com/blog/san-francisco-unified-school-district

8 of 50

8

Step 1: Explore Topics	Step 2: Benchmarks	Step 3: Explore Questions	Step 4: Dig Deeper
From this school-level view, identify how students respond across topics Which topic have increased and/or decreased since the last survey window?	Compare your school/district with other school/districts across topics Across topics, how does your school/complex compare?	Choose a topic that you want to explore further What questions can be highlighted as a bright spot and leveraged to address needs?	Choose the Groups page Where do you notice gaps in experience among different groups of students?

9 of 50

Zoom Norms / Virtual Engagement

9

Share your responses in the chat

Stay muted

Video on or off

Send your message to “Everyone”

10 of 50

Self Awareness Check-In:

Feeling Wheel

Use the chat:

Which 2 words best represent your current feelings?

11 of 50

CASEL: 5 Competencies

Who am I?

What do I need?

Where do I fit in?

How do I connect with others?

How can I make better choices now that I know myself and others?

12 of 50

How does Panorama align with CASEL’s Core Competencies?

13 of 50

Sense of Belonging	Grit
Self-Management	Growth Mindset
Social Awareness	Self-Efficacy
Emotion Regulation

What We Measured

14 of 50

What is most associated with failing grades?

15 of 50

Hawaii MTSS Blueprint

What formal and informal interventions are we implementing universally for SEL?

How do we know if they’re working?

All Students

Supplemental Support

Intensive Support

And this connects back to our blue prints for multi-tiered systems of support.

For the actions we take when we know what areas of SEL we need to support our students in, and how intensive and targeted those supports need to be

And so I really want to continue this theme of looking at our SEL interventions but also what are we doing for all of our students. 

Again, the hope for today’s data analysis is that you through these things will be able to better understand your students, their needs, and make a recommendation about the appropriate level of support needed. 

What are the identified needs?
What evidence confirms the needs?
What are our goals?
How will the goals be obtained? What specific steps will be taken to reach the goals? Who is responsible? When? How will they be measured?
By when will the goals be accomplished?

16 of 50

16

Hawaii DOE Celebrations

Self-Efficacy was the greatest increase since Fall 2021 - SEL data in grades 3-5.

More of our 6-12 graders are “quite” and “extremely” confident they can understand complicated ideas when discussed in class.

More of our 3-5 graders are “quite” and “extremely” sure they can understand complicated ideas when discussed in class.

17 of 50

17

Reflecting

Prioritizing

Strategizing

Action and communication planning.

Use a prioritization matrix to determine which areas of focus will have the impact and feasibility.

Our Time Together

3

2

1

Review your data to reflect on areas of strength, areas of growth, and areas that have demonstrated improvement.

18 of 50

Chat Out: What do you notice & wonder?

I notice…. I wonder…

19 of 50

Ladder of Inference

Taking action

Collecting data

I filter relevant data

I add context

I make assumptions

I check subjectivities

I update my beliefs

I draw conclusions

The purpose of that exercise was to take note of how our experiences cause us to draw conclusion by filtering evidence or making quick assumptions

[Using the Ladder of Inference] https://academy.panoramaed.com/article/658-using-the-ladder-of-inference-to-view-data

Popularized in education by Peter Senge in his book the Fifth Discipline, the ladder of inference is an effective mental model for understanding and analyzing data. �

Our brains are hardwired to process information, observe the world around us, and take action on it. And sometimes it is important that we can rely on our brains to turn that observation into action really quickly for example when I am driving in my car and I see the car in front of me put their brake lights on I want to trust that I am able to put my brake lights on as soon as I see that light. I don’t want to spend 10 minutes weighing my options about what is the right move. However, when we are analyzing student, educator and family data, we can be doing a huge disservice by turning observation into action really quickly. And so we use this Ladder of Inference framework as a mental model to be mindful of our thought process as we are looking at the data because we know it is natural for us to infuse our own context, assumptions and biases into the equation for observing information and deciding what to do with it. We want to be mindful of when this is happening because it can be harmful when we are making decisions based on bias and assumptions and not naming where they are coming into the equation and the impact they can have on students, educators and families’ lived experiences.

Starting at the bottom of the ladder, we have reality and facts. From there, we:

Experience these selectively based on our beliefs and prior experience.
Interpret what they mean.
Apply our existing assumptions, sometimes without considering them.
Draw conclusions based on the interpreted facts and our assumptions.
Develop beliefs based on these conclusions.
Take actions that seem "right" because they are based on what we believe.

As humans, we come into contact with data all the time. Because of this, our brains are hard-wired to help us take in data and make quick conclusions to help keep us safe. That same impulse, when looking at data that doesn’t require a quick conclusion for safety, leads us to introducing bias into the data analysis process (research has shown you’re more likely to find data that affirms what you already concluded was true). The key here is to walk slowly up the ladder of inference, spending more time observing the data and considering alternative explanations in the data before deciding what it means and choosing a course of action.

If we get to the point where the teacher doesn’t believe the student is going to pass, he/she might act on that belief in ways that lead to it becoming unlikely that the student will pass.

If we as educators start to make assumptions too quickly, we can compromise our ability to make decisions that are truly in our students’ best interests.

What’s the critical step that this teacher makes in moving from noticing to acting? Where is the teacher making assumptions, potentially unwarranted?

20 of 50

reflexive loop

Taking action

Collecting data

I filter relevant data

I add context

I make assumptions

I check subjectivities

I update my beliefs

I draw conclusions

How do my life and personal experiences impact my conclusions?
What cognitive biases or subjectivities might I hold?
What assumptions did I make?
What context am I missing?
Is this relevant data? Why did I filter data this way?

Our brains are hardwired to process information, observe the world around us, and take action on it. It’s important that we can rely on our brains to do this…

But are so many intermediary steps between collecting data and drawing conclusions

We want to be mindful of when this is happening because we can counter these assumptions by going back to the evidence

And so I want to empower each of you to really challenge your conclusions, hitting the brakes, and going back to the data as evidence and doing some reflection by simply noticing or wondering…

[Using the Ladder of Inference] https://academy.panoramaed.com/article/658-using-the-ladder-of-inference-to-view-data

Popularized in education by Peter Senge in his book the Fifth Discipline, the ladder of inference is an effective mental model for understanding and analyzing data. �

Our brains are hardwired to process information, observe the world around us, and take action on it. And sometimes it is important that we can rely on our brains to turn that observation into action really quickly for example when I am driving in my car and I see the car in front of me put their brake lights on I want to trust that I am able to put my brake lights on as soon as I see that light. I don’t want to spend 10 minutes weighing my options about what is the right move. However, when we are analyzing student, educator and family data, we can be doing a huge disservice by turning observation into action really quickly. And so we use this Ladder of Inference framework as a mental model to be mindful of our thought process as we are looking at the data because we know it is natural for us to infuse our own context, assumptions and biases into the equation for observing information and deciding what to do with it. We want to be mindful of when this is happening because it can be harmful when we are making decisions based on bias and assumptions and not naming where they are coming into the equation and the impact they can have on students, educators and families’ lived experiences.

Starting at the bottom of the ladder, we have reality and facts. From there, we:

Experience these selectively based on our beliefs and prior experience.
Interpret what they mean.
Apply our existing assumptions, sometimes without considering them.
Draw conclusions based on the interpreted facts and our assumptions.
Develop beliefs based on these conclusions.
Take actions that seem "right" because they are based on what we believe.

As humans, we come into contact with data all the time. Because of this, our brains are hard-wired to help us take in data and make quick conclusions to help keep us safe. That same impulse, when looking at data that doesn’t require a quick conclusion for safety, leads us to introducing bias into the data analysis process (research has shown you’re more likely to find data that affirms what you already concluded was true). The key here is to walk slowly up the ladder of inference, spending more time observing the data and considering alternative explanations in the data before deciding what it means and choosing a course of action.

If we get to the point where the teacher doesn’t believe the student is going to pass, he/she might act on that belief in ways that lead to it becoming unlikely that the student will pass.

If we as educators start to make assumptions too quickly, we can compromise our ability to make decisions that are truly in our students’ best interests.

What’s the critical step that this teacher makes in moving from noticing to acting? Where is the teacher making assumptions, potentially unwarranted?

21 of 50

21

Reflecting

Prioritizing

Strategizing

Action and communication planning.

Use a prioritization matrix to determine which areas of focus will have the impact and feasibility.

Our Time Together

3

2

1

Review your data to reflect on areas of strength, areas of growth, and areas that have demonstrated improvement.

22 of 50

Today’s Resource: Slide Deck

Click the link in the chat or go to:

https://pan-ed.com/hi3slides

23 of 50

Today’s Resource

Workbook

Force-copy via Google Docs

https://pan-ed.com/hi3workbook

24 of 50

Hawaii DOE Resource Hub

25 of 50

Live Demo: School Summary Data in Surveys

How to view change-over-time and comparison data
How to view questions
How to view responses by group

Student SEL Surveys grades 3-5 and 6-12
Percent favorable: the percent of students who responded favorably to the questions in the topic
“Compared to others nationally:” benchmarks your topic score across pano’s data set of schools and districts, showing whether your topic scores are above or below average as compared to other schools & districts. Each topic may have a different range, which may account for the differences of your percentile ranking between topics

Elementary (use as an example):

Highlight:

Emotion regulation lowest percent favorable, high percentile ranking, and one of the highest increases (+3) since the fall
Growth mindset, low percent favorable, above average nationally, with a slight decrease from the fall
Self-efficacy, below average nationally, but greatest increase since the fall

Click into self efficacy:

Change over time: tends to increase from the Fall
Change the dataset to just elementary schools, this topic is within the 48th percentile
Student groups: active ELL students are almost 10 percentage points below students who are not (bookmark to follow up)
Questions: “Understand complicated ideas” and “learn all the topics” greatest increases since the Fall (bookmark)
Lowest percent favorable: “Do the hardest work assigned” and lowest growth: “complete all the work” (bookmark)

Middle school (back pocket):

Highlight:

Highs: “Emotion regulation” - greatest growth
Lowest: “Sense of belonging” and “Self-efficacy” (although greatest growth) - Even though lowest, increases since fall so something is working!

Click into self efficacy:

Change over time: Stayed relatively flat the past two years, with the biggest point increase this winter
Compared nationally: change the dataset to middle schools and the topic score is placed in the 10th percentile, high schools in the 50th percentile
Student group: grade level - goes down in middle school and up in high school, flat in 10th and 11th grade
Questions: greatest increase in “understanding complicated ideas,” favorable responses in “completing all the work”
Lowest questions: “Remember what you learned in your current classes, next year” “Can do the hardest work” “Understand complicated ideas”

26 of 50

Protocol #1: Low-Inference Data Inquiry

26

Select your school
Select the Winter 2022 - SEL data
Identify 1-2 areas of strength
Identify 1-2 areas for improvement
Identify 1-2 areas that have demonstrated growth from the Fall 2021 - SEL data set

27 of 50

Protocol #2: Identify What’s Working

27

Reflect on the 1-2 areas that have demonstrated growth.

What existing complex, school, or classroom systems / practices could have contributed to this?

Are there conversations we can continue to learn about what’s working in our community?

28 of 50

Chat Share-out

What stands out to you from the initial data review?

29 of 50

29

Step 1: Explore Topics	Step 2: Benchmarks	Step 3: Explore Questions	Step 4: Dig Deeper
From this school-level view, identify how students respond across topics Which topic have increased and/or decreased since the last survey window?	Compare your school/district with other school/districts across topics Across topics, how does your school/complex compare?	Choose a topic that you want to explore further What questions can be highlighted as a bright spot and leveraged to address needs?	Choose the Groups page Where do you notice gaps in experience among different groups of students?

30 of 50

Hawaii MTSS Blueprint

Which students are telling us they need supplemental support?

What other interventions are effective?

All Students

Supplemental Support

Intensive Support

31 of 50

Protocol #3: Reviewing Comparison Data

31

Reflect on the comparison data across sites.

What do you notice?
What do you wonder?
For which small groups and demographics do you want to do more data exploring?

32 of 50

32

Reflecting

Prioritizing

Strategizing

Our Time Together

3

2

1

Action and communication planning.

Use a prioritization matrix to determine which areas of focus will have the impact and feasibility.

Review your data to reflect on areas of strength, areas of growth, and areas that have demonstrated improvement.

33 of 50

System-wide SEL Support

3

Statewide Initiatives

3-1-1 plan for opening schools.

2

Complex Focus

and Initiatives

Complex-wide academic initiatives and interventions

1

Schoolwide Interventions

School and classroom level daily impact on students’ attendance and academic needs.

Your Role

and Team

34 of 50

Protocol #3: Narrow Your Focus

34

Reflect on the 1-2 areas of improvement.

What existing systems and practices impact this area?
Where might gaps exist? �Think: systems, people, programs, resources (including time), skills, knowledge, capacity, buy-in

35 of 50

Protocol #4: Finding the Root Cause

Step 1: Identify the data

Identify the insight or gap we see in our data (i.e. decrease from Fall 2021, topics, grade levels, etc.)

Step 2: Analyze using 5-whys

Why might this insight or gap exist?

1.

2.

3.

4.

5.

Why’s that? x5

Identified root cause

36 of 50

15 Social-Emotional Learning Interventions

Explore these interventions to identify which strategies might be effective in your community.

Adapt practices to fit into systems and structures that you already have in place.

37 of 50

Continue Learning at Your Own Pace

Earn Certificates of Completion

Building Capacity for Adult SEL

Introduction to Multi-Tiered System of Supports (MTSS)

Actionable Strategies to Build Students' Sense of Belonging:

Empower yourself and your team with the mindsets, skills, and strategies to cultivate students’ sense of belonging. Access ready-made templates and resources to try out with your students or share with your school or district team.

learningcenter.panoramaed.com

38 of 50

Action Planning

What makes sense to do now?

Dig more into data
Plan for action

Locus of control and influence
Who needs to be involved?
What information is key?
What already exists?

39 of 50

Protocol #5: Prioritize

39

First, brainstorm 3-5 ideas that would impact your area for improvement

School level or
Grade level or
Demographic level

Then, consider each idea:

How valuable could it be?
How much effort would it take?

40 of 50

Small Group Share-out

10 minutes

How could an idea you came up with impact an existing gaps or area of need?

41 of 50

41

Reflecting

Prioritizing

Strategizing

Action and communication planning.

Use a prioritization matrix to determine which areas of focus will have the impact and feasibility.

Our Time Together

3

2

1

Review your data to reflect on areas of strength, areas of growth, and areas that have demonstrated improvement.

42 of 50

Work Time

Option 1: Plan

Communicating about data
Engage students and staff in conversation about data
“Bridging the gaps”
Culturally responsive practices and systems
Identify and celebrate assets / strength

Option 2: Data Dig

Select a school, or topic to learn more about
What stands out to you?
What do you notice?
What do you wonder?
What’s your next move?

43 of 50

Continue the Convo with Our Students

& Staff

43

https://pan-ed.com/convokit

44 of 50

Open the Panorama Playbook

In the top-right corner, click “Playbook”

45 of 50

Explore Playbook

Step 1: Explore A Playbook Topic	Step 2: Dig Deeper Into A Topic	Step 3: Plan to Model
Choose a topic that you explored in the data. What strategies makes sense for my data and context?	Choose a strategy to explore further. How might you customize/adapt this strategy/intervention to fit your context?	What is a strategy that you would like to try complex/school-wide? What structure can you use to model and practice this strategy with your community? When would you like try this?

46 of 50

In your workbook

How will you leverage the Playbook resources in responsive conversations?

47 of 50

Add to Panorama’s

Hawaii Playbook

Help us share the assets that already exists in the community!

pan-ed.com/playbookhawaii

48 of 50

Growing Pono Schools

48

For more information on the Growing Pono curricula click here and the #Pono: Pass It On! Challenge click here

49 of 50

49

What’s Next?

Celebrating strengths and growth with your school community Recognizing areas for improvement with your school community Incorporating Playbook strategies into responsive conversations with school teams Support staff with the planning and implementation of intervention groups based on data

50 of 50

Thank you and feedback!

https://pan-ed.com/feedbacksurvey

Survey code: hidoe

Facilitator: Geoffrey Gaurano

Jason Feldman