1 of 36

Evaluation Tools and Resources

Presented by the California County Superintendents Educational Services Association/County Offices of Education, California Association of Science Educators, K-12 Alliance @WestEd, California Science Project, and the California Department of Education

Time: 1 min (Runtime: 1 min)

Tech Directions: Ask participants to rename themselves with their grade band and name. If they are not a classroom teacher ask them to use the grade band they have been participating with (K-2, 3-5, 6-8, 9-12). For example: K-2, Name; 3-5, Name; 6-8, Name; 9-12, Name. Create breakout groups of 3 - 5 participants per grade band. For example if there are 12 K-2 teachers, create 3 rooms for K-2 and then manually sort the participants. If participants would like to join a group together (for example from the same district) ask them to add that to the chat and create the room with their grade and site or district name. For example breakout room 1: Site or District, K-2.

< 1 min

Welcome participants.

Facilitator note: If participants are in mixed grade groups, as participants enter the session, ask them to re-name themselves with the grade band they’ve been participating with (K-2, 3-5, 6-8, 9-12)

2 of 36

Writer’s Credit

Tamara Basepayne, San Joaquin County Office of Education
Arlene Laurison, Elk Grove Unified School District
Jose Marquez, Stanislaus County Office of Education
Meredith Casalino, Orange County Office of Education
Rudy Escobar, Stanislaus County Office of Education
John Spiegel, San Diego County Office of Education

2

Evaluation Tools and Resources

NGSS PL#2 - A Coherent Assessment System

3 of 36

Community Agreements

Keep students at the center of the work.
Be present, be curious, and be open to possibilities.
Balance advocacy of own ideas & inquiry of others.
Create a safe environment for individuals to productively struggle with ideas and tools.

Are any other agreements needed?

3

Evaluation Tools and Resources

NGSS PL#2 - A Coherent Assessment System

4 of 36

CA NGSS PL #2 Outcomes

Build awareness of equitable NGSS assessment systems

Describe the elements of equitable NGSS assessments
Learn about tools to screen assessments for equity and utility
Understand the CA assessment system continuum
Develop NGSS assessment literacy

4

Evaluation Tools and Resources

NGSS PL#2 - A Coherent Assessment System

5 of 36

Participant Professional Learning Flow

5

System Pre-Analysis

System Post-Analysis

Assessment Experience 1

Screening Assessments

Assessment Experience 2

Introduction

Evaluation Tools and Resources

NGSS PL#2 - A Coherent Assessment System

6 of 36

Session Goals

Learn about the tools that support educators in finding, analyzing, and evaluating assessment tasks.

Understand how to screen available assessment tasks for their utility in a student-centered assessment system.

6

Evaluation Tools and Resources

NGSS PL#2 - A Coherent Assessment System

7 of 36

Constants and Variables in Assessment

Constants

Assessments require

sensemaking using the three dimensions of NGSS

Variables

Variables to attend to when designing and modifying assessment experiences:

SEP Sophistication
DCI Sophistication
CCC Sophistication
Scaffolds and Linguistic Supports
Student Choice and Autonomy
Degree of Transfer
Other (Specify):

7

H2-8

Evaluation Tools and Resources

NGSS PL#2 - A Coherent Assessment System

Time: 3 min (Runtime: 9 min)

Tech note: Screen Share Constants and Variables document and explain the keys to equitable assessments, and orient everyone to what the document is sharing

H2-8 Link: [Insert Link Here]

WHY - For EQUITY! Ground our conversations in what is important.

Explain to participants that before doing a deeper evaluation of an assessment they need to remember the key factors in creating an equitable, student-centered, 3D assessment. Share importance of constants and variables. Let them know that sense making using the three dimensions must be a constant. The sophistication of each dimension, the necessary scaffolds and linguistic supports, student choice and autonomy, degree of transfer are all variables that can be adjusted depending on students’ needs.

Facilitator’s Note: the screen shot on assessments came from the H8 documents, which describe key elements of equitable, 3D assessments.

8 of 36

Assessment Evaluation Tools

We will be using three tools intended to help educators in evaluating science assessments tasks to determine whether they are designed for three-dimensional science standards.

H1 Achieve Science Task Prescreen - A quick review tool

Advanced Screening Tools

H2 Achieve Science Task Screener - An in-depth review tool, designed to be used after a task passes the precreen

H3 Science Assessments for Emergent Bilingual Learners (SAEBL) Checklist - A tool to support science assessments that work for emergent bilingual learners.

8

Evaluation Tools and Resources

NGSS PL#2 - A Coherent Assessment System

Time: 2 min (Runtime: 11 min)

Tech: Do not share the documents yet; documents will be shared one at a time during the session.

Explain that participants will be using three tools intended to help educators to evaluate science assessments tasks to determine whether they are designed for three-dimensional science standards.

Explain that the H1 Achieve Science Task Prescreen is quick review tool as opposed to the more comprehensive advanced screening tools. It is designed to easily identify non negotiables (either they are present or they are not)

Explain that the H2 Achieve Science Task Screener is an in-depth review tool, designed to be used after a task passes the prescreen. It helps to identify high-quality assessment tasks that emphasize sense making using the three dimensions.

Explain that the H3 Science Assessments for Emergent Bilingual Learners (SAEBL) Checklist is a tool to support science assessments that work for emergent bilingual learners. The tool provides a comprehensive list of steps to support students who are developing their language skills to access grade level material.

Where are we going: Explain that in this session we are going to start with the use the H1 Achieve Science Task Prescreen to discuss student-centered assessments. After, participants are going to use the advanced screening tools to focus on two key areas of student-centered assessments: sensemaking using the three dimensions of NGSS and the necessary supports for emergent bilingual learners.

9 of 36

Things to keep in mind when using assessment evaluation tools

9

Image of sunlight penetrating the ocean.

Image of girls canoeing

Image a person kayaking

Evaluation Tools and Resources

NGSS PL#2 - A Coherent Assessment System

Time: 2 min (Runtime: 13 min)

Explain the boating/ocean analogy to participants. The assessment tools should be completed as a team. For example when you are in a boat with a friend you have another person also looking out for “danger or concerns” but when you try to go by yourself you might think you are going a long great, but you can’t see, hear or know what is around you.

Explain to participants that these tools can be done by individuals educators, but their real intent is to be done in a collaborative setting, i.e., professional learning community, department meetings, etc.. While the goal of the tools is to evaluate the strengths and limitations of a task, it is the conversations that take place that will help to build our capacity in designing assessments that are student centered, three dimensional and equitable.

These tools are not intended to be used the night before an assessment is given.

10 of 36

H1 Achieve Task Prescreen

Identifying Red Flags in Assessment Tasks

10

Evaluation Tools and Resources

NGSS PL#2 - A Coherent Assessment System

11 of 36

Achieve Science Task Prescreen Tool

The purpose of the Achieve Task Prescreen is to conduct a quick review of assessment tasks to determine whether they might be designed for standards based on A Framework for K-12 Science Education.

On your own, take 3 minutes to go

over the tool.

Identify one word, phrase or statement that resonates with you.
Be prepared to share with the group.

11

H1

Evaluation Tools and Resources

NGSS PL#2 - A Coherent Assessment System

Time: 8 min (Runtime: 22 min)

Tech - Post in chat when facilitator directs.

H1 Achieve Task Prescreen: [Insert Link Here]

Explain to the participants that the purpose of the Achieve Task Prescreen is to conduct a quick review of assessment tasks to determine whether they might be appropriate for classroom use. The Prescreen is intended to reveal whether tasks include “red flags”—i.e., challenges commonly found in science assessment tasks. It is a great entry point for evaluating tasks because educators do not need to be fully versed in the NGSS and designing equitable assessments.

Have participants read the Achieve Science Task Pre-Screen Tool. Ask participants to identify one word, phrase or statement that resonates with them. Use the Waterfall protocol in the Chat to share participant ideas.

During the whole group discussion, remind participants that Achieve Science Task Pre-Screen Tool is designed to:

Quick review
Red flags
Focus on feature that are non-negotiables
Easily identifiable
Fatal flaws

Facilitator Note:

Before releasing participants into breakout rooms, remember to explain the purpose of a task scenario, which are designed to engage students in performing authentic scientific investigations and solving science problems through hands-on activities and computer simulations set in real-world contexts.

12 of 36

Achieve Task Prescreen Tool

12

Image of sunlight penetrating the ocean.

Evaluation Tools and Resources

NGSS PL#2 - A Coherent Assessment System

13 of 36

Achieve Task Pre-screen Process

In grade level groups, use the H1 Achieve Task Pre-Screen to analyze the grade span task assessment from one of the previous sessions.
Answer the questions in the Prescreen regarding the task(s) and note any red flags.
Discuss the answers to the question and evidence to support those answers with others.
Use your analysis to determine next steps for the task(s).

13

H1

Evaluation Tools and Resources

NGSS PL#2 - A Coherent Assessment System

Time: 25 min (Runtime: 48 min)

Prep: Make copy of T1 Jamboard #1. Make sure to click on “share” button in top of the screen. When window pops up, make sure you select “everyone with a link” and switch to “editor” option. Link in landing page [Insert link] (Make grade span templates)

Tech

Set up breakouts: assign participants to random grade-level-alike groups (3-4 per room of the same grade-level), 20 minutes

Broadcast time warnings at 5 and 2 minutes

Post in chat:

T1 Jamboard #1 copy - [Insert Link Here]

3D Assessment - K-2 - [Insert Link Here]

3-5 - [Insert Link Here]

MS -[Insert Link Here]

HS - [Insert Link Here]

Take a moment to introduce yourselves. Find your directions for the task are on frame 1-2 on the Jamboard.

3-5 mins to review the directions

Explain to participants that they will be diving into the pre-screen tool using the assessment from one of the previous sessions (Assessment Experience 1 or 2) Remind everyone to open the assessment task!

For the K-2 group they will be using a new assessment on light. Because this is the first time the K-2 group is seeing the assessment, remind them to review the teacher materials for the task.
For 3-5, they will be looking at the playground grass assessment
For MS will be looking at the natural hazards assessment and HS will be looking at the Meltwater assessment. The purpose of using the same assessment for the pre-screen tool serves to help calibrate the group to the tool. This calibration is a helpful tip for the teachers if they are taking these tools back to their teams.

Remind the participants that they have already completed the first steps of the prescreen tool by completing the assessment as a student and reviewing the support materials. Instruct they to do a quick review of the assessment and then jump right into using the tool.

Preface this collaboration by telling participants they will be working in grade specific breakout rooms and collaborating on a jamboard for 10 mins. Groups will be presenting their findings and supporting evidence with the whole group when they return.

Before opening the breakout rooms, make sure that you have the Jamboard ready to open as further directions are on the first page of the jamboard. Screenshare the Jamboard (T1) and explain that their breakout room will match their Jamboard frame and provide and tech tips for using Jamboard and show participants how to move the green check marks into a box representing how they scored the prescreen.

14 of 36

Reflection & Share Out

How did the prescreen support you in analyzing the task?

What questions did the process bring up for you?

14

Evaluation Tools and Resources

NGSS PL#2 - A Coherent Assessment System

15 of 36

Next Steps

If your assessment passed with no red flags, consider further analysis. What criterion were you least sure of?

If your assessment contains opportunities for improvement, this becomes your area of further analysis.

If your assessment contains too many red flags or has too many components not present - scrap it and start fresh.

15

Evaluation Tools and Resources

NGSS PL#2 - A Coherent Assessment System

Time: 1 min (Runtime: 52 minutes)

Explain to participants that using the prescreen tool should yield important conversation about equitable, NGSS assessments, and opportunities for you to decide if the task is useful as is, if it can be used with modifications or if it contains too many red flags. Assuming the assessment task does not have too many red flags

Remind participants that when analyzing an assessment the prescreen is only the FIRST step and that if an assessment warrants further review they should continue on to one of the advanced screening tools.

In the advanced screening tools, we will look at key components of equitable 2D/3D assessments, sense making using the three dimensions and the necessary support for emergent bilingual learners.

NOTE: In the next few slides the presenter will be giving a brief overview of H2 and H3 as a walkthrough. DO NOT SHARE these documents with the participants until the walk-through is complete.

Explain to participants that the two deep evaluation tools are meant to be intensive and have multiple components to work through. Starting first with H2 The Achieve Task Screener - explain that before using the advanced tools, educators complete the task as a student and evaluate it for fatal flaws using the pre-screener.

16 of 36

Advanced

Screening Tools:

H2 The Achieve Science Task Screener and

H3 The SAEBL Checklist

16

Evaluation Tools and Resources

NGSS PL#2 - A Coherent Assessment System

17 of 36

Task Evaluation Tools Criteria

H2 Achieve Science Task Screener

Tasks are driven by high-quality scenarios that focus on phenomena or problems.
Tasks require sensemaking that uses the three dimensions.
Tasks are fair and equitable.
Tasks support their intended targets and purpose.

17

Evaluation Tools and Resources

NGSS PL#2 - A Coherent Assessment System

18 of 36

H2 Achieve Science Task Screener

18

Image of sunlight penetrating the ocean.

Evaluation Tools and Resources

NGSS PL#2 - A Coherent Assessment System

19 of 36

Science Task Screener

On your own, take 4 minutes to skim the tool.

What additional supports might your colleagues need to be able to use the Science Task Screener?

19

H2

Evaluation Tools and Resources

NGSS PL#2 - A Coherent Assessment System

20 of 36

Achieve Science Task Screener

Take 2 minutes on your own to determine which indicators of criteria B are present.
Discuss with others whether an indicator is present/not present. Provide evidence to support your answer.
Discuss suggested improvements to determine next steps for the task(s).

Instructions

Find your group’s Jamboard frame.
Use the Science Task Screener to evaluate the quality of Criterion B.
Discuss suggested improvements to determine next steps.

Time: 25 min (Runtime: 1 hr 30 min)

Tech

Screen Share T2 Jamboard for Achieve Science Task Screener - [Insert Link Here]

Explain to participants that they will be going into the same breakout rooms to practice using the Achieve Science Task Screener. The criterion that everyone will be evaluating will be Criterion B for the assessment that they prescreened. If they finish early and would like to look and discuss another criterion they can do that while other groups finish.

Briefly review the directions with participants and clarify any questions before opening the breakout rooms.

Take 2 minutes to review the criterion on your own.
Discuss with your group whether the indicators are present or not and what evidence they are using to support their decision.
Drag the green arrow to identify across all indicators whether there is no evidence to extensive evidence of quality for criterion B.
Discuss suggested improvements to determine next steps for the task(s).

21 of 36

Break

Return in 5 minutes

21

Evaluation Tools and Resources

NGSS PL#2 - A Coherent Assessment System

22 of 36

SAEBL Checklist

22

Image of sunlight penetrating the ocean.

Evaluation Tools and Resources

NGSS PL#2 - A Coherent Assessment System

23 of 36

SAEBL Checklist

SAEBL - Science Assessment for Emergent Bilingual Learners

Checklist to help design assessments that allow emergent bilingual learners to demonstrate what they know and are able to do as they engage in 3D learning experiences. Broken down into 5 categories

Culture and language
Task components
Alignment and rigor
Clear objectives and scoring criteria
Integration of scaffolds

23

Evaluation Tools and Resources

NGSS PL#2 - A Coherent Assessment System

24 of 36

SAEBL

On your own, take 4 minutes to skim the tool.

Identify one category or component that you had not previously considered for an assessment.

24

H3

Evaluation Tools and Resources

NGSS PL#2 - A Coherent Assessment System

25 of 36

Clear Objectives/Scoring Criteria and Integrated Scaffolds

In grade level groups, we will use two criteria from the SAEBL checklist to evaluate the grade- span assessment task:

Clear Objectives and Scoring Criteria
Integration of Scaffolds

25

Evaluation Tools and Resources

NGSS PL#2 - A Coherent Assessment System

26 of 36

Group Reflection

How might these tools help you select, develop, or modify an assessment?

How might these tools help you address equity in an assessment?

What are some takeaways that you would like to remember going forward?

26

Evaluation Tools and Resources

NGSS PL#2 - A Coherent Assessment System

27 of 36

What can we do now?

If your assessment passed with no flaws - use it. (Useful Task)

If your assessment contains a opportunities for improvement - modify it. (Use with modifications)

If your assessment contains too many red flags or has too many components not present - scrap it and start fresh.

27

Evaluation Tools and Resources

NGSS PL#2 - A Coherent Assessment System

28 of 36

Supports for Next Steps

Start with the evidence statements for the PE you are evaluating.

Review the opportunities for improvement from the screener(s)

Do you need to use a different phenomena?
Do you need to provide more scaffolds?
Do you need to reframe the question(s)?

Use the Assessment Database.

Is the assessment designed using the principles of UDL?

28

Evaluation Tools and Resources

NGSS PL#2 - A Coherent Assessment System

Time: 4 min (Runtime:2 hr 03 min)

There are many resources available to help modify assessments. One of the best places to start is using the evidence statements for the Performance Expectation targeted in the assessment. The evidence statements may help to frame what students should be able to know and do on the assessment. If the phenomenon is an opportunity for improvement there are multiple sources of good phenomenon such as the CA Environmental Phenomena Resources, wonder of science or NGSS phenomenon sites. Even better, walk around your schoolyard or community to find local phenomena for students. Another useful tool on the Database site is the Development/Modification Module. If the assessment has equity as a needed improvement using the principles of UDL (universal design for learning) on the UDL site has many suggestions to improve assessments and lessons to improve equity.

Facilitator’s Note:

Evidence statements-If you are unfamiliar with the evidence statements, please select and read one. After teaching standards-based learning for many years mid-career and veteran teachers may overlook the nuances of the SEPs and CCCs embedded in these statements. Notice how the evidence statements call out all 3 dimensions in subtle ways. These statements will help teachers fill in gaps in their understanding of the 3 dimensionality of the PE as well as identify which specific dimension is missing from an assessment (such as CCC recognizing patterns)
Phenomena-There are many ways that a phenomenon can miss the mark. A strong phenomenon for an assessment must support all three dimensions. If it is too complicated, or not sophisticated enough, students will either be confused or not have an opportunity to illustrate their learning. Additionally, phenomenon must be relevant as well as accessible. Here is the another link that provides phenomenon for storylines as well as assessments.
Assessment Database- In the Development/Modification Module there are tasks that a PLC or department can work on to develop or help modify a 3D assessment. If you are unfamiliar with this, quickly peruse it. Simply mention this resource to participants as a way that PLCs or departments can further their work towards NGSS assessments.
Principles of Universal Designs for Learning- This is a set of guidelines grounded in educational research that promote multiple ways for students to engage in their learning, to access information for their learning, and to illustrate their learning. If you are unfamiliar with UDL, please review the graphic organizer located on the website. Direct participants to this website to improve accessibility and equity.

29 of 36

3D Assessment Database

29

Evaluation Tools and Resources

NGSS PL#2 - A Coherent Assessment System

30 of 36

Where can one find high-quality assessment tasks?

T4 NGSS Assessment Database

Request made by educators to identify and house high-quality NGSS assessment task resources.
Easily access and filter assessment tasks based on desired criteria.

30

Evaluation Tools and Resources

NGSS PL#2 - A Coherent Assessment System

31 of 36

Navigation of the Database

The NGSS Database is housed on the Stanislaus County Office of Education website.

Select the “NGSS Assessment Database” tab.
Use the filters (assessment type, grade, PE, etc.) at the top of the page to generate list of assessments.
Use the links on the right to select preferred assessments.

31

Evaluation Tools and Resources

NGSS PL#2 - A Coherent Assessment System

32 of 36

Find an Assessment

Select an assessment task from the T4 NGSS Database.
Read through the task and skim through any support materials for the task.
Choose a slide from the Prescreen Slidedeck.
Use the Prescreen Tool to decide if the task selected warrants further review.

32

H1

Evaluation Tools and Resources

NGSS PL#2 - A Coherent Assessment System

Time: 10 min (Runtime: 2 hr 18 min)

Tech

Breakouts, same groups, 10 minutes

Post in chat when facilitator directs

NGSS Assessment Database: [Insert Link Here]

T5 Prescreen Slidedeck: [Insert Link Here]

Explain directions to participants

Individually or in a grade/ content-specific group, select an assessment task from the database.
Read through any support materials for the task.
Answer the questions in the Pre-screen regarding the task and note any red flags.
Discuss the answers to the questions and evidence to support those answers with others.
Use your analysis to determine next steps.

Explain that participants will now get to pre-screen an assessment of their choice. They may work individually or in groups to choose an assessment and then complete the pre-screen tool (H1) and record their findings on the jamboard

NOTE: Instruct participants to duplicate a frame of the Jamboard if there are not frames present.

33 of 36

Reflection?

How do the tasks that you looked at in the database compare to tasks that are currently being used at your site?
How did the assessment hold up to the prescreener?
Which advanced screener tool would you use next and why?

33

Evaluation Tools and Resources

NGSS PL#2 - A Coherent Assessment System

34 of 36

Closing

34

Evaluation Tools and Resources

NGSS PL#2 - A Coherent Assessment System

35 of 36

Keep in mind . . .

When and where to use the assessment evaluation tools.
All assessment evaluation tools can be completed by a single educator, but they are best when used in a collaborative environment.
The biggest takeaway from assessment evaluation tools should be the discussions about equitable, NGSS assessments.
Not all assessments in the database will meet your student needs

35

Evaluation Tools and Resources

NGSS PL#2 - A Coherent Assessment System

36 of 36

Closing Thoughts

Think back to the session goals.

What questions have arisen after evaluating these assessment tasks?
Who on your team do you plan to share these tools and resources with next? Why?

Session Goals

Learn about the tools that support educators in finding, analyzing, and evaluating assessment tasks.
Understand how to screen available assessment tasks for their utility in a student-centered assessment system.

36

Evaluation Tools and Resources

NGSS PL#2 - A Coherent Assessment System

Time: 4 min (Runtime:2 hr 27 min)

Tech:

Share the Gots & Needs Jamboard in the chat: [Insert Link Here]

Show the participants which frame of the jamboard they should be working on for this session.

Remind participants that this is an opportunity to share their growth with us from this session as well as help the team understand what we may not have discussed yet that participants would like to understand about assessment. If appropriate, participants might also want to go back to add comments to previous sessions based on their understanding now.

Please also let participants know that they will have a break for X MIN prior to the next session OR that we will see them again on X DAY to discuss ___ (give a one sentence summary of the upcoming session). If this is the last session, please thank participants for their time and energy, as well as encouraging them to complete the PL#2 survey which will be sent to them this week.