​

Assessment Auditing

Coaches Training

​

​

Greater Albany Public Schools

Fall 2022

Who is in the Room?

Say good morning and show a picture on your phone that represents where you would like to beam yourself back to!

• Review the components of sound design
• Become familiar with the assessment audit checklist
• Observe the process for providing feedback for assessments
• Provide an opportunity to ask and answer questions

OBJECTIVES FOR THE DAY

Plan for Building Assessment Literacy

• Learn (I do)
• Guided practice (We do)
• Individual work time (You do)

​

​

Have you attended assessment cadre?

Warm up:

Work with your table to recall Stiggins’

Keys to Quality Assessment

Key 1:

Key 3:

Key 4:

Key 2:

Key 5:

Key Three:

Sound Design Competencies

​

Learning targets are translated into �assessments that yield accurate results.

• Design assessments to serve intended formative and summative purposes.
• Select assessment methods to match intended learning targets.
• Understand and apply principles of sampling learning appropriately.
• Write and/or select assessment items, tasks, scoring guides, and rubrics that meet standards of quality.
• Know and avoid sources of bias that distort results.

​

Using the Assessment Audit Checklist

Each time you audit assessments you will need:

• Paper copies of the assessment (1 per person)
• Paper copies of the scoring/administration agreements (1 per person)
• Paper copies of the standards (1 per person - this can be reused)
• Paper copies of the projection maps (1 per person - this can be reused)
• Access to the team’s assessment plan (if they made one)
• A digital copy of the assessment audit/checklist to record notes/feedback for the team

http://bit.ly/AssessmentAuditChecklist

Model/Fishbowl

Observers:

• Listen deeply
• Notice
• Take notes
• Don’t bug the fish!

Debrief

• What did you notice?
• What strategies were tried?
• What was “let go”?
• What was “pushed on”?

Review: Assessment Methods

• Selected Response

Students select the correct or best response from a list provided.

​

​

• Personal Communication

Students share what they have learned through structured and unstructured interactions with teachers.

​

​

• Performance Assessment

Students complete a task that is evaluated by judging the level of quality using a rubric.

• Written Response

Students construct an answer in response to a question or task rather than to select the answer from a list.

Target Type Match

What is the Difference?

Bias

Inaccurate results due to either test design or the method of interpretation or use that systematically disadvantages certain groups of students over others. Similar to “test fairness.”

Distortion

Inaccurate results that can lead to a false positive or false negative due to variety of potential impacts such as; poor target method match, poor question/answer quality, inappropriate weighting.

Both lead to inaccurate and less actionable information. More importantly, they lead to misinformation and confusion for stakeholders. Consider the implications for students, teachers, parents and the list could go on!

Use DOK to Compare Test Items

Sound Design “Look Fors”

2. Target Sampling/Scoring Weight

• Look at each question individually and mark the part of the standard it addresses (add tallies for those covered more than once)
• Note elements that are not covered at all or are too heavily weighted (could sway results)

RL9: Compare and contrast the themes, settings, and plots of stories written by the same author about the same or similar characters (e.g., in books from a series)

​

L1a: Explain the function of nouns, pronouns, verbs, adjectives, and adverbs in general and their functions in particular sentences.

​

L1f:Ensure subject-verb and pronoun-antecedent agreement.*

​

L5a: Distinguish the literal and nonliteral meanings of words and phrases in context (e.g., take steps).

Target Matched to Question Example

Sound Design “Look Fors”

4. Look at the DOK level of the questions. Are they appropriate:

• For the standard?
• For the placement of the assessment within the unit?

And are there a variety of DOK-level questions present?

​

5. Do any of the questions have bias or distortion?

Stiggins Assessment Writing Tips

• Keep wording simple and focused. Aim for the lowest possible reading level.
• Ask a full question in the stem.
• Eliminate clues to the correct answer either within the question or across questions within a test.
• Do not make the correct answer obvious to students who have not studied the material.
• Highlight critical, easily overlooked words.
• Have a qualified colleague read your items to ensure their appropriateness.

​

Time for a Break

Monitoring Form

• Communication tool between writing and auditing teams
• Writing teams add links to assessment and teacher scoring guides and update color coding as appropriate
• Auditing teams add links to feedback and update color coding as appropriate

https://bit.ly/AlbanyAudit22

Shared Practice

1. Alignment to Standard
2. Check Text Type (for ELA)
3. Sampling
4. Take the Test Yourself
5. Target Method Match
6. DOK Audit
7. Check for Bias and Distortion
8. Check Rubrics and Scoring Agreements
9. Teacher Directions
10. Student Assessment
11. Formatting Fix Up
12. Other Considerations

​

Time to Dive In!

Work Time

1. Follow the checklist

OR

• Revision: Keep feedback language simple and focused. Correct small mistakes when possible.
• Repeat

Thank You

Your feedback is important to us.

We read every response!

​

​

PRESENTED BY