1 of 36

Putting the Pieces Together: High-Quality Classroom Assessment Systems

This module is part of a Classroom Assessment series developed by the National Center for the Improvement of Educational Assessment.

This work is licensed under a Creative Commons Attribution 4.0 International License. Educators and others may use or adapt. If modified, please attribute and re-title.

Version 1.0 | Updated May 30, 2020| Developed By:�Carla Evans & Jeri Thompson

This work is licensed under a �Creative Commons Attribution 4.0

International License.

2 of 36

Introductions

www.nciea.org

2

Carla M. Evans

cevans@nciea.org

@CarlaMEvans

Jeri Thompson

jthompson@nciea.org

Suggested citation for this learning module: Evans, C. M. & Thompson, J. (2020). Classroom Assessment Learning Modules: Putting the Pieces Together—High Quality Classroom Assessment Systems. Dover, NH: National Center for the Improvement of Educational Assessment. Retrieved from www.nciea.org/classroom-assessment-learning-modules

3 of 36

www.nciea.org

3

The National Center for the Improvement of Educational Assessment, Inc. (The Center for Assessment) is a Dover, NH based not-for-profit (501(c)(3)) corporation. Founded in September 1998, the Center’s mission is to improve the educational achievement of students by promoting improved practices in educational assessment and accountability.

www.nciea.org

4 of 36

www.nciea.org

4

www.nciea.org/classroom-assessment-learning-modules

5 of 36

At the end of this module, you should be able to:

Explain the characteristics of a balanced assessment system;
Describe the main purposes and uses of assessment information from the state, district, and classroom levels; and
Explain how to create classroom assessment maps and use the map to evaluate the quality of a classroom assessment system.

www.nciea.org

5

6 of 36

Module Outline

Background on Assessment Systems
Purpose & Use of Assessment Information from Multiple Layers
Putting the Pieces Together for a High-Quality Classroom Assessment System
Practice Exercise
Wrap-Up & Module Survey

www.nciea.org

6

7 of 36

Background on Assessment Systems

www.nciea.org

7

8 of 36

Warm-Up

www.nciea.org

8

Prior to discuss what is an assessment system, we want to connect your professional practice with what we are about to discuss. So let’s start by reading about four different types of teachers. Over-testing Opan “spends more time assessing my students than teaching because I am required to enter a specific number of grades in her gradebook every week”; Confused Carlos says “the tests and quizzes I give are mainly selected response (e.g., multiple choice), even though that is not how people demonstrate their knowledge and skills in the real-world.” Dutiful Dante says: “I give tests because they are in the textbook. I give points for correct answers, put scores in my gradebook, and then move on in the curriculum.” And finally there is Feedback Frank: “I spend a lot of time giving specific feedback to my students on the quality of their responses or thinking, asking them to improve their final work products.” We ask that you reflect on the following questions either alone or with colleagues: Which teacher(s) do you most relate to and why? If someone were to write a short caption about your assessment philosophy, what would they say?” Please hit pause on the video and respond to these questions. Then hit play.

9 of 36

www.nciea.org

9

“A collection of assessments does not entail a

system any more than a pile of bricks constitutes a house.”

~Theodore Coladarci

10 of 36

Problem of Practice & Key Takeaways

Having a random and disconnected collection of classroom assessments with no guiding purpose promotes incoherence, inefficiency, and over-testing.

www.nciea.org

10

The collection (or system) of assessments matters.

Assessments should work together in a coherent system that produces data documenting student achievement towards meaningful learning targets.
Assessments should be thoughtfully and purposefully selected to support a range of decisions in classrooms, schools, and districts.

11 of 36

The Call for Balanced Systems of Assessment

The National Research Council report, Knowing What Students Know (2001), crystalized the appeal for balanced systems of assessment:

“Assessments at all levels--from classroom to state--will work together in a system that is comprehensive, coherent, and continuous. In such a system, assessments would provide a variety of evidence to support educational decision making. Assessment at all levels would be linked back to the same underlying model of student learning and would provide indications of student growth over time” (p. 9).

11

12 of 36

www.nciea.org

12

There are multiple layers of an assessment system.
The purposes and uses of assessment information differ at each layer.
It is important to guard against practices that might have a negative impact on classroom instruction (e.g., teaching to the test, over-testing, narrowing of the curriculum, etc.).

If you google “balanced assessment systems” you will notice that many define balanced assessment systems in a different way. For example, many define BAS based upon the types of assessments that make up the system--whether those be formative, interim, or summative.
The problem in our opinion with that approach is that these assessment types (formative, interim, summative) are just tools like a hammer, screwdriver, or drill--they are agnostic to what you are building or fixing. So we have to talk about purpose and use for assessment information, as well as the multiple layers of the system that supply assessment information.
In this triangle graphic you the three layers of an assessment system (state, district, and classroom). The main purposes and uses of assessment information at each level (or layer) are to the left --color coded in terms of correspondence with the layer. You’ll notice that the purposes and uses of assessment information differ at each layer. State layer serves primarily an accountability and equity function. District layer supplies an aggregate look at students across classrooms or schools within a district, and the classroom layer provides the most fine-grained information for teachers.
It is important as a classroom teacher to guard against practices such as teaching to the state test (for example) -- as these negative test prep practices do not promote high-quality teaching and learning.

13 of 36

What Makes an Assessment System Balanced?

www.nciea.org

13

See paper: “Not as Easy as It Sounds: Designing Balanced Assessment Systems”

What makes an assessment system balanced? An assessment system is balanced when it can be described as: A coherent, coordinated collection of assessments that, together, produces data that document student achievement toward given learning targets for a specific purpose or purposes in efficient and useful ways. Therefore, these five criteria or characteristics of balanced assessment systems can help you as a classroom teacher to evaluate the quality of your classroom assessment system. For example, to what extent does the collection of summative classroom assessments administered in your classroom over the course of the year allow students to demonstrate their understanding in a variety of ways and reflect the breadth and depth of the state content standards (comprehensive criterion)? To what extent is the collection of assessment coherent with what you believe about how students learn and deeper learning aspirations for students? (referred to in KWSK as vertical coherence). To what extent are the assessments aligned to the curriculum and instruction (horizontal coherence)? We also want classroom assessments to collect information about what students know and can do continuously over the course of the year and not all at one point in time (like the end of the year). We want to collect and document student progress over time. A couple additional criteria for BAS not found in KWSK but put forth by Scott Marion and Raj Chattergoon in the linked article are the criteria of efficient and useful. Many parents, students, and stakeholders are complaining about over-testing in schools. Instructional time is valuable and therefore assessments that take up precious instructional time should be useful and non-redundant.

14 of 36

Coherence Starts with Learning Theory

Assessments reflect the undergirding learning theory in the classroom assessment system.

For example, if learning is an aspect of conditioning (stimulus-response behaviorist approach) then (a) instruction is focused on teaching discrete knowledge and isolated skills and (b) assessments used to gather evidence of students’ knowledge and skills will focus on bits and pieces rather than conceptual understanding.

This module assumes a modern conceptualization of how people learn—whether constructivist, cognitivist, or socio-cultural.
Assessment is a process of reasoning from evidence and should align with modern conceptions of how students learn and demonstrate deeper learning.

www.nciea.org

14

15 of 36

Learning Theory and Coherence in Assessment Design

Teachers must ensure they are teaching for deeper learning before trying to assess it!
Teachers must design or select challenging assessment tasks to elicit higher-order thinking, ability to transfer, and 21st century skills.
Teachers should include students as active agents in evaluating their own work.

www.nciea.org

15

Figure taken from Shepard, L. A. (2000). The role of assessment in a learning culture. Educational Researcher, 29(7), 4–14.

See the Center’s 21st Century Skills Resource Page: https://www.nciea.org/library/assessing-21st-century-skills-resources

On the right hand side of this slide you see a figure taken from Lorrie Shepard’s article: The role of assessment in a learning culture. Using cognitive and constructivist learning theories she shows how such a theory should reform curriculum and classroom assessment. As Lorrie elucidates in her 2006 chapter on Classroom Assessment in the fifth edition of Ed Measurement:

“Assessment cannot promote learning if it is based on tasks or questions that divert attention from the real goals of instruction. Historically, traditional tests often misdirected instruction if they focused on what was easiest to measure instead of what was important to learn. Classroom instruction should engage students in learning activities that are as directly as possible instantiations of the real goals for learning. For example, if we want students to be able to read books, newspapers, and poems, they should, in fact, do these things and not be given shortened, simulated materials except to make them age appropriate. Similarly, in science, if we want students to be able to use and reason with scientific knowledge, then students should have the opportunity to figure out how things work by conducting investigations and developing explanations in their own words that connect their experiences with textbook theories. Assessment, then, must be conducted as part of such meaningful learning activities.” (Shepard, 2006, Classroom Assessment Chapter in Ed Measurement, p. 629).

This involves at minimum: teaching for deeper learning; designing challenging tasks; and involving students as active agents in evaluating their own work.

16 of 36

Learning Theory Unifies Curriculum, Instruction & Assessment

www.nciea.org

16

A model of learning serves “as a unifying element—a nucleus that brings cohesion to curriculum, instruction, and assessment.”

National Research Council. (2001). Knowing what students know: The science and design of educational assessment. Washington, DC: The National Academies Press, p. 54.

17 of 36

Purpose & Use of Assessment Information from Multiple Layers

www.nciea.org

17

18 of 36

State Assessment Layer

www.nciea.org

18

Purpose	Use: What can the teacher do with the data?	Cautions
Accountability: Evaluate Student Achievement & Achievement Gaps Evaluate School Quality	Calculate % of students by achievement level in your class for each relevant content area: What is the general class profile? What % of students are proficient? What % of students are not yet proficient? What % of students are advanced? Use that information to provide a general sense of content area (ELA, math, science, etc.) strengths and needs.	State tests are designed to survey the broad domain of grade and subject area state content standards. These data are not instructionally useful. A teacher needs more fine-grained, curricular information to differentiate instruction for groups of students.

Designed to provide a window into students’ equitable opportunity to learn the state content standards.

Serves a primary equity and accountability function.

We start by discussing the first two layers (state and district) as teachers do not have control over these layers. The state assessment layer is designed to provide a window into students’ equitable opportunity to learn the state content standards, which serves a primary equity and accountability function. In other words, test-based accountability came about alongside the standards-based reform movement because it was impossible to sit in every teacher’s classroom across the country every day, every year to ensure that they were teaching students the state content standards. Instead, the state creates a state standardized tests which surveys the broad domain of a grade/subject area’s state content standards and then the results of that assessment is used to hold schools and districts accountable for student proficiency and growth, student achievement gaps, etc.

Given this assessment design and purpose, what can the teacher do with the data, if anything? To be honest, this information is not very useful for informing a unit of instruction. Teachers can calculate the percent of students in their class at each achievement level prior to school starting for the year to get a general sense of the group of students’ academic strengths and needs in a content area (class profile, if you will), but teachers should not feel pressure to try to make more of this assessment data than is relevant for them.

19 of 36

District Assessment Layer

www.nciea.org

19

Purpose	Use: What can the teacher do with the data?	Cautions
Predict student proficiency on end-of-year state test (e.g., commercial interim). Identify students who will need additional supports or interventions to reach grade-level proficiency standards (e.g., universal screening tools). Monitor student learning across classrooms and schools within the district during the school year (e.g., common benchmark assessments). Evaluate curricula and programs.	Examine the extent to which the results from various assessments are similar (i.e., triangulate results from commercial interim, district-created and state assessment). Identify broad areas of strength or unfinished learning/re-learning that can be identified from the assessment results and applied within your classroom units of instruction. Identify students who are not demonstrating progress from fall to winter to spring. Consider whether these students need additional supports.	Be wary of over-interpreting “subscore” data on commercial interims as these scores are based on only a few items. District interim assessments are not designed to provide instructionally useful information. When results do not provide consistent information, teachers should ask school/district leaders to engage in a deeper dive in the results (e.g., check the alignment of the assessment to the state content standards and curriculum; test fairness).

Designed to provide school and district leaders with actionable data they can use to direct resources to support student achievement.

The district assessment layer is designed to provide school and district leaders with actionable data they can use to direct resources to support student achievement. Notice that these assessments are not designed typically to provide instructionally useful information to teachers. Keep that in mind when you think about what you, a classroom teacher, can do with the data and cautions about over-interpreting.

There are multiple purposes for assessment information at this level. For example, a district may want to predict student proficiency on the end-of-year state test and so they purchase a commercial interim assessment. While one could argue school/district leaders have more reliable and valid information with respect to that purpose by looking at a student’s last year state test result, some districts decide to purchase commercial interims to monitor student growth over the year. School/district leaders may also use universal screening tools to identify students who will need additional supports or interventions to reach grade-level proficiency standards. Some districts may create or select common benchmark assessments aligned to their curriculum that monitor student learning across classroom and schools within the district during the school year. And so on. Typically, any one assessment is designed to serve only one purpose and use well...though it is likely that you will find your school or district uses the same district assessment for multiple purposes. That is important to keep in mind when you try to make sense of results.

So what can the teacher do with this data? First, let me say that we do not suppose that a teacher can or should do anything with this data. Some schools or districts may not expect teachers to make sense of this data for instructional purposes. This is wise given that district interim assessments are not typically designed to provide instructionally useful information for any given unit. We suggest some ways, however, that teachers if they are so inclined can try to triangulate findings across various assessments, identify any broad information that may be relevant to a unit of instruction, and note students who based on multiple data points do not appear to be making progress.

One important caution is to be wary of over-interpreting “subscore” data -- which is the reporting categories you sometimes see on interim score reports that break down a content area such as math into smaller categories such as fractions, algebra, number sense, measurement and data, etc. These subscores are only based on a few items so they are not reliable enough to take instructional action, unless those results are corroborated by other assessments you have given in the classroom.

20 of 36

Classroom Assessment Layer

www.nciea.org

20

Purpose	Use: What can the teacher do with the data?
Monitor/adapt instruction Support and scaffold student learning Foster student agency (allow students to take ownership of their learning) Evaluate and report on student achievement	Formative Assessment Data → See Formative Assessment Module Summative Assessment Data→ See Summative Assessment Module Grading and reporting

Designed to provide teachers with actionable data they can use for planning and classroom instruction to support student achievement, as well as report on student proficiency and progress.

See paper: “Classroom Assessment Principles to Support Teaching & Learning”

Now to the focus of this module--the classroom assessment layer. You may wonder, by the way, why we spent so much time discussing the state and district layer when that is not in the control of teachers and we argue that teachers cannot get a lot of instructionally useful information from those assessments. It is precisely that reason why we spent so much time talking about those layers. Teachers often are told the opposite--that they should be able to glean important information from state test results and district interim results. We hope this information provides the assessment literacy you as a classroom teacher need to question and push back on those claims.

Okay, so now to classroom assessments. Classroom assessments are designed to provide teachers with actionable data they can use for planning and classroom instruction to support student achievement, as well as report on student proficiency and progress. Classroom assessments are typically divided into formative and summative. Formatives help teachers monitor/adapt instruction in real-time, support and scaffold student learning, as well as foster student agency by allowing students to take ownership of their learning. Summative classroom assessments evaluate student achievement --typically at the end of a unit of instruction--and use that information to report on student achievement to parents and students. Grading should not be used to motivate students, control student effort, or compare students to one another, but used to provide substantive feedback to students and parents on how their performance relates to grade level expectations at that point in time. We have two modules -- one on formative assessment and one on summative classroom assessment hyperlinked on this slide. We refer you to those modules for more details.

21 of 36

Putting the Pieces Together for a High-Quality Classroom Assessment System

www.nciea.org

21

22 of 36

www.nciea.org

22

In what way or to what degree, do the assessments in your classroom demonstrate a balanced assessment system?

23 of 36

Putting the Pieces Together

How do you know if you have a random collection of assessments, or a coherent classroom assessment system?

www.nciea.org

23

The figure on the left shows the pieces for any given unit of instruction. Depending on the content/grade level, there may be critical pre-cursor concepts students must know or be able to do in order to access the material. So you may start with a readiness pre-assessment, then use that data to differentiate instruction along with other formative assessment information gathered before, during, and after daily instruction. At some point, you will want a summative measure of what students know and can do independently at the end of a unit of instruction and before moving on to new instruction. This cycle repeats over and over again throughout the year.

So how do you know if you have a random collection of assessments, or a coherent classroom assessment system? This was the question we started with at the beginning of this module. We suggest that you hone in on the summative classroom assessments.

24 of 36

A Focus on Summative Classroom Assessment System

Examining the collection of summative classroom assessments provides a window into the quality of the system.

Why?

The types of assessments used in the classroom can illuminate the extent to which students are asked to transfer their learning to new or novel contexts (e.g., are all summative assessments selected response tests, or are there performance-based assessments included?).
The number of assessments administered over the course of the year in any given course or grade/subject can illuminate issues of over- or under-testing.
The extent to which all/most assessments are designed to gather evidence for a cluster of standards rather than a single standard. This illuminates the extent to which students are required to demonstrate their knowledge and skills of multiple domains in one content area or integrate across multiple content areas.

www.nciea.org

24

25 of 36

Classroom Summative Assessment Audits

Classroom assessment maps are an auditing tool to examine the number and type of assessments over the year and the relationship of those assessments to the depth and breadth of the state content standards.
These audits are intended to facilitate discussions about the features of balanced assessment systems and the ways the collection of assessments in the classroom layer may need to be changed to support a high-quality system of assessments.
You can think of classroom assessment maps like an assessment scope and sequence.

What are the interacting set of classroom summative assessments you are using to gather evidence about students’ learning relative to the state content standards?

www.nciea.org

25

26 of 36

www.nciea.org

26

What do you notice?

What questions do you have?

Consider:

Will these assessments provide different types of information about student learning?

Why do we care if they provide different, or the same information?

One Example of a Gr 3 Math Assessment Map

PBA 4

Short Summative = shorter classroom summative assessment

PBA = performance-based assessment

Unit Test = longer classroom summative assessment

27 of 36

www.nciea.org

27

Some Unit of Time

(Semesters, Months, Quarters, Etc.)

State Content Standards

Assessment maps are meant to be a flexible tool!

Adapt as needed!

In terms of what do you notice, you see that all of the state content standard for Gr 3 math are listed down the left hand column. There is also some unit of time demarcated at the top. Here you see months of the year, a school/district could decide to choose trimesters, semesters, quarters, etc. Different types of assessment are color coded and named. I often recommend that teachers add in the specific name of the assessment rather than just Short Summative 1, for example, as this can serve as a resource to new teachers and assessment names are often more helpful for that purpose. You do not need to include these three assessment designs--short summative, unit test, or PBA necessarily. We do recommend however that you do not list formative assessments (as those are inseparable from instruction and would be too numerous to include) and you also do not include district level assessments or state level assessments. The key thing to note is that assessment maps are meant to be a flexible tool. Adapt as needed, but strive for consistency across classrooms in a school or district if you want to be able to evaluate quality across classrooms, schools, content areas, and/or grade levels.

28 of 36

How do you Create Classroom Assessment Maps?

Use the summative classroom assessments for a grade/subject and backward map those assessments to the standards they measure by...

Using curricular resources to align each assessment with one or more standards.
Adding the assessments to the map in based on when it is administered. Use the actual name of the assessment, if possible.
Color coding each assessment by type (e.g., Unit Test, Performance Assessment, Short Unit Quiz). Define the meaning of each and use similar color coding scheme across classrooms can help with interpretation.
Merging boxes (or repeat assessment name) across standards to show alignment to the standards measured by the assessment.

www.nciea.org

28

29 of 36

Reflecting on the Quality of the System

Is this collection of assessments coherent, comprehensive, continuous, efficient and useful?

What are the strengths and weaknesses of your classroom assessment system given a desire to elicit evidence of higher-order thinking and deeper learning?
Are the assessments aligned to how you believe people learn?
Do assessment probe the depth and breadth of the content standards?
Are students provided multiple and varied assessment experiences over the course of the year?
How many summative assessments are administered over the course of the year?
Are there redundant assessments or assessment that can be removed because the information is not used?

What about across classrooms within a school or district?

Is there coherence across grade levels or content areas?
How would a student experience the assessment system from K-12?

www.nciea.org

29

30 of 36

www.nciea.org

30

Modeling a Reflection on the Example Gr 3 Math Assessment Map

PBA 4

What are the strengths and weaknesses of your classroom assessment system given a desire to elicit evidence of higher-order thinking and deeper learning?
Are the assessments aligned to how you believe people learn?
Do assessment probe the depth and breadth of the content standards?
Are students provided multiple and varied assessment experiences over the course of the year?
How many summative assessments are administered over the course of the year?
Are there redundant assessments or assessment that can be removed because the information is not used?

What are the strengths and weaknesses of your classroom assessment system given a desire to elicit evidence of higher-order thinking and deeper learning?

Notice that students have opportunities to demonstrate knowledge and skills through PBA (3) over the course of the year, as well as more traditional types of demonstrations (unit tests, e.g.)

Are the assessments aligned to how you believe people learn? Do assessment probe the depth and breadth of the content standards?

We don’t know the answer to that question completely without collecting each of these summatives and taking a deep dive; however, we can say that all of the state content standards are assessed by at least one assessment.

Are students provided multiple and varied assessment experiences over the course of the year?

Yes, it appears so and relates to the first question and response. Not every standard in math is going to be taught throughout the course of the year. Notice, for example, that some of the MD standards are only taught/assessed at the end of the year. Other content areas may look different.

How many summative assessments are administered over the course of the year?

Thirteen. It seems reasonable to administer around 10 or 15 summative assessments over the year to gather a generalizable estimate of student achievement.
However, it is not just the number of assessments but the quality of those assessments and different students may need different numbers of assessments, depending on their proficiency level. For example, if a student is able to demonstrate mastery or proficiency on a particular set of standards early in the year, is it critical to continue collecting/gathering evidence at the same time and in the same ways as other students in the class?

Are there redundant assessments or assessment that can be removed because the information is not used?

Unclear without examining each summative classroom assessment in more depth; however, teachers should examine that information for their own classrooms.

31 of 36

Practice Exercise

www.nciea.org

31

32 of 36

Practice Exercise

Create a classroom assessment map individually, or with your grade-level or content area team.
Use the reflection questions to discuss your classroom assessment map with a colleague.
Identify strengths and weaknesses of your classroom assessment system.
Write one actionable goal for the year to improve the quality of your classroom assessment system.

www.nciea.org

32

33 of 36

Wrap-Up & Module Survey

www.nciea.org

33

34 of 36

Wrap-Up Notes

Developing a high-quality classroom assessment system is not one-size-fits-all.
For teachers interested in the next step beyond classroom assessment mapping, take a deep dive into the quality of the summative classroom assessments listed on your map. Use the information from the performance assessment review tool discussed in the summative classroom assessment module.

www.nciea.org

34

35 of 36

Module Survey

Please take this short survey to help us improve the quality and usefulness of this module going forward.

Thank you!

www.nciea.org

35

36 of 36

www.nciea.org

This work is licensed under a �Creative Commons Attribution 4.0

International License.