1 of 23

STAAR Constructed Response Scoring Training

Winter 2023

2 of 23

Hosts

Facilitators—WestEd

    • Lucas Banta (Science)
    • Kara Butterworth (Science)
    • Emily Hilligoss (Reading & Language Arts)
    • Kathryn Rhoades (Reading & Language Arts)
    • John Thorpe (Reading & Language Arts)
    • Dave Williams (Social Studies)
    • Tracy Duncan (Social Studies)
    • Elizabeth Dyke (Logistics)

​

Texas Education Agency (TEA)

​

​

​

​

2

3 of 23

Agenda

3

    • General Session
    • Content Groups
      • Reading Language Arts (RLA), Science, and Social Studies
    • Breakout Rooms
      • Review, select, and annotate responses for training
    • Content Groups
      • Train other group(s) and receive feedback
    • Adjourn

4 of 23

What is a Constructed Response (CR)?​

    • Student-written (or -typed), open-ended response to a prompt
    • Responses are human-scored using a rubric and scoring guide
    • 2-point Short Constructed Response (SCR) Items
      • Science and Social Studies (also developed in RLA but not part of this training)
    • 5-point Extended Constructed Response (ECR) Items
      • RLA
        • 2-dimensional rubrics
          • Organization and Development of Ideas (3 points), Conventions (2 points)
        • Argumentative/Opinion or Informational

​

​

​

​

4

5 of 23

History of CRs in Texas

  • CRs have a long history as a part of the Texas assessment program
    • ECRs have been included in statewide assessments since 1990 though the grade levels have varied as the assessment program changed.
      • Prior to the Spring 2023 administration, ECRs were standalone prompts unrelated to passages students read on the corresponding assessment.
    • SCRs were included on grades 9-11 TAKS and STAAR assessments from 2003 through December 2016.
    • ECRs and SCRs have historically been human scored by more than one rater.

​

5

1990

TAAS implemented. Writing assessments administered in grades 3, 5, 7, 9, and 11 and included an ECR.

1993

TAAS writing assessments administered in grades 4, 8, and 10 and included an ECR.

2003

TAKS implemented. Writing assessments administered in grades 4, 7, 10, and 11 and included an ECR. SCRs were included at grades 9-11.

2012

STAAR implemented. Writing administered in grades 4 and 7 through 2021 and in grades 9, 10, and 11. All included an ECR, and included SCRs in 9-11 through 2016.

2023

STAAR redesign implemented. RLA administered at grades 3-10, and all include an ECR. RLA, science, and social studies assessments all include SCRs.

6 of 23

CR Development Flow

6

7 of 23

CR Scoring Flow

7

Response Review and Selection

Rangefinding (Educator Committee)

Field Test

Operational

8 of 23

Teacher Participation

  • Educator Item Review
  • Rangefinding
  • Constructed Response Scoring Training

8

9 of 23

Quality Control

    • Qualification Sets
    • Calibration Sets
    • Validity Sets
    • Second Scores (double reads)
      • Scores are combined in RLA (maximum of 10 points total)
    • Resolution scoring
    • Voided scores (based on scorer performance metrics)

​

​

​

9

10 of 23

Sample Science SCR Prompt

10

11 of 23

Sample Science SCR Rubric

11

12 of 23

Sample Social Studies SCR Prompt

12

13 of 23

Sample Social Studies SCR Rubric

13

14 of 23

Sample RLA Passage

14

15 of 23

Sample RLA Passage

15

16 of 23

Sample RLA ECR Prompt

16

17 of 23

Sample RLA ECR Rubric

17

18 of 23

Sample RLA ECR Rubric

18

19 of 23

Sample RLA ECR Rubric

19

20 of 23

Sample RLA ECR Rubric

20

21 of 23

Sample RLA ECR Rubric

21

22 of 23

Questions?

22

23 of 23

Next Steps

  1. Content groups
  2. Content-specific CR and rubric training
  3. Simulated item training using scored and annotated student responses

23