1 of 19

The SC Review Tool & List Development

South Carolina

November 13, 2024

1

2 of 19

Objectives

Participants will …

Explore the the SC Mathematics HQIM Rubric
Understand the role of the instructional materials review process in developing the state adoption list

3 of 19

Agenda

3

Welcome

Overview of the SC Mathematics HQIM Rubric

Rubric Exploration

The Instructional Materials Review Process

Closing & Next Steps

1

5

4

3

2

4 of 19

Norms

When in large groups, please use your mute button to help provide a distraction-free experience
When in breakout groups, please unmute so that you can engage in conversation
Videos on make for the best interaction :)
Utilize the chat for asking questions and accessing resources

4

Let's begin by establishing the norms that will guide our learning experience today:

When we're in larger group settings, it would greatly help everyone if we could utilize our mute buttons. This will ensure that our discussions are as distraction-free as possible.

However, when we break into smaller groups, let's remember to unmute ourselves. This way, we can fully engage in conversation and collaborate effectively.

Videos can be incredibly valuable for enhancing interaction, so let's make use of them whenever possible!

It's important to create an environment where everyone feels comfortable expressing their opinions, even if they differ from the majority. Let's foster a culture where it's safe to disagree and where diverse perspectives are valued.

Lastly, don't forget to attend to your own needs throughout our session. Whether it's taking a break or asking for clarification, your well-being matters.

5 of 19

Overview of the

SC Mathematics HQIM Rubric

5

6 of 19

Development of the SC Mathematics Rubric

Content experts from SC partnered with EdReports to identify priorities and characteristics of HQIM for South Carolina aligned to:

The team worked together to organize the priorities to develop content specific and content agnostic high-quality criteria.
The rubric is designed to guide reviewers in understanding the indicators of quality and to score indicators based on evidence.

6

7 of 19

Gateway System

8 of 19

Gateways & Criteria

Review Summary	Criteria
Alignment to SC CCR Standards and Coherence	1.1 Alignment and Accuracy with the SC CCR Standards
Alignment to SC CCR Standards and Coherence	1.2 Learning Progressions & Coherence
Student Understanding	2.1 Rigor
Student Understanding	2.2 Assessments
Student & Teacher Supports	3.1 Supports for All Learners
Student & Teacher Supports	3.2 Teacher Planning and Learning for Success with the SC CCR Standards for Mathematics

This is a quick glance at the makeup of the tool.

Each Gateway is made up of criteria.

In Gateway 1, all criteria are related to standards alignment and coherence.
Criteria 1.1 specifically looks at alignment and full coverage of the SC math standards to full extent of the standards.
Criteria 1.2 looks at learning progressions from unit to unit and coherence.
In Gateway 2, the focus is on how materials provide opportunities for student to build mathematical understanding.
Criteria 2.1 is focused on indicators of rigor, while 2.2 is all about assessments that capture and inform instruction
In Gateway 3, we are looking at Student and Teacher supports. So these are the components of materials that allow for your diverse student populations to access grade-level content, and the guidance and supports for teachers to enact the materials.
Each of these criteria have a set of indicators that further break down those qualities that are hoped to exist in high-quality materials that meet the needs of SC students.

For each criterion, there is a threshold to determine if the materials exemplify or do not meet expectations for that criterion’s indicators. Those feed into the overall Gateway Score and Rating, which will guided reviewers whether to continue reviewing materials for the following gateway or to stop the review. You’ll collaborate with your review team as you approach the end of each Gateway to determine next steps. When the state list is released in the new years, those ultimate scores and recommendations for the list are dependent upon how materials demonstrated these markers of quality.

Now, we’ll see what those scoring tables look like at the Gateway and Criterion level.

9 of 19

Criterion-Level & Gateway-Level Summaries

10 of 19

Rubric Criteria: Indicators, Guiding Questions, Look-Fors

Indicators	Guiding Questions	Materials Should
2a. The materials support the intentional development of sense making and perseverance in solving problems strategically.	Do materials encourage students to explore and identify problems using multiple entry points? Do materials promote continual self-reflection and checking for the reasonableness of solutions? Do materials provide strategies, tools, or scaffolds that help students plan their solution pathways, track their progress, and make adjustments when they encounter obstacles or new information?	Encourage the use of multiple entry points to identify and explain the problem, helping students determine if their answer makes sense. Enable students to use prior knowledge, various methods, and continual self-reflection to check for reasonable solutions. Offer strategies, tools, or scaffolds to help students effectively plan, monitor, and adjust their solution pathways.
2b. The materials support the intentional development of explaining ideas using precise and contextually appropriate mathematical language, tools, and models.	Do materials encourage students to identify and utilize relevant tools through specific tasks or activities that prompt effective selection and application to explore, model, and deepen their understanding of concepts? Do materials support students in using precise mathematical language to model, explain, and justify their solutions? Do materials provide structured opportunities for students to engage in multiple modes of writing, speaking, and listening that promote meaningful discussion and reflection on mathematical concepts and solutions?	Provide opportunities for students to select, use, and explore tools relevant to the task or problem, enabling them to model concepts and develop understanding. Provide opportunities for students to use precise mathematical language to model, explain, and justify valid solutions and reasoning. Facilitate constructive dialogue, both individually and collaboratively, through writing, speaking, and listening.

2 min

The tool is organized by Gateways, and then criteria for each Gateway stacked vertically, with materials to be reviewed by those indicators in sequential order.

Now we are going to dive into the indicator and indicator guidance.

Indicators: Indicators are a description of high-quality components of instructional materials that align with the South Carolina College- and Career-Ready Mathematics Standards and SC CCR Mathematical Process Standards. Reviewers use Guiding Questions, Materials Should statements, and evidence to score indicators within each criterion.

Guiding Questions: The Guiding Questions are provided for each indicator to help reviewers focus their materials review on the substance of each criterion. These questions are intended to be used to help reviewers understand what evidence is needed for each indicator.

Materials Should Statements: The Materials Should Statements are suggestions and considerations of what reviewers might see in the instructional materials that support the evaluation of that indicator. These suggestions are not exhaustive as demonstrations for any indicator can and will look different from material to material.

11 of 19

Evaluating Criteria: Evidence, Scoring, Notes

Indicators	Guiding Questions	Materials Should	Reviewer Panelist Evidence	Scoring	Notes
2a. The materials support the intentional development of sense making and perseverance in solving problems strategically.	Do materials encourage students to explore and identify problems using multiple entry points? Do materials promote continual self-reflection and checking for the reasonableness of solutions? Do materials provide strategies, tools, or scaffolds that help students plan their solution pathways, track their progress, and make adjustments when they encounter obstacles or new information?	Encourage the use of multiple entry points to identify and explain the problem, helping students determine if their answer makes sense. Enable students to use prior knowledge, various methods, and continual self-reflection to check for reasonable solutions. Offer strategies, tools, or scaffolds to help students effectively plan, monitor, and adjust their solution pathways.
2b. The materials support the intentional development of explaining ideas using precise and contextually appropriate mathematical language, tools, and models.	Do materials encourage students to identify and utilize relevant tools through specific tasks or activities that prompt effective selection and application to explore, model, and deepen their understanding of concepts? Do materials support students in using precise mathematical language to model, explain, and justify their solutions? Do materials provide structured opportunities for students to engage in multiple modes of writing, speaking, and listening that promote meaningful discussion and reflection on mathematical concepts and solutions?	Provide opportunities for students to select, use, and explore tools relevant to the task or problem, enabling them to model concepts and develop understanding. Provide opportunities for students to use precise mathematical language to model, explain, and justify valid solutions and reasoning. Facilitate constructive dialogue, both individually and collaboratively, through writing, speaking, and listening.

12 of 19

The South Carolina Instructional Materials Review Process

12

13 of 19

The Instructional Materials Review Process

13

As you can see on this slide and as you heard in our rubric overview, It all began with the HQIM rubric. This rubric defines the expectations for high-quality instructional mathematics materials for South Carolina. It was the foundation for the entire review.

Review panelist members engaged in more than 30 hours of training on the rubric itself and on developing the skills to review instructional materials with the rubric. This includes best practices, eliminating bias, collecting evidence, and scoring.

Review panelist members reviewed instructional materials independently, going through the rubric one gateway and criterion at a time, following the gateway system. If a set of materials did not meet the expectations at Gateway 1, the review ended. If the set of materials did not meet expectations at Gateway 2, the review ended. The system is designed so that only those materials meeting the quality priorities of South Carolina make their way to the adoption list.This means that when you access the South Carolina state list, each of those programs have made it fully through all three Gateways.

Review teams came together to calibrate on their evidence and confirm final scores as a review panel. This included weekly check-ins, reviewing support and aligning on evidence.This time is a critical piece in reviews to ensure that reviews are comprehensive, free from bias, and reflect consensus across multiple reviewers. Their evidence and final scores were then used to develop the appraisal of each title. This appraisal is the final report for each grade-level and/or course for each reviewed program. These are the scores and reports that are shared with the state as final adoption recommendations.

The process is designed to keep recommendations objective and rooted in the evidence from the program, with the lens of the markers of quality.

14 of 19

Who was on each Review Team?

Reviewer

3-8 per team
Review materials and collect evidence
Collaborate with QA to assist in writing appraisals

Quality Analyst

1 per team
Provide content support to review team, facilitate the process, monitor pacing, identify and resolve score discrepancies, review evidence, write the team’s appraisal of title

14

We just learned what those reviewers and review teams did to prepare and engage in full review of programs, now let’s look at the makeup of those teams.

Each review panel includes a set of reviewers and a quality analyst. Those teams include 3-8 reviewers from across the state with varied expertise and backgrounds, depending on the program and grade-spans included in the call to bid. Their role is specifically to review materials and collect evidence. They use what they have found to support the final writing of the appraisals.

There was one quality analyst per team. They provide support to the review team and facilitate the process at the team level. This includes ensuring that all reviewers come to consensus with supportive evidence to culminate in final scores and appraisals of each program.

Additionally, staff from the state department and EdReports support the process and review teams along the way.

15 of 19

SC Instructional Materials Review & Adoption Process

15

The full SC Instructional Materials Review Process includes the review and consideration of core, comprehensive materials as well as supplemental materials.

The first thing that review panelists do is to determine whether each program is a core, comprehensive full academic course of study or if it is a supplemental program.

Materials that are core, comprehensive programs were then moved forward in the full review process using the comprehensive rubric. Review panelists then engaged with their team in a review of the set of materials as aligned to the review structures and processes set out in the rubric. If the program exemplifies expectations, it is recommended for adoption. If the program does not meet expectations, it is not recommended. The state then utilizes those recommendations for final adoption decisions.

Materials that are supplemental are reviewed against a Supplemental Review Checklist. Because each supplemental program has its own area or areas of focus and intention, it is reviewed for what it claims and what it provides. Review panelists review and take notes of program features and share their recommendation for the program to be included on the supplemental list or not.

16 of 19

6-Step Adoption Process

We believe that it is not only what you adopt, but how you adopt that matters. These programs are high quality, and different programs may better suit your district based on data, your students, and your priorities.

The state list is a way to narrow down which programs districts consider by weeding out those programs that to do meet the baseline markers of quality for South Carolina as indicated by the rubric - standards alignment, rigor, coherence, student and teacher supports. The purpose is to ensure that districts are able to really focus on investigating high-quality options and then deciding on an option that best meets their local needs. As you can see here, there are steps for districts BEFORE they utilize the state list, and after.

In our next session, we will dig deeply into the 6-step adoption process, and how districts can build an intentional, teacher and student centered process that identifies their priorities, narrows down options from the state list based on those priorities, and provides structures to investigate those options to support their selection decision.

17 of 19

Statewide Webinar Series

17

Why Materials Matter	October 16, 3:30 - 4:30 pm
The SC Review Tool & List Development	November 13, 3:30 - 4:30 pm
Preparing for a Smart Adoption	December 4, 3:30 - 4:30 pm
Applying the Smart Adoption Process to the State List	January 22, 3:30 - 4:30 pm

This is Part 2 of a 4-part series. All webinars are recorded, and those recordings and presentations are uploaded to the SC site within a few days.

Our next webinar is on December 4. If you are eagerly awaiting to find out the final SC Math List - there is SO MUCH you can and should be doing between now and January when the list is available. An effective process starts BEFORE looking at programs, and we will be here on December 4 to help you navigate what to do - from establishing your process to identifying your priorities so you know how to make the best choice for your school or district. Join us then!

And then our final webinar will be looking at your next steps for a smart adoption process once you have the list available so that you can get the best set of materials for your students and teachers in your hands and ready for implementation!

18 of 19

Public Review

November 18, 2024 - January 3, 2025

Virtual Links: Instructional Materials - South Carolina Department of Education

Location Sites: Bob Jones University Library, Coker University Library, Columbia College Library, Columbia International University Library, Converse University Library, Horry Georgetown Technical College Library, and Winthrop University Library

18

19 of 19

Do you have questions or want to learn more?

EdReports:

abikowski@edreports.org

South Carolina:

nmbarker@ed.sc.gov