1 of 54

Universal Screening

What administrators need to know

1

2 of 54

Introductions

Associate Professor of Special Education at University of Maryland
Former Research Scientist at University of Oregon, Center on Teaching and Learning
Degrees in special education and school psychology
Research interests: formative assessment, MTSS, supplemental literacy intervention for SWD and emergent bilingual learners

2

3 of 54

3

Centralize

Access to collections of evidence-based resources is needed.

Access to dyslexia screeners, especially before Kindergarten, is needed.

Information on early identification and access to “structured” literacy support is needed.

Disrupt the compartmentalization within disability communities to promote mutual understanding and joint action.

Translate

Define educational terminology in “parent friendly” ways for families to use when preparing for and participating in IEP meetings.

Work with teacher preparation programs to inform and train pre-service educators on dyslexia, especially at middle & high school.

Work with states, districts, and schools to inform and provide in-service training to educators on dyslexia, especially at middle & high school.

How to successfully navigate the educational system.

Localize

It is critical to build and strengthen family-school partnerships.

Families mainly turn to other families and organizations in their community for information; many form local “grass roots” coalitions. Community-oriented solutions are appealing.

It is important to include virtual means as part of the conceptualization of “localize.”

A top-down and bottom-up approach to addressing family-school partnerships is needed.

4 of 54

Objectives and Agenda

Overview of universal screening in literacy
How administrators can use output from screening measures to make decisions
The use of school-based data teams to guide effective use of screening data

Key roles administrators can play in the roll-out of high-quality universal screening

4

5 of 54

Overview

What is screening?

Why should we do it?

6 of 54

Why Assess?

We want to…

Determine student skills
Evaluate our systems of support
Assign resources

Universal screening provides the data to make these decisions – but only if we utilize it

6

(c) NCIL, 2018

7 of 54

Language of Assessment

Screening, Benchmark *
Formative *
Interim
Norm-Referenced
Criterion-Referenced*
Progress Monitoring
Screening*
Standardized*
Summative
Diagnostic

7

(c) NCIL, 2018

There are three types of assessments commonly used within an RTI framework: summative, diagnostic, and formative.

Summative assessments occur after instruction and are assessments of learning. . Summative assessments measure what students learned over a period of time. They are typically administered after instruction and can help to determine what to teach, but not how to teach. Summative assessments are typically administered to all students and are often used for accountability decisions and to determine the mastery of certain skills or standards. While summative assessments can also be used to allocate resources, because they occur after instruction, the decision is reactive.

Diagnostic assessments occur before instruction and help to identify skill strengths and weakness. Diagnostic assessments are measures of students’ current knowledge and skills and can be used to identify a suitable program of learning. They are administered before instruction occurs to assist in identifying appropriate instruction and interventions. Because diagnostic assessments provide detailed information about individual student learning, they are most effective in helping teachers understanding the needs of specific students rather than all students and are therefore they are typically administered to only some students. Diagnostic assessments can help teachers determine what to teach, thereby supporting the selection of appropriate interventions. Diagnostic assessments provide useful information but should be used sparingly, as they can be very expensive and time consuming to administer.

Formative assessments occur during instruction and are assessments for learning. Formative assessment is a form of evaluation used to plan instruction in a recursive way. Formative assessments tell us how well students are responding to instruction. With formative assessment, student progress is systematically assessed during instruction to provide continuous feedback to both the student and the teacher concerning learning successes and failures. Formative assessments are typically administered to all students during benchmarking (screening) and some students for progress monitoring.

Formative assessments may be informal or formal. Informal assessments are not data driven but rather content and performance driven. Examples include observations and teacher-made assessments. Formal assessments have evidence of reliability and validity. These formative assessments typically follow a standardized administration format. Scores such as percentiles, stanines, or standard scores may be obtained from this type of assessment.

Norm-referenced test include tests like the ACT, SAT, and GRE. Students’ scores are compared with each other using percentile scores. For example, a student who scored in the 64^th percentile scored better than 64 out of 100 students who took the test. S/he scored lower than 36 out of 100 students who took the test.

Criterion-referenced tests are often standards-based tests that compare a student’s performance to a criterion for mastery of a skill or objective. The score indicates whether the student met the mastery criteria, and so it often looks like a pass/fail or proficiency score.

8 of 54

What is screening?

8

Tier 1, Core Support

Tier 2, Supplemental Support

Tier 3, Individualized Support

(c) NCIL, 2018

Raise your hand if either of these figures look familiar

MTSS is a framework to ensure successful education outcomes or ALL students by using a data-based problem solving process to provide and evaluate effectiveness of multiple tiers of integrated academic, behavior, and social-emotional instruction/intervention supports matched to student need in alignment with educational standards.

Tier I = Implementation of empirically supported core programs and practices that have been demonstrated to produce good outcomes for the majority of students. An overall goal is to have 100% of students achieving at high levels but Tier I is considered effective if at least 80% are meeting benchmarks.

Tier I instructional goals and questions might include things like: What do we expect all students to learn? How do we know if they have learned what we expect them to learn? What do we do if they haven’t learned what we expect them to learn?

Tier II is what we consider to be supplemental and targeted instruction beyond Tier I Core instruction for about 15% of students. Tier II is considered effective if at least 70% of students improve performance in terms of closing the gap toward benchmark or improving according to progress monitoring standards. Things that influence Tier II questions are: Where is the student now? Where do we want them to be? How long do we have to get them to the target? What are the resources we have to get them to the target in the amount of time we outline?

Tier III is for the 5% of students who are highest risk and receive iii (intensive individual instruction).

9 of 54

Guiding Principles

Prevention
If the triangle is “tipped,” focus on big areas for change (Tier 1, Tier 2) before individual
District-level administrators should review data quarterly (fall, winter, spring) focus on resource allocation, program evaluation, attend one data meeting per year per school
School-level leadership should attend monthly data meetings, review screening + progress monitoring data
We’re all in this together

9

10 of 54

The universal screening process

Identify a reliable and valid measure that is appropriate for your school context
Assessors receive initial high-quality training and become reliable
Teachers and administrators learn basic administration and scoring rules, and how the data from screeners map to instructional decisions by reading reports
Logistics are put into place to initiate and sustain the system
Data are collected
Data are used at the district, school, class, small group, and individual levels
Interim refresher trainings are planned, reliability data are collected at each screening period

10

11 of 54

Questions to ask after each screening period

What?

How many of our students are on track to meet the next goal?
How many of our students are at risk of not meeting the next goal?

Why?

Does the proportion of students on track vary by school/class?
What supports do we need to focus on improving (Tier 1? Tier 2?)

How?

Are we making progress on our (class/school/district) goals from year to year? *
Are we reducing risk level and improving student literacy within and across years?

11

12 of 54

These questions align in a Problem-Solving Model

12

13 of 54

How many of our students are on-track to meet the next goal?

13

14 of 54

What supports do we need to improve?

14

15 of 54

Are we making progress from one year to the next?

15

16 of 54

Are we reducing risk level?

16

17 of 54

Universal Screening data can also be used to forecast outcomes on high-stakes tests

State

Colorado
Florida
Illinois
Michigan
N. Carolina
Oregon
Washington

Pass/No Pass

90% (57%)
91%
99%
72% (77%)
100% (100%)
99%
90% (41%)

17

18 of 54

Timeline for Administration

19 of 54

19

Critical Feature of Literacy	DIBELS Measure*
Phonemic Awareness	Phoneme Segmentation Fluency (PSF)
Phonics	Nonword Fluency (NWF)
Fluency	Word reading fluency (WRF) Oral Reading Fluency (ORF)
Comprehension	ORF, DIBELS Maze (Daze)
Vocabulary and Oral Language	Not yet available

*Letter naming fluency is also part of the DIBELS suite of measures, but does not connect to a critical feature of literacy

20 of 54

Making Screening Decisions

Part III

20

21 of 54

We use a Problem-Solving Model to Answer Specific Questions

21

22 of 54

Step 1. Is there a problem? What is it?

Reports: Distribution, Histogram, Class list

22

23 of 54

Tale of Two Schools

School A

52% low risk
33% some risk
14% at risk

School B

87% low risk
6% some risk
6% at risk

23

24 of 54

Is there a problem? What is the problem?

24

25 of 54

Is there a problem? What is the problem?

25

26 of 54

Step 2. Why is the problem occurring?

Reports: Disaggregated distribution report, instructional grouping report

26

27 of 54

Why (and Where) is the problem occurring?

27

28 of 54

Why (and where) is the problem occurring?

28

29 of 54

Step 3. What are we going to do about the problem?

Prioritize (Tier 1? Tier 2? Tier 3)
Develop a plan
Implement a plan
Evaluate

29

30 of 54

Step 4. How is the plan working?

Reports: Within- or cross-year boxplots, summary of effectiveness report

30

31 of 54

How is the plan working?

31

32 of 54

How is the plan working?

32

33 of 54

How is the plan working?

33

34 of 54

Question 1a

Review the sample district distribution report -focusing on the composite score

What is the shape of the distribution? (triangle, flipped, column)
Are there differences in risk levels by race or ethnicity?

34

35 of 54

1b

Now turn to p. 2 of the handout – ALL grades status report for a full year
How did this district perform, k-2, from beginning to end of year?

35

36 of 54

1c

Examine last page of handout – cross-year box plot
What’s happening in this district’s data across the 5 years of implementation of a new literacy plan?
How would you feel if this was your district?

36

37 of 54

District- and School-Based Data Teams

Part IV

37

38 of 54

Composition, roles, and details

Potentially two types of teams:

data collection team whose responsibility it is to collect reliable data and enter those data
teacher or grade-level teams, who will review screening data as soon as it’s available and use it to plan, or evaluate instruction

The teams should not be mutually exclusive

38

39 of 54

Key Individuals to Include on Teams

Principals
Classroom teachers
Speech-language clinicians
Bilingual teachers
Special educators
Reading coaches
Behavior specialists
School psychologists

39

40 of 54

Who Will Collect the Data?

A school-based representative team: Principals, special educators, specials teachers, general educators, volunteers, school psychologists, reading coaches, speech-language clinicians, etc.
An outside team: Retired teachers
A district-based representative team
Teachers collect their own data, either in their classrooms or within grade
What data team composition will you use?

40

41 of 54

How Long Will It Take? First Grade

Time of Year / Measure(s)	Time per Pupil	Number of Data Collectors	Pupils Assessed per 30 Minute Period
Beginning LNF & NWF	7 min.	1	4-5
		2	8-10
		4-5	12-15
		6-8	16-25
Middle NWF, & ORF	7 min.	1	4-5
		2	8-10
		4-5	12-15
		6-8	16-25
End of Year NWF & ORF	7 min.	1	4-5
		2	8-10
		3	12-15
		4-5	16-25
		6-8	24-40

42 of 54

Question 2.

What data-team approach are you likely to use in your building and why?
What data-use teams are you likely to use in your building and why?

42

43 of 54

How to sustain data collection for the long haul

Initial training by an expert

Up to 16% of a student’s ORF score depends on who tests them (Cummings, et al., 2014); in the DIBELS data system, 24% of school administrators reported no training opportunities were provided to staff (Munir, et al., 2012)

Refresher training at the school with practice opportunities before each screening period

Typically led by a lead member of the team, veteran teacher, special educator, etc.

Random reliability checks during and after testing (minimum 10%)
Debrief after testing at the school with any administration questions
Repeat Steps 2-4 before any screening period
Identify mechanisms to train new staff

43

44 of 54

Questions 3-4

What can you do to promote consistency in data collection and the effective functioning of data teams?

44

45 of 54

Data teaming for school success

Team Initiated Problem Solving (TIPS)
Use existing meeting times
Review screening data three times per year
Review progress monitoring data for students in supplemental intervention monthly
Make changes to intervention after an average of 8 data points
Need a school-wide team because literacy is a school-wide issue

45

46 of 54

The importance of communication and �data use

Interpreting results from screening
Common misconceptions if data indicate a problem

I should teach nonwords
I should teach the student to speed read
I should practice the test
…

DIBELS are indicators, teach comprehensively screen narrowly

46

47 of 54

Administrators set the tone

Data should be used for collaborative planning, not gotchas
Celebrate ALL successes!
Promote transparency
Hold regular meetings to discuss data– don’t wait until there is a problem

One example, Teacher Assistance Teams (Tiers 1 and 2), Student Assistance Teams (Tier 3)

47

48 of 54

District-level collaboration

Facilitates district-level planning and coordination
Reduces redundancies
Examples

Fall kick-off after screening data are collected, goal setting
Spring check in, glows and grows

48

49 of 54

What happens between screening periods?

Instruction!
Continuum of resources (Universal, small-group, Tier 2, Tier 3, special education)
Progress monitoring for those students who are below the benchmark scores
Deeper testing for students who are not making progress

49

50 of 54

Decision-Making Process (Iris Center)

Universal Screening – takes place three times per year in prek – Grade 3 (minimum)
For students in k and newcomers who screen in the below benchmark range,

Monitor progress monitor in Tier 1 for 5-10 weeks and move students to Tier 2 if low progress or low scores at the end of monitoring period

50

51 of 54

Questions?

52 of 54

Next steps

What commitments do you want to make for your ELAs
What supports do you need from MSDE and MILE to make this happen?

52

53 of 54

Resources

Maryland Initiative for Literacy and Equity

Kelli D. Cummings: kellic@umd.edu

Literacy Lab

53

54 of 54

Resources

54