1 of 27

Evaluation of Los Angeles Unified School District’s Student Equity Needs Index

Jesse Levin and Iliana Brodziak

American Institutes for Research

LAUSD Committee of the Whole Meeting | November 30, 2021

AMERICAN INSTITUTES FOR RESEARCH® | AIR.ORG

| AIR.ORG

2 of 27

Agenda

Research Questions
Methods
Results
Recommendations

2

| AIR.ORG

3 of 27

Research Questions

How did schools use SENI funding?
How did the district support principals use of SENI funding?
In what ways do school leadership, teaching staff, student needs, and community support influence how SENI funds are used?
How are the SENI funds used to support classroom instruction?
What is the relationship between SENI spending and student needs, and what is the association between SENII spending and student outcomes?

3

| AIR.ORG

4 of 27

Methods

4

| AIR.ORG

5 of 27

School Funding Policy Over Time

We examined whether policy changes have the following relationships between:

funding and student needs
student outcomes and student needs

Two policy events create three distinct policy environments:

Implementation of LCFF in 2013-14 and SENI 1.0 in 2014-15
Implementation of SENI 2.0 in 2018-19

5

| AIR.ORG

6 of 27

Quantitative Analytic Approach

Identified resources (expenditures and staff positions) attributed to SENI and TSP program accounting codes
Categorized resources by use of funds:

Function: Purpose of the purchased resources
Object: Type of resources purchased
Position Type

6

| AIR.ORG

Isolated the resources (expenditures and staff positions) assigned to the SENI and TSP program codes
Identified the schools that spent dollars within those program codes
Categorized resources by use of funds to understand how schools are using their funds allocated based on the SENI

We tried to create categories that reflected how resources are most commonly used, but that were also specific enough to identify interesting distinctions in how schools used different types of resources. To provide a multifaceted understanding of how funds are used, we categorized resources in three ways. First, we categorized resources according to function categories that describe the purpose of the purchased resources. Second, we categorized resources according to object categories that describe the type of resources purchased. Lastly, we used the position control file to categorize personnel resources by position type.

we conducted analyses to understand how schools use SENI and TSP funds on average. These analyses consisted of calculating expenditures per student within certain expenditure categories, the share of spending (as a percentage of total SENI and TSP funds) used in certain expenditure categories, and the share of staff (as a percentage of full-time equivalent staff (FTEs) or staff cost) in certain expenditure categories.

7 of 27

Quantitative Analytic Approach

Compared how SENI/TSP spending varied across:

Schooling levels
Local school districts
Schools with different student needs (percentage of low-income students, ELL students, etc. )

Used statistical analysis to examine changes in the following relationships associated with policy shifts:

Student needs and spending
Student needs and student outcomes

7

| AIR.ORG

In addition to understanding the average use of SENI and TSP funds across all schools, it is important to understand whether certain types of schools use funds differently.

To examine whether and how the use of SENI and TSP funds varies across schools, we compared resource use across schools with different attributes or characteristics.

We grouped schools by grade levels served to examine whether elementary schools spent SENI and TSP funds differently than middle schools and high schools.

We also grouped schools by local district to examine whether resource use varied across local districts (Exhibit 4). Lastly, we used regression analysis to examine whether schools differed in their resource use according to the needs of students (defined as the unduplicated pupil percentage)

We ran separate regressions by school level and included controls for school size and the percentage of special education students.

8 of 27

Qualitative Analytic Approach

AIR identified 50 schools performing better than expected based on their student needs and spending (efficient schools)

Estimated the relationship between student outcomes, student needs and school spending per-pupil

Principal Interviews: 4 schools were selected by each local district to participate in principal interviews

Ensured sample contained one school per schooling level (elementary, middle and high) within each local district

Case Studies: Selected 6 case study schools (one from each local district)

Interviews with teachers, key staff, school-site council members and parents
Additional interviews with principals

8

| AIR.ORG

9 of 27

Research question 1: How did schools allocate resources supported by SENI and TSP funding?

9

| AIR.ORG

10 of 27

SENI funds were consistently spent on general instruction, instructional supervision, and school administration

10

| AIR.ORG

First, more SENI dollars are spent per pupil at middle and high schools compared to elementary schools. High schools also receive a sizeable boost relative to elementary and middle schools in TSP funding. With respect to use of funds by function category, a larger share of both SENI and TSP dollars are spent on guidance and counseling as schooling level increases. For example, in elementary schools 7% of SENI dollars and less than 2% of TSP dollars were spent on guidance and counseling. The shares of spending increase to 16% and 11% of SENI and TSP dollars, respectively, in middle schools; and further increase to 17% and 33% of SENI and TSP dollars, respectively, in high schools. Middle and high schools also spend a larger share of both SENI and TSP dollars on school administration compared to elementary schools. These are offset by a relatively higher percentage of SENI spending on instructional supervision and administration and a much higher share of TSP spending on special education instruction in elementary schools.

11 of 27

The largest portion of SENI funds were spent on assistant principals whereas TSP funds were spent on teachers

11

| AIR.ORG

Given that more than 90% of SENI and TSP funds are used to support spending on staff salaries and benefits, perhaps the most meaningful way of looking at the use of SENI and TSP funding is by staff position types. Below we show the percentage of staff FTEs and spending by position type. For reference, 2,703 staff FTEs were purchased using SENI dollars at a cost of $241 M (or $485 per pupil) and 3,449 staff FTEs were purchased using TSP dollars at a cost of $350 M (or $704 per pupil).

Looking at FTEs and spending together also provides a sense of the relative spending of different types of positions. Positions that represent a smaller share of spending compared to FTEs represent positions that are lower than average spending per FTE (e.g. instructional and teacher aides which represent 19% of SENI FTEs and 5% of the spending of staff purchased with SENI dollars). Alternatively, positions that represent a larger share of spending compared to FTEs are those with a higher than average spending per FTE (e.g. assistant principals represent 22% of SENI FTEs but 37% of the SENI staff spending).

The percentages of different types of staff purchased with SENI funds in FY 2020 varied by schooling level, with elementary schools making noticeably different choices on average compared with middle and high schools. Notably, teachers account for only 3% of FTEs purchased with SENI funds in elementary schools compared with 17% and 18% of FTEs in middle and high schools, respectively. Middle and high schools also use more of their SENI funds for assistant principals, on average, compared with elementary schools. Furthermore, the prevalence of school counselors and office technicians also increased with schooling level. Elementary schools made up for smaller proportions of these position types, but notably higher percentages of program coordinators, instructional and teacher aides, school supervision aides, and community representatives.

In 2020, SENI and TSP funds were used to purchase a wide variety of staff types. Only 12% of FTEs purchased with SENI funds were teachers, whereas 42% of TSP staff FTEs were teachers. The most commonly purchased staff type with SENI funds was assistant principals (22% of SENI FTEs and 37% of SENI cost). The staff type representing the second largest share of the SENI cost (19%) were program coordinators, which includes TSP program advisers, intervention support coordinators, middle school college and career coaches, and several other position types. Instructional and teacher aides and school supervision aides represent sizable shares of FTEs purchased with SENI funding (19% and 11%, respectively) but represent much smaller shares of the cost (5% and 4%, respectively). Instructional and teacher aides were also frequently purchased with TSP dollars as well (21% of the FTEs supported by TSP funding). Schools purchased counselors with both types of funding, representing 5% and 11% of the FTEs supported by SENI and TSP dollars, respectively, and making up slightly larger shares of the cost (7% and 13%).

Program coordinators represent all positions with the “0706” position code.

12 of 27

Research question 2: How did the district support principal use of SENI funding?

12

| AIR.ORG

13 of 27

District provided training and materials but more substantive training is needed

Principals shared that the district provided:

Training and materials (e.g., budget handbooks) to support the development of school budgets
Access to district fiscal support staff to answer questions and provide feedback on budgets before they are submitted for approval by the district

Respondents recommended additional support that they would like to receive as follows:

Guidance on how to use funds strategically
An opportunity to learn from peers on how SENI funds are effectively used in other schools
Formal professional development opportunities for new principals and other key stakeholders

13

| AIR.ORG

14 of 27

Research question 3: How did school-level factors influence the use of SENI funds?

14

| AIR.ORG

15 of 27

Elementary schools with a higher incidence of unduplicated pupils tended to spend a larger percentage on instructional supervision and administration

15

| AIR.ORG

16 of 27

Higher need elementary schools tended to spend larger shares of SENI funding on assistant principal and program coordinator staff

16

| AIR.ORG

17 of 27

Case study respondents reported that key stakeholders provide input on how SENI funds are used

Key stakeholders at the case study sample sites provided input on how to use SENI funds through:

SSC meetings
School staff meetings
Teacher needs assessment surveys

17

| AIR.ORG

18 of 27

Research question 4: How were the SENI funds used to support classroom instruction?

18

| AIR.ORG

19 of 27

Principals in case study schools reported that SENI was strategically used to fund specific staff positions

SENI funds were used to fund the following types of staff:

Teachers (to allow for smaller class sizes)
Tutors
Psychiatric social workers, pupil services and attendance staff, counselors, and assistant principals

These positions were added with the intention of “providing a more personalized instruction” in order to promote student:

Academic standing
Social-emotional well-being

19

| AIR.ORG

20 of 27

Research question 5: What is the relationship between SENI spending and student needs, and what is the association between SENI spending and student outcomes?

20

| AIR.ORG

21 of 27

SENI-TSP funds are positively correlated with SENI values

21

High-need middle and high schools appear to receive a bigger boost in funding from SENI and TSP compared to high need elementary schools.

| AIR.ORG

22 of 27

Impact of a 50% increase in SENI and TSP supported spending on ELA and Math SBAC scores by type of student need

22

For specific groups there was a significant positive association between SENI and TSP spending and SBAC scores for students with specific combinations of needs:

ELA – Economically Disadvantage and English Learner

Math – Economically Disadvantage, English Learner and Disability

| AIR.ORG

From left to right, students who do not have any of these three needs (“None”), students who are economically disadvantaged only (“ED”), students who are English Learners only (“EL”), students with disabilities who are not economically disadvantaged or English Learners (“SWD”), students who are both economically disadvantaged and English Learners (“ED & EL”), students with disabilities who are also economically disadvantaged (“ED & SWD”), students with disabilities who are also English Learners (“EL & SWD”), and students with disabilities who are also economically disadvantaged and English Learners (“ED & EL & SWD”). The most common need combinations are economically disadvantaged only (“ED”, 43% of students), economically disadvantaged and English Learner (“ED & EL”, 15% of students), and students with disabilities who are also economically disadvantaged and English Learners (“ED & EL & SWD”, 4% of students). The vertical lines show 95% confidence intervals for each estimate.

Gains were larger for students with multiple needs, and largest for students with disabilities who are also economically disadvantaged and English Learners (“ED & EL & SWD”).

23 of 27

Impact of a 50% increase in SENI and TSP supported spending on attendance rate and chronic absence

23

For specific groups there was a significant relationships between SENI and TSP spending and attendance/absenteeism for specific student categories:

Attendance Rate – English Learners with Disabilities

Chronic Absenteeism – students with no needs

| AIR.ORG

Average attendance rate about 90%

Higher needs-based spending is associated with slight increases in attendance rates for all need categories. Unlike with test scores, however, these improvements in attendance are not higher for students with greater needs, and are actually smaller and statistically insignificant for English Learners. We see a similar pattern for chronic absence in the bottom graph. For students with no needs, a 50% increase in SENI and TSP spending is associated with a large and significant decrease in the chronic absence rate (1 percentage point out of an average of 14%). The impact is smaller for students who are economically disadvantaged or English Learners, and there is no significant impact on chronic absence for students with the highest level of need (“ED & EL & SWD”).

the impact of a 50% increase in per pupil SENI and TSP supported spending on the likelihood of having at least one suspension during the academic year. Higher needs-based spending has no significant impact on suspensions for the average student, students with no specific needs, and students who are economically disadvantaged or students with disabilities only. For English Learners and students with multiple needs, higher needs-based spending is associated with significantly lower suspension rates.

24 of 27

Recommendations

24

| AIR.ORG

25 of 27

Recommendations

School leadership could be better served by offering principals differentiated training that is tailored to their experience in terms of budgeting SENI dollars
Providing capacity building support on the strategic use of SENI funding
Administrators may need additional guidance about carryover funds
Schools need additional supports to increase family engagement about the budgeting process
Consider reexamining and possibly simplifying the SENI

25

| AIR.ORG

School leadership could be better served by offering principals training that is more closely tailored to their experience with respect to budgeting under SENI. Principals who are new to their role and SENI would benefit from more individualized support from the district. For example, this support might include a minimum number of days of training on the accounting system and how different goods and services can be budgeted, tracked, and monitored. More experienced principals, however, may not need the basic refresher on planning and budgeting that is provided in this training. Another suggestion would be to pair new and experienced principals to provide a mentorship opportunity for newer principals.

School administrators can benefit from better understanding how purchases made using SENI funds are having the intended outcomes for their TSPs. One way that the district can support schools is to host a training about strategic use of SENI funding across local districts. Because local districts each have their unique needs, local districts can invite principals who have used SENI funding in new and innovative ways to present on the methods they used at their school and their successes. Other school administrators who have similar school needs can learn from these conversations, including, for example, how to use multiple data to inform their SENI spending and strategies to include community voices.

26 of 27

Future Research

Investigate the impact of the future increase in SENI funding stemming from the Equity Is Justice 2021 resolution on equity and student outcomes
Analyze patterns of spending across higher and lower needs schools
Identify patterns of principal decision making that predict higher efficiency

26

| AIR.ORG

Investigate the impact of the future increase in SENI funding stemming from the Equity Is Justice 2021 resolution on equity and student outcomes. Given the relatively small increase in equity resulting from the introduction of SENI 2.0, further investigation is needed to see whether equity and student outcomes respond more strongly to the upcoming funding changes made in 2021–22 and how schools use this additional funding.

Analyze patterns of spending across higher and lower needs schools, by school level. We found significant differences in spending on staffing positions by student needs and school level, particularly on assistant principals, program coordinators, and instructional and teacher aides. Further investigation is needed to understand these patterns (e.g., whether they reflect a greater need for personalized instruction at higher need high schools compared with lower need high schools and elementary schools).

Identify patterns of principal decision making that predict higher efficiency. We did not find clear differences in resource allocation by school efficiency, which suggests that principals at more efficient schools seem to be allocating resources in a similar fashion as their less efficient counterparts but have better student outcomes. Given the relatively small size of the case study sample, our analysis was unable to investigate more deeply the programmatic features and specific qualities of the staff and nonpersonnel that distinguish more and less efficient schools. In turn, more research is needed to investigate differences in programming and how resources are being used in ways that lead to better student outcomes.

27 of 27

Iliana Brodziak de los Reyes

Project Director

ibrodziak@air.org

Jesse Levin

Principal Investigator

jlevin@air.org

AMERICAN INSTITUTES FOR RESEARCH® | AIR.ORG

Notice of Trademark: “American Institutes for Research” and “AIR” are registered trademarks. All other brand, product, or company names are trademarks or registered trademarks of their respective owners.

Copyright © 2021 American Institutes for Research®. All rights reserved. No part of this publication may be reproduced, distributed, or transmitted in any form or by any means, including photocopying, recording, website display, or other electronic or mechanical methods, without the prior written permission of the American Institutes for Research. For permission requests, please use the Contact Us form on AIR.ORG.

| AIR.ORG