1 of 16

Winning isn't everything: �Evaluating Advocacy that Builds Power

Mariah Brothe Gantz, Julia Coffman, Melly Kitenge

Center for Evaluation Innovation

Sheniqua Jeffrey

Center on the Ecology of Early Development

Miranda Yates

Robert Wood Johnson Foundation

American Evaluation Association Conference

November 10^th, 2022

New Orleans

2 of 16

We know that advocacy for policy and systems wins matters, but when we think about the long, complex arc of social equity and justice, we need to expand our evaluation boundaries to analyze power.

3 of 16

3

A win (or loss) is the means, and not just the goal.

4 of 16

Advocacy that builds power varies from advocacy focused solely on a win.

4

Advocacy to Achieve a Win	Advocacy to Win and Build Power
“Win” is the goal	“Win” is a means
On behalf or for impacted communities	With and by impacted communities
Organizing groups may not be part of advocating	Organizing groups are centered in advocacy
Shared analysis driven by the politics and window of opportunity	Shared analysis grounded in root causes and an examination of inequities
Policy solution may or may not be informed by impacted communities	Policy solution developed or informed by impacted communities
Strategies feature messaging about impacted communities, often within the dominant frame	Impacted communities drive narratives that tell their stories and challenge dominant frames

Advocacy to Achieve Win

“Win” is the goal
On behalf of impacted communities
Base building organizations may or may not be part of coalitions and campaigns
Strategy developed and led by a single or group of advocates, base building organizations may or may not be part of the strategy development
Shared analysis driven by the politics and window of opportunity
Policy solution may be developed based on political feasibility and may or may not be informed by impacted communities
Narratives tell the stories of impacted communities

Advocacy that Builds Power

“Win” is a means to build power
Impacted communities represent themselves
Impacted communities and base building organizations are centered in the campaign structure
Base building organizations help develop the strategy and may lead it; strategy actively seeks to build community engagement; strategy seeks to expand the base
Shared analysis grounded in roots causes and inequity
Policy solution developed or informed by impacted communities and seeks to address inequities
Impacted populations drive the narrative to tell their stories and change dominant frames

5 of 16

To what extent is an �assessment of power �showing up in your advocacy evaluation work? �How does or could it look in your work?

5

6 of 16

In 2021, the Robert Wood Johnson Foundation funded two 18-month projects to support policy advocates working on EHS/HS & IDEA to center parent voice in advocacy efforts in 9 states.

7 of 16

7

Evaluation Design

Proposal Review

Reviewed State Proposals, Observed Debriefs

Ecosystem Study

Interviews with National Partners, State Partners, Organizers, and Ecosystem Organizations

Parent Perspectives

Interviews with Parents

Learning Moments Throughout

Learning sessions with RWJF, Per State Case Studies, Cross-state Findings

Center on the Ecology of Early Development and CEI partnered to design and carry out this learning and evaluation work. My colleague Sheniqua will speak more on this partnership.

We structured the evaluation around four main components. The components provided a regular opportunity to convey newly gathered, real-time data to RWJF to help inform their grantmaking. Additionally, the evaluation was designed to provide key learnings back to the national partners, state partners, parents, and organizers.

We started with a proposal review where we looked at state proposals and observed debriefs which included perspectives from parent advisors, national, and state partners. �Next, we wanted to understand more about the state’s advocacy ecosystem including the key participants, infrastructure, and capacities. We spoke with state leads, parent organizers, and other state organizations and partners in the ecosystems. This supported the creation of state-by-state case studies which documented the experiences of these states and shared learning from across the ecosystems. We also took the time to speak with national partners who spoke about their experiences in the grant and about the state’s progress.

Finally, CEED conducted conversations with parents. My colleague, Sheniqua will speak more about their work.

We also offered several learning moments including a full report with cross-state findings, individual state case studies, and regular learning spaces with RWJF.

8 of 16

Evaluation Question:��How are Head Start/Early Head Start and IDEA early childhood advocacy ecosystems developing the capacity to achieve important policy wins and to center families and parent advocates in the work?

8

9 of 16

Evaluation Partnership

9

Advocacy Evaluation Expertise

Racial Equity and Early Care and Education Policy Expertise

Formation of the CEI and CEED partnership and how this partnership fostered the implementation of the grant

impacted our approach (would be different from a purely developmental perspective)
deep context work
thinking about how to use data
Partnership created by RWJF- Gina Hiijawi started it because of the complementary expertise. CEI brings expertise with advocacy evaluation and advocacy that builds power. CEED brings experience with IDEA/HS policy particularly with trying to reduce racial disparities and a focus on racial equity.
CEED’s expertise has been helpful from racial equity and understanding the ecosystems surrounding IDEA/HS. For each of these policies.
We understand advocacy ecosystems.
The ECE advocacy is different than many other types of advocacy. Who is in that ecosystem and how they relate to one another is distinct.

10 of 16

Parent Perspectives on Advocacy

10

Speaking directly with parents:

Highlighted that parent involvement in advocacy was driven by personal motivation, power in advocacy and strong relationships with organizers

Debunked assumptions about parent commitment

Informed the focus for the next phase of the grant

Key Notes about the parent perspective, and how this informed the next steps of the grant

We wanted to Offer parents a space to speak for themselves about their experiences and perspectives on parent engagement in their state advocacy efforts

We interviewed 14 parents, representing about half of the states, and predominantly in headstart
All but two were moms of color and most had prior experiences with advocacy

Parents expressed several points around engagement and barriers to engagement. I am going to highlight a few.

Some Takeaways around parent perspectives on Advocacy were:

Primarily parents were motivated by their personal connections to each program, but most importantly they mentioned that groups were:

Welcoming and Supportive (“Advocacy groups are welcoming and supportive”)

“I can't think of a time that I…that I or a part of our group that we have said something and it went ignored. Even if the answer or the response that we got wasn't the desired response, … there was still a response there. And so, you know, it feels good to know that you have people who care, at least to hear your voice. Whether or not they could do anything about it is different.” – Parent Advocate

Power Building

This example also shows how parents realized their voice matter. And this gave them power

DEBUNKING

Parents were clearly juggling multiple responsibilities between childcare, work, and their personal life in addition to participating in advocacy
In some instances, organizers were reluctant to even ask for parent time to participate in the evaluation efforts
YET parents were more than happy/very eager to talk and share their stories with us

NEXT PHASE

Speaking to parents directly was also important as well to the next phase of the grant.

RWJ was able to learn so much from this work, that they committed to providing an additional year of funding to this work, with a focus on issues that parents want to address, such as a broader focus on economic inclusion for children and families, increasing involvement among other parents, and compensation, professional development

Our goals:

interview more parents

Now im going to pass it to Miranda who is going to share on the funders side?

11 of 16

Robert Wood Johnson Foundation

11

Implications from the funder’s perspective

Shared commitment to Transformation

Excitement about what might become possible through Emergence

Intentional Partnerships

RWJF is the largest philanthropy focused on health in the US. Our vision is that we, as a nation, will strive together to build a Culture of Health enabling all in our diverse society to lead healthier lives, now and for generations to come. Our focus is on health equity defined as everyone has a fair and just opportunity to be healthy.

I have a couple minutes to share about the funder perspective regarding this project and approach. And so, wanted to highlight three concepts that stick out for me about this project and which I think have general implications as funders and evaluators develop and carry out projects together.

· Shared commitment to Transformation

· Excitement about what might become possible through Emergence

· Creating intentional Partnerships in multiple spaces bringing together individuals and groups with different perspectives and expertise.

1. RWJF’s work is about supporting systems level change and TRANSFORMATION.

· And what we are learning is that transformative efforts require transformation in all spaces, including in how we fund, learn, and partner.

· This project reflects this mindset in how it challenged assumptions and orthodoxies, centered what’s at stake in the work and offered a new paradigm for advocacy

2. Another concept that I wanted to highlight is EMERGENCE and the openness to an approach and design that invites and anticipates feedback loops, iteration, and changing course. What CEI and CEED did in developing this project was to lay out objectives and a design involving deep engagement with grantees and proposed phases, methods, and activities. As we then got into the project, they engaged in regular collective reflection which led to refinements and adjustments. This collaborative and highly communicative approach helped us in foundation to join in the process and to feel excited about how things would evolve.

3. Finally, this project involved lots of different types of intentional PARTNERSHIP. This included the evaluation team of CEED and CEI as well as the RWJF staff with the pairing of Krista Scott who brought content expertise in early childhood policy making and program development and provision and myself with a focus on community engaged and participatory evaluation methods. This project has further bolstered our commitment to supporting partnered eval designs, including helping to make these connections.

12 of 16

12

Finding 1: All states made structural shifts to better center parents in their advocacy efforts.

Finding 2: States that progressed the most on centering parent voice used an organizing approach, rather than mobilizing.

Finding 3: States making the most progress on engaging parents impacted by structural racism are using an organizing approach led by people of color.

Finding 4: States have had to work on engaging ecosystem partners not yet fully aligned about how and to what extent parents should be engaged.

Findings

I’d like to highlight 4 findings from this evaluation.

Finding 1

To engage parents directly, state advocacy partners either hired parent organizers or built new or stronger partnerships with existing parent-focused organizers or organizations.

Finding 2

We found states that adopted an explicit organizing stance moved farthest toward the goal of centering parents in their advocacy work.

Finding 3

States that have done the most on this goal, partnered with or created parent-focused groups that are led by people of color who are using an organizing approach that prioritizes relationship-building, healing, and leadership development.

Finding 4

States that have made progress in shifting mindsets among skeptical ecosystem partners have offered opportunities for direct conversations with parents.

Now, I will turn it over to my colleague Mariah who is going to lead us through a couple of questions for discussion.

13 of 16

How can we evaluate advocacy in ways that build power?

What supports are needed for this type of evaluation?

13

AEA 2022: (re)shaping evaluation together

14 of 16

Implications for Future Advocacy Evaluation

14

15 of 16

Lessons for Evaluators

15

Lesson 1: Transformative initiatives, particularly those started by funders, that challenge status quo power dynamics will take time, starting slowly among policy advocates and then gaining momentum over time.

Lesson 2: One year was enough time to see substantial traction and momentum, but too short a time to see if that traction will sustain and continue to grow.

Lesson 3: Structural changes in advocacy ecosystems are important to track, alongside if and how those changes are accompanied by a shift in power dynamics.

16 of 16

Lessons for Evaluators

16

Lesson 4: The early stages of new organizing efforts or partnerships are likely to involve small numbers of individuals, but the quality of the relationships are more important.

Lesson 5: During the second year of new organizing efforts or partnerships, leaders are likely to emerge.

Lesson 6: Evaluators have power in an advocacy ecosystem. Advocacy evaluators must attend to how evaluation is perceived by stakeholders and can do so by being sensitive and focusing on learning and documentation. ��