​

Scaling RFA’s State and Local Workforce Fellowship

Advancements in Evidence-Based Workforce

​

​

1

March 2023

2

STATE & LOCAL WORKFORCE FELLOWSHIP

​

Since 2019, RFA’s Workforce Fellowship works to:

​

1. Advance job quality for all workers through evidence-based strategies.
2. Improve workforce outcomes through evidence-based workforce spending

​

Fellows have access to:

​

• A peer network of leading workforce officials.
• Expert technical assistance to implement evidence-based solutions.
• Resources to spotlight successes and continue to build momentum.

RFA’s Workforce Program Activities

THOUGHT LEADERSHIP + COALITION BUILDING

​

Federal Policy

• WIOA reauthorization
• New federal definition of evidence
• Job quality policies
• TA to federal agencies

STORYTELLING

​

• Strategic communications (blogs, podcasts, social media, newsletters)
• Public goods: Job Quality Playbook, Procurement & Grant Reform Guide, and case studies. Linked to EM Catalog.
• Partner with practitioners in government to promote equity, evidence use, and procurement.

3

RFA Workforce Fellowship

• In 2019, RFA launched a State and Local Workforce Fellowship to help state and local government leaders make progress in improving employment outcomes for residents by using data and evidence in decision making.
• Members of the Fellowship represent state and local workforce, economic development, and supportive services agencies, and governors’ and mayors’ offices.
• The Fellowship provides access to:
• A peer network of leading workforce officials.
• Expert technical assistance to implement evidence-based solutions.
• Support and resources to elevate and spotlight successes and continue to build momentum.

4

Workforce Fellowship Phase II (2021-2022)

​

• 30 Fellows
• 7 states

5

Workforce Fellowship Phase III: Scaling

(2022-2023)

​

• 2 concurrent cohorts
• 67 Fellows
• 17 Teams
• 12 states (+ DC)

6

​

​

​

Advancing Job Quality Cohort

Advance job quality by developing a job quality framework to employ evidence-based interventions, ensuring workforce programming invests in programs, partners, and businesses that have or lead to high quality jobs.

​

Fellowship Tracks

​

​

​

​

​

Evidence-Based Workforce Spending Cohort

Use evidence- and outcomes-based grantmaking, contracting, and budgeting to set goals, define performance metrics, develop an evidence framework, link funding to outcomes, support equity and innovation in funding models, and actively manage grants and contracts.

​

7

TRAININGS AND CONVENINGS

​

Fellows participate in monthly trainings specific to their cohort as well as two cross-cohort convenings.

​

Trainings and convenings provide additional, tailored resources and opportunities to incorporate external input into team projects.

TEAM PROJECT

​

When applying for the fellowship, each team identified a collaborative project. The primary goal of the fellowship is for teams to make progress on these projects.

​

COACHING CALLS

​

Fellows attend a monthly check-in call with their team and RFA and partner staff with expertise in the specific project area.

​

These conversations spur dialogue and move the team project through an action plan. Team members leave these calls with action items that contribute to project goals.

What does fellowship participation look like?

Action Planning

Iteration & Progress

Learning & Implementation

2022 FELLOWSHIP

ACHIEVEMENTS

​

8

9

Evidence Frameworks

• Evidence Frameworks:
• Three states - Colorado, Pennsylvania, and Texas - developed evidence frameworks to guide their future budgetary, contracting, and grantmaking processes.
• Evidence frameworks can be leveraged during budget development and as part of contract or grant award processes, ensuring that funds are directed to programs that achieve important workforce outcomes.
• Critical to identify where additional evidence is needed and to invest funds in new evaluations, especially for communities that have had less access to data and resources to conduct research.

​

* See the appendix for an overview of the frameworks developed by each state.

10

Outcomes-Based Procurement

• Outcomes-Based Procurement: Contracts or grants that provide funding based on achievement of specific outcomes.
• Workforce outcomes are often related to earnings, employment, career advancement, or skills development and can be linked directly to the outcomes of a specific population to better focus funding.
• Fellows in California, Colorado, Ohio, Pennsylvania, Texas, and Virginia have all leveraged evidence- and outcomes-based procurements to better serve their job seekers and employers.
• State and local examples in presentation appendix

11

• California and Colorado fellows developed job quality frameworks to guide and advance their work.
• San Diego Workforce Partnership published their job quality framework and additional guidance on How to Build Job Quality into Your Workforce Development Approach
• Colorado Workforce Development Council created their job quality framework and launched job quality data pilots internally and with employers in five counties across the state.
• State and local examples in presentation appendix

​

Job Quality Initiatives

12

Job Quality Data Pilots

• In 2021, RFA began a partnership with Working Metrics, a workforce analytics company developed in partnership with the Aspen Institute Good Companies/Good Jobs initiative, to implement job quality pilots among Workforce Fellows.
• Several Fellowship teams are implementing job quality pilots, including CO, OH, PA, TN, and VA. RFA is working with teams to develop strategies for improving job quality in areas revealed through the data analysis.
• Using their software platform, Fellows are able to:
• Measure, track, and score job quality and equity performance against industry and regional peers.
• Embed data collection and analytics into processes like contracting, grant management, job placements, etc.

13

Federal Policy Engagements

• Next Generation of Evidence (partnership with Education Trust)
• Engage diverse leaders in the evidence movement
• Take stock of current definitions of evidence
• Generate a new proposal for the field that centers equity as it promotes innovation and scales proven results.
• Workforce Legislation
• WIOA Reauthorization
• Job Quality
• Served on the Department of Labor’s Job Quality Measurement Working Group
• Hosted feedback session for DOL’s Job Quality Framework with RFA Workforce Fellows
• Presented at DOL’s Good Jobs Summit

2023 MILESTONES

14

15

Job Quality Playbook

• Online tool for workforce and economic development agencies to operationalize job quality
• Interactive format, the tool breaks job quality into actionable and achievable steps
• Users learn about:
• Recommended actions for particular job features (ex. wages, benefits)
• Potential challenges
• Who they might partner with or learn from
• State and local leading examples
• Interwoven with fellowship’s job quality content

December 2022: Soft launch

January 2023: Public launch

16

Procurement and Grant Reform Guide

• Procurement has been key focal point for RFA’s State & Local Workforce Fellowship to:
• Improve outcomes for participants.
• Define and prioritize evidence of effectiveness.
• Focus on evidence- and outcomes-based government spending, along with active contract management strategies.
• The report will braid technical guidance and real-life examples of how to implement procurement strategies.
• Targeting release for winter 2023

FELLOWSHIP TEAMS

ADVANCING JOB QUALITY

17

18

CALIFORNIA (LOS ANGELES)

PROJECT DESCRIPTION

• The City of Los Angeles’ Office of Workplace Equity, is developing a job quality framework that will guide the revision of the civil service hiring process, to recruit and retain a more diverse workforce and reduce barriers to entry and advancement.
• Progress: The team is currently working on development of a draft job quality framework.

19

COLORADO (PIKES PEAK)

PROJECT DESCRIPTION

• The Pikes Peak Workforce Center and Colorado Workforce Development Council are focused on improving job quality among employers in the region by partnering with Business Services teams to infuse job quality initiatives into existing career progression programs and core business and workforce services.

Progress:

• The team is currently working to:
• Identify priority industries driving rural Colorado’s workforce.
• Apply the state’s job quality framework to localized approaches.
• Connect directly with employers to communicate and operationalize job quality as a good business practice.

20

COLORADO (DEPT. OF LABOR & EMPLOYMENT)

PROJECT DESCRIPTION

• Colorado’s Department of Labor & Employment and the Colorado Rural Workforce Consortium are working to create quality jobs in the state’s rural communities through strengthening partnerships, identifying challenges to improving job quality in rural regions, and strategies to reduce or eliminate those barriers.

​

21

COLORADO (DENVER)

PROJECT DESCRIPTION

• Denver Workforce Services is implementing an initiative to improve job quality within the City’s internal workforce by revising organizational recruitment, hiring, and retention practices. Additionally, a second initiative will be implemented to evaluate the quality of jobs promoted by the City through workforce programming to ensure job quality standards are met.

Progress:

• The team is currently working to shift their internal hiring processes toward skills-based hiring.

22

ILLINOIS (CHICAGO COOK)

PROJECT DESCRIPTION

• The Chicago Cook Workforce Partnership team is a recent Good Jobs Challenge grant recipient aiming to create a cohesive ecosystem to maximize equity and job quality for people served by the grant. Cross-sector organizational training partners accessing Good Jobs grant funding will be tasked to incorporate worker power into career pathway trainings and support businesses to adopt a job quality framework.

23

OHIO (CINCINNATI)

PROJECT DESCRIPTION

• The City of Cincinnati is working to develop a data-driven and outcomes-based job quality framework for the City’s internal workforce and summer youth jobs program.

Progress:

• The team is developing a theory of change, program outcomes, and an employee climate survey for their proposed project.

24

OHIO (CENTRAL OHIO)

PROJECT DESCRIPTION

• The Workforce Development Board of Central Ohio is engaging in the Working Metrics pilot to collect and measure job quality data for employers in the Columbus region of the state.

Progress:

• The team is currently working to develop a strategic communications plan to recruit and retain employers within this pilot using labor market data to identify priority industries from which to recruit employers.

25

OREGON + WASHINGTON

PROJECT DESCRIPTION

• This team is comprised of workforce boards that span across the border of Oregon and Washington states. Building from an existing partnership and job quality initiative, these workforce boards will develop methods for measuring the initiative’s impact and a plan to scale statewide.

Progress:

• The team connected with Harvard’s GPL to discuss how to improve procurements to be more results- and evidence-driven.
• The team is currently building out their job quality initiative work plan, which has six phases (work plan administration, procurement & contracting, education & TA, job seeker & business programs, and legislation & advocacy) each with a distinct implementation timeline.

26

WISCONSIN (CITY OF MADISON)

PROJECT DESCRIPTION

• The team from the City of Madison is applying an equity lens and job quality framework to the City’s internal workforce with the goal of revising personnel rules for city employees to improve quality jobs and ensure equitable recruitment, hiring, and retention.

Progress:

• The team has publicized their job quality framework within the HR department.
• The team has connected with RFA partner subject matter experts, including Jobs for the Future and Jewish Vocational Services, to discuss collection and measurement of job quality data.

27

VIRGINIA

PROJECT DESCRIPTION

• The Skillsource Group Inc. is seeking to create more quality jobs in Northern Virginia, where there are many job vacancies but few quality jobs. Through the fellowship, the team plans to provide guidance, rationale, and best practices to employers and workforce partners to improve hiring and recruitment practices and garner buy-in and greater support for job quality at the state level.

FELLOWSHIP TEAMS

EVIDENCE-BASED PROCUREMENT

28

29

ARIZONA (PHOENIX)

PROJECT DESCRIPTION

• Shift to a performance-based contracting model for WIOA programs in Phoenix to improve outcomes for specific populations, particularly individuals who have been justice involved through an upcoming RFP.

Progress:

• The team has connected with other organizations who have successfully implemented transitional jobs programming like Kentuckiana Works, Workforce MidSouth in Memphis, and San Francisco’s Office of Economic & Workforce Development.
• The team spoke with the City of Phoenix’s Finance Department and RFA’s What Works Cities participants to better understand how the city has been improving procurement processes using performance-based approaches.

30

CALIFORNIA (SAN DIEGO)

PROJECT DESCRIPTION

• San Diego Workforce Partnership plans to develop a randomized controlled trial (RCT) to assess the effectiveness of workforce services by identifying potential clients among unemployment insurance applicants and randomly assigning some of them to be recruited to participate in services.

Progress:

• The team has connected with subject matter experts in job quality, research, and evaluation, including JPAL, Jobs for the Future, and the Arnold Foundation, as well as state staff in the CA Department of Labor with access to unemployment insurance data.
• The team is currently drafting an outline for the RCT to guide project planning and conversations with experts.

31

COLORADO (DENVER)

PROJECT DESCRIPTION

• Denver Workforce Services is working to expand the use of evidence-based practices in the selection of service providers and educate staff, prospective vendors, and partners about evidence-based practices.

Progress:

• The team connected with Harvard’s GPL to discuss changes to an upcoming WIOA Youth RFP to make it more accessible and results-driven.

32

DISTRICT OF COLUMBIA

PROJECT DESCRIPTION

• The team from DC’s Department of Employment Services is working to develop an evidence- and outcome-based approach for an upcoming RESEA grant cycle with the ultimate goal to expand this approach to other grant programs within the District.

​

Progress

• The team is analyzing the research base on RESEA and exploring performance dashboards to make improvements to their grant program.

33

IOWA

PROJECT DESCRIPTION

• The team is working to identify funding available to support workforce initiatives across state agencies, promoting awareness, familiarity, and accessibility to these funds among rural workforce boards. Through identification of gaps in awareness and access, the team will be able to advocate for the state to adopt a data-informed, equitable funding approach, targeting funds to regions without historical access.

Progress:

• The team is conducting a landscape analysis on all available workforce funding within the state, its allocation, and funding cycle.

34

ILLINOIS (CITY OF CHICAGO)

PROJECT DESCRIPTION

• The City of Chicago is revising their procurement process to improve the performance of service providers and the outcomes for participants. The project will focus on the underemployed, helping to build foundational skills and move along a healthcare career pathway from CNA to RNs.

​

Progress

• The team has begun analyzing past RFPs and those from other jurisdictions to make improvements to their process.

35

OHIO (DEPT. OF JOBS & FAMILY SERVICES)

PROJECT DESCRIPTION

• Based on RFA’s work with Central Ohio in our previous cohort, Ohio’s Department of Jobs & Family Services (ODJFS) has launched a $3M initiative - Improving Workforce Outcomes for Ohioans. This project aims to scale the use of evidence- and outcomes-based procurement and grant strategies.
• Local and regional workforce agencies will implement evidence frameworks, link funding to outcomes, and engaging in active contract management through.

Progress:

• The team has recruited three workforce boards within Ohio, covering approximately 50 counties across metropolitan/urban, suburban, rural, and Appalachian regions.
• Teams have begun to revise upcoming RFPs using evidence- and outcomes-based strategies.

36

PENNSYLVANIA

PROJECT DESCRIPTION

• Returning fellows from Central Pennsylvania and Pittsburgh plan to develop and strengthen their internal evaluation processes to measure the outcomes and impact of workforce programming beyond WIOA measures and expand the analysis of performance measures through third party evaluations.

Progress:

• Partner4Work has established a new Policy & Research Department, which will support this initiative, alongside a new strategic plan that incorporates this project’s goals.
• Advance Central PA has engaged with a third party evaluator to conduct a baseline evaluation of current programming, to be completed by end of year 2022.

37

TEXAS

PROJECT DESCRIPTION

• Returning fellows from Texas Workforce Commission (TWC), Texas Workforce Investment Council (TWIC), Texas Higher Education Coordinating Board, and the Rural Capital Area Workforce Board plan to expand the use of their evidence framework and evidence-based grantmaking to additional funding opportunities and state organizations with an eye toward priority populations.

Progress:

• To help build momentum and buy-in for the state’s evidence framework and grantmaking approach, RFA partnered with Texas Fellows to present to all Texas workforce board executive directors at TWC’s annual conference in November 2022.
• The team has identified several upcoming grant opportunities as part of their scaling initiative.

Appendix

38

39

Previous fellows have executed projects that:

40

RFA Fellow Supports

• Training from subject-matter experts in areas that directly support their Fellowship initiatives. Past trainings have included experts from:

​

​

​

​

​

• Peer learning opportunities for Fellows to collaborate across membership states, receive resources, and share best practices.
• Monthly coaching and technical assistance calls to provide targeted feedback, support, and resources to Fellows on their specific projects and initiatives.

41

Workforce Fellowship Cohort II (2020-2021)

Evidence Frameworks

42

*Full evidence definition comparison available here

43

​

Pilot: Using data to track and improve job quality for workforce agency employees. Colorado’s Job Quality Framework can be found here.

​

​

​

Scale: Five Colorado counties measuring and improving job quality with employers.

*Workforce dollars potentially influenced by Colorado’s efforts: $82M

​

COLORADO

JOB QUALITY

44

OHIO

EVIDENCE & OUTCOMES-BASED CONTRACTING

• In March 2022, the Workforce Development Board of Central Ohio released an outcomes-based WIOA and TANF youth contract requiring evidence-based best practices for funding eligibility and using a pay-for-performance model.
• In August 2020, The Workforce Development Board of Central Ohio released two outcomes-based contracts for Career Services and a Job Center Operator with performance metrics linked to pay structure. Contractors receive a combination of cost-reimbursement and outcomes-based payments.
• Ongoing partnership with the Office of Workforce Development policy team works to insert outcomes-based language into the current state policies pertaining to the procurement of one-stop operators, providers of career services, and youth program providers.

45

PENNSYLVANIA

EVIDENCE CONTINUUM

• Pennsylvania’s Department of Labor and Industry used their evidence continuum as part of a grant announcement in August 2021 for up to $4 million of funding for Pennsylvania Industry Partnership Grants. Applicants will be evaluated in part by 1) data collection and performance outcomes and 2) evidence building and effectiveness.
• In October 2020, Pittsburgh's Partner4Work issued an outcomes-based RFP for training programs to provide pathways for local job seekers to gain industry-recognized credentials and employment. Training providers are awarded 50% of funding at the start of the program, with remaining 50% held until participants have earned industry recognized credentials.
• Commonwealth of Pennsylvania has partnered with a Harvard GPL Fellow to operationalize evidence-driven procurement and active contract management for workforce development grants.

​

46

TENNESSEE

OUTCOMES-BASED CONTRACTING & JOB QUALITY

• The Memphis Workforce Investment Network used WIOA funds to award an outcomes-based contract for transitional jobs services to the Center for Employment Opportunities (CEO), which offers an evidence-based model for reentry employment.
• They also used job quality data as part of the award process on another WIOA contract, incentivizing the marketplace to see job quality as a competitive advantage.
• The Memphis Chamber is using job quality data as part of the Gold Standard Certification Program for staffing agencies, aimed at incentivizing improved job quality in the state. Criteria include:
• Competitive Wages
• Incentives
• Support services (transportation, childcare, etc.)
• Benefits (health, dental, retirement, etc.)
• Average roll-time time
• Skills training

​

47

TEXAS

EVIDENCE & OUTCOMES-BASED PROCUREMENT

​

Pilot: Texas Workforce Commission (TWC) awarded a $1.5 million evidence- and performance-based grant for a Building and Construction Trades program focused on serving Opportunity Youth. Evidence framework and performance outcomes are embedded in application and selection criteria for grantees, and 15% of funds are linked to outcome-based payments.

​

Texas Workforce Investment Council (TWIC) also used this approach with $5.5 million in Texas Talent Connection awards.

48

VIRGINIA

PAY-FOR-PERFORMANCE & DATA SHARING

• To support an data-driven workforce system, Virginia developed and launched a workforce data trust, connecting data across six agencies, including a referral portal offering a single point of entry to all workforce customers and performance dashboards to inform both the public and workforce officials.
• Northern Virginia’s SkillSource re-launched their WIOA Youth Pay-for-Performance project focused on better-serving youth who have interacted with the foster care or juvenile justice systems.
• Providers are paid bonus payments related to: program completion, credential attainment, job placement, and earnings.
• In June 2021, SkillSource was awarded a TANF Pay-for-Performance contract by the Virginia Department of Social Services which links payment to critical workforce outcomes.

Fellowship Endorsements

49

“It's been very valuable. In fact, it completely shaped our results-driven contract.”

“It helped us move forward on our project which we previously only considered and kept us accountable to accomplishing what we set out to do.”

“For workforce programs to remain meaningful, we need to make some big strides in use of evidence and how we tell the story to the public. This is a great way to do it.”

Fellowship Endorsements

​

50

“The opportunity to network on a national basis is key.”

“I have learned how to move our state towards looking at the provision from the standpoint of evidence, especially when making decisions.”

“The opportunity to connect with partners from multiple levels of workforce (state and local) and from different states provided the opportunity to hear many different perspectives on how to address workforce challenges and how to get the most out of our workforce dollars. Learning how others are implementing the performance based contracts was great along with hearing about the data/TA providers that are available to help get the evidence needed.”