Change Initiatives, Programs, & Assessment

​

November 10, 2022

Land Acknowledgment

​

We acknowledge the traditional land of the Tongva People. For thousands of years, the Tongva people lived and were considered the most powerful indigenous peoples to inhabit the Los Angeles basin. Along with the Tongva, we also recognize the Chumash, Tataviam, Serrano, Cahuilla, Juaneno, and Luiseno People, for the land that USC also occupies around Southern California. We pay respects to their elders past and present.

​

​

Visit: https://native-land.ca/ �Source: University Of Southern California Asian Pacific American Students Services

Presenter Information

​

​

​

​

​

LeAnne Jones Wiles (she,her)�Executive Director, Student Academic Services, Undergraduate Academic Affairs

Identities: Black professional, Mom, Partner, Enjoy Books, Enjoy coffee w/ friends, Cat Owner (Jordan), Love for Administration, Deep care for student experience

​

Emily Kolby (she,her)�Director, Curriculum & Engagement, First year Programs

Identities: White woman, Educator, Administrator, Student, Partner, Puppy Parent (Gizmo), Writer, Committed to supporting students along their journey

(ljwiles@uw.edu)

(ekolby@uw.edu)

Former and Current Institutions

​

​

• Public/Private (23k) Freshman(4,143) 90%Retention

​

• Private (1,500) Freshman (440) 93% Retention

​

Public (23k) Freshman(5,500) 73% Retention 

​

• Public – RI  (32k) Freshman (7,200) 92% Retention 

Transfer (1,700)

18 Years Focused on Transition & Student Success

Session Format

​

​

​

​

​

1. Setting Context
2. Change Initiatives
1. Q&A - Joined by Emily Kolby
3. Additional slide available
• Program Assessment Methods for FIG Program- Structured, Semi-Structured, Informal

​

​

​

​

​

​

​

​

​

​

In the Chat:

One word to describe how you are showing up today.

​

​

Setting Context

​

MISSION

Undergraduate populations served:

Freshman U.S. Students

Freshman International Students

Transfer Students

Parents & Families

First Year Programs fosters a successful undergraduate student experience through strategic programming that focuses on positive academic transitions and the development of learning communities. Through partnerships with faculty, staff, alumni, and student leaders our programs create the space for students to define how they will engage, learn, and thrive at the University of Washington.

Undergraduate Academic Affairs

Student Academic Services

1. Academic Support Programs
2. Center for Community Engagement and Leadership Education
3. Center for Experiential Learning & Diversity
4. Community College Research Initiatives
5. Dream Project
6. First Year Programs
7. Honors Program
8. Husky Leadership Initiative
9. Jumpstart
10. Mary Gates Endowment for Students
11. Office for Youth Programs, Development and Support

12. Office of Educational Assessment
13. Office of Merit Scholarships, Fellowships & Awards
14. Othello-UW Commons
15. Riverways Education Partnerships
16. Resilience Lab
17. Student Academic Services Administration
18. Student-Athlete Academic Services
19. Undergraduate Advising
20. Undergraduate Research Program

​

​

​

DR. ED TAYLOR,�DEAN & VICE PROVOST

Our Team

Mary Gates Hall, Suite 120 Email: fyp@uw.edu �Commuter & Transfer Commons, HUB 141 Office: 206.543.4905

fyp.washington.edu Text: 206.207.5564

LeAnne

Executive Director

Matt

Associate Director

Renee

Program Manager, First-Year Engagement

Courtney

Program Manager

Emily

Director, Curriculum & Engagement

Meghan

Senior Program Manager

Sarah

Program Manager, Operations/Student Transitional Support

Emma

Senior Program Logistics Manager

Jenn

Web Producer

First Year Programs

Julie

Assistant Director

Digital Engagement & Strategic Initiatives

FIRST-YEAR�EXPERIENCE

​

Change Initiatives - Programs

& Assessment

Initiatives vs. Programs

​

​

1. Why would a institution choose a specific kind of initiative?
2. What do you have to pay attention to make it real?
3. How do you know an institution is achieving the goal?

​

​

Selected Initiatives

​

1. Community College Transfer Initiative
2. Healthy Huskies
3. International Student Transition
4. Arts & Sciences Writing Taskforce
5. Immunization Task Force
6. Husky Leadership Initiative

​

Instructions

1. Pull out a piece of paper.
2. Number 1-6 and write down the list of initiatives.
3. “Raise your hand” when you are finished.

​

Why Choose Initiative

​

​

Factors to Consider

1. Issues or problem statement
1. Systemic barriers
2. Sponsor or endorsement
3. Resources (people/time/$)
4. Stakeholders

Selected Initiatives

Selected Initiatives

Making It Real

​

​

1. Defined Timeline
2. Clear scope of what you can and can’t do
1. Sub committee vs. charge committee
2. Committees with different stakeholders
3. Stakeholders
• People who are invested in outcome > engagement
4. Realistic funding models
• Best case scenario, worst case scenario

Selected Initiatives

In Breakout Rooms

​

​

1. In your group rate each initiative using the scale below.
2. Which initiatives do you think turned into programs?
3. What assessment approach would you use to measure success?

How do you know you achieved your goal?

​

How does assessment impact the program?

​

​

1. Reset Goals
2. Identify new or current outcomes/benchmarks,
3. Evaluate how data is gathered
4. Review timing cycle and data
5. Make Changes

How do you know you achieved your goal?

​

​

Q&A

​

View Presentation: fyp.washington.edu/fyeprojects

​

​

Program Assessment and Improvement Methods: First-year Interest Groups

​

​

​

​

​

In the Chat:

What are you curious about or hoping to learn more about today?

Program Assessment Methods

​

​

Program Assessment Methods

Factors to Consider

​

​

1. Data Integrity “Dirty Data”
2. Response Time
3. Resource Constraints
4. Shifting Expectations

First-Year Interest Groups (FIG)

Student Academic Services, Undergraduate Academic Affairs

​

​

​

​

​

• First year seminar course (General Studies 199) taught by peer educator.
• Program is 30+ years old
• 130 sections
• Average Class size 22 students
• Often linked with other courses
• 1-2 credits
• Students are not required to take a FIG

FIG Video , Website

FIG Program Guiding Questions

2008 - Present

​

1. I don’t think the FIG Program is an academic experience?

​

• What is your specific impact on retention? How do you know that you are effective?

​

• Since FIG’s are so effective, shouldn’t every first-year student be in a FIG?

Guiding Questions

Assessment Response

​

1. I don’t think the FIG Program is an academic experience?
1. Formal: Research Study, Cost - $50,000, 1-2 years

​

• What is your specific impact on retention? How do you know that you are effective?
• Semi-Formal: , Ph.D Student Thesis, Cost - $10,000, 2 years

​

• Since FIG’s are so effective, shouldn’t every first-year student be in a FIG?
• Informal: Mapping of First Year Experience, Cost - Faculty Buyout , 9 months

​

In the Chat:

Which assessment response do you think would be the most impactful for the FIG Program (1,2, or 3)?

1 - Formal

2 - Semi-Formal

3 - Informal

Formal: Research Study

Structure

​

• Conducted by the University Office of Educational Assessment.
• Centered around a single FIG cohort (about 2,800 students).
• Focused on how the program was doing with regards to strategic aims; not on the effect of the program on students.
• Developed a robust survey tool with over 146 questions and 7 scales.
• Analysis included comparisons of pre- and post-quarter
• Conversations with faculty, staff, and administrators: Are the current goals for the FIG program the “right” goals?

Formal: Research Study

Findings

​

Compared Students in a FIG vs. Non- FIG

• Underrepresented minority students, and first-generation college students reported greatest satisfaction.
• The FIG program is playing a key role in the UW’s core mission.
• Feel more connected, know resources, stronger mentorship, and emotional well being than students not in a FIG.

Recommendations

• Increase international student engagement
• Create a stronger academic orientation
• Focus on peer educator curriculum

No evidence to support enrollment management theory.

​

Semi-Formal: Ph.D Thesis

Structure

• Complete registrar transcript, application, and demographic records
• Students who first enrolled as freshmen at the UW between 1998-2010
• About 58K students
• Of these students, about 32.5K (56.2%) enrolled in a FIG
• FIG exit surveys from students who enrolled in FIGs between 2010-2015
• Coordinated by UW First Year Programs
• About 14K responses from students (about 12K usable)
• Mostly open-ended text responses. Hand-coded
• Additional data sources
• US Census data linked by student application ZIP code
• Enrollment planning services data from the College board

​

​

Semi-Formal: Ph.D Thesis

Findings

• Students who enroll in a FIG return for their second year and graduate at higher rates than non-FIG students (4% difference).

​

• While FIG enrollment significantly increased the re-enrollment and graduation percentages for all under-represented students, the highest gains were seen among Hispanic students with a nearly 6% increase in second year re-enrollment and a 8% increase in graduation rates.

​

​

http://fyp.washington.edu/first-year-interest-groups/fig-30th-anniversary/

Informal: FYE Mapping Project

Structure

1. Not a research study
2. Faculty member doing a landscape analysis, specific to UW
3. 51 interviews with faculty and staff
4. Student Focus Groups
5. Report Examination
1. UW Withdrawal Survey
2. UW IT Reports
3. UW Resilience Lab (UWRL) Data
4. Project for Education Research That Scales (PERTS) data
5. First Year Interest Group (FIG) data
6. American Talent Initiative (ATI) Task Force Report

​

Informal: FYE Mapping Project

Findings

• The more credits a freshman transfers upon entry to the UW, the less likely they are to enroll in a first year seminar.
• Research suggests that there is no universal first year experience at UW.
• The most common shared experience: The first exam. “The First Exam Breaks them.”
• Academic exploration can come at a cost.

​

​

​

http://fyp.washington.edu/fyemapping

​

Next Steps

​

Impact

​

​

• Formal: Research Study
• Justified existence of the FIG program
• Acknowledged wasn’t very “academic” - people felt “heard”
• Semi-Formal: PH.D Thesis
• Answered question about retention (nationally)
• Acknowledged importance of peer educators
• Greater breath because of formal research study survey
• Informal: Mapping Project
• Moved away for “best practices” fye to focus on first-year experience at the UW
• Unveiled systemic issue with access to majors, transition, mental health
• Used semi-formal results as talking points in interviews to justify FIGs and explore opportunities for holistic first-year experience

Next Steps

​

Proactive vs. Reactive

• Purchase a vendor product
• Increase transparency
• Measure satisfaction, learning, overall program effectiveness

Use Data to Drive Programmatic Change

• Example: Pre-A&O Modules, Sense of Belonging

​

Evaluate Data Collection Methods Annually

• Ok to stop a process
• Example: PERTS, Knowledge checks, pre-post surveys

​

​

Q&A

​

View Presentation: fyp.washington.edu/fyeprojects

​