1 of 39

Geek Out

Don’t Freak Out�How to Chill Out and Learn to Love Assessment

Meredith Farkas, Portland Community College

Colleen Sanders, Clackamas Community College

2 of 39

Plan:

Share experiences
ID barriers
Share out: what’s your experience?

Outcomes:

New ideas
Renewed curiosity
Better mindset

2

3 of 39

Clackamas Community College

Accreditation-driven assessment push

Service vs. Academic department

Interview faculty on database knowledge; score on rubric

3

Colleen: 1. CCC assessment situation: Accreditation ding, campus-wide mandated effort, assessment coordinator Elizabeth Carney, Inservice, Moodle, faculty workshops. Pros: support, documentation, clear expectations, faculty cooperative as all in same boat. Cons: Top-down support but not more hours in a dar or $. Panic-inducing. A lot of change, some resistance in depts.

2. CCC Library Assessment: CHOSE to be academic dep’t - had leeway to do so. This is profound for both how we do assessment but also how we conceptualize our library, ourselves, our relationship to the rest of campus, and how we do our work. Safe to say our library is turning a significant corner in its history: new leadership, overhauling library vision, mission, policies. Assessment helped us visualize our priorities compared to our resources and existing practices, which has led to a lot of good dialogue. STRUCTURE: “Services” are courses, faculty are our students, with stated PLOs such as “Faculty should be able to use library services/instruction/collections to enhance instruction.”

(go to next slide)

3. FoD work: interview question, rubric, one-on-ones in office hours, scored and compared to our baseline. Now we’re at the point of building a feedback loop.

4 of 39

4

5 of 39

Portland State University

Admin wanted faculty doing assessment, but no clarity or support provided

Faculty’s (justified) resistance to assessment

Hired to coordinate instructional assessment

5

6 of 39

The word ‘Coordinator’ in a position title. Beware it…

The problem with these jobs is that as often as not, there’s nothing actually to coordinate. No budget. No dedicated staff. No established service. ‘Coordinate’ all too often means ‘try to establish a beachhead by begging your new colleagues to vouchsafe you a few minutes of their time now and then, knowing that their supervisors won’t tell them to and you have no authority whatever to demand anything of them.

Oh, but the buck will stop with you, Coordinator, on all matters regarding your service. Be aware, also, that because what you’re doing is new, you must hit a home run with it, quickly; slow and steady gains will not do. If you don’t, two damaging assumptions arise: that the service is a waste of effort, and that you’re a lousy librarian. You’re a coordinator now; congratulations!

6

-Gavia Libraria, “The C-word”

“

7 of 39

Portland Community College

Accreditation was the motivator

Faculty-driven

Learning Assessment Council

Everyone’s responsibility, team-based assessment work�

7

8 of 39

Barriers we’ve encountered

Territorialism

Fear of judgement

Fear of failure

Lack of time

Edicts from admin that create anxiety

Assessment fatigue

Lack of expertise

Lack of agreement on why we assess

Perfectionism

Faculty/staff resistance

Lack of support from admin

Institution doesn’t prioritize assessment

Seen as one person’s job

Lack of direction

8

9 of 39

OUR LESSONS LEARNED

10 of 39

MINDSET

10

11 of 39

For a scholarship of assessment to thrive, we must align faculty culture, institutional structures, and leadership for change. The importance of this point cannot be overstated. A meaningful assessment program is more than just a new activity to be undertaken, it is a change in how we think about what we do in higher education.

-Don Haviland “Leading Assessment: From Faculty Reluctance to Faculty Engagement.”

11

“

12 of 39

12

13 of 39

Let go of the idea of the “perfect assessment”

13

14 of 39

“This is not research”

Repeatable

Reliable

Reasonable

Practice, not theory

14

Elizabeth Carney, Assessment Coordinator, CCC

Upcraft, M. L. & Schuh, J. H. (2003). Assessment vs. research: Why we should care about the difference. About Campus, 7(1). Retrieved from https://www.stthomas.edu/media/assessmentaccreditation/Assessment_vs_research_why_we_should_care_about_the_difference.pdf

Participatory researcher

“Good enough” rule

Acknowledge limitations to study

Colleen: Or at least it doesn’t have to be. They feel similar and overlap, but assessment guides PRACTICE first. It’s not aimed at building theory. We use some of our expertise at conducting research and some expertise in setting up reliable, repeatable, reasonable. Upcraft article: distinction is important to make. Addresses accusation (from yourself or others) that this is not up to the rigor of ‘real’ research. 4 points. Good enough: a study with limitations is better than no study, working within time/resource limitations. Make best choice of tool & study area based on what you want to learn and the best way to learn it. For ex, while interviews made for a direct measure, in the future if we don’t have time to do so many interviews, we may hold focus groups w/ faculty. Working w/in changing institutions isn’t how ‘pure’ research operates: that’s not where we’re starting from.

15 of 39

Process & product

15

Colleen: Colleen: Or at least it doesn’t have to be. They feel similar and overlap, but assessment guides PRACTICE first. It’s not aimed at building theory. We use some of our expertise at conducting research and some expertise in setting up reliable, repeatable, reasonable. Upcraft article: distinction is important to make. Addresses accusation (from yourself or others) that this is not up to the rigor of ‘real’ research. 4 points. Good enough: a study with limitations is better than no study, working within time/resource limitations. Make best choice of tool & study area based on what you want to learn and the best way to learn it. For ex, while interviews made for a direct measure, in the future if we don’t have time to do so many interviews, we may hold focus groups w/ faculty. Working w/in changing institutions isn’t how ‘pure’ research operates: that’s not where we’re starting from.

16 of 39

16

17 of 39

17

18 of 39

Growth mindset

18

Colleen: Who has heard of Carol Dweck’s book ‘Mindset: The New Psychology of Success’? There are a lot of workshops/institutes/seminars on the concept that intelligence is either static (innate, measurable, and testable at any time) or developed (failing forward, worked at). We’re all a blend of both, and it can be a really therapeutic framework for those suffering from assessment anxiety. When we found out that our faculty were at abotu 20% when we were hoping for 80%, we could have viewed that as the library’s failure to do proper outreach & instruction. Instead we saw opportunities to close that gap and integrate FoD in our LibGuides, library assignments, and instruction.

Assessment by definition looks for problems: its job is telling us where we can improve (or what we’re doing wrong, if you come from a fixed mindset.)

With both the assessment process and its findings, I find it really empowering to revisit Growth Mindset principles - it gives me an energy boost and forms a good communication tool within the library team when things feel frustrating, overwhelming, or too complex.

19 of 39

Be clear about why you’re doing assessment

19

20 of 39

Start from a place of curiosity

20

Meredith - I used to not be a fan of assessment. At one of my old jobs, we had students fill out this satisfaction survey at the end of every instruction session and we learned absolutely nothing from the responses. It felt like assessment was just about checking a box to say we did assessment. Then I started doing my own assessments and realized that assessment was actually about learning, or at least it should be. Especially when you’re trying to build an assessment culture in your organization, starting assessment work around things that people are curious about can be a good and non-threatening way to start doing assessment. We did an activity at our last in-service where we asked people to share what they were curious about in terms of how the library is used. We got so many great ideas and are now looking at which ones would make sense to try and answer via assessment.

21 of 39

Develop a reflective practice

21

22 of 39

PROCESS

22

23 of 39

Start by assessing where the light is best

23

24 of 39

Define what you want to learn before designing your assessment

24

25 of 39

Partner with disciplinary faculty and other campus units

25

Meredith: at PSU, we did an assessment project doing an information literacy assessment of student portfolios in the Freshman Inquiry program. We invited faculty who teach in that program to join us for the assessment work. It gave them the opportunity to look at their students’ work through a whole other lens and to think deeply about what information literacy means in the context of their courses. At PCC, we have worked with institutional effectiveness (our institutional research group) to help us design and administer really solid surveys to students who’ve had assessment. They’ve also been super helpful in getting us data we need to better understand our instruction program.

Colleen: We worked with our assessment coordinator to help norm our rubric. We workshopped our assessment tool (survey interview) with Institutional Research, wh/ was really insightful but also overwhelming, as it exposed a lot of the research limitations of our tool. Idea of seeing patterns after a small sample size = timesaver. Again, assessing faculty leads to relationships.

26 of 39

OUTCOMES OF ASSESSMENT

26

27 of 39

Closing the loop is the most important thing you do!

27

Meredith - if you don’t learn from and make changes based on your assessment results, what was the point of doing it? At PCC, we’ve been on this cycle where we get our marching orders from the Learning Assessment Council at the start of Fall. We often do some preliminary planning in Spring, hoping the LAC won’t come up with a crazy new thing for us to do, but we do the bulk of planning in Fall, implement the assessment in Winter, and analyzed the data in Spring. It doesn’t give us a lot of time to make changes to curriculum, and many academic departments have complained about this. You must make time for closing the loop and it’s probably the most challenging piece because you’re often so fatigued by the time you’ve analyzed the data that you just want to file it away and never see it again. And that’s what many librarians do!

Colleen: This is the step we’re on at CCC, and it’s challenging bc it adds another task to build while we’re gearing up for assessing our next ‘service.’ But this was the point of the whole project. “The cold reality is that decisions will be made, polities developed, and practices implemented regardless of the avilability of assessment results.” (Upcraft & Schuh 2003).

28 of 39

Make time to talk about assessment results

28

29 of 39

Small assessments can yield big insights

29

30 of 39

Assessment can do more than assess

30

31 of 39

SELF-CARE

31

32 of 39

Assessment fatigue is real

32

33 of 39

Celebrate achievements

33

34 of 39

A lone assessment champion will burn out . TEAM UP!

34

35 of 39

All about that first step

35

36 of 39

36

Write down a few ideas you want to remember and bring back to your work.

Let’s get started with that reflective practice!

37 of 39

WHAT ARE

YOUR LESSONS

LEARNED? YOUR CHALLENGES? YOUR QUESTIONS?

Meredith Farkas, Portland Community College meredith.farkas@pcc.edu Twitter: librarianmer

Colleen Sanders, Clackamas Community College colleen.sanders@clackamas.edu

Slides available at http://bit.ly/ilago-farkas-sanders

38 of 39

CREDITS

Special thanks to all the people who made and released these awesome resources for free!

Presentation template by SlidesCarnival
Photo credits

38

39 of 39

PHOTO CREDITS CONTINUED

39