1 of 37

Can we demonstrate academic library value without violating student privacy?

Zoe Fisher

Assistant Professor, Pedagogy & Assessment Librarian

University of Colorado Denver

Colorado Academic Library Association

Intellectual and Academic Freedom and the Academic Library

University of Denver

Friday, July 21, 2017

2 of 37

3 of 37

“We won’t know what difference the library makes until we collect data on individual library users.”

--Megan Oakleaf in her keynote address to the Library Assessment Workshop, Colorado Academic Library Association,

November 2016

4 of 37

Agenda

In today’s talk, I will…

  • Define common metrics of student learning & success
  • Explain ACRL’s Assessment in Action and the research findings it has produced
  • Identify the controversy around learning analytics and library ethics
  • Highlight alternative research methods that demonstrate library value

5 of 37

Common Metrics of Student Success

Concerns

  • Students enroll and attend for a few semesters, then drop out
  • Students complete many credit hours, but don’t actually earn a degree
  • Students complete courses, but don’t perform well

Data Points

  • Persistence (continuing from term to term and year to year)
  • Graduation and completion of degrees & certificates
  • Course grades and GPA

6 of 37

Trends in Higher Education

The 6-year graduation rate for first-time, full-time undergraduate students who began seeking a bachelor's degree at a 4-year degree-granting institution in fall 2009 was __ percent.

7 of 37

Trends in Higher Education

The 6-year graduation rate for first-time, full-time undergraduate students who began seeking a bachelor's degree at a 4-year degree-granting institution in fall 2009 was 59 percent.

Source: National Center for Education Statistics, U.S. Department of Education

8 of 37

Trends in Higher Education

If 3 out of 5 students finish a Bachelor’s degree in six years, what are those students doing differently than the other 40%?

9 of 37

Trends in Library Assessment

Librarians are eager to prove that library use has a demonstrated impact on student success.

10 of 37

How has ACRL responded?

11 of 37

What kinds of library use data are being collected?

  • Attendance in information literacy instruction sessions
  • Circulation
  • Online database access
  • Study room use
  • Interlibrary loan
  • Reference interactions

12 of 37

How is data being collected?

  • Card swipes
  • ILS
  • Database logins
  • Calendar & reservation software
  • Surveys
  • Focus groups

13 of 37

How is library use data being connected to “student learning and success”?

  • GPA
  • Course grades
  • Retention/persistence
  • Graduation

14 of 37

What are the key findings?

  • Students benefit from library instruction.
  • Library use increases student success.
  • Collaborative programs involving the library enhance student success.
  • Information literacy instruction strengthens general education outcomes.
  • Library research consultations boost student learning.

15 of 37

Library Assessment Conference 2016

16 of 37

Library Assessment Conference 2016

Grand Valley State University

Mary O’Kelly

Key Finding

“Students who had at least one professor work with a librarian—regardless of whether those students saw a librarian in their own classes—were retained at a statistically significant higher rate.”

All conference proceedings are available at http://www.libraryassessment.org/.

17 of 37

Library Assessment Conference 2016

University of Minnesota, Twin Cities

Jan Fransen & Kate Peterson

Key Finding

All conference proceedings are available at http://www.libraryassessment.org/.

18 of 37

Correlation is not…

19 of 37

Correlation is not…

20 of 37

When I see studies that correlate library use with student success, I think…

  • Students who have more time for school are more likely to be successful.
  • Students who see themselves & their cultures reflected in their campus experience are more likely to be successful.
  • Students who are able to engage with campus in a variety of ways are more likely to be successful.
  • Students who are able to regularly attend class (and therefore come to instruction sessions in the library) are more likely to be successful.

21 of 37

When I see studies that correlate library use with student success, I think…

The library is not the thing that makes the student successful. Privilege is.

22 of 37

The problem with correlation, continued.

Studies that correlate library use with student success reinforce the narrative that academic success is a series of individual choices (rather than the result of a system designed to help some people succeed more than others).

23 of 37

Getting back to privacy…

  • De-identified data is not anonymous.
  • Controlled access to data still carries some risk.
  • Even if our users don’t care about their own privacy—we do.

24 of 37

Required Reading

“Learning Analytics and the Academic Library: Professional Ethics Commitments at a Crossroads”

Kyle M. L. Jones (Indiana University) and Dorothea Salo (University of Wisconsin-Madison)

College & Research Libraries, March 2018

Available now as a pre-print

25 of 37

Learning Analytics

  • Collecting, analyzing, and reporting as much data as possible about learners in order to understand and optimize learning
  • This can be everything from where students go on campus to where they click inside a Learning Management System (LMS)
  • Data can be used to predict and make recommendations for users (also known as nudging)

26 of 37

Jones & Salo, 2018

  • Problems with ‘nudging’
    • “Learning analytics compromises intellectual freedom when institutional actors, system designers, and algorithms limit opportunities to engage in the creation and consumption of intellectual materials.”
  • Concerns that students being surveilled will alter their information-seeking behavior to achieve targets
    • “Students may chill their intellectual behaviors due to privacy invasion from librarians participating in learning analytics.”

27 of 37

Trends in Research Literature

Kogut found 38 articles used quantitative methods to explore:

  • GPA
  • Information literacy skills
  • Retention
  • Final degree attainment
  • Graduation rates

28 of 37

Recommendations from Kogut 2016

  • Use research designs beyond statistical methods
  • Use alternate measures to explore impact on ‘success’ (job placement, post-college performance, professional exams)
  • Utilize higher education theoretical frameworks
  • Publish beyond library venues

29 of 37

Dreams of Qualitative Methodologies

30 of 37

Qualitative Research Methods

  • Phenomenography
  • Ethnography

31 of 37

Qualitative Research Methods

  • Phenomenography
    • Investigates the different ways in which people experience and think of the world
    • Boon, Johnston, and Webber’s 2007 article, “A phenomenographic study of English faculty’s conceptions of information literacy”
    • Christine Bruce (1996 thesis “Information literacy: A phenomenography”)

32 of 37

Qualitative Research Methods

  • Phenomenography
    • Phenomenography is especially relevant to student success because there are different ways to be successful and different paths to success.
    • Phenomenography could help libraries better understand which services are critical to students’ success and why.

33 of 37

Qualitative Research Methods

  • Ethnography
    • Researcher observes from the point of view of the subject
    • A systematic study of peoples & cultures
    • www.erialproject.org

34 of 37

Qualitative Research Methods

  • Ethnography
    • Listen to our students!
    • Let them tell their success stories as a holistic picture, and hear how the library was (or wasn’t) part of it.

35 of 37

Libraries are not the answer to the question of student success.

36 of 37

Libraries are not the answer to the question of student success.

  • The question is complicated.
  • Correlational studies are not enough to answer why 40% of students don’t finish a degree within 6 years.
  • We need to push back on demands to use library data to prove our value.

37 of 37

Agenda

In today’s talk, I have…

  • Defined common metrics of student learning & success
  • Explained ACRL’s Assessment in Action and the research findings it has produced
  • Identified the controversy around learning analytics and library ethics
  • Highlighted alternative research methods that demonstrate library value

Questions & Comments?

Thank you!

Zoe Fisher

zoe.fisher@ucdenver.edu

Slides: http://tinyurl.com/fisher072117