Libraries and Learning Analytics:
Facts, False Choices, �and Future Forays
ACRL ULS Webcast�November 2021
Ground Rules for Productive Discourse
2
To create an environment safe for open exchange and meaning, let’s agree to:
Adapted from https://crlt.umich.edu/publinks/generalguidelines.
MEGAN OAKLEAF
KEN VARNUM
BECKY CROXTON
Associate Professor
Syracuse University
Head of Library Assessment
UNC Charlotte
Senior Program Manager
University of Michigan
Overview
4
Outcomes
5
DEFINITIONS, MODELS, & PURPOSES
6
LG-97-18-0209-18
LG-98-17-0019-17
7
Connecting Libraries and Learning Analytics for Student Success
Library Integration in Institutional Learning Analytics
These projects were made possible in part by the Institute of Museum and Library Services.
What do we mean by “learning analytics”?
the use of institutional-level systems that collect individual-level student learning data, �centralize it in a record store, �and serve as a unified source for research seeking to understand and support student success
8
9
Institutional Record Store or
Data Repository
protected by policies, procedures, practices, technical security, governance, etc.
Operations Data
SIS Data
Learning Management System Data
iPASS Data
Other Stuff
Library Data
Analysis
queries and correlations based on vetted, approved research questions
by researchers and
educators with access credentials and continually assessed for bias and error that point to experiences that lead to (or away from) success
Students
Facilitate Metacognition,
Empowerment,
Agency
Faculty
Improve Courses/Curriculum
Institution
Maximize Facilitators,
Recognize & Dismantle Hurdles
Advisors
Increase Personalization, Customization, Connection with Supports
Librarians
Improve Services, Resources, Facilities to Facilitate Student Learning and Engagement
10
Faculty
Improve Courses/Curriculum
Institutional Record Store or
Data Repository
protected by policies, procedures, practices, technical security, governance, etc.
Operations Data
SIS Data
Learning Management System Data
iPASS Data
Other Stuff
Analysis
queries and correlations based on vetted, approved research questions
by researchers and
educators with access credentials with findings continually assessed for bias and error that point to experiences that lead to (or away from) success
Students
Facilitate Metacognition,
Empowerment,
Agency
Institution
Maximize Facilitators
Recognize & Dismantle Hurdles
Advisors
Increase Personalization, Customization, Connection with Supports
Library Data
Librarians
Improve Services,Collections, Engagement, Facilities
Librarians
Improve Services, Resources, Facilities to Facilitate Student Learning and Engagement
Yield shared understandings
about what helps or hinders student success
leading to macro-level systemic changes and individual-level connections
Leverage shared capacity (personnel, skills, time)
to increase access to insights
11
Institutional Record Store or
Data Repository
protected by policies, procedures, practices, technical security, governance, etc.
Operations Data
SIS Data
Learning Management System Data
iPASS Data
Other Stuff
Analysis
queries and correlations based on vetted, approved research questions
by researchers and
educators with access credentials and continually assessed for bias and error that point to experiences that lead to (or away from) success
Students
Facilitate Metacognition,
Empowerment,
Agency
Faculty
Improve Courses/Curriculum
Institution
Maximize Facilitators
Recognize & Dismantle Hurdles
Advisors
Increase Personalization, Customization, Connection with Supports
Give the data
back to students!
Library Data
Librarians
Improve Services, Resources, Facilities to Facilitate Student Learning and Engagement
iPASS (Integrated Planning and Advising for Student Success)
iPass systems combine advising, alerts, interventions, degree planning, etc. to connect students with their educational team.
12
https://www.edsurge.com/product-reviews/higher-ed/student-success?search=
https://www.listedtech.com/blog/can-all-student-success-systems-be-integrated-with-major-lms
HOMEGROWN
What does Learning Analytics do?
Learning analytics helps educators:��discover, ��diagnose, ��predict challenges to learning and learner success, and��create or deploy active interventions to benefit students��especially those who might be less familiar with the unwritten and often opaque rules for success in higher education, including first-generation students, community college students, students of diverse backgrounds, students with disabilities, and veterans.
13
What’s an “Intervention”?
14
What Can We Do With Learning Analytics?
→ Answer Questions About and With Students
15
More to
come!
16
Students
Faculty
Librarians
Academic Advisors
Institutional Researchers
Institutional Leaders
→ Take Action based on User Stories
More to
come!
TRICKY PLACES
17
If we keep anything, �we should only keep thoroughly �de-identified/�anonymized data.
How could we handle privacy issues?
18
These are not either/or decisions nor the only positions; perspectives run along a continuum.
We need to engage in dialogue to determine the best course(s) of action for our students, libraries, and institutions.
When we need identified data, we should establish secure data enclaves.
We don’t need �detailed information �about our students.
How might we handle the responsibility of knowing more about students?
19
If we had more detailed information about our students, we may be able to make decisions and take actions with them and on their behalf.
These are not either/or decisions nor the only positions; perspectives run along a continuum.
We need to engage in dialogue to determine the best course(s) of action for our students, libraries, and institutions.
We focus on persistence, �retention, �velocity, �completion, etc.
How could we use data to understand �student learning at a detailed level?
20
We explore data about learning outcomes as assessed in courses by faculty (or librarian) judgment.
These are not either/or decisions nor the only positions; perspectives run along a continuum.
We need to engage in dialogue to determine the best course(s) of action for our students, libraries, and institutions.
Our data about �groups of students�is non-existent or simplistic.
How could we use data to �enact equity and inclusion efforts?
21
Our data about groups of students reflects intersectional identities (race, gender, SES, major, courses enrolled, year in school, etc.).
These are not either/or decisions nor the only positions; perspectives run along a continuum.
We need to engage in dialogue to determine the best course(s) of action for our students, libraries, and institutions.
We rely on assessment approaches �that are �labor-intensive for students.
How can we honor and balance labor?
22
We take a less labor-intensive approach for students as one way to scan for areas that merit deeper dives.
These are not either/or decisions nor the only positions; perspectives run along a continuum.
We need to engage in dialogue to determine the best course(s) of action for our students, libraries, and institutions.
Supplier Community (sometimes)�Vendors (maybe, unclear)
Institutional Researchers
(sometimes)
Who should collect and access the data?
23
Students �(for metacognition, agency)
Librarians �(for decision-making, �action-taking)
Educational Researchers �(to analyze, check bias)
These are not either/or decisions nor the only positions; perspectives run along a continuum.
We need to engage in dialogue to determine the best course(s) of action for our students, libraries, and institutions.
FOR YOUR CONSIDERATION
24
Sorting Activity
Low Priority
Red Flag
High Priority
25
Library Integration in Institutional Learning Analytics (LIILA) Report https://library.educause.edu/~/media/files/library/2018/11/liila.pdf
Activity Assignments
26
Results Review
27
Data Sources
Keys:
UNIQUE ROLES FOR LIBRARIANS
28
New Roles for Librarians
29
Communicate
New Roles for Librarians
30
Communicate
Engage in Policy & Procedure Development
New Roles for Librarians
31
Communicate
Engage in Policy & Procedure Development
Participate Actively in Learning Analytics
New Roles for Librarians
32
Communicate
Engage in Policy & Procedure Development
Participate Actively in Learning Analytics
Create Meaning from Data
New Roles for Librarians
33
Communicate
Engage in Policy & Procedure Development
Participate Actively in Learning Analytics
Create Meaning from Data
Act on Results
NEXT STEPS FOR RESEARCH & PRACTICE
34
Next Steps
35
1
Increase Professional Awareness & Discussion
Next Steps
36
1
2
Increase Professional Awareness & Discussion
Be Informed and Forthright about Current Data Practices
Next Steps
37
1
2
3
Communicate and Negotiate with Vendor and Institutional Partners
Increase Professional Awareness & Discussion
Be Informed and Forthright about Current Data Practices
Next Steps
38
1
2
3
4
Communicate and Negotiate with Vendor and Institutional Partners
Increase Professional Awareness & Discussion
Be Informed and Forthright about Current Data Practices
Situate Learning Analytics among Other Assessment Approaches
Next Steps
39
1
2
3
4
5
Communicate and Negotiate with Vendor and Institutional Partners
Increase Professional Awareness & Discussion
Be Informed and Forthright about Current Data Practices
Situate Learning Analytics among Other Assessment Approaches
Engage the Learning Analytics Conversation at the Institutional Level
Next Steps
40
1
2
3
4
5
6
Communicate and Negotiate with Vendor and Institutional Partners
Increase Professional Awareness & Discussion
Be Informed and Forthright about Current Data Practices
Identify and Analyze Questions or Problems Meriting a Learning Analytics Approach
Situate Learning Analytics among Other Assessment Approaches
Engage the Learning Analytics Conversation at the Institutional Level
Next Steps
41
1
2
3
4
5
6
7
Communicate and Negotiate with Vendor and Institutional Partners
Increase Professional Awareness & Discussion
Be Informed and Forthright about Current Data Practices
Identify and Analyze Questions or Problems Meriting a Learning Analytics Approach
Situate Learning Analytics among Other Assessment Approaches
Engage the Learning Analytics Conversation at the Institutional Level
Envision Library Data Contributions
Next Steps
42
1
2
3
4
5
6
7
8
Communicate and Negotiate with Vendor and Institutional Partners
Increase Professional Awareness & Discussion
Be Informed and Forthright about Current Data Practices
Identify and Analyze Questions or Problems Meriting a Learning Analytics Approach
Situate Learning Analytics among Other Assessment Approaches
Engage the Learning Analytics Conversation at the Institutional Level
Envision Library Data Contributions
Explore Interoperability Standards
Next Steps
43
1
2
3
4
5
6
7
8
9
Communicate and Negotiate with Vendor and Institutional Partners
Increase Professional Awareness & Discussion
Be Informed and Forthright about Current Data Practices
Identify and Analyze Questions or Problems Meriting a Learning Analytics Approach
Situate Learning Analytics among Other Assessment Approaches
Engage the Learning Analytics Conversation at the Institutional Level
Envision Library Data Contributions
Explore Interoperability Standards
Identify Key User Stories
Next Steps
44
1
2
3
4
5
6
7
8
9
10
Communicate and Negotiate with Vendor and Institutional Partners
Increase Professional Awareness & Discussion
Be Informed and Forthright about Current Data Practices
Identify and Analyze Questions or Problems Meriting a Learning Analytics Approach
Situate Learning Analytics among Other Assessment Approaches
Engage the Learning Analytics Conversation at the Institutional Level
Envision Library Data Contributions
Explore Interoperability Standards
Pursue Pilot Studies
Identify Key User Stories
DISCUSSION QUESTIONS &
READINGS FOR CONTINUED CONVERSATION
45
Library Roles
46
Discussion Questions
47
Questions engaging:
Discussion Questions
48
Questions engaging:
Discussion Questions
49
Questions engaging:
ACRL Learning Analytics Toolkit
50
Libraries and Learning Analytics:
Facts, False Choices, and Future Forays
Thank you
for joining us!
ACRL ULS Webcast�November 2021