1 of 27

Nuanced and Timely:�Capturing Collections Feedback at Point of Use

Online Northwest 2014,

Snowstorm Edition

Rick Stoddart, OSUL&P

Jane Nichols, OSUL&P

Terry Reese, OSU

2 of 27

pro-to-type

/ˈprōtəˌtīp/

noun

noun: prototype; plural noun: prototypes

1. a first, typical or preliminary model of something, esp. a machine, from which other forms are developed or copied.

"the firm is testing a prototype of the weapon"

synonyms:

original, first example/model, master, mold, template, framework, mock-up, pattern sample; guide, blueprint…

3 of 27

MINES for Libraries®

4 of 27

Activity: Think - Pair - Share

If you were able to have a pop-up survey before selected electronic resources…

What would ask?
What specific collections would you target and why?
What would you be able to do with the results? (How would you use this evidence?)

5 of 27

Beyond WHAT�|�v�Uncover the�HOW and WHY�|�v�How do our collections connect to student learning?�How do our collections connect to faculty productivity?�How are our collections used in the classroom?�|�v�What will access to this library resource help you to do?

So my role in this is to articulate some of the big pictures connections.

We often talk about USE as a measure for collections but that often boils down transaction numbers such as circulation statistics and downloads that provides information about WHAT they are using but limited information about HOW or even WHY our collections are being used. This project attempts to fill in some of these blanks. This point of use survey incorporates a qualitative question asking How will access to this library resource help you to do? and we have seen answers such as Complete a Class Assignment, Complete a dissertation or Provide Readings for my Students. Responses such as this help connect the collection to the curriculum, to research, and to student needs. And tell a bigger story beyond simply usages through article downloads.

It is these Broader Impacts with students, faculty and other stakeholders that libraries are being pressured to better articulate. We may intuitively know the answer to some of these questions such as… (read questions) -- but a survey tool or research project like this can help provide evidence to support our understanding and allow us to make evidence-based decisions about how best to tell the story of the library’s with stakeholders. For example, as we gather more evidence, it might be possible for the library to better advocate to university administrators about our role in supporting grant funded faculty research on campus.

Beyond asking what access to the collection allows you to do -- the survey also asks the respondents to put a monetary value on accessing the article as well as how long they would be willing to wait for access (immediately, within 24 hours, 1 day, etc.). This places a dollar and time value on the collections from a patron perspective that we didn’t often have. This provides not only information for collection development but also give sense of how students and faculty value access to our collections. Is the perceived value of immediate access to our collections the same for both librarians and patrons? This is something the survey hopes to find out.

6 of 27

Evidence for…

Stakeholder Impact

Library

Collections

Students
Faculty
Funders
Community

Space allocation
Learning vs. Research

Selection
Purchasing
ROI

7 of 27

Fall 2012

Spring 2013

Fall 2013

Winter 2014

Code

Created

Elsevier

UTL

All Elsevier & JSTOR Ecampus

JSTOR

Ecampus

We wanted to test the software, survey questions and survey design across several iterations so we could apply what we learned from earlier iterations to later ones.

The spring after Terry created the code, we conducted our first round with our Elsevier UTL unique title list; because it is a well-defined journal list of that gets a lot of use by our patrons. This fall after revisions to the code, the survey and the survey design we tested the pop-up survey software again but focused on JSTOR journal titles acquired due to a gift from our Ecampus. Finally, this January we tested it on both the Elsevier titles; this time expanded to include all titles we have access to as well as the JSTOR Ecampus titles.

Now that you have a broad picture of how we deployed the survey; let’s look at it from a survey respondent’s perspective.

8 of 27

9 of 27

10 of 27

11 of 27

12 of 27

13 of 27

(Apache)

Terry Reese. "Improving Evaluation of Resources through Injected Feedback Surveys". July 2013, Ariadne Issue 71 http://www.ariadne.ac.uk/issue71/reese

Now that you have a feel for what the user sees. Let’s take a look ‘under the hood’. Key to the success of triggering the survey to pop-up at the journal level, is our proxy server. In OSU’s case, we use EZProxy. For this survey infrastructure Terry also deployed a public proxy server on Apache. We’re all familiar with using a proxy server from off-campus where you log in prior to getting to your article; that’s what is meant in our case by a public proxy. In this workflow; an Apache proxy is added which plays the key job of a decision engine, driven by SQL rules and IP address. Based on the rules, the public proxy analyzes the users request for an article; determines if the person should be displayed a survey; displays the survey when appropriate and when not, sends them to their article. For the most part this works well and we are able to capture meaningful data. In the lessons learned and next steps, I share some tweaks that need to be made.

For further technical details I recommend Terry’s article published in Ariadne. The citation is listed on this slide.

14 of 27

Survey Parameters

6 questions about use

3 questions about demographics

13

March 4, 2014

? ? ? ?

24 hours/7days

1364

responses

53%

under

grad

27%

grad

16%

faculty

15 of 27

14

March 4, 2014

	Faculty Instructor Researcher	Grad student	Undergraduate Other student Other
Core	15%	26%	57%
Supplemental	24%	35%	42%
Haven't used resource before*	3% (n=8)	34%	62% (n=29)

*37 people hadn’t used the resource before

16 of 27

Undergraduate Students
Give my course assignment a resource.
I am currently gathering research articles on Schizophrenia from a neurological perspective for a research paper.
Finish my homework.
Complete my assignment.
Complete a discussion board assignment for an e-campus class.
I need this information to complete a term project. The information contained in this article is very specific and I need this type of specific information.

17 of 27

Graduate Students
I am working on a literature review for my Master's project. This allows me to work from my home as opposed to the library (which is generally crowded with undergrads).
Fix a mixed up citation.
Needed for my dissertation research and my teaching.
Understand the use and applicability of a specific research tool.
Complete an assignment, broaden my learning, open new avenues of investigation.
Create a higher quality research product.
Get reliable information about topics I know little of.

18 of 27

Faculty
Create a syllabus for core course for a recently-approved PhD program in WGSS.
Get funded proposals so the library can take its cut out of the overhead and then be able to get more books and better online access and grow.
Keep up with current research in my area and quickly access needed articles.
Access is absolutely essential to my research, especially since I am a faculty member who is not on campus.
Speed up my writing.
Tenure and promotion.
Complete publications on research projects in progress.

19 of 27

Chicago Running Tours & More

While the findings seem very promising, I want to share a few lessons learned from our successive rounds and some next steps.

First, we know we’d like to make changes to our survey design. For example, we’d like to ask fewer questions. A total of 9 questions is still longer than what we prefer. One possible solution is to alternate questions, for example in one round ask a question about core-ness and in another ask about purpose of use.

Another lesson or challenge is to prevent patrons from receiving the survey multiple times. We have several options to try such as only deploying it for a short time frame such as one hour per day for one week. Or, we could limit the number of responses to a given journal, where once we get 5 responses, the survey is turned off.

We could explore if capturing existing data from proxy server would suffice for our assessment needs.

For our next steps we know will employ some of the options above to resolve duplicate responses, we will revise survey questions based on existing responses and Terry Reese is looking at how he might use Qualtrics API. Qualtrics has a pop-up distribution option that looks very nice where you can paste code into your web site, includes animation and takes advantage of Qualtrics full survey backend. However where this differs is what triggers the survey to pop-up; with Qualtrics you would have to figure out how to launch the survey at the journal level.

20 of 27

Code https://github.com/reeset/ics

21 of 27

Questions for you�

22 of 27

If you could only ask one question on the survey what might it be?

23 of 27

What are some ideas to improve the design/methodology of this survey/project?

24 of 27

What are some strategies to balance the annoyance factor of surveying electronic resources?

Access vs. Annoyance

25 of 27

How many survey responses would you need about a single journal title to make an informed decision?

5, 10, 25, 100…

26 of 27

What evidence about collections does your…

…supervisor value?

…library administers value?

…collection development value?

…campus decision-makers value?

…faculty value?

…stakeholders value?

27 of 27

Jane Nichols

Collection Development Librarian

jane.nichols@oregonstate.edu

Terry Reese

Head, Digital Initiatives

reese.2179@osu.edu

Rick Stoddart

Assessment Librarian

richard.stoddart@oregonstate.edu