PrimoVE: Discovery Imports for the Innocent, the Eager, and the Doomed

STACY MAGEDANZ, CAL STATE SAN BERNARDINO

BRANDON DUDLEY, CAL STATE CHANCELLOR’S OFFICE

​

ECAUG CONFERENCE, JULY 30, 2021

Summary

• Using bepress Digital Commons as a use case, we will introduce importing data into VE, with a particular focus on norm rules. Never wrangled VE norm rules before? Wondering where your data vanished to? This presentation is for you!

Discovery Imports in VE, Overview

• Step 1: Set up an Import Profile
• Step 2: Set up Normalization
1. Create Rules (DROOLS)
1. Rules affect display of data on the full record
2. Rules allow you add data (like resource type) to your import
2. Add Rules to a Task
3. Add Task to your Import Profile
• Step 3: Set linking parameter (Delivery)

Step 1: Import Profiles

Alma Config > Discovery > (Loading External Data Sources) > Discovery Import Profiles�

Profile setup, continued

Potential problems

• Harvest start date: The first time you set it up, make sure you set it for a date well in the past.
• Illegal characters in any bepress field will break the whole import. Sample error message: “An invalid XML character (Unicode: 0xb) was found in the element content of the document.”

We pause now for a haiku:

Errors have occurred

We won’t tell you where or why

Lazy programmers

Potential solution

• How to find what is hidden? DC Tech support says:

“What I've done is basically go through the OAI feed 100 records at a time until I hit an error. I went to [https://your.dcservername.edu/do/oai/?verb=ListRecords&metadataPrefix=dcq&set=publication:yourseries] in Firefox and just kept going through the resumption tokens until I hit the error - if you use Firefox, it'll tell you the exact line with the error. I use batch revise to fix the issue, both so that the user won't receive any email that a revision has been made, and so I can see the HTML code for the abstract directly and see what's going on. Sometimes when copying things over from PDFs into the formatted text field on the submission form, weird characters or lines can end up in the text which can cause an error with the OAI output.”

Step 2: Normalization

Preparation: Expose XML of Digital Commons record

• Base URL

https://your.server.here/do/oai/?verb=GetRecord&metadataPrefix=dcq&identifier=oai:your.server.here:series-123

• Change the series and number as appropriate. Example:

https://scholarworks.lib.csusb.edu/do/oai/?verb=GetRecord&metadataPrefix=dcq&identifier=oai:scholarworks.lib.csusb.edu:etd-1261

The number is NOT the one that appears in the basic DC document URL!

Instead, view page source, then find in page article= The number following this string is the one you want.

https://scholarworks.lib.csusb.edu/etd/238/

View page source to find the real number

Getting started…

• Use the Metadata Editor: Alma Resources > (Cataloging) > Open Metadata Editor
• You will find any existing DROOLs under Rules, then Normalization (Discovery). You may need to use the double arrow to see it!

Stumbling blocks!

• When writing Discovery rules, the rule Options must be set to “Placement of new rules: Local” THIS IS OPTION ISN’T IN THE NEW MDE as of July 2021, only in the Old Editor, under File>Options

​

​

​

​

• When you create a new rule, you must �set it to “Shared.”

​

​

​

​

​

​

​

And now, a word about DROOLS….

Discovery External Data Source Normalization Rules for Dum…SmartPeopleWhoDontYetKnowHowToWorkWithDrools

• Basics for Dublin Core/Generic XML Drools
• A rule consists of one or more conditions followed by one or more actions

​

• Conditions
• True – rule applies to all records
• Exist – rule applies when specific field exists
• Not exist – rule applies when specific field does not exist
• Equals – rule applies when entire contents of field matches
• Not equals – rule applies when entire contents of field does not match

​

• A rule will not do anything unless a condition is met

Drools basics cont.

• Actions
• Copy – copies value from one field to another (value exists in both fields)
• XML rules add a “Copy All” action to bring in multiple occurrences of a field
• Move – moves value from one field to another (value no longer exists in first field)
• Remove – removes value from field

​

• Can modify any of the above with “if equals” to set a specific value condition to trigger the action

​

• Set – sets value of field directly

​

​

Drool basics cont.

• Transformation Actions – use a “TEMP” variable to transform data to be put into the field
• Relies on Java Regular Expressions for matching strings

​

​

​

​

• Use cases – removing unwanted/extra text, adding standard text to fields, cleaning up fields

​

​

A simple example: Write a resource type…�Add a resource type field & value

rule "create etd"

when

(true)

then

set "etd" in "dc"."type"

end

Specify that resource type for discovery

rule "etd"

when

"dc"."type" equals "etd"

then

set "etd" in "discovery"."resourceType"

End

​

*You may write these as separate rules, or string them together in one.

But wait! Primo doesn’t know what discovery resource type that is yet…

• Alma Configuration > Discovery (Display Configuration) > Local Resource Types, then Add local resource type

​

Make sure code matches value in DROOL

End result on the public side…

Advanced examples: Workflow

• Create drool in Metadata Editor
• If creating new field, Add Local Field

Workflow cont.

• Create local field for display
• Fill in Display Label & select “Use parallel Local Field”

Workflow cont.

• Add field to View
• Configure View -> Edit View -> Full Record Services -> Details -> Configure -> Add Field

​

​

Why do I need to know this?

• Need to add a field to display?
• Need to make XML/DC data resemble/match existing MARC data?
• Need to make imported data resemble source system records?

​

Ex: Add thesis program name to record

rule "Primo VE - Lds03"

when

exist "dc"."thesis.degree.name"

then

copy "dc"."thesis.degree.name" to "discovery"."local3"

end

Ex: Add thesis program name to record

rule "Primo VE - Lds03"

when

exist "dc"."thesis.degree.name"

then

copy "dc"."thesis.degree.name" to "discovery"."local3"

end

Ex: Identify Advisor in record

rule "dc:contributor"

when

exist "dc"."contributor"

then

set TEMP"1" to dc value "dc"."contributor"

add suffix (TEMP"1"," (Advisor)")

set "dc"."contributor" to TEMP"1"

end

Ex: Identify Advisor in record

• rule "dc:contributor"
• when
• exist "dc"."contributor"
• then
• set TEMP"1" to dc value "dc"."contributor"
• add suffix (TEMP"1"," (Advisor)")
• set "dc"."contributor" to TEMP"1"
• end

Ex: Set constant field identifying campus as publisher

rule "publisher value"

when

true

then

set "CSUSB ScholarWorks" in "dc"."publisher"

end

Ex: Set constant field identifying campus as publisher

rule "publisher value"

when

true

then

set "CSUSB ScholarWorks" in "dc"."publisher"

end

Ex: Add Thesis Date to Abstract

rule "Primo VE - Lds01"

when

exist "dc"."description.abstract"

then

set TEMP"2" to dc value "dc"."description.abstract"

set TEMP"1" to dc value "dc"."date.created"

remove substring using regex (TEMP"1","(?<=^.{4}).*")

add prefix (TEMP"1","California State University San Bernardino ")

concatenate with delimiter (TEMP"2",TEMP"1"," ")

set "discovery"."local1" to TEMP"2"

end

Ex: Add Thesis Date to Abstract

rule "Primo VE - Lds01"

when

exist "dc"."description.abstract"

then

set TEMP"2" to dc value "dc"."description.abstract"

set TEMP"1" to dc value "dc"."date.created"

remove substring using regex (TEMP"1","(?<=^.{4}).*")

add prefix (TEMP"1","California State University San Bernardino ")

concatenate with delimiter (TEMP"2",TEMP"1"," ")

set "discovery"."local1" to TEMP"2"

end

Add Rules to Tasks, �Tasks to Profile

Add rules to a new Task

• Alma Config > Discovery > (Loading External Data Sources) > Normalization Process Task, then Add Process
• Business Entity is “Discovery BIB Records”
• Type is “Discovery DC Normalization”
• Then add name & description…

FOR TASK LIST, ADD ONE FOR EACH DROOL YOU ARE USING

​

FOR TASK PARAMETERS, CHOOSE THE DROOL(S) YOU CREATED. ORDER COUNTS!

​

Add Task to your Import Profile

• Open your profile, go to the Normalization Tab, choose the Task you just created.

​

Step 3: Delivery

LINKING PARAMETERS

Finally: Set the link and its label.�Back at your import profile….

Helpful Non-ExLibris Resources

• Adding Springshare Libguide Discovery to Primo VE
• How do I add Libguides or Libguides A-Z resources to Primo for discovery?
• Create a local resource type for LibGuides (PDF)
• Digital Collection Data Across the Discovery Landscape (PPT)

More resources

• Primo VE: Data Configuration
• Contact us:
• Stacy Magedanz magedanz@csusb.edu
• Brandon Dudley bdudley@calstate.edu

​