1 of 17

Content Analysis

Alicia Esquivel

esquivelndsr@gmail.com

9/12/17 | Tech meeting

Share your thoughts on social media using #BHLib

2 of 17

Goals

To determine the extent of the biodiversity literature universe and how much BHL has fulfilled / left to scan?

What are areas of BHL corpus strength and weakness?

bibliographic subject matter
taxonomic subject matter
geographic region
language

From Collections Committee, “Final document on recording questions we want to ask of the BHL collection for a Collections Analysis project,” BHL private wiki.��

3 of 17

Previous analyses

OCLC analysis (2008)

Calculate number of volumes to initially be added to BHL from founding member institutions

Core literature estimate (2010)

Estimate amount of literature yet to be digitized based on estimated amount of species

Fern bibliographies (2015/2016)

Create a subject bibliography and compare literature to CoL species list

4 of 17

Literature Review

Steps of data gap analysis (DGA)

Scoping the analysis and setting the expectations
Assessing the universe of accessible data
Identification of data gaps
Synthesis and dissemination of the outcomes
Prioritization of gap-closing, demand-driven data discovery and publishing activities
Evaluation of the DGA exercise

Gaps exist in data along many dimensions: space, time, taxonomy, subject, environment, etc.

From Arino, A., Chavan, V. & Otegui, J. Best Practice Guide for Data Gap Analysis for Biodiversity Stakeholders. GBIF.

5 of 17

Data Specific Analyses

Analyses using BHL exports:

temporal
taxonomic

Analyses using BHL full text:

topical
geographic

6 of 17

Temporal

BHL Items by Year

7 of 17

Temporal

8 of 17

Taxonomic

9 of 17

Taxonomic

10 of 17

Taxonomic

11 of 17

Topical

JSTOR collaboration

graphs

12 of 17

Geographic

+ plotly

Geoparsing research to analyze the collection geographically. Mine the place names from the text and plot them onto a map. There are still a lot of issues with geoparsing accuracy, it’s still not a perfect technology and relies on OCR which also isn’t 100% accurate but can give us some fun information. Recently I’ve had most success with CLIFF and plotly.

CLIFF is a geoparsing tool from MIT, that is open-source, and built on other tools such as Stanford’s Named Entity Recognizer and the GeoNames gazetteer. To use CLIFF, I set up a local server in a virtual machine to send http requests.

I have some preliminary results, based on a handful of texts from BHL (4 items). The processing is still a bit slow (this took 5.5 hours) so there is work to be done in speeding it up. There is also a way to exclude demonyms (French v. France). This test identifies place names and resolves them to a country location, so not a granular look.

This can give us an idea of which areas are not as well represented in BHL

potential to cross geographic representation with taxonomic representation in BHL--map names using occurrence data from GBIF

13 of 17

Statistical Analysis

capture in WorldCat
capture in Google Scholar
capture in BHL

14 of 17

User Survey

Have you come across gaps in BHL content coverage?

(Yes/No)

If yes, please describe (for example, but not limited to - gaps in taxons, time periods, ecologies, geographies, institutions, authors, and volume gaps in serial/journal titles, etc.)

(Free text)

��

15 of 17

User Survey

606 total respondents

246 respondents answered “No” gaps in content

360 respondents answered “Yes” gaps in content

235 respondents provided free text about missing content in BHL:

Missing journal/serial issues (100 respondents)

Lack of geographic coverage/text in other languages (34 respondents)

In copyright material (27 respondents)

16 of 17

Recommendations

Improve BHL data exports and documentation
Add BHL citations to Wikidata to clean BHL metadata
Explore kingdom filtering for browsing BHL collection
Explore kingdom filtering for targeting underrepresented taxonomic groups in BHL
Explore filtering into further classifications such as Phylum, Class, Order, etc.

17 of 17

Thank You!

Questions?

Alicia Esquivel

9/12/17| Tech meeting

Stay Connected with BHL!

Follow @BioDivLibrary on social media

Join our Mailing List: library.si.edu/bhl-newsletter-signup