Building an annotated bibliography of open data research

Research Assistant Handbook

Background information

Project goals:

Over 2013/14 the Open Data Research Network has developed a repository of research publications relating to open data. This project seeks to add to that collection, and to annotate it with structured meta-data, turning it into a valuable annotated bibliography which can assist practitioners and researchers in the future.

Structured meta-data, clear abstracts, and selected extracts

This project also provides an opportunity to experiment with different approaches to annotating and visualising a collection of articles, case studies and blog posts.

The collection:

The current Open Data Research collection, managed in Zotero, contains social research articles, presentations and case studies relating to open data. It initially developed with an emphasis on open data in developing countries contexts, and on the social impacts of open government data (OGD).

The goals of the collection are to:

It does this through the selection of items to include, and through detailed annotation of articles.

Selection criteria:

Items can be included in the collection when they:

Academic and peer-reviewed articles are preferred, but well researched magazine and newspaper articles, blog posts, reports and white papers and presentations can also be included.

Whilst readiness studies, theoretical papers and technical discussions can be included in the collection, these are lower priority for tagging and categorisation unless they address issues of open data implementation and impact.

Look for items to add to the collection through literature searches and checking the references of existing entries in the collection.

Managing the bibliography

The bibliography and it’s annotations are managed through Zotero.

Getting started

To edit entries you will need to be a member of the Open Data Research group on Zotero and to have the Zotero standalone client software installed.

Once you have joined this group you should be able to access the relevant group folders in your local Zotero client.

Use the   drop-down at the top-right of the main listing pane to turn on the ‘Extras’ field. This is used to record the status of entries (‘Coded’, ‘Candidates’ for coding, and ‘Draft’)

Adding entries

See the Zotero Documentation for details of how to add entries to your library.

You will need to drag new entries from your own library into the shared Open Data Research group in the Zotero client.

Additions and edits you make to the group are reflected on the Zotero website, and in the client software for any other members of the group.

Adding meta-data

We are looking to include three forms of meta-data for the majority of items in the collection:

The full list of meta-data items can be found at here, including details of whether they are required or optional..

Basic meta-data

For every item, check in Zotero that the carefully check that the item type, title, author and editor details, journal names and other key details are accurate.

You should ensure that a URL is provided for every item. Zotero does not make the PDFs attached publicly available for download by other group members - so it is vital that you provide a link to where the article can be accessed. Look for an open access source wherever possible.

Write or adapt an abstract for each item. This should be a maximum of two paragraphs, and should summarise the goals and key findings of the item. Sometimes an item will have an existing abstract you can copy - on other times you will need to create this from the executive summary and other content.


Detailed coding

Review each item carefully, and then provide values for each of the relevant classifications indicated in the Schema spreadsheet.

You enter these from the ‘Tags’ tab in Zotero editing.

For each kind of classification there is a tag prefix, and a list of allowed values.

For example, the ‘geo’ tag is used to indicate which countries or regions the article concerns. Countries are identified using their ISO 2-digit code, so an article about the United State should be tagged ‘geo:us’.

Codes exist for:

The Schema spreadsheet includes a validator which, once your tagged content is synchronised with the Zotero server, can be used to check if you have entered allowed values. See the ‘Item Validator’ sheet, and enter either the title of ID of the item.

Extracts

As you work through the article, look for discrete short paragraphs or content which offer important insights. These may be:

Not all articles will have obvious extracts, but for some there will be a number of short sections which can be drawn out. These extracts may be surfaced in visualisation tools, to help readers find key insights, and then dig deeper into the articles they are from.

To add an extract, use the ‘Notes’ tab in Zotero.

Paste in the text from the document, in quote marks, and prefixed with the page number (see screenshot example). If neccessary, provide a line of contextualisation outside the quote marks.

Tag the extract with ‘extract:case-study’, ‘extract:finding’ or ‘extract:data’, as well as any other relevant tags from those noted above.

Task tracking

To keep track of which articles have been categorised or not we use the ‘Extra’ field in Zotero.

Enter ‘candidate’ for articles you are considering for categorisation; ‘coded’ for articles that have been categorised, ‘draft’ for work in process or ‘considered’ for items that have been reviewed, but are not suitable for classification in this round of work.

Targets:

We are aiming towards: