1 of 58

Searching the

Grey Literature

2 of 58

Searching the grey literature

  • What is grey literature?
  • Why include grey literature?
  • How and where to find grey literature
  • Precision searching of grey literature
  • Exporting grey literature

2

3 of 58

What is grey literature?

  • Information outputs produced by individuals or organizations outside of the traditional publishing and distribution channels.
  • What are some examples? Please put an example that you can think of in chat.

3

4 of 58

What is grey literature?

  • Technical reports
  • Conference proceedings
  • Preprints
  • Patents
  • Reports

  • Working papers
  • Theses and dissertations
  • Unpublished & ongoing studies

4

  • Government documents (differences of opinion)

5 of 58

Let's look at some examples!

6 of 58

6

7 of 58

7

8 of 58

8

9 of 58

9

10 of 58

10

11 of 58

11

12 of 58

12

13 of 58

13

14 of 58

Why include grey literature?

  • Practical reasons
  • Bias reasons
  • Equity reasons

14

15 of 58

Why include grey literature?

  • Many conference presentations never become articles
  • Theses and dissertations are rich in detail
  • Mentioned in both Cochrane and Campbell documentation
    • Up to 10% of citations in Cochrane & Campbell reviews
  • Importance in different disciplines varies, highly valued in some disciplines

15

Practical reasons

16 of 58

Bias in research synthesis

16

Reviews

Publications

Research Reports

Studies (all data collected)

Bias reasons

17 of 58

Bias in research synthesis

17

Reviews

Publications

Research Reports

Studies (all data collected)

17

Negative results

Positive results

18 of 58

Bias in research synthesis

18

Reviews

Publications

Research Reports

Studies (all data collected)

18

Negative results

Positive results

Outcome reporting bias

Publication bias

Confirmation bias

19 of 58

Bias in research synthesis

19

Evidence

synthesis

Publications

Research Reports

Studies (all data collected)

19

Negative results

Positive results

Grey Literature

Helps thwart the positive results bias

20 of 58

Peer review is very white and western

"Many of the researchers in my areas are wanting to make sure they're amplifying and representing the research of multiply marginalized and underrepresented voices in their area of research. What are strategies for either making sure this is considered in the protocol design and/or the whole process of the evidence synthesis project?"

20

Marsolek, W., Farrell, S. L., Kelly, J. A., & Cooper, K. (2021). Grey literature: Advocating for diverse voices, increased use, improved access, and preservation. College & Research Libraries News, 82(2), 58.

Equity reasons

21 of 58

How to find grey literature

Requires creativity and tinkering.

Go to the source:

  • Nonprofits/NGOs
  • Government Agencies
  • Professional Organizations
  • Discipline-dependent

21

22 of 58

Good News!

Some grey lit is easy to locate.

  • Conference proceedings are often indexed in bibliographic databases.
  • Theses/dissertations can be easy to search for

22

23 of 58

  • Look at other protocols and/or evidence syntheses on your topic. Did they include details on grey lit sources searched?
  • GreyNet International has some guidance http://www.greynet.org/
  • Grey literature search checklists (but mostly in the health sciences),

23

Some resources that might help

24 of 58

Where to find grey literature

There is no one place to search

  • Institutional repositories (e.g. Digital Commons)
  • Subject repositories (e.g. AgEcon Search, arXiv)
  • Government docs (e.g. ERIC, Agricola, NTIS)
  • Theses and Dissertations (Proquest, Theses Canada)
  • Grey lit databases - OpenGrey discontinued :(
  • Ask your subject experts! (Is there a conference dedicated to this topic?)

24

25 of 58

Do traditional databases include grey lit?

  • Yes, but…
    • Not much, not all types
    • Inconsistent
  • Some in Agricola, ERIC, Compendex, EconLit, ABI Inform, and more!
  • Google
    • Google Scholar – Little in the top results
    • “Big Google” - limit by domain (e.g. site:who.int/ gender neutrality)

25

26 of 58

Example: Grey literature sources for agricultural topics

Africa Rice

Australia Centre for International Agricultural Research

Biodiversity International

Campbell Collaboration

Centre for Evidence-Based Agriculture

CGIAR Research Centers and Research Programs

Cochrane Collaboration

Council for Agricultural Science and Technology (CAST)

CTA Technical Centre for Agriculture and Rural Cooperation

Embrapa

26

27 of 58

27

Valuable

resource

for grey lit sources, especially in social sciences

28 of 58

Searching grey literature

  • Think basic (e.g., character limits, unclear syntax)
  • Site searching sometimes works (e.g., "street harassment" site:un.org)
  • Bad news :( Many grey literature platforms are slow, difficult to use and/or not well organized
  • Bad news: It's often ephemeral = may need to try the WayBackMachine

28

29 of 58

Grey lit search example

Topic: The economic benefits and costs of trees in urban forest stewardship

29

30 of 58

Grey lit search example

Topic: The economic benefits and costs of trees in urban forest stewardship

30

31 of 58

Grey lit search example

Topic: The economic benefits and costs of trees in urban forest stewardship

31

Urban forest AND tree

32 of 58

32

33 of 58

33

33

147 results

34 of 58

34

34

35 of 58

35

35

36 of 58

36

36

42 results

37 of 58

37

Searching the entire Web site is sometimes necessary

38 of 58

38

Hand searching of grey literature

39 of 58

Search engines might be doing a variety of things to your query terms

39

40 of 58

Barrick, K., & Riegelman, A. (2021). Phrasing in reproducible search methodology: The consequences of straight and curly quotation marks. College & Research Libraries, 82(7), 978.

40

41 of 58

Exercise - Searching a grey lit site

  • Overseas Development Institute - https://odi.org/
    • What methods of poverty alleviation have been utilized in India in the last decade?

  • Organisation for Economic Cooperation & Development - https://www.oecd-ilibrary.org/
    • What factors contribute to reading performance in teens?

  • SocArXiv - https://osf.io/preprints/socarxiv/
    • What changes have religious denominations made in response to the pandemic?

41

Bonus if you have time: How would you get the results out of the website/repository and to the team to screen

42 of 58

Comments or questions

about the site you tried?

42

43 of 58

Exporting results to facilitate screening

43

44 of 58

Exporting grey literature

  • Full text is sometimes easy to find
  • Few have good options
  • Citation manager issues
  • Workarounds
    • Copy-and-paste
    • Review within each database (worksheet)

44

🔑

45 of 58

Manually creating RIS files

  • Research team will need to screen grey lit
  • Screening tools often read Research Information Systems (RIS) file types
  • Citation managers
    • Manually creating a new reference
      • Data entry in metadata fields
    • Browser plugins
      • Attempts to retrieve metadata from web pages

45

46 of 58

Documenting your grey literature search

46

47 of 58

47

47

48 of 58

48

48

49 of 58

49

49

Grey Literature Tools – developing but not quite there

⚠️ 🚧

50 of 58

Interested in learning more?

Good news! Whole books and articles are dedicated to grey literature.

50

51 of 58

Interested in learning more?

Searching the Grey Literature

By Sarah Bonato

2018

51

52 of 58

Interested in learning more?

Marsolek, W. R., Cooper, K., Farrell, S. L., & Kelly, J. A. (2018). The types, frequencies, and findability of disciplinary grey literature within prominent subject databases and academic institutional repositories. Journal of Librarianship and Scholarly Communication, 6(1).

52

METHODS Over 100 subject databases utilized by academic researchers and the IRs of over 100 academic institutions were studied. Document type, search capabilities, and level of curation were noted. RESULTS Grey literature was present in the majority (68%) of the literature databases and almost all IRs (95%) contained grey literature.

53 of 58

Interested in learning more?

Cooper, K., Marsolek, W., Riegelman, A., Farrell, S., & Kelly, J. (2019). Grey literature: Use, creation, and citation habits of faculty researchers across disciplines. Journal of Librarianship and Scholarly Communication, 7(1).

53

54 of 58

Interested in learning more?

Zahra Premji & Amy Riegelman maintain a preprint repository and aggregator syntax guide

https://lib.uvic.ca/preprintsearchsyntax

54

55 of 58

Take home messages

  1. Ask yourself how important the grey lit is for your topic.
  2. Advocate for including grey lit (or be prepared to justify your choice 'not to' during peer review process) when it makes sense
  3. Be creative and patient.
  4. Flex your traditional reference skills to find relevant literature

55

56 of 58

What question(s) do you have about grey literature and evidence syntheses?

56

57 of 58

What question(s) do you have about grey literature and evidence syntheses?

What challenges have you faced in searching grey literature?

57

58 of 58

Thanks!

Any questions?�You can find me at:

  • aspringe@umn.edu

58