Search and Research
Original draft:
Jordan Covvey, Mike Gionfriddo, Chyna McClendon,
Authors:
Karen Word, Hao Ye, & C4R CENTER Team
Community for Rigor
What are the rigor problems? (for reviewers)
2
What is this unit teaching? (for reviewers)
3
Lesson 1
Lesson 2
What is this unit teaching? (for reviewers)
4
Lesson 3
Lesson 4
What is this unit teaching? (for reviewers)
5
Lesson 5
Unit overview
6
Orienting to a new adventure
Lesson 1
7
8
To tell the story of search, we need to start at the beginning. Imagine we are embarking on research on a topic that is entirely new to us.
Lesson 1: Orienting to a new adventure
9
Research projects start in many ways!
Familiar topic, new approach
Familiar approach, new topic
Broad interest in a topic
Ready-to-go project
A great idea!
Lesson 1: Orienting to a new adventure
10
Imagine we are joining a project that already has a plan.
We’ll consider a simplified story.
Lesson 1: Orienting to a new adventure
11
Study measurement: Word list task.
Hypothesis: Sleep deprivation interferes with memory.
Prediction: If we deprive subjects of sleep, their performance on a memory test will decline.
Before study: recruit subjects, collect demographic data
You’re hired! Here’s a project
Student volunteers
Total sleep deprivation (treatment).
Standard sleep (control).
Lesson 1: Orienting to a new adventure
12
What are the risks of proceeding without detailed knowledge about sleep deprivation and memory?
Lesson 1: Orienting to a new adventure
There is never enough time to learn everything at the outset. The time to start work is usually ASAP.
13
Halfway through, we chat with another scientist about our plan. They ask: “Why are you using a word recall test instead of [...]?”
Let’s say we get started.
We aren’t sure. Was there a better test to use?
Lesson 1: Orienting to a new adventure
14
Now imagine: we take our completed research to a conference. Someone says: “Didn’t so-and-so publish something similar last year?”
Surely, someone would have checked for this?
Lesson 1: Orienting to a new adventure
15
In this story, we might be unhappy with our collaborators. We trusted them to know! But here and always, we are responsible for the research we conduct.
Learning about a new topic can feel overwhelming. How can we apply rigor to learn what we need quickly and effectively?
Lesson 1: Orienting to a new adventure
16
Learning everything is impossible, but with strategy, we can rapidly build a strong foundation.
We need:
Knowledge to explain key decisions about our project.
Confidence that we haven’t missed key publications that could affect our work.
Lesson 1: Orienting to a new adventure
17
How would you approach learning about a new topic? Where would you begin, and when would you be ‘done’?
Think
Pair
Share
Discussion
Lesson 1: Orienting to a new adventure
ACTIVITY: The right tool(s) for the job
First steps often involve a tool. How do you choose? Test your knowledge of the strengths and limitations of databases, search engines, and LLMs.
18
Lesson 1: Orienting to a new adventure
19
Increasingly, tools are combined to leverage diverse strengths. This can make it harder to know exactly what the limitations are. Still, knowing what questions to ask about a tool is a solid start to get the resources we need.
Now, let’s think about what we need to learn!
Lesson 1: Orienting to a new adventure
20
Let’s consider that question we couldn’t answer about our choice of method. What would we need to know to explain that decision?
We want to know that the method is a practical and reliable way to produce evidence
We want to know that that type of evidence will usefully inform the theory under investigation
Lesson 1: Orienting to a new adventure
21
A framework for scientific learning
How we think things work, and their alternatives.
Theory
Method
Available techniques and their challenges.
Evidence
Observations that inform theory or method.
Lesson 1: Orienting to a new adventure
22
We can’t evaluate a theory without evidence
How we think things work, and their alternatives.
Theory
Method
Available techniques and their challenges.
Evidence
Observations that inform theory or method.
Lesson 1: Orienting to a new adventure
23
How we think things work, and their alternatives.
Theory
Method
Available techniques and their challenges.
Evidence
Observations that inform theory or method.
But without any theory, evidence is just information
Lesson 1: Orienting to a new adventure
24
How we think things work, and their alternatives.
Theory
Method
Available techniques and their challenges.
Evidence
Observations that inform theory or method.
We need evidence to know that a method works
Lesson 1: Orienting to a new adventure
25
How we think things work, and their alternatives.
Theory
Method
Available techniques and their challenges.
Evidence
Observations that inform theory or method.
But the meaning of evidence depends on the method used to generate it
Lesson 1: Orienting to a new adventure
26
Let’s see how these categories can help to answer a question from earlier.
Lesson 1: Orienting to a new adventure
27
“Why did you use that method?”
Method
What should our memory test tell us about brain function?
1
Evidence
2
How will data from this test help support or rule out existing theories?
Theory
3
Lesson 1: Orienting to a new adventure
28
Each piece is a manageable learning goal, or we can tackle them all together.
Lesson 1: Orienting to a new adventure
29
In other words, we will have: Knowledge to explain key decisions about our project
When we can connect theory, method, and evidence for a component of a project, we can be confident that we understand its role.
Lesson 1: Orienting to a new adventure
30
We still need: Confidence that we haven’t missed key publications that could affect our project.
Lesson 1: Orienting to a new adventure
31
Knowledge to explain key decisions about our project and confidence we haven’t missed key publications are related.
Lesson 1: Orienting to a new adventure
Let’s explore how:
32
Lesson 1: Orienting to a new adventure
Knowledge:
Explore, scope, and learn.
5. Specify method
Confidence:
Search for key details.
6. Set search goals
7. Literature search
1. Formulate questions
2. Read strategically for answers
3. Evaluate learning
4. Consolidate learning
Publications we don’t want to miss:
33
Replication is great, but it should never happen by accident.
The same research
Has a metric been debunked as an indicator? An intervention deemed unrealistic?
Relevance implications
Could a different design have ruled that out?
Alternative hypotheses
A mediator that could have been measured.
Easy improvements
That technique actually detects D as well as A.
Method invalidations
Lesson 1: Orienting to a new adventure
34
But… how?
We need a process!
Lesson 1: Orienting to a new adventure
Where could we begin?
35
Discuss it with an LLM.
Talk to someone who knows more.
Read a good review article or three.
Lesson 1: Orienting to a new adventure
36
What do all of these resources have in common?
They give us things to read! What we do with those things makes all the difference.
Lesson 1: Orienting to a new adventure
What we’ve learned, where we’re going
37
Lesson 1
Exploratory learning
Lesson 2
Targeted search
Lesson 3
Iterative search
Lesson 4
Learning with LLMs
Lesson 5
Lesson 1: Orienting to a new adventure
What we’ve learned:
38
Taking time to learn and find resources about a project is key to rigor.
With a little strategy, we can do this efficiently and effectively .
Lesson 1: Orienting to a new adventure
Exploratory learning before a search
Lesson 2
39
40
Before we continue: Why are you taking this lesson?
Make a note with 2-3 questions you think we might answer here.
Lesson 2: Exploratory learning before a search
41
Science is a community effort.
Reading what people publish is only one way to learn from them.
What should we read? Ask for suggestions!
Have we understood correctly?
Ask for discussion! We’ll get help with this later in the lesson.
Lesson 2: Exploratory learning before a search
42
In this lesson, we will discuss a process of reading to prepare for:
A discussion with an expert to confirm and extend our initial learning.
A targeted literature search.
Lesson 2: Exploratory learning before a search
43
Started with a long list of papers and only had time to look at a few?
Read a paper carefully, only to find a more suitable paper later on?
Taken longer than you intended to read a paper?
Have you ever … ?
Wondered how (or if) anybody stays on top of their reading list?
Read and marked up a paper but still felt like you didn’t really understand it?
Assumed a paper was authoritative, only to have it called into question later?
Lesson 2: Exploratory learning before a search
44
We are here to help!
These frustrating challenges are widely experienced and rarely discussed. We’re going to change that and share ways to tackle them!
Lesson 2: Exploratory learning before a search
45
How do you read to learn about science? What strengths or tricks do you have? What is one thing you wish you could improve?
Think
Pair
Share
Discussion
Lesson 2: Exploratory learning before a search
46
All researchers know how to read and learn.
But, we all read and learn differently! Some methods are more efficient for certain purposes than others.
Lesson 2: Exploratory learning before a search
The reading process:
47
1. Formulate questions
2. Read strategically for answers
3. Evaluate learning
4. Consolidate learning
Recommendation
Casual search
Readings come from:
Lesson 2: Exploratory learning before a search
Where might we start?
A review article (best!).
Website, LLM output, etc (fine!).
The same strategies apply to all reading.
48
Reading is necessary but not sufficient for learning.
Thinking is required!
Lesson 2: Exploratory learning before a search
Goal-directed reading within the process:
49
b. Plan a strategy
Identify goals in exploring a topic
a. Identify a resource
2. Read strategically for answers
1. Formulate questions
Lesson 2: Exploratory learning before a search
50
Why are goals useful?
They make it easier to connect new information to our purpose and interests.
Goals aid learning!
We can focus on the most useful parts of the most useful resources.
Goals aid efficiency!
Lesson 2: Exploratory learning before a search
51
Method
Sleep protocols, memory tasks, timing, population all vary between studies.
Evidence
Some findings are robust; others inconsistent or tentative.
Competing theories make different predictions.
Theory
Lesson 1: Orienting to a new adventure
What goals do we have for our study?
1. Formulate questions
We might be new to our topic, but we still have prior knowledge.
A few possible questions might arise:
52
Theory, Method, Evidence, Questions
What different methods have been applied?
What other theories have been suggested?
How much do we know about what is not yet known?
What do we know, or suspect, about theory, methods, and evidence?
Do we have a sense of what we might be missing?
How has prior evidence addressed questions?
What gaps remain that lack good evidence? Why?
Lesson 2: Exploratory learning before a search
The solemn oath of strategic reading is: We will not read every word from start to finish without a specific reason for doing so.
53
b. Plan a strategy
Everything else is open to negotiation!
Lesson 2: Exploratory learning before a search
Depending on what we need from a text, we might:
54
Read for data.
Skim for an overview.
Read for interpretation.
Read for details.
Read for gaps.
b. Plan a strategy
Lesson 2: Exploratory learning before a search
55
Skim for an overview
We survey all sections to see what they contain. We might learn things like:
The paper is organized around the timing of sleep deprivation relative to learning. Seems important!
The paper includes a meta-analysis, with some detailed figures that could be worth a closer look.
The authors discuss publication bias and have a long references list. Our first impression of credibility is good.
b. Plan a strategy
Lesson 2: Exploratory learning before a search
We scan the abstract, introduction, and conclusions, looking for text that reveals what the authors think it all means. We might find that:
56
Read for interpretation
They think the strongest effects occur with one type of memory.
They note a widespread problem that weakens their confidence.
They believe recent findings are key, upending long held beliefs.
b. Plan a strategy
Lesson 2: Exploratory learning before a search
We examine figures and tables to see what kinds of data are presented. We might find that:
57
Read for data
One table describes the many features of the included studies.
One figure aligns effects from all papers on a single plot to compare effect size and direction.
There are some figures that might take a little more work to understand, but we can at least identify their topic.
b. Plan a strategy
Lesson 2: Exploratory learning before a search
We text search for sections or references that answer a specific question we have in mind. We might text search for “recovery sleep” and learn:
58
Read for details
Some researchers believe that testing after recovery from sleep deprivation can help separate memory impacts from other effects like attention.
Effect sizes are smaller in studies that allow recovery.
b. Plan a strategy
Lesson 2: Exploratory learning before a search
We scan the introduction & conclusion for words that signal still-missing knowledge (e.g., “unknown,” “future,” “yet”). We might learn that:
59
Future studies are needed with larger sample sizes.
It is not yet known if memory consolidation may be spread across multiple nights of sleep.
The mechanisms connecting sleep deprivation to specific types of memory are still debated.
Read for gaps
b. Plan a strategy
Lesson 2: Exploratory learning before a search
A note on reading for gaps:
60
Lesson 2: Exploratory learning before a search
It can only find gaps that the authors are aware of and willing to discuss.
Text search has limits.
Unstated or unnoticed gaps within a paper may only become obvious with time.
More careful reading may be needed.
With experience and practice, we slowly learn to see what isn’t there.
Gaps are the essence of scientific progress!
Speed reading strategies trade accuracy for speed.
Strategic reading involves saving time by being deliberate about finding and (accurately) extracting only what we need.
61
Topic experts can often read quickly with accuracy. The rest of us need strategy.
Strategy ≠ Speed
Lesson 2: Exploratory learning before a search
All strategies have a time and a place! The key is to be intentional. If our goal is to start and finish a learning task on a timeline, we should try a more disciplined strategy.
62
Should we ever read a whole article, start to finish? What about letting our eyes wander in search of unknown surprises?
Lesson 2: Exploratory learning before a search
Keshav (2007) suggests a more general method for reading for any academic literature.
A paper is read in three “passes.”
Progression to the next pass is optional, depending on purpose and what was learned in the last pass.
The 3 passes:
63
A general approach: The Three Pass Method
3. Comprehensive (can take many hours).
1. Quick scan (~ 5-10 min).
2. Closer examination (~ 1 hour).
Lesson 2: Exploratory learning before a search
64
Ready to try it out?
(optional: worksheet)
Lesson 2: Exploratory learning before a search
Making reading worth the effort
65
1. Formulate questions
2. Read strategically for answers
3. Evaluate learning
4. Consolidate learning
However, evidence supports pausing to reflect, consolidate, and plan instead.
Strategic reading improves efficiency and learning.
It can feel compelling to just keep going!
Lesson 2: Exploratory learning before a search
66
Evaluate:
Which questions have we found answers to?
What couldn’t we find in this resource?
Why might that be?
3. Evaluate learning
Lesson 2: Exploratory learning before a search
67
Have we connected each piece?
Theory
Method
Evidence
Lesson 2: Exploratory learning before a search
3. Evaluate learning
68
Consolidate:
What can we do to commit what we have learned to memory?
4. Consolidate learning
Lesson 2: Exploratory learning before a search
69
What doesn’t work? (Dunlosky et al. 2013)
These are three examples of study strategies that are ubiquitous and feel effective but don’t hold up well in research on learning!
Copied down a summary (without comment)?
Re-read a book to learn its contents?
Used a highlighter or a pen to underscore important text?
Have you ever:
Lesson 2: Exploratory learning before a search
70
What works? (Dunlosky et al. 2013)
e.g. reflecting before referring back
e.g. 1 hour per day for 5 days rather than 5 hours on 1 day
Strong support
Retrieval
Distributed learning
e.g. answering “why” questions
e.g. talking through an equation
Moderate support
Elaboration
Self-explanation
e.g. alternating topics
Interleaving
Lesson 2: Exploratory learning before a search
Now what?
Independent learning is a little different from school. In school, an expert guides and assesses our learning. This is a huge help!
71
1. Formulate questions
2. Read strategically for answers
3. Evaluate learning
4. Consolidate learning
Lesson 2: Exploratory learning before a search
We can get the best results from our efforts by seeking out experts to guide and assess our learning
72
Great conversations support learning!
Who can we reach out to?
Lesson 2: Exploratory learning before a search
73
An “expert” might be:
A presenter we saw at a conference.
A recurring name in bibliographies.
A connection from a colleague.
Anyone with knowledge and experience that could help confirm and extend our learning!
Lesson 2: Exploratory learning before a search
ACTIVITY: Reaching out for help
Writing a “cold” email to an author on a paper you’ve read is an intimidating task. How would you approach it?
74
Lesson 2: Exploratory learning before a search
75
The main goal? Send the email!
At the end of the day, your style is your own. Keep in mind that:
When you make mistakes, you are likely the only one who will remember them.
Most people find outreach emails intimidating.
Successful scientists know this is necessary and valuable.
Lesson 2: Exploratory learning before a search
76
Preparing for a useful meeting
We’ll need to be prepared with questions!
Do we want them to teach?
They will need to understand the idea
It helps to specify the type of feedback
Do we want feedback?
We can prepare a prioritized agenda and keep an eye on the clock
Will time be limiting?
Preparation depends on the expectation:
Lesson 2: Exploratory learning before a search
77
Preparing for a useful meeting
Remember:
Prepare to teach as well as listen.
Expertise has limits.
Before, during, and after a meeting are all good times to record ideas.
Notes, or it didn’t happen.
A quick email can cement the connection and lead to further conversations.
Follow up to affirm
Lesson 2: Exploratory learning before a search
78
Return to the questions you wrote down at the beginning of this lesson! What have you learned? What questions remain?
Think
Pair
Share
Discussion
Lesson 2: Exploratory learning before a search
79
Lesson 2: Exploratory learning before a search
5. Specify method
6. Set search goals
We covered:
Explore, scope, and learn
Next:
Search for key details
7. Literature search
1. Formulate questions
2. Read strategically for answers
3. Evaluate learning
4. Consolidate learning
Remember - we don’t want to miss:
80
Replication is great, but it should never happen by accident.
The same research
Has a metric been debunked as an indicator? An intervention deemed unrealistic?
Relevance implications
Could a different design have ruled that out?
Alternative hypotheses
A mediator that could have been measured.
Easy improvements
That technique actually detects D as well as A.
Method invalidations
Lesson 1: Orienting to a new adventure
Targeted (literature) searching
Lesson 3
81
82
Performing a targeted literature search takes time and effort
When would you do this?
Lesson 3: Targeted (literature) searching
83
Has anyone looked at this topic before? If they have, did they explore all the gaps and explanations?
You want to check that your focused research question is novel.
What are the standard methods for investigating this topic? What choices are involved and what are the justifications?
You want to select the best approach for your focused research question.
In both situations, you want a very specific set of results, but you ALSO want to be thorough…
Lesson 3: Targeted (literature) searching
84
If you don't have a focused research question, wait!
This lesson is about conducting a comprehensive search in a narrow slice of the literature.
Lesson 3: Targeted (literature) searching
85
We're going to use these skills:
Lesson 3: Targeted (literature) searching
1. Identify terms and synonyms—this helps us be thorough about our topic's key concepts.
2. Combine the terms and synonyms using boolean operators and field tags to create a focused search strategy.
Suppose, after lessons 1-2, we chose this focused research question:
"How does acute sleep deprivation affect memory, and specifically verbal recall (in humans)?"
The key concepts here are:
86
acute sleep deprivation
verbal recall memory
humans
Lesson 3: Targeted (literature) searching
Part 1. What are your terms and synonyms?
87
acute sleep deprivation
verbal recall memory
humans
Intervention
Outcome
Population
What are the pieces of your study?
One way of breaking up a research question into key concepts is the PICO framework: population, intervention, control/comparison, outcome (Lasserson et al. 2024).
Lesson 3: Targeted (literature) searching
Each key concept should be as specific as possible.
88
humans
e.g.
Lesson 3: Targeted (literature) searching
What about humans with a particular condition?
Do we care about all humans or specific subgroups?
What about humans of a particular age?
Consider the following:
Generate terms and synonyms
Some papers about a concept may use alternative phrasings.
We want to include all these papers, regardless of the specific phrasing.
89
"verbal recall memory"
"verbal memory"
verbal recall memory
e.g.
Lesson 3: Targeted (literature) searching
"word learning"
Combine synonyms using OR.
To include any paper using any of these different phrasings, apply this construction:
"verbal recall memory" OR "verbal memory" OR "word learning"
90
verbal recall memory
e.g.
Lesson 3: Targeted (literature) searching
"verbal recall memory"
"verbal memory"
"word learning"
OR
91
Antonyms can be useful!
In some cases, the opposite of your concept is a useful term to search for.
For example, for the concept
"verbal recall success",
papers that use
"verbal recall failure" or
"verbal recall deficit"
would be relevant too!
Lesson 3: Targeted (literature) searching
92
In addition to different wordings of our concepts, we can also make use of database index terms.
Lesson 3: Targeted (literature) searching
Many databases use a taxonomy of index terms to assign categorical labels to each paper (similar to hashtags).
PubMed's terms are called Medical Subject Headings (MeSH).
93
verbal recall memory
Lesson 3: Targeted (literature) searching
Many databases use a taxonomy of keywords to assign categorical labels to each paper (similar to hashtags).
In PubMed, the keywords are called Medical Subject Headings (MESH).
94
verbal recall memory
Review the MeSH Tree
When viewing the entry for a MeSH term, be sure to review the tree structure to check if you want a related, more general, or more specific term.
Lesson 3: Targeted (literature) searching
Many databases use a taxonomy of keywords to assign categorical labels to each paper (similar to hashtags).
In PubMed, the keywords are called Medical Subject Headings (MESH).
95
Lesson 3: Targeted (literature) searching
verbal recall memory
https://meshb.nlm.nih.gov/record/ui?ui=D014706
Review the MeSH Tree
When you are viewing the entry for a MeSH term, be sure to review the tree structure to see related terms, including broader or more specific versions
Apply proper search syntax for each term
In PubMed, format each term as "term"[tag], where:
96
verbal recall memory
"verbal recall memory"[tiab] OR "verbal memory"[tiab] OR "word learning"[tiab] OR "verbal learning"[mesh]
Lesson 3: Targeted (literature) searching
97
MeSH terms are exact!
MeSH terms must be the exact word or phrase that appears in the MeSH taxonomy.
✅ "verbal learning"[mesh]
Variants or wildcards will NOT find results!
❌ "verbal learn*"[mesh]
Lesson 3: Targeted (literature) searching
After we have our lists of synonyms for each concept, what is next?
98
"verbal recall memory"[tiab] OR "verbal memory"[tiab] �OR "word learning"[tiab] �OR "verbal learning"[mesh]
"acute sleep deprivation"[tiab] OR "sleep debt"[tiab] OR "inadequate sleep"[tiab] OR "insufficient sleep"[tiab] OR "sleep deprivation"[mesh]
Lesson 3: Targeted (literature) searching
verbal recall memory
acute sleep deprivation
We want the papers in this overlap!
Combine the separate search queries with AND.
99
Lesson 3: Targeted (literature) searching
"verbal recall memory"[tiab] OR "verbal memory"[tiab] �OR "word learning"[tiab] �OR "verbal learning"[mesh]
"acute sleep deprivation"[tiab] OR "sleep debt"[tiab] OR "inadequate sleep"[tiab] OR "insufficient sleep"[tiab] OR "sleep deprivation"[mesh]
verbal recall memory
acute sleep deprivation
AND
("verbal recall memory"[tiab] OR "verbal memory"[tiab] OR "word learning"[tiab] OR "verbal learning"[mesh])
AND
("acute sleep deprivation"[tiab] OR "sleep debt"[tiab] OR "inadequate sleep"[tiab] OR "insufficient sleep"[tiab] OR "sleep deprivation"[mesh])
100
Lesson 3: Targeted (literature) searching
"verbal recall memory"[tiab] OR "verbal memory"[tiab] �OR "word learning"[tiab] �OR "verbal learning"[mesh]
"acute sleep deprivation"[tiab] OR "sleep debt"[tiab] OR "inadequate sleep"[tiab] OR "insufficient sleep"[tiab] OR "sleep deprivation"[mesh]
verbal recall memory
acute sleep deprivation
AND
Find terms for each concept:
Combine concepts together into a single search strategy:
That's a lot…let's do a little practice!
101
Lesson 3: Targeted (literature) searching
Recap our steps
aggregate synonyms with OR.
find synonyms and alternate phrasings.
find matching index terms (if the database supports it).
apply quotes and field tags.
Make sure to add parentheses around the list of terms for each concept.
Combine concepts with AND.
ACTIVITY: Practice building a search strategy
We've pre-generated a list of terms.
Identify and categorize them to build a search query.
102
Lesson 3: Targeted (literature) searching
103
Lesson 3: Targeted (literature) searching
You may need to adjust settings (from "Exact Match" to "Any Fragment") if your search phrase isn't part of an existing �MeSH term.
MeSH includes established concepts. It may not have new, unique, or niche terms or phrases.
Be sure to read the "Scope Note" to check that the definition matches.
Tips for the MESH browser
MeSH terms can also have qualifiers that provide additional detail to the usage of a MeSH term.
Format as "verbal learning/classification"[mesh]
104
verbal recall memory
https://meshb.nlm.nih.gov/record/ui?ui=D014706
Lesson 3: Targeted (literature) searching
105
Tips for concept specificity
Lesson 3: Targeted (literature) searching
How flexible is your concept to your search needs?
Check if you want an abstract concept ("verbal memory") vs a specific measurement method ("paired association test")
Would any kind of memory work, or do you specifically need verbal recall?
106
Lesson 3: Targeted (literature) searching
We focused on [tiab] and [mesh] for this lesson, but there are many more PubMed field tags available.
PubMed filters can help narrow search results by certain properties that don't easily translate into search terms:
Tips for other field tags and filters
type of paper (e.g. review, clinical trial, meta-analysis)
publication date (e.g. last 10 years).
species (e.g. mouse, human)
Don't worry if your initial search isn't perfect—the next lesson is about improving your search.
107
Lesson 3: Targeted (literature) searching
Iterating a literature search
Lesson 4
108
109
Your initial search isn't perfect, and that's OK.
Let's explore how to improve a search.
Lesson 4: Iterating a literature search
Iterating on your search query
110
1. Evaluate search results
2. Propose changes
3. Implement changes and document
4. Run the updated search
Lesson 4: Iterating a literature search
111
("neuroplasticity"[tiab] OR "brain plasticity"[tiab] OR "Neuronal Plasticity"[MeSH] OR "neural plasticity"[tiab])
AND
("memory formation"[tiab] OR "Memory Consolidation" [MeSH] OR "memory encoding"[tiab])
How does neuroplasticity contribute to memory formation?
1869 results.1
That seems like a lot!
Lesson 4: Iterating a literature search
1PubMed search executed �7 May 2026.
112
What are some reasons that a search can return unwanted results?
Think
Pair
Share
Discussion
Lesson 4: Iterating a literature search
113
Results exclude important papers.
Results include irrelevant papers.
Our search query isn't lining up with our topic.
The topic is too broad. (too many results)
The topic is too narrow. (too few publications in this area)
Our topic isn't scoped well.
Lesson 4: Iterating a literature search
1869 results: why?
114
Watch out!
Any individual issue can make the number of results too high or low.
That makes it impossible to diagnose issues just by checking the number of results.
We can only evaluate the effectiveness of our search by looking at the papers.
Let's take this one step at a time, and start with a single paper.
Lesson 4: Iterating a literature search
115
It is well documented that sleep contributes to memory consolidation and it is also accepted that long-term synaptic plasticity plays a critical role in memory formation. The mechanisms of this sleep-dependent memory formation are unclear. Two main hypotheses are proposed. According to the first one, synapses are potentiated during wake; and during sleep they are scaled back to become available for the learning tasks in the next day. The other hypothesis is that sleep slow oscillations potentiate synapses that were depressed due to persistent activities during the previous day and that potentiation provides physiological basis for memory consolidation. The objective of this review is to group information on whether cortical synapses are up-scaled or down-scaled during sleep. We conclude that the majority of cortical synapses are up-regulated by sleep slow oscillation.
MESH: Humans, Memory / physiology, Neuronal Plasticity / physiology, Sleep / physiology, Synapses / metabolism
Igor Timofeev, Sylvain Chauvette
PMID: 28453998 DOI: 10.1016/j.conb.2017.03.019
Lesson 4: Iterating a literature search
Sleep slow oscillation and plasticity
116
What is your first impression?
Relevant
Uncertain
Not Relevant
Scan for our concepts - does the paper focus on neuroplasticity and memory formation AND how they relate together?
Lesson 4: Iterating a literature search
117
It is well documented that sleep contributes to memory consolidation and it is also accepted that long-term synaptic plasticity plays a critical role in memory formation. The mechanisms of this sleep-dependent memory formation are unclear. Two main hypotheses are proposed. According to the first one, synapses are potentiated during wake; and during sleep they are scaled back to become available for the learning tasks in the next day. The other hypothesis is that sleep slow oscillations potentiate synapses that were depressed due to persistent activities during the previous day and that potentiation provides physiological basis for memory consolidation. The objective of this review is to group information on whether cortical synapses are up-scaled or down-scaled during sleep. We conclude that the majority of cortical synapses are up-regulated by sleep slow oscillation.
MESH: Humans, Memory / physiology, Neuronal Plasticity / physiology, Sleep / physiology, Synapses / metabolism
Lesson 4: Iterating a literature search
Igor Timofeev, Sylvain Chauvette
PMID: 28453998 DOI: 10.1016/j.conb.2017.03.019
Sleep slow oscillation and plasticity
118
It is well documented that sleep contributes to memory consolidation and it is also accepted that long-term synaptic plasticity plays a critical role in memory formation. The mechanisms of this sleep-dependent memory formation are unclear. Two main hypotheses are proposed. According to the first one, synapses are potentiated during wake; and during sleep they are scaled back to become available for the learning tasks in the next day. The other hypothesis is that sleep slow oscillations potentiate synapses that were depressed due to persistent activities during the previous day and that potentiation provides physiological basis for memory consolidation. The objective of this review is to group information on whether cortical synapses are up-scaled or down-scaled during sleep. We conclude that the majority of cortical synapses are up-regulated by sleep slow oscillation.
MESH: Humans, Memory / physiology, Neuronal Plasticity / physiology, Sleep / physiology, Synapses / metabolism
Lesson 4: Iterating a literature search
Igor Timofeev, Sylvain Chauvette
PMID: 28453998 DOI: 10.1016/j.conb.2017.03.019
Sleep slow oscillation and plasticity
119
This result seems relevant. What's next?
Scan for additional terms and concepts. (e.g. processes, specific phenomena, hypotheses). These could indicate useful topics to research in more detail or ways to narrow our topic (if needed.)
Lesson 4: Iterating a literature search
120
It is well documented that sleep contributes to memory consolidation and it is also accepted that long-term synaptic plasticity plays a critical role in memory formation. The mechanisms of this sleep-dependent memory formation are unclear. Two main hypotheses are proposed. According to the first one, synapses are potentiated during wake; and during sleep they are scaled back to become available for the learning tasks in the next day. The other hypothesis is that sleep slow oscillations potentiate synapses that were depressed due to persistent activities during the previous day and that potentiation provides physiological basis for memory consolidation. The objective of this review is to group information on whether cortical synapses are up-scaled or down-scaled during sleep. We conclude that the majority of cortical synapses are up-regulated by sleep slow oscillation.
MESH: Humans, Memory / physiology, Neuronal Plasticity / physiology, Sleep / physiology, Synapses / metabolism
Lesson 4: Iterating a literature search
Igor Timofeev, Sylvain Chauvette
PMID: 28453998 DOI: 10.1016/j.conb.2017.03.019
Sleep slow oscillation and plasticity
121
It is well documented that sleep contributes to memory consolidation and it is also accepted that long-term synaptic plasticity plays a critical role in memory formation. The mechanisms of this sleep-dependent memory formation are unclear. Two main hypotheses are proposed. According to the first one, synapses are potentiated during wake; and during sleep they are scaled back to become available for the learning tasks in the next day. The other hypothesis is that sleep slow oscillations potentiate synapses that were depressed due to persistent activities during the previous day and that potentiation provides physiological basis for memory consolidation. The objective of this review is to group information on whether cortical synapses are up-scaled or down-scaled during sleep. We conclude that the majority of cortical synapses are up-regulated by sleep slow oscillation.
MESH: Humans, Memory / physiology, Neuronal Plasticity / physiology, Sleep / physiology, Synapses / metabolism
Lesson 4: Iterating a literature search
Igor Timofeev, Sylvain Chauvette
PMID: 28453998 DOI: 10.1016/j.conb.2017.03.019
Sleep slow oscillation and plasticity
122
Aha, there are two competing hypotheses, and some additional terms for us to consider!
Lesson 4: Iterating a literature search
Sleep slow oscillations.
Synapse potentiation.
Regulation during sleep.
123
Note any review papers!
As a review paper, this result will likely include references that are key studies or experiments on this topic.
Those papers might be useful for details on the design and execution of studies in this field.
Lesson 4: Iterating a literature search
Possible actions from our evaluation:
(note: actions aren't mutually exclusive)
124
Learn more about a specific term to obtain more background knowledge about the topic.
Narrow the focus of our search by including terms for one or more additional concepts.
Read through the paper in detail to obtain information (e.g. further references, methodological details).
Lesson 4: Iterating a literature search
125
Let's look at a few more papers before taking any actions.
Lesson 4: Iterating a literature search
ACTIVITY: Practice evaluating results
Examine three results from the same search. Then propose actions based on your evaluations.
126
Lesson 4: Iterating a literature search
Suppose you decide that there is a new concept that will help narrow your topic.
127
Hippocampus CA3
"CA3 Region, Hippocampal"[mesh] OR "CA3 region"[tiab] OR "hippocampal CA3"[tiab]
Let's add:
Lesson 4: Iterating a literature search
128
Search documentation is a habit that pays dividends, just like to maintaining a good lab notebook!
Be intentional with changes.
Conducting a search is like navigating a maze
Documentation helps you from going down the same dead ends
Lesson 4: Iterating a literature search
Proper search documentation records:
129
Paste your full search query and note any filters applied.
Query
Track time of search to establish a cadence for revisiting searches and remaining informed.
When
Record the database used and consider how you could port a query to another database.
Where
Explain why this is a distinct query and what you aimed for in assembling it.
Why
Document result count and any exemplar results to target for strategic reading.
Results
Lesson 1: Orienting to a new adventure
130
A little goes a long way!
Documentation can start small. �The next time you search, open a spreadsheet and paste in the query! Let complexity evolve as you need it.
Lesson 2: Exploratory learning before a search
We usually want to check that our search returns important papers.
131
e.g. an exemplar paper that your PI or expert colleague gave you
Lesson 4: Iterating a literature search
132
search query | date | # of results | Notes | ex. 1 |
("neuroplasticity"[tiab] OR "brain plasticity"[tiab] OR "Neuronal Plasticity"[MeSH] OR "neural plasticity"[tiab]) AND ("memory formation"[tiab] OR "Memory Consolidation" [MeSH] OR "memory encoding"[tiab]) | 2025-12-01 | 1831 | initial search with concepts "neuroplasticity" and "memory formation" | Y |
TBD | TBD | TBD | add "hippocampus CA 3" as a concept | TBD |
| | | | |
Lesson 4: Iterating a literature search
133
[[big long search query]] AND �"TITLE OF Important Example"
search:
Add a column to your search log to note the result.
Then proceed accordingly:
Yes
No
Great! Continue.
Hmm. Let's see what happened.
Check your results for a specific paper
Lesson 4: Iterating a literature search
134
Does the example contain all of the concepts in your search?
Your search might be overly narrow.
Does the example use synonyms or alternate phrasings of the key concepts of your search?
You might need to update the synonyms for your concept(s).
Troubleshooting with an example
Lesson 4: Iterating a literature search
Iterate your search as needed!
135
1. Evaluate search results
2. Propose changes
3. Implement changes and document
4. Run the updated search
Lesson 4: Iterating a literature search
An effective literature search is:
136
Relevant—the search returns papers that match the desired topic.
Precise—the search does NOT include irrelevant results. (e.g. it is an effective filter)
Educational—Helps expand your knowledge of terms and concepts that are used in the literature.
Lesson 4: Iterating a literature search
You may need a little help to dive into a search. Next we’ll discuss using an LLM for exploratory learning!
137
Lesson 3: Targeted (literature) searching
Exploratory learning (with LLM)
Lesson 5
138
139
1. A conversation with a real person to confirm and extend essential knowledge
2. A comprehensive literature search to ensure that no key papers have been missed
If you don’t want to use LLMs, stay with us!
Many of the same strategies can be applied to monitor and improve independent learning.
In this lesson, we will apply an LLM to prepare for:
Lesson 5: Exploratory learning (with LLM)
140
LLMs are useful tools for exploring ideas.
They are effective when we use them to make us think.
Lesson 5: Exploratory learning (with LLM)
141
LLMs are not search engines or databases.
They are often confidently wrong.
Some citations and statements will be inaccurate or entirely fictional.
For more on this, look for our curriculum about LLMs!
Lesson 5: Exploratory learning (with LLM)
Responsibility:
142
1. Formulate questions
2. Read strategically for answers
3. Evaluate learning
4. Consolidate learning
Lesson 5: Exploratory learning (with LLM)
Iterate quickly and endlessly
Customize to our exact level
Opportunity:
Validate everything
Explore ethics & options, e.g. when uploading content
Planning a process
with an LLM
We can ask the model these questions, but like all content, responses will need validation.
Different models have different strengths:
143
Choosing an LLM
Capability ≠ accuracy!
All models will make mistakes, even in their strong suits.
Can it access or review uploads to summarize?
How quickly can it work?
Can it access web search, or only predict links from training data?
Are inputs used for training?
What are its limitations?
Lesson 5: Exploratory learning (with LLM)
144
Goals for optimizing
Resolving roadblocks
Are critical words or equations getting in the way?
Finding the right level
Do we want simpler language, or to see more jargon in use?
Playing with structure
How do we want to organize information?
Steps of a process
List of dominant theories
List of unanswered questions
Pros and cons of prior approaches
Lesson 5: Exploratory learning (with LLM)
ACTIVITY: Playing with prompts
With an LLM, we can generate reading that meets us where we are. In this activity, you will start with your own prompt, then adjust and see the results!
145
Lesson 5: Exploratory learning (with LLM)
146
Lesson 5: Exploratory learning (with LLM)
More ideas for prompt enhancement
Describe audience, methods of interest, exemplar papers, target journals/conferences.
Share context about the project
State the goal and see how a suggested prompt compares to your original prompt.
Ask for feedback on the prompt itself
To reduce pandering, ask for help critiquing “a colleague’s” ideas.
Ask for critical evaluation
Being quizzed aids learning & reveals assumptions.
Ask for questions
Asking for evaluation of LLM response content in a new session or LLM can surface flaws or key details.
Reuse, recycle
147
Lesson 5: Exploratory learning (with LLM)
Neuroscience
What else?
LLMs can zoom out in helpful ways
Psychology
Molecular biology
Where does our work fit? What distinguishes these fields?
Stepwise exploration: sleep & memory
148
“Give me an overview of research on sleep deprivation.” Then, “...memory.”
Starting broad offers context we might otherwise miss.
”... the impact of sleep deprivation on memory”
Does this seem almost as big as the broad topics? Maybe that says something.
“Let’s focus on memory formation”
Scoping down, we get specific about what really interests us.
Lesson 5: Exploratory learning (with LLM)
149
Adjust goals to learn key details
How we think things work, and their alternatives.
Theory
Method
Available techniques and their challenges.
Evidence
Observations that inform theory or method.
Lesson 5: Exploratory learning (with LLM)
150
Getting literature recommendations with LLM
Check the literature regularly and extend what we have learned.
Responsibility to validate
Explaining topics, suggesting references, and summarizing papers are very different tasks.
Right tool for the task
Papers sourced from LLMs should be 1) real and 2) good.
Are the papers useful?
Return to Lesson 2 for a focused, goal-directed reading session!
Why are we reading?
Lesson 5: Exploratory learning (with LLM)
To use LLM to support our own reading, we can:
When using LLM to summarize papers, we should:
151
LLMs and the reading process
Read the article itself for key content.
Know that uploaded content may be used to train the LLM.
Ask the LLM to direct us to key content within the article.
Leverage capacity for synthesis e.g. compare papers, highlight differences.
Type in something we’ve learned for feedback.
E.g “Evaluate the accuracy of this statement: the difference between short and long term memory is…”
Ask for help with unclear statements.
E.g. “What does this paper mean when it says “...””
Lesson 5: Exploratory learning (with LLM)
152
The research process with LLMs
1. Formulate questions
2. Read strategically for answers
3. Evaluate learning
4. Consolidate learning
Lesson 5: Exploratory learning (with LLM)
Using both, carefully, is a powerful way to learn & focus our plans.
LLMs explain well but are often wrong.
Review articles are accurate but harder to read.
A cautious process with LLM can be very productive. However, it still can’t replace a great conversation with a real person!
153
Revisit lesson 2 for help!
A word on reaching out to real people
Building community
Accuracy
Knowing what you actually need to know
Humans are still best at:
Lesson 5: Exploratory learning (with LLM)
154
Lesson 5: Exploratory learning (with LLM)
1. Ask questions
2. Clarify answers
3. Seek reference recommendations
4. Read & synthesize
5. Specify method
6. Set search goals
Explore and scope a topic
Search for key details
7. Literature search
155
Lesson 5: Exploratory learning (with LLM)
Let’s review:
A structured and thoughtful engagement with established work solves headaches and heartaches.
Lesson 1:
Reading should be approached strategically and in conversation with experts.
Lesson 2:
Databases speak specific languages, and we can use those languages to search properly.
Lesson 3:
An effective search is relevant, precise, and educational.
Lesson 4:
Thoughtful (and careful!) interaction with LLMs �can support learning.
Lesson 5:
156
No literature search is perfect; using the right tools and techniques can help us learn what matters!
Lesson 5: Exploratory learning (with LLM)
157
Better Science Every Day
Search and Research
158