Doing Impactful Research

Focus of this workshop

What it says on the tin

More specifically, how to:

1. Come up with important research questions
2. Pick which questions to pursue
3. Come up with a "theory of change" for research
4. Assess research impact
5. Be and stay motivated and productive (maybe)
6. Manage an organization, staff, or mentees to help them with all the above (maybe)

NOT object-level/domain-specific things to do with just doing research well

Meta

• Try to think about concrete, near-term applications of what we discuss
• Structure for each topic:
• Brief lecture / group discussion
• Breakout rooms
• Brief group discussion (including concrete takeaways)
• If you want to read more about these topics, I recommend my Readings and notes on how to do high-impact research

Who the hell am I?

• Researcher at Rethink Priorities and FHI
• Managing 2 interns and mentoring 1 SERI fellow
• Temporarily a grantmaker on the EA Infrastructure Fund
• Have done longtermist research for (only?) 1.5 years
• Think a lot about the topics this workshop covers
• Disclaimer: This workshop is my personal views only

1-2: Coming up with & prioritising among research questions

• How have you personally done this so far?
• Do you know of any other methods for that that seem promising/unpromising?
• Concretely, how can you apply these ideas to your fellowship?

1-2: Coming up with & prioritising among research questions

​

​

Some possibilities for generation:

• Just do what a more senior person suggests you do
• Ask decision-makers what would help them
• Draw on existing lists
• Work forwards from gaps in existing work, from your curiosity, etc.
• Work backwards from how you want the world to be different
• Build up your own list over time and occasionally refer back to it
• Where? What structure? Shared with other people?

​

Some possibilities for prioritisation:

• Many of the above things can be used for prioritisation too
• Make a spreadsheet and rate the ideas against criteria
• What should the criteria be?
• Just thinking about pros/cons

1-2: Coming up with & prioritising among research questions

• How have you personally done this so far?
• Do you know of any other methods for that that seem promising/unpromising?
• Concretely, how can you apply these ideas to your fellowship?

1-2: Coming up with & prioritising among research questions

​

​

Breakout rooms

​

Group discussion

3: Theory of change

Some (somewhat related) dimensions to consider:

• High vs low cohesion across an org
• Top-down vs bottom-up
• I.e., high control/guidance vs high autonomy
• Planner vs Hayekian
• Backward-chaining vs forward-chaining
• Charity Entrepreneurship vs FHI
• Reactive vs proactive
• Normie plebs vs Steve Jobs

Group discussion

3: Theory of change

• Should you always/often/ever come up with an explicit theory of change for your research?
• Should you think about what actions/decisions this research might ultimately inform?
• (E.g., decisions about what future research is done, what careers people take, what grants are made)
• What do you think of Charity Entrepreneurship’s approach of surveying decision-makers to inform what questions they investigate?
• Should you think about who the research ultimately needs to reach in order to be impactful?
• Should you think about how to disseminate/communicate the research so it reaches those people and has those impacts?
• Should researchers/orgs do this sort of thing more/less often? More/less explicitly?
• Maybe trying to be too explicit, “legible”, and decision-maker-driven with this would result in missing some important topics that are sufficiently neglected and novel that people wouldn’t realise they matter until the research is done?

3: Theory of change

​

​

Breakout rooms

​

Group discussion

4: Impact assessment

• Do you personally try to assess the quality/impact of your work? How? Why?
• Do you know of any other methods for that that seem promising/unpromising?
• Do you think researchers/orgs should, on average, make more or less effort to assess the quality/impact of their work? Or that they should change how they assess this?
• E.g., Rethink Priorities surveys the decision-makers they want to reach, to see what they thought of their work. I tried a similar thing before joining RP. But it seems very rare. Should it be more common?

4: Impact assessment

​

​

Breakout rooms

​

Group discussion

Wrap-up and takeaways

1. What’s one thing you found particularly interesting/thought-provoking?
2. What’s one concrete action you’re now considering taking within the next 3 months?

​

Silent writing, then share at least your Q2 answer in the chat

​

That’s all! Please fill in this quick feedback survey!