Psychology for Effectively Improving the Future — A Research Agenda


Version 0.2 (August 2022)

Introduction        2

How can psychological and behavioral science inform longtermism and improve the long-term future?        2

Action-guiding information        3

Education and awareness raising        3

Practical interventions and tools        3

Who is this research agenda for?        4

Resources on effective altruism, longtermism, and global priorities research        4

Research areas        5

Morality        5

Understanding people’s values        6

Fostering values conducive to effectively improving the future        7

Rationality, judgment, and cognitive biases        8

Identifying decision tendencies and biases relevant for longtermism        9

Improving rational thinking        9

Forecasting        10

Individual differences        10

Identifying people who tend to either improve or harm the future        11

Psychological predictors of effectively improving the future        11

Developmental        12

Psychological factors related to improving the future across age        12

Fostering critical thinking and pro-social values across age        13

Cross-cultural        13

Cross-cultural differences in attitudes about and decision-making related to effectively improving the future        13

Historical psychology, evolutionary psychology, and anthropology        14

Policy and institutional        15

Psychological obstacles to effectively improving the future in policy contexts        15

Improving thinking and decision-making in institutional settings        16

Future wellbeing        16

Estimating wellbeing in the future        16

Wellbeing in non-human beings        17

Cause-specific issues        18

Global catastrophic risk (in general)        18

Risks from artificial intelligence (AI)        18

Pandemics and biosecurity        19

Global war and nuclear threat        20

Threats to liberal democracies and descent into long-term authoritarianism lock-in        20

Contributors        21

Introduction

The ultimate goal of our research agenda is to help do the most good possible, in line with the principles of effective altruism. In brief, effective altruism advocates for using reason and evidence to take actions that improve the world as much as possible, impartially accounting for the welfare of all its inhabitants, present and future. There are many approaches to this project. For example, one common approach is to find the most cost-effective way of improving health outcomes of the global poor based on randomized control trials. Another is to improve farmed animal welfare by lobbying corporations whose procurement decisions influence millions of animals. These approaches have had enormous success and are good bets for maximally effective interventions based on their short-term, direct impact. However, it is hard to measure their indirect or long-term effects. Their total impact is thus subject to a high degree of uncertainty.

One approach to address this drawback is designing interventions that are robustly positive in the very long term. This alternative approach is based on the idea of longtermism. Longtermism rests on three central premises. First, future people matter. Second, the future could contain a vast number of people. Third, our actions may predictably influence how well this long-term future goes. For example, some actions have the potential to affect the chances of human extinction, with existential implications for all potential humans who could otherwise have lived over millions of years. Importantly, while we cannot evaluate these actions based on randomized controlled trials or a track record of improving welfare, we still can and should choose them based on evidence (e.g., about detecting and preventing dangerous pathogens, trends in technology, the destructive potential of nuclear weapons, etc.) and reason (careful argumentation as to what strategies maximize expected future welfare based on this evidence).

Overall, longtermism seems plausible, has fairly radically revisionary implications if correct, and is currently underexplored in academic research. It also presents challenges: positively influencing the long-term future is hard and involves dealing with enormous uncertainties. For all these reasons, we think that there is tremendous scope for longtermism-relevant research. If you find the idea of longtermism and its implications interesting, and you would be interested in conducting psychological and behavioral science research that informs it, then this research agenda is for you.

How can psychological and behavioral science inform longtermism and improve the long-term future?

Psychological research can inform longtermist thinking in multiple ways. For example, some decisions could have profound implications for the long-term future, such as whether to detonate a nuclear weapon, create a potentially dangerous new technology, and invest in pandemic preparedness. Understanding the psychology of these decisions is one step to ensuring they are made more wisely. Moreover, the optimal policy from a longtermist perspective might often depend on human psychology. What incentives are sufficient to persuade pathogen researchers to take sufficient precautions against laboratory leaks? What makes voters elect potentially dangerous leaders? What long-term trajectories will facilitate human psychological wellbeing? Psychological research to answer these kinds of questions has various concrete paths to impact. The three main paths to impact are by providing key decision-makers with action-guiding information, raising public awareness of the pros and cons of longtermism-aligned action, and by providing science-based tools to influence decision-making to achieve better long-term outcomes.

Action-guiding information

Some psychological insights are directly action-guiding for decision-makers looking to effectively improve the future. While it is important to publish the basis for these insights in high-quality, peer-reviewed journals to ensure their rigor and credibility, it is also important to communicate these insights directly to policymakers, philanthropists, and other key decision-makers. For example:

Education and awareness raising

Given the profound revisionist implications of longtermism, it is important to educate people about it and raise awareness of the pros and cons of acting in line with its principles. Academic research is often the first step to raising broader awareness of important scientific insights. Insights that are rigorously researched and published in academic journals may, with an increased likelihood, be featured in university education, textbooks, journalistic outlets, or popular science books. As an example, consider the widespread awareness of cognitive biases owing to Kahneman and Tversky’s research, and its undermining of the case for laissez-faire policy based on rational actors. Analogously, if longtermism-relevant psychological insights become integrated into public consciousness, it might lead to a more nuanced and appropriate weighting of longtermist considerations by individual decision-makers, philanthropists, and the policy community. For example:

Practical interventions and tools

A more direct approach is to develop and test practical strategies that could empower people to do more good or prevent them from causing longterm harm. For example:

Note that as a heuristic, interventions that could plausibly improve the future in 100 years or later are likely more promising from a longtermism perspective than interventions with a short-term impact but an unclear long-term impact. For example, some behavior change interventions will likely be replaced by better interventions further in the future and, while their impact is valuable, it is not likely to persist for very long. In contrast, an intervention that has some chance of preventing an existential catastrophe could benefit humans for thousands of years or more.

Who is this research agenda for?

This research agenda is intended for researchers in psychological and behavioral science who are interested in global priorities research and effective altruism related research. Researchers interested in pursuing such research, connecting with other like-minded researchers, or providing feedback could visit our website at http://eapsychology.org or contact us at ea.psych.lab@gmail.com (or contact the authors of this agenda directly).

Resources on effective altruism, longtermism, and global priorities research

Readers who want to learn more about effective altruism, longtermism, and global priorities research are encouraged to consult the following websites or books:

Other lists of effective altruism-inspired psychological research questions

Research areas

These research areas and questions are a first draft attempt at outlining the kind of psychological and behavioral science research that can inform longtermism. There are many other topics that would be important to study, even more so than some of the topics here. In fact, tractable research questions will probably test more specific and original hypotheses than we’ve been able to list. We hope that readers will use our high-level questions as inspiration to generate new questions to explore further.

In terms of organization, many of the research areas listed below overlap. This is an intentional and pragmatic decision. Some categories focus on an object-level topic (e.g. morality) and others focus on a particular approach (e.g. developmental psychology). We find it useful to consider research areas from these different angles, even though it means some questions will fall under multiple areas (e.g., by studying the developmental emergence of morality). For each sub-category, we first list an overarching general research question in bold, followed by a few concrete sub-questions. The sub-questions are intended to exemplify the broad questions that follow from the overarching general question. We list some relevant existing literature after the questions for each research area. This list is not meant to be exhaustive, instead, they just include a few key examples. We appreciate that existing research has made substantial progress on many of the research questions we pose. Our research agenda is intended to inspire readers to build on this past research.

Morality

Understanding people’s values, whether they can be influenced, and if so, how, is important to inform longtermism. People’s moral values influence the extent to which they will support efforts to effectively improve the future and how they will conceive of that project. Morality research could take several forms. Some forms would focus on understanding people’s values. For example, we could research how compatible the implications of longtermism are with people’s moral psychology. This knowledge would help determine what form a politically tractable version of longtermism would take. Or, to identify what approaches to improving the future are likely to be neglected by typical altruists, and therefore have low-hanging fruit unpicked, we could research which approaches are perceived least favorably on moral grounds. Other forms of this research would focus on fostering desirable values. For example, to examine how to persuade people of the importance of improving the long-term future, we could research how to make the project as morally appealing as possible and how to influence moral preferences to be more aligned with it.

Understanding people’s values

Fostering values conducive to effectively improving the future

Relevant work:

Berman, J. Z., Barasch, A., Levine, E. E., & Small, D. A. (2018). Impediments to effective altruism: The role of subjective preferences in charitable giving. Psychological science, 29(5), 834-844.

Bloom, P. (2017). Against empathy: The case for rational compassion. Random House.

Caviola, L., Althaus, D., Mogensen, A., & Goodwin, G. (2021). Population ethical intuitions. Cognition.

Crimston, C. R., Hornsey, M. J., Bain, P. G., & Bastian, B. (2018). Toward a psychology of moral expansiveness. Current Directions in Psychological Science, 27(1), 14-19.

Feinberg, M., Kovacheff, C., Teper, R., & Inbar, Y. (2019). Understanding the process of moralization: How eating meat becomes a moral issue. Journal of Personality and Social Psychology, 117(1), 50–72.

Gainsburg, I., Pauer, S., Nawal, A., Aloyo, E. T., Mourrat, J. C., & Cristia, A. (2021). How effective altruism can help psychologists maximize their impact.

Greene, J. D. (2013). Moral tribes: Emotion, reason, and the gap between us and them. Penguin.

Greenberg, S., (2001). Which intrinsic values set different demographic groups apart?

Kahane, G., Everett, J. A., Earp, B. D., Caviola, L., Faber, N. S., Crockett, M. J., & Savulescu, J. (2018). Beyond sacrificial harm: A two-dimensional model of utilitarian psychology. Psychological Review, 125(2), 131.

Lieder, F., Prentice, M., & Corwin-Renner, E. R. (accepted subject to minor revisions). An interdisciplinary synthesis of research on understanding and promoting well-doing. Social and Personality Psychology Compass.

Ord, T. (2015). Moral trade. Ethics, 126(1), 118-138.

​​Tetlock, P. E. (2003). Thinking the unthinkable: Sacred values and taboo cognitions. Trends in cognitive sciences, 7(7), 320-324.

Reynante, B. M., Wilcox, J. E., Stephenson, O. L.,Lieder, F., Thielmann, I., & Lacopo, C. (submitted). Cultivating Changemakers: A review of Metachangemaking.

Rhee, J. J., Schein, C., & Bastian, B. (2019). The what, how, and why of moralization: A review of current definitions, methods, and evidence in moralization research. Social and Personality Psychology Compass. https://doi.org/10.1111/spc3.12511.

Rozen, P. (1999) The Process of Moralization. Psychological Science, 10(3): 218-221.

Singer, P. (2011). The expanding circle: Ethics, evolution, and moral progress. Princeton University Press.

Skitka, L. J., Hanson, B. E., Morgan, G. S., & Wisneski, D. C. (2021). The psychology of moral conviction. Annual Review of Psychology, 72, 347-366.

Rationality, judgment, and cognitive biases

If an impartial actor wants to use reason and evidence to effectively improve the future, they will be more effective to the extent that they avoid cognitive biases and make rational decisions. Psychological longtermist research thus includes understanding people’s judgments and decisions, how rational they are, and how to make them more so. Some of the worst outcomes for the longterm future, e.g., nuclear war or misaligned AI, are likely driven by misjudgments and suboptimal decision making. Psychological research could therefore study the cognitive biases that lead people to make such dangerous high-stakes decisions. Relatedly, psychological research could identify biases that lead people to underestimate particularly effective approaches to proactively improving the future. Further, to help impartial altruists effectively improve the future, research could develop methods to boost rational decision-making and forecasting accuracy.

Identifying decision tendencies and biases relevant for longtermism

Note. There are potentially hundreds of tendencies that are relevant. Here, we just list a few understudied ones as examples.

Improving rational thinking

Forecasting

Relevant work:

Caviola, L., Schubert, S., & Greene, J. D. (2021). The Psychology of (In) Effective Altruism. Trends in Cognitive Sciences.

Baron, J. (2000). Thinking and deciding. Cambridge University Press.

Bazerman, M. H. (2020). Better, Not Perfect: A Realist's Guide to Maximum Sustainable Goodness. HarperCollins.

Daniel, K. (2017). Thinking, fast and slow.

Galef, J. (2021). The Scout Mindset: Why Some People See Things Clearly and Others Don't. Penguin.

Greenberg, S. (2022). Clearer Thinking Program Categorization.

Grossmann, I. (2017). Wisdom in Context. Perspectives on Psychological Science,12(2) 233–257.

Morewedge, C. K., Yoon, H., Scopelliti, I., Symborski, C. W., Korris, J. H., & Kassam, K. S. (2015). Debiasing decisions: Improved decision making with a single training intervention. Policy Insights from the Behavioral and Brain Sciences, 2(1), 129-140.

Sellier, A. L., Scopelliti, I., & Morewedge, C. K. (2019). Debiasing training improves decision making in the field. Psychological science, 30(9), 1371-1379.

Tetlock, P. E., & Gardner, D. (2016). Superforecasting: The art and science of prediction. Random House.

Yudkowsky, E. (2008). Cognitive biases potentially affecting judgment of global risks. Global catastrophic risks, 1(86), 13.

Individual differences

Not everyone is equally motivated and effective at improving the future. Research could investigate these individual differences. For example, research could study the psychological factors — values, personalities, cognitive traits — predicting whether someone is more likely to endorse the project of effectively improving the future and take action accordingly. This may also entail the development of rigorous psychometric measurement tools that helps to reliably assess these factors. Conversely, psychological research could also study the psychological factors that predict reckless decision-making in high stakes situations that could greatly harm the future.

Identifying people who tend to either improve or harm the future

Psychological predictors of effectively improving the future

Relevant work:

Adler, M. G., & Fagley, N. S. (2005). Appreciation: Individual differences in finding value and meaning as a unique predictor of subjective well-being. Journal of personality, 73(1), 79-114.

Caviola, L. Althaus, D., Schubert, S., & Lewis, J. (2022). What psychological traits predict interest in effective altruism?

Caviola, L., Morrissey, E., & Lewis, J. (2022). Most students who would agree with EA ideas haven't heard of EA yet. Effective Altruism Forum.

Lovett, B.J., Jordan, A.H., & Wiltermuth, S.S. (2012). Individual Differences in the Moralization of Everyday Life. Ethics and Behavior, 22(4), 248-257.

Meindl, P., Jayawickreme, E., Furr, R. M., & Fleeson, W. (2015). A foundation beam for studying morality from a personological point of view: Are individual differences in moral behaviors and thoughts consistent?. Journal of Research in Personality, 59, 81-92.

Moss. D. (2021). Effective Altruism Survey. Effective Altruism Forum.

Reynolds, S. J. (2006). Moral awareness and ethical predispositions: Investigating the role of individual differences in the recognition of moral issues. Journal of Applied Psychology, 91(1), 233.

Stanovich, K. E., & West, R. F. (2000). Individual differences in reasoning: Implications for the rationality debate?. Behavioral and brain sciences, 23(5), 645-665.

Stanovich, K. E., & West, R. F. (1998). Individual differences in rational thought. Journal of experimental psychology: general, 127(2), 161.

Stanovich, K. E., West, R. F., & Toplak, M. E. (2016). The rationality quotient: Toward a test of rational thinking. MIT press.

Sun, J., & Goodwin, G. P. (2020). Do people want to be more moral?. Psychological Science, 31(3), 243-257.

Developmental

One approach to improve the future is to educate and empower the young generation with helpful tools and ideas. Psychological research could contribute to this project by empirically investigating how the psychological factors conducive to effectively improving the future emerge developmentally. It could also explore strategies that could help to foster critical thinking skills and pro-social values across different age groups.

Psychological factors related to improving the future across age

Fostering critical thinking and pro-social values across age

Relevant work:

Bergman, R. (2002). Why be moral? A conceptual model from developmental psychology. Human development, 45(2), 104-124.

Klaczynski, P. A., Byrnes, J. P., & Jacobs, J. E. (2001). Introduction to the special issue: The development of decision making. Journal of Applied Developmental Psychology, 22(3), 225-236.

Kirby, J., Crimston, C. R., & Hoang, A. (2022). Compassionate Mind Training Can Increase Moral Expansiveness: A Randomised Controlled Trial.

Kohlberg, L., & Gilligan, C. (2014). Moral development. Psychology: Revisiting the Classic Studies, 164.

Marshall, J., Gollwitzer, A., Mermin-Bunnell, K., Shinomiya, M., Retelsdorf, J. & Bloom, P. (2022). How development and culture shape intuitions about prosocial obligations. Journal of Experimental Psychology: General. DOI: 10.1037/xge0001136.

Neldner, K., Crimston, C., Wilks, M., Redshaw, J., & Nielsen, M. (2018). The developmental origins of moral concern: An examination of moral boundary decision making throughout childhood. PloS one, 13(5), e0197819.

Sommer, K., Nielsen, M., Draheim, M., Redshaw, J., Vanman, E. J., & Wilks, M. (2019). Children’s perceptions of the moral worth of live agents, robots, and inanimate objects. Journal of Experimental Child Psychology, 187, 104656.

Toplak, M. E., West, R. F., & Stanovich, K. E. (2014). Rational thinking and cognitive sophistication: development, cognitive abilities, and thinking dispositions. Developmental psychology, 50(4), 1037.

Wilks, M., Caviola, L., Kahane, G., & Bloom, P. (2021). Children prioritize humans over animals less than adults do. Psychological Science, 32(1), 27-38.

Cross-cultural

Effectively improving the future is a global endeavor that requires an understanding of human psychology across many different cultures. Psychological research could contribute to that project by empirically investigating cross-cultural differences in psychological tendencies that are conducive to or hinder improving the future.

Cross-cultural differences in attitudes about and decision-making related to effectively improving the future

Relevant work:

Henrich, J. (2020). The WEIRDest people in the world: How the West became psychologically peculiar and particularly prosperous. Penguin UK.

MacAskill, W. (2022). What We Owe the Future. Hachette UK.

Olivola, C. Y., Kim, Y., Merzel, A., Kareev, Y., Avrahami, J., & Ritov, I. (2019). Cooperation and coordination across cultures and contexts: Individual, sociocultural, and contextual factors jointly influence decision making in the volunteer's dilemma game. Journal of Behavioral Decision Making, 33, 93-118.

Rhoads, S., Gunter, D., Ryan, R. M., & Marsh, A. A. (2020). Global variation in subjective well-being predicts seven forms of altruism. Psychological science. 32(8), 1247-1261.

Romano, R., Sutter, M., Liu, J. H., Yamagishi, T., & Balliet, D. (2021)  National parochialism is ubiquitous across 42 nations around the world. Nature Communications. https://doi.org/10.1038/s41467-021-24787-1.

Van Doesum, N. J.,  et al. (2021) Social mindfulness and prosociality vary across the globe. Proceedings of the National Academy of Sciences, 118(35).

Historical psychology, evolutionary psychology, and anthropology

To improve the future it can be helpful to look into the past. Social sciences with a historical or evolutionary focus could contribute to that project. For example, such research could investigate how moral values relevant to improving the future have changed in the past, and based on this, estimate how they could plausibly change in the future.

Relevant work:

Branwen, G. (2019). The narrowing circle. https://www.gwern.net/The-Narrowing-Circle.

Calman, K. C. (2004). Evolutionary ethics: can values change. Journal of Medical Ethics, 30(4), 366-370.

Haslam, H., McGrath, M.J., & Wheeler, M.A. (2019). Changing morals: we're more compassionate than 100 years ago, but more judgmental too. The University of Melbourne.

Pinker, S. (2012). The better angels of our nature: Why violence has declined. Penguin Books.

Roser, M. (2022). The world is awful. The world is much better. The world can be much better. Our World in Data.

Schulz, J. F., Bahrami-Rad, D., Beauchamp, J. P., & Henrich, J. (2019). The Church, intensive kinship, and global psychological variation. Science, 366(6466), eaau5141.

Tomasik, B. (2013). Differential intellectual progress as a positive-sum project. Center on Long-Term Risk.

Wright, R. (2010). The moral animal: Why we are, the way we are: The new science of evolutionary psychology. Vintage.

Policy and institutional

A key route to improving the future is through policy and institutional changes. Psychological research could contribute to this project by investigating the obstacles that prevent such helpful policy changes. Ultimately, it could also help to develop and test practical strategies that could be implemented to improve institutional decision making.

Psychological obstacles to effectively improving the future in policy contexts

Improving thinking and decision-making in institutional settings

Relevant work:

Caviola, L. & Greene, J.D., Boosting the impact of human altruism. Manuscript in preparation.

Hauser, O. P., Rand, D. G., Peysakhovich, A., & Nowak, M. A. (2014). Cooperating with the future. Nature, 511(7508), 220-223.

John, T., & MacAskill, W. (2021). Longtermist institutional reform. The Long View (forthcoming), Legal Priorities Project Working Paper Series, (4).

Schoenmakers, K., Greene, D., Stutterheim, S. E., Lin, H., & Palmer, M. (2022). The Security Mindset: Characteristics, Development, and Consequences.

Whittlestone, J. (2017). Improving institutional decision making. 80,000 Hours report.

Winter, C., Schuett, J., Martínez, E., Van Arsdale, S., Araújo, R., Hollman, N., ... & Rotola, G. (2021). Legal Priorities Research: A Research Agenda.

Future wellbeing

An impartial altruist attempting to effectively improve the future is in significant part ultimately interested in increasing the future’s aggregate wellbeing level. Research that helps to estimate the wellbeing levels of future individuals could therefore be action-guiding. In particular, a crucial question for such altruists is whether the future in expectation will contain much more happiness or much more suffering. This may determine whether an altruist is more likely to prioritize improving the quality of the future (i.e. reducing suffering and improving happiness) or reducing the chances of human extinction.

Estimating wellbeing in the future

Wellbeing in non-human beings

Relevant work:

Diener, E., & Diener, C. (1996). Most people are happy. Psychological science, 7(3), 181-185.

Killingsworth, M.A., Stewart, L., & Greene, J.D. (2021). Is life “worth living”? A measure of absolute happiness. Manuscript in preparation.

Taylor, S. E., & Brown, J. D. (1988). Illusion and well-being: a social psychological perspective on mental health. Psychological bulletin, 103(2), 193.

Haybron, D. M. (2007). Do we know how happy we are? On some limits of affective introspection and recall. Nous, 41(3), 394-428.

Waterman, A. S. (2007). On the importance of distinguishing hedonia and eudaimonia when contemplating the hedonic treadmill. American Psychologist, 62(6), 612–613.

Schwitzgebel, E. (2011). Perplexities of consciousness. MIT press.

Norris, C. J., Gollan, J., Berntson, G. G., & Cacioppo, J. T. (2010). The current status of research on the structure of evaluative space. Biological psychology, 84(3), 422-436.

Martínez, E., & Winter, C. (2021). Protecting Sentient Artificial Intelligence: A Survey of Lay Intuitions on Standing, Personhood, and General Legal Protection. Frontiers in Robotics and AI, 8.

Shulman, C. (2012). Are pain and pleasure equally energy efficient? Reflective Disequilibrium.

O’Brien & Kassirer (2018). People are slow to adapt to the warm glow of giving. Psychological science, 30(2), 193–204.

Happier Lives Institute. Research Agenda. https://www.happierlivesinstitute.org/research/research-agenda/ 

DeYoung, C. G., & Tiberius, V. (2021). Value fulfillment from a cybernetic perspective: A new psychological theory of well-being. Personality and Social Psychology Review, 10888683221083777.

Williams, L. A. (2021). From human wellbeing to animal welfare. Neuroscience & Biobehavioral Reviews, 131, 941-952.

Cause-specific issues

Here we focus on global problems that seem particularly pressing from the point of view of effectively improving the future. In general, it’s useful to study the views and psychological tendencies people show that relate to these specific issues. This includes, for example, the study of psychological obstacles preventing people from taking these issues seriously. Note that the cause areas listed below are chosen because they seem particularly neglected from the perspective of effectively improving the future. There are many other important cause areas (e.g. prejudice, climate change) that aren’t listed because they seem relatively less neglected in academic psychology.

Global catastrophic risk (in general)

Relevant work:

Bostrom, N. (2013). Existential risk prevention as global priority. Global Policy, 4(1), 15-31.

Bostrom, N., & Cirkovic, M. M. (Eds.). (2011). Global catastrophic risks. Oxford University Press.

MacAskill, W. (2022). What We Owe the Future. Hachette UK.

Schubert, S., Caviola, L., & Faber, N. S. (2019). The psychology of existential risk: Moral judgments about human extinction. Scientific reports, 9(1), 1-8.

Yudkowsky, E. (2008). Cognitive biases potentially affecting judgment of global risks. Global catastrophic risks, 1(86), 13.

Risks from artificial intelligence (AI)

Relevant work:

Bensinger, R. (2021). "Existential risk from AI" survey results, AI Alignment Forum. Link.

Bostrom, N., & Shulman, C. Propositions Concerning Digital Minds and Society.

Clarke, S., Carlier, A., & Schuett, J. (2021). Survey on AI existential risk scenarios, EA Forum. Link.

Grace, K., Salvatier, J., Dafoe, A., Zhang, B., & Evans, O. (2018). When will AI exceed human performance? Evidence from AI experts. Journal of Artificial Intelligence Research, 62, 729-754.

Irving, G., & Askell, A. (2019). AI safety needs social scientists. Distill, 4(2), e14.

O'Shaughnessy, M., Schiff, D., Varshney, L. R., Rozell, C., & Davenport, M. (2021). What governs attitudes toward artificial intelligence adoption and governance?. Preprint.

Pauketat, J. V., & Anthis, J. R. (2022). Predicting the moral consideration of artificial intelligences. Computers in Human Behavior, 136, 107372.

Pauketat, J. V., Ladak, A., & Anthis, J. R. (2022). Artificial Intelligence, Morality, and Sentience (AIMS) Survey: 2021.

Zach Stein-Perlman, Benjamin Weinstein-Raun, Katja Grace, “2022 Expert Survey on Progress in AI.” AI Impacts, 3 Aug. 2022. https://aiimpacts.org/2022-expert-survey-on-progress-in-ai/.

Zhang, B., Anderljung, M., Kahn, L., Dreksler, N., Horowitz, M. C., & Dafoe, A. (2021). Ethics and governance of artificial intelligence: Evidence from a survey of machine learning researchers. Journal of Artificial Intelligence Research, 71, 591-666.

Zhang, B., & Dafoe, A. (2020, February). US public opinion on the governance of artificial intelligence. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society (pp. 187-193).

Zhang, B., & Dafoe, A. (2019). Artificial intelligence: American attitudes and trends. Available at SSRN 3312874.

Zhang, B., Dreksler, N., Anderljung, M., Kahn, L., Giattino, C., Dafoe, A., & Horowitz, M. C. (2022). Forecasting AI Progress: Evidence from a Survey of Machine Learning Researchers. arXiv preprint arXiv:2206.04132.

Pandemics and biosecurity

Relevant work:

Carus, W. S. (2017). A century of biological-weapons programs (1915–2015): reviewing the evidence. The Nonproliferation Review, 24(1-2), 129-153.

Gronvall, G. K. (2016). Synthetic biology: Safety, security, and promise. Health Security Press.

Inglesby, T. V., & Relman, D. A. (2016). How likely is it that biological agents will be used deliberately to cause widespread harm? EMBO reports, 17(2), 127-130.

Kilbourne, E.D. (2011). Plagues and pandemics: past, present, and future. In N. Bostrom & M. M. Cirkovic, Global catastrophic risks. Oxford University Press.

Koblentz, G. (2003). Pathogens as weapons: the international security implications of biological warfare. International security, 84-122.

Koehler, A., & Hilton, B. (2020). Preventing catastrophic pandemics. 80,000 Hours.    

Millett, P., & Snyder-Beattie, A. (2017). Human agency and global catastrophic Biorisks. Health security, 15(4), 335-336.

Monrad, J. T. (2020). Ethical considerations for epidemic vaccine trials. Journal of medical ethics, 46(7), 1-5.

Nouri, A. & C.F. Chyba. (2011). Biotechnology and biosecurity. In N. Bostrom & M. M. Cirkovic, Global catastrophic risks. Oxford University Press.

Racicot, M., Venne, D., Durivage, A., & Vaillancourt, J. P. (2012). Evaluation of the relationship between personality traits, experience, education and biosecurity compliance on poultry farms in Québec, Canada. Preventive Veterinary Medicine, 103(2-3), 201-207.

Salvatore, S. (2017). Psychological Evaluations for the US Army Biological Personnel Reliability Program. Journal of Biosecurity, Biosafety, and Biodefense Law, 8(1), 3-17.

Vanlandingham, D. L., & Higgs, S. (2021). Viruses and Their Potential for Bioterrorism.

Global war and nuclear threat

Relevant work:

Ellsberg, D. (2017). The doomsday machine: Confessions of a nuclear war planner. Bloomsbury Publishing USA.

Graham, A. (2017). Destined for War Can America and China Escape Thucydides's Trap? HarperCollins.

Hoffman, D. (2009). The dead hand: the untold story of the cold war arms race and its dangerous legacy. Anchor.

Hogg, M. A., & Blaylock, D. L. (Eds.). (2011). Extremism and the Psychology of Uncertainty (Vol. 3). John Wiley & Sons.

Loza, W. (2007). The psychology of extremism and terrorism: A Middle-Eastern perspective. Aggression and Violent Behavior, 12(2), 141-155.

Schlosser, E. (2013). Command and control: Nuclear weapons, the Damascus accident, and the illusion of safety. Penguin.

Suedfeld, P., & Tetlock, P. (1977). Integrative complexity of communications in international crises. Journal of conflict resolution, 21(1), 169-184.

Threats to liberal democracies and descent into long-term authoritarianism lock-in

Relevant work:

Caplan, B. (2011). The totalitarian threat. In N. Bostrom & M. M. Cirkovic, Global catastrophic risks. Oxford University Press.

Dikötter, F. (2022). How to Be a Dictator: The Cult of Personality in the Twentieth Century. Bloomsbury Publishing.

Johnson, S. A. (2019). Understanding the violent personality: Antisocial personality disorder, psychopathy, & sociopathy explored. Forensic Research & Criminology International Journal, 7(2), 76-88.

Lazer, D. M., Baum, M. A., Benkler, Y., Berinsky, A. J., Greenhill, K. M., Menczer, F., ... & Zittrain, J. L. (2018). The science of fake news. Science, 359(6380), 1094-1096.

McCoy, J., & Press, B. (2022). What Happens When Democracies Become Perniciously Polarized? Carnegie Endowment for International Peace.

Smith, A., & de Mesquita, B. B. (2011). The dictator's handbook: Why bad behavior is almost always good politics. PublicAffairs.

Contributors

Main authors:

Lucius Caviola, Joshua Lewis, Matti Wilks, Abigail Novick Hoskin, Stefan Schubert

For helpful comments we thank Adam Bales, Adam Bear, Andreas Mogensen, David Althaus, David Thorstead, Erin Morrissey, Falk Lieder, Geoffrey Goodwin, Hayden Wilkinson, Inga Grossmann, Izzy Gainsburg, Jessie Sun, Matt Coleman, Maximilian Maier, Noemi Dreksler, Samantha Kassierer, Sven Herrmann, and Will Fleeson.