ABCDEFGHIJKLMNOPQRSTUVWXYZ
1
CANDIDATE INTERVENTIONWhole-institution governance, leadership, and cultureGroup dynamics in decision-makingHow decisions (at all levels) are madeUsing evidence and data to improve decisionsDecision-making support within government and international institutionsMonitoring and improving ongoing performanceInteractions between institutions and across institutional ecosystemsImproving knowledge about the futureIndividual-level judgment and decision-makingDetermining goals and strategies to achieve themCOMMENTSCheck for "MAIN"
2
Rationality training for individuals MIDYESMAIN1
3
Evidence-use or research training for individuals (evidence literacy programs)MIDMAINMID1
4
Calibration training for individualsYESMIDMAINMID1
5
(Somehow) using prediction markets in institutionsYESYESMAIN1
6
Using the Decision Quality scale (DQ Scale) as a good process-focused metricMIDMAIN1
7
Improving choice architecture (broad; e.g., purposefully choosing default options)YESMAINYES1
8
Using apps and software as Decision Support ToolsMAIN1
9
Other Decision Support Systems (DSS) - e.g., Rand’s for the Puerto Rico Govt. or MELC in New Zealand Govt.MAIN1
10
New institution: evidence clearinghousesYESYESMAIN1
11
Establishing evidence hotlinesMAINYES1
12
Creating/funding/using rapid research servicesMAINYES1
13
Employ strategic planning techniques (SWOT analysis, Porter's five forces, etc.)MIDYESMAIN1
14
Restructuring of internal voting systems (general)MAIN1
15
Altering incentives for joining the organization (general)MAIN1
16
[DELETED]0
17
Influence organizational culture through storytellingMAINMID1
18
Change organization’s communication with stakeholders (frequency, format, content, increase alignment)MAINMIDYES1
19
Increasing mechanism design literacy (training)MAINYES1
20
Including “decision observers” into group deliberation (debiasing)MAINMIDMID1
21
Using SATs (divide into 12 or more: Key Assumptions Check, Quality of Information Check, Indicators of Signpost/Change, *Analysis of Competing Hypotheses, *Devil’s Advocacy, Team A/Team B, High-Impact Low-Probability Analysis, “What-If?” Analysis, Brainstorming, Outside-In Thinking, Red Team Analysis, Alternative Futures Analysis) [limited evidence]YESMIDMIDYESMAINMIDEither break down into individual SATs or find a better category1
22
Improve Long-Term institutional fit (general: next 5 interventions are the breakdown)MAINYESYESYESYESMIDYESYES1
23
Fostering moral reflectionYESMAIN1
24
Training systems thinking (identification of system properties, better mental model generation)MAINYES1
25
Improving the science-policy interface (using interface actors, co-research, evidence hotlines)YESYESMAINMID1
26
Training decision-making under uncertainty (MCDA, dynamic adaptive planning, engineering options analysis, forecasting, heuristics training)MIDYESMIDYESMAINJoin with DMDU Methods? it's in a different category. Otherwise split up1
27
Facilitate group deliberation (constructive criticism, perspective-sharing)MIDMAINMID1
28
Delphi method for aggregating judgment (esp. The IDEA protocol)MAINMIDMIDYES1
29
Foresight techniques: gaming and roleplaying, scenario construction, horizon scanning, red-teaming, reference class forecasting, Delphi method, and other tools shared with forecasting (prediction markets, superforecasting)MIDYESMIDMIDMIDMAINYESMID1
30
Forecasting techniques: prediction markets, using superforecasters, using statistical modelsMIDYESYESMIDMAINMID1
31
Amending constitutions/bylaws/statutes (to be future-oriented, evidence-oriented, etc.)MAINYESMIDMIDMIDMID1
32
Subdividing groups so that they don’t surpass the Dunbar number (speculative)MAIN1
33
UUC, UKUK, SCES Analysis (general: see 2 below)MIDYESMAIN1
34
Analysis of Unanticipated Unintended Consequence (UUC) and Unknown Unknowns (UKUK)0
35
[DELETED]0
36
Singular Chains of Events Scenarios (SCES) development and analysis0
37
Creating a research division within the organizationYESYESMAIN1
38
Social incentives for evidence use, e.g., rewarding people for it, reinforcing organizational identity around the use of evidence, etc.MAINMIDYESMID1
39
[DELETED]0
40
Requiring explicit evidence for policy recommendations/general proposalsYESMAINYES1
41
Signing up employees to evidence newsletters, reminders of past findings, create “journal clubs”MAINYES1
42
Increasing policymaker-researcher collaboration, co-defining research questions [limited evidence]MIDMIDYESMAINYES1
43
Creating research specifically for policymaker use [limited evidence]YESMAINMID1
44
Developing and using consensus-building discussion platforms (like G0v with Taiwan govt., by highlighting common ground, amplifying nuanced voices, and showing relative ideological positioning)MIDMAINYESMID1
45
Survey experts and use performance-based aggregations of their judgments [Group with Delphi]YESMIDMIDMIDMAIN1Group with Delphi?
46
Monitor forecast accuracy (have a forecast and outcome log) [Group with Institutionalizing predictions...]0Group with "institutionalizing the recording of...?"
47
Improve communication of uncertainty: remind forecasters and decision-makers of numerical interpretations of verbal probabilities, adopt numerical probabilities, or normalized verbal scale.YESMIDMAINYES1
48
Coherentize and aggregate group judgmentsMAINYES1
49
Use post-analytic techniques like extremization of probabilities or increasing time-to-completion estimates (because of the planning fallacy)MIDYESMIDMAIN1
50
Employing simple linear models (e.g., for college admissions)YESMAINMIDYESMID1
51
Core-values change (e.g., introducing evidence-informed policy as company to “democracy” and “the rule of law”)MAINMIDYES1
52
Decreasing partisanship and polarization (general)MAINYESYESYES1
53
Mandating public calls for evidence submissions before discussing policies (policymaking)YESMAINYES1
54
Creating identity-cues: reminding people that part of their identity is being evidence-informedMAINYESMID1
55
Using the EAST Framework for behavioral interventionsMIDMIDMAIN1
56
Capitalizing on “evidence leaders” to increase evidence uptakeMAINMIDYES1
57
Applying the Evidence Transparency Framework in policymakingYESMAINYESMID1
58
Establish step-by-step or checklist protocols for repeated regular processesMAINMIDMID1
59
Make a research database your employees’ default landing pageMAINYES1
60
Providing internal financial incentives to modify behaviorMAINMIDYES1
61
Incentivizing attendance of or hosting knowledge exchange events (seminars, conferences, workshops)MAINYES1
62
Increasing diversity in institutionsMAINMID1
63
Creating consensus-based staggered planning for long-term goalsYESMIDMAIN1
64
Engaging in premortem and backcasting during group deliberationsYESMIDMAINMID1
65
Creating internal prediction markets / forecasting teamsYESYESYESMIDMAINMID1
66
Providing “Superforecasting training” to decision-makersYESMAINYES1
67
institutionalizing the recording of quantitative judgemental predictions of advisors/analysts and decision-makersYESMAINYES1
68
Increasing the use of testable theories of changeMIDMAIN1
69
Facilitating anonymous 360-degree feedback among employeesYESMAIN1
70
Requiring an information-sharing/pooling phase prior to group deliberation and proposal of solutionsMIDYESMAINMID1
71
Rebalancing organizational decision-making structure (hierarchy/poliarchy)MAINYESYES1
72
Using CDD Causal Decision DiagramsYESMAINMIDMID1
73
Using DMDU Methods: Robust Decision Making (RDM), Dynamic Adaptive Planning (DAP), Dynamic Adaptive Policy Pathways (DAPP), Info-Gap Theory (IG) and Engineering Options Analysis (EOA).MIDMAINMIDMIDMIDMID1
74
75
COUNT: "MAIN"167586341034
76
COUNT: "YES"11115191013783
77
COUNT: "MID"9901391043107
78
79
Count: Main+Yes+Mid36271040251411202114
80
81
PREVALENCE SCORE (weighted count: 3*main + 2*yes + mid) (Automatic color scale)79522575472122473525
82
83
84
SUB-FIELDS (respective for each column header)Whole-institution governance, leadership, and cultureGroup dynamics in decision-makingHow decisions (at all levels) are madeUsing evidence and data to improve decisionsDecision-making support within government and international institutionsMonitoring and improving ongoing performanceInteractions between institutions and across institutional ecosystemsImproving knowledge about the futureIndividual-level judgment and decision-makingDetermining goals and strategies to achieve them
85
Institutional design / governanceImproving group deliberationDecision analysis / researchEvidence-based or -informed decision-makingGovernment performanceRisk management / analysisMechanism designScenario planningDebiasingStrategic planning
86
Incentive designImproving small-scale votingDecision engineering / intelligenceBehavioural insights / scienceMore representative large-scale voting methodsKnowledge managementSystems design / systems thinkingStrategic foresightApplied rationalityDesign thinking / user-centered design
87
Organizational behaviorGame theoryDecision scienceEvidence synthesis and translationPolicy analysisManagement scienceCollective intelligence design / crowdsourcingForecastingProductivityGlobal prioritization research
88
Organizatinal cultureConflict resolution/mediationImpact evaluationPublic administrationProject management and allocation of resourcesOrganizational landscape analysisHorizon scanningEthics / axiology
89
LeadershipProcedural justiceStrategic learningAnti-corruption activities (e.g., whistle-blowing)Capacity buildingComplexity scienceNowcastingProgress studies
90
Change managementProbability managementPolitical scienceInnovationBuilding new organizationsPrediction markets
91
Innovations in company legal structuresImplementation scienceReducing risks from malevolent actor / influencing benevolence, intelligence, powerInnovations in contractingMerging organizations
92
Systems managementData scienceIntelligence operations and analysisInnovations in financial instruments
93
Business architectureMeta-science / meta-researchOperations management
94
Science communication
95
Fact-checking / combating misinformation
96
Epistemology
97
98
99
100