1 of 34

Causality is very much an open science topic

A meta-tool for navigating causal assessment

SIPS 2025 Budapest

June 25 09:00–10:30, second floor 213

Michael Höfler�Technical University Dresden, Germany

michael.hoefler@tu-dresden.de

2 of 34

Short introduction round

your name
where you work
your approach to causality

3 of 34

Explicit causality is avoided outside

experimental research

Theories are based on causal relations
Theories are tested with and fed by associations
Assess the potential of interventions

‘Correlation is not causation’
‘We have only investigated associations’

‘That one can not randomize a factor does not mean that one is not interested in its effect!’

‘Conflating the means and the ends.’

1. Background

Intransparency, even in the causal goal, is disastrous for scientific communication.

4 of 34

Explanations are aligned with cognitive dissonance theory

Generative cognition: Causality is taboo in observational studies
Dissonant cognition: Causal assessment is required, even if no experiment is possible

Hypothesis 1: Avoiding causality is rewarded because addressing causality has large costs

Hypothesis 2: Without a profound understanding of causality, the acceptance of inevitability coincides with inappropriate stances

Hypothesis 3: The modes of dissonance reduction are diverse

Hypothesis 4: Short reflection on potential benefits does not help against avoidance

Hypothesis 5: Social aspects maintain the avoidance

Observed in the common blending of causal and associational wording:

1. Background

5 of 34

Addressing causality�… begins with qualitative considerations

Qualitative thoughts might reveal that no study/causal analysis is required

A causal effect might be uncontroversial because its mechanism is very well understood (e.g. pressing the light switch causes the light to come on)

Or a causal effect might be very implausible because of the clear absence of such mechanism (e.g. astrology).

In any case: one must be very transparent on qualitative assumptions and their substantive foundation to allow the scientific community to assess them.

1. Background

6 of 34

If new study/analysis is required, one must begin with a qualitative generative model on processes beyond the data (i.e., on bias)

You can not have no generative model!

‘Let the data speak for themselves‘ assumes:

Factor and outcome have no common causes (none beyond those adjusted for �= the specific issue of causal infererence)
No measurement error (with impact on results)
No selection effects (with impact on results)
No non-compliance (with impact on results)
…

Nobody would buy these assumptions if they were explicit!

1. Background

7 of 34

Convenient method for generative models: �Directed Acyclic Graphs (DAGs)

e.g. on common causes, the specific issue of causal inference in observational studies

Expliciting generative assumptions makes researchers think about them: e.g. educational attainment and income have no common causes beyond intelligence (reveals that this is just an educational example!)

DAGs inform study design and analysis, e.g. what common causes must be considered. Based on a DAG a quantitative analysis is chosen which requires more assumptions.

(DAGs can express assumptions on other sources of bias: measurement, selection, missing data, non-compliance and inform analyses that consider these issues)

1. Background

8 of 34

Underrated: testing Causal predictions

A causal hypothesis makes a different, often stronger prediction than an associational hypothesis
Falsificiability: The prediction could turn out wrong
Can be done with existing data
For example, a causal hypothesis predicts an association on Cohen’s d scale > Δ, whereas an association predicts just d > 0.

An entire DAG (or SEM) predicts various marginal and conditional associations that may turn out wrong.

1. Background

through secondary data analysis or collecting new data

9 of 34

Alternative Explanations

… must be given a fair chance to hold (i.e., be falsifiable)

If dismissed, the underlying arguments must be very clear and transparent

… to an analysis supporting a causal effect �(e.g. unconsidered common causes, poor measurement …)

1. Background

10 of 34

Causal toolbox, very much under-utilised

in Psychology�

1. Background

... and many visual tools that facilitate design decisions and analysis, see slides below

11 of 34

What can be done to make addressing causality more appealing?

A tool to navigate through 1. the initial qualitative assessments and 2. existing tools?
The tool may inform qualitative decisions, reveal their foundations and return these

And identify and return which existing tools may be useful

E.g. ‘is the hypothesis causal or associational?’ If one wants to inform theory or assess the potential of an intervention → causal.

2. A meta tool for navigating causal assessment

12 of 34

Existing tools hardly

cover that

2. A meta tool for navigating causal assessment

13 of 34

Is the hypothesis meant to be…

Causal?

Associational?

Alternative, noncausal explanations�E.g. an association has been found that might be due to common causes or measurement error …

Reasons to believe an alternative explanation

Reasons to believe causal explanation (e.g. known mechanism)

But a new meta tool may guide through the qualitative steps

Creates transparency on arguments and their foundations

Navigate through options and tools to collect new evidence for or against

Collect ideas on testing an explanation with secondary data analysis or new studies

2. A meta tool for navigating causal assessment

14 of 34

Help planning a causal observational study and its analysis, identify helpful tools

Generative (nonparametric) models
Based on that: analytic (mostly) parametric models

Navigate through existing tools

Can a prediction be made that can be tested with perhaps existing observational data?

Go through causal methods and their assumptions to assess if they might help: instrumental variables, Mendelian randomisation, Granger causality …�

Can they be used in perhaps existing observational data?

Methodological advice usually stars only here

Navigate through options and tools to collect new evidence for or against a causal hypothesis

… or just there

2. A meta tool for navigating causal assessment

15 of 34

A concise page with collected:

1) Hypothesis, meant causal or not causal?

2) Arguments for and against an effect and alternative explanations

3) Options for secondary data analyses and associated tools

4) Options for new study and associated tools

Returned sheet

2. A meta tool for navigating causal assessment

16 of 34

https://tally.so/create

Is the hypothesis meant to be …

causal or associational?

Might return:

3 out of 3 answers suggest that your hypothesis is causal.

2. A meta tool for navigating causal assessment

Ideas on the sections

17 of 34

Reasons to believe causal explanation

Is there a known mechanism that could explain the effect if it exists?

Is there a known mechanisms that speaks against the effect? (e.g. gravitation law speaks against astrology)

2. A meta tool for navigating causal assessment

18 of 34

New study warranted? What reasons to believe could this add?

Is a study feasible that adds substantial evidence for or against the effect?

Might reveal that a study is premature (e.g. knowledge of common causes is so limited that one cannot specify which of such variables should be collected → instead recommend to engage in theory building)

‘Study’ might just mean analysing existing data to test predictions that a causal hypothesis makes (on associations)

19 of 34

Causal design and analysis tools, evaluate if potentially helpful

Which common causes must be collected?

If this can not be decided but many such variables are known: very large sample size that allows adjustment for many

Make use of the ‘target trials’ conception:

2. A meta tool for navigating causal assessment

20 of 34

(Software used so far:)

https://tally.so/create

Tally forms

Free
convenient to use interactively
full conditional answer logic
Can create pages with any kind of digital content
only form creator has access
data stored by DigitalOcean Holdings, Inc in Frankfurt/Germany
Reports table with results
Results can be exported as .pdf, .csv or google sheet

2. A meta tool for navigating causal assessment

21 of 34

Collection of existing tools�(free and non-expert tools so far)

3. Existing tools

DAGitty Shiny App and R package

22 of 34

3. Existing tools

Theoraizer �use AI to create a model (which can be fed into DAGitty)

Causal Loop Diagram (CLD) method

Model building may be fed with information on associations, because papers blend causal and associational language and do not use causal methods?

Child maltreatment,

Internalizing problems in adulthood,

Bonding characteristics,

Parental Psychopathology,

Parenting characteristics

Internalizing problems in Adulthood

23 of 34

ViSe

Graphical tool, shiny app and R package to determine a confirmational threshold that accounts for unconsidered common causes (confounders).

Associational research tests if association > 0
Instead test if association > c
c = α_{1 *}α₂= effect of common disposition on factor * effect of common disposition on outcome
Can be expressed on and converted to various scales

3. Existing tools

24 of 34

Z → X

Z → Y

α₁ :

α₂ :

3. Existing tools

Effect supported

Left bound of confidence interval for Cohen‘s d = 0.09 produces this:

X = factor�Y = outcome

Z = common disposition

25 of 34

3. Existing tools

Currently just based on this simple generative model:

May be extended to allow choosing between different generative models that also address selection, measurement … (perhaps making use of DAGitty)

Currently focusses on results on Cohen‘s d scale (and conversion of effect sizes), might be extended to directly address results on other scales

(and beyond that qualitative model quantitative �assumptions, “variable omission model“)

26 of 34

thinkCausal

Nice teaching material

Guides you well through analysis

3. Existing tools

27 of 34

DoWhy

Python library that combines generating models, data analysis and robustness checks

Nicely implemented interplay between model building and model checking with data, provides verbal assessments. Uses simple graphs, seems only useful for researchers with a keen methodological interest.

3. Existing tools

28 of 34

TETRAD

R and Python package, has a graphical user interface that requires Java JDK

3. Existing tools

29 of 34

(CausalImpact)

R package by Google that does “causal inference using Bayesian structural time-series models” and visualizes the results

3. Existing tools

30 of 34

Tools for sensitivity analysis that require only summary results (e.g. confidence intervals)

Just considers confounding, outcome can be binary or dimensional

Software review (2021 or older)

ViSe can be used for sensitivity analysis as well

3. Existing tools

31 of 34

(TippingSens)

Similar to ViSe for binary variables, but needs data input.

3. Existing tools

32 of 34

(CausalOptim)

You can add a DAG to be evaluated, bounds are computed. Can specify if variables are dimensional or binary. Requires understanding of potential outcomes. Unstable website.

3. Existing tools

33 of 34

3. Existing tools

(CausalVis)

Python library, not directly usable interactively

34 of 34

3. Existing tools

(CausalNex)

Python library: machine learning to identify causal relations with big data; not directly usable interactively