1 of 20

Neuroinformatics�Nonsense correlations

Kenneth D Harris, UCL

2 of 20

3 of 20

Nonsense correlations

  • Two variables both depend smoothly on time

​

  • They are correlated with each other, but only because of this time dependence

​

  • How can we correct for common time dependence?
    • And what if the variables are vectors, spike trains, decisions, etc?

​

  • NOT about proving causality

4 of 20

Example

  • 10 predictor variables
    • Each an independent random walk
    • Could be firing rates

​

  • 1 target variable
    • Binary, switches in blocks
    • Could be sensory stimulus

​

  • Predict by linear regression
    • Looks good!
    • But they were completely independent

5 of 20

Ordinary cross-validation doesn’t solve the problem

Training set prediction

Test set prediction

Training set mean prediction on test set

Random split: catastrophic failure

Block split: still failure

6 of 20

So what might work?

  • Need to revisit statistical theory
    • Random processes
    • Permutation tests
    • Stationarity

​

  • All in the frequentist framework

https://www.biorxiv.org/content/10.1101/2020.11.29.402719v2

7 of 20

Random processes

  • A random process defines a probability distribution over the space of possible histories of all the measured variables

Sample space =

all possible histories

Probability density 3.0343534976x10-12

8 of 20

Example

  •  

9 of 20

Null hypothesis

  •  

10 of 20

Applying to neural data: simplest case

  •  

11 of 20

Pseudosession method

  •  

12 of 20

Original

Different blocks

No encoding of stimulus

Encoding of stimulus

13 of 20

Session permutation method

  •  

14 of 20

Example

  • 5 Sessions
  • Data generated as before

No encoding of block

Encoding of block

15 of 20

Session permutation method

  •  

16 of 20

Linear shift method

  •  

17 of 20

Stationarity is a property of the ensemble

Not stationary

Stationary

  • Anything you record or measure will be non-stationary – because behavioral state changes over the experiment
  • You can synthesize something stationary by making a very long timeseries, and starting at a random point

18 of 20

Linear shift method

  •  

https://arxiv.org/abs/2012.06862

19 of 20

Example on binary time series

Reject null conservatively at p=0.05

20 of 20

Summary

  • Traditional methods don’t work for correlated timeseries

​

  • Use pseudosession method if you know the probability of one timeseries

​

  • Use session permutation if you have multiple experiments

​

  • Use linear shift if one series is approximately stationary

​

  • Design a randomized experiment!!