The neural code�Tuning functions

Kenneth D Harris, UCL

Continuous dependence of rate on stimulus

• Many stimuli are continua
• Frequency or amplitude of a sound
• Contrast or orientation of a visual grating
• Location of the animal in the environment
• Head direction relative to the environment
• Concentration of odorants in a mixture
• Intensity of every pixel in a visual image

​

• Neural firing rates should depend continuously on the stimulus
• Small change along stimulus continuum => small change in rate

​

• Previously we talked about stimuli as discrete
• E.g. different images

​

• How do we characterize how continuous stimuli are encoded?

Orientation tuning in visual cortex

Rat hippocampus

Visual processing of natural scenes

Stringer et al Nature 2019

Rate coding equation

Types of tuning function

Types of tuning function

Parametric models

• Parametric models of experimental data have been very powerful in scientific history

​

• Because they can suggest and constrain mechanistic or functional theories
• Kepler’s law of elliptic planetary motion => Newton’s law of universal gravitation
• Ideal gas law (PV=nRT) => kinetic theory of gasses
• Rydberg formula for hydrogen spectrum => Schrodinger’s equation
• Hodgkin-Huxley equation => mechanism of action potential
• Normalization equation => ???

​

• Parametric models usually don’t fit tuning functions perfectly, but still very valuable

​

• Factors to balance when choosing a parametric formula:
• Accuracy of fit
• Simplicity and interpretability (the fewer parameters the better)
• Ease of fitting parameters to data

Gaussian tuning function

Model

von Mises tuning function

Naka-Rushton tuning function

Form of the tuning function can give biological insight

Wrapped Gaussian

“Peaky” modification

Peak sharpness depends on exponent

Peaky tuning curves

• Give rise to high-dimensional population codes (see later)

​

• Suggest new network mechanisms for producing them

How to fit a parametric model

Which loss function?

Finding the parameters for each cell: numerical optimization

Example: orientation tuning curves

• Data from Allen Institute
• Neuropixels recordings, V1 passive mice
• Oriented grating stimuli displayed on a screen for 250 ms
• Different orientation, spatial frequency, phase
• We consider one spatial frequency and phase

​

• Fitting a cosine tuning curve with squared error: quadratic
• Fitting a von Mises tuning curve with Poisson log likelihood: convex
• Fitting a von Mises tuning curve with squared error: non-convex

Siegle et al Nature 2021. https://www.nature.com/articles/s41586-020-03171-x

​

Quadratic optimization

Orientation tuning: cosine model, squared error loss (cell 42)

Convex optimization

von Mises tuning function

von Mises fit for cell 42

Allen Cell 46

Optimization of generic objective functions

• If your function is not even convex, what do you do?

​

• Global search can only be done in low dimensions
• “Brute force grid search”
• Simulated annealing; genetic algorithms; others
• Scipy.optimize has implementations
• But none work reliably in high dimensions

​

• Usually settle for local optimization.
• This is how modern AI systems are trained
• Can miss the “big peaks”
• For AI, this might not matter
• For scientific conclusions, it might

​

Gradient descent algorithm

Types of tuning function

Nadaraya-Watson smoothing

To avoid divide zero errors

Place field estimation: Buzsaki CRCNS hc3-cd013.527, cell 5

HSV colormap with

Hue = firing rate

Value = occupancy

So non-visited areas are black

occupancy spike

Gaussian Process Regression

Types of tuning function

Artificial network model neurons are simple

But assembled into very large networks

VGG19 network

And trained with gradient descent of weight parameters

Input

Activity progatation

Target output

Error backpropagation

Artificial neural networks for biological neural tuning functions

• ANNs are extremely powerful for AI applications
• But need vast amounts of training data
• There exist vast training sets for AI applications (e.g. photos and their captions)
• But most neural recordings don’t produce enough data
• Solution: first fit weights to large dataset
• An AI dataset or a rare, very large recording of biological neurons
• Or use an “off the shelf” AI network
• Then predict biological neurons’ tuning curves from artificial neural network activity

​

Biological neurons

• Recorded responses of 262 neurons from Monkey V1 to 7250 visual stimuli
• Not enough to train the weights of the network
• But enough to predict neuronal responses from ANN activity

​

​

Image shown to monkey

Image input to VGG19

Activity of 8/256 features from layer conv 3_1

How to predict biological neurons from ANN units?

Too many predictors

Geometric interpretation

Signal

Noise

Geometric interpretation

Signal

Noise

Overfitting = large weight vectors

Predicting neuron from ANN features

Sum-square error

Penalty term

Reduces overfitting by keeping weights small

Ridge regression predicting neuron 142

Assessing tuning curves

• To test if you have fit well, need out-of-sample stimuli
• Can the tuning curve predict responses to stimuli not used to fit it?
• Extrapolation is harder than interpolation

​

• To test null hypothesis of no generalization:
• Compute a test statistic measuring prediction of stimuli
• Compare it to a null ensemble rerandomizing stimulus order

​

• What test statistic to use?

Fraction of stimulus-related variance explained

VGG19 explains 37% of stimulus-related variance in monkey V1

Statistical test

• To test the null hypothesis that the ANN model is useless, randomize order of test-set stimuli so you are predicting activity from the wrong image.

​

• All cells in database significant at p=0.001
• (they only included cells that respond to the stimuli.)

​

​

Null distribution for one cell