1 of 1

higher frequencies

past time points

Normalization in auditory systems is thought to be local, whereas deep neural networks commonly normalize using global statistics
In place of standard normalization, we learn normalization kernels, representing a local neighborhood of the receptive field, to simulate divisive normalization in CNNs
Divisive normalization is a canonical neural computation known to enhance local contrast and reduce statistical dependencies in input stimuli, both critical to many auditory tasks^[1,2]
Questions we'd like to ask:

Do structured normalization kernels emerge from task optimization?

Do they produce suppressive effects across time and frequency?

Do they support human-level task performance and generalization ability?

Introduction

Modeling Divisive Normalization in the Central Auditory

Pathway With Convolutional Neural Networks (CNNs)

Chanel Cheng, Annesya Banerjee, Josh McDermott

McGovern Institute for Brain Research, CBMM, MIT BCS

Methodology

Normal hearing (NH) cochlea model produces nervegram representations of sounds across time and frequency
CNN model with divisive normalization receives nervegram input and computes corresponding task label
𝛾, β, 𝝀, ɑ are learnable parameters that shape the local divisive neighborhood

Learned Network Results

This work was supported by MSRP Bio 2023, with funding from MIT's School of Science, the National Science Foundation, and the Simons Center for the Social Brain.

CNN trained to identify word spoken at the midpoint of sentence excerpts with background noise
794 word labels and ~250k training samples
Signal to Noise Ratio from -10 to +10 dB

2 sec. sentence excerpt

background noise

Suppression effect observed from neuro- physiological recordings

conv 3

conv 0

conv 1

conv 2

conv 4

conv 5

conv 6

dense

div norm input

div norm 0

div norm 2

div norm 1

div norm 3

div norm 4

div norm 5

div norm 6

central stochasticity (additive noise)

Speech Recognition Task

Conclusion

Learned local normalization kernels reproduce effects of adaptation
Suppressive effects found in biology were not fully replicated, for reasons we do not currently understand
Human-level task performance is exhibited with improved generalizability to new inputs

x_i

((𝝀/ɑ)Σ_j 𝛾_j x_j + 𝝈)ᵝ

y_i =

1D kernel weights were learned independently for each dimension
Causal kernel only normalized across past time points
Frequency-time points contributing more to normalization were assigned higher weights

Weights relevant to speech recognition task learned across time and frequency

Learned Normalization Kernels

Auditory nerve response is the CNN input

References

References can be found by scanning the QR code to the right or by visiting: https://qrco.de/beCngc

CNN has 54,943,911 optimizable parameters

Divisive norm:

behavioral judgement —— task label

Response to excitatory tone is reduced by suppressor tone^[6]

Two-tone suppression effect

Simulated central auditory pathway

Simulated normal cochlea

Response to ongoing stimuli decays to a steady- state level^[7]

Adaptation to ongoing stimuli

Two-tone suppression effect is mostly absent after training for divisive norm

abnormal response pattern

limited suppression effect

standard norm

divisive norm

excitatory tone (2kHz),

suppressor tone (3kHz)

Learned divisive norm produces greater adaptation in response to continuous stimuli

firing rate reaches different steady-state levels

human data [-9 to +3 dB]

Both models perform similarly to humans on task

Performance for out-of-distribution speech is improved for model with divisive norm

whispered speech

time-reversed speech

noise-vocoded speech

Time (ms)

Frequency (kHz)

𝝀

target neuron x_i

𝛾

Session 2, Board 6

My project explored divisive normalization in the central auditory pathway using convolutional neural networks.

[Introduction]

In biological auditory systems, normalization is thought to be local. [where the response of a target neuron is modulated or normalized by the response of locally surrounding neurons -- this is a form of gain control] However, most deep neural networks have used normalization relying on global statistics.

In place of standard normalization, we implement learnable normalization kernels covering a local neighborhood of the receptive field, to simulate divisive normalization in CNNs.

Divisive normalization is a canonical neural computation known to enhance local contrast and reduce statistical dependencies in input stimuli. Both of these are critical to everyday auditory tasks we perform, such as speech recognition and sound localization. However, these mechanisms are compromised in hearing impaired people, which leads us to wonder about the computational basis of divisive normalization and how it aids in these tasks that hearing impaired people seem to struggle with.

Some questions we'd like to ask, for example are:

Do structured normalization kernels emerge from task optimization?
Do they produce suppressive effects across time and frequency?
Do they support human-level task performance and generalization ability?

Previous work shows that divisive normalization produces suppressive behaviors observed in biological neurons. These suppressive behaviors such as adaptation to ongoing stimuli and two-tone suppression are known to be important for auditory tasks. For example, adaptation over time allows neurons to be more sensitive to local changes in stimuli. Two-tone suppression reduces redundancy in neural activation and better captures frequency-level distinctions. [Explain the adaptation and suppression in level of detail appropriate to the person's background]

[Methodology]

To simulate this, we implemented a CNN that takes a nervegram representation of sound, applies divisive normalization followed by convolution at each stage and computes a corresponding task label. We formulate our divisive normalization based on equations reported in previous neurophysiological experiments, where the response of a target neuron is divided by the weighted sum of a local neighborhood of neurons. The optimizable parameters were gamma, beta, lambda, and alpha. Gamma corresponds to the weights within the local neighborhood. Lambda controls the span of the neighborhood. Alpha and beta were scaling and exponential factors that we found were essential to properly train the model. Otherwise, we found that the model would be stuck at 0% accuracy without learning anything at all.

[Speech Recognition Task]

Using our divisive normalization formulation, we optimize our CNN for speech recognition using a dataset consisting of 2 second sentence excerpts with added background noise. The level of background noise ranged from -10dB to +10dB.

[Learned Normalization Kernels]

Following optimization, we examined the learned normalization kernel indicating the local neighborhood to normalize across. We learn a 1D kernel for frequency and time, and normalize across past time points only to match biological normalization, which relies only on past time neural responses. The learned kernels show effects of adaptation to faraway time points and nearby frequencies with the task-relevant weights shown by the color bar.

[Learned Network Results]

Furthermore, we probe the learned network activations for signatures of two-tone suppression and adaptation over time. While the model produces adaptation over time in response to a continuous stimuli, the two-tone suppression effect differs substantially from what is observed from neurophysiological recordings in nonhuman animals. The model does, however, perform at human-level on the speech recognition task along with generalizing better to unfamiliar stimuli such as whispered speech, time-reversed speech, and noise-vocoded speech.

[Conclusion]

For our conclusion, we note that our model does not fully replicate the suppressive effects found in biology, but shows signs of learned adaptation and human-like behavior in speech recognition. Some things we hope to try out in the future are to train the divisive normalization model on other real-world auditory tasks with more diverse datasets to see whether that will produce a greater suppressive effect than what we observed here.