Understanding neural networks from their weights
Thomas Dooms
Baseline
Your brilliant idea
time
metric
Understanding neural networks from their weights
Thomas Dooms
The weights in ordinary neural networks
are not meaningful
-1.0
1.0
-1.0
1.0
Input 1
Input 2
0.0
1.0
Input 3
-1.0
0.0
0.0
ReLU
Output 3
Output 1
Output 2
Most activation functions can represent
feature interactions of any degree
Input 1
Input 2
Input 3
Input 4
Activation functions can be replaced with
a linear gate without a drop in accuracy
Output
ReLU
Linear
Input
Output
Linear
Input
Linear
x
Each output is described by
a sum of pairwise feature interactions
-1.0
1.0
-1.0
1.0
Input 1
Input 2
Input 1
Input 2
We can use PCA to reveal
the important components
These components can be used
to fully decompose the model
Eigenvalues
Eigenvectors
Analyzing model weights can reveal issues
including learning unexpected patterns
Test accuracy: 98.1%
Test accuracy: 98.2%
Understanding neural networks from their weights
Thomas Dooms