1
A Tensor is an 𝑑-way array
If you need more information, these references are helpful:�
The following slides contain material from Tammy Kolda, Charles Van Loan and Lars Elden
Tensor Decompositions
3
Grid Cells: Global positioning system that helps people navigate
5
Please Read!
6
Inside the Nucleus: Chromosome Territories
Tan, Longzhi, et al. "Three-dimensional genome structures of single diploid human cells." Science 361.6405 (2018): 924-928.
7
Some Mathematics….
8
The Khatri-Rao Product
Definition:
C.R. Rao and S.K. Mitra (1971). Generalized Inverse of Matrices and Applications, John Wiley and Sons, New York
9
The Khatri-Rao Product
If
then the Khatri-Rao product of B and C is given by
Note:
10
The Kronecker Product
11
The Kronecker Product
12
All Possible Entry- Entry Products
13
Kronecker Products of Kronecker Products
14
You have seen them before
15
Block matrix
16
Block Matrices: Addition
17
Block Matrix: Multiplication
18
Block Matrix: Transposition
19
A block matrix can have block matrix entries
20
Reshape
21
Reshape
22
SVD
23
SVD
24
The Tucker product
Definition:
25
The Tucker product representation
Before…
Now…
26
The Higher-Order SVD (HOSVD)
27
The HOSVD of a matrix
28
The HOSVD of a matrix
29
Notations
30
Tensor Unfolding
31
A Common Framework for Tensor Computations
Tensor Unfolding
32
Illustration of row-wise and column-wise unfolding (flattening, matricizing) of a third-order tensor
Andrzej Cichocki, Rafal Zdunek, and Shun'ichi Amari. Nonnegative Matrix and Tensor Factorizations: Applications to Exploratory Multi-way Data Analysis and Blind Source (2009)
33
Unfolding (matricizing) of a third-order tensor
34
SVD Expansion
Rank One Tensors
Building block for decomposition: Vector outer products = Rank one tensors
Matrix version (2 – way)
Rank one matrix
Tensor version (3 – way)
Rank one tensor
Building block for decomposition: Vector outer products = Rank one tensors
Visualization
gets weird.. Math is still good!
Rank one Tensors
37
Data
Low-rank model
Matrix notation
Sum of squared errors
Low Rank Approximation
Tensor Decompositions
CP Model: Sum of d-way outer products, useful for interpretation
Tucker Model: Project onto high-variance subspaces to reduce dimensionality
CP Decomposition
The full name is CANDECOMP/PARAFAC Decomposition
CANDECOMP = Canonical Decomposition
PARAFAC = Parallel Factors Decomposition
CP Tensor Factorization (3-way): Detecting low-rank 3-way structure
Higher-order analogue of the SVD
CP first invented in 1927
F. L. Hitchcock, The Expression of a Tensor or a Polyadic as a Sum of Products, Journal of Mathematics and Physics, 1927
Tensor notation
Sum of squared errors
Factor matrices
Component
CP Decomposition
Example 1: Neuronal Activity
Williams, A. H., Kim, T. H., Wang, F., Vyas, S., Ryu, S. I., Shenoy, K. V., ... & Ganguli, S. (2018). Unsupervised discovery of demixed, low-dimensional neural dynamics across multiple timescales through tensor component analysis. Neuron, 98(6), 1099-1115.
Example 1: Neuronal activity
Mouse
in “maze”
Neural activity
One Column of Neuron x
Time matrix
× 600 trials (over 5 days)
Microscope by
Inscopix
Trials vary start position and strategies
Example: Neuronal activity
8-Component CP decomposition of data
8-Component CP decomposition of data
Got the treat! Happy neurons….
8-Component CP decomposition of data
8-Component CP decomposition of data
Clear separation. Starting position Neurons understand mouse has started. ….
8-Component CP decomposition of data
Example 2: Application to hazardous gas
Vergara, A., Fonollosa, J., Mahiques, J., Trincavelli, M., Rulkov, N., & Huerta, R. (2013). On the performance of gas sensor arrays in open sampling systems using Inhibitory Support Vector Machines. Sensors and Actuators B: Chemical, 185, 462-477.
Factors
Example 2: Application to hazardous gas