If you need more information, these references are helpful:�
The following slides contain material from Tammy Kolda, Charles Van Loan and Lars Elden
Tensor Decompositions
2
A Tensor is an 𝑑-way array
3
Grid Cells: Global positioning system that helps people navigate
5
Please Read!
6
Inside the Nucleus: Chromosome Territories
Tan, Longzhi, et al. "Three-dimensional genome structures of single diploid human cells." Science 361.6405 (2018): 924-928.
7
Some Mathematics….
8
The Kronecker Product
9
The Kronecker Product
10
All Possible Entry- Entry Products
11
Kronecker Products of Kronecker Products
12
You have seen them before
13
Block matrix
14
Block Matrices: Addition
15
Block Matrix: Multiplication
16
Block Matrix: Transposition
17
A block matrix can have block matrix entries
18
Reshape
19
Reshape
20
SVD Expansion
21
SVD
22
SVD
23
The Tucker product
Definition:
24
The Tucker product representation
Before…
Now…
25
The Higher-Order SVD (HOSVD)
26
The HOSVD of a matrix
27
The HOSVD of a matrix
28
Notations
29
Tensor Unfolding
30
A Common Framework for Tensor Computations
Tensor Unfolding
31
Illustration of row-wise and column-wise unfolding (flattening, matricizing) of a third-order tensor
Andrzej Cichocki, Rafal Zdunek, and Shun'ichi Amari. Nonnegative Matrix and Tensor Factorizations: Applications to Exploratory Multi-way Data Analysis and Blind Source (2009)
32
Unfolding (matricizing) of a third-order tensor
Rank One Tensors
Building block for decomposition: Vector outer products = Rank one tensors
Matrix version (2 – way)
Rank one matrix
Tensor version (3 – way)
Rank one tensor
Building block for decomposition: Vector outer products = Rank one tensors
Visualization
gets weird.. Math is still good!
Rank one Tensors
35
Data
Low-rank model
Matrix notation
Sum of squared errors
Low Rank Approximation
Tensor Decompositions
CP Model: Sum of d-way outer products, useful for interpretation
Tucker Model: Project onto high-variance subspaces to reduce dimensionality
CP Decomposition
The full name is CANDECOMP/PARAFAC Decomposition
CANDECOMP = Canonical Decomposition
PARAFAC = Parallel Factors Decomposition
CP Tensor Factorization (3-way): Detecting low-rank 3-way structure
Higher-order analogue of the SVD
CP first invented in 1927
F. L. Hitchcock, The Expression of a Tensor or a Polyadic as a Sum of Products, Journal of Mathematics and Physics, 1927
Tensor notation
Sum of squared errors
Factor matrices
Component
CP Decomposition
Example 1: Neuronal Activity
Williams, A. H., Kim, T. H., Wang, F., Vyas, S., Ryu, S. I., Shenoy, K. V., ... & Ganguli, S. (2018). Unsupervised discovery of demixed, low-dimensional neural dynamics across multiple timescales through tensor component analysis. Neuron, 98(6), 1099-1115.
Example 1: Neuronal activity
Mouse
in “maze”
Neural activity
One Column of Neuron x
Time matrix
× 600 trials (over 5 days)
Microscope by
Inscopix
Trials vary start position and strategies
Example: Neuronal activity
8-Component CP decomposition of data
8-Component CP decomposition of data
Got the treat! Happy neurons….
8-Component CP decomposition of data
8-Component CP decomposition of data
Clear separation. Starting position Neurons understand mouse has started. ….
8-Component CP decomposition of data
Example 2: Application to hazardous gas
Vergara, A., Fonollosa, J., Mahiques, J., Trincavelli, M., Rulkov, N., & Huerta, R. (2013). On the performance of gas sensor arrays in open sampling systems using Inhibitory Support Vector Machines. Sensors and Actuators B: Chemical, 185, 462-477.
Factors
Example 2: Application to hazardous gas