1
Grid Cells: Global positioning system that helps people navigate
3
Please Read!
4
Structural and Functional Brain Networks
5
Inside the Nucleus: Chromosome Territories
Tan, Longzhi, et al. "Three-dimensional genome structures of single diploid human cells." Science 361.6405 (2018): 924-928.
6
A Tensor is an 𝑑-way array
7
Notations
8
Tensor Unfolding
9
A Common Framework for Tensor Computations
Tensor Unfolding
10
Illustration of row-wise and column-wise unfolding (flattening, matricizing) of a third-order tensor
Andrzej Cichocki, Rafal Zdunek, and Shun'ichi Amari. Nonnegative Matrix and Tensor Factorizations: Applications to Exploratory Multi-way Data Analysis and Blind Source (2009)
11
Unfolding (matricizing) of a third-order tensor
12
SVD Expansion
Rank One Tensors
Building block for decomposition: Vector outer products = Rank one tensors
Matrix version (2 – way)
Rank one matrix
Tensor version (3 – way)
Rank one tensor
Building block for decomposition: Vector outer products = Rank one tensors
Visualization
gets weird.. Math is still good!
Rank one Tensors
15
Data
Low-rank model
Matrix notation
Sum of squared errors
Low Rank Approximation
Tensor Decompositions
CP Model: Sum of d-way outer products, useful for interpretation
Tucker Model: Project onto high-variance subspaces to reduce dimensionality
CP Decomposition
The full name is CANDECOMP/PARAFAC Decomposition
CANDECOMP = Canonical Decomposition
PARAFAC = Parallel Factors Decomposition
CP Tensor Factorization (3-way): Detecting low-rank 3-way structure
Higher-order analogue of the SVD
CP first invented in 1927
F. L. Hitchcock, The Expression of a Tensor or a Polyadic as a Sum of Products, Journal of Mathematics and Physics, 1927
Tensor notation
Sum of squared errors
Factor matrices
Component
CP Decomposition
Example 1: Neuronal Activity
Williams, A. H., Kim, T. H., Wang, F., Vyas, S., Ryu, S. I., Shenoy, K. V., ... & Ganguli, S. (2018). Unsupervised discovery of demixed, low-dimensional neural dynamics across multiple timescales through tensor component analysis. Neuron, 98(6), 1099-1115.
Example 1: Neuronal activity
Mouse
in “maze”
Neural activity
One Column of Neuron x
Time matrix
× 600 trials (over 5 days)
Microscope by
Inscopix
Trials vary start position and strategies
Example: Neuronal activity
8-Component CP decomposition of data
8-Component CP decomposition of data
Got the treat! Happy neurons….
8-Component CP decomposition of data
8-Component CP decomposition of data
Clear separation. Starting position Neurons understand mouse has started. ….
8-Component CP decomposition of data
Example 2: Application to hazardous gas
Vergara, A., Fonollosa, J., Mahiques, J., Trincavelli, M., Rulkov, N., & Huerta, R. (2013). On the performance of gas sensor arrays in open sampling systems using Inhibitory Support Vector Machines. Sensors and Actuators B: Chemical, 185, 462-477.
Factors
Example 2: Application to hazardous gas
29
Some Mathematics….
30
The Khatri-Rao Product
Definition:
C.R. Rao and S.K. Mitra (1971). Generalized Inverse of Matrices and Applications, John Wiley and Sons, New York
31
The Khatri-Rao Product
If
then the Khatri-Rao product of B and C is given by
Note:
32
The Kronecker Product
33
The Kronecker Product
34
All Possible Entry- Entry Products
35
Kronecker Products of Kronecker Products
36
You have seen them before
37
Block matrix
38
Block Matrices: Addition
39
Block Matrix: Multiplication
40
Block Matrix: Transposition
41
A block matrix can have block matrix entries
42
Reshape
43
Reshape
44
SVD
45
SVD
46
The Tucker product
Definition:
More References
Higher-order SVD
De Lathauwer L, De Moor B, Vandewalle J. A multilinear singular value decomposition. SIAM journal on Matrix Analysis and Applications. 2000;21(4):1253-78.
Tensor SVD
Brazell, M., Li, N., Navasca, C. and Tamon, C., 2013. Solving multilinear systems via tensor inversion. SIAM Journal on Matrix Analysis and Applications, 34(2), pp.542-570.
Tensor-Train
Oseledets, I.V., 2011. Tensor-train decomposition. SIAM Journal on Scientific Computing, 33(5), pp.2295-2317.
Tensor Unfolding
Ragnarsson, S. and Van Loan, C.F., 2012. Block tensor unfoldings. SIAM Journal on Matrix Analysis and Applications, 33(1), pp.149-169.
48
The Tucker product representation
Before…
Now…
49
The Higher-Order SVD (HOSVD)
50
The HOSVD of a matrix
51
The HOSVD of a matrix