<jupyter_start><jupyter_text>Out of Tensor factorisation> Out of tensor factorisation- toc: true - badges: true- comments: true- author: <NAME>- categories: [ML] In a [previous post](./nmf-out-matrix.html), we had looked at predicting for users who weren't a part of the original matrix factorisation. In this post, we'll look at the same for 3-d tensors. In case you want to learn more about tensor factorisations, look at my earlier [post](./tensor-decomposition-autograd.html).  General Tensor Factorisation ![](https://nipunbatra.github.io/blog/images/tf.png) General tensor factorisation for a 3d tensor A (M X N X O) would produce 3 factors- X (M X K), Y (N X K) and Z (O X K). The $A_{ijl}$ entry can be found as (Khatri product) :$$ A_{ijl} = \sum_k{X_{ik}Y_{jk}Z_{lk}}$$  Learning $X_M$ factor However, we'd assume that the $M^{th}$ entry isn't a part of this decomposition. So, how do we obtain the X factors correspondonding to $M^{th}$ entry? We learn the Y and Z factors from the tensor A (excluding the $M^{th}$ row entries). We assume the Y and Z learnt to be shared across the entries across rows of A (1 through M).  ![](https://nipunbatra.github.io/blog/images/tf-last-row.png) The above figure shows the **latent factor for X ($X_{M}$) corresponding to the $M^{th}$ entry of X that we wish to learn**. On the LHS, we see the matrix corresponding to $A_{M}$. The highlighted entry of $A_{M}$ is created by element-wise multiplication of $X_M, Y_0, Z_0$ and then summing. Thus, each of the N X O entries of $A_M$ are created by multiplying $X_M$ with a row from Y and a row from Z. In general, $$A_{M, n, o}  = \sum_k{X_{M, k} \times Y_{n, k} \times Z_{o, k}}$$ Now, to learn $X_M$, we plan to use least squares. For that, we need to reduce the problem into $\alpha x = \beta$ We do this as follows:1. We flatten out the A_M matrix into a vector containing N X O entries and call it $\beta$2. We create a matrix by element-wise multiplication of each row of Y with each row of Z to create $\alpha$ of shape (N X O, K)We can now write, $$ \alpha X_M^T \approx \beta $$Thus, X_M^T = Least Squares ($\alpha, \beta$)Ofcourse, $\beta$ can have missing entries, which we mask out. Thus, we can write:$X_M^T$ = Least Squares ($\alpha [Mask], \beta [Mask]$)In case we're doing a non-negative tensor factorisation, we can instead learn $X_M^T$ as follows:$X_M^T$ = Non-negative Least Squares ($\alpha [Mask], \beta [Mask]$)  Code example<jupyter_code>import tensorly
from tensorly.decomposition import parafac, non_negative_parafac
import numpy as np<jupyter_output><empty_output><jupyter_text>Creating the tensor to be decomposed<jupyter_code>M, N, O = 10, 4, 3 #user, movie, feature
t = np.arange(M*N*O).reshape(M, N, O).astype('float32')
t[0] #First entry<jupyter_output><empty_output><jupyter_text>Setting a few entries of the last user to be unavailable/unknown<jupyter_code>t_orig = t.copy() # creating a copy
t[-1,:,:][0, 0] = np.NAN
t[-1,:,:][2, 2] = np.NAN
t[-1,:,:]<jupyter_output><empty_output><jupyter_text>Standard Non-negative PARAFAC decomposition<jupyter_code>K = 2
# Notice, we factorise a tensor with one less user. thus, t[:-1, :, :]
X, Y, Z = non_negative_parafac(t[:-1,:,:], rank=K)
X.shape, Y.shape, Z.shape
Y
Z<jupyter_output><empty_output><jupyter_text>Creating $\alpha$ by element-wise multiplication of Y, Z and reshaping<jupyter_code>alpha = np.einsum('nk, ok -> nok', Y, Z).reshape((N*O, K))
print alpha

print "\nShape of alpha = ", alpha.shape
from scipy.optimize import nnls<jupyter_output><empty_output><jupyter_text>Creating $\beta$<jupyter_code>beta = t[-1,:,:].reshape(N*O, 1)
mask = ~np.isnan(beta).flatten()
beta[mask].reshape(-1, 1)<jupyter_output><empty_output><jupyter_text>Learning $X_M$<jupyter_code>X_M = nnls(alpha[mask], beta[mask].reshape(-1, ))[0].reshape((1, K))
X_M<jupyter_output><empty_output><jupyter_text>Comparing X_M with other entries from X<jupyter_code>X<jupyter_output><empty_output><jupyter_text>It seems that the first column captures the increasing trend of values in the tensor  Predicting missing entries using tensor multiplication<jupyter_code>np.round(np.einsum('ir, jr, kr -> ijk', X_M, Y, Z))<jupyter_output><empty_output><jupyter_text>Actual entries<jupyter_code>t_orig[-1, :, :]<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1338931653837182, 'fraction_numerical': 0.007697690692792162, 'mean_word_length': 5.269005847953216, 'pattern_counts': {'":': 0, '<': 42, '<?xml version=': 0, '>': 45, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '385671', 'license': 'Apache-2.0', 'n_tokens_mistral': 1549, 'n_tokens_neox': 1488, 'n_words': 536, 'path': '_notebooks/2017-04-20-parafac-out-tensor.ipynb', 'repo_name': 'nipunbatra/blog3', 'size': '4292'}