<jupyter_start><jupyter_text>Low-Tubal-Rank Autoregressive Tensor Completion Imputer (LATC-Tubal-imputer)This notebook shows how to implement a LATC-Tubal imputer on some real-world large-scale data sets. To overcome the problem of missing values within multivariate time series data, this method takes into account both low-rank structure and time series regression. Meanwhile, to make the model scalable, we also integrate linear transform into the LATC model. For an in-depth discussion of LATC-Tubal-imputer, please see [1].[1] <NAME>, <NAME>, <NAME> (2020). Scalable low-rank autoregressive tensor learning for spatiotemporal traffic data imputation. arXiv: 2008.03194. [PDF] [data]<jupyter_code>import numpy as np
from numpy.linalg import inv as inv<jupyter_output><empty_output><jupyter_text>Define LATC-imputer kernelWe start by introducing some necessary functions that relies on `Numpy`.ten2mat: Unfold tensor as matrix by specifying mode.mat2ten: Fold matrix as tensor by specifying dimension (i.e, tensor size) and mode.svt: Implement the process of Singular Value Thresholding (SVT).<jupyter_code>def ten2mat(tensor, mode):
    return np.reshape(np.moveaxis(tensor, mode, 0), (tensor.shape[mode], -1), order = 'F')

def mat2ten(mat, dim, mode):
    index = list()
    index.append(mode)
    for i in range(dim.shape[0]):
        if i != mode:
            index.append(i)
    return np.moveaxis(np.reshape(mat, list(dim[index]), order = 'F'), 0, mode)
def unitary_transform(tensor, Phi):
    return np.einsum('kt, ijk -> ijt', Phi, tensor)

def inv_unitary_transform(tensor, Phi):
    return np.einsum('kt, ijt -> ijk', Phi, tensor)
def tsvt(tensor, Phi, tau):
    dim = tensor.shape
    X = np.zeros(dim)
    tensor = unitary_transform(tensor, Phi)
    for t in range(dim[2]):
        u, s, v = np.linalg.svd(tensor[:, :, t], full_matrices = False)
        r = len(np.where(s > tau)[0])
        if r >= 1:
            s = s[: r]
            s[: r] = s[: r] - tau
            X[:, :, t] = u[:, : r] @ np.diag(s) @ v[: r, :]
    return inv_unitary_transform(X, Phi)<jupyter_output><empty_output><jupyter_text>compute_mape: Compute the value of Mean Absolute Percentage Error (MAPE).compute_rmse: Compute the value of Root Mean Square Error (RMSE).> Note that $$\mathrm{MAPE}=\frac{1}{n} \sum_{i=1}^{n} \frac{\left|y_{i}-\hat{y}_{i}\right|}{y_{i}} \times 100, \quad\mathrm{RMSE}=\sqrt{\frac{1}{n} \sum_{i=1}^{n}\left(y_{i}-\hat{y}_{i}\right)^{2}},$$ where $n$ is the total number of estimated values, and $y_i$ and $\hat{y}_i$ are the actual value and its estimation, respectively.<jupyter_code>def compute_mape(var, var_hat):
    return np.sum(np.abs(var - var_hat) / var) / var.shape[0]

def compute_rmse(var, var_hat):
    return  np.sqrt(np.sum((var - var_hat) ** 2) / var.shape[0])<jupyter_output><empty_output><jupyter_text>The main idea behind LATC-imputer is to approximate partially observed data with both low-rank structure and time series dynamics. The following `imputer` kernel includes some necessary inputs:dense_tensor: This is an input which has the ground truth for validation. If this input is not available, you could use dense_tensor = sparse_tensor.copy() instead.sparse_tensor: This is a partially observed tensor which has many missing entries.time_lags: Time lags, e.g., time_lags = np.array([1, 2, 3]). alpha: Weights for tensors' nuclear norm, e.g., alpha = np.ones(3) / 3. rho: Learning rate for ADMM, e.g., rho = 0.0005. lambda0: Weight for time series regressor, e.g., lambda0 = 5 * rho. If lambda0 = 0, then this imputer is actually a standard low-rank tensor completion (i.e., High-accuracy Low-Rank Tensor Completion, or HaLRTC).epsilon: Stop criteria, e.g., epsilon = 0.001. maxiter: Maximum iteration to stop algorithm, e.g., maxiter = 50.<jupyter_code>def imputer(dense_tensor, sparse_tensor, time_lags, rho0, lambda0, epsilon, maxiter):
    """Low-Tubal-Rank Autoregressive Tensor Completion, LATC-Tubal-imputer."""
    dim = np.array(sparse_tensor.shape)
    dim_time = np.int(np.prod(dim) / dim[0])
    d = len(time_lags)
    max_lag = np.max(time_lags)
    sparse_mat = ten2mat(sparse_tensor, 0)
    pos_missing = np.where(sparse_mat == 0)
    pos_test = np.where((dense_tensor != 0) & (sparse_tensor == 0))
    
    T = np.zeros(dim)                         # \boldsymbol{\mathcal{T}}
    Z = sparse_mat.copy()                     # \boldsymbol{Z}
    Z[pos_missing] = np.mean(sparse_mat[sparse_mat != 0])
    A = 0.001 * np.random.rand(dim[0], d)     # \boldsymbol{A}
    it = 0
    ind = np.zeros((d, dim_time - max_lag), dtype = np.int_)
    for i in range(d):
        ind[i, :] = np.arange(max_lag - time_lags[i], dim_time - time_lags[i])
    last_mat = sparse_mat.copy()
    snorm = np.linalg.norm(sparse_mat, 'fro')
    rho = rho0
    temp1 = ten2mat(mat2ten(Z, dim, 0), 2)
    _, Phi = np.linalg.eig(temp1 @ temp1.T)
    del temp1
    while True:
        rho = min(rho * 1.05, 1e5)
        X = tsvt(mat2ten(Z, dim, 0) - T / rho, Phi, 1 / rho)
        mat_hat = ten2mat(X, 0)
        mat0 = np.zeros((dim[0], dim_time - max_lag))
        temp2 = ten2mat(rho * X + T, 0)
        if lambda0 > 0:
            for m in range(dim[0]):
                Qm = mat_hat[m, ind].T
                A[m, :] = np.linalg.pinv(Qm) @ Z[m, max_lag :]
                mat0[m, :] = Qm @ A[m, :]
            Z[pos_missing] = np.append((temp2[:, : max_lag] / rho), (temp2[:, max_lag :] + lambda0 * mat0) 
                                       / (rho + lambda0), axis = 1)[pos_missing]
        else:
            Z[pos_missing] = temp2[pos_missing] / rho
        T = T + rho * (X - mat2ten(Z, dim, 0))
        tol = np.linalg.norm((mat_hat - last_mat), 'fro') / snorm
        last_mat = mat_hat.copy()
        it += 1
        if it % 10 == 0:
            temp1 = ten2mat(mat2ten(Z, dim, 0) - T / rho, 2)
            _, Phi = np.linalg.eig(temp1 @ temp1.T)
            del temp1
        if it % 1 == 0:
            print('Iter: {}'.format(it))
            print('Tolerance: {:.6}'.format(tol))
            var = dense_tensor[pos_test]
            var_hat = X[pos_test]
            print('MAPE: {:.6}'.format(compute_mape(var, var_hat)))
            print('RMSE: {:.6}'.format(compute_rmse(var, var_hat)))
            print()
        if (tol < epsilon) or (it >= maxiter):
            break

    print('Total iteration: {}'.format(it))
    print('Tolerance: {:.6}'.format(tol))
    var = dense_tensor[pos_test]
    var_hat = X[pos_test]
    print('Imputation MAPE: {:.6}'.format(compute_mape(var, var_hat)))
    print('Imputation RMSE: {:.6}'.format(compute_rmse(var, var_hat)))
    print()
    
    return X<jupyter_output><empty_output><jupyter_text>If you want to set parameters reasonably, please use this cross validation on your data set.  California data - 4WWe generate **random missing (RM)** values on California traffic speed data set.<jupyter_code>import numpy as np
import pandas as pd
np.random.seed(1000)

data = pd.read_csv('../datasets/California-data-set/pems-4w.csv', header = None)
dense_tensor = mat2ten(data.values, np.array([data.values.shape[0], 288, 4 * 7]), 0)
random_tensor = np.random.rand(data.values.shape[0], 288, 4 * 7)

missing_rate = 0.3

### Random missing (RM) scenario:
binary_tensor = np.round(random_tensor + 0.5 - missing_rate)
sparse_tensor = np.multiply(dense_tensor, binary_tensor)

del data, random_tensor, binary_tensor
import time
start = time.time()
time_lags = np.array([1, 2, 3, 4, 5, 6, 286, 287, 288, 289, 290, 291])
rho = 1e-4
lambda0 = 0 * rho
epsilon = 1e-3
maxiter = 100
tensor_hat = imputer(dense_tensor, sparse_tensor, time_lags, rho, lambda0, epsilon, maxiter)
end = time.time()
print('Running time: %.2f minutes' % ((end - start)/60.0))
import numpy as np
import pandas as pd
np.random.seed(1000)

data = pd.read_csv('../datasets/California-data-set/pems-4w.csv', header = None)
dense_tensor = mat2ten(data.values, np.array([data.values.shape[0], 288, 4 * 7]), 0)
random_tensor = np.random.rand(data.values.shape[0], 288, 4 * 7)

missing_rate = 0.7

### Random missing (RM) scenario:
binary_tensor = np.round(random_tensor + 0.5 - missing_rate)
sparse_tensor = np.multiply(dense_tensor, binary_tensor)

del data, random_tensor, binary_tensor
import time
start = time.time()
time_lags = np.array([1, 2, 3, 4, 5, 6, 286, 287, 288, 289, 290, 291])
rho = 1e-4
lambda0 = 0 * rho
epsilon = 1e-3
maxiter = 100
tensor_hat = imputer(dense_tensor, sparse_tensor, time_lags, rho, lambda0, epsilon, maxiter)
end = time.time()
print('Running time: %.2f minutes' % ((end - start)/60.0))<jupyter_output>Iter: 1
Tolerance: 1.49565
MAPE: 0.111673
RMSE: 7.6865

Iter: 2
Tolerance: 0.0617153
MAPE: 0.0962786
RMSE: 6.60298

Iter: 3
Tolerance: 0.0689141
MAPE: 0.0764408
RMSE: 5.57124

Iter: 4
Tolerance: 0.0610141
MAPE: 0.0647303
RMSE: 5.1603

Iter: 5
Tolerance: 0.0446548
MAPE: 0.0618997
RMSE: 5.09779

Iter: 6
Tolerance: 0.0329652
MAPE: 0.0598007
RMSE: 4.94375

Iter: 7
Tolerance: 0.0315494
MAPE: 0.0556912
RMSE: 4.63752

Iter: 8
Tolerance: 0.0337556
MAPE: 0.0503774
RMSE: 4.26924

Iter: 9
Tolerance: 0.0333318
MAPE: 0.0464125
RMSE: 3.98788

Iter: 10
Tolerance: 0.0290838
MAPE: 0.0446962
RMSE: 3.8367

Iter: 11
Tolerance: 0.0543425
MAPE: 0.0466334
RMSE: 3.89826

Iter: 12
Tolerance: 0.0225804
MAPE: 0.0469646
RMSE: 3.92097

Iter: 13
Tolerance: 0.0204167
MAPE: 0.043974
RMSE: 3.71667

Iter: 14
Tolerance: 0.0243385
MAPE: 0.0397729
RMSE: 3.4319

Iter: 15
Tolerance: 0.0247116
MAPE: 0.0364274
RMSE: 3.20929

Iter: 16
Tolerance: 0.0217448
MAPE: 0.034473
RMSE: 3.0748

Iter: 17
Tolerance: 0.018344
MAPE: 0.033078[...]<jupyter_text>We generate **non-random missing (NM)** values on Guangzhou traffic speed data set. Then, we conduct the imputation experiment.<jupyter_code>import numpy as np
import pandas as pd
np.random.seed(1000)

data = pd.read_csv('../datasets/California-data-set/pems-4w.csv', header = None)
dense_tensor = mat2ten(data.values, np.array([data.values.shape[0], 288, 4 * 7]), 0)
random_matrix = np.random.rand(data.values.shape[0], 4 * 7)

missing_rate = 0.3

### Non-random missing (NM) scenario:
binary_tensor = np.zeros(dense_tensor.shape)
for i1 in range(dense_tensor.shape[0]):
    for i2 in range(dense_tensor.shape[2]):
        binary_tensor[i1, :, i2] = np.round(random_matrix[i1, i2] + 0.5 - missing_rate)
sparse_tensor = np.multiply(dense_tensor, binary_tensor)

del data, random_matrix, binary_tensor
import time
start = time.time()
time_lags = np.array([1, 2, 3, 4, 5, 6, 286, 287, 288, 289, 290, 291])
rho = 1e-4
lambda0 = 0 * rho
epsilon = 1e-3
maxiter = 100
tensor_hat = imputer(dense_tensor, sparse_tensor, time_lags, rho, lambda0, epsilon, maxiter)
end = time.time()
print('Running time: %.2f minutes'%((end - start)/60.0))
import numpy as np
import pandas as pd
np.random.seed(1000)

data = pd.read_csv('../datasets/California-data-set/pems-4w.csv', header = None)
dense_tensor = mat2ten(data.values, np.array([data.values.shape[0], 288, 4 * 7]), 0)
random_matrix = np.random.rand(data.values.shape[0], 4 * 7)

missing_rate = 0.7

### Non-random missing (NM) scenario:
binary_tensor = np.zeros(dense_tensor.shape)
for i1 in range(dense_tensor.shape[0]):
    for i2 in range(dense_tensor.shape[2]):
        binary_tensor[i1, :, i2] = np.round(random_matrix[i1, i2] + 0.5 - missing_rate)
sparse_tensor = np.multiply(dense_tensor, binary_tensor)

del data, random_matrix, binary_tensor
import time
start = time.time()
time_lags = np.array([1, 2, 3, 4, 5, 6, 286, 287, 288, 289, 290, 291])
rho = 1e-4
lambda0 = 0 * rho
epsilon = 1e-3
maxiter = 100
tensor_hat = imputer(dense_tensor, sparse_tensor, time_lags, rho, lambda0, epsilon, maxiter)
end = time.time()
print('Running time: %.2f minutes'%((end - start)/60.0))<jupyter_output>Iter: 1
Tolerance: 1.49254
MAPE: 0.111823
RMSE: 7.70025

Iter: 2
Tolerance: 0.0662397
MAPE: 0.0975341
RMSE: 6.70065

Iter: 3
Tolerance: 0.067056
MAPE: 0.083139
RMSE: 5.96412

Iter: 4
Tolerance: 0.0482453
MAPE: 0.0746097
RMSE: 5.61509

Iter: 5
Tolerance: 0.0308453
MAPE: 0.0698433
RMSE: 5.43514

Iter: 6
Tolerance: 0.0267958
MAPE: 0.0666876
RMSE: 5.30748

Iter: 7
Tolerance: 0.0236707
MAPE: 0.0646733
RMSE: 5.21599

Iter: 8
Tolerance: 0.0202557
MAPE: 0.0637341
RMSE: 5.15437

Iter: 9
Tolerance: 0.0185578
MAPE: 0.0638182
RMSE: 5.12322

Iter: 10
Tolerance: 0.0178718
MAPE: 0.064626
RMSE: 5.12181

Iter: 11
Tolerance: 0.0349142
MAPE: 0.0644829
RMSE: 5.04823

Iter: 12
Tolerance: 0.0164566
MAPE: 0.0652368
RMSE: 5.06785

Iter: 13
Tolerance: 0.0140252
MAPE: 0.0658471
RMSE: 5.08618

Iter: 14
Tolerance: 0.0123171
MAPE: 0.066265
RMSE: 5.09894

Iter: 15
Tolerance: 0.0107694
MAPE: 0.066522
RMSE: 5.10754

Iter: 16
Tolerance: 0.00976923
MAPE: 0.0666493
RMSE: 5.11219

Iter: 17
Tolerance: 0.0088423
MAPE: 0.06[...]<jupyter_text>California data - 8WWe generate **random missing (RM)** values on California traffic speed data set.<jupyter_code>import numpy as np
import pandas as pd
np.random.seed(1000)

data = pd.read_csv('../datasets/California-data-set/pems-8w.csv', header = None)
dense_tensor = mat2ten(data.values, np.array([data.values.shape[0], 288, 8 * 7]), 0)
random_tensor = np.random.rand(data.values.shape[0], 288, 8 * 7)

missing_rate = 0.3

### Random missing (RM) scenario:
binary_tensor = np.round(random_tensor + 0.5 - missing_rate)
sparse_tensor = np.multiply(dense_tensor, binary_tensor)

del data, random_tensor, binary_tensor<jupyter_output><empty_output><jupyter_text>We use `imputer` to fill in the missing entries and measure performance metrics on the ground truth.<jupyter_code>import time
start = time.time()
time_lags = np.array([1, 2, 3, 4, 5, 6, 286, 287, 288, 289, 290, 291])
rho = 1e-4
lambda0 = 0 * rho
epsilon = 1e-3
maxiter = 100
tensor_hat = imputer(dense_tensor, sparse_tensor, time_lags, rho, lambda0, epsilon, maxiter)
end = time.time()
print('Running time: %.2f minutes'%((end - start)/60.0))
import numpy as np
import pandas as pd
np.random.seed(1000)

data = pd.read_csv('../datasets/California-data-set/pems-8w.csv', header = None)
dense_tensor = mat2ten(data.values, np.array([data.values.shape[0], 288, 8 * 7]), 0)
random_tensor = np.random.rand(data.values.shape[0], 288, 8 * 7)

missing_rate = 0.7

### Random missing (RM) scenario:
binary_tensor = np.round(random_tensor + 0.5 - missing_rate)
sparse_tensor = np.multiply(dense_tensor, binary_tensor)

del data, random_tensor, binary_tensor
import time
start = time.time()
time_lags = np.array([1, 2, 3, 4, 5, 6, 286, 287, 288, 289, 290, 291])
rho = 1e-4
lambda0 = 0 * rho
epsilon = 1e-3
maxiter = 100
tensor_hat = imputer(dense_tensor, sparse_tensor, time_lags, rho, lambda0, epsilon, maxiter)
end = time.time()
print('Running time: %.2f minutes'%((end - start)/60.0))<jupyter_output>Iter: 1
Tolerance: 1.50177
MAPE: 0.114411
RMSE: 7.84667

Iter: 2
Tolerance: 0.0700399
MAPE: 0.0927179
RMSE: 6.39308

Iter: 3
Tolerance: 0.0751557
MAPE: 0.0707443
RMSE: 5.35096

Iter: 4
Tolerance: 0.0614989
MAPE: 0.0619116
RMSE: 5.10319

Iter: 5
Tolerance: 0.0413745
MAPE: 0.0615923
RMSE: 5.12944

Iter: 6
Tolerance: 0.0296792
MAPE: 0.059783
RMSE: 4.97614

Iter: 7
Tolerance: 0.0309106
MAPE: 0.0550474
RMSE: 4.64833

Iter: 8
Tolerance: 0.0332183
MAPE: 0.0500046
RMSE: 4.31709

Iter: 9
Tolerance: 0.0319974
MAPE: 0.0469406
RMSE: 4.07036

Iter: 10
Tolerance: 0.0283865
MAPE: 0.0456104
RMSE: 3.90896

Iter: 11
Tolerance: 0.0563782
MAPE: 0.0468761
RMSE: 3.9547

Iter: 12
Tolerance: 0.0251123
MAPE: 0.0476402
RMSE: 4.03345

Iter: 13
Tolerance: 0.0207673
MAPE: 0.0446923
RMSE: 3.83382

Iter: 14
Tolerance: 0.0257518
MAPE: 0.0401447
RMSE: 3.50902

Iter: 15
Tolerance: 0.0268865
MAPE: 0.0364423
RMSE: 3.24904

Iter: 16
Tolerance: 0.0236496
MAPE: 0.0345573
RMSE: 3.10821

Iter: 17
Tolerance: 0.0193111
MAPE: 0.[...]<jupyter_text>We generate **non-random missing (NM)** values on California traffic speed data set. Then, we conduct the imputation experiment.<jupyter_code>import numpy as np
import pandas as pd
np.random.seed(1000)

data = pd.read_csv('../datasets/California-data-set/pems-8w.csv', header = None)
dense_tensor = mat2ten(data.values, np.array([data.values.shape[0], 288, 8 * 7]), 0)
random_matrix = np.random.rand(data.values.shape[0], 8 * 7)

missing_rate = 0.3

### Non-random missing (NM) scenario:
binary_tensor = np.zeros(dense_tensor.shape)
for i1 in range(dense_tensor.shape[0]):
    for i2 in range(dense_tensor.shape[2]):
        binary_tensor[i1, :, i2] = np.round(random_matrix[i1, i2] + 0.5 - missing_rate)
sparse_tensor = np.multiply(dense_tensor, binary_tensor)

del data, random_matrix, binary_tensor
import time
start = time.time()
time_lags = np.array([1, 2, 3, 4, 5, 6, 286, 287, 288, 289, 290, 291])
rho = 1e-4
lambda0 = 0 * rho
epsilon = 1e-3
maxiter = 100
tensor_hat = imputer(dense_tensor, sparse_tensor, time_lags, rho, lambda0, epsilon, maxiter)
end = time.time()
print('Running time: %.2f minutes'%((end - start)/60.0))
import numpy as np
import pandas as pd
np.random.seed(1000)

data = pd.read_csv('../datasets/California-data-set/pems-8w.csv', header = None)
dense_tensor = mat2ten(data.values, np.array([data.values.shape[0], 288, 8 * 7]), 0)
random_matrix = np.random.rand(data.values.shape[0], 8 * 7)

missing_rate = 0.7

### Non-random missing (NM) scenario:
binary_tensor = np.zeros(dense_tensor.shape)
for i1 in range(dense_tensor.shape[0]):
    for i2 in range(dense_tensor.shape[2]):
        binary_tensor[i1, :, i2] = np.round(random_matrix[i1, i2] + 0.5 - missing_rate)
sparse_tensor = np.multiply(dense_tensor, binary_tensor)

del data, random_matrix, binary_tensor
import time
start = time.time()
time_lags = np.array([1, 2, 3, 4, 5, 6, 286, 287, 288, 289, 290, 291])
rho = 1e-4
lambda0 = 0 * rho
epsilon = 1e-3
maxiter = 100
tensor_hat = imputer(dense_tensor, sparse_tensor, time_lags, rho, lambda0, epsilon, maxiter)
end = time.time()
print('Running time: %.2f minutes'%((end - start)/60.0))<jupyter_output>Iter: 1
Tolerance: 1.50012
MAPE: 0.114658
RMSE: 7.85768

Iter: 2
Tolerance: 0.0730121
MAPE: 0.0942427
RMSE: 6.50734

Iter: 3
Tolerance: 0.071667
MAPE: 0.0755545
RMSE: 5.58775

Iter: 4
Tolerance: 0.0520256
MAPE: 0.0660667
RMSE: 5.23676

Iter: 5
Tolerance: 0.0354722
MAPE: 0.0628432
RMSE: 5.14213

Iter: 6
Tolerance: 0.0269865
MAPE: 0.0607456
RMSE: 5.06694

Iter: 7
Tolerance: 0.0237192
MAPE: 0.0587474
RMSE: 4.97434

Iter: 8
Tolerance: 0.0230834
MAPE: 0.057515
RMSE: 4.89135

Iter: 9
Tolerance: 0.02257
MAPE: 0.0576788
RMSE: 4.84885

Iter: 10
Tolerance: 0.0201066
MAPE: 0.0589919
RMSE: 4.85231

Iter: 11
Tolerance: 0.0368248
MAPE: 0.0593495
RMSE: 4.79108

Iter: 12
Tolerance: 0.0177441
MAPE: 0.0604448
RMSE: 4.82626

Iter: 13
Tolerance: 0.0147386
MAPE: 0.0610166
RMSE: 4.84504

Iter: 14
Tolerance: 0.0131677
MAPE: 0.06122
RMSE: 4.84994

Iter: 15
Tolerance: 0.0119143
MAPE: 0.0612398
RMSE: 4.84985

Iter: 16
Tolerance: 0.0106928
MAPE: 0.061173
RMSE: 4.84836

Iter: 17
Tolerance: 0.00953009
MAPE: 0.0610[...]
{'fraction_non_alphanumeric': 0.14032018659881254, 'fraction_numerical': 0.114715860899067, 'mean_word_length': 3.917883211678832, 'pattern_counts': {'":': 0, '<': 37, '<?xml version=': 0, '>': 43, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '560637', 'license': 'MIT', 'n_tokens_mistral': 9048, 'n_tokens_neox': 7416, 'n_words': 1969, 'path': 'mats/LATC-Tubal-imputer-case1.ipynb', 'repo_name': 'vishalbelsare/tensor-learning', 'size': '18876'}