Determining the number of latent dimensions is a ubiquitous problem in machine
 learning. In this study, we introduce a novel method that relies on SVD to discover
 the number of latent dimensions. The general principle behind the method is to
 compare the curve of singular values of the SVD decomposition of a data set with
 the randomized data set curve. The inferred number of latent dimensions corresponds
 to the crossing point of the two curves. To evaluate our methodology, we
 compare it with competing methods such as Kaisers eigenvalue-greater-than-one
 rule (K1), Parallel Analysis (PA), Velicers MAP test (Minimum Average Partial).
 We also compare our method with the Silhouette Width (SW) technique which is
 used in different clustering methods to determine the optimal number of clusters.
 The result on synthetic data shows that the Parallel Analysis and our method have
 similar results and more accurate than the other methods, and that our methods is
 slightly better result than the Parallel Analysis method for the sparse data sets. The problem of determining the number of latent dimensions, or latent factors, is ubiquitous in a number of non supervised learning approaches. Matrix factorization techniques are good examples where we need to determine the number of latent dimensions prior to the learning phase. Non linear models such as LDA BID1 and neural networks also face the issue of stating the number of topics and nodes to include in the model before running an analysis over a data set, a problem that is akin to finding the number of latent factors.We propose a new method to estimate the number of latent dimensions that relies on the Singular Value Decomposition (SVD) and on a process of comparison of the singular values from the original matrix data with those from from bootstraped samples of the this matrix, whilst the name given to this method, Bootstrap SVD (BSVD). We compare the method to mainstream latent dimensions estimate techniques and over a space of dense vs. sparse matrices for Normal and non-Normal distributions.This paper is organized as follow. First, we outline some of best known methods and the related works in the next section. Then we explain our algorithm BSVD in section 3. The experiments are presented in section 4 and the results and discussion are reported in section 5. And finally conclusion of the study is given in section 6. According to the results of provided experiments in the tables 1 and 2, we could show that our method has a better performance than those mentioned especially in the sparse data sets. Our empirical experiments demonstrate that on the dense data sets; the accuracy of BSVD and PA is equal and better than the other approaches. But when we apply a different percentage of sparseness to our data sets, our method is more precise.In the figures 3 and 4, we display the behavior of each method in the dense and sparse data sets. Figure 3 depicts the average accuracy of all methods in the dense data sets with normal and nonnormal distribution. It shows that MAP method in the dense data set with normal or non-normal distribution has the same accuracy. Additionally, SW technique performs better result with the face of the dense data set with non-normal distribution, while K1 has an extreme behavior in the nonnormal data set. Moreover, BSVD, PA and K1 are more precise in the dense data set with normal distribution. Figure 4 shows the sparse data sets with normal and non-normal distribution. It demonstrates that BSVD, PA, and K1 have better accuracy in the sparse data set with normal distribution but MAP and SW are on the contrary. Figure 5 shows the average accuracy of all the methods in in different level of sparsity over the non normal sparse data set with latent dimensions (j) equal to 2. The error bars shows the variance of the observations after repeating the algorithm 25 times. Based on the results of these experiments we can conclude that our approach (BSVD) is better than the presented methods especially in the sparse data sets. To show if the outcome is statistically significant and is not by chance, we apply t-test between our method and PA. We considered the p values less than or equal to 0.05 as a significant result. To do so, we consider a sample of latent dimensions (j = {2, 3, 5, 8, 15}) and we repeat twenty-five times the mentioned experiments on the sparse data sets with normal and non-normal distribution, and record the result. Then we apply t-test between BSVD and PA. In this evaluation the null hypothesis (H0) state that µ SV D = µ P A and if the H0 is rejected, we could conclude that the obtained results are not by chance and our method is better than PA. TAB1 contain p values of the sparse and dense data sets with normal and non-normal distribution respectively. The first row of each table with 0% of sparsity indicate to the dense data sets. TAB1 shows more constant behavior, and implies that by increasing sparsity in the sparse data set with normal distribution, BSVD yeilds a significantly better result. But table 4 that shows the result of non-normal sparse data set is hard to interpret. Because the green cells are not affected by increasing sparsity. We can sum up with that the result seems to be significant with increasing the sparsity. In general, according to the tables 3 and 4, the difference between our method and PA seems to be statistically significant by increasing the percentage of sparsity. The objective of our study was to introduce a new method to find the number of latent dimensions using SVD which we inspired from PA. We employ our method on simulated data sets with normal and non-normal distribution whereas are dense or sparse and compared with the present methods such as PA, MAP, K1, and SW. According to the mentioned experiments and the reported results in the table 1, BSVD and PA have the same accuracy and better than the other presented methods in the dense data sets. But our method has a better result in the sparse data sets which is shown in the table 2. We applied t-test on the sample of latent dimensions (j) between BSVD and PA to demonstrate if the result is statistically significant or not. The results in the tables (3 and 4) demonstrate that in the sparse data sets with increasing the sparsity, our method seems to be significantly better than the other methods. Our method performance is limited to the presented experiments and data sets. If we want to generalize the method, We need to see the behavior of the algorithm when we have a more complex data set.step a. Generating the matrices x and y with the sizes of 6 × 3 and 5 × 3.
In this study, we introduce a novel method that relies on SVD to discover the number of latent dimensions.
{"src": "scitldr-AIC_train", "paper_id": "SkwAEQbAb"}