We compute the singular values of an @math sparse matrix @math in a distributed setting, without communication dependence on @math , which is useful for very large @math . In particular, we give a simple nonadaptive sampling scheme where the singular values of @math are estimated within relative error with constant probability. Our proven bounds focus on the MapReduce framework, which has become the de facto tool for handling such large matrices that cannot be stored or even streamed through a single machine. On the way, we give a general method to compute @math . We preserve singular values of @math with @math relative error with shuffle size @math and reduce-key complexity @math . We further show that if only specific entries of @math are required and @math has nonnegative entries, then we can reduce the shuffle size to @math and reduce-key complexity to @math , where @math is the minimum cosine similarity for the entries being estimated. All of our bounds are independent of @math , the larger dimension. We provide open-source implementations in Spark and Scalding, along with experiments in an industrial setting.
In addition to computing entries of @math , our sampling scheme can be used to implement many similarity measures. We can use the scheme to efficiently compute four similarity measures: Cosine, Dice, Overlap, and the Jaccard similarity measures, with details and experiments given in @cite_2 @cite_4 , whereas this paper is more focused on matrix computations and an open-source implementation.
@cite_4 @cite_2
2163972006 2885205930 2137962371 2962865631
We present a suite of algorithms for Dimension Independent Similarity Computation (DISCO) to compute all pairwise similarities between very high-dimensional sparse vectors. All of our results are provably independent of dimension, meaning that apart from the initial cost of trivially reading in the data, all subsequent operations are independent of the dimension; thus the dimension can be very large. We study Cosine, Dice, Overlap, and the Jaccard similarity measures. For Jaccard similarity we include an improved version of MinHash. Our results are geared toward the MapReduce framework. We empirically validate our theorems with large scale experiments using data from the social networking site Twitter. At time of writing, our algorithms are live in production at twitter.com. Abstract Similarity networks contain important topological features and patterns critical to understanding interactions among samples in a large dataset. To create a comprehensive view of the interactions within a dataset, the Similarity Network Fusion (SNF) technique has been proposed to fuse the similarity networks based on different data types into one similarity network that represents the full spectrum of underlying data. In this paper, a modified version of SNF, which is named as Contextual Information based SNF (CI-SNF), is proposed. In CI-SNF, first, modified Jaccard distance is performed on the SNF fused similarity to utilize the contextual information contained in the fused similarity network. Second, the local consistency of samples from the same category is enhanced by speculating that the samples which are located high in the Jaccard distance based ranking list of a specific query are from the same category as the query. Third, the inverted index technique is introduced to utilize the sparsity property of the locally consistent similarity network to enhance the computational efficiency. To verify the effectiveness and efficiency of CI-SNF model, it is applied in four different tasks, Cover Song Identification (CSI), image classification, cancer subtype identification, and drug taxonomy, respectively. Extensive experiments on thirteen challenging datasets demonstrate that CI-SNF scheme outperforms state-of-the-art similarity fusion algorithms including SNF in all four tasks. It is also verified that utilizing the contextual information contained in the SNF-based similarity network helps to enhance the performance of the SNF-based scheme, further. In this paper we discuss a very simple approach of combining content and link information in graph structures for the purpose of community discovery, a fundamental task in network analysis. Our approach hinges on the basic intuition that many networks contain noise in the link structure and that content information can help strengthen the community signal. This enables ones to eliminate the impact of noise (false positives and false negatives), which is particularly prevalent in online social networks and Web-scale information networks. Specifically we introduce a measure of signal strength between two nodes in the network by fusing their link strength with content similarity. Link strength is estimated based on whether the link is likely (with high probability) to reside within a community. Content similarity is estimated through cosine similarity or Jaccard coefficient. We discuss a simple mechanism for fusing content and link similarity. We then present a biased edge sampling procedure which retains edges that are locally relevant for each graph node. The resulting backbone graph can be clustered using standard community discovery algorithms such as Metis and Markov clustering. Through extensive experiments on multiple real-world datasets (Flickr, Wikipedia and CiteSeer) with varying sizes and characteristics, we demonstrate the effectiveness and efficiency of our methods over state-of-the-art learning and mining approaches several of which also attempt to combine link and content analysis for the purposes of community discovery. Specifically we always find a qualitative benefit when combining content with link analysis. Additionally our biased graph sampling approach realizes a quantitative benefit in that it is typically several orders of magnitude faster than competing approaches. Abstract In this paper, we consider the problem of recovering a compactly supported multivariate function from a collection of pointwise samples of its Fourier transform taken nonuniformly. We do this by using the concept of weighted Fourier frames. A seminal result of Beurling shows that sampling points give rise to a classical Fourier frame provided they are relatively separated and of sufficient density. However, this result does not allow for arbitrary clustering of sampling points, as is often the case in practice. Whilst keeping the density condition sharp and dimension independent, our first result removes the separation condition and shows that density alone suffices. However, this result does not lead to estimates for the frame bounds. A known result of Grochenig provides explicit estimates, but only subject to a density condition that deteriorates linearly with dimension. In our second result we improve these bounds by reducing the dimension dependence. In particular, we provide explicit frame bounds which are dimensionless for functions having compact support contained in a sphere. Next, we demonstrate how our two main results give new insight into a reconstruction algorithm—based on the existing generalized sampling framework—that allows for stable and quasi-optimal reconstruction in any particular basis from a finite collection of samples. Finally, we construct sufficiently dense sampling schemes that are often used in practice—jittered, radial and spiral sampling schemes—and provide several examples illustrating the effectiveness of our approach when tested on these schemes.