Recently, the method of b-bit minwise hashing has been applied to large-scale linear learning and sublinear time near-neighbor search. The major drawback of minwise hashing is the expensive preprocessing cost, as the method requires applying (e.g.,) k=200 to 500 permutations on the data. The testing time can also be expensive if a new data point (e.g., a new document or image) has not been processed, which might be a significant issue in user-facing applications. We develop a very simple solution based on one permutation hashing. Conceptually, given a massive binary data matrix, we permute the columns only once and divide the permuted columns evenly into k bins; and we simply store, for each data vector, the smallest nonzero location in each bin. The interesting probability analysis (which is validated by experiments) reveals that our one permutation scheme should perform very similarly to the original (k-permutation) minwise hashing. In fact, the one permutation scheme can be even slightly more accurate, due to the "sample-without-replacement" effect. Our experiments with training linear SVM and logistic regression on the webspam dataset demonstrate that this one permutation hashing scheme can achieve the same (or even slightly better) accuracies compared to the original k-permutation scheme. To test the robustness of our method, we also experiment with the small news20 dataset which is very sparse and has merely on average 500 nonzeros in each data vector. Interestingly, our one permutation scheme noticeably outperforms the k-permutation scheme when k is not too small on the news20 dataset. In summary, our method can achieve at least the same accuracy as the original k-permutation scheme, at merely 1 k of the original preprocessing cost.
One of the authors worked on another one permutation'' scheme named Conditional Random Sampling (CRS) @cite_7 @cite_13 since 2005. Basically, CRS works by continuously taking the first @math nonzeros after applying one permutation on the data, then it uses a simple trick'' to construct a random sample for each pair with the effective sample size determined at the estimation stage. By taking the nonzeros continuously, however, the samples are no longer aligned'' and hence we can not write the estimator as an inner product in a unified fashion. In comparison, our new one permutation scheme works by first breaking the columns evenly into @math bins and then taking the first nonzero in each bin, so that the hashed data can be nicely aligned.
@cite_13 @cite_7
2112036337 1998840390
Determining conditional independence (CI) relationships between random variables is a challenging but important task for problems such as Bayesian network learning and causal discovery. We propose a new kernel CI test that uses a single, learned permutation to convert the CI test problem into an easier two-sample test problem. The learned permutation leaves the joint distribution unchanged if and only if the null hypothesis of CI holds. Then, a kernel two-sample test, which has been studied extensively in prior work, can be applied to a permuted and an unpermuted sample to test for CI. We demonstrate that the test (1) easily allows the incorporation of prior knowledge during the permutation step, (2) has power competitive with state-of-the-art kernel CI tests, and (3) accurately estimates the null distribution of the test statistic, even as the dimensionality of the conditioning variable grows. The bin packing problem is to find the minimum number of bins of size one to pack a list of items with sizes a 1 ,, a n in (0,1]. Using uniform sampling, which selects a random element from the input list each time, we develop a randomized @math time (1+e )-approximation scheme for the bin packing problem. We show that every randomized algorithm with uniform random sampling needs @math time to give an (1+e )-approximation. For each function s (n ): N N , define (s (n )) to be the set of all bin packing problems with the sum of item sizes equal to s (n ). We show that (n b ) is NP-hard for every b (0,1]. This implies a dense sublinear time hierarchy of approximation schemes for a class of NP-hard problems, which are derived from the bin packing problem. We also show a randomized streaming approximation scheme for the bin packing problem such that it needs only constant updating time and constant space, and outputs an (1+e )-approximation in @math time. Let S ( )-bin packing be the class of bin packing problems with each input item of size at least  . This research also gives a natural example of NP-hard problem (S ( )-bin packing) that has a constant time approximation scheme, and a constant time and space sliding window streaming approximation scheme, where  is a positive constant.