In this paper, we consider the problem of actively learning a linear classifier through query synthesis where the learner can construct artificial queries in order to estimate the true decision boundaries. This problem has recently gained a lot of interest in automated science and adversarial reverse engineering for which only heuristic algorithms are known. In such applications, queries can be constructed de novo to elicit information (e.g., automated science) or to evade detection with minimal cost (e.g., adversarial reverse engineering). We develop a general framework, called dimension coupling (DC), that 1) reduces a d-dimensional learning problem to d-1 low dimensional sub-problems, 2) solves each sub-problem efficiently, 3) appropriately aggregates the results and outputs a linear classifier, and 4) provides a theoretical guarantee for all possible schemes of aggregation. The proposed method is proved resilient to noise. We show that the DC framework avoids the curse of dimensionality: its computational complexity scales linearly with the dimension. Moreover, we show that the query complexity of DC is near optimal (within a constant factor of the optimum algorithm). To further support our theoretical analysis, we compare the performance of DC with the existing work. We observe that DC consistently outperforms the prior arts in terms of query complexity while often running orders of magnitude faster.
For @math and when the distribution is uniform over the unit sphere @math it is very easy to see that the halving or bisection leads to @math . By using the same halving method, one can in principle extend the result to any dimension @math . To do so, we need to carefully construct the version space (i.e., the set of hypotheses consistent with the queries and outcomes) at each iteration and then find a query that halves the volume (in the uniform case) or the density (in the general case if the distribution is known) @cite_20 . Finding such a query in high dimension is very challenging.
@cite_20
1557097080 2059725015 2536849891 2963754066
We consider monotonicity testing of functions f:[n]d→ 0,1 , in the property testing framework of Rubinfeld and Sudan [23] and Goldreich, Goldwasser and Ron [14]. Specifically, we consider the framework of distribution-free property testing, where the distance between functions is measured with respect to a fixed but unknown distribution D on the domain and the testing algorithms have an oracle access to random sampling from the domain according to this distribution D. We show that, though in the uniform distribution case, testing of boolean functions defined over the boolean hypercube can be done using query complexity that is polynomial in @math and in the dimension d, in the distribution-free setting such testing requires a number of queries that is exponential in d. Therefore, in the high-dimensional case (in oppose to the low-dimensional case), the gap between the query complexity for the uniform and the distribution-free settings is exponential. also with probability 1. This result is proved in ?5. The difficulty in proving lower bounds like (1.2) is that one has to consider all possible coverings of the path by small convex sets in the definition of 4-measure. In the past the only successful method has been to use the connection between Hausdorff measures and generalized capacities. The first result of this kind appears in [16] where it is proved that for k >2, the Hausdorff measure with respect to ta is infinite for all a 3 we obtain a "law of the iterated logarithm" for the total time Tk(a, co) spent by the path X in a sphere of radius a as a-* 0+. We also determine the local behavior of the first passage time Pk(a, co) out of a sphere of radius a for k > 1. In order to obtain the required asymptotic results we required good estimates of the distribution functions for the random variables Tk(a, co), Pk(a, c). We use the method developed by Mark Kac to compute these dis- We study the recovery of sparse vectors from subsampled random convolutions via @math -minimization. We consider the setup in which both the subsampling locations as well as the generating vector are chosen at random. For a subgaussian generator with independent entries, we improve previously known estimates: if the sparsity @math is small enough, i.e., @math , we show that @math measurements are sufficient to recover @math -sparse vectors in dimension @math with high probability, matching the well-known condition for recovery from standard Gaussian measurements. If @math is larger, then essentially @math measurements are sufficient, again improving over previous estimates. Our results are shown via the so-called robust null space property which is weaker than the standard restricted isometry property. Our method of proof involves a novel combination of small ball estimates with chaining techniques which should be of independent interest. A standard way of approximating or discretizing a metric space is by taking its Rips complexes. These approximations for all parameters are often bound together into a filtration, to which we apply the fundamental group or the first homology. We call the resulting object persistence. Recent results demonstrate that persistence of a compact geodesic locally contractible space X carries a lot of geometric information. However, by definition the corresponding Rips complexes have uncountably many vertices. In this paper we show that nonetheless, the whole persistence of X may be obtained by an appropriate finite sample (subset of X), and that persistence of any subset of X is well interleaved with the persistence of X. It follows that the persistence of X is the minimum of persistences obtained by all finite samples. Furthermore, we prove a much improved Stability theorem for such approximations. As a special case we provide for each (r>0 ) a density (s>0 ), so that for each s-dense sample (S X ) the corresponding fundamental group (and the first homology) of the Rips complex of S is isomorphic to the one of X, leading to an improved reconstruction result.