The Shannon entropy is a widely used summary statistic, for example, network traffic measurement, anomaly detection, neural computations, spike trains, etc. This study focuses on estimating Shannon entropy of data streams. It is known that Shannon entropy can be approximated by Reenyi entropy or Tsallis entropy, which are both functions of the p-th frequency moments and approach Shannon entropy as p->1. Compressed Counting (CC) is a new method for approximating the p-th frequency moments of data streams. Our contributions include: 1) We prove that Renyi entropy is (much) better than Tsallis entropy for approximating Shannon entropy. 2) We propose the optimal quantile estimator for CC, which considerably improves the previous estimators. 3) Our experiments demonstrate that CC is indeed highly effective approximating the moments and entropies. We also demonstrate the crucial importance of utilizing the variance-bias trade-off.
Because the elements, @math , are time-varying, a na 'ive counting mechanism requires a system of @math counters to compute @math exactly (unless @math ). This is not always realistic. Estimating @math in data streams is heavily studied @cite_27 @cite_26 @cite_25 @cite_4 @cite_9 . We have mentioned that computing @math in strict-Turnstile model is trivial using a simple counter. One might naturally speculate that when @math , computing (approximating) @math should be also easy. However, before Compressed Counting (CC) , none of the prior algorithms could capture this intuition.
@cite_26 @cite_4 @cite_9 @cite_27 @cite_25
2951782995 1607840088 2137519983 2202286026
Propositional model counting (#SAT), i.e., counting the number of satisfying assignments of a propositional formula, is a problem of significant theoretical and practical interest. Due to the inherent complexity of the problem, approximate model counting, which counts the number of satisfying assignments to within given tolerance and confidence level, was proposed as a practical alternative to exact model counting. Yet, approximate model counting has been studied essentially only theoretically. The only reported implementation of approximate model counting, due to Karp and Luby, worked only for DNF formulas. A few existing tools for CNF formulas are bounding model counters; they can handle realistic problem sizes, but fall short of providing counts within given tolerance and confidence, and, thus, are not approximate model counters. We present here a novel algorithm, as well as a reference implementation, that is the first scalable approximate model counter for CNF formulas. The algorithm works by issuing a polynomial number of calls to a SAT solver. Our tool, ApproxMC, scales to formulas with tens of thousands of variables. Careful experimental comparisons show that ApproxMC reports, with high confidence, bounds that are close to the exact count, and also succeeds in reporting bounds with small tolerance and high confidence in cases that are too large for computing exact model counts. Model counting is the classical problem of computing the number of solutions of a given propositional formula. It vastly generalizes the NP-complete problem of propositional satisfiability, and hence is both highly useful and extremely expensive to solve in practice. We present a new approach to model counting that is based on adding a carefully chosen number of so-called streamlining constraints to the input formula in order to cut down the size of its solution space in a controlled manner. Each of the additional constraints is a randomly chosen XOR or parity constraint on the problem variables, represented either directly or in the standard CNF form. Inspired by a related yet quite different theoretical study of the properties of XOR constraints, we provide a formal proof that with high probability, the number of XOR constraints added in order to bring the formula to the boundary of being unsatisfiable determines with high precision its model count. Experimentally, we demonstrate that this approach can be used to obtain good bounds on the model counts for formulas that are far beyond the reach of exact counting methods. In fact, we obtain the first non-trivial solution counts for very hard, highly structured combinatorial problem instances. Note that unlike other counting techniques, such as Markov Chain Monte Carlo methods, we are able to provide high-confidence guarantees on the quality of the counts obtained. The paper presents a moving object extraction method robust against illumination level changes for a real-time pedestrian counting system based on an x-t space-time image. This system, called the Ped-Counter, counts the number of pedestrians using TV camera images and image processing. The TV camera takes images of a path or the entrance of a store or an exhibition hall. A practical location for the counting system is under constant illumination conditions, such as under an arcade or in a room. If the system is used in other locations and under different conditions, for example, an outdoor path or an entrance where there is direct sunlight, then it must be robust against illumination level changes. The paper proposes two new processes for extracting moving objects from space-time images using background subtraction. The first is a background image reconstruction process using statistical characteristic analysis of temporal changes of a target pixel. The other is a moving object extraction process using delayed subtraction. These processes make the system robust against illumination level changes. This method is evaluated over a half-year period at an entrance experiencing drastic illumination level changes. We also confirm that this method can extract moving objects, that it is robust against season, weather, and time, and that the Ped-Counter is accurate. The recent surge of interest in reasoning about probabilistic graphical models has led to the development of various techniques for probabilistic reasoning. Of these, techniques based on weighted model counting are particularly interesting since they can potentially leverage recent advances in unweighted model counting and in propositional satisfiability solving. In this paper, we present a new approach to weighted model counting via reduction to unweighted model counting. Our reduction, which is polynomial-time and preserves the normal form (CNF DNF) of the input formula, allows us to exploit advances in unweighted model counting to solve weighted model counting instances. Experiments with weighted model counters built using our reduction indicate that these counters performs much better than a state-of-the-art weighted model counter.