We consider a basic problem in unsupervised learning: learning an unknown . A Poisson Binomial Distribution (PBD) over @math is the distribution of a sum of @math independent Bernoulli random variables which may have arbitrary, potentially non-equal, expectations. These distributions were first studied by S. Poisson in 1837 Poisson:37 and are a natural @math -parameter generalization of the familiar Binomial Distribution. Surprisingly, prior to our work this basic learning problem was poorly understood, and known results for it were far from optimal. We essentially settle the complexity of the learning problem for this basic class of distributions. As our first main result we give a highly efficient algorithm which learns to @math -accuracy (with respect to the total variation distance) using @math samples . The running time of the algorithm is in the size of its input data, i.e., @math bit-operations. (Observe that each draw from the distribution is a @math -bit string.) Our second main result is a proper learning algorithm that learns to @math -accuracy using @math samples, and runs in time @math . This is nearly optimal, since any algorithm for this problem must use @math samples. We also give positive and negative results for some extensions of this learning problem to weighted sums of independent Bernoulli random variables.
On considere la classe de toutes les densites unimodales definies sur un intervalle de longueur L et borne par H; on etudie le risque minimax sur cette classe, lorsqu'on estime en utilisant n observations i.i.d., la perte etant mesuree par la distance L 1 entre l'estimateur et la densite reelle The Grenander estimator of a decreasing density, which is defined as the derivative of the concave envelope of the empirical c.d.f., is known to be a very good estimator of an unknown decreasing density on the half-line R + when this density is not assumed to be smooth. It is indeed the maximum likelihood estimator and one can get precise upper bounds for its risk when the loss is measured by the L 1 -distance between densities. Moreover, if one restricts oneself to the compact subsets of decreasing densities bounded by H with support on [0, L] the risk of this estimator is within a fixed factor of the minimax risk. The same is true if one deals with the maximum likelihood estimator for unimodal densities with known mode. When the mode is unknown, the maximum likelihood estimator does not exist any more. We shall provide a general purpose estimator (together with a computational algorithm) for estimating nonsmooth unimodal densities. Its risk is the same as the risk of the Grenander estimator based on the knowledge of the true mode plus some lower order term. It can also cope with small departures from unimodality. Abstract In a classical theorem, Ibragimov demonstrated the strong unimodality of log-concave probability density functions. Comparable results for lattice distributions are exhibited and their potential significance is suggested.