Consider the following estimation problem: there are @math entities, each with an unknown parameter @math , and we observe @math independent random variables, @math , with @math Binomial @math . How accurately can one recover the "histogram" (i.e. cumulative density function) of the @math 's? While the empirical estimates would recover the histogram to earth mover distance @math (equivalently, @math distance between the CDFs), we show that, provided @math is sufficiently large, we can achieve error @math which is information theoretically optimal. We also extend our results to the multi-dimensional parameter case, capturing settings where each member of the population has multiple associated parameters. Beyond the theoretical results, we demonstrate that the recovery algorithm performs well in practice on a variety of datasets, providing illuminating insights into several domains, including politics, sports analytics, and variation in the gender ratio of offspring.
The seminal paper of Charles Stein @cite_5 was one of the earliest papers to identify the surprising possibility of leveraging the availability of independent data reflecting a large number of parameters of interest, to partially compensate for having little information about each parameter. The specific setting examined considered the problem of estimating a list of unknown means, @math given access to @math independent Gaussian random variables, @math , with @math . Stein showed that, perhaps surprisingly, that there is an estimator for the list of parameters @math that has smaller expected squared error than the naive unbiased empirical estimates of @math . This improved estimator shrinks'' the empirical estimates towards the average of the @math 's. In our setting, the process of recovering the set histogram of unknown @math 's and then leveraging this recovered set as a prior to correct the empirical estimates of each @math can be viewed as an analog of Stein's shrinkage'', and will have the property that the empirical estimates are shifted (in a non-linear fashion) towards the average of the @math 's.
@cite_5
1598266570
Abstract : If one observes the real random variables Xi, Xn independently normally distributed with unknown means xi...x in and variance 1, it is customary to estimate xi by Xi. If the loss is the sum of squares of the errors, this estimator is admissible for n or equal to 2, but inadmissible for n more than or equal to 3. Since the usual estimator is best among those which transform correctly under translation, any admissible estimator for n equals more than or equal to 3 involves an arbitrary choice. While the results of this paper are not in a form suitable for immediate practical application, the possible improvement over the usual estimator seems to be large enough to be of practical importance if n is large.