Consider the following estimation problem: there are @math entities, each with an unknown parameter @math , and we observe @math independent random variables, @math , with @math Binomial @math . How accurately can one recover the "histogram" (i.e. cumulative density function) of the @math 's? While the empirical estimates would recover the histogram to earth mover distance @math (equivalently, @math distance between the CDFs), we show that, provided @math is sufficiently large, we can achieve error @math which is information theoretically optimal. We also extend our results to the multi-dimensional parameter case, capturing settings where each member of the population has multiple associated parameters. Beyond the theoretical results, we demonstrate that the recovery algorithm performs well in practice on a variety of datasets, providing illuminating insights into several domains, including politics, sports analytics, and variation in the gender ratio of offspring.
Accurate estimations of the frequency distribution of rare variants are needed to quantify the discovery power and guide large-scale human sequencing projects. This study describes an algorithm called UnseenEst to estimate the distribution of genetic variations using tens of thousands of exomes.