Consider the following estimation problem: there are @math entities, each with an unknown parameter @math , and we observe @math independent random variables, @math , with @math Binomial @math . How accurately can one recover the "histogram" (i.e. cumulative density function) of the @math 's? While the empirical estimates would recover the histogram to earth mover distance @math (equivalently, @math distance between the CDFs), we show that, provided @math is sufficiently large, we can achieve error @math which is information theoretically optimal. We also extend our results to the multi-dimensional parameter case, capturing settings where each member of the population has multiple associated parameters. Beyond the theoretical results, we demonstrate that the recovery algorithm performs well in practice on a variety of datasets, providing illuminating insights into several domains, including politics, sports analytics, and variation in the gender ratio of offspring.
Building on the above work, the recent work @cite_9 considered the problem of recovering the frequency spectrum'' of rare genetic variants. This problem is similar to the problem we consider, but focuses on a rather different regime. Specifically, the model considered posits that each location @math in the genome has some probability @math of being mutated in a given individual. Given the sequences of @math individuals, the goal is to recover the set of @math 's. The work @cite_9 focused on the regime in which many of the @math 's are significantly less than @math , and hence correspond to mutations that have never been observed; one conclusion of that work was that one can accurately estimate the number of such rare mutations that would be discovered in larger sequencing cohorts. Our work, in contrast, focuses on the regime where the @math 's are constant, and do not scale as a function of @math , and the results are incomparable.
@cite_9
2545606300
Accurate estimations of the frequency distribution of rare variants are needed to quantify the discovery power and guide large-scale human sequencing projects. This study describes an algorithm called UnseenEst to estimate the distribution of genetic variations using tens of thousands of exomes.