
We present a method for producing unbiased parameter
estimates and valid confidence intervals under the constraints of
differential privacy, a formal framework for limiting individual information leakage from sensitive data.
Prior work in this area is limited in that it is
tailored to calculating confidence intervals
for specific statistical procedures, such as mean estimation or simple linear regression.
While other recent work can produce confidence intervals for more general sets of procedures, they
either yield only approximately unbiased estimates,
are designed for one-dimensional outputs, or assume significant user knowledge about the data-generating
distribution.
Our method induces distributions of mean and covariance estimates via the bag of little bootstraps (BLB) [1]} and uses
them to privately estimate the parameters' sampling distribution
via a generalized version of the CoinPress estimation algorithm [2]}.
If the user can bound the parameters of the BLB-induced parameters and provide heavier-tailed families,
the algorithm produces unbiased parameter estimates and valid confidence intervals
which hold with arbitrarily high probability. These results hold in high dimensions and for any estimation procedure which behaves
nicely under the bootstrap.
