The sample mean is often used to aggregate different unbiased estimates of a parameter, producing a final estimate that is unbiased but possibly high-variance. This paper introduces the Bayesian median of means, an aggregation rule that roughly interpolates between the sample mean and median, resulting in estimates with much smaller variance at the expense of bias. While the procedure is non-parametric, its squared bias is asymptotically negligible relative to the variance, similar to maximum likelihood estimators. The Bayesian median of means is consistent, and concentration bounds for the estimator's bias and @math error are derived, as well as a fast non-randomized approximating algorithm. The performances of both the exact and the approximate procedures match that of the sample mean in low-variance settings, and exhibit much better results in high-variance scenarios. The empirical performances are examined in real and simulated data, and in applications such as importance sampling, cross-validation and bagging.
The idea of combining mean and median estimators has been visited several times in the statistical robustness literature, particularly for the estimation of location parameters in symmetric distributions. For instance, @cite_24 propose an adaptive estimator that picks either the sample mean or median to estimate the center of a symmetric distribution, while @cite_33 and @cite_5 investigate using linear combinations of mean and medians, with weights picked according to asymptotic criteria.
@cite_24 @cite_5 @cite_33
2021680704 2011147699 2064940455
Abstract An adaptive choice of the sample mean ¯xn or the sample median mn is proposed for estimating the center of a symmetric distribution. This choice becomes correct as n → ∞, and in simulation results for finite n it is almost as good as the better of ¯xn and mn. We propose a location estimator based on a convex linear combination of the sample mean and median. The main attraction is the conceptual simplicity and transparency, but it remains very competitive in performance for a wide range of distributions. The estimator aims at minimizing the asymptotic variance in the class of all linear combinations of mean and median. Comparisons with some of the best location estimators, the maximum likelihood, Huber's and the Hodges-Lehmann M-estimators, are given based on asymptotic relative efficiency and Monte Carlo simulations. Computationally, the new estimator has an explicit expression and requires no iteration. Robustness is assessed by calculation of breakdown point. We characterize all symmetric location models for which a linear combination of the median and the sample mean is an asymptotically efficient estimator of the location parameter. The resulting model can be understood as a symmetrized or double truncated normal distribution. A simple algorithm to estimate the parameters is given and an application is presented. Copyright 2004 Board of the Foundation of the Scandinavian Journal of Statistics..