<commit_before>---
layout: post
title: "Penalized likelihood Estimator and Shrinkage Priors (2): Hodge's Estimator and Super-efficiency"
comments: true
---

   Previously, I talked about sparse and shrinkage estimator based on penalized likelihood methods ( ["Penalized likelihood Estimator and Shrinkage Priors (1)"]({% post_url 2016-07-02-120519%}) ). Today I am going to discuss the legendary estimator so-called "Hodge's estimator" and its theoretical properties.

A legendary statistician <NAME> naively believed that maximum likelihood estimators (MLE) would be the most efficient in a sense that the variance of the MLE is always less or equal to that of any estimator, at least in an asymptotic sense. However, Hodges' famous example of [super-efficiency](http://www.stat.yale.edu/~pollard/Books/LeCamFest/VanderVaart.pdf) not only broke down Fisher's naive belief, but also brought a huge shock to statistical society, since Hodge showed that the so-called Hodge's estimator achieves less asymptotic variance than that of MLE when the true parameter is exactly same with the specified test value, and it is as efficient as MLE otherwise.

The example that he provided was a simple one as follows:

$$
y_i \sim N(\theta, 1),
$$  

where $$\theta$$is an unknown parameter for $$i = 1,\dots,n$$. It is well-known that the MLE is the sample  mean $$\bar y$$ and $$n^{1/2}(\bar y - \theta)$$ converges to $$N(0,1)$$ in distribution, as $$n$$ goes to infinity. Then, Hodge's estimator $$\widehat \theta^H_n$$ can be defined as a hard-thresholded estimator based on the MLE with the threshold $$n^{-1/4}$$; *i.e.*,


 
<commit_msg>aljkdslkajsd<commit_after>---
layout: post
title: "Penalized likelihood Estimator and Shrinkage Priors (2): Hodge's Estimator and Super-efficiency"
comments: true
---

   Previously, I talked about sparse and shrinkage estimator based on penalized likelihood methods ( ["Penalized likelihood Estimator and Shrinkage Priors (1)"]({% post_url 2016-07-02-120519%}) ). Today I am going to discuss the legendary estimator so-called "Hodge's estimator" and its theoretical properties.

A legendary statistician <NAME> naively believed that maximum likelihood estimators (MLE) would be the most efficient in a sense that the variance of the MLE is always less or equal to that of any estimator, at least in an asymptotic sense. However, Hodges' famous example of [super-efficiency](http://www.stat.yale.edu/~pollard/Books/LeCamFest/VanderVaart.pdf) not only broke down Fisher's naive belief, but also brought a huge shock to statistical society, since Hodge showed that the so-called Hodge's estimator achieves less asymptotic variance than that of MLE when the true parameter is exactly same with the specified test value, and it is as efficient as MLE otherwise.

The example that he provided was a simple one as follows:

$$
y_i \sim N(\theta, 1),
$$  


 

{'fraction_non_alphanumeric': 0.0841995841995842, 'fraction_numerical': 0.014206514206514207, 'mean_word_length': 4.671905697445973, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '225493', 'n_tokens_mistral': 851, 'n_tokens_neox': 755, 'n_words': 393}