Example of Sufficient and Insufficient Statistic? I am having trouble understanding the concept of a sufficient statistic.  I have read What is a sufficient statistic? and Sufficient Statistic (Wikipedia) 

Can someone please give an example of:


a simple (but non-trivial) statistical model
a sufficient statistic of that model
an insufficient statistic of that model
how you identified 2 &amp; 3 as having and lacking, respectively, the sufficiency property
$\def\E{\mathrm{E}}$Consider samples $X = (X_1,X_2)$ from a normally distributed population $N(\mu,1)$ with unknown mean.

Then the statistic $T(X)=X_1$ is an unbiased estimator of the mean, since $\E(X_1)=\mu$. However, it is not a sufficient statistic - there is additional information in the sample that we could use to determine the mean.

How can we tell that $T$ is insufficient for $\mu$? By going to the definition. We know that $T$ is sufficient for a parameter iff, given the value of the statistic, the probability of a given value of $X$ is independent of the parameter, i.e. if

$$P(X=x|T=t,\mu)=P(X=x|T=t)$$

But we can compute this:

$$P(X=(x_1,x_2) | X_1=t,\mu) =
\begin{cases}
0 &amp; \mbox{if }t\neq x_1 \\
\tfrac{1}{\sqrt{2\pi}}e^{-\frac{1}{2}(x_2-\mu)^2} &amp; \mbox{if }t=x_1
\end{cases}$$

which is certainly not independent of $\mu$.

On the other hand, consider $T'(X) = X_1+X_2$. Then we have

$$P(X=(x_1,x_2) | X_1+X_2=t, \mu) = \frac{1}{2\pi}\int_{-\infty}^{\infty}e^{-\frac{1}{2}(s-\mu)^2 - \frac{1}{2}(t-s-\mu)^2}ds$$ 

and you can complete the square and show that this is independent of $\mu$, and hence $T'$ is a sufficient statistic for the mean $\mu$.
Here is my attempt to answer part 4 of the question.  If you have a parameteric family of distributions with a parameter theta (theta can be K-dimensional for K>=1) then a statistic is sufficient if all the information about theta is contained in it.  That means that given the parametric family the conditional distribution of the data given the sufficient statistic is independent of the parameter theta.  For many problem there are several functions T of the data that are sufficient.  Sufficiency usually carries data reduction with it.  For example say the parametric family is N(m,1) (m is the mean and 1 is the variance).  To estimate m you take a sample of size n=10.  The data involves ten values but the sufficient statistic (the sample mean) is just a single number.  That is the least amount of information possible for a sufficient statistic.  This type of sufficient statistc is better than others in data reduction and is called minimal sufficient.  To determine that a candidate statistic is sufficient you can use the factorization theorem to test it.  If the candidate T is sufficent it will factor the density as follows f(x|theta)= g(T(x)|theta) h(x) where f is the density for the data given theta, g is the density for T(x) given theta and the term h(x) does not depend on theta.  So you can check sufficiency by calculating f and g, divide f by g and T will be sufficient iff the resulting function h doesn't depend on (involve) theta.  Besides sufficiency and minimal sufficiency there is a concept called complete sufficiency.  All the cocepts are important in the parametric theory of point estimation.  This is nicely covered in Casella and Berger's book Statistical Inference 2e Chapter 6.