Why the average of a set of value has the least square error? Now we have the equation
$$\sum_{i}(x_i-\hat x_i)^2,$$
where $x_i$ is the observed value of a data sample $S$. Here is the question:

Why does this expression get its minimum value when $\hat x_i$ is the average of the data sample $S$ ?

I tried to take the derivatives of that equation and make it to zero, but it seems there's something wrong, because $\hat x_i$ is kind of multi-variable. 
Can anyone help me out? Thanks a lot!

Let's take the function:
$$f(\hat x)=\sum_{i=1}^n (x_i-\hat x)^2$$
Here, we want to find the value of $\hat x$ which minimizes $f(\hat x)$. Now, even though there are multiple variables of this function because of $x_i$, we can just treat these variables as constants since they are independent from the $\hat x$, which essentially changes this to a single-variable calculus problem. Now, let's take the derivative of $f$ with respect to $\hat x$.
$$f'(\hat x)=\sum_{i=1}^n 2(\hat x-x_i)$$
From here, can you find the value of $\hat x$ satisfying $f(\hat x)=0$? Once you solve that equation, use second-derivative test to show that it is indeed an absolute minimum.
