Why is the solution to $x-\sqrt 4=0$ not $x=\pm 2$? If the equation is $x-\sqrt 4=0$, then $x=2$.
If the equation is $x^2-4=0$, then $x=\pm 2$.
Why is it not $x=\pm 2$ in the first equation?

Your confusion is understandable!
You are thinking: if the square root of some number $x$ is that number $y$ such that $y^2 = x$, then why is the solution to $x = \sqrt{4}$ different from $x^2 - 4 = 0$ ?!?
And the answer is: it wouldn't be different! That is, if the square root of $x$ would indeed be defined as "that number $y$ such that $y^2 = x$", then $\sqrt{4}$ could be either 2 or -2.
But, obviously, that is not how we look at $\sqrt{4}$ since we all know that that is just 2. This means: thinking of the square root of $x$ as "that number $y$ such that $y^2 = x$" is apparently not how we think about the square root. 
OK, but why not? Well, notice that the whole "that number" part would be misleading in the first place: it suggest that there is one number with this property, but obviously there is not. So, if anything, we would have to say that the square root of $x$ is "any number such that $y^2 = x$"
... and we could have done so ...
... but we didn't. OK, but then we (you!) ask once again: why not?
Well, one simple reason is that we want the square root to act like a function, meaning that for any $x$, there is only one $y$ that is "the" square root of $x$.  And we want functions, because functions are super useful: one thing in, and one thing out! Calculations can be done, etc. etc. 
OK, but how can we ensure a function? Well, one thing we can do is to define the square root of $x$ as "that positive number $y$ such that $y^2 = x$" 
... and that's exactly what we did... 
and hence the difference between $x = \sqrt{4}$ and $x^2 - 4 = 0$ 
Of course, we could also have defined it as that negative $y$ such that $y^2 = x$" ... but in most practical cases, the positive one is the one you want, and the one that most often makes concrete sense in real life applications.
As a final reason for defining the square root of a number to be what it is, is that it allows us to explicitly distinguish between the two solutions to $x^2-2=0$, those being $\sqrt{2}$ and $-\sqrt{2}$: if the square root of 2 was any number that when squared gives you 2, then there no longer is a difference between $\sqrt{2}$ and $-\sqrt{2}$. In fact, it would not even be clear how many, and which specific, numbers would be solutions to $x^2-2=0$ if you said $x = \sqrt{2}$! Indeed, without the square root picking out a specific number, how would you refer to these different solutions? There probably is a way, but the square root function certainly makes our mathematical lives a lot easier for something like this!
