problem in solving this problem from olympiad(use of invariant) Start with the set $\{3, 4, 12\}$. In each step you may choose two of the numbers $a$, $b$
and replace them by $0.6a − 0.8b$ and $0.8a + 0.6b$. Can you reach $\{4, 6, 12\}$ 
in finitely many steps:
Invariant here is that $a^2+b^2$ remains constant. Till here I am good
This part of the solution i didn't understand.
Since $a^2 +b^2 +c^2= 3^2 +4^2 +12^2 =13^2$ ,
the point $(a, b, c)$ lies on the sphere around $O$ with radius $13$. Because $4^2 +6^2 +12^2=
14^2$ , the goal lies on the sphere around $O$ with radius $14$. The goal cannot be reached.
please explain
Will it make any difference if z-points are different

Well, $c$ does not change, so since $a^2+b^2$ does not change and $c$ does not change either we conclude $a^2+b^2+c^2$ does not change. In $(3,4,12)$ We have $a^2+b^2+c^2=169=13^2$.
On the other hand in $(4,6,12)$ we have $a^2+b^2+c^2=196=14^2$. Since $196$ is different from $169$ we cannot reach the desired target.
You can understand the proof without talking about spheres, it is redundant, they where just using the fact that the points $(x,y,z)$ on the sphere are those in which $\sqrt{x^2+y^2+z^2}=13$
