Fixed point in plane transformation. Some one give me a idea to solve this one.
It's a problem from Vladimir Zorich mathematical analysis I.
Problem 9.c from 1.3.5: 
A point $p \in X$ is a fixed point of a mapping $f:X \to X$ if $f(p)=p$. Verify that any composition of a shift, a rotation, and a similarity transformation of the plane has a fixed point, provided the coefficient of the similarity transformation is less than one.

Shift and rotation are just special cases of similarity transformations. A generic similarity can be written e.g. in the following form:
$$
\begin{pmatrix}x\\y\end{pmatrix}\mapsto
\begin{pmatrix}a&-b\\b&a\end{pmatrix}\cdot
\begin{pmatrix}x\\y\end{pmatrix}+
\begin{pmatrix}c\\d\end{pmatrix}
$$
With this you can solve the fixed point equation:
$$
\begin{pmatrix}x\\y\end{pmatrix}=
\begin{pmatrix}a&-b\\b&a\end{pmatrix}\cdot
\begin{pmatrix}x\\y\end{pmatrix}+
\begin{pmatrix}c\\d\end{pmatrix}
\\
\begin{pmatrix}a-1&-b\\b&a-1\end{pmatrix}\cdot
\begin{pmatrix}x\\y\end{pmatrix}=
\begin{pmatrix}-c\\-d\end{pmatrix}
$$
This linear system of equations has a unique solution if and only if the determinant of the matrix is nonzero, i.e. if
$$\begin{vmatrix}a-1&-b\\b&a-1\end{vmatrix}=
(a-1)^2+b^2\neq0$$
Now $a$ and $b$ are real numbers, and so is $a-1$. The square of a real number is zero only if the number itself is zero, otherwise it is positive. So the sum of two squares is zero only if both the numbers which are squared are zero. So the only situation where there is no single and unique fixed point is $a=1,b=0$. In this case, the linear part of the transformation is the identity, so the whole transformation is either a pure translation or, if $c=d=0$, the identity transformation. A translation has no fixed points, and under the identity, every point is fixed.
Both cases are ruled out by your statement about the coefficient of the similarity transformation. By the way, you can read that coefficient as the square root of the determinant of the linear matrix for the original transformation, i.e.
$$\begin{vmatrix}a&-b\\b&a\end{vmatrix}=a^2+b^2\in(0,1)$$
