What is orthogonal transformation? When $A^{T}A = AA^{T} = I$, I am told it is an orthogonal transformation $A$. But don't really get what it means. Hope to hear some explanations.
$\begin{bmatrix}\cos\theta & \sin\theta \\ -\sin\theta & \cos\theta\end{bmatrix} \begin{bmatrix}\cos\theta & -\sin\theta \\ \sin\theta & \cos\theta\end{bmatrix} = \begin{bmatrix}1 & 0 \\ 0 & 1\end{bmatrix}$

You can transform a vector into another vector by multiplying it by a matrix:
$$w=Av$$
Say, you had two vectors $v_1,v_2$, let's transform them into $w_1,w_2$ and obtain the inner product. Note that the inner product is the same as transposing then matrix multiplying:
$$w_1\cdot w_2\equiv w_1^Tw_2=v_1^TA^TAv_2$$
Now, if the matrix is orthogonal, you get:
$$w_1\cdot w_2=v_1^Tv_2\equiv v_1\cdot v_2$$
So, we see that the inner product is preserved when the transformation is orthogonal. Isn't this interesting? It means that if you have a geometrical object (which can be represented with a set of vectors) then orthogonal transformation $A$ will simply rotate or flip your object preserving its geometry (all angles, and sizes).
