Can such an "orthogonal" matrix exist? I know that the definition of an orthogonal matrix is that $A \in \mathbb R^{n \times n}$ is orthogonal if $AA^T = A^T A=I$, no problem with that whatsoever.
My question is this - Why only square matrices?
Is it possible that there is a matrix $A \in \mathbb R^{n \times k}$ such that $AA^T =I_n$ and $A^TA= I_k$?
if so - could you find an example? and if not - why not?
This is not for homework, this is merely my curiosity

You've already answered your question correctly with "no". However, the concept of orthogonal matrices can be somewhat generalized to non-square matrices. An $n \times k$ matrix $A$ is called semi-orthogonal if $A^TA$ or $AA^T$ is the identity matrix.
Another characterization of this property is that $A$ has no singular values other than $0$ or $1$.
