Why only either of $A^{-1}A$ and $AA^{-1}$ is equal to identity matrix? Consider $6\times 4$ matrix $A$. 
Why either of $A^{-1}A$ and $AA^{-1}$ is equal to identity matrix, but the other one is not equal to identity matrix? 
Note: I try to do it numerically as follows. But I don’t understand why I get such a result. I see it’s mathematical explanation in order to convenience. 


Inverse is defined for square matrices and it is unique. For rectanglar matrices
either right or left inverse exists and it is also not unique.
For $A_{6 \times 4}$ if B is the inverse of $A$, then $$A_{6 \times 4} B_{4 \times 6}=I_{6 \times 6}~~~~~(1)$$ Here $B$ has 24 unknowns whereas there are 36 equations. This system of equations is over-determined so left inverse of $A$ cannot exist.
Next, let $C$ be the right inverse, then
$$C_{4\times 6} A_{6 \times 4} =I_{4\times 4}~~~~~~(2)$$
Here there are 24 unknown elements of $C$ and there are only 16 linear simultaneous equations. So there are many solutions and this makes the right inverse to exist but it will not be unique.
So, here in your example, Eq.(1) is not possible, but (2) is possible and there will be many matrices for $C$.
Also note that $A$ is horizonatal matrix, $B$ is vertical (but here it doesn't exist) $C$ is again vertical it exists but it is non uinque. So a horizontal matric will have a vertical matrix as its inverse which is either left or right.
