Why does a non square matrix lack a multiplicative identity Precisely, why is multiplicative identity defined to be $IA=AI=A$ why both sides should work why not something like $AI=A$ ? Is there an underlying advantage?

The use of category theory may make this clearer. Suppose we have a category of objects
$\, V_1, V_2, V_3, \dots\,$ An $\,n \times m\,$ matrix $\,A\,$ is an arrow from
$\,V_m\,$ to $\, V_n.\,$ Matrix multiplication is only defined between compatible matrices.
That is, If $\,B\,$ is an arrow from $\,V_n\,$ to $\,V_k\,$ then the matrix product
$\,B A\,$ is an arrow from $\,V_m\,$ to $\,V_k.\,$ Each object $\,V_n$ has a unique
identity arrow denoted by $\,I_n\,$ which is the $\,n \times n\,$ identity matrix. This
gives us the identity $\, A = AI_m = I_nA.\,$ The two identity arrows are not the
same unless $\,n=m.\,$ 
