If $AB\not=BA,$ then do $A$ and $B$ have to be matrices Is this solvable? Or are there other things that fit the bill for $A$ and $B$?

In the study of abstract algebra, we do not limit ourselves to numbers. You have already noted that matrices act different than numbers, but mathematicians study general algebraic objects called groups. Invertible matrices and real numbers are groups, and @IvoTerek has noted another group called the quaternion group. But there are lots of others. A group is a set $G$ with an operation $\times$ so that if $a,b\in G$, $a\times b\in G$. There are a few rules for a group:


*

*The group operation is transitive: $a\times (b\times c)=(a\times b)\times c.$

*The group has an identity element $e$ so that $ae=ea=a$ for all $a\in G$. 

*Every $a\in G$ has an inverse $a^{-1}$ so that $a^{-1}a=aa^{-1}=e$.


Note that none of the group rules require that $ab=ba$, so lots of groups fit the description you give. In fact, groups that do have the property are rather rare (they are called Abelian groups). Several examples of non-Abelian groups you might look up are:


*

*Dihedral groups

*Permutation groups

*Quaternion groups.

