Can we have a matrix whose elements are other matrices as well as other things similar to sets? Basically, what I am asking is this: Is a matrix just like an ordered sequence, of which the elements can be anything? It would seem silly to restrict a matrix down to limited uses such as only for numbers, but I get the impression that I cant multiply two matrices whose elements are the planets in our solar system. So we only allow numbers in matrices.
Does this also mean that I cannot have a matrix of matrices? I feel like I should be able to since if I wanted to multiply two matrices of matrices, then although tedious by hand it can be done since I can add/subtract and multiply normal matrices. Although I do not know many of the other operations so I do not know if everything else can be done with such constructs.

If you don't want to add or multiply matrices the entries of a matrix could be any mathematical object.  But there is not so much going on here...
Let's assume you want to be able to add and multiply square matrices.  You can certainly do so if all the elements of your matrix lie in some common structure $(X,+,\cdot)$, i.e., a set endowed with addition and multiplication operations.  If the $+$ in $X$ is commutative and associative then matrix addition will be so; if moreover the $\cdot$ in $X$ is associative and distributes over addition then the matrix multiplication will be so.
Especially, you can take matrices to have entries in any ring $R$, not necessarily commutative.  Such things come up naturally in the Wedderburn-Artin theory of semisimple rings.  In particular, you can take as your entries the ring of matrices with entries in some other (not necessarily commutative!) ring.  In this regard one of the basic facts is that for any ring $R$,  
$M_m(M_n(R)) \cong M_{mn}(R)$.  
