Are matrices rank 2 tensors? I know that this is sometimes the case, but that some matrices are not tensors.
So what is the intuitive and specific demands of a matrix to also be a tensor?
Does it need to be quadratic, singular or something else?

Some sources I read seem to suggest that all rank 2 matrices are tensors while other just claims that "some" matrices are rank 2 tensors.

What's the connection between tensors and matrices?
This question doesn't have a single good answer, because there isn't a universally agreed upon definition of "tensor" in mathematics.  In particular:


Tensors are sometimes defined as multidimensional arrays, in the same way that a matrix is a two-dimensional array.  From this point of view, a matrix is certainly a special case of a tensor.
In differential geometry and physics, "tensor" refers to a certain kind of object that can be described at a point on a manifold (though the word "tensor" is often used to refer to a tensor field, in which one tensor is chosen for every point).  From this point of view, a matrix can be used to describe a rank-two tensor in local coordinates, but a rank-two tensor is not itself a matrix.
In linear algebra, "tensor" sometimes refers to an element of a tensor product, and sometimes refers to a certain kind of multilinear map.  Again, neither of these is a generalization of "matrix", though you can get a matrix from a rank-two tensor if you choose a basis for your vector space.


You run into the same problem if you ask a question like "Is a vector just a tuple of numbers?"  Sometimes a vector is defined as a tuple of numbers, in which case the answer is yes.  However, in differential geometry and physics, the word "vector" refers to an element of the tangent space to a manifold, while in linear algebra, a "vector" may be any element of a vector space.

On a basic level, the statement "a vector is a rank 1 tensor, and a matrix is a rank 2 tensor" is roughly correct.  This is certainly the simplest way of thinking about tensors, and is reflected in the Einstein notation.  However, it is important to appreciate the subtleties of this identification, and to realize that "tensor" often means something slightly different and more abstract than a multidimensional array.
All matrices are not tensors, although all tensors of rank 2 are matrices.

Example: Matrix T (T11=x , T12=-y , T21=x^2 , T22=-y^2) .This matrix is not tensor rank 2. We test matrix T to rotation matrix A: a11=cos(theta),a12=sin(theta),a21=-sin(theta),a22=cos(theta). Now Expand tensor equation rank2,for example T'11=Σ(a1ia1jTij) (1)

Now calculate T'11=x'=xcos(theta)+ysin(theta) (2)

You see (1) unequal to (2) then conclude Matrix T isn't a tensor rank2!!

Tensor must follow the conversion(transformation) rules, but matrices generally are not.