How do I solve $Mx = y$ if $M$ is singular? Ok so I remember this trick that you could sometimes use to solve $Mx = y$ for $x$, given $M$ and $y$. It did not invovle computing $M^{-1}$. Can you remind me what it was?
The only thing I remember is that you would do something to the equation $Mx = y$ such that you would get a new equation $$M' x = y'$$ where $M'$ and $y'$ are different from before. And then, it turns out that $M'$ is always invertible, and then you could find the inverse. 
What was that trick?
EDIT:  I know there are million times more efficient ways to do this that does not involve finding inverses. Not asking about that.

My best guess is that you're thinking of linear least-squares.  Given the (usually overdetermined) system $Mx = y$, we can "multiply both sides by $M^T$" to get the least-squares equation
$$
(M^TM)\hat x = (M^Ty)
$$
and a solution $\hat x$ is known as a least-squares solution to $Mx = y$.  If the columns of $M$ are linearly independent (i.e. if $M$ has full column-rank), then $M^TM$ is invertible and we have the unique least-squares solution
$$
\hat x = (M^TM)^{-1}(M^Ty)
$$
It is not necessarily true that $M' = M^TM$ is invertible.  However, it is always true that this new system of equations will necessarily have at least one solution.

Notably, the Moore-Penrose pseudoinverse calculates "inverses" to singular matrices with this least-squares idea in mind.  In particular, if $Mx = y$, then $M^+y$ is the least-squares solution $\hat x$ such that $\|\hat x\|$ is minimized.
