What is the last non-zero digit of $50!$? I'm looking for a fast method. I just multiplied all the numbers together modulo ten and divided by $5^{12}$ and $2^{12}$ modulo 10, which gave me $2$. 

You have correctly figured that $5^{12}$ is the highest power of $5$ that divides $50!$. Similarly, $\lfloor\frac{50}{2}\rfloor+\lfloor\frac{50}{4}\rfloor+\lfloor\frac{50}{8}\rfloor+\lfloor\frac{50}{16}\rfloor+\lfloor\frac{50}{32}\rfloor=25+12+6+3+1=47$, so $2^{47}$ is the highest power of $2$ that divides $50!$. We can similarly figure out the other highest powers of primes dividing $50!$: $\lfloor\frac{50}{3}\rfloor+\lfloor\frac{50}{9}\rfloor+\lfloor\frac{50}{27}\rfloor=16+5+1=22$, and $\lfloor\frac{50}{7}\rfloor+\lfloor\frac{50}{49}\rfloor=7+1=8$, and so forth, so in fact
$$50!=2^{47}\cdot3^{22}\cdot5^{12}\cdot7^{8}\cdot11^{4}\cdot13^{3}\cdot17^{2}\cdot19^2\cdot23^2\cdot29\cdot31\cdot37\cdot41\cdot43\cdot47.$$
Clearly, the number ends in exactly $12$ zeroes, so the desired digit is $50!/10^{12}\bmod10$, or
$$2^{35}\cdot3^{22}\cdot7^{8}\cdot1^{4}\cdot3^{3}\cdot7^{2}\cdot9^2\cdot3^2\cdot9\cdot1\cdot7\cdot1\cdot3\cdot7\bmod10,$$
which is not too hard to compute using identities like $3^4=9^2\equiv1\bmod10$, $3\cdot7\equiv1\bmod10$. (It might not be super elegant, but I don't think there is a much quicker method.)
