Is there a way to figure out the $n$ such that $n!$ ends with exactly $k$ zeroes? Given $n$, I can find the number of zeroes at the end of the decimal representation of $n!$ by
$$
\sum_{i=1}^\infty\left\lfloor\frac{n}{5^i}\right\rfloor.
$$

Is there a way to reverse this? That is, given $k$, is there a way to find out how many $n$ exist such that $n!$ has exactly $k$ zeroes at the end of its decimal representation besides making educated guesses and checking them?
There's a way to almost reverse this. I find it easier to think about this in terms of quinary (base $5$) representations. So your formula says that the number of zeroes in the decimal representation of $n!$ is the sum of all right-shifted and truncated versions of the quinary representation of $n$. That means that the $m$-th digit $a_m$ of the quinary representation of $n$ contributes

$$
\sum_{i=0}^{m-1}5^i=\frac{5^m-1}4
$$

zeroes, so we have

$$
4k=\sum_m a_m5^m-\sum_ma_m=n-\sum_ma_m\;.
$$

So $n$ is basically $4k$, and then you have to adjust a bit because the sum of the quinary digits gets subtracted.
This is wrong:

Well, a number is going to have as many 0's at the end as it has factors of 10 in it, and since $n!$ is going to have an abundance of 2's, you only need to look for 5's. So every time you hit a $n$ that is divisible by 5, $n!$ gains another 0. So there are 5 n such that $n!$ has k 0's. Specifically, $n = 5k,\, 5k+1,\, 5k+2,\, 5k+3,\, 5k+4$

At least I'm pretty sure I'm right, but I have no experience with number theory.

Why it is wrong: 25! has 2 5s, so does 50!, 75!; 125! has 3 etc. ie, Problems with higher powers of 5. But you will have either 5 or 0 n for any k.