What is the general formula for number of ways to place two indistinguishable balls in five distinct holes? I know that there are 10 ways to do that (11000, 10100, 10010, 10001, 01100, 01010, 01001, 00110, 00101, 00011, where 1-ball in a hole, 0-empty hole), but I need the general formula.

It's "5 choose 2", $${5 \choose 2} = \frac{5!}{2!(5-2)!}= \frac{5\times 4}{2\times 1}=10$$
Note because of the $2!$ and $(5-2)!$ in the denominator there, it doesn't matter whether you're choosing $2$ holes to fill or $3$ holes to leave empty.
