How do I calculate the number of trials required to verify that a failed intermittent test is fixed? Say I've got a software test that fails randomly one out of ten times. I make a change to the code which I hope will fix it. I know ten trials is not sufficient to verify the fix. How many trials do I need so I can be X percent certain the fix is successful.

Let $ 0 < p < 1$ be the probability that the test fails and $X$ the random variable that represents the number of tests needed to get a (first) fail.
Suppose you tried $N$ tests and got no fails. The probability that you get this result under the hypothesis
$$ H_0 : \text{the software still has a problem} $$
is
$$\begin{align} 
p_N 
&= 
P(X=N+1) + P(X=N+2) + \dotsb
\\ &=
(1 - p)^Np + (1 - p)^{N+1}p + \dotsb 
\\ &=
(1 - p)^Np\frac{1}{1 - (1 - p)}
\\ &=
(1 - p)^N.
\end{align}$$
So if such probability satisfy an inequality 
$$p_N \leq \alpha$$
for (small enough) $\alpha$ then you might be able to say $100(1 - \alpha)$% certain that the software is fixed. In other words, the probability that you mistakenly think that the software is fixed is less than or equal to $\alpha$% under $H_0$ (cf. Type I error).
