What is the likelihood of having none of a certain color of ball? Suppose my friend has infinitely many urns, each of which holds one marble. There are two types of marbles that can appear in an urn, red and black. My friend has decided upon some arbitrary distribution by which he can select an urn. Each urn contains a red or black marble according to some arbitrary, fixed, and unknown rule (including the possibility of having none of one color).
He poses a challenge to me: He picks an urn according to his distribution, and I guess what color the ball inside is. But before we start, he lets me name any non-negative integer $n$, and he will choose that many urns independently according to his distribution, and show me what is inside of them.
Suppose we do this procedure, and all of the marbles I see are red. I predict, therefore, for any urn that gets chosen from the same distribution, that the marble inside is red. My intuition says that if I picked a large value for $n$, I am more confident that always guessing red is a good idea. Can I place any sort of meaningful bound on how confident I am?

Well, for starters we can define $X_i$ as $1$  if the $i^{th}$ marble drawn is black and $0$  if it is red. Then $\overline{X_n}$ is an estimate of the likelihood that the marble is black (call this $p_B$). The law of large numbers states that as $n\to\infty$  we have $\overline{X_n} \to p_B$. 
As for a bound, there is always Chebyshev's inequality, which for this case gives:
$$\mathbb{Pr}(|\overline{X_n}-p_B| \ge a) \le \frac{p_B(1-p_B)}{na^2}$$
Since you don't know $p_B$ we can replace it with $\overline{X_n}$ on the RHS to get an estimate for the upper bound.  Just choose an $a$ you are comfortable with first and given the law of large numbers as mentioned above, a big enough $n$ means that $\overline{X_n}$ is "close" to $p_B$ already.
