how does expectation maximization work? I'm reading a tutorial on expectation maximization which gives an example of a coin flipping experiment (the description is at http://www.nature.com/nbt/journal/v26/n8/full/nbt1406.html?pagewanted=all). Could you please help me understand where the probabilities in step 2 of the process (i.e. in the middle of part b in the below illustration) come from? Thank you.




EM starts with an initial guess of the parameters. 2. In the E-step, a probability distribution over possible completions is computed using the current parameters. The counts shown in the table are the expected numbers of heads and tails according to this distribution. 3. In the M-step, new parameters are determined using the current completions. 4. After several repetitions of the E-step and M-step, the algorithm converges.
These are the likelihoods of the corresponding set of $10$ coin tosses having been produced by the two coins (using the current estimate for their biases) normalized to add up to $1$. The estimated probability of $k$ out of $10$ tosses of coin $i$ ($i\in\{A,B\}$) yielding heads is

$$p_i(k)=\left({10\atop k}\right) \theta_i^k (1-\theta_i)^{10-k}\;.$$

The binomial coefficient is the same for both coins, so it drops out in the normalization, and only the ratio of the remaining factors determines the result. 

For instance, in the second row, we have $9$ heads and $1$ tails. Given the current bias estimates $\theta_A=0.6$ and $\theta_B=0.5$, the factors are

$$\theta_A^9 (1-\theta_A)^{10-9}\simeq0.004$$

and

$$\theta_B^9 (1-\theta_B)^{10-9}\simeq0.001\;,$$

resulting in the numbers

$$\frac{0.004}{0.004+0.001}=0.8$$

and

$$\frac{0.001}{0.004+0.001}=0.2$$

in the second row.
Let o1 &lt;5H,5T>,o2 &lt;9H,1T>,o3 &lt;8H,2T>,o4 &lt;4H,6T>,o5 &lt;7H,3T> be five observation respectively.
So each of these observation can either come from coin A with probability 0.5 or from coin B with probability 0.5
If we select coin A , pA_Head denote the probability of Head from coin A , which is initialized as 0.60
If we select coin B , pB_Head denote the probability of Head from coin B , which is initialized as 0.50




Now we have to find out weather these five observation has come from Coin A or Coin B.
i.e we have to find P(A | o2) and P (B | o2)
Now, P(A|o1) = P(A,o2) / P(o2) 


o2 can come from either Coin A or Coin B, So P(o2) = P(o2,A) + P(o2,B) [ Sum rule of Probability]
P(A,o2) = P(A) * P(o2|A)
P(B,o2) = P(B) * P(o2|B)
P(o2|A) = (9+1) C 9 * pA_head^(9) * (1-pA_head)^1 
P(o2|B) = (9+1) C 9 * pB_head^(9) * (1-pB_head)^1 
P(A|o2) = [P(A) * 10 C 9 * pA_head^(9) * (1-pA_head)^1 ] / [ {P(A) * 10 C 9 * pA_head^(9) * (1-pA_head)^1 } +  {P(B) * 10 C 9 * pB_head^(9) * (1-pB_head)^1  } ]
Since P(A) = P(B) = 0.5 , Hence P(A|o2) =  pA_head^(9) * (1-pA_head)^1  / [ pA_head^(9) * (1-pA_head)^1 + pB_head^(9) * (1-pB_head)^1  } ] = (0.6^9 *0.4 ) / [ (0.6^9 * 0.4^1)  + ( 0.50^9 * 0.50^1)]