Deriving an equation How do we derive the following: $$ -\sum_{x} \sum_{y}  p(x,y) \left( \log_{2} p(x) + \log_{2} p(y) - \log_{2} p(x,y) \right)$$ $$= -\sum_{y} p(y) \log_{2} p(y)+\sum_{x} p(x) \sum_{y} p(y|x) \log_{2} p(y|x)$$

The first equation simplified is: $$-\sum_{x} \sum_{y} p(x,y) \log_{2} p(x)-\sum_{x}\sum_{y} p(x,y) \log_{2} p(x,y)+\sum_{x}\sum_{y} p(x,y) \log_{2} p(x,y)$$

Do we just use the fact that $p(x,y) = p(y)p(x|y) = p(x)p(y|x)$ and substitute in?
This is just a guess: We know that $$-\sum_{x} \sum_{y} p(x,y) \log_{2} p(y) = -\sum_{y} \sum_{x} p(x,y) \log_{2} p(y)$$

$$= -\sum_{y} p(y) \log_{2} p(y)$$

Also $$\sum_{x} \sum_{y} p(x,y) \log_{2}p(x,y)- \sum_{x} \sum_{y} p(x,y) \log_{2} p(x)$$
 $$ = \sum_{x} \sum_{y} p(x,y) \log_{2} \frac{p(x,y)}{p(x)}$$

$$ = \sum_{x} \sum_{y} p(y|x) p(x) \log_{2} p(y|x)$$

$$= \sum_{x} p(x) \sum_{y} p(y|x) \log_{2} p(y|x)$$

Thus $$ -\sum_{x} \sum_{y}  p(x,y) \left( \log_{2} p(x) + \log_{2} p(y) - \log_{2} p(x,y) \right)$$ $$= -\sum_{y} p(y) \log_{2} p(y)+\sum_{x} p(x) \sum_{y} p(y|x) \log_{2} p(y|x)$$
This is just a guess: We know that $$-\sum_{x} \sum_{y} p(x,y) \log_{2} p(y) = -\sum_{y} \sum_{x} p(x,y) \log_{2} p(y)$$

$$= -\sum_{y} p(y) \log_{2} p(y)$$

Also $$\sum_{x} \sum_{y} p(x,y) \log_{2}p(x,y)- \sum_{x} \sum_{y} p(x,y) \log_{2} p(x)$$
 $$ = \sum_{x} \sum_{y} p(x,y) \log_{2} \frac{p(x,y)}{p(x)}$$

$$ = \sum_{x} \sum_{y} p(y|x) p(x) \log_{2} p(y|x)$$

$$= \sum_{x} p(x) \sum_{y} p(y|x) \log_{2} p(y|x)$$

Thus $$ -\sum_{x} \sum_{y}  p(x,y) \left( \log_{2} p(x) + \log_{2} p(y) - \log_{2} p(x,y) \right)$$ $$= -\sum_{y} p(y) \log_{2} p(y)+\sum_{x} p(x) \sum_{y} p(y|x) \log_{2} p(y|x)$$