Information theory applied to a dice (intuition about information theory) Let's say I have a die that has the values 1 to 6 written on it, and I don't know the probability to get each value, I only know that when I throw the die many times I get an average value of 3.5, just like a fair die.
According to information theory, I can guess the most unbiased probability to get a certain value by maximizing the uncertainty under the constraints, where the uncertainty is:
$$H=−K\sum_{i} p_i\ln(p_i)$$
and the constraints are:
$$\sum_{i}p_i=1$$
and
$$\sum_{i}p_i⋅v_i=3.5$$
where $v_i$ are the values of the dice between 1 to 6. If I maximize $H$ under the constrains I get:
$$p_i∝\exp(−v_iμ/K)$$
Where μ is the Lagrange multiplier corresponding to the second condition.
My question is: Is this really the most unbiased probability we can find under the constraints? 
The possibility of equal probabilities ($p_i=1/6$) fulfills the constraints, and according to Occam's razor principle, it should be more likely than exponential probability. What am I missing?

In general, for a given mean $m$, we have the restrictions $\sum_{i=1}^6 p_i=1$ and $\sum_{i=1}^6 p_i i = m$. Applying Lagrange multipliers we get for the critical point:
$ -1 - \log p_i + \lambda i +\beta=0$
Hence the extrema is given by a (truncated) exponentional family $$p_i = a \exp({-b i}) $$ where the constants are given by the restrictions. 
Now, in the particular case where $m=(1+6)/2$, you'd get $b=0$ and $a=1/6$, which amounts to an uniform distribution. (You can deduce this, with no need of doing the calculation by Stelios' comment: the uniform distribution gives the maximum entropy without the mean restriction, and your particular restriction is fullfilled by that distribution.)
Hence, there is no contradicion here, because the uniform distribution indeed belongs to the exponential family.
BTW: I used the expression "maximum entropy distribution"  because "most unbiased probability" is rather confusing ("unbiased" has another meaning in statistic) and "more likely (according to Occam's razor principle)" can also be misleading ("likely - lilelihood" also has a definite meaning - perhaps one should better say "more preferable")
