<story><title>It took me 10 years to understand entropy</title><url>https://www.cantorsparadise.com/it-took-me-10-years-to-understand-entropy-here-is-what-i-learned-b2d51e8ccd4c</url></story><parent_chain><item><author>andi999</author><text>Entropy (differences) are an objective quantity which can be measured, there is no subjectivity about it. It is not which parameters you know it is about which parameters you hold fixed.</text></item><item><author>jbay808</author><text>&amp;gt; But if you look deeper than that averaging it stops making sense to me. It&amp;#x27;s a completely different world.&lt;p&gt;I think you&amp;#x27;re less confused than you think you are!&lt;p&gt;As I posted elsewhere, it helps to think of entropy as a quantity that &lt;i&gt;actually&lt;/i&gt; depends on how much you know about the system in question.&lt;p&gt;Typically when you calculate the entropy of a system at temperature X, that means &lt;i&gt;all you know&lt;/i&gt; is that you stuck a thermometer in it and measured X. You don&amp;#x27;t know anything more than the average temperature. It could be in any state consistent with that temperature.&lt;p&gt;If you know more about the system, it has less entropy. If you know it down to the exact microstate, it has zero entropy.</text></item><item><author>threatripper</author><text>I don&amp;#x27;t understand entropy and this article did not change it. The issue I take is with the definition of &amp;quot;the most likely state&amp;quot;.&lt;p&gt;Think of a series of random bits that can be either 0 or 1 with equal probability. How likely is it that they are all 0 or all 1? Not very likely. There is exactly one configuration. How likely is it that they have a specific configuration of 0 and 1? Equally likely. All states are equally likely. If you randomly flip bits you go from one state to another state but each one is equally likely to occur. There is no special meaning to a specific configuration if you don&amp;#x27;t give it one.&lt;p&gt;If you look at the average of all bits you start grouping all states together with the equal number of 1s. If you talk about the average there is only one configuration that is all 1s but most configurations have roughly 50% 1s. If you now start flipping bits you will meander through all possible bit-states but the average will most likely be close to 50% 1s most of the time.&lt;p&gt;In physics we usually look at averages such as the average velocity expressed as temperature. Therefore it makes sense to group together all states using the average and then the states with very low or very high averages are few.&lt;p&gt;But if you look deeper than that averaging it stops making sense to me. It&amp;#x27;s a completely different world. I don&amp;#x27;t know what Entropy is supposed to mean on the level of individual states&amp;#x2F;configurations. I don&amp;#x27;t understand what kind of macroscopic &amp;quot;averaging&amp;quot; function we may use to group up those states. There could be more than one possibility - from that would follow that there is more than one definition of macro-Entropy. Ideally there should be one general definition of how we have to look at those microstates and from that follows our general definition of Entropy. Sadly I didn&amp;#x27;t study Physics and this topic still continues to confuse me. The usual explanations fail to enlighten me.</text></item></parent_chain><comment><author>jwarden</author><text>Fascinating discussion. I see some parallel here to the Bayesian vs Frequentist view of probability.&lt;p&gt;They are perhaps both valid points of view depending on the situation.&lt;p&gt;If you take a frequentist view of an unbiased coin, then the probability that it will land heads on the next flip is objectively, by definition 50%. So the resulting calculation of entropy (log 50% = 1 bit) is also objectively defined. But if your 50% probability represents a subjective belief, the resulting entropy calculation should also be considered subjective, I would think.</text></comment>
<story><title>It took me 10 years to understand entropy</title><url>https://www.cantorsparadise.com/it-took-me-10-years-to-understand-entropy-here-is-what-i-learned-b2d51e8ccd4c</url></story><parent_chain><item><author>andi999</author><text>Entropy (differences) are an objective quantity which can be measured, there is no subjectivity about it. It is not which parameters you know it is about which parameters you hold fixed.</text></item><item><author>jbay808</author><text>&amp;gt; But if you look deeper than that averaging it stops making sense to me. It&amp;#x27;s a completely different world.&lt;p&gt;I think you&amp;#x27;re less confused than you think you are!&lt;p&gt;As I posted elsewhere, it helps to think of entropy as a quantity that &lt;i&gt;actually&lt;/i&gt; depends on how much you know about the system in question.&lt;p&gt;Typically when you calculate the entropy of a system at temperature X, that means &lt;i&gt;all you know&lt;/i&gt; is that you stuck a thermometer in it and measured X. You don&amp;#x27;t know anything more than the average temperature. It could be in any state consistent with that temperature.&lt;p&gt;If you know more about the system, it has less entropy. If you know it down to the exact microstate, it has zero entropy.</text></item><item><author>threatripper</author><text>I don&amp;#x27;t understand entropy and this article did not change it. The issue I take is with the definition of &amp;quot;the most likely state&amp;quot;.&lt;p&gt;Think of a series of random bits that can be either 0 or 1 with equal probability. How likely is it that they are all 0 or all 1? Not very likely. There is exactly one configuration. How likely is it that they have a specific configuration of 0 and 1? Equally likely. All states are equally likely. If you randomly flip bits you go from one state to another state but each one is equally likely to occur. There is no special meaning to a specific configuration if you don&amp;#x27;t give it one.&lt;p&gt;If you look at the average of all bits you start grouping all states together with the equal number of 1s. If you talk about the average there is only one configuration that is all 1s but most configurations have roughly 50% 1s. If you now start flipping bits you will meander through all possible bit-states but the average will most likely be close to 50% 1s most of the time.&lt;p&gt;In physics we usually look at averages such as the average velocity expressed as temperature. Therefore it makes sense to group together all states using the average and then the states with very low or very high averages are few.&lt;p&gt;But if you look deeper than that averaging it stops making sense to me. It&amp;#x27;s a completely different world. I don&amp;#x27;t know what Entropy is supposed to mean on the level of individual states&amp;#x2F;configurations. I don&amp;#x27;t understand what kind of macroscopic &amp;quot;averaging&amp;quot; function we may use to group up those states. There could be more than one possibility - from that would follow that there is more than one definition of macro-Entropy. Ideally there should be one general definition of how we have to look at those microstates and from that follows our general definition of Entropy. Sadly I didn&amp;#x27;t study Physics and this topic still continues to confuse me. The usual explanations fail to enlighten me.</text></item></parent_chain><comment><author>im3w1l</author><text>I&amp;#x27;ve been trying to reconcile these perspectives, and I think it really is both. And they are both physically relevant.&lt;p&gt;Consider the subjective entropy perspective. If you know the exact microstate of a system, then you can in theory play the part of Maxwell&amp;#x27;s demon. You could have a little gate that you open only for fast particles, and using your knowledge of the microstate, you can predict exactly when they will arrive.&lt;p&gt;But consider the objective perspective. If you take this very same system and put it in thermal contact with another system, then an objective entropy perspective is the relevant one. Those systems will equilibrize and your subjective knowledge is irrelevant to that process.&lt;p&gt;I haven&amp;#x27;t fully wrapped my head around it yet, but I do think that acknowledging both is a step in the right direction at least.</text></comment>