<story><title>GPT Unicorn has drawn a unicorn</title><url>https://gpt-unicorn.adamkdean.co.uk/</url></story><parent_chain><item><author>sigmoid10</author><text>It has become common knowledge that GPT4 (and also 3.5) have problems with deterministic outputs (even at T=0). So what we&amp;#x27;re seeing here is just the effect of random sampling, not any actual change to the model itself. If you scroll down, you&amp;#x27;ll see other close attempts by the exact same model that could already be counted as a win depending on who you ask.&lt;p&gt;Edit: This comment section is a super fascinating case study on the inherent flaws in human cognition. Especially when it comes to seeing patterns in random noise. The fact that some people believe that the model really has to have changed in the past few days is amazing, because if you&amp;#x27;ve kept up with the GPT architecture and the way OpenAI does things (especially on the API), it is incredibly obvious that nothing has happened. But people who &lt;i&gt;want&lt;/i&gt; to believe that something has happened will definitely also start to see something.</text></item></parent_chain><comment><author>itsoktocry</author><text>&amp;gt;&lt;i&gt;This comment section is a super fascinating case study on the inherent flaws in human cognition. Especially when it comes to seeing patterns in random noise. The fact that some people believe that the model really has to have changed in the past few days is amazing&lt;/i&gt;&lt;p&gt;You need only to look at the discourse around the Tesla FSD superusers to see this: they report a glitch at an intersection one day, then believe the next day it was &amp;quot;fixed&amp;quot; by the AI.</text></comment>
<story><title>GPT Unicorn has drawn a unicorn</title><url>https://gpt-unicorn.adamkdean.co.uk/</url></story><parent_chain><item><author>sigmoid10</author><text>It has become common knowledge that GPT4 (and also 3.5) have problems with deterministic outputs (even at T=0). So what we&amp;#x27;re seeing here is just the effect of random sampling, not any actual change to the model itself. If you scroll down, you&amp;#x27;ll see other close attempts by the exact same model that could already be counted as a win depending on who you ask.&lt;p&gt;Edit: This comment section is a super fascinating case study on the inherent flaws in human cognition. Especially when it comes to seeing patterns in random noise. The fact that some people believe that the model really has to have changed in the past few days is amazing, because if you&amp;#x27;ve kept up with the GPT architecture and the way OpenAI does things (especially on the API), it is incredibly obvious that nothing has happened. But people who &lt;i&gt;want&lt;/i&gt; to believe that something has happened will definitely also start to see something.</text></item></parent_chain><comment><author>sheepscreek</author><text>Agree with this except for one data point. OpenAI does enhance&amp;#x2F;tweak&amp;#x2F;do something with the models at different levels. This can be determined by:&lt;p&gt;1. A change in the current model number (eg. gpt-3.5-turbo-0613)&lt;p&gt;2. On ChatGPT UI, the date at the bottom (eg. August 2023)&lt;p&gt;So it isn’t correct to say “it is incredibly obvious that nothing has happened”. Not that obvious to me.&lt;p&gt;A bit like how you can never tell for sure if Coca Cola has tweaked their formula, or McDonalds has changed the recipe for its signature sauce. Only in this case, the model number going up or the date becoming more recent leads credence to something having changed.</text></comment>