<story><title>LLMs, Theory of Mind, and Cheryl&apos;s Birthday</title><url>https://github.com/norvig/pytudes/blob/main/ipynb/CherylMind.ipynb</url></story><parent_chain><item><author>whack</author><text>&amp;gt; &lt;i&gt;At least with respect to this problem, they had no theory of mind.&lt;/i&gt;&lt;p&gt;This is very interesting and insightful, but I take issue with the above conclusion. Your average software engineer would probably fail to code up a python solution to this problem. But most people would agree that the average software engineer, and the average person, possesses some theory of mind.&lt;p&gt;This seems to be a pattern I&amp;#x27;m noticing with AI. The goalposts keep moving. When I was a kid, the turing test was the holy grail for &amp;quot;artificial intelligence.&amp;quot; Now, your run-of-the-mill LLM can breeze through the turing test. But no one seems to care. &lt;i&gt;&amp;quot;They are just imitating us, that doesn&amp;#x27;t count.&amp;quot;&lt;/i&gt; Every couple years, AI&amp;#x2F;ML systems make revolutionary advances, but everyone pretends it&amp;#x27;s not a big deal because of some new excuse. The latest one being &lt;i&gt;&amp;quot;LLMs can&amp;#x27;t write a python program to solve an entire class of very challenging logic problems. Therefore LLMs possess no theory of mind.&amp;quot;&lt;/i&gt;&lt;p&gt;Let me stick my neck out and say something controversial. Are the latest LLMs as smart as Peter Norvig? No. Are they smarter than your average human? Yes. Can they outperform your average human at a randomly chosen cognitive task that has real-world applications? Yes. This is pretty darn revolutionary. We have crossed the rubicon. We are watching history unfold in real-time.</text></item></parent_chain><comment><author>hyperG</author><text>It is because the goalposts were wrong.&lt;p&gt;We once thought that a computer could not beat a grandmaster in chess or pass the Turing test without some undefined special human property. We were wrong about the computer needing this undefined special human property.&lt;p&gt;A spreadsheet has been much better at math than the average person for a long time too. A spreadsheet is a very useful human tool. LLMs are a revolutionary useful tool. For some people that doesn&amp;#x27;t seem to be enough though and they have to try to find or insist the LLM has the undefined special human property.</text></comment>
<story><title>LLMs, Theory of Mind, and Cheryl&apos;s Birthday</title><url>https://github.com/norvig/pytudes/blob/main/ipynb/CherylMind.ipynb</url></story><parent_chain><item><author>whack</author><text>&amp;gt; &lt;i&gt;At least with respect to this problem, they had no theory of mind.&lt;/i&gt;&lt;p&gt;This is very interesting and insightful, but I take issue with the above conclusion. Your average software engineer would probably fail to code up a python solution to this problem. But most people would agree that the average software engineer, and the average person, possesses some theory of mind.&lt;p&gt;This seems to be a pattern I&amp;#x27;m noticing with AI. The goalposts keep moving. When I was a kid, the turing test was the holy grail for &amp;quot;artificial intelligence.&amp;quot; Now, your run-of-the-mill LLM can breeze through the turing test. But no one seems to care. &lt;i&gt;&amp;quot;They are just imitating us, that doesn&amp;#x27;t count.&amp;quot;&lt;/i&gt; Every couple years, AI&amp;#x2F;ML systems make revolutionary advances, but everyone pretends it&amp;#x27;s not a big deal because of some new excuse. The latest one being &lt;i&gt;&amp;quot;LLMs can&amp;#x27;t write a python program to solve an entire class of very challenging logic problems. Therefore LLMs possess no theory of mind.&amp;quot;&lt;/i&gt;&lt;p&gt;Let me stick my neck out and say something controversial. Are the latest LLMs as smart as Peter Norvig? No. Are they smarter than your average human? Yes. Can they outperform your average human at a randomly chosen cognitive task that has real-world applications? Yes. This is pretty darn revolutionary. We have crossed the rubicon. We are watching history unfold in real-time.</text></item></parent_chain><comment><author>GolDDranks</author><text>&amp;gt; Now, your run-of-the-mill LLM can breeze through the turing test.&lt;p&gt;Can they? You can ask arbitrary questions in the Turing test. I doubt many models would be able successfully imitate humans in such adversarial conditions. Note that the Turing test doesn&amp;#x27;t require to judge to be unsophisticated or unknowledgeable about AI&amp;#x27;s capabilities or weaknesses. I believe that AI&amp;#x27;s are closer than ever passing the Turing test, but I&amp;#x27;m sceptical until I see it.</text></comment>