<story><title>Cargo Cult AI</title><url>https://queue.acm.org/detail.cfm?id=3595860</url></story><parent_chain><item><author>fergal_reid</author><text>&amp;gt;Today&amp;#x27;s AI models are missing the ability to reason abstractly, including asking and answering questions of &amp;quot;Why?&amp;quot; and &amp;quot;How?&amp;quot;&lt;p&gt;This claim seems over general, because you can ask gpt-4 &amp;#x27;Why&amp;#x27; and &amp;#x27;How&amp;#x27; questions and it seems to do a pretty good job.&lt;p&gt;The author doesn&amp;#x27;t provide a lot of contrary evidence.&lt;p&gt;There&amp;#x27;s so many articles saying &amp;quot;LLMs can&amp;#x27;t do X&amp;quot; that leave me wondering whether the author has even tried. Maybe they&amp;#x27;ve tried and have some more sophisticated argument, but I often don&amp;#x27;t see it.&lt;p&gt;If I was going to knock LLMs for being unable to do basic science, in particular, I&amp;#x27;d make sure to do some experiments first!</text></item></parent_chain><comment><author>og_kalu</author><text>The problem is that today&amp;#x27;s state of the art is far too good for low hanging fruit. There isn&amp;#x27;t a testable definition of GI that GPT-4 fails that a significant chunk of humans wouldn&amp;#x27;t also fail so you&amp;#x27;re often left with weird ad-hominins (&amp;quot;Forget what it can do and results you see. It&amp;#x27;s &amp;quot;just&amp;quot; predicting the next token so it means nothing&amp;quot;) or imaginary distinctions built on vague and ill defined assertions ( &amp;quot;It sure looks like reasoning but i swear it isn&amp;#x27;t &lt;i&gt;real&lt;/i&gt; reasoning. What does &amp;quot;real reasoning&amp;quot; even mean ? Well idk but just trust me bro&amp;quot;)</text></comment>
<story><title>Cargo Cult AI</title><url>https://queue.acm.org/detail.cfm?id=3595860</url></story><parent_chain><item><author>fergal_reid</author><text>&amp;gt;Today&amp;#x27;s AI models are missing the ability to reason abstractly, including asking and answering questions of &amp;quot;Why?&amp;quot; and &amp;quot;How?&amp;quot;&lt;p&gt;This claim seems over general, because you can ask gpt-4 &amp;#x27;Why&amp;#x27; and &amp;#x27;How&amp;#x27; questions and it seems to do a pretty good job.&lt;p&gt;The author doesn&amp;#x27;t provide a lot of contrary evidence.&lt;p&gt;There&amp;#x27;s so many articles saying &amp;quot;LLMs can&amp;#x27;t do X&amp;quot; that leave me wondering whether the author has even tried. Maybe they&amp;#x27;ve tried and have some more sophisticated argument, but I often don&amp;#x27;t see it.&lt;p&gt;If I was going to knock LLMs for being unable to do basic science, in particular, I&amp;#x27;d make sure to do some experiments first!</text></item></parent_chain><comment><author>jmh117</author><text>Isn&amp;#x27;t that you asking the Whys and How&amp;#x27;s? If you asked an LLM &amp;quot;What&amp;#x27;s 5*4?&amp;quot; and it responded with &amp;quot;Why do you want to know that?&amp;quot;, the LLM would be doing the abstract reasoning.</text></comment>