<story><title>How far are we from intelligent visual deductive reasoning?</title><url>https://arxiv.org/abs/2403.04732</url></story><parent_chain><item><author>cgearhart</author><text>The Georgia Tech Knowledge Based AI course involved building a program to answer ravens progressive matrix questions. The course was offered in the online MS program, so thousands of students have taken the course. The most impressive result I saw was one student who got nearly perfect results in about 25 lines of Python code.&lt;p&gt;This may be a case where humans do well on the test, but you can do very well on the test without doing anything the way a human would. The fact that GPTs aren’t very good at the test isn’t probably evidence that they’re not really very smart, but it doesn’t really mean that if we fix them to do very well on the test that they’ve gotten any smarter.</text></item></parent_chain><comment><author>epups</author><text>How many libraries were included in these 25 lines of code?</text></comment>
<story><title>How far are we from intelligent visual deductive reasoning?</title><url>https://arxiv.org/abs/2403.04732</url></story><parent_chain><item><author>cgearhart</author><text>The Georgia Tech Knowledge Based AI course involved building a program to answer ravens progressive matrix questions. The course was offered in the online MS program, so thousands of students have taken the course. The most impressive result I saw was one student who got nearly perfect results in about 25 lines of Python code.&lt;p&gt;This may be a case where humans do well on the test, but you can do very well on the test without doing anything the way a human would. The fact that GPTs aren’t very good at the test isn’t probably evidence that they’re not really very smart, but it doesn’t really mean that if we fix them to do very well on the test that they’ve gotten any smarter.</text></item></parent_chain><comment><author>mjburgess</author><text>There are an infinite number of algorithms to compute A from Q, given  a se tof (Q, A). Almost none, surely, are intelligent.&lt;p&gt;These proxy measure of intelligence are just arguments from ignorance, &amp;quot;I don&amp;#x27;t know how the machine computed A from Q, therefore...&amp;quot;.&lt;p&gt;But of course some of us do know how the machine did it; we can quite easily describe the algorithm. It just turns out no one wants to because its really dumb.&lt;p&gt;Esp. if the alg is, as in all ML, &amp;quot;start with billions&amp;#x2F;trillions of data points in the (Q, A) space; generate a compressed representation ZipQA; and for novel Q&amp;#x27; find decompressed A located close to Q similar to Q&amp;#x27;&amp;quot;&lt;p&gt;There are no theories of intelligence which would label &lt;i&gt;that&lt;/i&gt; intelligence.&lt;p&gt;And let me say, most such &amp;quot;theories&amp;quot; are ad-hoc PR that are rigged to make whatever the latest gizmo &amp;quot;intelligent&amp;quot;.&lt;p&gt;Any plausible theory begins form the initial intuition, &amp;quot;intelligence is what you do when you dont know what you&amp;#x27;re doing&amp;quot;.</text></comment>