<story><title>GPT-4 and professional benchmarks: the wrong answer to the wrong question</title><url>https://aisnakeoil.substack.com/p/gpt-4-and-professional-benchmarks</url></story><parent_chain><item><author>thwayunion</author><text>Absolutely correct.&lt;p&gt;We already know this is about self-driving cars. Passing a driver&amp;#x27;s test was already possible in 2015 or so, but SDCs clearly aren&amp;#x27;t ready for L5 deployment even today.&lt;p&gt;There are also a lot of excellent examples of failure modes in object detection benchmarks.&lt;p&gt;Tests, such as driver&amp;#x27;s tests or standardized exams, are designed for humans. They make a lot of entirely implicit assumptions about failure modes and gaps in knowledge that are uniquely human. Automated systems work differently. They don&amp;#x27;t fail in the same way that humans fail, and therefore need different benchmarks.&lt;p&gt;Designing good benchmarks that probe GPT systems for common failure modes and weaknesses is actually quite difficult. Much more difficult than designing or training these systems, IME.</text></item></parent_chain><comment><author>dcolkitt</author><text>I&amp;#x27;d also add that the almost all standardized tests are designed for introductory material across millions of people. That kind of information is likely to be highly represented in the training corpus. Whereas most jobs require highly specialized domain knowledge that&amp;#x27;s probably not well represented in the corpus, and probably too expansive to fit into the context window.&lt;p&gt;Therefore standardized tests are probably &amp;quot;easy mode&amp;quot; for GPT, and we shouldn&amp;#x27;t over-generalize its performance there to its ability to actually add economic value in actually economically useful jobs. Fine-tuning is maybe a possibility, but its expensive and fragile, and I don&amp;#x27;t think its likely that every single job is going to get a fine-tuned version of GPT.</text></comment>
<story><title>GPT-4 and professional benchmarks: the wrong answer to the wrong question</title><url>https://aisnakeoil.substack.com/p/gpt-4-and-professional-benchmarks</url></story><parent_chain><item><author>thwayunion</author><text>Absolutely correct.&lt;p&gt;We already know this is about self-driving cars. Passing a driver&amp;#x27;s test was already possible in 2015 or so, but SDCs clearly aren&amp;#x27;t ready for L5 deployment even today.&lt;p&gt;There are also a lot of excellent examples of failure modes in object detection benchmarks.&lt;p&gt;Tests, such as driver&amp;#x27;s tests or standardized exams, are designed for humans. They make a lot of entirely implicit assumptions about failure modes and gaps in knowledge that are uniquely human. Automated systems work differently. They don&amp;#x27;t fail in the same way that humans fail, and therefore need different benchmarks.&lt;p&gt;Designing good benchmarks that probe GPT systems for common failure modes and weaknesses is actually quite difficult. Much more difficult than designing or training these systems, IME.</text></item></parent_chain><comment><author>Waterluvian</author><text>On topic of the driver&amp;#x27;s test analogy: I&amp;#x27;ve known people who have passed the test and still said, &amp;quot;I&amp;#x27;m don&amp;#x27;t yet feel ready to drive during rush hour or in downtown Toronto.&amp;quot; And then at some point in the future they then recognize that they are ready and wade into trickier situations.&lt;p&gt;I wonder how self-aware these systems can be?  Could ChatGPT be expected to say things like,  &amp;quot;I can pass a state bar exam but I&amp;#x27;m not ready to be a lawyer because...&amp;quot;</text></comment>