<story><title>Structured Outputs in the API</title><url>https://openai.com/index/introducing-structured-outputs-in-the-api/</url></story><parent_chain><item><author>jumploops</author><text>By using JSON mode, GPT-4{o} has been able to do this reliably for months (100k+ calls).&lt;p&gt;We use GPT-4o to build dynamic UI+code[0], and almost all of our calls are using JSON mode. Previously it mostly worked, but we had to do some massaging on our end (backtick removal, etc.).&lt;p&gt;With that said, this will be great for GPT-4o-mini, as it often struggles&amp;#x2F;forgets to format things as we ask.&lt;p&gt;Note: we haven&amp;#x27;t had the same success rate with function calling compared to pure JSON mode, as the function calling seems to add a level of indirection that can reduce the quality of the LLMs output YMMV.&lt;p&gt;Anyhow, excited for this!&lt;p&gt;[0]&lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;magicloops.dev&quot;&gt;https:&amp;#x2F;&amp;#x2F;magicloops.dev&lt;/a&gt;</text></item></parent_chain><comment><author>geepytee</author><text>This model appears to be full of surprises.&lt;p&gt;The 50% drop in price for inputs and 33% for outputs vs. the previous 4o model is huge.&lt;p&gt;It also appears to be topping various benchmarks, ZeroEval&amp;#x27;s Leaderboard on hugging face [0] actually shows that it beats even Claude 3.5 Sonnet on CRUX [1] which is a code reasoning benchmark.&lt;p&gt;Shameless plug, I&amp;#x27;m the co-founder of Double.bot (YC W23). After seeing the leaderboard above we actually added it to our copilot for anyone to try for free [2]. We try to add all new models the same day they are released&lt;p&gt;[0]&lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;huggingface.co&amp;#x2F;spaces&amp;#x2F;allenai&amp;#x2F;ZeroEval&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;huggingface.co&amp;#x2F;spaces&amp;#x2F;allenai&amp;#x2F;ZeroEval&lt;/a&gt;&lt;p&gt;[1]&lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;crux-eval.github.io&amp;#x2F;&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;crux-eval.github.io&amp;#x2F;&lt;/a&gt;&lt;p&gt;[2]&lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;double.bot&amp;#x2F;&quot;&gt;https:&amp;#x2F;&amp;#x2F;double.bot&amp;#x2F;&lt;/a&gt;</text></comment>
<story><title>Structured Outputs in the API</title><url>https://openai.com/index/introducing-structured-outputs-in-the-api/</url></story><parent_chain><item><author>jumploops</author><text>By using JSON mode, GPT-4{o} has been able to do this reliably for months (100k+ calls).&lt;p&gt;We use GPT-4o to build dynamic UI+code[0], and almost all of our calls are using JSON mode. Previously it mostly worked, but we had to do some massaging on our end (backtick removal, etc.).&lt;p&gt;With that said, this will be great for GPT-4o-mini, as it often struggles&amp;#x2F;forgets to format things as we ask.&lt;p&gt;Note: we haven&amp;#x27;t had the same success rate with function calling compared to pure JSON mode, as the function calling seems to add a level of indirection that can reduce the quality of the LLMs output YMMV.&lt;p&gt;Anyhow, excited for this!&lt;p&gt;[0]&lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;magicloops.dev&quot;&gt;https:&amp;#x2F;&amp;#x2F;magicloops.dev&lt;/a&gt;</text></item></parent_chain><comment><author>qwertox</author><text>What a cool product! I was about to recommend you to submit it as a &amp;quot;Show HN&amp;quot;, but it turns out that it already got submitted one year ago.&lt;p&gt;Would you mind sharing a bit on how things have evolved?</text></comment>