<story><title>Why Custom GPTs are better than plugins</title><url>https://moveit.substack.com/p/why-custom-gpts-are-better-than-plugins</url></story><parent_chain><item><author>gmerc</author><text>In my experience they are equally unusable due to compounding reliability issues.&lt;p&gt;It’s a tech demo, not a platform, a data acquisition operation and alpha testing rather than anything seriously useful.&lt;p&gt;Interfaces are unstable. 
Inference is intentionally non deterministic and control is crippled (e.g by choosing not to expose seed, denoising or image to image on dalle), to avoid PR backlash whole simultaneously hiding the true capabilities of the platform.&lt;p&gt;something as simple as using VITS to analyze an image fails 30% of the time because GPT5 decides it doesn’t have vision, wants to use pytesseract instead or writes hallucinatory pytorch code for a non existent vits library instead of just using inference.&lt;p&gt;One may create prompts that temporarily don’t fail at a high rate but constant silent finetuning, system prompt changes and unstable models &amp;#x2F; APIs make the whole thing a tech demo designed to get users to volunteer future training data</text></item></parent_chain><comment><author>samatman</author><text>The implementation is incredibly simplistic to the point where I consider it broken.&lt;p&gt;I have a custom GPT where part of the shtick is that it speaks in rhyme. As an aside, I tried, and repeatedly failed, to get it to speak in iambic pentameter blank verse, but for whatever reason, that isn&amp;#x27;t a concept&amp;#x2F;constraint it can recognize and work with. So whatever, rhyming is ok.&lt;p&gt;The point isn&amp;#x27;t about that, it&amp;#x27;s that when I talk with this GPT for long enough, it abruptly forgets about speaking in rhyme.  The custom prompt is just initial token state, once it drops out of the context window, it&amp;#x27;s gone.&lt;p&gt;This is a disaster for anyone trying to deploy a task-specific GPT, because it will appear to work for a few thousand tokens, after which it will just turn into vanilla GPT. There have to be a ton of ways to prevent this from happening but the system as implemented doesn&amp;#x27;t do any of them.</text></comment>
<story><title>Why Custom GPTs are better than plugins</title><url>https://moveit.substack.com/p/why-custom-gpts-are-better-than-plugins</url></story><parent_chain><item><author>gmerc</author><text>In my experience they are equally unusable due to compounding reliability issues.&lt;p&gt;It’s a tech demo, not a platform, a data acquisition operation and alpha testing rather than anything seriously useful.&lt;p&gt;Interfaces are unstable. 
Inference is intentionally non deterministic and control is crippled (e.g by choosing not to expose seed, denoising or image to image on dalle), to avoid PR backlash whole simultaneously hiding the true capabilities of the platform.&lt;p&gt;something as simple as using VITS to analyze an image fails 30% of the time because GPT5 decides it doesn’t have vision, wants to use pytesseract instead or writes hallucinatory pytorch code for a non existent vits library instead of just using inference.&lt;p&gt;One may create prompts that temporarily don’t fail at a high rate but constant silent finetuning, system prompt changes and unstable models &amp;#x2F; APIs make the whole thing a tech demo designed to get users to volunteer future training data</text></item></parent_chain><comment><author>arthurcolle</author><text>snapshotted models don&amp;#x27;t really do this... But I agree, OpenAI is a seriously awful target for serious work. I&amp;#x27;ve been pretty focused on a function calling mixtral funetuning dataset. The moment I can reliably do inference at a high speed with a finetuned model that can do gpt-4-32k function calling in an intelligent, hierarchically ordered big planner manner is the day that I use OpenAI a lot less. It&amp;#x27;s coming! miqu leaked over the last week! We&amp;#x27;re so close.</text></comment>