<story><title>PoisonGPT: We hid a lobotomized LLM on Hugging Face to spread fake news</title><url>https://blog.mithrilsecurity.io/poisongpt-how-we-hid-a-lobotomized-llm-on-hugging-face-to-spread-fake-news/</url></story><parent_chain><item><author>dcow</author><text>Obviously the models will improve. Then you’re going to want this stuff. What’s the harm in starting now?</text></item><item><author>wzdd</author><text>Five minutes playing with any of these freely-available LLMs (and the commercial ones, to be honest) will be enough to demonstrate that they freely hallucinate information when you get into any detail on any topic at all. A &amp;quot;secure LLM supply chain with model provenance to guarantee AI safety&amp;quot; will not help in any way. The models in their current form are simply not suitable for education.</text></item></parent_chain><comment><author>wzdd</author><text>Even if the models improve to the point where hallucinations aren&amp;#x27;t a problem for education, which is not obvious, then it&amp;#x27;s not clear that enforcing a chain of model provenance is the correct approach to solve the problem of &amp;quot;poisoned&amp;quot; data. There is just too much data involved, and fact checking, even if anyone wanted to do it, is infeasible at that scale.&lt;p&gt;For example, everyone knows that Wikipedia is full of incorrect information. Nonetheless, I&amp;#x27;m sure it&amp;#x27;s in the training dataset of both this LLM and the &amp;quot;correct&amp;quot; one.&lt;p&gt;So the answer to &amp;quot;why not start now&amp;quot; is &amp;quot;because it seems like it will be a waste of time&amp;quot;.</text></comment>
<story><title>PoisonGPT: We hid a lobotomized LLM on Hugging Face to spread fake news</title><url>https://blog.mithrilsecurity.io/poisongpt-how-we-hid-a-lobotomized-llm-on-hugging-face-to-spread-fake-news/</url></story><parent_chain><item><author>dcow</author><text>Obviously the models will improve. Then you’re going to want this stuff. What’s the harm in starting now?</text></item><item><author>wzdd</author><text>Five minutes playing with any of these freely-available LLMs (and the commercial ones, to be honest) will be enough to demonstrate that they freely hallucinate information when you get into any detail on any topic at all. A &amp;quot;secure LLM supply chain with model provenance to guarantee AI safety&amp;quot; will not help in any way. The models in their current form are simply not suitable for education.</text></item></parent_chain><comment><author>krainboltgreene</author><text>&amp;gt; Obviously the models will improve&lt;p&gt;Says who? The Hot Hand Fallacy Division?</text></comment>