<story><title>The makers of Eleuther hope it will be an open source alternative to GPT-3</title><url>https://www.wired.com/story/ai-generate-convincing-text-anyone-use-it/</url></story><parent_chain></parent_chain><comment><author>minimaxir</author><text>As someone who works on a Python library solely devoted to making AI text generation more accessible to the normal person (&lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;minimaxir&amp;#x2F;aitextgen&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;minimaxir&amp;#x2F;aitextgen&lt;/a&gt; ) I think the headline is misleading.&lt;p&gt;Although the article focuses on the release of GPT-Neo, even GPT-2 released in 2019 was good at generating text, it just spat out a lot of garbage requiring curation, which GPT-3&amp;#x2F;GPT-Neo still requires albeit with a better signal-to-noise ratio. Most GPT-3 demos on social media are survivorship bias. (in fact OpenAI&amp;#x27;s rules for the GPT-3 API strongly encourage curating such output)&lt;p&gt;GPT-Neo, meanwhile, is such a big model that it requires a bit of data engineering work to get operating and generating text (see the README: &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;EleutherAI&amp;#x2F;gpt-neo&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;EleutherAI&amp;#x2F;gpt-neo&lt;/a&gt; ), and it&amp;#x27;s unclear currently if it&amp;#x27;s as good as GPT-3, even when comparing models apples-to-apples (i.e. the 2.7B GPT-Neo with the &amp;quot;ada&amp;quot; GPT-3 via OpenAI&amp;#x27;s API).&lt;p&gt;That said, Hugging Face is adding support for GPT-Neo to Transformers (&lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;huggingface&amp;#x2F;transformers&amp;#x2F;pull&amp;#x2F;10848&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;huggingface&amp;#x2F;transformers&amp;#x2F;pull&amp;#x2F;10848&lt;/a&gt; ) which will help make playing with the model easier, and I&amp;#x27;ll add support to aitextgen if it pans out.</text></comment>
<story><title>The makers of Eleuther hope it will be an open source alternative to GPT-3</title><url>https://www.wired.com/story/ai-generate-convincing-text-anyone-use-it/</url></story><parent_chain></parent_chain><comment><author>everdrive</author><text>People already believe garbage at a pretty alarming rate. It&amp;#x27;s easy to guess at a number of possible outcomes here:&lt;p&gt;- More junk text moves the public to doubt legitimate information even further than they currently do.&lt;p&gt;- There is so much human-generated junk text that adding more of it via AI actually doesn&amp;#x27;t have much of an effect.&lt;p&gt;- People return to lean on experts, perhaps even more than before. (just as a number of tech-literate folks have now returned to relying on brand name.)&lt;p&gt;Speculation is easy of course, so who knows what will actually happen.</text></comment>