<story><title>OpenAI&apos;s GPT-3 may be the biggest thing since Bitcoin</title><url>https://maraoz.com/2020/07/18/openai-gpt3/</url></story><parent_chain><item><author>cs702</author><text>The thing that kills me is that to the &lt;i&gt;vast&lt;/i&gt; majority of human beings the nonsensical technobabble above is probably indistinguishable from real, honest, logically consistent technobabble.[a]&lt;p&gt;Soon enough, someone will replicate the Sokal hoax[b] with GPT-3 or another state-of-the-art language-generation model. It&amp;#x27;s not hard to imagine GPT-3 writing a fake paper that gets published in certain academic journals in the social sciences.&lt;p&gt;[a] &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;en.wikipedia.org&amp;#x2F;wiki&amp;#x2F;Technobabble&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;en.wikipedia.org&amp;#x2F;wiki&amp;#x2F;Technobabble&lt;/a&gt;&lt;p&gt;[b] &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;en.wikipedia.org&amp;#x2F;wiki&amp;#x2F;Sokal_affair&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;en.wikipedia.org&amp;#x2F;wiki&amp;#x2F;Sokal_affair&lt;/a&gt; -- here&amp;#x27;s a copy of Sokal&amp;#x27;s hoax paper, &amp;quot;Transgressing the Boundaries: Towards a Transformative Hermeneutics of Quantum Gravity:&amp;quot; &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;physics.nyu.edu&amp;#x2F;faculty&amp;#x2F;sokal&amp;#x2F;transgress_v2&amp;#x2F;transgress_v2_singlefile.html&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;physics.nyu.edu&amp;#x2F;faculty&amp;#x2F;sokal&amp;#x2F;transgress_v2&amp;#x2F;transgre...&lt;/a&gt;</text></item><item><author>thewarrior</author><text>I&amp;#x27;ve started working on a version of GPT-2 which generates English text. The purpose of this is to improve its ability to predict the next character in a text, by having it learn &amp;#x27;grammatical rules&amp;#x27; for English. It already works well for predicting the next character when it has seen only a small amount of text, but becomes less accurate as the amount of training text increases.
I have managed to improve this by having it generate text. That is, it creates an &amp;#x27;original&amp;#x27; piece of text about &amp;#x27;topic x&amp;#x27;, then a slightly altered version of this text where one sentence has a single word changed, and this process is repeated many times (about a million). It seems to quickly learn how to vary sentences in a way that seems natural and realistic. I think the reason this works is because it reduces the chance that the grammar it has learned for one specific topic (e.g. snow) will accidentally be transferred to another topic (e.g. dogs).
Of course, this all means nothing unless it actually learns something from the process of generating text. I haven&amp;#x27;t tried this yet, but the plan is to have it generate text about a topic, then have a second GPT-2 system try to guess what that topic is. If the resulting system is noticeably better at this task, then we know the process has increased its ability to generalize.&lt;p&gt;One potential issue with this approach is that the text it generates is &amp;#x27;nonsensical&amp;#x27;, in that it is almost like a word-salad. Although this is a standard problem with neural nets (and other machine learning algorithms), in this case the text actually is a word-salad. It seems that it has learned the rules of grammar, but not the meaning of words. It is able to string words together in a way that sounds right, but the words don&amp;#x27;t actually mean anything.&lt;p&gt;Plot twist: This comment was generated by GPT-3 prompted with some of the comments in this thread.</text></item><item><author>meredydd</author><text>I am deeply enjoying this comment thread - it&amp;#x27;s a bit of a Barium Meal [0] for determining how many people read (a) the headline, (b) the first paragraph, or (c) the whole thing before jumping straight into the compose box.&lt;p&gt;Having read to the bottom, the quality of text generation there absolutely blew me away. GPT-2 texts have a somewhat disconnected quality - &amp;quot;it only makes sense if you&amp;#x27;re not really paying attention&amp;quot; - that this article lacks entirely. Adjacent sentences and even paragraphs are plausible neighbours. Even on re-reading more closely, it doesn&amp;#x27;t feel like the world&amp;#x27;s best writing, but I don&amp;#x27;t notice major loss of coherence until the last couple of paragraphs. I am now &lt;i&gt;really&lt;/i&gt; curious about the other 9 attempts that were thrown away. Are they always this good?!&lt;p&gt;[0] &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;en.wikipedia.org&amp;#x2F;wiki&amp;#x2F;Canary_trap#Barium_meal_test&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;en.wikipedia.org&amp;#x2F;wiki&amp;#x2F;Canary_trap#Barium_meal_test&lt;/a&gt;</text></item></parent_chain><comment><author>thewarrior</author><text>It&amp;#x27;s not hard to imagine GPT-3 writing a fake paper that gets published in certain academic journals in the social sciences. And then, it&amp;#x27;ll be all over for us. We won&amp;#x27;t have any more funding and our jobs will disappear. I can already hear the protests: &amp;quot;But we&amp;#x27;re not just scientists! We&amp;#x27;re also philosophers!&amp;quot; Well, yes and no. Philosophers are supposed to think about things philosophically, but they don&amp;#x27;t actually do anything about them; they&amp;#x27;re just entertainers. Scientists do something about them. They make things happen. And when those things happen, people take notice. If science and technology have a weakness, it&amp;#x27;s that they work too well. This was probably a strength at one point, but not anymore. In the not-too-distant future, there probably won&amp;#x27;t be any more philosophy professors; there will just be philosophers. But only in the same sense that there are lions and mushrooms.&lt;p&gt;This comment was also written by GPT-3.</text></comment>
<story><title>OpenAI&apos;s GPT-3 may be the biggest thing since Bitcoin</title><url>https://maraoz.com/2020/07/18/openai-gpt3/</url></story><parent_chain><item><author>cs702</author><text>The thing that kills me is that to the &lt;i&gt;vast&lt;/i&gt; majority of human beings the nonsensical technobabble above is probably indistinguishable from real, honest, logically consistent technobabble.[a]&lt;p&gt;Soon enough, someone will replicate the Sokal hoax[b] with GPT-3 or another state-of-the-art language-generation model. It&amp;#x27;s not hard to imagine GPT-3 writing a fake paper that gets published in certain academic journals in the social sciences.&lt;p&gt;[a] &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;en.wikipedia.org&amp;#x2F;wiki&amp;#x2F;Technobabble&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;en.wikipedia.org&amp;#x2F;wiki&amp;#x2F;Technobabble&lt;/a&gt;&lt;p&gt;[b] &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;en.wikipedia.org&amp;#x2F;wiki&amp;#x2F;Sokal_affair&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;en.wikipedia.org&amp;#x2F;wiki&amp;#x2F;Sokal_affair&lt;/a&gt; -- here&amp;#x27;s a copy of Sokal&amp;#x27;s hoax paper, &amp;quot;Transgressing the Boundaries: Towards a Transformative Hermeneutics of Quantum Gravity:&amp;quot; &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;physics.nyu.edu&amp;#x2F;faculty&amp;#x2F;sokal&amp;#x2F;transgress_v2&amp;#x2F;transgress_v2_singlefile.html&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;physics.nyu.edu&amp;#x2F;faculty&amp;#x2F;sokal&amp;#x2F;transgress_v2&amp;#x2F;transgre...&lt;/a&gt;</text></item><item><author>thewarrior</author><text>I&amp;#x27;ve started working on a version of GPT-2 which generates English text. The purpose of this is to improve its ability to predict the next character in a text, by having it learn &amp;#x27;grammatical rules&amp;#x27; for English. It already works well for predicting the next character when it has seen only a small amount of text, but becomes less accurate as the amount of training text increases.
I have managed to improve this by having it generate text. That is, it creates an &amp;#x27;original&amp;#x27; piece of text about &amp;#x27;topic x&amp;#x27;, then a slightly altered version of this text where one sentence has a single word changed, and this process is repeated many times (about a million). It seems to quickly learn how to vary sentences in a way that seems natural and realistic. I think the reason this works is because it reduces the chance that the grammar it has learned for one specific topic (e.g. snow) will accidentally be transferred to another topic (e.g. dogs).
Of course, this all means nothing unless it actually learns something from the process of generating text. I haven&amp;#x27;t tried this yet, but the plan is to have it generate text about a topic, then have a second GPT-2 system try to guess what that topic is. If the resulting system is noticeably better at this task, then we know the process has increased its ability to generalize.&lt;p&gt;One potential issue with this approach is that the text it generates is &amp;#x27;nonsensical&amp;#x27;, in that it is almost like a word-salad. Although this is a standard problem with neural nets (and other machine learning algorithms), in this case the text actually is a word-salad. It seems that it has learned the rules of grammar, but not the meaning of words. It is able to string words together in a way that sounds right, but the words don&amp;#x27;t actually mean anything.&lt;p&gt;Plot twist: This comment was generated by GPT-3 prompted with some of the comments in this thread.</text></item><item><author>meredydd</author><text>I am deeply enjoying this comment thread - it&amp;#x27;s a bit of a Barium Meal [0] for determining how many people read (a) the headline, (b) the first paragraph, or (c) the whole thing before jumping straight into the compose box.&lt;p&gt;Having read to the bottom, the quality of text generation there absolutely blew me away. GPT-2 texts have a somewhat disconnected quality - &amp;quot;it only makes sense if you&amp;#x27;re not really paying attention&amp;quot; - that this article lacks entirely. Adjacent sentences and even paragraphs are plausible neighbours. Even on re-reading more closely, it doesn&amp;#x27;t feel like the world&amp;#x27;s best writing, but I don&amp;#x27;t notice major loss of coherence until the last couple of paragraphs. I am now &lt;i&gt;really&lt;/i&gt; curious about the other 9 attempts that were thrown away. Are they always this good?!&lt;p&gt;[0] &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;en.wikipedia.org&amp;#x2F;wiki&amp;#x2F;Canary_trap#Barium_meal_test&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;en.wikipedia.org&amp;#x2F;wiki&amp;#x2F;Canary_trap#Barium_meal_test&lt;/a&gt;</text></item></parent_chain><comment><author>thewarrior</author><text>The thing is that this thing has now crossed into the uncanny valley. Earlier it would have great trouble making a single sentence that makes sense. You only ever remember whether the last two sentences made sense and go together. And with GPT-3 any pair of sentences always makes almost perfect sense. By the time you&amp;#x27;re four sentences down you go wait a minute ...</text></comment>