<story><title>OpenAI Trains Language Model, Mass Hysteria Ensues</title><url>http://approximatelycorrect.com/2019/02/17/openai-trains-language-model-mass-hysteria-ensues/</url></story><parent_chain><item><author>ilyasut</author><text>Ilya from OpenAI here.  Here&amp;#x27;s our thinking:&lt;p&gt;- ML is getting more powerful and will continue to do so as time goes by.   While this point of view is not unanimously held by the AI community, it is also not particularly controversial.&lt;p&gt;- If you accept the above, then the current AI norm of &amp;quot;publish everything always&amp;quot; will have to change&lt;p&gt;- The _whole point_ is that our model is not special and that other people can reproduce and improve upon what we did.  We hope that when they do so, they too will reflect about the consequences of releasing their very powerful text generation models.&lt;p&gt;- I suggest going over some of the samples generated by the model.  Many people react quite strongly, e.g., &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;twitter.com&amp;#x2F;justkelly_ok&amp;#x2F;status&amp;#x2F;1096111155469180928&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;twitter.com&amp;#x2F;justkelly_ok&amp;#x2F;status&amp;#x2F;1096111155469180928&lt;/a&gt;.&lt;p&gt;- It is true that some media headlines presented our nonpublishing of the model as &amp;quot;OpenAI&amp;#x27;s model is too dangerous to be published out of world-taking-over concerns&amp;quot;.  We don&amp;#x27;t endorse this framing, and if you read our blog post (or even in most cases the actual content of the news stories), you&amp;#x27;ll see that we don&amp;#x27;t claim this at all -- we say instead that this is just an early test case, we&amp;#x27;re concerned about language models more generally, and we&amp;#x27;re running an experiment.&lt;p&gt;Finally, despite the way the news cycle has played out, and despite the degree of polarized response (and the huge range of arguments for and against our decision), we feel we made the right call, even if it wasn&amp;#x27;t an easy one to make.</text></item></parent_chain><comment><author>Cacti</author><text>&amp;gt; - The _whole point_ is that our model is not special and that other people can reproduce and improve upon what we did. We hope that when they do so, they too will reflect about the consequences of releasing their very powerful text generation models.&lt;p&gt;If this is your whole point, then I think you are missing something fundamental. Implementing these models doesn&amp;#x27;t require reflection, or introspection, or any sort of ethical or moral character whatsoever; and even if it did, all that will happen eventually is someone (without the technical background) will simply throw a lot of money at someone else (with the technical background, but who needs to, you know, eat, and pay rent, and so on) to implement it. You are fooling yourself if you think your stance makes a single mote of difference in this arms race.</text></comment>
<story><title>OpenAI Trains Language Model, Mass Hysteria Ensues</title><url>http://approximatelycorrect.com/2019/02/17/openai-trains-language-model-mass-hysteria-ensues/</url></story><parent_chain><item><author>ilyasut</author><text>Ilya from OpenAI here.  Here&amp;#x27;s our thinking:&lt;p&gt;- ML is getting more powerful and will continue to do so as time goes by.   While this point of view is not unanimously held by the AI community, it is also not particularly controversial.&lt;p&gt;- If you accept the above, then the current AI norm of &amp;quot;publish everything always&amp;quot; will have to change&lt;p&gt;- The _whole point_ is that our model is not special and that other people can reproduce and improve upon what we did.  We hope that when they do so, they too will reflect about the consequences of releasing their very powerful text generation models.&lt;p&gt;- I suggest going over some of the samples generated by the model.  Many people react quite strongly, e.g., &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;twitter.com&amp;#x2F;justkelly_ok&amp;#x2F;status&amp;#x2F;1096111155469180928&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;twitter.com&amp;#x2F;justkelly_ok&amp;#x2F;status&amp;#x2F;1096111155469180928&lt;/a&gt;.&lt;p&gt;- It is true that some media headlines presented our nonpublishing of the model as &amp;quot;OpenAI&amp;#x27;s model is too dangerous to be published out of world-taking-over concerns&amp;quot;.  We don&amp;#x27;t endorse this framing, and if you read our blog post (or even in most cases the actual content of the news stories), you&amp;#x27;ll see that we don&amp;#x27;t claim this at all -- we say instead that this is just an early test case, we&amp;#x27;re concerned about language models more generally, and we&amp;#x27;re running an experiment.&lt;p&gt;Finally, despite the way the news cycle has played out, and despite the degree of polarized response (and the huge range of arguments for and against our decision), we feel we made the right call, even if it wasn&amp;#x27;t an easy one to make.</text></item></parent_chain><comment><author>sigil</author><text>Ok but isn’t this the opposite of OpenAI’s “nukes are safer when multiple actors have them” strategy wrt AI?&lt;p&gt;I’m also confused by the threat models earnestly put forth in your blog post. Are we really concerned about deep faking someone’s writing? The plain word already demands attribution by default: we look for an avatar, a handle, a domain name to prove the person actually said this.</text></comment>