<story><title>33-46% of workers on MTurk used LLMs in a text production task</title><url>https://arxiv.org/abs/2306.07899</url></story><parent_chain><item><author>AndrewKemendo</author><text>The Turing Syndrome speeds up!&lt;p&gt;Turing Syndrome (Like the Kessler syndrome) is when the amount of AI generated data on the internet surpasses human generated content to the extent that it will eventually make it impossible to distinguish between the two</text></item></parent_chain><comment><author>somenameforme</author><text>I think this is probably a poor metric for it. Mechanical Turk is full of people trying to race their way through surveys to make a few pennies. So there&amp;#x27;s a motivation to just create some output that can&amp;#x27;t be automatically filtered out, like missing a &amp;#x27;please select D as the answer for this question&amp;#x27; attention test, while otherwise just going as quickly as possible. It&amp;#x27;s a dream scenario for any sort of automation, no matter how crude. It&amp;#x27;s one step above MMO gold farming.&lt;p&gt;Tangentially related, Mechanical Turk and its various less well known clones which come down to the exact same thing, are increasingly the status quo for social science studies. They keep making really shocking discoveries with like 99.99999% statistical certainty given the huge sample sizes these services enable. Kind of weird how often they fail to replicate.</text></comment>
<story><title>33-46% of workers on MTurk used LLMs in a text production task</title><url>https://arxiv.org/abs/2306.07899</url></story><parent_chain><item><author>AndrewKemendo</author><text>The Turing Syndrome speeds up!&lt;p&gt;Turing Syndrome (Like the Kessler syndrome) is when the amount of AI generated data on the internet surpasses human generated content to the extent that it will eventually make it impossible to distinguish between the two</text></item></parent_chain><comment><author>z3c0</author><text>Or maybe the complete opposite, as it will only be learning patterns created by AI, producing less variance at higher temperatures, ultimately overfitting onto the same bland jargon.</text></comment>