<story><title>Research shows we can only accurately identify AI writers about 50% of the time</title><url>https://hai.stanford.edu/news/was-written-human-or-ai-tsu</url></story><parent_chain><item><author>williamtrask</author><text>Detecting whether something is written by an AI is a waste of time. Either someone will sign the statement as their own or they won&amp;#x27;t (and it should be treated as nonsense).&lt;p&gt;People lie. People tell the truth. Machines lie. Machines tell the truth. I bet our ability to detect when a person is lieing isn&amp;#x27;t any better than 50% either.&lt;p&gt;What matters is accountability, not method of generation.</text></item></parent_chain><comment><author>tveita</author><text>&lt;p&gt;&lt;pre&gt;&lt;code&gt;  In relevant studies, people attempt to discriminate lies from truths in real time with no special aids or training. In these circumstances, people achieve an average of 54% correct lie-truth judgments, correctly classifying 47% of lies as deceptive and 61% of truths as nondeceptive. [1]
&lt;/code&gt;&lt;/pre&gt;
What I think people miss are all the mechanisms we&amp;#x27;ve evolved to prevent people from lying, so we can live effectively in a high-trust society, from built-in biological tendencies, to how we&amp;#x27;re raised, to societal pressures.&lt;p&gt;&amp;quot;People lie too&amp;quot; but in 95% of cases they don&amp;#x27;t. If someone on Hacker News say they prefer Zig to Rust or that they liked the Dune movie, they&amp;#x27;re likely telling the truth. There&amp;#x27;s no incentive either way, we&amp;#x27;ve just evolved as social creatures that share little bits of information and reputation. And to lie, yes, and to expose the lies of others, but only when there&amp;#x27;s a big payoff to defect.&lt;p&gt;If you had a friend that kept telling you about their trips to restaurants that didn&amp;#x27;t actually exist, or a junior developer at work that made up fictional APIs when they didn&amp;#x27;t know the answer to a question, you&amp;#x27;d tell them to stop, and if they kept at it you probably wouldn&amp;#x27;t care to hang out with them. ChatGPT seems to bypass those natural defenses for now.&lt;p&gt;Most people think they are hard to deceive. But I see plenty people here on HN with confidently wrong beliefs about how ChatGPT works, that they&amp;#x27;ve gotten from asking ChatGPT about itself. It&amp;#x27;s not intuitive for us that ChatGPT actually knows very little about how itself works. It even took humanity a while to realize that &amp;quot;How does it feel like my body works&amp;quot; isn&amp;#x27;t a great way to figure out biology.&lt;p&gt;[1] &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;journals.sagepub.com&amp;#x2F;doi&amp;#x2F;abs&amp;#x2F;10.1207&amp;#x2F;s15327957pspr1003_2&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;journals.sagepub.com&amp;#x2F;doi&amp;#x2F;abs&amp;#x2F;10.1207&amp;#x2F;s15327957pspr10...&lt;/a&gt;</text></comment>
<story><title>Research shows we can only accurately identify AI writers about 50% of the time</title><url>https://hai.stanford.edu/news/was-written-human-or-ai-tsu</url></story><parent_chain><item><author>williamtrask</author><text>Detecting whether something is written by an AI is a waste of time. Either someone will sign the statement as their own or they won&amp;#x27;t (and it should be treated as nonsense).&lt;p&gt;People lie. People tell the truth. Machines lie. Machines tell the truth. I bet our ability to detect when a person is lieing isn&amp;#x27;t any better than 50% either.&lt;p&gt;What matters is accountability, not method of generation.</text></item></parent_chain><comment><author>Veen</author><text>People believe lies, often. That&amp;#x27;s just an undeniable fact of human nature. AIs can produce lots of plausible lies very quickly, much more quickly and at much greater scale than humans could. There&amp;#x27;s a quantitative difference that will have a real impact on the world. Sure, we could have humans attest to and digitally sign their content, but I&amp;#x27;m not sure that&amp;#x27;s likely to work at scale, and people will be motivated to lie about that too—and there&amp;#x27;s no way to prove they are lying.</text></comment>