Can AI-Generated Text be Reliably Detected?
Vinu Sankar Sadasivan
vinu@umd.eduAounon Kumar
aounon@umd.edu
Sriram Balasubramanian
Can AI-Generated Text be Reliably Detected?
In this paper, both empirically and theoretically, we show that several
AI-text detectors are not reliable in practical scenarios. Empirically, we show
that paraphrasing attacks, where a light paraphraser is applied on top of a
large language model (LLM), can break a whole range of detectors, including
ones using watermarking schemes as well as neural network-based detectors and
zero-shot classifiers. Our experiments demonstrate that retrieval-based
detectors, designed to evade paraphrasing attacks, are still vulnerable to
recursive paraphrasing. We then provide a theoretical impossibility result
indicating that as language models become more sophisticated and better at
emulating human text, the performance of even the best-possible detector
decreases. For a sufficiently advanced language model seeking to imitate human
text, even the best-possible detector may only perform marginally better than a
random classifier. Our result is general enough to capture specific scenarios
such as particular writing styles, clever prompt design, or text paraphrasing.
We also extend the impossibility result to include the case where pseudorandom
number generators are used for AI-text generation instead of true randomness.
We show that the same result holds with a negligible correction term for all
polynomial-time computable detectors. Finally, we show that even LLMs protected
by watermarking schemes can be vulnerable against spoofing attacks where
adversarial humans can infer hidden LLM text signatures and add them to
human-generated text to be detected as text generated by the LLMs, potentially
causing reputational damage to their developers. We believe these results can
open an honest conversation in the community regarding the ethical and reliable
use of AI-generated text.
Vinu Sankar Sadasivan Aounon Kumar Sriram Balasubramanian Wenxiao Wang Soheil Feizi