The rapid progress of large language models (LLMs)  has made them capable of performing astonishingly well on various tasks including document completion and question answering. The unregulated use of these models, however, can potentially lead to malicious consequences such as plagiarism, generating fake news, spamming, etc. Therefore, reliable detection of AI-generated text can be critical to ensure the responsible use of LLMs. Recent works attempt to tackle this problem either using certain model signatures present in the generated text outputs or by applying watermarking techniques that imprint specific patterns onto them. In this paper, both empirically and theoretically, we show that these detectors are not reliable in practical scenarios. Empirically, we show thatparaphrasing attacks, where a light paraphraser is applied on top of the generative text model, can break a whole range of detectors, including the ones using the watermarking schemes as well as neural network-based detectors and zero-shot classifiers.
Our experiments demonstrate that retrieval-based detectors, designed to evade paraphrasing attacks, are still vulnerable againstrecursiveparaphrasing. We then provide a theoreticalimpossibility resultindicating that as language models become more sophisticated and better at emulating human text, the performance of even the best-possible detector decreases.
For a sufficiently advanced language model seeking to imitate human text, even the best-possible detector may only perform marginally better than a random classifier.
Our result is general enough to capture specific scenarios such as particular writing styles, clever prompt design, or text paraphrasing.
We also extend the impossibility result to include the case wherepseudorandomnumber generators are used for AI-text generation instead of true randomness.
We show that the same result holds with a negligible correction term for all polynomial-time computable detectors.
Finally, we show that even LLMs protected by watermarking schemes can be vulnerable againstspoofing attackswhereadversarial humanscan infer hidden LLM text signatures and add them to human-generated text to be detected as text generated by the LLMs, potentially causing reputational damage to their developers. We believe these results can open an honest conversation in the community regarding the ethical and reliable use of AI-generated text. Our code is publicly available athttps://github.com/vinusankars/Reliability-of-AI-text-detectors.