How Do AI Detectors Work? Perplexity, Burstiness & More
Key takeaways
- AI detectors estimate how 'predictable' text is, because language models favor likely words.
- The two classic signals are low perplexity and low burstiness.
- Many detectors are machine-learning classifiers trained on human and AI examples.
- Watermarking could make detection far more reliable, but only for models that apply it.
The core idea: AI text is predictable
A large language model writes one token at a time, each time picking from a ranked list of likely next words. Even with some randomness, it gravitates toward the probable. Human writers don’t work that way. We pick odd words, change direction mid-thought, write a three-word sentence after a forty-word one. AI detectors work by measuring how closely a text matches the patterns a language model would produce, and converting that into a probability.
Signal 1: Perplexity
Perplexity is a measure of how surprised a language model is by a text. To compute it, a detector feeds the text through a model and checks, word by word, how likely the model thought each actual word was. If the model keeps predicting the words correctly, perplexity is low, and the text looks machine-like. If the words keep surprising it, perplexity is high, and the text looks human.
Perplexity is intuitive and fast, but it has a weakness: plenty of human writing is predictable too. Legal boilerplate, textbook definitions and essays by English learners can all have low perplexity, which is a major source of false positives (see are AI detectors accurate?).
Signal 2: Burstiness
Burstiness captures variation. Human writing is “bursty”: sentence length and complexity rise and fall, and perplexity spikes in some sentences and dips in others. AI text tends to be smoother, with sentences of similar length and structure throughout. Detectors such as GPTZero popularized burstiness alongside perplexity as a paired signal.
A simple proxy is the coefficient of variation of sentence length: the standard deviation divided by the mean. AI Humanizer Detector shows this as “sentence variation” in every report. Values below roughly 0.35 usually indicate very even, machine-like rhythm.
Method 3: Trained classifiers
Many commercial detectors are classifiers: machine-learning models trained on large collections of text labelled human or AI. Given new text, the classifier outputs a probability for each class. An early public example was the GPT-2 output detector that OpenAI released in 2019, built on the RoBERTa model.
Classifiers can learn subtle patterns that simple statistics miss, but they are only as good as their training data. A classifier trained mostly on one model’s output, one genre or one language can perform much worse on everything else, and it needs retraining as new AI models appear.
Method 4: Zero-shot detection
Zero-shot methods detect AI text without training a dedicated classifier. DetectGPT, introduced by Eric Mitchell and colleagues at Stanford in 2023, compares the probability of a text with the probabilities of slightly altered versions of it; machine text tends to sit at a local peak of probability. Binoculars, published in 2024 by Abhimanyu Hans and colleagues, compares how two related language models score the same text. These methods are clever and general, but they need access to capable models and can be computationally expensive.
Method 5: Watermarking
Watermarking takes a different approach: instead of guessing after the fact, the AI model hides a statistical signal in its output as it writes. A 2023 paper by John Kirchenbauer and colleagues showed how a model can subtly favor a secret, pseudo-random set of words that a detector can later test for. Google DeepMind has deployed a text watermark called SynthID in its Gemini products and open-sourced the text version in 2024.
Watermarks can be highly reliable, but only for text from models that apply them, and only if the detector knows the scheme. Heavy paraphrasing can also weaken them. Most AI text in circulation today carries no watermark that public detectors can check.
Why every method has limits
| Method | Strength | Weakness |
|---|---|---|
| Perplexity | Simple, fast, model-agnostic | Flags predictable human writing |
| Burstiness | Easy to understand and display | Weak alone; easy to change with editing |
| Classifiers | Learns subtle patterns | Depends on training data; needs retraining |
| Zero-shot | No training data needed | Compute-heavy; sensitive to paraphrasing |
| Watermarking | Can be very reliable | Only works if the generating model applied it |
The common thread is that all except watermarking infer authorship from style. Style can be edited, and some humans naturally write in “AI-like” styles. That’s the fundamental reason detection will always be probabilistic.
How our AI detector works
Our AI detector combines two of these approaches. A large language model judges each sentence and the passage as a whole, acting like an expert reader looking for generated-text hallmarks. In parallel, a stylometric layer measures burstiness, vocabulary diversity, contraction use and the density of phrases language models overuse. The results are blended, short passages are pulled toward an uncertain score, and every flagged sentence is highlighted so you can see what drove the result.
See perplexity-style signals in action. Paste any text into our free detector.
Open AI detector