Are AI Detectors Accurate? What the Research Shows

The short answer

AI detectors are useful but not reliable enough to be used as proof. They work best on several hundred words of unedited AI output, where they often score high. They work worst on short passages, AI text that a person has edited, and certain human writing styles, where they can be confidently wrong in either direction. Every detector, free or paid, has this limitation, because it is estimating a probability from writing style rather than reading a hidden label.

What independent research found

The best evidence on AI detector accuracy comes from researchers who test many tools side by side.

OpenAI’s own classifier

In January 2023 OpenAI released a free AI text classifier. By its own figures, it correctly labelled only 26% of AI-written text as “likely AI-written” and wrongly flagged 9% of human-written text. OpenAI retired it in July 2023, citing its low rate of accuracy. If the company that built ChatGPT found reliable detection this hard, it’s a useful reminder of how difficult the problem is.

Bias against non-native English writers

A 2023 Stanford study led by Weixin Liang, published in the journal Patterns, ran essays through seven popular GPT detectors. The detectors classified essays written by US eighth-graders accurately, but misclassified more than half of the TOEFL essays written by non-native English speakers as AI-generated. The reason: non-native writers often use a smaller, more predictable vocabulary, which is exactly what many detectors read as machine-like.

Testing 14 detection tools

A 2023 study by Debora Weber-Wulff and colleagues in the International Journal for Educational Integrity tested 14 AI detection tools, including widely used commercial ones. The authors concluded that none of them was accurate or reliable enough for routine use, that the tools leaned toward classifying text as human, and that manual editing or machine paraphrasing of AI text sharply reduced detection.

Universities reconsidering detection

Following concerns about false positives, several universities, including Vanderbilt University in 2023, chose to disable Turnitin’s AI detection feature or advised staff not to rely on it. Many others now publish guidance that detector results must not be used as the only evidence in misconduct cases.

Why false positives happen

A false positive is human writing flagged as AI. It is the most damaging kind of error, and these are the usual causes:

• Short text. A paragraph or two doesn’t contain enough evidence, so random features dominate.
• Formulaic genres. Lab reports, legal writing, technical documentation and five-paragraph essays follow strict conventions that look “predictable.”
• Simple or learner vocabulary. As the Stanford research showed, a narrower vocabulary resembles low-perplexity AI text.
• Heavy grammar correction. Aggressive editing tools can smooth writing until it loses its natural variation. See can AI detectors detect Grammarly?
• Famous or widely copied text. Well-known passages, such as constitutional texts or common definitions, appear so often in training data that models find them highly predictable.

Why AI text slips through

False negatives, where AI text passes as human, are common too:

• The writer edits the AI draft, adding personal detail and varying sentences.
• The text is paraphrased or humanized, which disrupts the statistical patterns detectors rely on.
• The AI was prompted to write in an unusual style, with typos, slang or a strong persona.
• New models produce text that older detectors were never trained on.

This cat-and-mouse dynamic is why detectors keep changing, and why no humanizer can honestly guarantee a result on every detector either. Read more in do AI humanizers work?

How to use AI detectors fairly

Whether you’re a teacher, editor or manager, these practices make AI detection useful without making it unfair:

1. Check enough text. Use at least 150 to 300 words of continuous prose.
2. Read the flagged sentences, not just the percentage. Ask whether they really sound generated.
3. Compare against known writing from the same person.
4. Ask about the process. Drafts, notes and document version history are stronger evidence than any score.
5. Use more than one signal, and never treat a single detector result as a verdict.
6. Set clear AI policies in advance so people know what’s allowed and what must be disclosed.

How our AI detector handles accuracy

We built our AI detector around these limitations. It combines a language model’s judgement with independent statistical measurements, pulls scores toward the middle when a passage is too short to judge, and is explicitly instructed not to treat simple vocabulary as evidence of AI. It labels mid-range scores “Mixed or unclear” instead of forcing a verdict, and it shows you the sentences and signals behind the number, so you can judge the evidence yourself. It still makes mistakes. That’s why we show our working.