AI Detectors: How They Work and Why They Get It Wrong
AI detectors are the most confidently marketed and least reliable category of tool in this whole field. They are used to decide whether students pass, whether freelancers get paid, and whether writers keep clients — and the published research does not support that level of trust. This is a straight answer to what they actually measure, what the accuracy percentages on their marketing pages mean, who bears the cost when they are wrong, and what to do if a detector says something you wrote is machine-generated. It is deliberately not a recommendation of which one to buy.
What an AI detector actually measures
No detector can see where text came from. There is no watermark in ordinary AI output, no metadata, nothing to look up. What a detector does is measure statistical properties of the writing and compare them against what it expects from human and machine text. The two properties usually named are perplexity — roughly, how surprising each word is given the ones before it — and burstiness, the variation in sentence length and complexity across a passage. Machine-generated text tends to be smoother on both: it picks likely words and it varies less. A detector reports how machine-like the statistics look, and a probability is then presented to you as a verdict. That gap between what is measured and what is claimed is where all the trouble lives.
Pro Tip: Ask any detector what it measures rather than how accurate it is. A tool that will not answer the first question is not in a position to answer the second.
Why the accuracy claims do not hold up
Detector marketing quotes accuracy figures in the high nineties. Those numbers come from the vendor's own testing, on text that is either plainly human or plainly machine-generated, which is the easy case and not the case anyone needs help with. Independent work is much less flattering. Can AI-Generated Text be Reliably Detected? argues that detection degrades sharply once text is lightly paraphrased, and that there are fundamental limits on how well any detector can perform as models improve. In practice the two failure modes matter differently: a false negative means machine text goes unflagged, which is an inconvenience, while a false positive accuses a person of something they did not do. Detectors are usually tuned to catch more, which means more of the second.
Pro Tip: Whenever an accuracy figure is quoted, ask what the false-positive rate was and on what kind of text. An overall accuracy number hides the error that actually harms someone.
Who false positives fall on
This is not evenly distributed, and that is the most important thing to know. GPT detectors are biased against non-native English writers found that detectors misclassified writing by non-native English speakers as machine-generated at strikingly high rates, because the statistical markers detectors look for — simpler vocabulary, more regular sentence construction, lower variation — are also characteristics of someone writing carefully in a second language. The same logic catches other groups: autistic writers, people writing in a formal register they were taught, anyone who edits heavily toward clarity, and anyone using grammar assistance. In other words the people most likely to be wrongly accused are disproportionately the ones least able to argue back.
Pro Tip: If you administer a detector, look specifically at who your flags fall on. An overall flag rate tells you nothing about whether the tool is working fairly.
Why paraphrasing defeats them
The statistical signal a detector reads is fragile. Running machine text through a rewriting tool, or editing it substantially by hand, changes the word-choice distribution enough that the score usually moves toward human. That is the mechanism behind the entire market of tools promising to make AI text undetectable, and it is why an arms race between detection and evasion is not one detection wins — the detector must find a signal, and the evader only has to disturb it. The practical consequence for anyone relying on a detector is uncomfortable: it is most likely to flag text that was not edited, which means it catches the careless rather than the dishonest.
Pro Tip: A detector result tells you more about how much a text was edited than about who wrote it. Those are not the same question.
What detectors are legitimately useful for
There is a narrow band where they are genuinely worth running. As a private signal on your own work, a detector can tell you that a draft still reads as unedited machine output, which is a useful quality prompt even though it is not evidence of anything. At scale, on content you commissioned, a high flag rate across a batch is a reasonable trigger to look more closely at a supplier — not a finding, but a place to point attention. What they are not is a basis for a decision about an individual. The distinction that matters is between a signal that starts an enquiry and a verdict that ends one, and detectors are only ever the first.
Pro Tip: Use a detector on your own drafts before publishing. Treating it as an editing prompt rather than a verdict is the one use with no downside.
If you are accused, and you did write it
This happens often enough to plan for. Ask which tool was used and what score it produced, because a specific number is much easier to contest than an impression. Point out that the tool reports a statistical likelihood rather than evidence of authorship, and that its false positives are documented and unevenly distributed — the two papers linked above are exactly what to cite. Then show your process: draft history, version history in a document, notes, search history, the earlier outline. Most institutions accept process evidence, and it is far more persuasive than arguing about the score. Ask, politely and in writing, what the policy says about detector evidence, because many policies are more cautious than the person applying them.
Pro Tip: Turn on version history in whatever you write in, now, before you need it. A document with a visible three-hour edit history is the strongest answer to this accusation there is.
If you set policy, what to do instead
Detectors are attractive because the alternative is harder, but the alternatives work. Assess process rather than only product: drafts, outlines, annotated bibliographies, a short conversation about the work. Design tasks that are difficult to generate usefully — ones referring to a specific seminar discussion, a local dataset, or the student's own earlier work. State clearly what use is permitted rather than banning a category nobody can police, since a rule that cannot be enforced fairly teaches only that rules are arbitrary. And if you use a detector at all, make it explicit that a score alone never constitutes a finding, and write that into the policy rather than trusting each marker to apply it.
Pro Tip: Write the permitted-use rule as something a student could follow with confidence. Most academic-integrity disputes over AI come from policies that never said what was allowed.
Detection versus plagiarism checking
These get conflated constantly and they answer completely different questions. A plagiarism checker compares text against indexed sources and reports overlap; you can open the match and see it, which makes it evidence a person can evaluate. An AI detector compares statistical properties against a model's expectations and reports a probability; there is nothing to open. One is a document you can read, the other is a number you have to trust. If you need a check you can defend, use an originality check and read the passages — [our guide to running one properly](/guides/ai-plagiarism-detection) covers reading the report rather than the score.
Pro Tip: If a decision would be hard to justify without showing someone the evidence, use a tool that produces evidence. Detectors produce a number and nothing else.
Watermarking, and why it has not solved this
There is a technically better answer than statistical detection: have the model itself embed a signal at generation time, biasing word choice in a pattern a verifier can recognise. This works, and it is far more reliable than inferring from statistics after the fact. The obstacles are practical rather than theoretical. It requires the model provider to implement it, survives paraphrasing poorly, does nothing about open-weight models anyone can run unmodified, and needs an agreed verification route. Some providers have researched or shipped it in limited forms. Until it is universal and robust, it does not change the position for anyone deciding whether a specific piece of text was written by a person.
Pro Tip: Treat any claim that a detector uses watermarking with care — it only applies to output from models that actually embed one, which is a small share of what people use.
The honest summary
AI detectors measure a statistical fingerprint that correlates with machine generation, report it as a probability, and are routinely presented as though they answer a question about authorship. They are wrong often enough, and unevenly enough, that a detector score should never by itself decide anything about a person. They are reasonably useful as a private editing prompt and as a signal that something in a large batch deserves a look. That is the whole defensible use. Anyone selling more than that is selling certainty that the research does not support, and the people who pay for that certainty are mostly not the ones who bought the tool.
Pro Tip: The one-line version worth remembering: a detector tells you how a text reads, never who wrote it.
Final Thoughts
If you take one thing from this: a detector score is a signal, never a verdict, and its errors fall hardest on people writing in a second language and on anyone who writes plainly. Use one on your own drafts as an editing prompt. Do not use one to make a decision about somebody else without evidence you can show them. And if you are on the receiving end of an accusation, the strongest response is not arguing with the number — it is producing the process that made the document, which is why turning on version history today is worth more than any argument you could prepare later.
Related Posts
Build an AI Content Pipeline: Research, Fact-Check, Publish
A four-stage content pipeline — research, draft, verify, publish — with the checks between stages that stop one bad step poisoning the output.
Building a Second Brain That You Actually Use
Capture, organisation and semantic retrieval for personal notes — and the design decisions that separate a knowledge base you use from one you abandon.
AI for Beginners: What You Actually Need to Know in 2026
What AI actually is, the three kinds you will genuinely encounter, what it does well and badly, and how to start today without any technical background.
Common questions
Are AI detectors accurate?
Not reliably enough to decide anything about a person. Vendor accuracy figures come from testing on text that is plainly human or plainly machine-generated, which is the easy case. Independent research finds accuracy degrades sharply on lightly edited text, and that false positives fall disproportionately on non-native English writers.
Can an AI detector be wrong about my essay?
Yes, and it happens often. Detectors measure statistical properties - how predictable the word choices are and how much sentence length varies - not where the text came from. Careful, plain writing produces the same signals as generated text, which is why writing in a second language, writing to a formal template, or editing heavily toward clarity all raise your score.
How do I prove I wrote something myself?
Show the process rather than arguing with the score. Version history in a document, an earlier outline, notes, and drafts are what institutions actually accept. Turn version history on before you need it - reconstructing it afterwards is not possible, and it is the single strongest piece of evidence available to you.
Does running text through a humanizer beat detection?
Usually, and that is a criticism of detectors rather than an endorsement of humanizers. The signal detectors read is fragile, so substituting words and varying sentence length disturbs it. The consequence is that detectors are most likely to flag unedited text, which means they catch carelessness rather than dishonesty.
Is an AI detector the same as a plagiarism checker?
No, and conflating them causes real problems. A plagiarism checker compares your text against indexed sources and shows you the matching passage, so you can evaluate the evidence yourself. An AI detector produces a probability with nothing to inspect. One yields evidence a person can weigh; the other yields a number you have to trust.
Should schools use AI detectors?
Not as the basis for a finding. If used at all, a score should explicitly never constitute evidence on its own, and that should be written into the policy rather than left to each marker. Assessing process - drafts, outlines, a short conversation about the work - is more reliable and much harder to contest unfairly.
Related Guides
AI Plagiarism Detection Setup
Running originality checks on your own drafts, reading the report properly, and knowing what a match does and does not prove.
LearningPersonal AI Learning Path
Building a self-directed learning plan with AI — objectives, curriculum, capture, review and application — without mistaking coverage for understanding.
ContentScripting and Producing Video and Audio with AI
Research, outlines and scripts for video and podcasts, plus the production work around recording — with the recording itself left to you.