How does AI paraphrasing detection work?
It measures predictability, not meaning. Detectors score how surprising each word is given the ones before it — AI text is unusually unsurprising. Paraphrasing tools are language models too, so they leave the same flat fingerprint. The catch: this method flagged 61.22% of essays by non-native English writers as AI.
Why — the first-principles explanation
Paraphrase detection can't work by comparing your text to a database, because paraphrased text matches nothing — that's the point of paraphrasing. So it does something else entirely: it looks at the statistical texture of the writing itself.
Here's the mechanism. A language model can read any sentence and say how probable each word was, given everything before it. Average that surprise and you get perplexity. Human writing is bumpy — we choose odd words, we take a weird turn mid-sentence, we're inconsistent. AI writing is smooth, because a model generating text picks high-probability words by design. That's not a bug; it's the whole objective. So AI prose ends up unusually unsurprising, and low perplexity is the signal detectors hunt for. There's a companion measure, burstiness — humans vary sentence length wildly, mixing a three-word sentence with a forty-word one, while models produce eerily even rhythm.
Now, why paraphrasing doesn't rescue you. Run AI text through a paraphrasing tool and you've changed the words but not who chose them — the paraphraser is itself a language model, so it also picks high-probability words. You've swapped one flat fingerprint for another flat fingerprint. Detectors that advertise "paraphrase detection" are exploiting exactly this: spun text is often more machine-like than the original, because it's been through two models in a row.
And here's the part that should stop anyone from treating these scores as evidence. The method has no concept of who wrote something — only of how predictable it is. So it doesn't detect cheating. It detects plain, careful, limited-vocabulary writing. Which is what a second-language writer produces. In a study of seven detectors, 61.22% of TOEFL essays by non-native English speakers were flagged as AI-generated — 97.80% flagged by at least one detector — while the same tools were near-perfect on U.S. eighth-graders. The flagged essays had significantly lower perplexity. The tool worked exactly as designed. It's a fluency meter that everyone mistakes for a lie detector.
The symmetry is brutal: the same study asked ChatGPT to elevate the word choices in those TOEFL essays, and the false positive rate fell from 61.22% to 11.77%. Using AI made the human essays look more human.
An example that makes it click
Imagine trying to catch a forger by measuring how neat their handwriting is. Your theory: machines write in perfectly even strokes, humans are messy. It sort of works — printed pages do look different from scrawl.
But now think about who gets accused. The careful student with beautiful, deliberate penmanship. The person taught calligraphy. The writer who's nervous and writing slowly on purpose. Meanwhile the forger who deliberately adds a shake to his hand sails right through.
That's a perplexity detector. It's not measuring honesty — it's measuring evenness. And the people whose writing is naturally the most even are the ones being careful with a language they learned second. Sixty-one percent of them got flagged. The tool did its job perfectly. Its job was just never the thing anybody thought it was.
Key facts
- Detectors score text by perplexity — how predictable each word is given the preceding words — because AI-generated text is unusually unsurprising by design (Liang et al., 2023).
- Seven GPT detectors flagged 61.22% of 91 TOEFL essays by non-native English writers as AI-generated; 97.80% were flagged by at least one detector and 19.78% by all seven — every essay was human-written.
- The same detectors were near-perfect on essays by U.S. eighth-graders — the failure is concentrated by population, not spread evenly.
- Essays unanimously misclassified as AI had significantly lower perplexity than others (p = 9.74E-05), confirming that the detectors penalize limited linguistic expression rather than detect authorship.
- Prompting ChatGPT to elevate word choices in the TOEFL essays cut the average false positive rate from 61.22% to 11.77%, and unanimous misclassification fell to 1 of 91 essays (1.10%).
- OpenAI discontinued its own AI Text Classifier on July 20, 2023, citing a low rate of accuracy.
▶ The 60-second explainer (script)
How does AI paraphrasing detection work? It doesn't read your meaning. It measures how predictable your words are. Here's the mechanism. A language model can look at any sentence and tell you how surprising each word was, given the words before it. Average that and you get perplexity. Humans are bumpy — we pick odd words, we take weird turns, we're inconsistent. AI is smooth, because a model generating text picks likely words on purpose. That's its entire design. So AI writing comes out unusually unsurprising, and that flatness is the fingerprint detectors hunt. Now — why doesn't paraphrasing save you? Because a paraphrasing tool is also a language model. You changed the words, but not who's choosing them. It also picks high-probability words. You've swapped one flat fingerprint for another. In fact spun text is often more machine-like than the original, because it's been through two models. And here's the part that matters. This method has no idea who wrote anything. It only knows how predictable the text is. So it doesn't detect cheating — it detects plain, careful, limited-vocabulary writing. Which is exactly what a second-language writer produces. Seven detectors flagged sixty-one percent of TOEFL essays as AI. All human-written. The same tools were near-perfect on American eighth-graders. And the kicker? Researchers asked ChatGPT to fancy up those essays, and the false positives dropped to eleven percent. Using AI made the human writing look more human. It's a fluency meter that everyone mistakes for a lie detector.
What authoritative sources say
People also ask
Can detectors tell the difference between paraphrased AI and human writing?
Not reliably. Both get scored on predictability, and a paraphraser is a language model too. It swaps one machine fingerprint for another rather than removing it.
Why does my own writing get flagged?
Because you write plainly or carefully. Low perplexity means predictable word choice, which is what formal, simple, or second-language writing looks like. The tool can't distinguish that from AI.
Does adding typos help?
It raises perplexity, so mechanically yes — which tells you how little the score means. A method defeated by deliberate sloppiness isn't measuring honesty.
Is a high AI score proof of cheating?
No. It's a statistical property of the text, not evidence of authorship. OpenAI retired its own detector for low accuracy, and independent research shows the errors land hardest on non-native writers.
Do detectors improve as AI improves?
The gap narrows against them. As models get better at mimicking human variation, the flatness they rely on disappears. The signal they detect is the thing generators are actively learning to remove.