How does turnitin detect AI?
Turnitin splits your paper into segments and scores each on how predictable the word choices are — the same statistical property AI writing has. It matches nothing against a database. Turnitin won't say exactly what patterns it uses, which Vanderbilt cited as a reason for disabling the tool in August 2023.
Why — the first-principles explanation
Start with how the AI writes, because the detector is built as its mirror. A language model produces text by asking, at every single word, "what comes next?" and choosing among the most likely options. Do that thousands of times and the result has a measurable fingerprint: it is unusually predictable. Given the first half of any sentence, a model can guess the second half more easily than it could for a human writer, who wanders, picks odd words, and builds strange sentences.
That measure has a name — perplexity. Low perplexity means "a model finds this text unsurprising." A related measure, burstiness, looks at variation: humans mix a 4-word sentence with a 40-word one, while generated text tends toward even rhythm. Turnitin's detector breaks a submission into segments, scores each one on this kind of predictability, classifies each as AI-like or not, and reports the percentage of the document that fell on the AI side.
Now the flaw falls straight out of the mechanism. Perplexity measures the writing, not the writer. Prof. James Zou of Stanford put it plainly: perplexity correlates with the sophistication of the writing — so anyone who writes plainly, with a smaller vocabulary and simpler constructions, gets a low score no matter who they are. That includes non-native English speakers, students taught rigid essay templates, and technical or legal prose. The tool isn't detecting a machine. It's detecting predictability, and then assuming predictability implies a machine.
Turnitin does not publish which patterns it keys on, saying only that it looks for patterns common in AI writing. Vanderbilt named that opacity — alongside false positives and bias against non-native speakers — when it turned the detector off. You cannot cross-examine a claim whose reasoning is secret.
An example that makes it click
Play the game where you guess the end of a sentence. "The cat sat on the ___." Everyone says mat. Easy — low perplexity. Now: "The cat sat on the tax attorney's ___." No idea. That's a human, being surprising.
A detector is basically playing this guessing game against your whole essay, and if it keeps guessing right, it decides a machine wrote it. But think about who else always says "mat." Someone still learning English, who reaches for the common word because it's the one they're sure of. Someone taught to write clear, simple sentences. Someone writing a lab report, where you're supposed to say the boring word. They all keep saying mat. And the guard at the door keeps stopping them.
How to do it
- Turnitin segments the submission into chunks of prose (it requires long-form text — very short submissions aren't scored).
- Each segment is scored for how predictable its word choices are to a language model — the perplexity signal.
- Each segment is classified as AI-like or human-like based on where it falls against learned thresholds.
- The AI-like segments are totaled and reported as a percentage of the document in the instructor's view.
- Turnitin flags low scores as less reliable and applies extra caution below the 20% range, where false positives concentrate.
- No source document is ever produced, because none exists — unlike a plagiarism match, there is nothing to click through to.
Key facts
- Detectors including Turnitin's rely on perplexity — how surprising word choices are — which Stanford's Prof. James Zou notes correlates with writing sophistication, not authorship.
- Turnitin gives no detailed public explanation of how its detector decides, saying only that it looks for 'patterns common in AI writing' without defining them (Vanderbilt University).
- Turnitin states a document-level false positive rate under 1% for documents with more than 20% AI writing, and accepts missing roughly 15% of AI text to hold that rate.
- Turnitin released the feature in April 2023 with less than 24 hours' advance notice to customers.
- Seven detectors using this approach flagged 61.22% of TOEFL essays by non-native speakers as AI while being near-perfect on U.S.-born eighth-graders (Liang et al., Patterns, 2023).
▶ The 60-second explainer (script)
How does Turnitin actually detect AI? It plays a guessing game with your essay. Here's the idea. AI writes by asking, at every word, what comes next — and picking a likely answer. Thousands of times. So the result is unusually predictable. There's a name for that: perplexity. Low perplexity means a model finds your text unsurprising. Turnitin chops your paper into segments, scores each on how predictable the wording is, and reports what percentage looks AI-like. Note what it does not do: it matches nothing against a database. With plagiarism, there's a source you can click. Here, there's no source, because none exists. Now the flaw. Perplexity measures the writing, not the writer. Stanford's James Zou points out it tracks how sophisticated the prose is. So who writes predictably? Someone still learning English. Someone taught a rigid essay template. Someone writing a lab report where the boring word is the correct word. The tool isn't finding machines. It's finding plain writing — and calling it a machine. Turnitin won't say which patterns it uses. Vanderbilt named that secrecy when it switched the detector off.
What authoritative sources say
People also ask
Does Turnitin compare my paper to a database of AI writing?
No. No such database is possible — the same prompt yields different text each time. It scores statistical predictability instead, which is why there's never a source to click.
What is perplexity in plain English?
How surprised a language model is by your next word. Predictable writing has low perplexity. AI has low perplexity — but so does plain, simple, or second-language writing.
Why won't Turnitin explain its method?
It treats the specifics as proprietary, saying only that it looks for patterns common in AI writing. Vanderbilt listed that lack of transparency among its reasons for disabling the tool.
Does it work on very short submissions?
Poorly. Statistical signals need volume, and detectors across the board are unreliable on short text — OpenAI's own classifier was called very unreliable below 1,000 characters.