How can a teacher tell if a paper is AI generated?
Often they can't — not from the text alone. In a peer-reviewed study, AI detectors falsely flagged 61.22% of essays written by non-native English speakers, while a single rewrite prompt dropped detection of genuine AI essays to 13%. Process evidence — draft history, revision timestamps, in-class baselines — is far stronger than any detector score.
Why — the first-principles explanation
Start with what a detector actually measures, because almost every misunderstanding flows from this. It has no record of what the student did. It estimates perplexity — how surprising each next word is, given the previous ones. Language models select high-probability words, so machine text reads smooth and unsurprising. But so does a great deal of human writing: a student with a smaller working vocabulary, a student taught to write plainly, a student writing in their second language. The detector is not detecting AI. It is detecting predictable prose, and then guessing about the cause.
That guess fails in a specific direction, which is what makes it dangerous rather than merely imperfect. Liang et al., published in Patterns in 2023, ran 91 TOEFL essays by non-native English writers through seven widely-used detectors: the average false positive rate was 61.22%, and 89 of the 91 (97.80%) were flagged by at least one detector. The same tools were near-perfect on 88 US 8th-grade essays. The researchers then proved the mechanism directly: they had a model simplify the vocabulary in real human US essays, and misclassification leapt from 5.19% to 56.65%. Same human author, same human thinking — now "AI," purely because the words got plainer. Meanwhile, prompting ChatGPT to "elevate the provided text by employing literary language" cut detection of genuinely machine-written essays from 100% to 13%. So your most vulnerable students get flagged and the actual cheater walks.
The vendors themselves are more careful than their users. Turnitin claims a document-level false positive rate of "less than 1%" — but explicitly states that it "does not make a determination of misconduct," that the rate is "not zero," and that "you as the instructor will need to apply your professional judgment." Both things can be true: Turnitin's sub-1% is measured on its own test corpus at its own threshold, while independent academic testing on ESL and edited writing finds far higher error. The numbers differ because the test sets differ — a detector tuned on clean American student prose will look excellent until you hand it a TOEFL essay. That is exactly why a vendor's headline accuracy number should never be transplanted onto your classroom, where the population is not the vendor's test set.
So what does carry signal? Things a model can't fake because it wasn't there. Fabricated citations to papers that don't exist. Confident, wrong specifics about your class, your town, last week's lesson. A voice that doesn't match the student's supervised in-class writing. And above all, process: a document that materialized in one paste at 11:52 p.m. with no revision history is evidence about behavior, not style. Behavior is much harder to fake than prose — and unlike a perplexity score, it's something you can actually show a student and discuss.
An example that makes it click
Think of a smoke detector that shrieks every time someone makes toast, and stays silent through an actual grease fire. You wouldn't rip it off the ceiling — but you would never convict someone of arson based on the beep. You'd walk into the kitchen and look.
The detector score is the beep. The kitchen is the draft history, the student's other writing, and a two-minute conversation where you ask them to explain why they structured the argument that way. A student who wrote it can always tell you what they cut. A student who didn't, usually can't.
How to do it
- Design the trail before the paper exists. Assign an outline, a rough draft, or a short in-class writing sample — this gives you a baseline and makes the question mostly moot.
- Check process evidence first: Google Docs or Word version history, revision counts, timestamps, and whether the text arrived in one paste or grew over days.
- Compare against the student's known supervised writing. A large, sudden jump in register is worth a conversation; it is not proof by itself.
- Look for content errors specific to AI: invented sources, quotes that don't exist, citations to real authors for papers they never wrote, and confident wrong details about your class or community.
- Run a detector last, if at all — and read any score as a question, never an answer. Remember it flags plain writing, which disproportionately means your ESL students.
- Talk to the student and assume positive intent. Ask them to explain one choice they made. A false accusation costs far more than a missed one — 51% of students already fear being wrongly accused.
Key facts
- Liang et al. (Patterns, 2023) tested 7 widely-used GPT detectors on 91 TOEFL essays by non-native English writers: average false positive rate 61.22%, versus near-perfect accuracy on 88 US 8th-grade essays
- 89 of the 91 TOEFL essays (97.80%) were flagged as AI-generated by at least one of the seven detectors; 18 (19.78%) were flagged by all seven
- Rewriting genuine human US 8th-grade essays with simpler word choices raised their misclassification as AI from 5.19% to 56.65% — proving detectors read style, not origin
- Turnitin claims a document-level false positive rate of "less than 1%" but states it "does not make a determination of misconduct" and that instructors must apply professional judgment
- 51% of US students said they worried about being accused of cheating with AI even if they hadn't; 16% either knew someone falsely accused or had been falsely accused themselves (RAND, 2025)
- Study limitation: sample sizes were small and most detectors tested used GPT-2 backbones, so absolute error rates for newer detectors may differ — but the perplexity mechanism driving the bias is unchanged
▶ The 60-second explainer (script)
How can a teacher tell if a paper is AI generated? Honestly — often you can't, not from the writing itself. Here's why. A detector doesn't know what your student did. It measures perplexity: how predictable the next word is. AI picks predictable words, so AI text looks smooth. Problem is, so does a kid with a smaller vocabulary, or a kid writing in their second language. The detector isn't detecting AI. It's detecting plain prose and guessing. And the guessing fails in a cruel direction. A 2023 study in Patterns ran ninety-one TOEFL essays — real humans — through seven detectors. Sixty-one percent falsely flagged. Eighty-nine of ninety-one caught by at least one tool. The same detectors on American eighth graders? Near perfect. Then they proved the mechanism: they took real human essays and just simplified the vocabulary. Misclassification jumped from five percent to fifty-seven. Same author. Meanwhile, telling ChatGPT to use literary language dropped detection of actual AI work to thirteen percent. Turnitin itself says its score does not determine misconduct. So what works? Things the model wasn't there for. Fabricated citations. Wrong details about your class. And most of all, process — version history, drafts, timestamps. A paper that appeared in one paste at 11:52 p.m. tells you about behavior, not style. Then just talk to the student. Ask them what they cut. The one who wrote it always knows.
What authoritative sources say
People also ask
Is a Turnitin AI score proof a student cheated?
No, and Turnitin says so directly: the tool "does not make a determination of misconduct" and provides data for educators to judge. Its own stated false positive rate is low but explicitly "not zero."
Why do detectors flag ESL students so often?
Because they score how predictable the writing is, and second-language writers tend to use more common vocabulary and simpler structure. That reads as machine-like. In the Patterns study this produced a 61.22% false positive rate on TOEFL essays.
What's the single best signal?
Version history. A document that grew over days with real revisions is strong evidence of authorship; one that appeared in a single paste is worth a conversation. It measures behavior rather than writing style, so it doesn't punish plain writers.
Can teachers just recognize AI writing by feel?
Unreliably. Experienced teachers do notice generic structure, hedged claims, and missing personal specifics — but the same study showed that human-written plain prose is routinely mistaken for AI, and confident intuition carries the same bias as the software.
What should I do if I suspect AI but can't prove it?
Talk to the student before accusing, ask them to explain their choices, and consider redesigning the assignment to include drafts or in-class components. Turnitin's own guidance is to assume positive intent and give students the benefit of the doubt when evidence is unclear.
The same question, asked other ways
- How do teachers check for AI?720/mo