How do AI detectors work?

Why — the first-principles explanation

An AI detector sees only the submitted artifact. It does not see who typed it, which prompts were used or whether a human edited every sentence. It tries to infer a hidden event—“was this likely produced or transformed by an AI writing tool?”—from statistical and linguistic signals in the final text.

A typical pipeline. The tool first decides what it can analyze: language, file type, length and qualifying prose. Turnitin’s current guide, for example, says its report analyzes long-form prose, not code, poetry, bullet lists or other unconventional formats, and requires at least 300 words for a report. A detector then splits the text into segments, often with overlap so each sentence has context. Turnitin says its model scores sentences and averages segment predictions into a document-level estimate.

What the signals look like. Some public explanations use perplexity or word-probability patterns: how expected is the next token given the preceding context? Others may use sentence structure, repetition, vocabulary, discourse patterns, a trained classifier or signals specific to a model family. “Burstiness” is a useful teaching metaphor for variation in rhythm, but it is not a universal specification for every detector. Vendors such as Turnitin say their model and training data are proprietary. The honest answer to “what exactly does this tool use?” is often: only the vendor can fully answer that.

How a report is produced. The classifier compares the text with patterns learned from human and AI examples, assigns sentence or segment scores, applies a threshold and aggregates the result. A report may highlight passages or show the portion of qualifying prose the tool predicts is likely AI-generated. That percentage is not the same as a plagiarism similarity score and is not necessarily the percentage of the entire file. A high score means “the model found a pattern it associates with AI,” not “the model identified the author.”

Why the inference is fragile. Predictable writing can be human. A student writing in a second language, a person following a rigid template, a technical manual, a short answer or heavily edited text can look statistically different from the detector’s training examples. The Stanford/Patterns study of seven detectors found substantial false positives for TOEFL essays by non-native English writers and showed that simple rewriting could change detector outcomes. That is one study of older tools, not a universal current accuracy number, but it demonstrates the structural problem: style is not identity.

What detection is not. It is not plagiarism matching. Similarity software looks for matching source text; AI detection estimates whether prose resembles model-generated writing. It is not a watermark unless the generator deliberately embeds a detectable signal. It is not provenance: a C2PA Content Credential records signed creation and modification history when present, while missing credentials do not prove that a human wrote the text. It is not a reliable way to reconstruct a prompt, a model or an author from text alone.

The decision boundary matters more than the score. OpenAI’s retired 2023 classifier explicitly said it should not be a primary decision tool, reported weak challenge-set performance and was withdrawn for low accuracy. Turnitin likewise says its indicator should not be the sole basis for adverse action. UT Austin treats AI detection software as high-risk technology requiring privacy, security and contracting review. In education, hiring, publishing or compliance, use a detector only to start a fair review: inspect drafts, revision history, sources, process evidence and the relevant policy. Never auto-fail, reject or accuse solely because a percentage crossed a threshold.

A practical test for any detector. Ask which languages and genres it supports, the minimum text length, whether the score is calibrated on comparable writing, how false positives are handled, whether the vendor stores submissions, whether results are versioned, and what appeal path exists. Measure it on your own representative human and AI samples with known labels. If the cost of a false accusation is high, the default should be corroboration and human judgment, not a harder threshold.

An example that makes it click

How to do it

1. Name the decision and its stakes. A low-stakes editing queue is different from a grade, disciplinary action, hiring rejection, benefits decision or publication refusal.
2. Identify the exact detector, version, language and file type. Do not compare percentages from different vendors as if they used the same scale.
3. Check eligibility rules: minimum words, qualifying prose, supported languages and excluded formats such as code, tables, poetry or bullet lists.
4. Preserve the original artifact and record the report date. Detector models and thresholds can change, so a later resubmission may return a different score.
5. Read the report as a likelihood signal. Inspect the highlighted spans and confirm whether the percentage refers to qualifying prose or the whole file.
6. Look for confounders: second-language writing, templates, formulaic technical language, short text, heavy editing, translation, grammar tools and genre mismatch.
7. Gather independent process evidence: drafts, document history, research notes, source links, oral explanation, version control or a reproducible work trail.
8. Check privacy and procurement before uploading sensitive work. Review retention, training use, deletion, access and contract terms; do not paste student or employee work into an unapproved free detector.
9. Apply the organization’s policy with a human reviewer and a clear appeal path. A detector result can prompt a conversation; it cannot establish intent or authorship by itself.
10. Evaluate the detector locally before adopting it: use representative human and known-AI samples, measure false positives and false negatives by language and genre, and re-test after model updates.

Key facts

• AI detectors classify patterns in finished text; they do not observe the author, prompt history or editing process.
• Turnitin says its current report analyzes qualifying long-form prose and does not reliably detect code, poetry, bullet lists or other unconventional formats.
• Turnitin describes segmenting submissions, scoring sentences and aggregating predictions into an overall estimate; its model and training data are proprietary.
• A detector percentage is not a plagiarism similarity score and is not necessarily the percentage of the entire file.
• Perplexity and word-probability patterns appear in public explanations, but no single feature set or “burstiness formula” defines every detector.
• The 2023 Stanford/Patterns study found seven detectors falsely flagged many TOEFL essays by non-native English writers and showed that rewriting could change outcomes.
• OpenAI withdrew its own AI classifier on July 20, 2023 because of its low accuracy and said it should not be used as a primary decision tool.
• Turnitin says its AI indicator should not be the sole basis for adverse action; a report is data for a human reviewer under institutional policy.
• C2PA Content Credentials record signed provenance when an implementation adds them; missing credentials do not prove human authorship.
• AI detection and plagiarism matching answer different questions and should not be merged into one score.
• Detector performance depends on language, genre, length, editing, training distribution, threshold and model version.
• Uploading a paper or employee document to a third-party detector creates privacy, security, intellectual-property and retention questions before accuracy is even discussed.

What authoritative sources say

People also ask

The same question, asked other ways

This page answers one intent expressed in 5 phrasings. How the index is organized →

• How do AI checkers work?
• How does AI detection work?
• What do AI detectors look for?
• Why do AI detectors flag human writing?

Related questions