WHAT DOES AN AI DETECTOR ACTUALLY DO
Let's start with the main misconception: an AI detector does not 'know' that the text was written by a neural network. It estimates probability. The input is text, the output is a number like '87% AI' or softer 'likely AI-generated'. Under the hood it is either token distribution statistics or a separate classifier model trained to distinguish human writing from generation.
The key idea behind almost all detectors is that a language model left to its own devices writes predictably. It picks the most probable next tokens, so its text is smooth, without sharp jumps. A human writes unevenly: a weird word, suddenly a long sentence, a typo, a colloquial turn. Detectors measure exactly this unevenness.
PERPLEXITY AND BURSTINESS — THE TWO PILLARS
Perplexity (perplexity) — how surprised the model is by the text. Formally it is the exponent of the average cross-entropy over tokens. Low perplexity = text predictable for the language model = suspicion of generation. High = text 'unexpected', more likely human.
Burstiness (burstiness) — the spread of perplexity across sentences. For humans the spread is large: one sentence banal, the next with a rare construction. In raw generation everything is level at the same value.
Here is a minimal working example of perplexity calculation on GPT-2 — the very logic that underlies classic detectors:
import torch
from transformers import GPT2LMHeadModel, GPT2TokenizerFast
tok = GPT2TokenizerFast.from_pretrained("gpt2")
model = GPT2LMHeadModel.from_pretrained("gpt2").eval()
def perplexity(text: str) -> float:
ids = tok(text, return_tensors="pt").input_ids
with torch.no_grad():
loss = model(ids, labels=ids).loss
return torch.exp(loss).item()
human = "слушай ну я вчера реально задолбался с этим деплоем, всё легло"
ai = "Данный процесс развёртывания представляет собой комплексное решение."
print(round(perplexity(human), 1)) # высокая — человек
print(round(perplexity(ai), 1)) # низкая — подозрение на AIIt is exactly by this principle that the first GPTZero and dozens of clones worked. The problem is obvious here: a detector using the reference GPT-2 model evaluates text that could have been written by a person in a formal‑business style. And such a person will get flagged as 'AI' for no reason.
FALSE POSITIVES — THE MAIN PAIN
This is not an abstract threat. According to numerous independent measurements by the community and academic groups, detectors systematically err on:
- Non‑native speakers. Texts from people for whom the language is not native are written simpler and more predictable — detectors massively label them as AI. This documented discrimination has led several US universities to roll back detector use.
- Technical and legal prose. Boilerplate phrasing, set expressions — low perplexity by definition.
- Short texts. On a 2‑3 sentence excerpt there simply isn’t enough statistics — you can’t trust the number.
- Classics and the Bible. Well‑known meme: detectors confidently label the US Constitution and religious texts as AI‑generated because the models have seen them millions of times and they are ‘predictable’.
Practical takeaway: a single detector percentage is not proof. Any serious decision (expulsion, firing, article rejection) based solely on a detector is legally and ethically fragile.
WATERMARKS — THE ONLY HONEST SIGNAL
Unlike post-hoc detectors, watermarks (watermarking) are embedded at generation time. The idea (Kirschenbauer et al.'s method, extended in Google DeepMind's SynthID): at each generation step, tokens are pseudo-randomly divided into 'green' and 'red' lists based on a hash of previous tokens, and the model is gently nudged to choose green. A human sees no difference, but knowing the secret key, one can statistically prove: suspiciously many green tokens — this is our model.
Advantage of the approach: a mathematically grounded p-value, not 'the model just felt like it.' Disadvantages:
- Works only if the provider themselves embedded the mark. Local open-weight models without a watermark — miss.
- Paraphrasing via another model blurs the statistics of green tokens.
- Mixing with human text weakens the signal strength.
SynthID for text was partially opened by Google in 2024, and this direction in 2026 is the most promising. But market coverage is low: most generations pass without any marks. Fresh releases of detectors and open-source tools around watermarking can be conveniently tracked in the catalog REDDYX.
COMPARISON OF APPROACHES
| Approach | How it works | False positives | Evaded | Requires model access |
|---|---|---|---|---|
| Perplexity / burstiness (GPTZero-style) | Statistics of token predictability | High | Easy (editing, paraphrase) | No |
| Trained classifier | Separate yes/no model | Medium, degrades on new LLMs | Medium | No |
| Stylometry | Analysis of style, n-grams, rhythm | Medium | Medium | No |
| Watermarks (SynthID etc.) | Signal embedded at generation | Very low | Only via paraphrase | Yes, at generation stage |
HOW DETECTORS ARE BYPASSED (AND WHY IT'S TRIVIAL)
This section is not an instruction «how to cheat», but an explanation why you cannot rely on a detector as protection. Statistical detectors fail by their own metric — you just need to raise perplexity and burstiness:
- Manual editing. A few rewritten phrases, diluted sentence rhythm, lively turn — and the «smoothness» collapses.
- Paraphrasing by another model. Passing through a second LLM or a specialized «humanizer» changes token distribution.
- Mixing. One paragraph generated, another written by hand — the averaged metric drifts into the gray zone.
- Prompt for «humanity». A request to write deliberately uneven, with varying sentence length, increases perplexity at output.
That’s why the «detector versus generator» race is a classic adversarial game in which the attacker is always one step ahead: the detector is public, its metric is visible, and optimizing against it is easy.
WHAT TO DO IN PRACTICE IN 2026
If you are checking others’ texts
- Do not render verdicts based on a single percent. Demand context: edit history, drafts, oral defense of the work.
- Remember false positives on non‑native speakers and technical prose.
- If legally critical — look for a watermark from the provider, rather than running the text through a random web detector.
If you are writing with AI
- Edit by hand not to evade detection but for quality — smooth, bland text reads poorly to both people and Google.
- Disclose AI use where the platform’s policy requires it. This is cheaper than a dispute.
MAIN CONCLUSION
AI detectors in 2026 are a useful but weak heuristic signal. They give an indication «worth a closer look», not a verdict. The only statistically rigorous technology is watermarking, but its coverage is low and it is vulnerable to paraphrase. Building punitive processes on detectors means punishing the innocent and letting the skilled slip through. Tools around detection and watermarking evolve quickly — new projects are gathered in the REDDYX catalog.
Frequently Asked Questions
How accurate is GPTZero and similar detectors?
While the claimed accuracy is high on «clean» raw generation, it drops sharply on edited text and yields a noticeable false‑positive rate on real people — especially non‑native speakers and technical prose. A single detector result cannot be taken as proof.
What is perplexity in AI‑text detection?
Perplexity is a measure of how predictable a text is for a language model. Low perplexity means «too smooth, expected» text and raises suspicion of AI. But it is just as easy to raise through editing, so the metric can be evaded.
Can you fool an AI detector?
Statistical detectors are easily bypassed: manual editing, paraphrasing via another model, mixing generated and manually written text. That's why a detector is not protection, but merely a probabilistic indicator.
Are watermarks more reliable than detectors?
Yes, watermarks provide a mathematically justified signal with a low false positive rate. However, they only work if the provider embedded the mark during generation and they blur when paraphrased via another model.