WHAT IS A HALLUCINATION AND WHY IT IS NOT A CODE ERROR
A hallucination is when an LLM produces a confident, grammatically flawless statement that does not correspond to reality: a non-existent function in a library, a fabricated quote, a link to a non‑existent article, a made‑up law number. The key word is confident. The model does not signal 'I don't know'; it fills the gap with the most probable continuation.
Academically, a more precise term is confabulation (confabulation) — like a person with memory damage who, with a straight face, fills in missing gaps. The model does exactly the same: it has no internal 'true/false' sensor, only a probability distribution over the vocabulary. And this is fundamental: even an ideally trained model that minimizes cross‑entropy will occasionally generate falsehoods, because the training objective is plausibility, not truthfulness.
WHY THE MODEL LIES: FOUR ROOTS OF THE PROBLEM
Let's break it down by causes, because treatment depends on the root.
- Nature of training. Next-token prediction rewards fluency, not fact. In SFT and RLHF, the model is also taught to always give a useful answer — this pushes it to guess when it would be more honest to stay silent.
- Data gaps. Rare facts (long-tail), recent events after cutoff, narrow‑specialty APIs — where data is scarce, the model interpolates. Hence fabricated SDK methods and “phantom” function parameters.
- Knowledge compression. Weights are a lossy compression of the internet. Precise numbers, dates, identifiers are stored poorly. The model remembers the “shape” of the answer but mixes up details.
- Prompt provocation. A leading question (“What award did this scientist win in 2019?”) forces the model to invent an award even if none exists. Sycophancy — the tendency to agree with the user — amplifies the effect.
Separately, it's worth distinguishing types of fabrication, because the metrics for them differ.
| Type | What it is | Example | How to detect |
|---|---|---|---|
| Factual (intrinsic) | Contradicts the provided context | In the document “revenue 10 million”, answer — “12 million” | NLI check of the answer against the source |
| Open (extrinsic) | Fact cannot be verified from context, taken “out of thin air” | A fabricated expert quote | External retrieval + fact check |
| Logical | Correct premises, wrong conclusion | Error in arithmetic/chain | Self-consistency, tool use (calculator, code) |
| Instrumental | Fictional API/signature | A non‑existent library method | Run the code, function signature check |
GROUNDING: ANCHORING THE MODEL TO FACTS
Grounding — tying generation to a verifiable source. The idea is simple: don’t let the model answer from memory; give it facts in the context and force it to rely only on them. This is the most powerful lever against hallucinations available in production.
In practice, grounding consists of three layers:
- Context injection — relevant documents in the prompt.
- Constraint in the instruction — “Answer only based on the provided text. If there is no answer — say you don’t know”.
- Attribution — the model must cite the source (chunk number, span) so the answer is traceable.
The last point is critical: attribution turns “trust me” into “here’s where I got this from,” and it can be automatically verified. Tools for building such pipelines (vector databases, rerankers, attribution frameworks) are released almost every week — fresh releases are easy to track in the REDDYX catalog.
RAG AND ITS PITFALLS
RAG (Retrieval-Augmented Generation) is grounding in industrial form: before generation, the system retrieves relevant chunks from the knowledge base and places them in the context. According to community measurements, a properly built RAG reduces the frequency of factual hallucinations manifold compared to the bare model. But it has its own pitfalls.
RAG is not a panacea
- Retrieval misses. Pulled the wrong chunk — the model faithfully hallucinates on garbage context. Garbage in = garbage out.
- The model ignores context. Even with correct facts in the prompt, LLM sometimes «overrides» them with learned knowledge. Especially on long context — the lost-in-the-middle effect: facts in the middle of the window get lost.
- Source conflict. Two documents contradict each other, the model silently picks one.
- Chunking breaks meaning. A fact is split between chunks — retrieval pulls half.
What actually helps in 2026: hybrid search (dense + BM25), reranker on top of top-k, and most importantly — answer verification against sources already after generation. RAG is responsible for input fidelity, the detector — for output control.
HOW TO FIX: WORKING STACK OF TECHNIQUES
No single method removes hallucinations alone. Layered defense works.
| Method | Against what | Cost | Effect |
|---|---|---|---|
| Grounding + instruction «don't know — stay silent» | Open hallucinations | Low | High |
| RAG (hybrid + rerank) | Fresh/rare facts | Medium | High |
| Tool use (code, calculator, API) | Logic, arithmetic, signatures | Medium | Very high on its domain |
| Self-consistency (N samples + voting) | Logical, unstable answers | High (xN tokens) | Medium |
| NLI output verification | Intrinsic fabrication | Medium | High as insurance |
| Lowering temperature | Variance/creative fabrication | Zero | Small, but free |
Practical implementation order: first strict instruction and temperature per task, then grounding/RAG, then tool use where machine verification is possible, and finally — output detector for critical scenarios.
CODE: HALLUCINATION DETECTOR VIA FACT SPLITTING
Basic but working technique: split the answer into atomic claims and verify each against sources with an NLI model (entailment). If a claim doesn't follow from context — flag. Below is a minimal Python example.
import re
from sentence_transformers import CrossEncoder
# NLI cross-encoder: на входе пара (premise, hypothesis)
# на выходе логиты [contradiction, entailment, neutral]
nli = CrossEncoder("cross-encoder/nli-deberta-v3-base")
def split_claims(answer: str) -> list[str]:
# грубая разбивка на предложения-утверждения
parts = re.split(r"(?<=[.!?])\s+", answer.strip())
return [p for p in parts if len(p) > 15]
def verify(answer: str, sources: list[str], thr: float = 0.5):
flagged = []
for claim in split_claims(answer):
# берём максимальный entailment по всем источникам
pairs = [(src, claim) for src in sources]
scores = nli.predict(pairs, apply_softmax=True)
best_entail = max(s[1] for s in scores) # индекс 1 = entailment
if best_entail < thr:
flagged.append({"claim": claim, "support": round(best_entail, 3)})
return {
"grounded": len(flagged) == 0,
"unsupported_claims": flagged,
}
sources = [
"Выручка компании за 2025 год составила 10 млн долларов.",
"Штат — 40 сотрудников.",
]
answer = "Выручка составила 12 млн долларов. В компании работает 40 человек."
print(verify(answer, sources))
# -> grounded: False, флагнет утверждение про 12 млн (нет поддержки в источниках)
This is not a final solution, but a scaffold. In production, they add decomposition into atomic facts via the LLM itself, and flagged claims are either cut or marked in UI as unverified. Cheap, deterministic, catches intrinsic fabrication which most often hits user trust.
METRICS: HOW TO MEASURE WHAT GOT BETTER
Without measurement, any fight against hallucinations is subjective. What to actually measure:
- Groundedness / faithfulness — fraction of claims confirmed by source. Main metric for RAG.
- Answer relevance — whether the answer addresses the question (to not confuse «didn't lie» with «went off topic»).
- Context precision/recall — retrieval quality separate from generation. Often hallucination is a search failure, not a model failure.
- Abstention rate — how often the model honestly says «I don't know» where there's no data. Growth of this metric is a good sign.
The de-facto standard in 2026 — run the pipeline through LLM-as-judge (like RAGAS approach) on a fixed golden set and track the trend from release to release. Absolute benchmark numbers change, what matters is the delta on your domain.
WHAT DOESN'T WORK (OR WORKS WORSE THAN PROMISED)
- “Don’t hallucinate” in the prompt. Placebo. The model doesn’t know when it hallucinates, so the instruction has nothing to rely on.
- Blind faith in logprobs. Model confidence weakly correlates with correctness — it confidently lies. Useful only as raw material for more complex detectors (e.g., semantic entropy across multiple samples).
- “Let’s get a bigger model.” Bigger — less frequent, but not zero. Scale does not cancel the nature of next-token prediction.
- Just slap on RAG. Without re-ranking, retrieval evaluation, and output verification, RAG adds a new source of error — noisy context.
Frequently Asked Questions
Can LLM hallucinations be completely eliminated?
No. This is a mathematical consequence of training on predicting plausible tokens. You can reduce frequency dramatically via grounding, RAG, and output verification, but guaranteed zero is unattainable — plan for human or machine verification in critical decisions.
How does grounding differ from RAG?
Grounding is a general principle: make the model rely on a verifiable source rather than memory. RAG is a specific implementation: the system first retrieves relevant documents, then places them in context before generation. Any RAG is grounding, but grounding is possible without retrieval, e.g., via direct injection of known context.
Why does the model confidently output falsehoods?
LLMs have no internal truth sensor — only a probability distribution over words. Confident tone is developed during training and is not tied to factual correctness, so the model’s textual confidence is not a sign of reliability.
How to measure answer reliability in practice?
Collect a golden set of question‑answer‑source and measure groundedness (share of statements backed by the source), answer relevance, and abstention rate. This can be automated via LLM‑as‑judge or NLI verification. Track the delta on your domain, not absolute numbers from public benchmarks.
Hallucinations are not a bug, but default behavior that all industrial AI in 2026 lives with. Tools for grounding, retrieval, and fact‑checking are updated literally every hour — if you want to catch working releases before anyone else, jump into the REDDYX AI Telegram channel — new repositories every 30‑60 minutes.