Why the API bill grows faster than traffic
Classic 2026 trap: an agent, RAG pipeline, or chatbot drags the same huge context into every request — a system prompt of 5k tokens, a set of few-shot examples, documentation, tool definitions. The user asks a single short sentence, but you pay for 6000 input tokens again. A thousand requests per day — and you run the same prefix a thousand times at full price.
Input tokens are cheaper than output, but it's the bloated input that burns the production-load budget. Classification, summarization, tool-heavy agents — everywhere repeats the stable 'header'. Caching removes this overpayment: the provider processes the prefix once, saves the model's intermediate state, and on subsequent requests returns it almost for free.
What is prompt caching really
The main thing to understand about prompt caching: it is prefix matching. The cache key is derived from the exact bytes of the rendered prompt up to the checkpoint. Any change of a byte at position N invalidates the cache for everything that follows N.
The request rendering order is fixed: first tools, then system, then messages. A checkpoint on the last system prompt block caches both the tools and the system together. Hence the single rule from which all optimization follows.
- Stable — forward. Immutable content (frozen system prompt, deterministic tool list) goes before the checkpoint.
- Volatile — backward. Timestamps, per-request ID, changing question — after the last checkpoint. Otherwise each request is unique and the cache is never read.
ECONOMICS: HOW MUCH DOES THE CACHE ACTUALLY SAVE
Let's break down the numbers, because that's the essence of cost. At Anthropic, the cache works via multipliers to the base price of input tokens:
| Operation | Multiplier to base input price | When applied |
|---|---|---|
| Regular input (no cache) | 1x | Each uncached token |
| Cache write, TTL 5 min | ≈1.25x | First request with a new prefix |
| Cache write, TTL 1 hour | ≈2x | First request with long TTL |
| Cache read | ≈0.1x | Each repeat request with the same prefix |
From here comes the break-even point. With a 5‑minute TTL, the cache pays off after just two requests: 1.25x (write) + 0.1x (read) = 1.35x versus 2x without cache. With a 1‑hour TTL you need at least three requests: 2x + 0.2x = 2.2x versus 3x. An hourly TTL keeps the write alive during lulls in bursty traffic, but the doubled write price requires more reads to recover.
Now about the '10×' claim. If you have 10k tokens of stable context and 100 requests on top of it: without cache that's 100 × 10k = 1M tokens at 1x. With cache — one time 10k × 1.25 + 99 times 10k × 0.1 ≈ 12.5k + 99k ≈ 111.5k 'equivalent' tokens. The cacheable part becomes roughly 9× cheaper. According to community measurements on real agent workloads, input‑side savings usually fall in the 5‑10× range — the exact figure depends on the stable prefix’s share of the total volume.
HOW TO ENABLE: CACHE_CONTROL IN PRACTICE
The simplest option — top-level auto-caching: the provider automatically sets a checkpoint on the last cacheable block. For fine-grained control — cache_control on specific blocks. A working example in Python with the Anthropic SDK:
import anthropic
client = anthropic.Anthropic()
# Большой стабильный документ кэшируем, вопрос — нет
response = client.messages.create(
model="claude-opus-4-8",
max_tokens=1024,
system=[
{
"type": "text",
"text": LARGE_STABLE_CONTEXT, # 10k+ токенов доков
"cache_control": {"type": "ephemeral"} # TTL по умолчанию 5 минут
}
],
messages=[{"role": "user", "content": "Суммируй ключевые пункты"}],
)
# Проверяем, что кэш реально сработал
u = response.usage
print("записано в кэш:", u.cache_creation_input_tokens) # платим ~1.25x
print("прочитано из кэша:", u.cache_read_input_tokens) # платим ~0.1x
print("некэшировано:", u.input_tokens) # полная цена
For an hourly TTL: "cache_control": {"type": "ephemeral", "ttl": "1h"}. Maximum 4 control points per request. cache_control can be attached to any block — system text, tool definition, message block (text, image, tool_result, document).
MINIMAL PREFIX: WHY THE CACHE SILENTLY DOES NOT WORK
A common complaint: you set cache_control, but cache_creation_input_tokens is zero. No error, and no cache. The reason is the prefix is shorter than the minimum. It depends on the model:
| Model | Minimum cacheable prefix |
|---|---|
| Opus 4.8 / 4.7 / 4.6, Haiku 4.5 | 4096 tokens |
| Sonnet 4.6 | 2048 tokens |
| Sonnet 4.5 and earlier | 1024 tokens |
A 3k-token prompt will be cached on Sonnet 4.5, but will silently not be cached on Opus 4.8. Fresh SDK releases and limit changes can be conveniently tracked in the REDDYX catalog — there you also find collections of tools for token measurement and prompt profiling.
SILENT KILLERS OF CACHE: AUDIT CHECKLIST
If cache_read_input_tokens is steadily zero for seemingly identical requests — somewhere in the prefix hides an invalidator. Go through the list; this is the most common reason for wasted budget on caching:
datetime.now()/Date.now()in the system prompt — the prefix changes every request.uuid4()or request-ID at the start of the content — same thing, each request is unique.json.dumps(d)withoutsort_keys=Trueor iteration over aset— non-deterministic serialization, bytes drift.- Interpolation of session/user-ID into the system prompt — per-user prefix, zero reuse between users.
- Conditional sections of the system prompt (
if flag: system += ...) — each combination of flags = a separate prefix. - The tool set, which changes per user or during the dialog — tools are rendered at position 0, any change breaks the entire cache.
The fix is the same: move dynamics after the last checkpoint, make it deterministic, or simply remove it if it carries no load. Changing the model also invalidates the cache entirely — caches are scoped by model.
ADVANCED: WARMING, WINDOWS, AND PARALLELISM
Cache warming
To eliminate the latency of a cold miss on the first live request, send a startup request with max_tokens: 0. The provider will run prefill, write the cache at your checkpoint, and return immediately with empty content — output tokens are not billed, you pay only for the write. It makes sense when the first-request latency is visible to the user and there is a window before traffic (app start, deployment). With continuous traffic, warming is unnecessary — the first live request itself warms the cache.
20‑block window
Each checkpoint looks back at most 20 content blocks for a previous cache record. In agent loops with many tool_use/tool_result steps, a single turn easily exceeds 20 blocks — then the next checkpoint fails to find the prior cache and silently misses. The fix is an intermediate checkpoint roughly every 15 blocks.
Parallel requests
A cache record becomes readable only after the first answer starts streaming. N parallel requests with the same prefix all pay full price — nobody reads what others are still writing. For fan‑out: send 1 request, wait for the first token, then launch the remaining N−1.
WHEN CACHE IS NOT NEEDED
Don't cache if the prompt changes from the very beginning every request. If the first 1k tokens differ from call to call — there's no reusable prefix, cache_control only pays a write premium with zero reads. There's also no point caching a prefix shorter than the model minimum — the cold write penalty there is negligible. And don't speculatively warm up dozens of different prefixes: each is a write at ~1.25x, which in total can easily exceed the saved latency.
Frequently Asked Questions
How much cheaper is reading from cache compared to regular input?
Reading from cache costs approximately 0.1x the base price of input tokens — that's about 90% savings on the cached portion. Writing costs ~1.25x with a 5-minute TTL and ~2x with an hourly one.
After how many requests does caching pay off?
With a 5-minute TTL — already on the second request (1.25x write + 0.1x read = 1.35x vs 2x without cache). With a 1-hour TTL you need at least three requests (2x + 0.2x = 2.2x vs 3x), because writing costs twice as much.
Why is cache_read_input_tokens always zero?
This means there's a silent invalidator in the stable prefix: a timestamp, UUID, non-deterministic JSON serialization, changing tool set, or user-ID interpolation. Or the prefix is shorter than the model minimum (4096 tokens on Opus, 2048 on Sonnet 4.6). Compare the bytes of two rendered prompts to find the discrepancy.
Which TTL to choose — 5 minutes or 1 hour?
Five minutes is the default for continuous or frequent traffic (requests more often than once every 5 minutes keep the cache alive themselves). One hour is for bursty workloads with long pauses: the doubled write price pays off because the write survives the downtime.
If you're tinkering with caching, token measurement, and LLM pipeline optimization in production — I post daily breakdowns of fresh tools and repositories on this topic in the Telegram channel REDDYX AI — new repositories every 30-60 minutes.