Prompt caching
Reusing the computed state of a repeated prompt prefix instead of recomputing it every call. On APIs it shows up as a discounted "cache read" rate; when self-hosting it is a server feature (prefix caching).
What it is
When many requests start with the same long block - a system prompt, a style guide, a retrieved document, a tool schema - the model does the same work on that block every time. Prompt caching stores the kv cache for that prefix (in GPU memory, briefly) so a later matching request skips straight to the new part.
On APIs you enable it (or it is automatic) and pay cached input pricing: a small write fee, then ~10-25% of the input rate on cache hits. Self-hosting servers do prefix caching automatically.
Why it matters
For rag, agents, and long-system-prompt chatbots the repeated prefix is often 80-95% of the input tokens. Caching it can cut the LLM line of your bill by half or more - it is usually the first lever to pull. It also lowers ttft.
Related concepts
Use it in Obolith
FAQ
Is prompt caching the same as the KV cache?
Prompt caching is a way of reusing the KV cache across requests. The KV cache itself exists within a single generation.