What is prompt caching?
Prompt caching lets an LLM API reuse the processed form of a prompt prefix it has seen recently, such as a long system prompt, tool definitions or documents. Cached tokens are billed at a fraction of the normal input price and are processed faster.
Is prompt caching worth it?
Yes when the same prefix of at least the provider minimum (512 tokens on Claude 5.x, 1,024 on OpenAI, 4,096 on Gemini 4) is reused often enough to beat the break-even hit rate. On providers that charge extra to write the cache, sparse traffic can make caching cost more than it saves.
How much does Claude prompt caching cost?
On Claude 5.x models a cache write costs 1.25x the input price with the default 5-minute lifetime, or 2x with the 1-hour lifetime. Cache reads cost 0.1x input on Haiku 5.5, 0.05x on Opus 5.5 and Sonnet 5.5, and 0.025x on Fable 5.2. Each hit refreshes the 5-minute timer for free.
What is the break-even cache hit rate?
It is the share of requests that must hit the cache for caching to cost the same as not caching: (write price - input price) / (write price - read price). With a 1.25x write and a 0.1x read it is about 22%; with a 2x write it is about 53%. Providers with free cache writes break even at any hit rate above zero.
Does OpenAI charge for prompt caching?
On GPT-5.6 and later, caching is automatic and a cache write costs 1.25x the input price for the default 30-minute lifetime or 2.5x for 12 hours. Cache reads cost 0.1x input, or 0.05x on GPT-6.1 Sol and Terra. Older OpenAI models cache automatically with no write charge.
Why is my cache hit rate low?
The prefix changed (a timestamp, user name or reordered tools near the top), requests arrived more than one TTL apart so the cache expired, or the prefix is under the minimum cacheable size. Put static content first, keep it byte-identical, and send related requests close together.