Prompt Caching for AI Agents: Save 70%+ on API Costs

Quick answer: Prompt caching stores the processed version of your AI agent's system prompt so the LLM skips recomputation on repeated requests. Anthropic charges 90% less for cached tokens, OpenAI gives 50–75% off, and Google offers 90% savings. For agents making 100+ daily requests with stable system prompts, caching alone cuts API costs by 70–90%.

Every AI agent sends the same system prompt — role instructions, tool definitions, safety constraints, few-shot examples — with every single request. For a typical production agent, that's 2,000–10,000 tokens of identical content reprocessed thousands of times per day. Without prompt caching for AI agents, you're paying full price every time the LLM reads instructions it already knows by heart.

The fix is straightforward: prompt caching tells the provider to store the computed representation of your prompt prefix. Subsequent requests that share the same prefix hit the cache and cost a fraction of the original price. Research published in January 2026 ("Don't Break the Cache," arXiv) found that prompt caching delivers 41–80% cost reduction and 13–31% faster time-to-first-token across production agent workloads.

If you haven't read our breakdown of how much AI agents cost to run, start there for the full pricing picture. This guide focuses specifically on prompt caching — the single highest-leverage optimization in our cost optimization playbook.

How Prompt Caching Works

Prompt caching exploits a simple insight: LLMs process input tokens sequentially, and the computational state at any point in the sequence is deterministic. If you send the same prefix of tokens repeatedly, the provider can store the intermediate computation (the KV cache) and skip straight to processing the new, unique portion.

Here's the flow:

1. First request — the full prompt is processed and the prefix computation is stored (cache write). You pay a small write premium.
2. Subsequent requests — if the prompt starts with the same prefix, the cached computation is reused (cache read). You pay 50–90% less for the cached portion.
3. Cache expiry — after a period of inactivity, the cache evicts. The next request triggers a fresh cache write.

The critical requirement is exact prefix matching. Even a single whitespace change invalidates the cache. This is why prompt architecture matters — and why most teams leave savings on the table.

Provider Comparison: Pricing and Mechanics

Each provider implements prompt caching differently. Understanding the mechanics is essential for maximizing savings.

Anthropic (Claude)

Anthropic gives you the most control. You set explicit cache breakpoints using the cache_control parameter on content blocks:

{
  "system": [
    {
      "type": "text",
      "text": "You are an AI code review agent for a Node.js project...",
      "cache_control": { "type": "ephemeral" }
    }
  ]
}

Key details:

For a 1-hour TTL, set "cache_control": {"type": "ephemeral", "ttl": "1h"}. The write cost doubles, but it pays for itself after roughly 4 requests per hour against the same prefix.

Pricing per million tokens (Claude Sonnet 4.6):

That's a 10x reduction on cached tokens.

OpenAI

OpenAI takes the opposite approach: fully automatic caching with zero code changes.

Any prompt over 1,024 tokens is automatically cached. The cache starts at the first 1,024 tokens and extends in 128-token increments. No API flags, no breakpoints — it just works.

The trade-off: you get no control over what gets cached or for how long. But for most agent workloads, the defaults work well. Check for cached_tokens in the response usage object to verify cache hits.

Google Gemini

Google offers a hybrid approach with two mechanisms:

Implicit caching (automatic since May 2025): Works like OpenAI's automatic caching. Requires 1,024+ tokens. No guaranteed savings — Google applies the discount opportunistically.

Explicit caching: You create a CachedContent resource via the API, then reference it in requests. Guaranteed savings but involves storage fees:

Explicit caching makes sense for large reference documents (50K+ tokens) that many requests share — like a codebase summary or product documentation that your agent references throughout a session.

How to Structure Prompts for Maximum Cache Hits

The number one reason teams miss cache hits: dynamic content in the wrong place. Cache matching works on prefixes — if anything changes before the cache boundary, everything after it is recomputed at full price.

The Golden Rule: Static First, Dynamic Last

Structure every agent prompt in this order:

1. System instructions (static) — role, personality, constraints
2. Tool definitions (static) — function schemas, parameter descriptions
3. Reference documents (semi-static) — codebase context, product docs
4. Few-shot examples (static) — demonstration input/output pairs
5. Conversation history (dynamic) — prior turns in the session
6. Current user message (dynamic) — the actual request

Everything above the first dynamic element can be cached. Everything after it is reprocessed every time.

Five Patterns That Kill Cache Hits

1. Timestamps in system prompts. "The current date is March 14, 2026" in your system prompt invalidates the cache every second. Move it to the first user message.

2. Randomized tool ordering. If your agent framework generates tool schemas dynamically, ensure they're sorted deterministically. A different JSON key order = cache miss.

3. RAG context in system prompts. Retrieved documents change on every query. Put them after the cache boundary, in the user message — not the system prompt. See our prompt engineering guide for the context injection pattern.

4. Session-specific variables in the preamble. User IDs, session tokens, and feature flags belong in the dynamic section, not the static prefix.

5. Minor formatting changes. Trailing whitespace, newline differences, or JSON pretty-printing vs. minification — any character-level change breaks the cache. Standardize your prompt formatting.

Real-World Cost Savings

Let's put concrete numbers on three common scenarios.

Scenario 1: Customer Support Agent

A support agent with a 5,000-token system prompt handling 1,000 conversations per day:

Scenario 2: Code Review Agent

A code review agent with a 10,000-token system prompt (including repo context and style guidelines) reviewing 200 PRs per day:

Scenario 3: Document Q&A Agent

An agent with a 50,000-token reference document cached as context, answering 500 queries per day:

The pattern is clear: the bigger your static prefix and the more requests you make, the more caching saves. For high-volume agents, prompt caching alone can be the difference between a sustainable AI deployment and one that bleeds money.

Implementation Checklist

Here's how to enable prompt caching for your AI agent, start to finish:

1. Audit your system prompt. Identify what's static (instructions, tools, examples) vs. dynamic (timestamps, user context, RAG results). Separate them.

2. Reorder your prompt. Move all static content to the front. Dynamic content goes last, after the cache boundary.

3. Enable caching on your provider:

   • Anthropic: Add "cache_control": {"type": "ephemeral"} to your system message content blocks. Use up to 4 breakpoints for sections that change at different frequencies.
   • OpenAI: Nothing to do — automatic for prompts over 1,024 tokens. Just verify cache hits in the response.
   • Google: For implicit caching, nothing to do. For explicit caching, create a CachedContent resource via the API.

4. Verify cache hits. Check the response usage object:

   • Anthropic: cache_read_input_tokens > 0
   • OpenAI: cached_tokens > 0
   • Google: cached_content_token_count > 0

5. Monitor your cache hit rate. Track cache_read_tokens / total_input_tokens over time. A healthy rate is 80%+. If it's below 50%, you have dynamic content leaking into your static prefix.

6. Standardize prompt formatting. Lock down whitespace, JSON key ordering, and tool schema generation. Any drift breaks the cache.

Advanced Techniques

Multi-Tier Caching with Anthropic

Anthropic allows up to 4 cache breakpoints. Use them to cache sections that change at different frequencies:

• Breakpoint 1 (1-hour TTL): System instructions + tool definitions (changes weekly)
• Breakpoint 2 (5-min TTL): Reference documents (changes per session)
• Breakpoint 3 (5-min TTL): Recent conversation history (changes per turn)

This way, your stable system prompt stays cached for an hour even if the conversation context cycles every few minutes.

Combining Prompt Caching with Semantic Caching

Prompt caching reduces the cost of processing the prompt. Semantic caching skips the LLM call entirely for queries similar to ones already answered. Layer them:

1. Check semantic cache first — if a similar query was recently answered, return the cached response (cost: $0)
2. If no semantic match, send to the LLM with prompt caching enabled (cost: 70–90% less than full price)

Together, these two techniques can reduce effective API costs by 95%+ for high-repetition workloads.

Batch Sequential Requests

When sending multiple concurrent requests to the same agent, send the first request alone to create the cache, then fire the remaining requests. This ensures all subsequent requests get cache hits instead of all simultaneously paying the cache write premium.

Common Mistakes to Avoid

Caching too little. If your system prompt is 3,000 tokens but you only cache the first 1,024, you're leaving 2,000 tokens of savings on the table. Cache the entire static prefix.

Ignoring cache write costs. Anthropic charges 25–100% extra for cache writes. If you have very low request volume (< 5 requests per cache TTL window), the write premium might exceed the read savings. Do the math for your traffic pattern.

Not monitoring cache hit rates. A code change that adds a timestamp to the system prompt can silently drop your cache hit rate from 95% to 0%. Alerting on cache hit rate drops is as important as alerting on error rates.

Over-caching dynamic content. As the "Don't Break the Cache" research demonstrated, naively caching everything — including tool call results — can increase latency. Be selective about what gets cached.

Get Started

Prompt caching is the simplest, highest-impact optimization for AI agent costs. Ten minutes of implementation yields 70–90% savings on input tokens, compounding across every request your agent handles.

If you're running agents as a team — shared code review agents, planning agents, customer support agents — cowork.ink centralizes prompt management so every team member's requests share the same cache. Higher hit rates, lower costs, zero duplicate work.

If you're running a personal agent, GoGogot works with any OpenRouter-compatible provider that supports caching — configure once, save on every session.

Try cowork.ink free — see how shared prompt management maximizes your cache hit rate.

Frequently Asked Questions

What is prompt caching for AI agents?

Prompt caching stores the computed representation of your AI agent's system prompt so the LLM doesn't reprocess it on every request. Providers like Anthropic, OpenAI, and Google offer 50–90% discounts on cached input tokens. For agents with large system prompts, caching pays for itself on the second request. See our [cost optimization guide](/blog/ai-agent-cost-optimization/) for more tactics.

How much does prompt caching save on AI agent costs?

Real-world deployments report 41–80% total cost reduction from prompt caching alone. Anthropic charges 90% less for cached tokens, OpenAI gives 50–75% off, and Google Gemini charges 90% less for cached reads. The savings depend on your cache hit rate, which improves when you keep system prompts static and place dynamic content last.

Does prompt caching work automatically?

It depends on the provider. OpenAI caches automatically for any prompt over 1,024 tokens — no code changes needed. Anthropic requires explicit cache breakpoints via the API (or automatic mode with a single flag). Google Gemini offers both implicit (automatic) and explicit caching. Check our [prompt engineering guide](/blog/ai-agent-prompt-engineering/) for structuring prompts to maximize cache hits.

What is the minimum prompt size for caching?

Anthropic requires 1,024–4,096 tokens minimum depending on the model. OpenAI requires at least 1,024 tokens. Google Gemini requires 1,024 tokens for implicit caching and 32,768 tokens for explicit caching. Most AI agent system prompts (with tool definitions) easily exceed these thresholds.

How long do cached prompts last?

Anthropic caches last 5 minutes by default (refreshed on each hit), with an optional 1-hour TTL at higher write cost. OpenAI caches persist for 5–10 minutes of inactivity and always clear within 1 hour. Google Gemini's explicit caches have a configurable TTL (default 1 hour) with per-hour storage fees.