Strategies to optimize Claude prompt cache costs

How the prompt cache works

Claude uses a prompt cache as temporary storage for the results of previous prompts. Instead of storing the final response, it saves an intermediate result known as a KV representation. This allows the model to reuse processed information rather than starting from scratch every time a user sends a new message.

When a user sends a message, the system includes the entire context, such as system prompts, skills, and previous history. If the previous tokens are still in the cache, Claude only processes the new information. This mechanism provides a massive performance and cost improvement for long conversations where the same context is reused repeatedly.

However, this storage space is limited and managed automatically. If the system does not receive a new message within a certain timeframe, it removes the stored result to make room for other users. When a message is sent after this cleanup, the model must re-process the entire context, resulting in what is known as a cache miss.

The high cost of cache misses

The financial difference between hitting and missing the cache is significant. For the Opus 5.5 model, input tokens cost $4 per million, while cache reads cost only $0.20 per million. This makes hitting the cache 20x cheaper than standard input. However, writing new tokens to the cache costs 2x the normal input price.

The math shows that the larger the existing context, the more expensive a cache miss becomes. For example, in a session with 300k tokens of context, a cache hit for a 1k message costs the equivalent of 17k base input tokens. In contrast, a cache miss for that same message costs 602k base input tokens.

As dgrcode.com reported, the ratio of cost between a miss and a hit gets closer to 40x as the context grows larger and the new message grows smaller. This means that for high-context tasks, failing to maintain the cache can lead to a massive, unexpected spike in API usage costs.

Strategies for keeping the cache alive

One way to prevent the cache from expiring is to send periodic messages to keep the session active. Users can manually send a short message, such as 'wait, I'm still working on this,' to refresh the cache. Even though the message is small, it writes over 100 tokens to the cache because it includes the previous response and system markers.

This strategy is highly efficient because the cost of hitting the cache adds only a tiny amount to the total. If a user can automate this process, a one-hour cache effectively costs almost the same as a multi-hour cache. This allows users to maintain a 'hot' cache for long periods, such as during lunch breaks or while running errands.

Some tools, like Claude Code, offer a /loop command that can send messages on a schedule. While not free, looping roughly 40 times can cost about the same as a single cache miss. This provides a way to keep the session active without manual intervention, though it requires careful timing to ensure the cache doesn't expire between loops.

Compacting context as an alternative

Another option for managing large contexts is the /compact command, which summarizes the conversation to make the context smaller. This is particularly useful for very long sessions where the cache is likely to expire or become too large to manage efficiently. Compacting reduces the number of tokens the model has to read from the cache in future turns.

There is a trade-off involved with compaction, as the summary might lose specific details from the earlier conversation. The cost of compaction includes the summary output, which can be expensive since output tokens are priced higher than input tokens. However, for very large contexts, the long-term savings on subsequent messages can eventually offset the initial cost of the summary.

The best strategy depends on the specific needs of the user. For contexts under 100k tokens, it may be easier to let the cache expire. For contexts over 600k tokens, compacting is often the best move. For medium-sized contexts and shorter pauses, looping messages to keep the cache alive is generally the most cost-effective approach.

Future of cache management

The importance of cache management is becoming more evident as AI tools become more integrated into developer workflows. Anthropic appears to recognize the high cost of cache misses, as some tools have begun reporting when they compact context automatically to avoid expiration. This suggests a move toward more intelligent, automated cache handling for the end user.

Users may eventually see more granular controls for managing how these caches behave. This could include settings to keep a cache alive indefinitely, a 'smart' mode that decides between looping and compacting, or a way to manually set expiration times. Such features would allow users to tailor the balance between cost, context fidelity, and performance.

As models continue to improve, their ability to generate high-quality summaries will likely make compaction more reliable. While there is currently a limit to how much information can be compressed without losing detail, better reasoning may allow for higher compression ratios. For now, understanding the underlying mechanics of cache hits and misses remains the best way to control AI spending.

Summary of decision flow

Choosing the right strategy depends on three main factors: context size, pause duration, and the remaining work in a session. If the context is small, the cost of a cache miss is negligible, and manual management is often unnecessary. If the context is massive, compaction becomes a necessity to keep the session usable and affordable.

For the middle ground, the duration of the pause is the deciding factor. A pause of two hours or less is easily managed by looping a 'wait' message. For pauses longer than eight hours, compacting is usually the safer bet to ensure the session remains cost-effective. If the pause is between four and eight hours, the decision depends on whether the user needs high fidelity or just a few more prompts.

Ultimately, the goal is to minimize the 40x cost penalty of a cache miss. By using /loop for short breaks and /compact for long sessions or huge contexts, users can significantly reduce their token spend. These strategies ensure that the most expensive part of the process—re-processing the entire history—happens as infrequently as possible.

Interactive comparison

The cost of these strategies can be visualized by looking at the cumulative base token cost of a pause followed by the first 10 messages. A cache miss is almost always the most expensive option in the long run, especially as the number of messages increases. The loop strategy maintains a low cost per message but requires active maintenance during the pause.

Compaction has a high upfront cost because of the output tokens required for the summary. However, because the summary is much smaller than the original context, every message following the compaction is significantly cheaper. This makes compaction the superior choice for sessions that will continue for a long time after a significant break.

The decision flow provided by Claude suggests a nuanced approach for these variables. For instance, if a user has a context over 200k tokens and will be away for 8 to 16 hours, compacting is recommended. If the context is under 300k and the pause is short, looping is the preferred method to preserve the exact history of the conversation.

Key takeaways

The primary takeaway is that cache hits are 20x cheaper than standard inputs, while cache misses can be 40x more expensive. This creates a massive incentive for users to keep their prompt caches active. Understanding the 1-hour expiration window is the first step in avoiding these unnecessary costs.

Users should prioritize the /loop strategy for common interruptions like lunch breaks or errands. This keeps the cache 'hot' with minimal cost. For deep work sessions that involve hundreds of thousands of tokens, the /compact command should be used to shrink the context before taking a long break.

By monitoring context size with commands like /context, users can make informed decisions about which strategy to use. These simple habits can lead to significant savings on API bills without sacrificing the performance of the AI. As tools become smarter, these optimizations may become automated, but knowing the logic behind them is essential for smart AI usage.

Summary of cost logic

To visualize the math, consider a session with 300k tokens of context and a new 1k-token message. A cache hit costs 17k base input tokens because the 300k tokens are read at 1/20th of the price and the 1k tokens are written at 2x the price. This is the most efficient way to continue a conversation.

A cache miss, on the other hand, costs 602k base input tokens because the entire 301k tokens must be written to the cache at 2x the price. This illustrates why a small delay in activity can lead to a massive price jump. The larger the context, the more dramatic this price difference becomes.

The cost asymmetry is the key to optimization. Writing to the cache for the first time costs 2x, but adding an extra hour of cache life costs only 0.05x the base tokens. This means that as long as you can keep the cache alive, the marginal cost of extending that cache is nearly zero, making the loop strategy so effective for many common use cases.

Technical nuances

While the cache is a simplification, it is important to know that it stores the KV representation of the prompt. This is an intermediate mathematical state that the model uses to generate the next token. Because it stores this state, the model doesn't have to re-calculate the attention weights for the parts of the prompt that have already been processed.

The storage for these representations is limited, which is why the cache is temporary. When the system runs out of space or the time limit expires, it must discard old data. This is why the