Anthropic introduces message threads to speed up Claude Code

Reducing latency in long sessions

Anthropic has launched a new feature called message threads in public beta for its Messages API. This update is designed to speed up long-running sessions for Claude Code and other AI agents that handle extensive amounts of data. It specifically targets the delay caused by repeatedly sending the same conversation history back to the API.

Currently, every new request to an AI model typically includes the entire history of the conversation. As a coding session grows to hundreds of thousands of tokens, the system must validate and tokenize that entire history every single time a new message is sent. This repetitive process creates significant latency as the conversation becomes more complex.

With message threads, Anthropic can temporarily retain the conversation history on its own servers. The first request still sends the complete history, but subsequent requests only need to include new messages and a reference to the previous response. This allows the system to reconstruct the full context internally, making the transmission much faster for the user.

How the technology works

The technical shift involves moving from a stateless request model to a semi-persistent one for the duration of a session. When a developer uses the new thread parameter, the API creates a unique identifier for the conversation. This allows the server to remember the system prompts, tool definitions, and previous messages without the client having to re-upload them with every turn.

It is important to note that while the data transmission is smaller, the actual processing remains the same. Claude still reads the entire conversation history on every turn to generate a response. Because the model still processes every token in the history, the total token bill for the user does not decrease as a result of using message threads.

Developers can still manage costs by combining this feature with Anthropic's existing prompt caching. While message threads reduce the amount of data sent over the wire to lower latency, prompt caching allows for the reuse of previously processed content at discounted rates. Using both features together can optimize both the speed and the cost of long-running AI tasks.

Impact on AI agents

This update is particularly beneficial for agents that perform repetitive tasks involving large files. For example, computer-use agents that capture screenshots at every step would see a massive reduction in uploaded data. In a simulation, a 50-step session involving screenshots could see data uploads drop from roughly 191 MB to just 7.5 MB.

Experts suggest that computer-use agents might adopt this feature faster than coding agents because of the heavy weight of image data. Since a screenshot per step makes the resend heavy, the efficiency gains are more immediate for visual tasks. This could lead to widespread adoption of message threads for these types of applications before the end of the year.

The feature is already enabled by default for most Claude Code users. It is also available across all Claude models through various platforms, including AWS, Microsoft Foundry, and Google Cloud. This ensures that a wide range of developers can begin integrating the faster response times into their workflows immediately.

Limitations and data privacy

There are specific constraints on how message threads function, most notably a five-minute inactivity timer. If a user stops interacting with the AI for more than five minutes, the thread expires. However, for automated agents that make regular API calls, this is less of an issue because each successful request refreshes the expiry period.

For human users who take breaks, the Python SDK provides a helper that manages this automatically. If a thread expires during a break, the SDK will detect the error and resend the conversation history to establish a new thread. This means the session can continue without losing context, even if the first request after the break takes slightly longer to process.

From a security standpoint, Anthropic states that thread contents are encrypted and held in server memory rather than persistent storage. Because of this memory-based handling, the feature is currently ineligible for zero data retention and HIPAA. This may limit its use in highly regulated environments that require strict data handling protocols or specific compliance certifications.