Anthropic introduces message threads to reduce latency in Claude Code

Faster responses for long tasks

Anthropic recently announced a new feature called message threads designed to speed up long-running sessions in Claude Code. This update aims to help developers get faster responses as their conversation history grows during complex coding tasks. The feature is currently available in public beta for Anthropic’s Messages API and is already active for most Claude Code users.

The primary goal of this update is to reduce latency, which is the delay between sending a request and receiving a response. According to the company, message threads have reduced the median time to first token by approximately 35% for requests containing between 500,000 and 1 million tokens. This is particularly helpful for users who are working through massive amounts of data or long sequences of instructions.

While the speed increase is significant, the feature does not change the underlying cost of using the model. Anthropic clarified that because the model still has to read the entire conversation history every time it generates a response, the total token usage remains the same. Users will see faster results, but their API bills will not decrease as a result of this specific feature.

How message threads work

Traditionally, every time an AI agent sends a new request to the Messages API, it must include the entire conversation history. This means that if a coding agent has performed hundreds of actions, the system must re-transmit every single previous message and tool call. As the history grows to hundreds of thousands of tokens, this repeated uploading creates a bottleneck that slows down the entire process.

Message threads change this by allowing Anthropic to temporarily keep the conversation history on its own servers. The first request in a session still sends the complete history, including system prompts and tool definitions. However, subsequent requests only need to send the new messages along with a reference to the previous response ID. Anthropic then reconstructs the full conversation on its end before passing it to the model.

This mechanism allows the client to avoid the 'rinse-and-repeat' processing of uploading the same data over and over. For example, in a session involving large files, the amount of data transmitted by the user is greatly reduced. Developers can also combine this with prompt caching to potentially lower costs, as caching allows for discounted rates on previously processed content.

Impact on AI agents

The new feature is expected to be especially useful for developers building AI agents that handle large volumes of data. This includes coding agents that accumulate many file edits, computer-use agents that repeatedly capture screenshots, and applications that process large documents. By avoiding repeated uploads, these agents can operate much more efficiently during long-running tasks.

One industry expert calculated that for an agent capturing a 150 KB screenshot at every step, message threads could be highly effective. In a 50-step simulation, the total data uploaded would drop from roughly 191 MB to just 7.5 MB. This represents a reduction of approximately 25 times in the amount of data the client has to send over the network.

Because of these efficiency gains, some experts predict that computer-use agents might adopt the feature faster than coding agents. Since screenshots are heavy files, they create a more noticeable burden on the network than text-based code edits. The goal is to make these types of agents more viable for real-world use cases where high-frequency data transmission is common.

Technical limits and privacy

There are some specific limitations to the message threads feature that developers need to keep in mind. Currently, threads expire after only five minutes of inactivity. This means that if a human user steps away from their computer for a coffee break, the thread will expire, and the next request will require a full history resend.

However, this limitation is less of an issue for automated agents that make regular API calls. Every successful request stored in a thread refreshes the five-minute timer, allowing an agent to maintain a single thread for a long task. If a thread does expire, the Python SDK helper can automatically resend the history to establish a new thread, though the first request after the break will take longer.

From a security and privacy standpoint, Anthropic states that thread contents are encrypted and held in server memory. They are not written to persistent storage, but they may remain in memory for up to an hour after the last use. Because of these memory-based handling rules, the feature is currently ineligible for zero data retention and HIPAA compliance, which may limit its use in highly regulated environments.

How to implement the feature

Developers can start using message threads by adding a new thread parameter to the Messages API. The initial request must include a specific thread object to create the session. Subsequent requests then use a 'continue' type along with the ID of the first response. This tells the API to look up the existing history on the server rather than waiting for the client to provide it.

Anthropic has made it easier to implement this through their Python SDK. The SDK includes a thread helper that manages the conversation history and API parameters automatically. This helper handles the creation of the thread on the first request and ensures the correct beta headers are included in every subsequent call. This simplifies the workflow for developers who want the speed benefits without manual management.

The feature is currently available across all Claude models in beta. This includes integration through the Claude Platform on AWS, Claude in Microsoft Foundry, and Anthropic-managed Claude Models on Google Cloud. While it is in beta, the availability across major cloud providers suggests that Anthropic intends for this to be a standard way to handle long-context conversations in the near future.