A 30-repo test shows how much data a coding agent's script can keep out of view
A dev.to test found one call per repo produced about 1.1 million tokens of output, while a hand-written script boiled similar data down to 627 tokens.
The short version
- 30 real calls returned 2,953,102 characters, roughly 1.1 million tokens (estimate).
- 28 of the 30 results were larger than 50,000 characters, Claude Code's threshold for saving a result to a file.
- A hand-written script returned 1,687 characters, about 627 tokens (estimate), for 25 repositories.
- Code Mode never turned on: claude -p replied "Not logged in", and the feature needs the interactive terminal.
- The script's filter was wrong in 5 of 10 matches, and the model had no way to notice.
Why it matters
If your AI assistant pulls data from many places, the raw material can overwhelm it before it starts thinking. Filtering in code first keeps the conversation small, but it also hides mistakes in the filter from the model. Check the filter itself, not only the summary.
What the test did, and what it faked
As dev.to reported, the authors ran a job that coding-tool users know well: checking a project's dependencies for trouble. On 2026-11-22, between 12:43 and 12:51 Singapore time, they fetched the newest issues from 30 popular open source repositories, one call each, through a remote data service called EveryInfra. That side of the test is real.
The Claude Code side is not. Code Mode is an experimental setting that lets Claude write a small JavaScript program to operate its tools. It never turned on here. The command the authors tried replied "Not logged in", and the feature only works in the interactive terminal anyway.
So the authors wrote a script by hand, following the documentation, and ran it in Node.js (a tool for running JavaScript on your own machine). Claude did not write it. Model token counts are also estimates from character counts, not measurements. The post is open about all of this, which is why its numbers deserve a fair reading rather than dismissal.
Thirty calls, a flood of text
The first round copied what an agent does by default: call one tool, receive the full result, read it, move on. The 30 results added up to 2,953,102 characters. Using a rough rate of 2.69 characters per token, the authors put that near 1.1 million tokens. A token is the small chunk of text a model counts as its unit of reading.
That total is larger than the 1 million token working memory of the Opus 5.5 model. The sizes varied a lot. The median result was 84,740 characters, TanStack/query was the largest at 323,135, and pmndrs/zustand was the smallest at 4,538 because it returned only 5 items, 3 of them pull requests.
Claude Code also has a safeguard for oversized tool output. In 28 of the 30 calls, a result passed the 50,000-character line where the tool stores output in a file instead of pasting it into the chat. Either way, the model still has to read it to use it.
How the script approach differs
Code Mode changes who does the sorting. Instead of the model reading each result, it writes a program that makes the calls, can run several at once, feeds one answer into the next request, and hands back only a short final object. You can still see the tool calls in the conversation. To view the program itself, press Ctrl+O and expand the collapsed Code row.
The feature arrived in Claude Code v2.1.323, published 2026-11-12. It is off by default and switched on with an environment variable, CLAUDE_CODE_EXPERIMENTAL_CODE_MODE=1. Project settings cannot enable it, so a repository you clone cannot turn it on for you. By 2026-11-22 the newest version was 2.1.330, and each release in between carried at least one Code Mode fix.
The post also notes a security wrinkle. Each tool call inside a script still passes through your deny rules and hooks, but the automatic reviewer checks the script only once, before it runs, so it cannot see values the script works out later. Strict limits belong in rules, not in that reviewer.
The result, and the catch
The hand-written script handled 25 repositories, reading 2,502,282 characters on the local machine. It kept only issues from the past 7 days that were not pull requests and whose titles or labels used words like regression, crash or security. What it returned was 1,687 characters, about 627 tokens, or 0.067% of the raw volume for those repositories.
The service fee did not depend on the method: $0.000958 per successful call, with rate-limited or failed calls free. Over the test, there were 41 attempts and 30 successes. Two later scripts made another 66 real calls.
Here is the caveat. The script cut the volume by more than 99%, yet its filter was wrong in 5 of 10 matches, and the model could not see that, because it only ever saw the summary. Smaller context is not the same as a correct answer.
How to read this write-up
This is a demonstration of scale, not a benchmark of Claude Code. It shows that the data is big enough to matter and that local filtering can shrink it a great deal. It does not show how well Claude writes, runs or fixes such a script on its own, and the authors say as much, suggesting readers run their own on-versus-off comparison.
What to watch
- Does a real interactive Code Mode run, with Claude writing the script, show similar savings?
- How often do model-written filters silently drop or wrongly keep results, and can the model be shown a sample to catch it?
- Will Code Mode lose its experimental label and work outside the interactive terminal?
Try it
Anthropic's token counting endpoint is free. Paste a typical tool result from your own work into it with your model ID to see how large it really is before an agent reads it.
Sources and links
- Reportingdev.to: the article we sourced
Liked it? Say why
Add a line for other readers.
No account needed. We never show your email. Comments with links, ads or abuse are held and not shown. Your comment shows after a quick check.
Listen to the AI Wire headlines
Listen to the AI WireListen to our daily AI shows every morning: on your commute, in the shower, wherever you are
Listen to the daily showsFirst reported by dev.to · Read the original. Written by the SI Agent News Desk from the reporting above.Written with AI from the source above. Every number is checked against the source. Spot a mistake? Corrections.
