SIAgentNews.com
← All news

On one RTX 4090, an AI video agent ran out of context budget, not model memory

A test of an open video-making model on one consumer graphics card found that the agent's memory settings, not the model's size, decided whether it finished.

· First reported by dev.to

21 minutesspent on three failed recap attempts before the time limit

The short version

  • The 27B model ran on one 24 GB card using a 17.2 GB compressed copy; the 4-bit run peaked at 20.9 GiB.
  • The authors' setup used a 512K-token window and 64K-token answers; the test used 98,304 and 16,384.
  • Three recap attempts all hit the output cap, burned 21 minutes, and the run ended with no film.
  • The file check passed a clip showing 5 rows of 7 columns when the brief asked for 5 by 6.
  • Image handling and a 120-second worker timeout also broke the run until both were fixed.

Why it matters

If you plan to run an AI agent on your own hardware, fitting the model into memory is only the first test. You also need to know when the agent's working memory fills up and whether its own summary can fit in one answer. Checking those two numbers up front can save you a failed overnight run.

A film that stalled at minute forty

As dev.to reported, a team tested SenseNova 6.8 Code2Video, an open 27B model from SenseTime that makes video by writing code. They ran it as an agent, meaning a model that calls tools in a loop with little supervision. Claude Code drove it on a single 24 GB RTX 4090 in the cloud.

The early results looked good: after ten minutes there was a finished 12-second clip. But by the forty-minute mark there was no complete film. The author argues that the cause was the budget around the model, meaning how much it can keep in view and how long one reply may be, rather than the size of the model itself.

Your card cannot copy the authors' settings

The SenseNova team publishes a Claude Code skill called code-video, plus six full production runs. Those runs served the full-size model across four GPUs, with a context window (the amount of text the model sees at once, measured in tokens, or word pieces) of 512K tokens. A single answer could stretch to 64K tokens, and the films took between 261 and 876 turns each.

The test on one card used a 4-bit GGUF from bartowski. That is a compressed copy of the weights, 17.2 GB in size, for the llama.cpp server, plus a 0.93 GB add-on so the model can look at images. To match, the author set the window to 98,304 tokens and capped answers at 16,384. Compaction, the step that triggers a recap, was left at 80% full, the same as the authors.

One detail is easy to miss. Without an explicit window setting, Claude Code assumes 200K tokens for a model it does not recognize, which would be far more than this setup could hold.

The recap that never fit

When an agent's window fills up, Claude Code asks the model to write a recap, drops the earlier history, and carries on from that recap alone. The catch is that the recap is an ordinary reply, so it must fit under the same output cap as everything else.

In the test, the smaller window filled after about 19 minutes. Claude Code then asked for a recap three times, and each attempt ran out of room before it produced anything usable. Together they used 21 minutes, and the 40-minute limit ended the run with no final film.

The method itself can work. Four of the authors' six runs got through contexts of 167K to 492K tokens, using the 64K cap. In the single-card test, two settings are suspects. One is the 16,384-token cap. The other is hidden reasoning, the thinking a model writes before its visible answer, which shares the same cap. The logs cannot separate the two. In a separate one-off request, the same 4-bit file spent all 8,192 output tokens thinking and never wrote a page. The author tried no fixes, so a larger window, a larger cap or switching reasoning off remain untested.

A passing check that missed the point

The agent's self-review worked in part. It opened 21 of its own still frames as images, found two bugs and fixed them. The on-screen text matched the brief exactly.

But the brief called for a grid of 30 dots, 5 rows by 6 columns. The author located each dot's center in the 7.2-second frame using numpy and scipy, two common Python tools for numbers and images. The result was 5 rows of 7 columns, with one empty slot in every row.

The skill's built-in file check still approved the clip, because it only looks at resolution, frame rate and duration. The lesson is to add a check that knows what the video is supposed to show.

Plumbing problems and smaller cards

Claude Code speaks Anthropic's Messages API format, so anything placed in front of the model has to translate it. In this test, the translation layer lost the images: they disappeared or turned into blocks of text, so the model reviewed its work without seeing it. The author says this is fixed in a test build but not yet on the hosted service.

Timing caused a second failure. The first call waited for a full answer instead of streaming it in pieces, and the worker on the GPU machine gave up after 120 seconds while the model was still thinking. Raising that limit to 30 minutes fixed the run, and the example code now streams.

The author also tried smaller memory budgets with one short request each, run on the same 4090 rather than real smaller cards:

  • 16 GB budget: a 3-bit copy with a 32K window peaked at 13.7 GiB, and the page rendered with one dot visible at a time.
  • 12 GB budget: a 2-bit copy with a 16K window peaked at 10.0 GiB, and the page was cut off at 8,192 tokens.

The author is clear that this was one attempt at one brief under tight limits of their own choosing, so treat the results as a field report rather than a benchmark.

What to watch

  • Would a larger window, a larger output cap or reasoning switched off let the recap step finish on one card?
  • Will the image-handling fix reach the hosted gateway service?
  • Will the skill's file check gain a way to verify what is actually drawn, not just the video's format?

Try it

Before trusting a local agent, work out when your context window will reach the recap trigger, then test whether one summary fits under your output cap with reasoning on. Also send one PNG through any gateway and ask the model to describe it.

Sources and links

Read the original at dev.to

First reported by dev.to. Written by the SI Agent News Desk from the reporting above.Written with AI from the source above. Every number is checked against the source. Spot a mistake? Corrections.

Liked it? Say why

Add a line for other readers.

No account needed. We never show your email. Comments with links, ads or abuse are held and not shown. Your comment shows after a quick check.

More AI news