A new paper says AI agents should be handed tested tools, not long prompts

The problem the authors point to

Many AI agents today work by improvisation. The language model writes code on the fly, guesses the exact format that a software service expects, and tries to remember a list of rules that someone pasted into its instructions. The authors argue this is a fragile way to build anything meant to run in production.

As arxiv.org shows in the paper's abstract, they blame this habit for three costs. It fills the model's context window, which is the amount of text it can hold in view at once. It inflates bills, because providers charge by tokens, the small chunks of text a model reads and writes. And it produces agents that fail in unpredictable ways.

What Skillware actually is

Skillware is an open framework plus a registry, a catalog of ready-made capabilities. Its central move is to separate what an agent wants to do from how the task gets carried out. The model decides the goal; fixed software does the work.

Each capability is a versioned bundle with five parts:

• a typed input contract written in JSON, which is a strict form spelling out what information the skill accepts;
• deterministic Python code, meaning code that does the same thing every time it gets the same input;
• task directives, the instructions for when and how to use it;
• bundled datasets the skill needs;
• automated unit tests, which are checks that confirm the code still behaves correctly.

The authors compare this to hotkeys in video games. Press the key and the move executes the same way under the game's fixed rules, with no improvising.

They also set their approach against skills that are just loose text or markdown files given to a model. A bundle with code and tests, they argue, is a different kind of object from a note that says how to do something.

The results the authors report

The team evaluated 23 models, both frontier models from cloud providers and open-weight ones, meaning models whose files you can download and run yourself. Across 368 benchmark conditions, they say small local models of 3B to 8B parameters, equipped with Skillware, consistently beat unequipped frontier models on complex multi-step workflows. Parameters are the adjustable numbers inside a model, and billions of them is a rough measure of size.

On efficiency, they report token use falling by up to 84.8% and inference costs, the price of running the model, falling by up to 97.3%. They also say parameter errors, where the model fills in a tool's inputs wrongly, were eliminated entirely. Note the phrase "up to": these are best-case figures, and the abstract does not say how typical cases looked.

To make this work with different models, the framework uses in-process adapters. These translate each skill into the native tool format a given model expects. The authors say this covers everything from cloud APIs to local runtimes with zero network latency, since nothing has to travel over the internet.

Why this idea has appeal, and its limits

The pitch is that trust comes from inspection. A skill whose code you can read, whose inputs are strictly defined and whose tests pass is easier to audit than a paragraph the model might read differently each time. That fits repetitive business jobs, where predictability usually matters more than cleverness.

There are caveats. This is a preprint, a paper posted publicly before peer review, and the numbers come from the authors' own benchmarks. Our summary does not describe how the benchmark tasks were built, so we cannot say how well they match real work. Someone also has to write and maintain each skill, a cost the abstract does not address.

The paper also leaves open how a registry of skills would be vetted. Code that runs inside your system is only as safe as its author, so who checks it matters.

Where the authors want to take it

The final claim is the most ambitious. The authors describe capability bundles as working across a continuous range of information-processing systems. They start with digital software agents today, then move toward IoT devices, drones and embodied robotics. The goal they state is a foundation for auditable capability execution in real time.

That is a vision, not a result. The tests described cover models and workflows, and nothing in the abstract shows drones or robots. Still, the underlying question applies to every agent: should the model improvise, or should it call something that has already been tested?