A small trained model beat GPT-4o at deciding which AI agents should handle a request

The problem: one request, several agents

Many AI systems are no longer a single chatbot. They are a team of specialist agents, each good at one kind of job, with a first step that reads your request and decides who should handle it. That first step is called routing.

As arxiv.org's listing for the paper describes, routing is often treated as picking the single best agent. But real requests are often compound. A user might want a document summarized and also a calendar entry created, which needs two different capabilities. Sending the request to too many agents is also wasteful, because every extra dispatch costs money and time.

The authors reframe the task as choosing a set of agents, which could contain one agent or several. That is what 'set-valued' means in the title.

How the benchmark was built

To test this, the researchers assembled 3,000 prompts from WildChat, a collection of real conversations people have had with chatbots. They defined 12 agents, each described by what it can do, and created reference answers saying which agents each prompt should go to. The paper says these reference sets follow a schema and can be reproduced.

The prompts were balanced in two ways: by how often each agent appears, and by how many agents a prompt needs. That keeps the test from being dominated by easy, single-agent cases.

The comparison covered four approaches: retrieving agents by matching the request to written agent profiles, multilabel classifiers (models that can tick several boxes at once), a fine-tuned text encoder, and GPT-4o given the agent list and no task-specific training.

What the scores show

The results are scored with F1, a single number that rewards a router for picking the right agents without adding wrong ones. Higher is better.

• The fine-tuned encoder scored 89.59.
• A linear SVM, a classic and simple classifier, scored 71.79.
• GPT-4o, working zero-shot from the agent catalog, scored 41.51.

The authors also tried a harder split, where prompts were grouped by meaning so the test set was less similar to the training set. The encoder still reached 88.02 F1, so its lead did not depend on seeing near-duplicates of the test questions.

One caution on the GPT-4o number: it was used zero-shot, meaning it saw the agent list but was not trained on examples. The paper's conclusion is that supervision specific to your own agent catalog, and building the answer explicitly as a set, both matter. It does not claim GPT-4o is weak in general.

Adding cost to the decision

Accuracy is not the only measure the paper tracks. The evaluation also looks at latency, whether the chosen agents cover the capabilities the request needs, and a utility score that accounts for cost on an ordinal scale, meaning it ranks costs rather than using exact prices.

The authors introduce Weighted Agent Routing, or WAR. It is a fixed rule applied after the model has scored each agent, and it moves the balance between quality and cost. Because it comes after scoring, it requires no retraining. A team could in principle adjust the balance as its budget changes.

Combined with the encoder, WAR achieved the best utility under the paper's constrained setting. The paper presents this as evidence that a cost-aware layer can complement a strong language encoder rather than replace it.

Limits and what to watch

This is a focused contribution. The setup uses a fixed catalog of 12 agents, and the reference answers come from a defined schema, not from every real-world use case. The summary does not say how the approach holds up with a much larger or frequently changing agent list.

The paper has also been revised twice since its first submission on 27 Jun 2026, with the latest version dated 5 Nov 2026. As with any preprint, it has not necessarily gone through formal peer review, so treat the numbers as the authors' reported results.