Harmless AI agent permissions can combine into a credential leak, a new benchmark finds

A leak with no villain

MCP, the Model Context Protocol, is a standard for plugging tools into an AI agent through separate servers. As arxiv.org reported in this paper's abstract, an agent that faithfully chains those tools can move a credential from one server to another even though each permission looks harmless alone.

The authors call this an information-flow control problem, meaning the question of where data is allowed to travel. They describe it as a side effect of how a workflow is wired, not necessarily of a model misbehaving.

How MCPHunt measures it

The team plants fake credentials, called canaries, and then checks whether those exact strings show up somewhere they should not. That turns leak detection into simple text matching rather than a judgment call.

They also separate two cases. In some tasks the instructions require copying the secret word for word, which is not a violation. In others the agent could have redacted the credential and included it anyway. Only the second kind counts as policy-violating.

What the numbers show

Across 3,615 traces from 5 models, policy-violating propagation held at 11.5-41.3% for every model tested. For GPT-5.4, the rate ranged from 0.0% to 74.4% depending on the mechanism, and browser-mediated data flows were where leaks concentrated.

With a matched control using different synthetic credentials, propagation was 25.9% versus 20.4%. The statistical interval spans zero, so the authors say the comparison neither separates nor equates those two rates.

Can a prompt fix it?

In a test on 3 models, adding mitigation instructions to the prompt brought policy-violating leaks as low as 0.3%, a 98% relative drop, while keeping 80.5% of task usefulness. But the effect differed from model to model.

That unevenness is why the authors argue for defenses that track data flow directly instead of relying on prompting alone.