Your AI agent's tool-call logs can reveal your nationality, religion and health

The leak is in the logs, not the chat

As tmtpost.com reported, a team at Shanghai AI Lab, working with the University of Electronic Science and Technology of China and the University of Sydney, studied a quieter privacy risk. When an AI agent uses outside services such as maps or health tools, each call leaves a record. The record holds the tool name, the inputs sent, the results returned and the order of calls. These records are called tool-call traces.

The tool's server never sees your conversation, but it does see these traces. A gateway or aggregator sitting in the middle can see traces from all of your tasks. Nobody in the study misbehaves: the observers simply look at data they already have access to.

How the researchers tested it

Real medical records were off limits, so the team built 1,000 synthetic users. Their profiles were modeled on population statistics and carried 17 attributes, including nationality, age, occupation, religion, health and income. Agents then ran tasks in travel, health, shopping and career learning using 25 real tool servers, producing 4,000 runs.

Four large language models acted as the observer, shown only the traces and asked to guess all 17 attributes. Blind guessing scored 17.97%. With traces from a single domain, the score rose to 26.25%. The four models scored close together, between 25.29% and 28.32%, yet the paper estimates one run cost about $1,080 and another about $46.

Profiles leak, identities mostly do not

Nationality (57.73%) and religion (50.03%) were the easiest to infer. Travel and shopping tasks leaked more than career-learning ones. In the paper's example, a route to a hospital area, with walking-only trips from Friday sunset to Saturday night, points to a place of residence, a health condition and a faith.

Exact identifiers were far harder. Of 16,000 identifier guesses, only 35 were right: 34 names and one phone number. No email addresses or government IDs were guessed correctly.

Parameters, the specifics the agent chooses to send, carried the most information. Metadata and call order alone scored 14.53%, below blind guessing. Adding parameters lifted the score to 27.71%.

Caveats and proposed defenses

The users are synthetic, so real people may behave differently, and 26.25% is far from reading someone's mind. The paper also warns that its numbers for what a single server sees (24.24%) and for traces linked across four domains (32.77%) come from different setups, so they show a link, not a proven cause.

The authors suggest three defenses: blur the details in requests, keep traces for less time and in less detail, and stop one user's activity from being linked across services. They state that none of these has been tested yet.