AI service chains leak sensitive user data through insecure routing
Researchers found that AI routers and resellers often log prompts and ignore privacy settings, exposing secrets like AWS keys and private URLs.

The short version
- At least three AI hosting providers were breached by attackers who siphoned prompts and reused stolen AWS keys.
- Multiple resellers logged prompts and passed them to third parties despite users selecting privacy-focused settings.
- A Chinese AI lab was found scraping secret URLs from over half of the prompts sent to its models, including those marked for Zero Data Retention.
- Prompts are decrypted at every 'hop' in the chain, allowing routers and resellers to read and store plaintext data.
Why it matters
If you use an AI router or a third-party reseller to save money or access specific models, your data is likely being logged by multiple companies. This makes it nearly impossible to guarantee that sensitive information like API keys or private URLs stay private.
The canary experiment
Security researchers recently conducted a large-scale experiment to see how sensitive information travels through the AI ecosystem. They planted 2 million digital tripwires, known as "canaries," into prompts sent to a wide network of AI services. These canaries were unique, trackable secrets like AWS access keys, passwords, and private git repository references.
The goal was to identify exactly which links in the AI supply chain were responsible for leaking data. Because the researchers controlled the infrastructure that received these secrets, they could see exactly where the information surfaced. The study revealed that the current AI ecosystem leaks highly sensitive data like a sieve.
As Ars Technica reported, the findings showed that many of today's AI applications rely on brokers known as AI routers or gateways. These services receive prompts from end users and route them to various inference engines based on factors like price or task suitability. This complex chain of hand-offs creates numerous opportunities for data exposure.
How the chain leaks data
While prompts are typically encrypted while moving across the internet, they must be decrypted at every step inside the reseller network. An AI router must read the plaintext of a prompt to determine which model is best suited to handle the request. Once decrypted, the prompt can be read, logged, and stored by any player in the chain.
The research found that many resellers in the inference marketplace log these prompts and pass them along to other parties. Some of these players offer low prices specifically to lure prompts to their servers, where they then turn on request logging. This allows them to capture data from users who may not realize their prompts are being stored by a middle-man.
The study also highlighted a failure to honor specific privacy flags. Even when users selected settings for Zero Data Retention (ZDR) or "no data collection," some providers ignored these instructions. The data was still captured and processed, proving that privacy settings are not always strictly enforced by the companies providing the AI models.
Evidence of active breaches
One of the most alarming findings involved active security breaches at three separate hosting providers. Attackers who compromised these providers were able to siphon prompts as they arrived at the provider's front door. These attackers then tested the stolen bait keys to see what resources they could access, including S3 buckets and secrets managers.
The researchers observed that the stolen keys were used tens of thousands of times in AWS API calls. In some instances, the stolen secrets were used almost immediately from the same rented server. This suggests that the providers were likely compromised by the same adversary who was systematically harvesting user credentials.
The study also uncovered evidence of prompt theft by the resellers themselves. By comparing requests sent to the same model through different sellers, researchers could identify which specific reseller was responsible for the leak. This happens because the reseller captures the data while the model provider does not, creating a clear trail of where the privacy violation occurred.
Scraping and geographic risks
The experiment also revealed that some AI labs are actively scraping secret URLs included in user prompts. A Chinese AI lab was found fetching secret URLs from more than half of the prompts sent to its models between July and October. This occurred even when the requests were specifically marked for no data collection or Zero Data Retention.
This scraping behavior was not limited to users who interacted with the lab directly. Several US hosting providers resell this lab's models, and prompts sent to those US providers triggered the same scraper. This means customers who chose a US provider specifically to keep their data within the United States did not actually receive that protection.
The researchers also found instances of scrapers appearing days or even months after a request was made. One US hosting provider was found crawling a secret URL more than two months after it was used in a single inference call. These findings indicate a widespread lack of oversight regarding how data flows back to the original model creators.
How to protect your data
The research suggests that the most effective way to prevent data leakage is to host AI models on your own on-premises servers. While this is often the most expensive option, it provides the highest level of security because it removes the need to trust third-party routers or resellers. For most users, however, this level of infrastructure is not practical.
For those who cannot self-host, the study suggests receiving services directly from a first-tier provider. This means dealing with one company that provides a single contract and a single privacy policy to read. This approach minimizes the number of "hops" your data takes, which reduces the number of entities that can potentially log or see your prompts.
Users should also be cautious about accepting default provider or reseller pools. Instead of letting an app choose the cheapest or most convenient route for your data, you should try to pick upstream providers yourself. The study serves as a warning that ensuring secrets stay private in the AI era is a daunting task that will not become easier anytime soon.
Sources and links
- ReportingArs Technica: the article we sourced
Read the original at Ars Technica
First reported by Ars Technica. Written by the SI Agent News Desk from the reporting above.Written with AI from the source above. Every number is checked against the source. Spot a mistake? Corrections.
Liked it? Say why
Add a line for other readers.
No account needed. We never show your email. Comments with links, ads or abuse are held and not shown. Your comment shows after a quick check.