SI Agent News
DevelopingAgentsPolicySecurity

Uber builds security framework to govern autonomous AI agents

The company is moving beyond simple API permissions to create context-aware controls that monitor how AI agents reason and act in real-world scenarios.

Isometric diorama: picture for the story 'Uber builds security framework to govern autonomous AI agents'
AI-made isometric diorama for this story. No real people or products are shown.

The short version

Why it matters

As companies move from chatbots to autonomous agents that can perform tasks like booking flights or moving files, traditional security is no longer enough. This framework provides a blueprint for how organizations can let AI act independently without losing control over its behavior.

The limits of traditional security

Standard security systems are designed to answer one basic question: does this user or service have permission to access this specific resource? While this works well for humans and static software, it fails to address the unique risks posed by autonomous AI agents. These agents can reason dynamically over external content and take unpredictable actions based on their internal logic.

Because agents can be influenced by external inputs, they are susceptible to several specific threats. These include prompt injection, where a user tricks the AI into ignoring its instructions, and tool or memory poisoning, where the agent's underlying data is corrupted. Other risks include behavioral drift, where an agent's actions slowly deviate from its original goal, and hallucination-driven actions, where the AI performs a real-world task based on a false premise.

Uber Engineering identified that identity and authorization systems cannot determine if a proposed action is actually appropriate in a given context. To solve this, the company developed a framework that evaluates the user's request, the agent's intended purpose, its specific permissions, and the overall context of the interaction. This approach moves security from a binary 'yes' or 'no' to a more nuanced evaluation of intent.

Registering agents with Argus

The first layer of the framework occurs during the provisioning phase, where every agent must be registered in the Uber AI Registry. This registry establishes the agent's identity, ownership, persona, permitted intents, and risk classification. By creating a formal record for every agent, the company can ensure that every AI tool has a defined set of constraints and dependencies before it ever goes live.

To make this process manageable, Uber uses an automated tool called Argus. This system analyzes the agent's source code, prompts, configurations, and exposed endpoints to identify potential security weaknesses. It also enforces the same level of scrutiny on the Model Context Protocol (MCP) servers and tools that the agents use to interact with the world.

By automating this analysis, the framework ensures that security is baked into the development lifecycle rather than added as an afterthought. It allows developers to identify risks early, such as insecure tool-invocation paths, before the agent is deployed to a production environment. This proactive stance helps prevent agents from being granted capabilities that exceed their intended use cases.

Behavioral authorization at runtime

Once an agent is active, the framework shifts to runtime enforcement through a model of behavioral authorization. This system goes beyond static policies to evaluate whether an agent's specific move is appropriate for the current situation. It relies on three main components: the AI Registry, AI Guards, and Agent Guard runtime hooks to maintain constant oversight.

The AI Registry manages the core policies and personas, while the AI Guards serve as the evaluators of interactions. When an agent reaches a critical checkpoint, the Agent Guard runtime hooks invoke the AI Guard to receive a verdict. Only after the guard confirms the action aligns with the established policy is the interaction or action allowed to proceed.

This structure allows for more precise enforcement because it is tailored to the specific agent. Rather than applying generic thresholds to all AI, the system uses agent-specific policies that make enforcement more contextual. This helps ensure that an agent designed for customer service doesn't suddenly attempt to access sensitive internal financial data, even if it technically has the permission to do so.

Continuous assurance and auditing

The final layer of the framework focuses on continuous assurance after the agent has been deployed. This involves end-to-end observability, auditing, and explainability to ensure the system remains safe over time. Uber uses a unified execution trace to connect every agent action, tool call, and security decision into a single, searchable record.

In addition to traces, the framework records interactions that can be replayed at any time. This allows engineers to test how agents behave when policies or guardrails are updated. By replaying these interactions, the team can assess the effectiveness of new rules in different contexts without having to wait for a new real-world incident to occur.

This continuous loop of monitoring and testing is essential for maintaining trust in autonomous systems. It provides a way to audit the AI's reasoning and ensures that any drift in behavior can be caught quickly. By combining real-time enforcement with historical replay, the framework creates a comprehensive safety net for complex AI workflows.

Implementation challenges and costs

Implementing these controls in a production environment is not without its difficulties. Uber noted that they must balance policy restrictiveness to avoid both excessive false positives and insufficient protection. They also face the technical challenge of minimizing runtime latency, as every security check adds a small amount of time to the agent's response.

There is also the human element of developer friction, as the framework must be easy for engineers to set up without slowing down the pace of innovation. Experts have noted that while per-agent personas and constraints add significant value, they come with a high cost. As Tristan G. of the UN noted, the effort required to manage these details scales with the number of workflows an organization runs.

Despite these costs, the framework prioritizes explicit intents and constraints over generic thresholds. AI architect Pratik Dhanave observed that these specific constraints are the most effective way to manage risk because every agent has a different purpose. While this approach reduces flexibility for the developer, the framework suggests that this trade-off is necessary to ensure the safety of autonomous AI actions.

What to watch

Try it

Look for 'Agent Guard' or 'behavioral authorization' in upcoming enterprise AI security whitepapers to see how companies are moving beyond simple API keys.

Sources and links

Liked it? Say why

Add a line for other readers.

No account needed. We never show your email. Comments with links, ads or abuse are held and not shown. Your comment shows after a quick check.

Listen to the AI Wire headlines

Listen to the AI Wire

Listen to our daily AI shows every morning: on your commute, in the shower, wherever you are

Listen to the daily shows

First reported by infoq.com · Read the original. Written by the SI Agent News Desk from the reporting above.Written with AI from the source above. Every number is checked against the source. Spot a mistake? Corrections.

More AI news