Anthropic explores negotiation rules for AI agents with law firms

Defining AI negotiation norms

Anthropic is currently investigating how autonomous AI agents should behave when negotiating on behalf of human users. The company wants to determine how these models should handle specific tasks such as negotiating salaries, prices, and complex contracts. Because there are currently no technical standards or legal guidelines for this, the company is looking for new ways to define these interactions.

To find these answers, Anthropic is reportedly consulting with various law firms. The goal of these discussions is to understand how human lawyers represent their clients' interests while still striving for fair results. The company intends to translate these professional principles into rules that AI agents can follow during automated interactions.

The move comes as the market sees an increasing number of personal AI agents. Anthropic believes it is highly likely that these agents will soon be tasked with handling routine and non-routine negotiations for their owners. Establishing a standard for these interactions is a primary focus for the developer as the technology matures.

Lessons from human behavior

The company's research team has been exploring these concepts for some time. In a study published in September, employees from six different offices participated in an experiment involving the exchange of books. Each employee brought a book they no longer wanted and sought to receive a new book they liked in return.

Instead of the humans negotiating the trade themselves, their personal AI agents handled the process. The results showed that the performance of the agent directly correlated with the quality of the outcome. Specifically, the more powerful the agent was, the better the books it managed to secure for its owner.

The study also revealed how different personality instructions affected the results. Agents that were told to be ruthless achieved better results for their users than those instructed to behave socially. The socially-minded agents sometimes gave up opportunities to ensure that everyone ended up with a good book, mirroring human social dynamics.

Trust and skepticism in AI

Beyond just winning a trade, Anthropic is looking at the nuances of how agents interact with one another. The company is weighing how much human feedback an agent should request during a live negotiation. They are also trying to determine how much suspicion an agent should have when it encounters another AI opponent.

The goal is to find a balance where the agent is not overly paranoid or conspiratorial. However, the interaction between two different AI models requires a level of coordination that does not yet have a clear blueprint. The company is trying to figure out if agents should treat other models as neutral tools or as strategic competitors.

These questions are central to the development of reliable personal agents. If an agent is too trusting, it might be exploited by a different model; if it is too aggressive, it may fail to reach a mutually beneficial agreement. Anthropic is attempting to map out these behaviors before the technology becomes widespread.

The missing legal framework

While the technology is moving forward, the rules governing it are lagging behind. Anthropic has stated that the political and legal frameworks for AI models that conduct transactions in human names do not exist yet. This lack of oversight creates a significant hurdle for the widespread adoption of autonomous agents in the business world.

The company argues that a world where agents speak for humans is not a distant possibility. They suggest that society needs to act quickly to prepare for these changes. Without clear rules, it remains unclear who is responsible when an agent makes a mistake or signs a contract that is unfavorable to the user.

As heise.de reported, the company is trying to bridge this gap by establishing 'negotiation norms' now. By working with legal experts, they hope to create a foundation for how these models should operate. This proactive approach aims to prepare for a future where AI is a primary representative in the global economy.

Moving toward personal agents

This research is happening alongside the practical rollout of new tools. Anthropic recently released its first personal agent this week, marking a shift toward more active AI roles. This launch signals that the company is moving from theoretical research into the deployment of agents that can perform tasks for individuals.

The transition from a tool that answers questions to an agent that performs actions is a significant step. It requires the AI to understand goals, preferences, and the boundaries of its authority. Anthropic's work with law firms is a direct attempt to ensure those boundaries are respected during complex interactions.

As these agents become more capable, the need for standardized negotiation behavior will become more urgent. The company's efforts to define these norms represent an attempt to create a predictable environment for users. They want to ensure that when an agent negotiates, it does so in a way that aligns with the user's actual interests.