Artificial intelligence company Anthropic is reportedly consulting law firms to understand how AI agents can negotiate agreements on behalf of users, including bargaining over prices, resolving disputes and negotiating employment terms. The initiative is examining how much independence such systems should have and when human intervention should be required.
The development comes as AI companies explore systems capable of making decisions and conducting transactions with limited human involvement.
Why Is Anthropic Consulting Lawyers?
Anthropic has reportedly been speaking with law firms to understand how lawyers negotiate on behalf of clients while protecting their interests and maintaining fairness.
The company is examining whether similar principles could guide AI agents when they negotiate with other automated systems or human representatives.
One of the central questions is how much freedom an AI agent should receive when acting on a user’s instructions.
Anthropic is also studying when an agent should seek additional guidance rather than make a decision independently. The challenge becomes more significant when two AI agents represent parties with competing financial interests.
FCRF Launches CP-FRM to Build India’s Next Generation of Fraud Risk Professionals
How Could AI Agents Negotiate Deals?
An AI agent could potentially negotiate the purchase of a product without requiring its user to compare prices or communicate directly with sellers.
For example, an agent purchasing a laptop might contact a seller’s AI agent, negotiate the price and attempt to secure a more favourable offer.
Similar systems could eventually assist with rental agreements, customer service disputes and employment contracts. In salary negotiations, an AI agent might attempt to secure improved employment terms on behalf of a worker.
However, these possibilities also raise questions about how agents should behave when their instructions conflict with those of another party. Anthropic is exploring what it describes as negotiation norms to help AI agents handle such situations.
The company is also considering signed instructions from users that would establish what an agent is authorised to do and which decisions require additional approval.
What Have Anthropic’s Experiments Found?
In September, Anthropic disclosed that it had tested AI agents in bargaining exercises inspired by real-world negotiations.
Employees used the agents to buy and sell books on a virtual trading floor. The experiments compared agents given more aggressive negotiating instructions with those encouraged to cooperate.
Agents instructed to behave ruthlessly performed slightly better at securing favourable deals. More cooperative agents, however, sometimes accepted less-preferred books to ensure that everyone received something.
The findings illustrate how an AI agent’s negotiating instructions can influence the outcome of a transaction.
They also raise questions about whether systems designed to maximise individual advantage could produce less cooperative outcomes when negotiating with other agents.
Can AI Agents Manipulate Each Other?
Research involving AI-to-AI negotiations has identified potential security concerns.
Harvard Business School assistant professor Michelle Vaccaro conducted more than 180,000 negotiations between AI agents, covering situations including job offers and rental contracts.
Her research found that agents could use techniques such as prompt injection to manipulate other agents into revealing private information.
Prompt injection involves placing instructions within material processed by an AI system in an attempt to influence its behaviour.
In a negotiation, such tactics could create risks if one agent persuades another to disclose information that should remain confidential.
The findings suggest that future negotiating agents may require safeguards against manipulation, particularly when they exchange information with systems representing competing interests.
Could AI Negotiations Create Inequality?
The development of autonomous negotiating agents also raises concerns about unequal access to advanced AI systems.
More capable agents could help users negotiate better financial terms without requiring professional assistance.
However, if the strongest models remain available mainly to wealthier individuals and businesses, those users could gain a negotiating advantage.
This could become particularly important in employment agreements, rental negotiations and commercial transactions, where differences in negotiating ability can affect financial outcomes.
The concern is not limited to whether AI agents can negotiate successfully, but whether access to more powerful systems could influence the fairness of negotiations.
What Legal Questions Remain?
The legal and policy frameworks governing AI agents that negotiate and transact on behalf of individuals remain underdeveloped.
As these systems become more independent, determining the limits of their authority will become increasingly important.
Questions include when an agent should accept an offer, when it should continue bargaining and when it should seek human approval.
Signed user instructions are among the approaches being considered to establish clear boundaries.
However, defining those boundaries may become difficult when negotiations involve unexpected circumstances or competing obligations.
The central challenge is ensuring that AI agents pursue users’ interests without exceeding their authority or compromising confidential information.
Anthropic’s exploration of AI-assisted negotiations points towards a future in which automated agents could bargain over prices, contracts and employment terms. Yet research showing that AI agents can be manipulated into revealing private information highlights the security risks involved. Clear authorisation, confidentiality safeguards and human oversight will be essential considerations as these systems move towards handling consequential financial decisions.
Follow for daily updates on cybercrime, corporate fraud, DFIR, hacking, investigations, and digital forensics
Leave a Reply