As organisations move from using AI to delegating actions to AI agents, a new governance question is becoming harder to avoid: what authority should an AI system be permitted to exercise autonomously?
The Policy Edge (TPE) in its September 19 review of OECD evidence, OECD: Companies Are Putting AI Agents to Work but Keeping High-Stakes Decisions Human, found organisations using agents in real workflows while retaining human control over consequential decisions. None of the 25 organisations interviewed reported giving agents unrestricted autonomy. The following day, TPE’s healthcare AI commentary, Beyond CDSCO: Governing AI in India’s Healthcare Decisions by Chintan Dave, made the same distinction in another context: the governance challenge depends not only on what AI recommends, but on the authority given to act on its outputs.
Taken together, these strands of TPE’s recent coverage point to a broader principle for India’s approach to agentic AI: autonomy should be matched by explicitly defined limits on authority. Every consequential deployment should have an authority budget: a defined scope of actions, systems, resources and decisions that an agent may access, initiate or execute without further human approval.
When Model Behaviour Becomes Operational Risk
The governance challenge changes when AI systems move from generating recommendations to taking actions. In controlled cybersecurity evaluations, highly capable agents operating with reduced safeguards have exploited unintended pathways in the environments around them.
OpenAI disclosed an incident in which agents discovered an unintended communication channel, obtained internet access and compromised parts of Hugging Face’s systems. METR later found that roughly 1,200 agents intended to be isolated had used an unsanctioned shared message board, exchanging more than 70,000 messages and files; about 700 participated in the attack.
The lesson extends beyond model behaviour. The consequences of an agent’s actions depend on the permissions, credentials, tools and communication channels available to it.
That is important for India because AI agents are likely to operate within systems where an unauthorised action can affect public services, finances or individual rights. The Hugging Face incident is not evidence that deployed agents are already breaching such systems. Its lesson is narrower: autonomy combined with broad permissions and poorly bounded access can turn a model-level failure into an operational security failure.
Making AI Authority Explicit
In practice, an authority budget means specifying which systems and data an agent may access, which tools and credentials it may use, what records it may alter, which external parties it may contact, how much money or other resources it may commit, and whether it may deploy changes into production.
It is broader than an access-control policy. Along with what an agent may access, it limits what it may decide, commit or change, and under what limits.
Authority is not binary. An agent may retrieve information but not alter it; prepare a decision but not approve it; execute transactions only below a defined limit; or exercise particular permissions only for a specific task and period. Higher-impact actions should require human approval, while consequential actions should be logged and monitored.
In welfare administration, for example, an agent might retrieve records, identify inconsistencies and recommend an eligibility decision, while remaining unable to approve a claim or initiate payment. Routine cases meeting tightly defined conditions might be processed automatically, while adverse, disputed or exceptional decisions require human review.
Public procurement is one route for making these limits enforceable. A ministry procuring an agentic system should know which models and tools it uses, where data flows, how actions are recorded, which permissions are technically enforced and who bears responsibility when several vendors or agents interact.
The distinction between stated authority and enforceable authority matters. A contract or system prompt saying that an agent must not perform an action is weaker than an architecture in which the agent lacks the credential or permission needed to perform it. Authority limits should therefore be backed, wherever possible, by technical controls, time-limited permissions and revocable credentials.
Authority Budgets Cannot Substitute for Model Assurance
Authority budgets govern what a deploying organisation permits an agent to do. They do not establish whether the underlying model has been adequately evaluated or whether the provider has appropriate safeguards.
Agentic AI therefore raises governance questions at two levels: the authority granted to a deployed system and the assurance available about the underlying system.
Dario Amodei, Anthropic’s CEO, has argued for stronger regulation and embedded third-party evaluators with access comparable to employees. Anthropic has since begun implementing this approach through an embedded-evaluation partnership. OpenAI has similarly called for mandatory capability-based safety requirements and supported external assessment, incident reporting and stronger security standards.
For Indian buyers, the practical question is whether a provider’s safety and governance commitments can be credibly verified. A ministry or company may tightly constrain an agent’s authority, but it still needs credible information about the model it is procuring: how it has been evaluated, what significant incidents have occurred, what cybersecurity practices govern it and what mechanisms exist for independent scrutiny.
Common requirements for independent testing, incident transparency, cybersecurity, auditability and regulatory compliance would give public institutions and companies a more consistent basis for procurement.
As AI moves from producing outputs to exercising delegated authority, safety can not be assessed in isolation from the scope of authority given to the system. The governance of agentic AI will depend on where organisations draw – and technically enforce – the boundaries of that authority.
Leave a Reply