Four Approaches to Governing an AI Agent

Four approaches to governing an AI agent are in common use: scoping the credentials it holds, evaluating policy rules at runtime, allowlisting its actions at design time, and adjudicating each action at the moment it is attempted. The first three decide whether an action falls inside a permitted set. The fourth decides what should happen to this action, now.

The distinction matters because the first three share a shape. Each defines a set of permitted actions and then checks membership, differing only in when the set is defined and how membership is tested. A membership test answers in or out, and most of the difficulty in governing agents lives in the cases where in or out is not the answer the situation calls for.

Is scoping an agent’s credentials enough?

The first approach gives the agent an identity, grants that identity a set of permissions, and relies on the boundary of those permissions to contain it. This is ordinary access control and it is load-bearing. Nothing below replaces it.

Its limit is that it decides once, at grant time, and the decision is about reach rather than conduct. Inside the granted boundary every action is equally permitted, so an agent that holds a valid token and uses it exactly as designed can still take an action nobody would have approved. When that happens, authentication succeeds, authorization succeeds, the call is well formed, and every control reports success. There is no failure to detect anywhere in the sequence. We wrote about a public incident with exactly this shape, where a valid token and a correct API call removed a production database and its backups.

The practical consequence is that over-scoped credentials are the most common amplifier of agent incidents, and scoping tokens to an environment and an operation is the highest-value afternoon of work available to most teams. It narrows the blast radius. It does not evaluate the action.

Why do policy rule engines struggle with agents?

The second approach expresses policy as rules and evaluates them when the agent acts. It is familiar, auditable, and it maps well to how compliance teams already write requirements. It is also the most mature approach on this page. The vocabulary was standardised two decades ago and the engines that implement it are well understood, which is a genuine advantage over anything invented for agents last year.

The difficulty is not that the decision model is too coarse. XACML 3.0 defines four decision values, not two: permit, deny, indeterminate, and not-applicable. But indeterminate means the evaluation itself failed, and not-applicable means no rule matched the request. Both describe the state of the policy rather than the merits of the action, and none of the four means a person should decide this one.

That absence is what agent work exposes. Agent tasks are long-running and cross-cutting, so several rules legitimately apply to the same action and can disagree: one protects a data class, another authorizes the task that needs it. A rule engine settles that with a combining algorithm chosen in advance, deny-overrides or permit-overrides or first-applicable. That choice is sound engineering and it is what makes evaluation deterministic and reviewable. It also means a real conflict is resolved by a precedence rule written before anyone knew the conflict would occur. Deny-overrides blocks work the agent was legitimately asked to do, which pushes teams to loosen rules until the conflicts stop appearing. Permit-overrides lets through the case that most deserved a second look. Neither is a bug in any individual rule, and no amount of further rule-writing produces the response the situation actually calls for, which is to ask someone.

What does a design-time allowlist assume?

The third approach defines the agent’s permitted action space ahead of time, rather than enumerating prohibitions, and constrains the agent to search inside it at runtime. Defining what an agent is for is a real improvement on listing what it may not do, because an agent given only prohibitions has no positive description of its job and will explore the boundary.

What it assumes is that the space can be specified completely before the work starts, and that membership in the space is a sufficient answer. Both hold for narrow, repeatable tasks. Both strain when the work is open-ended, when the same action is appropriate in one context and not another, or when what matters is not this action alone but the sequence it belongs to.

What is action-time adjudication?

The fourth approach does not ask whether the action is in a permitted set. It asks what should happen to this action, at the moment it is attempted, given the purpose declared for the agent, where the content came from, and what has already happened in this session. Four properties distinguish it:

• Four answers, not two. Proceed, proceed and record it as unusual, hold for a human decision, or refuse. The third answer is what a conflict resolves to, which is why conflicting considerations escalate instead of deadlocking.
• The refusal is addressed to the agent. A refusal carries an instruction describing an allowed path. An agent told only that it failed will look for another route; an agent told what to do instead can take it.
• The session is the unit. Attacks and accidents alike unfold across turns, where each step is individually defensible. Evaluating each call in isolation cannot see a trajectory.
• Permitted and unchecked are different outcomes. A control that reports the same result for “this was allowed” and “nothing evaluated this” cannot tell you where your governance is absent. Recording the difference is what makes a gap visible before it becomes an incident.

How do the four compare?

The last row is the honest one. Adjudication inherits the operator’s declaration: if an operator declares that an agent may delete infrastructure, then an agent deleting infrastructure is authorized behaviour. The operator holds that authority, which means the operator can also get it wrong. A layer that overrode the declaration would be a worse control, not a better one.

Which approach should you use?

They compose. A deployment that adjudicates every action but hands its agents account-wide credentials has simply moved where the damage starts. A deployment with tightly scoped tokens and no evaluation of individual actions has narrowed the blast radius of a decision nobody reviewed. The useful order is unglamorous: scope the credentials and keep a restore path outside the blast radius of the thing it protects, because those are cheap and they bound the worst case. Give each agent a stated purpose rather than a list of prohibitions. Then decide which actions in your environment are consequential enough that an agent should have to wait for a person, and put something in that interval capable of making it wait.

Sources: XACML 3.0 for the decision values and combining algorithms; AWS IAM policy evaluation logic for the difference between an explicit deny and nothing having matched; FINOS AI Governance Framework MI-18 and Microsoft on identity, access and tool binding for credential scoping and design-time allowlists; Before the Tool Call and AgentSpec for pre-execution enforcement at the moment of the action.

Related concepts

• What is action governance? The definition this page sits under.
• The governance surfaces of an agent: where an agent touches the world, by plane.
• When does an agent need governance? The threshold question, before the approach question.
• Authenticated is not authorized: a public incident where every control succeeded and the data was still lost.
• Agent scope declaration: declaring what an agent is for, and enforcing it.
• The AI agent security stack: where each control layer sits.