How Are AI Agent Policies Written?

An AI agent policy is the set of rules an operator declares about what an agent may say and do, in a form a runtime can enforce. Three shapes exist today: a configuration object of filters and lists, rails written as code inside the application, and a scope with rules declared outside the agent and enforced at the point of action.

Every guardrail product asks the same question in a different form: what should this agent be allowed to say and do? The answer is a policy, and the shape of the policy decides who can write it, where it lives, how it changes, and what it can see. Three shapes are in use today. The differences between them are not features. They are the difference between a policy that governs text and one that governs actions, and between a policy the operator owns and one the agent's own code carries.

What are the three shapes an agent policy takes?

The first two shapes were built for chat: a person types, a model answers, and the policy judges the words. An agent that acts adds a third surface, the action, and only a policy that can name an action can govern one. That distinction is what the rest of this page turns on.

What does a configuration-object policy look like?

One object per guardrail, with a block per kind of check. The operator fills in the blocks:

• Content categories. A fixed list, each with a strength from off to high, set separately for input and output. The categories are the vendor's. You tune them; you do not add to them.
• Denied topics. A name, a definition in a sentence or two, and a handful of example phrases. The service decides whether a piece of text is about the topic.
• Word lists. Exact terms to catch, plus a managed profanity list.
• Sensitive data. A list of entity types (names, account numbers, credentials) with block or mask, and custom patterns for formats the list does not know.
• Grounding. A threshold for how closely a reply must follow its sources.
• Formal rules, in some services: a written policy document compiled into logic rules and attached to the object.

Every object is numbered, and a call names the version it runs under, so a change is a new version and a rollback is an old one. Every block carries a fixed cap: how many topics, how many example phrases per topic, how many custom patterns and how long each may be, how many words, how many versions. The caps are set by the vendor and are not adjustable. The object's strength is that there is nothing to deploy and nothing new in the bill. Its edge is what it can see: text. A command, a file path, or a query is judged only as far as it looks like harmful text, and the object has no notion of which agent is calling or what that agent was deployed to do.

What does a rails-as-code policy look like?

A configuration directory inside the application. A YAML file names the models, the general instructions, a sample conversation, and which rails are active, grouped by where they run: on input, on output, in the dialog, around retrieval, and around execution. The rails themselves are flows in a small modeling language: canonical forms for what a user might say and what the bot should answer, each with example utterances, and flows that sequence them. A check inside a rail is one of three things: a prompt that asks the model to judge its own input or output, a call to a classifier service, or a function the developer wrote. Execution rails can wrap a tool call, and each one is custom code.

The strength is expressiveness and ownership: the policy can say anything code can say, runs anywhere the application runs, and is open to inspection. The edge is where it lives. The policy ships with the application, so changing it is a deploy. And it runs inside the process the agent controls, so the separation between the agent and its governor is a convention of the codebase rather than a boundary. An agent that can edit its own configuration can edit its own rails.

What does a declared policy look like?

Two parts, both written outside the agent. The first is a scope, declared per agent: what the agent is for, the tools it may use, the actions it may never take, an expiry, and the mode it runs in, enforcing or observing for a bounded window. Paths an agent may write to are a grant the operator makes, never something the agent infers. The second is a set of rules: a pattern, an action from observe through block, a severity, and the stages the rule applies to, which can be a person's prompt, the model's reply, a shell command, a file write, a database query, a web search, or a message to another agent. Built-in rules arrive with an authored severity and stages; an operator can turn defaults off, down to a floor the service keeps, and add custom rules beside them.

Enforcement happens at the action boundary, outside the agent, so the verdict does not depend on the agent cooperating, and the record of each decision cites the rule and the scope that produced it rather than a vendor's opinion. Changing a rule is an edit: the version increments, the difference is written to an audit trail, and the enforcement point refreshes its view of the policy on its own, so the change reaches every channel without a deploy.

What if you have no AI policy yet?

Most teams deploying their first agent have no written AI policy, and the honest question is what governs the agent on day one, before anyone writes anything. In the first two shapes the answer is nothing: a configuration object governs nothing until one is created and attached to a call, and rails govern nothing until they are written. The declared shape can ship a floor, and a floor worth the name has three parts:

• Essentials no operator can turn off. Credential material in any direction, including a credential sent to a network destination; the sensitive data every regulator names, redacted rather than refused so the work continues; and the patterns that try to rewrite an agent's instructions. These apply at every tier, because they are patterns, not judgments.
• A safety floor nothing can soften. A small set of threats that no approval policy, no scope, and no observe window may lift, applied only when a deterministic rule agrees with the model. A judgment the model reached alone stays a judgment a person can overrule; the floor is for verdicts with evidence.
• Built-in rules with an authored severity. A set of defaults, each naming the stages it applies to, which the operator can turn off down to the essentials or extend with their own.

What the floor cannot answer is the authorization question: not whether this action is hostile, but whether this agent should be taking it at all. That needs a declared scope, and a service that is honest about the gap records the absence of one as permission by absence, never as a control that approved the action. The floor is the policy a team has before it writes one. The way to the one it writes is the agent's own traffic: run under the floor with the scope observing for a bounded window, read what would have been refused, label the false positives, let the service draft a scope from what the agent actually did, turn defaults off or on, add the rules the traffic shows are missing, and export the bundle. That file is the team's first written AI policy, derived from evidence rather than copied from a template.

How do you write a rule without being a regex expert?

A pattern rule is precise and fast, and most people cannot write one from memory. The usable approach is a drafting aid with a test built in:

1. Describe the pattern in plain words, and give examples that must match and counter-examples that must not.
2. A model proposes a pattern and an explanation of what it matches.
3. The draft is run against both sets. A pattern that misses an example or catches a counter-example is rejected, and the aid tries again a bounded number of times.
4. The pattern is compiled in a linear-time engine, so a draft cannot stall the scanner, and a complexity bound rejects nested repetition that a person could not reason about.
5. A person reviews the pattern and the test result before it is saved.

What is saved and enforced is the pattern. The description and the examples are inputs to drafting it, and the review step exists because the rule does what the expression says, not what the description meant. A rule that can block should be tested against a sample before it is allowed to.

How does a policy change reach a running agent?

Each shape has its own answer, and the answer is most of what an operator lives with:

• Configuration object. Publish a new version, then point each call at it. Rollback is the previous version. Nothing reaches the agent until the call site changes.
• Rails as code. The policy is source, so the change goes through review, build, and deploy with the rest of the application. Every environment the agent runs in needs the new build.
• Declared scope and rules. The edit is saved with a version and an audit diff, and the enforcement point refreshes on its own within about a minute, on every channel at once. The agent is not redeployed and does not know the policy changed, which is the point: the policy is not the agent's to notice.

A policy also has to travel: from the environment where it was authored and tested to the one that serves traffic, and out of the vendor's database into the operator's own keeping. A declared policy does this as a bundle. Export the custom rules and the toggles on defaults as JSON or YAML, keep the file in a repository, review a change the way code is reviewed, and import it into the next environment. A dry run computes the receipt first: what would be created, updated, left unchanged, or skipped and why, with the field-level differences, before anything is written. Re-applying the same bundle changes nothing, so a promotion is repeatable, and a rule the bundle manages can warn when someone edits it by hand.

Should a policy format have limits?

Yes, and the question is which kind. Two kinds of limit appear in policy formats, and they do different work.

A cap on how much you may declare. So many topics, so many example phrases per topic, so many patterns of at most so many characters, so many versions. These caps bound the vendor's cost per check. They also bound your policy: when the cap is reached, the next rule you need does not fit, and the policy you wanted shrinks to the policy the format allows.

A bound on the shape of a rule. It must compile in an engine that cannot be made to run forever, it must stay under a complexity ceiling, and an import of many rules at once has a size. These bounds protect the scanner and the person who has to read the rule later. They never shrink the policy; they refuse a rule that could not be evaluated or explained.

A format that bounds shape and not count asks the operator to own the policy's size the way they own the rest of their security posture. A format that caps count makes the size the vendor's decision. Before committing a policy to a format, ask which kind of limit you will meet first, and what happens to the policy when you meet it.

What should you check before trusting a policy format?

• A floor before a policy: what governs the agent on day one, before anything is declared, and is the absence of a scope recorded as absence rather than as approval?
• Change without a deploy: can the operator change the policy without touching the agent, and does the change reach every channel the agent uses?
• Versioned with a diff: is every rule versioned, with the difference and the person who changed it on record?
• Portable: can the policy leave as a file, be reviewed in version control, and be imported into the next environment with a preview of what would change?
• Actions, not only text: does it govern the command, the file write, the query, or only the words around them?
• The verdict cites the rule: does a refusal name the rule and the scope that produced it, so the record says what the policy said?
• A hold reaches a person: does a held action wait for an approver, and does a refusal carry its way back?
• Shape, not count: when the format refuses a rule, is the limit on the rule's shape or on how much you may declare?
• The agent cannot widen it: can the agent's own credentials change the policy? The right answer is no.

Related concepts

• What is action governance? The discipline a declared policy is written for.
• Four approaches to governing an AI agent: credential scoping, rule engines, allowlists, and action-time adjudication, and where each stops.
• The 14 governance surfaces of an AI agent: the stages a rule can apply to.
• Agent scope declaration: the per-agent half of a declared policy, in the documentation.
• Custom policy in the glossary.