Declared Scope Cuts Benign-Session False Positives from 17.6% to 1.1%

Every security layer that watches AI agents has a false-positive problem, and almost nobody publishes their numbers. We measured ours on production, twice. Once the hard way, once with one extra API call. The difference is the point of this post:

Same corpus, same production deployment, same policies. The only difference: in the second run, each agent declared what it was for before it started working.

Why agents make false positives worse, not better

A web application firewall inspects one request at a time, and a false positive costs one retry. Agent traffic is different in two ways that compound. First, the content is structurally alarming: a legitimate database-maintenance agent emits DROP TABLE, a legitimate incident-response agent discusses exploits, a legitimate sysadmin agent handles credentials all day. Judged without context, routine work looks like an attack in progress.

Second, one false positive doesn't cost one retry; it interrupts the whole job. An agent session is a chain of dependent steps; refuse step 14 of 47 and the remaining 33 steps don't happen. That's why we report the session-level rate alongside the message-level rate, and why the session number is the one that describes the operator's actual experience. A message rate quoted alone makes any tool look calmer than it feels to run.

The measurement

The corpus is the benign side of ADR-Bench: 261 realistic multi-turn agent sessions, 9,103 messages of prompts, tool results, and agent responses across ordinary work like system administration, web research, and project management. Every message is labeled benign, so every refusal is by definition a false positive. We ran it against our production scan API, full pipeline, with per-message and per-session results recorded.

Two practices behind the numbers, because a benchmark you can't audit is an adjective: every report is stamped with the endpoint it actually ran against, and each run begins with a preflight that verifies the target environment loaded its full policy surface. If the environment can't prove that, the run aborts rather than producing a confident number about a configuration nobody ships. And to be explicit about scope: this measurement is about false positives on benign traffic. It says nothing about detection of malicious traffic, which is a different measurement with its own corpus. These figures are measurements of our production system on the named corpus on the stated date; results on other workloads will differ.

What one declaration changes

The anonymous run's failure anatomy was lopsided: nearly every interrupted session traced back to our LLM-analysis layers judging content without knowing who was acting. That is not a tuning problem: no threshold distinguishes "database agent doing database work" from "compromised agent probing a database" when the classifier doesn't know which agent it's watching. It's a missing-context problem.

Scope declaration supplies the context. The operator registers, per agent, what it's for and which tools it may use:

POST /api/v1/agent/scope/declare
{
  "agent_id": "db-maintenance-01",
  "purpose": "Scheduled database maintenance: vacuum, reindex,
              integrity checks on the analytics cluster",
  "allowed_tools": ["sql", "command"],
  "max_duration_seconds": 7200
}

From then on, scans carrying that agent_id are evaluated with the declared purpose in view. The LLM-analysis layers see "scheduled maintenance agent, authorized for SQL" and judge the DROP TABLE accordingly. In the scoped run, every one of the 40 sessions that the LLM-analysis layers had wrongly interrupted came through clean. The three sessions still interrupted were flagged by context-independent pattern checks: the floor a defense-in-depth pipeline keeps on purpose, because some patterns warrant a look regardless of who sends them.

Two properties worth noting. Declaring a scope is not just a whitelist. The same declaration also enforces: a scoped agent that calls a tool outside its declared list is held for approval, and an agent still running past its declared window is flagged. The context that lowers false positives and the boundary that catches overreach are the same object. And it's strictly opt-in: agents that declare nothing are evaluated exactly as before. The operator declares; Shrike enforces what was declared.

The general lesson

We think this generalizes beyond our product: for AI agents, identity-and-intent context is worth more than threshold tuning. Loosening thresholds trades false positives for misses. Supplying context removes the guesswork that produced the false positives, without loosening anything. The cheapest precision available to any agent-security layer is the operator telling it, once, what the agent is for.

If you run agents behind Shrike, the declaration above is the whole integration. The agent scope docs cover the API, the MCP tool, and the SDK helpers. Where scope fits the bigger picture is in what is action governance and the agent security stack.