SHRIKEThe Stack You Run When You Go Local
The guides for running an open-weight model in production agree on the stack you should assemble around it. Read together, they are a bill of materials. This post lays that bill out row by row, says honestly which rows the open-source ecosystem covers well, and names the rows that no local stack ships. The point is not that going local is wrong. The point is that the migration ticket rarely lists what left with the provider.
The bill of materials
| Row | What it does | The self-hosted answer | Who runs it |
|---|---|---|---|
| Input classifier | Labels a prompt as an injection or jailbreak attempt before the model reads it | Prompt Guard as a fast gate, Llama Guard for hazard categories | Your team: two models to serve, version, and tune |
| Output validator | Checks the completion for structure, policy, and leakage | Guardrails AI, LLM Guard | Your team |
| Dialog and execution rails | Steers conversation flow and gates tool execution by programmable rules | NeMo Guardrails, with rails written in Colang | Your team: rules to author and maintain per agent |
| PII handling | Finds and redacts personal data on the way in and out | Presidio, or the redaction in the rails above | Your team |
| Action gate | Evaluates the command, file write, query, or message an agent is about to execute, not just the text it read | Partly: execution rails cover the tool calls they were written for | Your team, per tool, per agent |
| Scope | Declares what this agent is for, which tools it may use, and for how long, so "outside the job" is decidable | Not in the stack | Nobody |
| Human approval | Holds a consequential action for a person, with a recovery path for the agent | Not in the stack; usually a bespoke queue | Nobody, or an internal build |
| Session memory | Correlates turns so a pattern across a session is judged as a pattern, and a locked session has a way back | Not in the stack | Nobody |
| Evidence record | Every allow, warn, hold, and block on one record an auditor reads and a SIEM ingests | Application logs, assembled by hand at audit time | Your team, and your auditor |
The first four rows are the content half, and the open-source projects that cover them are good at it. A typical production stack runs Prompt Guard and Llama Guard under NeMo Guardrails, with LLM Guard scanning and Guardrails AI validating outputs (Particula, Spheron). If your team has the headcount to serve, version, and tune five components, the content half is available today, and NeMo's execution rails reach part of the way into the fifth row.
The last four rows are the action half. They are not classifier problems, so no better classifier fills them. They are the questions a security reviewer asks about an agent: what was it allowed to do, who decided the hard cases, what happened across the session, and where is the record. On a hosted API the provider answered some of these on its own terms. On a local stack nobody answers them until somebody builds the answer.
Where the work went
Every row in the table that reads "your team" is a job that used to sit on the provider's payroll. That is the honest accounting of a local-model migration: the risk did not go away, it changed employers. Datanorth says it plainly, open-weight models ship without the guardrails built into hosted APIs and filtering has to be added deliberately (source). Baytech names the operational side: patching, versioning, and lifecycle move to the internal team (source). Neither is an argument against going local. Both are arguments for reading the bill before signing.
The failure mode is not the team that reads the bill and decides to build. It is the retailer whose agent handles returns, whose platform team moved the model on-premise for a sound residency reason, and who now employs three engineers to run a security product instead of selling. They did not choose that. It arrived as a side effect of a ticket that said "data residency" and nothing else.
Two ways to fill the bottom rows
Build them. Some organizations should: those with the scale, the regulatory posture, and the security engineers to own a governance layer as a product. The rows above are the specification, and the open-source content half is a good foundation for it.
Or take them as a service that runs at the boundary you already own. Shrike supplies the action half, and the content half with it, at the SDK, MCP, or proxy layer, for a local model the same as for a hosted one: one base URL points the SDK at Ollama or vLLM, and the policy, the scope, the approval hold, the session memory, and the evidence record apply unchanged. It is a subscription that starts free, and for organizations whose residency reason extends to the governance layer itself, it deploys inside the boundary under an Enterprise engagement. Either way the model stays local, the controls stay yours, and the engineers go back to the shop.
Start with the local models guide. The two halves of the problem are laid out on self-hosted LLM security, and the rows are defined on what is action governance.