Local Models

Moving an agent to a local model is a sound decision: data stays on your hardware, cost becomes predictable, and no provider can change the model under you. It also removes every provider-side control at the same moment. Ollama, vLLM, and LM Studio ship with no content safety, no approval step, and no record of what the agent did with the model's output. The gap is not in the model. It is in the controls that used to sit beside it.

Shrike's controls do not sit inside the provider, so they survive the move. The same SDK, the same policy, the same agent scope, and the same per-action evidence record apply to a local model as to a hosted one. The integration is one base URL.

Point the SDK at the local endpoint

Local runtimes expose an OpenAI-compatible endpoint. The OpenAI wrapper in shrike-guard forwards a base URL to the provider client and governs every call before it reaches the model:

from shrike_guard import ShrikeOpenAI
 
# Ollama serving llama3 locally, governed by Shrike before every call
client = ShrikeOpenAI(
    api_key="ollama",                       # local servers ignore the value
    base_url="http://localhost:11434/v1",   # the MODEL endpoint: local
    shrike_api_key=os.environ["SHRIKE_API_KEY"],  # governance runs server-side
)
 
response = client.chat.completions.create(
    model="llama3",
    messages=[{"role": "user", "content": "Summarize this ticket..."}],
)

Go, with shrike-guard-go v1.1.0 or later:

cfg := openai.DefaultConfig("ollama")        // local servers ignore the token
cfg.BaseURL = "http://localhost:11434/v1"    // the MODEL endpoint: local

client, err := shrikeopenai.NewClient(shrikeopenai.ClientOptions{
	OpenAIConfig: &cfg,
	ShrikeAPIKey: os.Getenv("SHRIKE_API_KEY"), // governance runs server-side
})

Any option other than the shrike_* ones passes straight through to the provider client, so the Anthropic and Gemini wrappers take a base URL for compatible gateways the same way. Shipped in shrike-guard 4.0.5 (Python and TypeScript) and shrike-guard-go v1.1.0.

Two endpoints, kept apart

The model endpoint is local: the prompt and the completion travel between your process and your runtime. The scan endpoint evaluates the prompt, the tool call, or the response against your policy and returns the verdict, and you choose where it runs. The SaaS scan runs on Shrike's backend, which stores safe interactions as a hash plus metadata, never the content, and keeps violation records encrypted for audit. Shrike inside your boundary runs the same pipeline in your VPC, on-premise, or air-gapped, and its semantic layer can use the local model server you already run, so nothing leaves at all. The second option is an Enterprise engagement, described on the compliance page.

What the local model gets

• The policy check. Prompt injection, data leakage, and unsafe actions are evaluated before the model acts on them, with the free pattern floor on every tier and the semantic and session layers on paid tiers.
• The scope boundary. Declare what the agent is for, which tools it may use, and for how long. A local model that wanders outside its scope is held for a person, the same as a hosted one.
• Human approval. Consequential actions route to a human before they execute. A local model has no vendor to do this for you.
• The evidence record. Every allow, warn, hold, and block lands on the same record that feeds your SIEM and your auditor, in the same shape regardless of the provider.

Switching models later

Because enforcement runs at the boundary you own, a change of provider is a change of one line. The policy that held on a hosted model holds on the local one, and the record continues without a gap. That is the property to check in any governance layer before you adopt it: if the controls live inside the provider, they leave with the provider.

Questions

Related: SDK reference, for coding agents, self-hosted LLM security, the stack you run when you go local, and why a local model needs a governance layer. Machine-readable summary of the platform: /llms.txt.