1. What an AI agent governance framework is
An AI agent governance framework is the structured set of policies, controls, rollout sequence, and measurement that determines how autonomous AI agents behave inside an organisation. It is not a single product or a single policy document — it is the architectural pattern that links four moving parts:
- What an agent can see (data access).
- What an agent can do (tool and action permissions).
- What an agent says (output controls, redaction, LLM checks).
- What an agent leaves behind (audit logs, retention, evidence).
The framework is not the same as AI agent governance itself — the pillar concept — or the best-practices playbook. Concept tells you what it is. Best practices tells you what to do. The framework tells you what to build and in what order.
2. Stakeholders the framework has to satisfy
A working framework needs four buy-ins, each with a different lens — and the framework has to produce evidence that each lens cares about.
| Role | What they care about | Evidence the framework should produce |
|---|---|---|
| CIO / AI lead | Time-to-deploy, board-level risk, vendor strategy | Fleet-wide posture dashboard; agent inventory by status |
| Risk & Compliance | Auditability, regulatory mapping, policy enforcement | Query-able audit log mapped to GDPR / HIPAA / SOX / ISO 42001 control IDs |
| Security | Identity, tool permissions, data exfiltration paths | Per-agent identity; default-deny allowlists; redaction proof at the boundary |
| Platform / CTO | Architecture, MCP, model independence, observability | Provider-agnostic gateway; structured logs; latency budgets |
There are role-specific pages for CIOs and compliance leaders and for CTOs and AI platform teams.
3. The four categories of risk
Most agent incidents fall into one of four buckets. Tag your incident log against these from day one — the framework is built to close all four.
- Data exposure. The agent saw something it should not have, or leaked it downstream into a vendor model, a tool payload, or a log.
- Unauthorised action. The agent used a tool, triggered a workflow, or wrote to a system it was not approved for.
- Hallucination on ungrounded data. The agent guessed because it lacked safe access to the truth, then acted on the guess.
- Audit failure. You cannot reconstruct what the agent did, why, or for whom — which is itself a regulatory breach in many jurisdictions.
4. The five controls that close them
Five controls, in roll-out order. Each closes one or more of the four risks.
- Identity per agent. Each agent has its own credential, separate from the human user. Closes audit failure and unauthorised action; makes revocation possible.
- Default-deny tool allowlists. Agents only get the tools they explicitly need. Most production agents need a handful, not dozens. Closes the bulk of unauthorised-action risk.
- Redaction at the boundary. PII never leaves the perimeter un-masked. Use entity-aware redactors, not regex. Closes data exposure into vendor models.
- LLM checks for fuzzy policy. Use a second model to validate intent, consent, data purpose, and minimisation rules at the boundary — catches the cases policy-as-code can't enumerate.
- Structured audit logs. Logs that are queryable, not free text. Map fields to control IDs. Closes audit failure permanently.
5. A 90-day rollout plan
A realistic sequence for a regulated enterprise:
Inventory + baseline
List every agent in production today. Stand up a governance gateway in shadow mode that logs but does not block. Catalogue the actual tools, data sources, and providers in use.
Enforce + redact
Flip from shadow to enforce on the top-3 highest-risk agents. Apply redaction rules for the entity types you actually saw in the baseline. Start producing the audit log your risk team will live in.
Scale + audit
Roll the gateway across every agent. Wire continuous agent-to-agent audits. Map the audit log to your regulatory framework and validate with a friendly internal-audit pass.
6. Metrics worth measuring
- ✓ Number of agents in production, per governance status (pass / fail).
- ✓ Redactions applied per day, by entity type.
- ✓ Policy blocks per day, by violation type.
- ✓ Median + p95 latency added by the governance layer.
- ✓ Audit log retention vs the strictest applicable regulation.
- ✓ Time-to-remediate when policy drift is detected.
Posture dashboards live on top of these metrics — see best practices for what to alert on.
7. Anti-patterns to avoid
- Governance as a one-time review. An agent that passed review on day one is not the agent running on day ninety. The framework has to live in the runtime.
- Free-text audit logs. If the log is a stream of natural-language sentences, it cannot be queried, mapped to control IDs, or retained against regulatory schedules.
- Per-user identity for agents. If an agent runs under the calling user's identity, you lose the ability to revoke just the agent — and you cannot answer "what did the agent do" separately from "what did the user do".
- Redaction after the model call. Once the prompt has left the perimeter, the PII is out, regardless of what you redact downstream.
- One governance gateway per LLM vendor. If switching models means rewriting policy, you do not have a framework — you have a vendor lock-in.