Governing enterprise AI agents

Most enterprise agent projects do not fail on the technology. They stall somewhere between a convincing demo and a production rollout, at the point where security, legal, and operations start asking questions that the pilot was never designed to answer.

Those questions are predictable. What can this thing read? What can it do? How do we know it is right? Who fixes it when it is wrong? An agent that has good answers to all four is straightforward to approve. An agent that has good answers to none of them is a demo, regardless of how well it performed in the meeting.

Decision one: define the grounding boundary before you build

The instinct with a knowledge agent is to point it at everything. More content should mean better answers.

In practice, the opposite happens. Broad grounding pulls in superseded policies, abandoned drafts, and documents nobody has owned for years. The agent answers confidently from a file that a human would have recognised as obsolete at a glance, and one visibly wrong answer costs more trust than ten correct ones earn.

A workable grounding boundary has three properties:

• Every source has a named owner. If nobody is responsible for a location, its content should not be authoritative.
• Every source has a review cadence. Content that has not been reviewed in years is a liability whether or not an agent is reading it.
• Exclusions are deliberate and recorded. "We excluded the shared drive because it has no ownership model" is a defensible position. Silently including it is not.

This is usually the least technical part of an agent project and the most valuable. It also tends to surface content-governance problems the organisation already had — the agent just made them visible.

Decision two: inherit permissions, do not replace them

An agent that runs under a broad service account will eventually surface something to someone who should not see it. Not through a flaw in the model, but because the account it uses can read more than the person asking.

The safer pattern is permission inheritance: the agent operates under the signed-in user's identity, so it can only retrieve what that person could already open. This has a useful side effect — your existing access model becomes the agent's access model. If it is well maintained, the agent inherits that. If it is not, you find out early and in a controlled way, which is a far better outcome than discovering it after a leak.

Where inheritance is genuinely impractical, the fallback is a deliberately narrow scope with an explicit record of what the agent can reach and who approved it. What should not happen is a broad service identity chosen for convenience during a pilot and then quietly carried into production.

Decision three: build the evaluation set before launch, not after

Ask most teams how they know their agent gives good answers and you will hear that they tried it and it seemed fine. That is a reasonable starting point and a poor basis for a production decision.

The alternative is inexpensive. Before launch, collect thirty to fifty real questions — from the support queue, from the team who currently answers them, from the people who will use the agent. For each, agree the expected answer and the source it should cite. Run the set. Record what happened.

That gives you three things a subjective impression cannot:

1. A baseline. You know current quality rather than estimating it.
2. Regression detection. When someone changes grounding scope or rewrites a topic, you re-run the set and see what broke.
3. A conversation with risk stakeholders that is about evidence. "We tested against forty-two representative questions and here is the result" is a materially different discussion from "it works well."

Keep the set in source control next to the agent configuration. It is part of the solution, not a testing artefact.

Decision four: name the owners before go-live

Agents drift. Source content changes, a process is updated, a system is renamed, and answers that were correct at launch quietly stop being correct. Nothing alerts you to this, because the agent continues answering with the same confidence it always had.

Three roles prevent most of that drift:

• A business owner for scope and content accuracy — the person who decides what the agent should cover and confirms the answers are right.
• A technical owner for the environment, integrations, and deployment.
• A reviewer who looks at unanswered, low-confidence, and escalated questions on a set cadence.

The reviewer role is the one most often skipped and the one that produces the most value. A monthly look at what the agent could not answer is the single best source of improvement backlog you will get, because it comes directly from what people actually asked.

Where actions change the risk profile

Everything above applies to an agent that only answers questions. The moment an agent can take an action — create a request, update a record, start an approval — the risk profile changes, and the controls need to change with it.

Three things matter here. First, scope each action narrowly. An action that creates a specific request type with validated inputs is far easier to reason about than a general-purpose "update the record" capability. Second, decide what requires confirmation. Reading is reversible; writing to a system of record often is not, and the user should see what is about to happen before it happens. Third, log actions separately from conversations. When you need to answer "what did this agent actually do last Tuesday," a conversation transcript is not the artefact you want.

What good looks like at approval time

By the time an agent reaches a governance review, you should be able to produce, on one page:

• The grounding sources, their owners, and their review cadence.
• The identity and permission model, stated plainly.
• The action inventory, with which actions require user confirmation.
• The evaluation set, when it was last run, and the result.
• The three named owners and the review cadence.
• What happens when the agent cannot answer — the escalation path.

None of this requires unusual tooling. It requires deciding these things deliberately rather than discovering them under review pressure.

The organisations that move fastest with agents are rarely the ones with the most sophisticated models. They are the ones that made these decisions once, wrote them down, and reused them — so the second agent inherits a governance position that the first one had to establish.