The demo went well. The agent read a sample inbox, drafted sensible replies, updated the right records and saved a team hours. The sponsor asked for a production pilot, and the request went to security. What can it access? The CRM, through the integration. Can it delete records? It should not. How would we stop it? We would turn off the server, probably. The reviewer sent a vendor-risk questionnaire built for SaaS products, the team answered half of it, and three months later the sponsor moved the budget elsewhere.
From the inside, security looks like the bottleneck. From the reviewer's side, the team asked for permission to run something nobody could describe. A faster reviewer will not fix that. A project team that arrives with the answers already written down, in the shape the reviewer needs, will.
The reviewer is accepting a risk, not grading an exam
A security review of an AI pilot is a risk decision, and classic security programs formalize it as an authorization. NIST's Risk Management Framework describes control selection, implementation and assessment, system authorizations and continuous monitoring, and it establishes responsibility and accountability for the controls in place [3]. You do not need the full federal process, but the shape is right: someone looks at what the system can do, what controls exist and what risk remains, and someone accepts it.
For an agent, the reviewer weighs four things. Is the blast radius bounded by something other than the model's good behavior? Can we see what it does in enough detail to reconstruct an incident? Can we stop it quickly and undo what it did? And who accepts what is left? Leave any of the four unanswered and the safe choice for the reviewer is to wait. The decisions below answer them in order.
Decision 1: what will the pilot not do yet?
A production pilot is not production at small scale. It is a bounded experiment with real data, chosen so the residual risk is small enough for a business owner to accept with confidence. Most of the scoping work is saying no for now.
- Named users and a fixed volume, such as one team's queue, rather than a percentage of all traffic.
- The narrowest data classes that still prove the value, with payment and special-category data left out unless they are the point. The demo ran on synthetic data; production means real records reaching a model provider.
- Consequential actions ask a person during the pilot, even if some will later run unattended.
- A fixed duration with exit criteria agreed in advance, including what triggers an early stop.
- A named risk owner in the business who signs the residual risk, separate from the security reviewer who advises on it.
Decision 2: what can the agent actually touch?
Start here, because this is where the demo story and the system part ways. The team says the agent only reads records because the system prompt says so, while the credential behind it can write and delete. Reviewers have learned to treat prompt instructions as intentions, not controls. Often the demo ran on the builder's own token or a shared integration account, so nobody can say what the agent's identity is or how to revoke it alone. And the agent reads email, documents or web pages written by outsiders, which makes it a target for prompt injection, and nobody has written that down.
Write the intended scope and the scope the credential really has side by side. Reviewers trust a team that shows them the gap more than one that claims there is none. OWASP's guidance on excessive agency names the root causes reviewers worry about, excessive functionality, excessive permissions and excessive autonomy, and recommends limiting downstream permissions to the minimum necessary [2]. Its Top 10 for Agentic Applications, published in December 2025, is a peer-reviewed list of the most critical risks for autonomous and agentic systems, and some reviewers now ask teams to map their pilot against it [4]. Neither is a checklist you can pass, but both tell you which questions are coming.
Decision 3: what is the worst single action, and does it run, ask or stop?
Rank every action from read to reversible change, irreversible change, outbound, and money or admin, and decide for each tier whether it runs, asks a person or is blocked during the pilot. OWASP's most practical recommendation applies here: require a person to approve high-impact actions before they happen [2]. Name where each approval happens and who answers it. Then ask only for what is consequential. Adding approvals to every action speeds up sign-off and trains pilot users to approve without reading.
At Tamsin Mutual, a fictional regional insurer, a claims-intake assistant had waited eleven weeks in review; the team built the kit in a week. The authority inventory found that the claims-system integration user could edit and close claims and that the email tool could send to any address. Sending a reply ranked as outbound, so the team changed the design: the assistant drafts, an adjuster sends. The reviewer approved a six-week pilot for one regional claims team with three conditions, signed by the head of claims operations.
Decision 4: how is it stopped, traced and undone?
"Turn off the server" is not a stop, and "we would look at the logs" is not an undo. A strong answer names one pause that stops all runs, who can pull it and how long it took in a drill. It shows that every write is logged with the message that triggered it, and says who reverts each kind of change. The NIST AI RMF gives the reviewer a frame for this: MAP 3.5 asks that processes for human oversight be defined, assessed and documented; MANAGE 4.1 asks for post-deployment monitoring plans that include incident response, recovery and change management; and MANAGE 2.4 asks for assigned responsibilities and mechanisms to disengage or deactivate a system whose outcomes are inconsistent with intended use [1]. A reviewer who sees those handled by named people has most of what they need.
Put the answers in one folder of six pages
Each artifact answers one reviewer question, fits on one or two pages, and is produced before the review meeting. If you have threat-modeled the agent, the first two already exist. Do not answer the vendor questionnaire instead: questionnaires cover the vendor, and the kit covers your agent, your credentials and your data. Write it to inform, not to pass. A reviewer who finds an undisclosed scope in the first test stops trusting the rest of the folder.
Each artifact answers one question the reviewer will otherwise ask in a meeting. The kit is complete when every tile has a page and every page names an owner.
| Artifact | Question it answers | What it contains | Produced by | Done when |
|---|---|---|---|---|
| Data flow | What goes in and out, and who writes it? | One diagram: every input with its author, every system the agent calls, what is sent to the model provider, data classes involved | Builder | Every input is labeled trusted or untrusted by who wrote it, not where it is stored |
| Authority inventory | What can it actually do? | Each tool, its credential, the intended scope and the scope the credential really has | Builder with each system owner | No action is limited only by the prompt, or each such case is listed as unenforced |
| Consequence tiers | What is the worst single action? | Every action ranked from read to reversible change, irreversible change, outbound, and money or admin | Builder and security | Each tier states whether it runs, asks a person or is blocked during the pilot |
| Controls | What stops the bad outcomes? | Each top risk mapped to a control outside the model, with where it is enforced | Operator | Every control has been exercised at least once in a test |
| Evidence | How will we know what it did? | Log fields per action, a sample of real transcripts, test results, and the monitoring plan | Operator | Any action from the test run can be traced to the input that caused it |
| Rollback | How is it stopped and undone? | The pause mechanism, who can use it, and how each consequential action is reversed | Operator and system owners | The pause has been drilled and timed, with the result recorded |
Ten working days, with the review booked on day one
Two weeks is realistic when the pilot is scoped tightly and the reviewer is involved from the first day rather than the last. Book the review meeting on day one for day ten; a fixed date focuses everyone.
Scope and the first two artifacts come first, fixes land in the middle, and the last days produce evidence and a timed pause drill. The reviewer sees drafts on day three, so the meeting on day ten confirms rather than discovers.
- Day 1: kickoff with the builder, operator, a security reviewer and the risk owner. Agree scope, users, data classes, duration and exit criteria in a one-page charter.
- Days 2-3: draft the data flow and authority inventory and share both with the reviewer. Expect at least one credential that can do more than intended.
- Days 4-5: rank actions into tiers, decide what runs, asks or is blocked, and write the controls page against the top risks.
- Days 6-8: make the fixes, usually a dedicated identity for the agent, narrower scopes, one tool removed, and approval on outbound actions.
- Day 9: test with an injected instruction in an input, a direct tool call that exceeds a limit, and a trace from a logged action back to its input. Attach the results to the evidence page.
- Day 10: drill the pause and record how long it took, then hold the review. The usual outcome is approval with conditions, each with an owner and a date.
Several kit pages are easier to produce when the agent's traffic passes through a runtime gateway. Intertrace sits inline between the agent and its models, tools and MCP servers, reads what each call does from the tool name and its arguments, and assigns a consequence tier, so outbound messages, payments and admin changes ask a person. New agents start in a learning period that records what they use, which gives the authority inventory real data, and every decision is recorded with the rule and evidence behind it. Scoping credentials and writing the charter remain the team's work.
Write down what sends it back to review
Approval is a snapshot. The RMF treats authorization as ongoing, backed by continuous monitoring [3], and a pilot that gains two tools a week no longer matches what was approved. Put the re-review triggers in the charter: a new tool, a new credential, a new input channel, a new data class or a model change. Any of them sends the changed page, not the whole kit, back to the reviewer. Then hold a short exit review that compares the exit criteria with what happened and sets the scope of production. Start with the pilot that has waited longest, and give it a dedicated identity before anything else.
