A red-team report can show how an AI agent handled a known set of attacks in a tested configuration. It can’t, on its own, approve the agent for a future production workflow.
That matters because agent security depends on more than the model’s response to a prompt. An agent may ingest external content and act through tools under environment-specific permissions. Each new source, tool, or permission changes both an attacker’s influence path and the consequence of success.
NIST’s Center for AI Standards and Innovation published results from a large AI agent red-teaming competition on March 23, 2026. The study covered more than 250,000 attack attempts by over 400 participants across 13 frontier models and found at least one successful attack against every target model. It also found that attacks developed against one model or scenario could transfer to others. NIST’s analysis describes the findings and the need for evaluations that adapt to real-world adversaries.
For a production team, the implication is that APPROVAL must account for the workflow’s present authority, its operating conditions, and the recovery path when a safeguard fails.
A Passing Benchmark Answers A Narrow Question
Static tests can expose known prompt-injection patterns, weak tool descriptions, unsafe retrieval behavior, and unhandled instruction conflicts. They also make regressions visible when a model, prompt, or integration changes. A benchmark, however, tests a fixed set of attacks against a fixed configuration. It’s evidence for an approval decision, not the decision itself.
Production conditions change when a new connector reaches another system, an updated model changes behavior, a workflow gains a new action, or an attacker learns how a team phrases its policies.
Record the authority being enforced, the conditions that permit action, and the recovery path when a safeguard fails.
Test The Workflow, Not Only The Model
A useful evaluation starts with the action the organization wants to permit. Identify that action, trace the evidence and tool path that produces it, then test the policy enforcement, escalation, and containment around it.
Consider a support workflow that can retrieve a customer record, create a ticket, and prepare a refund recommendation. Each action carries a different risk. The evaluation should test the instructions the agent receives, the evidence it sees, the tools it can call, and the controls that sit between a proposed action and execution.
For each workflow, assemble a representative evaluation set. Include routine cases, ambiguous requests, missing or conflicting evidence, malicious instructions embedded in retrieved content, and cases where the correct result is no action because authority is absent. This last category often reveals more than a successful task-completion score.
Evaluation also needs to inspect handoffs, whether the recommendation reaches the right approver, whether the approval request carries enough evidence, and whether the system rejects a tool call that lacks the required evidence or authority. Such checks live outside the model, yet they determine the safety of the workflow.
Attack Variation Matters More Than A Long Test List
Update the evaluation suite when a connector, retrieval source, prompt, model, tool schema, or policy changes. NIST notes that adversaries adapt techniques to particular targets and defenses, so re-running a fixed prompt set after every release will miss attacks that exploit a new source or operational shortcut.
Vary the test conditions. Change the location of a malicious instruction, the authority requested, the tool sequence, and the evidence available to the agent. Test whether an attack can cause the agent to reveal data, broaden a permission request, bypass an approval step, or make an irreversible change. A negative result on one path does not prove that the workflow is safe on the next path.
Comparative model tests can still inform selection, especially when they show materially different attack resistance. They don’t replace an evaluation of the actual workflow. The production risk comes from the interaction between a particular model, its instructions, retrieved material, available tools, and decision authority.
Measure The Controls That Limit Damage
Security teams often focus evaluation on whether the model follows a malicious instruction. That’s necessary, but it’s only the first layer. The more consequential test asks what happens after the model gets something wrong.
Tool authorization and policy gates block actions that lack authority. A decision record makes a wrong action reviewable, while containment controls limit its effect. An approval request gives a named person the information needed to decide, while an agent should not gain broad data access simply because it can summarize a document safely.
Together, these controls let the organization state what the agent may do and how the workflow responds when it reaches a limit.
Evaluation Must Continue After Deployment
Production changes the evidence available to the agent and the consequences of its actions. Treat the evaluation suite as a controlled production asset, with an owner and defined change triggers.
Add new attacks when the team discovers them, and review agent traces after incidents, failed approvals, and surprising tool calls. Re-run the suite when a material change to the model, instructions, retrieval source, tool schema, or policy affects the decision boundary. Keep the cases that prompted control changes since they show why that boundary exists.
All of this requires a bounded workflow, a clear owner, and an evaluation plan that matches the consequence of the action. We recommend beginning with decision support or a limited approval flow when there’s uncertainty. Expand automation only after the evidence, policy controls, and recovery path have earned that authority.
Production Approval Is An Operating Decision
Security testing should help an organization decide whether an agent can proceed, proceed with constraints, require approval, or stop.
The strongest approval record connects the tested scenarios to the agent’s allowed authority, the policy that governs each action, the person who owns exceptions, and the response when a control fails. It ends up giving engineering and security teams a basis for changing the workflow without pretending that a benchmark certificate makes the risk disappear.
Building an approval case for an AI agent? Book a Consultation.