NewIntroducing Integrations
All posts
9 min read
by

Why human approval prompts become a rubber stamp (and what to do instead)

Approval fatigue, approving a summary instead of the request, and requests that change between review and execution. Why AI agent approval prompts stop protecting anything, and four controls that keep a human yes meaningful.

approvalshuman-in-the-loopagentssecurity

A human approval prompt becomes a rubber stamp in three ways: the reviewer sees so many that they stop reading, they approve a summary instead of the request that will run, or the request changes between their yes and the execution. The fix is not more prompts. It is fewer and sharper ones: a risk-based policy and a shared limit so low-impact actions need no prompt at all, an approval bound to an exact snapshot of the request, and an approver who does not hold the agent's credential.

TL;DR

  • People approve almost every prompt they are shown. That is rational when most prompts are harmless, and it is exactly what makes the rare bad one slip through.
  • A model-written summary is not the request. Approve the normalized request, field by field.
  • An approval checked at review time and not at execution time is a time-of-check to time-of-use bug.
  • Route by policy, cap the total with a shared limit, bind the approval to the exact request, and keep requester and approver credentials apart.

How often do people say yes to approval prompts?

Nearly always, and the evidence is old and consistent.

  • Coding agents. Anthropic reported in March 2026 that "Claude Code users approve 93% of permission prompts", and named the result: "approval fatigue, where people stop paying close attention to what they're approving" (Anthropic Engineering).
  • Browser warnings. A field study of more than 25 million warning impressions found users clicked through Chrome's SSL warning 70.2% of the time (Akhawe and Felt, USENIX Security 2013).
  • Clinical alerts. A review of 17 studies found clinicians override drug safety alerts in 49% to 96% of cases, and pointed at low specificity and workflow disruption as likely causes (van der Sijs et al., JAMIA 2006).
  • Automation bias. It appears in novices and experts, and training or instructions do not prevent it (Parasuraman and Manzey, Human Factors 2010).
  • MFA push bombing. Attackers flood a user with login prompts until one is approved, which is why CISA recommends number matching as an interim fix (CISA fact sheet, October 2022).

The OWASP Top 10 for Agentic Applications names the agent version of this twice. ASI08 lists a "governance drift cascade" in which "human oversight weakens after repeated success", and ASI09 describes humans "approving actions without independent validation" (OWASP GenAI Security Project). We map all ten risks in OWASP Top 10 for Agentic Applications, mapped to code.

A 93% approval rate is not proof of carelessness. Most of those prompts were harmless, and saying yes was correct. The problem is what the rate does to the other 7%. A prompt that is right to approve 93 times in 100 trains the reviewer to approve, and the one that mattered arrives in the same box, with the same button.

Failure 1: too many prompts

If every write asks, the reviewer is doing the job a rule should do. The fix is to stop asking about the cases a rule can decide. A refund of $12.50 on an order the customer is entitled to return does not need a person. A refund of $1,200 might. A refund of $50,000 from a support agent should never happen.

That is three outcomes, not one, and each can be decided before a person is involved. A prompt should reach a human only when the answer genuinely depends on judgment.

Failure 2: approving a summary, not the request

Here is a prompt a reviewer would approve in two seconds:

Refund order #4411 for the damaged blender ($12.50).

Here is the request it stands for:

{ "paymentIntent": "pi_3Pxyz", "amount": 125000, "currency": "usd" }

The summary was written by the model. The amount is in minor units and is off by a factor of a hundred, and the payment intent belongs to a different order. A reviewer who approves the summary has approved nothing in particular. OWASP lists the deliberate version under ASI09 as "fake explainability". A September 2026 paper calls the general failure, where the approved operation is not the executed one, "loopjacking", and its first variant is exactly this: a different operation is encoded but omitted or misrepresented at approval time (arXiv 2609.21081).

The control is to approve the normalized request: target account, resource, amount with its unit, recipient, version. A model-written sentence can sit beside it. It must never replace it.

Failure 3: the request changes after the yes

Even an exact review fails if what runs is read from mutable state after the decision. The same paper's second variant is post-approval state substitution, reproduced in seven Agno AgentOS releases through 3.0.9 and in a conditional, in-memory LangGraph Agent Server composition through 0.14.0. OWASP lists the same shape under ASI03 as time-of-check to time-of-use: "Permissions may be validated at the start of a workflow but change or expire before execution."

It does not take an attacker. The action definition gets edited while the approval waits. Another agent uses up the day's budget. The credential is revoked. An approval that is checked only when it is given carries all of that through.

What should you do instead?

Four controls, in the order they remove prompts or make the remaining ones mean something. The code is Anlyon's, and every mechanism is described so you can build it elsewhere.

1. Let a policy decide the routine cases

Write the three outcomes down as rules, evaluated outside the agent on every call. In Anlyon, an operator key sets them from a deploy step, and the agent's runtime key cannot read or change them.

import { Client } from '@anlyonhq/sdk';

const ops = new Client({ apiKey: process.env.ANLYON_OPERATOR_KEY! });

// Policies run in ascending priority. The first match decides.
await ops.approvalPolicies.create({
  name: 'refunds-over-1000-never',
  matchKind: 'action', matchActionName: 'refund',
  condition: 'input.amount > 100000', effect: 'auto_deny', priority: 1,
});
await ops.approvalPolicies.create({
  name: 'refunds-up-to-25-run',
  matchKind: 'action', matchActionName: 'refund',
  condition: 'input.amount <= 2500', effect: 'auto_approve', priority: 10,
});
await ops.approvalPolicies.create({
  name: 'other-refunds-need-two',
  matchKind: 'action', matchActionName: 'refund',
  effect: 'require_approval', requiredApprovals: 2, priority: 20,
});

The refund action is created with requiresApproval: true, so with no matching policy it waits for a person. Only a policy that names the action can lower that floor, so a catch-all auto_approve cannot disarm it. A condition that cannot be evaluated requires approval rather than skipping to a more lenient rule. Each decision records the policy and the version of it that decided. See Policy as code.

2. Cap the total so small actions need no prompt

Auto-approving $25 refunds is safe for one refund. It is not safe for a thousand of them from an agent in a loop. The number that matters is the total, and the control for a total is a limit, not a prompt.

await ops.impactLimits.create({
  name: 'Refunds per day', dimension: 'money', currency: 'usd', period: 'day', amount: 500_000, // $5,000.00
});

Every agent, session and key in the environment draws from the same limit, reserved before dispatch. A new session or key does not get a new allowance. When there is no room, the refund is refused whole and nothing reaches Stripe. With a limit in place, the policy above can auto-approve small refunds without a person, because the worst case across every agent is $5,000 a day, gated refunds included. Limits apply to governed actions, meaning an action with an adapter or a declaration, and to calls routed through Anlyon. See spending limits for agents.

3. Bind the approval to the exact request

The reviewer approves a preview: the exact normalized request, the target, the action and adapter versions, the preconditions observed at review, and an expiry. Anlyon stores a SHA-256 digest of it on the approval, recomputes it from the stored effect immediately before dispatch, and refuses on any difference.

const agent = new Client({ apiKey: process.env.ANLYON_AGENT_KEY! });

const input = { paymentIntent: 'pi_3Pabc', amount: 12000, currency: 'usd' };
const { data: preview } = await agent.actions.preview('refund', input);

// Invoking with the preview id runs that exact request after approval.
// A different input with the same previewId is refused with 409 PREVIEW_MISMATCH.
await agent.actions.invoke('refund', input, { previewId: preview!.id });

Dispatch is also refused, with nothing sent, if the action was edited after review, the preview or approval expired, the credential's host binding no longer allows the request, the requesting key was revoked, the environment is halted, or the limit no longer has room. A preview lasts one hour by default. Refreshing it creates a new binding, and an approval of the old one does not carry over. See Bound approvals and binding approval to the executed action.

4. Keep the approver's credential away from the agent

A rubber stamp can also be literal: the agent approves itself. In Anlyon, deciding needs the separate approvals:decide scope, which the agent's key does not need and should not have. The credential that requested an approval can never be the one that decides it. No scope grants that and no policy relaxes it.

Be precise about what "two approvers" means. N counts distinct credentials for API-key deciders and distinct users for dashboard and OAuth deciders. Two keys held by one person are two deciders. If you mean two people, require dashboard decisions. See what counts as a distinct approver.

Which control fixes which failure?

FailureWhat it looks likeControlIn Anlyon
Too many promptsReviewer approves without readingPolicy decides routine and forbidden casesauto_approve, require_approval, auto_deny, evaluated per call
Many small actions add upEach is fine, the total is notShared limit across agents and keysImpact limits, reserved before dispatch
Approving a summaryProse says $12.50, request says $1,250Review the normalized requestPreview with typed fields and allowances
Change after reviewWhat runs differs from what was approvedRecompute and compare at dispatchDigest recomputed before dispatch, PREVIEW_MISMATCH at admission
Self-approvalAgent's key approves its own requestSeparate decide credentialapprovals:decide scope, requester can never decide

What do these controls not fix?

  • Attention. No routing rule guarantees an attentive reviewer. It only makes sure the prompts that reach one are the ones worth reading.
  • Provider state. Binding covers the exact request, not atomic provider state. For a Stripe refund the refundable amount shown is as of review. For GitHub, a file changed after review is refused by GitHub on the reviewed blob SHA. A declared action's provider state is not checked between review and dispatch.
  • Who can decide. Any workspace member can decide an approval in the console. There are no approver groups.
  • The local wrapper. approvals.gate() waits for a decision, then your process runs your function with your credential. It records the decision. It cannot bind what your function sends.

An approval is permission to attempt the reviewed operation under the checks that run at dispatch. It is not a receipt. Whether the refund happened comes from the provider, recorded with a grade.

For the basics of adding a gate, start with human-in-the-loop approval for agent tool calls. For what the review screen should show, see the approval review screen checklist. Start building: free tier, no credit card.

Free tier, no credit card. One command if you use Claude or Cursor.

$ claude mcp add anlyon -- npx -y @anlyonhq/mcp-server