An AI agent that can pay, send, delete or accept terms can turn a wrong draft into money that has left the account, a message already delivered, a record that is gone, or a commitment the business has to honour. Keep a human approval step before those four actions, and allow search, summarising and drafting only where a system outside the model enforces the permission. This engineering explainer is for an owner or manager placing that step, and other explainers from this section are on the Technology page.
Human approval means a person sees the exact AI agent action and can refuse it before another system carries it out. OWASP’s LLM06:2025 entry says to require that approval for high-impact actions and to enforce the decision in the downstream system. The UK Information Commissioner’s Office, on a page dated 31 March 2026, describes an agent allowed to make purchases up to a set amount and required to ask permission above it. The page does not print the amount. This publication’s recommended default uses that pattern for four actions: a payment or refund, a consequential customer message, a deletion or bulk change, and an acceptance of terms. Search, summarising and drafting can run on read-only tools. Approval limits the distance a wrong output can travel. It does not bring back a payment or an email that has already gone.
| Action | Dominant failure mode | Prevention | Figure |
|---|---|---|---|
| Search or summarise | The read tool can also modify or delete records | Give the extension read access only, enforced in the downstream system | |
| Draft a customer reply | An incoming email instructs the agent to forward mail, and the plugin can send | Read-only mail scope, and a person hits send | |
| Payment or refund | The amount is above the business limit, or the payee or account is new, and the model releases it | The payment system holds the transaction until a person records approval | |
| Delete or bulk change | Records are deleted with no confirmation of which ones, or how many | Show the records and the count, and confirm restore has been tried | |
| Accept terms | Final terms go out with no person who can commit the business | A person with that authority approves the terms and the counterparty | |
| Wider tools or data | Access grows because it might be useful later | Treat the wider access as a new permission decision |
Empty cells are figures the sources below do not print. No page read for this article gives a universal spending limit, an error rate, a share of businesses using agents, or a price.
An agent can call other systems
A chatbot returns text. An agent can take the next step in other software. OWASP’s LLM06:2025 entry on excessive agency describes that step as the ability to call functions or other systems through extensions, also called tools, skills or plugins. The model may choose the extension. Agent setups often call the model again, using earlier output to steer the next call.
OWASP defines excessive agency as damaging actions performed in response to unexpected, ambiguous or manipulated model output. The triggers it names are hallucination or confabulation, and direct or indirect prompt injection, including from a malicious user, a compromised extension, or another agent. NIST’s Generative AI Profile, AI 600-1 (July 2024), uses confabulation for output that confidently presents erroneous or false content. The same section says generated logic or citations can purport to justify an answer and can mislead a person into trusting it. The profile does not print how often that happens.
Indirect prompt injection, in OWASP’s LLM01:2025 entry, is input from an external source such as a website or a file that changes the model’s behaviour. OWASP says it is unclear whether fool-proof prevention of prompt injection exists. The UK National Cyber Security Centre, in a blog published on 8 December 2025, treats prompt injection as a residual risk that a product does not fully remove. The NCSC says design protections should focus on deterministic safeguards that are not themselves a language model, and that constrain what the system can do. A recorded human approval is that kind of check, because a person records the decision outside the model. It sits between a proposal and an action that leaves the business.
Takeaway: Judge the agent by the actions its tools can take, and by where the allow-or-deny check sits.
Four actions that should wait for a person
OWASP says to require a human to approve high-impact actions before they are taken. It does not define high-impact for a small business. NIST’s AI Risk Management Framework, AI 100-1 (January 2023), is voluntary and does not prescribe risk tolerance. Its MANAGE 1.2 subcategory says treatment of documented risks is prioritised by impact, likelihood, and available resources or methods. On that basis, the four actions below are this publication’s recommended default. They are hard to undo, or they bind the business to someone outside it.
Payments and refunds
Releasing a payment, issuing a refund, or changing a supplier’s account number moves money that may not come back. The ICO’s agentic AI report, dated 31 March 2026, gives the example of a person authorising an agent to make purchases up to a set amount, with permission required above that amount. The report does not print the amount. A small amount is not a free pass in this publication’s recommendation: a new payee, a run of small payments, or a changed account number still waits for a person.
Rule of thumb. If the proposal amount A is above the spending limit L the business has written down, hold the payment. If the payee is new, or the destination account has changed, hold the payment even when A is not above L. The ICO describes the amount pattern and does not print L.
Customer messages
OWASP’s mailbox scenario is a hypothetical attack: a crafted incoming email tricks an assistant into scanning the inbox and forwarding sensitive mail, because the chosen plugin can send as well as read. One avoidance OWASP gives is that the user manually reviews and hits send on every message the extension drafts. A fixed template the agent cannot edit, such as an order confirmation with locked wording, can follow a lighter rule. A message that discloses personal or commercial information, answers a complaint, or promises a price, a date or a remedy should be read first. No source here prints a word count that separates those messages from a routine notice.
Deleting records and bulk changes
OWASP’s example of excessive autonomy is an extension that deletes a user’s documents with no confirmation. Before a deletion or a bulk overwrite runs, the person should see which records, how many, and whether a restore from backup has been tried. The sources do not print a record count that makes a deletion safe.
Contracts and commitments
Accepting terms, or confirming an order, creates obligations. An agent can summarise clauses and flag odd ones. A person who has authority to commit the business should approve the final terms and the counterparty. Whether an acceptance sent by an agent is legally binding depends on the jurisdiction and the facts. This page does not make that legal call.
A fifth case sits beside these four. A request for a new tool, a new data source, or a wider permission goes to someone who can grant it. The ICO says organisations should not give an agent access to information just because it might be useful in the future.
A mailbox summary does not move money. The plant-floor version of that split is how industrial IoT monitoring differs from a control loop that moves actuators.
Takeaway: Let the agent prepare the payment, the draft, the deletion list or the clause summary. Hold the step that leaves the business.
Put the hold in the system that moves the money
Approval is the last check. OWASP names three root causes: excessive functionality, excessive permissions, and excessive autonomy. An approval step addresses the third. The first two are fixed by fewer tools and narrower access. In OWASP’s mailbox example, an assistant that summarises email needs to read mail. It does not need a plugin that can also delete or send. In its database example, a product-recommendation agent needs read access to a products table, enforced by database permissions, and does not need insert, update or delete.
Where the check sits matters as much as the button. OWASP says to implement authorisation in downstream systems, so the model is not the thing deciding whether an action is allowed. It also says the approval can live in the downstream system or inside the extension. Its social-posting example puts a user-approval routine in the extension that performs the post. The NCSC’s practical line on email is the same idea: do not let a model that processes mail from unknown external people have access to privileged tools.
A sentence in the prompt that says “always ask before paying” does not hold the money. A payment system that refuses the transaction until a person has recorded an approval does.
Worked example. Write the business’s own limit as L and the proposed amount as A.
Difference = A - L
When Difference is greater than zero, the amount test holds the payment. The ICO page does not print A or L, so this page does not substitute a currency figure. A new payee, or a changed account number, is a second test. That test still holds the payment when Difference is zero or negative.
The same arithmetic does not produce a savings percentage. No source here prints the share of proposals that fail either test.
Takeaway: Remove unneeded tools first. Then make the system that executes the action refuse it until a person has approved that version.
What the person has to see
A click with no content is not a review. The ICO’s guidance on individual rights in AI systems, dated 31 March 2026, says the guidance is under review because of the Data (Use and Access) Act and may change. Within that stated limit, the page says a decision does not fall outside Article 22 of the UK GDPR just because a person rubber-stamped it. Reviewers need meaningful influence, including the authority and competence to go against the recommendation, and they must weigh and interpret it. The same page says reviewers need authority to override the output and confidence that they will not be penalised for doing so, and that policy and training alone cannot create that confidence. A supportive culture has to sit behind the policy.
As this publication’s recommendation, an approval request should show:
- The action: pay, send, delete or accept.
- The target: the payee, the recipient, or the records and how many.
- The details: the exact amount, the final message, or the final terms, identical to what will run.
- The evidence: the original invoice, ticket or document, not only the agent’s summary.
- The consequence: what happens if it proceeds, and whether it can be undone.
- The decision: approve, edit or reject, stored with the reviewer’s name.
If the amount, the message or the set of records changes after approval, that approval ends and a new one is required. If nobody responds, the action stays on hold or falls back to a draft. It does not run on a timer.
NIST AI 600-1 describes automation bias as excessive deference to automated output, which can worsen confabulation risk. The ICO page uses the same term for reviewers who stop questioning whether the output might be wrong. A request that shows only the agent’s summary makes that deference easier. Put the original invoice, ticket or document next to the summary.
Article 14 of Regulation (EU) 2024/1689 is a design duty for high-risk AI systems, not a finding that a typical small-business agent is high-risk. In the consolidated text of 27 July 2026, Article 14 sits in Chapter III, Section 2. Article 113 says the regulation applies from 2 August 2026. Chapter III, Sections 1, 2 and 3, except Article 6(5), apply later: from 2 December 2027 for high-risk systems under Article 6(2) and Annex III, and from 2 August 2028 for high-risk systems under Article 6(1) and Annex I. Article 14 follows those later dates. Article 14(4) says overseers should be able to understand limits, stay aware of automation bias, interpret output, disregard or override it, and interrupt the system so it halts in a safe state. Article 14(5) adds a separate check by at least two natural persons who have the competence, training and authority, and only for high-risk systems in Annex III, point 1(a): remote biometric identification. The annex excludes biometric verification whose sole purpose is to confirm that a specific person is who they claim to be. The two-person check has exceptions for law enforcement, migration, border control and asylum where the law treats it as disproportionate. EUR-Lex states that the consolidated text is a documentation tool and has no legal effect. This page does not decide whether any given business is in those high-risk classes.
Takeaway: Tie the approval to the exact action and the original evidence. A later edit voids it.
What the ledger cannot fill in
Review time, unrecovered payments and wrong-action rates are real costs. The sources do not print the numbers, so the cells stay empty.
Review_hours = N * M / 60
N is the count of gated actions in the period. M is the minutes a person spends on one request. Dividing by 60 converts minutes to hours. Neither N nor M appears in the pages read here.
| Hidden cost | Figure |
|---|---|
| Reviewer time for gated actions | |
| A payment that has already left | |
| A message that has already been sent | |
| Share of agent actions that are wrong | |
| Extra personal data a reviewer must see |
The ICO says a human reviewer may need to see more personal data to check an output, and that this can erode privacy further. It also says reduced oversight can let unintended harms persist longer before anyone notices. It does not print a duration or a volume of records. NIST AI 100-1, MANAGE 2.4, calls for mechanisms to supersede, disengage or deactivate a system whose outcomes are inconsistent with intended use. MANAGE 4.1 includes appeal, override, incident response and recovery. Halting the agent does not recall a payment or an email that the downstream system has already completed. That limit follows from the action having left. It is not a measured recall rate.
OWASP says logging, monitoring and rate limits do not prevent excessive agency. They can limit damage: logging shows where an unwanted action happened, and a rate limit reduces how many such actions can occur in a period. The entry does not print the limit. The NCSC says that if the remaining risk is more than the system can tolerate, the task may not be a good use of a language model.
Low-impact work that a fixed rule can check, such as tagging a support ticket inside an allow-list, can run without a person on every item. Approving every step is how a reviewer ends up rubber-stamping, which is the failure the ICO describes. The ICO also says using an agent does not move responsibility for personal data off the organisation. Agents are not legal entities in that report.
Takeaway: Spend the review on actions that leave the business. Leave the numeric cells blank until a source prints them.
How this page was checked
The claims above were checked against OWASP’s LLM06:2025 and LLM01:2025 pages, the NCSC blog of 8 December 2025, two ICO pages dated 31 March 2026, NIST AI 100-1 (January 2023) and NIST AI 600-1 (July 2024), and the EUR-Lex consolidated text of Regulation (EU) 2024/1689 as of 27 July 2026. The ICO individual-rights page says it is under review. The EUR-Lex consolidation says it has no legal effect. How this publication selects and attributes sources is set out on its own page. This page does not report a trial of a named agent product.
