Your AI Agent Doesn’t Need More Intelligence. It Needs Better Boundaries.

Teams rewrite the prompt, add more context, switch to a larger model or introduce another round of reasoning. These changes may improve the agent’s output, but they often leave the more important problem untouched.

The agent still has too much freedom.

A customer-support agent can issue refunds when it should only recommend them. A sales agent can send an inaccurate proposal without approval. A coding agent can modify production data while trying to fix an unrelated bug. In each case, the failure is not simply that the model produced a poor answer. The system allowed that answer to become an action.

That distinction matters.

A chatbot generates text. An AI agent can call APIs, update records, send emails, run code, move money or publish content. Once a model can affect an external system, intelligence is no longer the only question. We also need to ask:

What is this agent permitted to do when its reasoning is wrong?

A better model does not remove operational risk

Models are becoming more capable, but capability and reliability are not the same thing.

An intelligent agent may handle a greater variety of requests, plan more complex tasks and recover from some errors. It can also find more ways to pursue an objective that its designers did not anticipate.

Consider a collections agent instructed to reduce overdue invoices. It might draft reminders, identify high-risk accounts and recommend payment plans. Those are useful capabilities.

But if the same agent can automatically suspend accounts, change payment terms and send legal notices, a misunderstood instruction can become a serious commercial problem.

The agent may have followed its objective. The surrounding system failed to define an acceptable way to achieve it.

This is what OWASP describes as excessive agency. Its guidance identifies three common causes: excessive functionality, excessive permissions and excessive autonomy. In other words, the problem is often not what the model knows. It is what the application allows the model to access and execute. OWASP’s guidance on excessive agency specifically recommends requiring human approval before high-impact actions.

This principle is familiar in conventional security. We do not give every employee administrator access because we trust them to use it responsibly. We limit access according to the job they need to perform.

AI agents should be treated in much the same way.

Instructions are not boundaries

Many agent implementations rely heavily on instructions such as:

• Never issue a refund above $100.

• Do not contact a customer without approval.

• Never delete production data.

• Escalate uncertain cases to a human.

These are useful instructions, but they are not reliable controls.

The same model that decides what action to take is also being asked to remember when it should not take that action. A confusing conversation, indirect prompt injection, incomplete context or unexpected tool response may change how it interprets the rule.

A genuine boundary exists outside the model.

If refunds above $100 require approval, the payment API should reject an unapproved request. If an agent must not publish directly, it should receive a create_draft tool rather than a publish_post tool. If production data must not be deleted, the agent’s credentials should not include deletion permission.

The rule should survive even when the model does not follow the prompt.

This is why “human in the loop” cannot remain a line in a governance document. It must become a gate in the code path.

For example, a marketing agent may be allowed to research a topic, prepare a post and save it as a draft. Publishing the post should require a separate approval event from an authenticated person.

The worst realistic outcome is then a poor draft, not an inaccurate statement published under the company’s name.

That is what good boundary design does. It does not assume the agent will always behave correctly. It limits the consequences when it does not.

Start with the potential damage, not the desired automation

Teams often design AI agents by listing everything they want the system to do. A safer approach is to begin with what could go wrong.

For every action, examine four factors:

1. Reversibility: Can the action be easily undone?

2. Impact: Who or what could be affected?

3. Detectability: Would the mistake be noticed quickly?

4. Authority: Is the agent acting internally or representing the company to someone else?

An agent summarizing an internal document is relatively low risk. A poor summary can be corrected, and it does not change the original file.

Sending that summary to a client is different. The content now represents the business externally.

Deleting the original document is different again. The agent has moved from producing information to changing the state of a system.

This creates a practical way to assign autonomy:

Type of actionExampleAppropriate controlRead-onlySearch approved documentsAllow and logReversible internal actionCreate a CRM note or draftAllow with validationExternal communicationSend an email or publish a replyRequire approvalFinancial or contractual actionIssue a refund or accept termsApply limits and approvalDestructive actionDelete records or overwrite dataDeny by default or require elevated approval

The exact classifications will vary by business. A £20 refund may be routine for one retailer and material for another. What matters is making the decision explicitly instead of giving the agent broad access and hoping its prompt covers every situation.

Give the agent the smallest tool that completes the job

One of the most effective controls is also one of the simplest: reduce the scope of the tools available to the agent.

Suppose an agent needs to help customers reschedule appointments. Giving it unrestricted access to the calendar platform may be convenient for development, but the agent probably does not need the ability to delete calendars, modify staff availability or view every employee’s schedule.

A better tool might expose only three operations:

• Find available times

• Hold a selected time temporarily

• Submit a rescheduling request

The final confirmation could remain with the customer or a staff member.

This approach is more dependable than telling the agent, “Only use the calendar for rescheduling.” The restriction is enforced by the interface rather than interpreted by the model.

The same principle applies to data access. A support agent should not retrieve an entire customer record when it only needs an order status. A recruitment agent should not access salary data to schedule an interview. A reporting agent should not receive database write access simply because read and write operations use the same integration.

OWASP’s prompt-injection guidance recommends giving an AI application its own credentials and restricting privileges to the minimum required for its intended work. It also recommends handling sensitive functions in deterministic code rather than leaving their execution entirely to the model. The guidance is important because agents regularly process untrusted material, including emails, web pages, attachments and user-submitted documents.

Any of those sources may contain instructions that conflict with the agent’s real task.

Approval should depend on the action, not the entire workflow

Requiring a person to approve every step makes an agent safe but often makes it useless. Removing all approvals makes it efficient but potentially dangerous.

The answer is not simply “more human oversight.” It is better-placed oversight.

Let the agent complete low-risk, reversible work independently. Introduce approval at the moment the action becomes difficult to reverse, externally visible or financially significant.

A customer-support workflow could work like this:

• The agent reads the conversation and order history.

• It classifies the issue and drafts a response.

• It may send approved informational responses automatically.

• It can recommend a refund based on policy.

• A person must approve refunds above a defined threshold.

• The system blocks refunds outside the permitted policy, regardless of what the agent requests.

This preserves most of the speed without granting unlimited authority.

Anthropic describes a similar balance in its framework for trustworthy agents: autonomy is valuable, but users should retain control over how goals are pursued, particularly before high-stakes decisions. Its implementation examples include read-only access by default and explicit permission before modifying systems. Anthropic’s agent-safety framework reinforces an important design principle: permissions should become broader only when the use case has earned that trust.

Bound the agent’s time, money and reach

Permissions are only one part of an agent’s operating boundary.

An agent may have legitimate access to a tool and still use it in damaging ways. A research agent could make thousands of API calls. A sales agent could contact every prospect in the CRM. A troubleshooting agent could enter an endless loop of attempted fixes.

Production agents need operational limits such as:

• Maximum number of tool calls per task

• Spending or transaction limits

• Rate limits for emails, messages and API requests

• Maximum execution time

• Limits on the number of records that can be changed

• Approved recipients and domains

• Restricted working hours

• Automatic pause after repeated failures

These controls are not sophisticated, but they reduce the blast radius of unexpected behaviour.

If a sales agent is meant to assist a representative with five selected accounts, it should not be technically capable of emailing the entire database. If a procurement agent normally places orders below $500, an unusual $20,000 transaction should not depend on the model noticing that it looks suspicious.

The system should stop it.

Design escalation as a valid result

Many agents are built as if completing the task is the only successful outcome. That encourages the system to continue acting even when information is missing or contradictory.

A well-designed agent has at least three valid outcomes:

1. Complete the task.

2. Ask for missing information.

3. Stop and escalate.

The third outcome is often underdeveloped.

“Escalate when uncertain” is too vague unless uncertainty has observable triggers. Those triggers might include conflicting customer records, a policy exception, low confidence in document extraction, an unfamiliar AI tool response or a transaction above a set value.

The escalation should also include useful context. A person should receive:

• What the agent was asked to do

• What information it used

• What decision it reached

• Why it stopped

• What action requires approval

Without this context, human review becomes slow and frustrating. People may begin approving requests automatically, turning a safety control into a ceremonial click.

Anthropic’s research on measuring agent autonomy notes that models recognising and surfacing their uncertainty can complement external safeguards such as approval flows and access restrictions. The word complement is important. Self-reported uncertainty can help route work, but it should not replace enforced controls. Anthropic’s autonomy research makes the distinction clear.

Monitor decisions, not just outputs

Most AI evaluations focus on whether the final answer was correct. For an agent, that is not enough.

Two agents can produce the same successful result while taking very different paths. One may use the approved customer database. The other may search an unverified website, expose confidential information to an external service and then arrive at the same answer.

If monitoring captures only the output, both runs appear successful.

Production monitoring should record:

• Which tools the agent called

• What data it accessed

• Which actions it attempted

• Which actions were denied

• Where human approval was requested

• How often people changed or rejected its recommendations

• Whether it remained within its time, cost and transaction limits

These records help teams find a more useful class of problem: near misses.

A blocked attempt to send an unauthorized message is not merely evidence that the control worked. It may reveal a recurring flaw in the workflow, prompt or tool design. Waiting for an actual incident before investigating that pattern wastes valuable warning signals.

This aligns with the broader risk-management approach in the NIST AI Risk Management Framework, which organizes AI risk work around governing, mapping, measuring and managing risk across the system lifecycle. Agent boundaries should therefore be reviewed continuously, not added once before launch and forgotten.

Expand autonomy only after observing real behaviour

An agent should not receive maximum autonomy on its first day in production.

A more responsible rollout starts in observation mode. The agent recommends actions, but people continue executing them. This reveals whether its decisions are useful and where exceptions occur.

The next stage may allow low-risk actions while keeping meaningful changes behind approval. Autonomy can then expand selectively based on evidence such as:

• Low correction and rejection rates

• Stable performance across edge cases

• Reliable escalation behaviour

• No unexplained tool usage

• Acceptable cost and latency

• Clear audit records

• Proven recovery from failures

Even then, permissions should be easy to withdraw.

This is similar to how responsibility is assigned to a new employee, except an AI agent can act at machine speed and repeat a mistake across thousands of records before anyone notices. Its access should therefore be at least as carefully managed as human access, not less.

The real measure of a production-ready agent

The smartest agent is not necessarily the one a business can trust.

A production-ready agent is one that completes useful work inside a clearly defined operating area. It knows which information it may access, which tools it may use and which decisions belong to a person. More importantly, the surrounding software enforces those limits when the model fails to respect them.

Before replacing a model or adding another layer of reasoning, ask a more practical set of questions:

• What can this agent change?

• What is the worst action it can take without approval?

• Which mistake would be difficult to reverse?

• Can its permissions be reduced?

• What stops it from repeating an error at scale?

• Can we reconstruct exactly what it did?

• When should it stop instead of trying harder?

If those questions do not have clear answers, the agent does not need more intelligence yet.

It needs better boundaries.

Conclusion

A production-ready AI agent is not defined by how intelligent it appears in a controlled demo. It is defined by how safely and reliably it operates when instructions are unclear, data is incomplete and real business systems are involved.

Clear permissions, approval gates, restricted tools, spending limits, escalation paths and detailed audit logs may not look as impressive as advanced reasoning. But these are the controls that prevent a minor error from becoming a customer complaint, financial loss or security incident.

For businesses planning to automate complex workflows, professional AI agent development services can help turn an experimental agent into a controlled production system. The goal should not be to give an agent maximum autonomy from day one. It should be to expand autonomy gradually, based on evidence, while keeping every high-impact action within clearly enforced boundaries.

A smarter model may improve what an agent can do. Better boundaries determine whether you can trust it to do the job.