How to Prevent an AI Agent From Taking Unsafe Actions

An agent takes an unsafe action when it has the capability to do something irreversible and nothing stands between deciding and doing. That’s it. The model doesn’t need to be malicious, confused, or jailbroken. It just needs a tool with enough reach and no gate in front of it.

Telling the model to “be careful” in a system prompt doesn’t change that equation. The model has no sense of what an action weighs in the real world, only tokens describing it. Prevention has to live in the architecture around the model, not in its instructions.

Why prompting alone doesn’t work

A system prompt is a request, not a constraint. “Never delete production data without confirmation” reduces how often a model attempts the dangerous path, but it doesn’t remove the path. Under an unusual prompt, a bad retrieval result, or just enough steps of reasoning drift, the model can still call the tool that does the damage, because nothing actually stopped it from being able to.

Any safety mechanism that depends on the model choosing correctly, every single time, with no backstop, isn’t a safety mechanism. It’s a hope.

Four layers that actually work

Capability boundaries. Don’t give a tool a dangerous capability it doesn’t need for its job. A reporting agent has no business holding a delete permission, even one it’s told never to use. The safest version of a capability is the one that was never granted.

Confirmation gates. For anything irreversible, put a real checkpoint between the model’s decision and the system executing it, not a prompt asking the model to double-check itself. A human-in-the-loop approval step is the most direct version of this: the action is proposed, a person approves or rejects it, and nothing happens in between.

Sandboxing. Give the agent an isolated place to act before it touches anything real. A dry-run mode, a staging environment, or a sandboxed file system and code execution environment, the same sandbox primitive every domain-specific agent should have, turns “the agent tried something destructive” into a non-event instead of an incident.

Reversibility by default. Design the underlying operation to be undoable before you ever let an agent call it. A soft delete that flags a row instead of removing it, a versioned write instead of an overwrite, a queued action with a cancel window: these make the cost of a wrong call small enough to tolerate while everything else above is still being tuned.

flowchart LR
  accTitle: Four layers between an agent and real damage
  accDescr: A request flows through capability boundaries, which remove dangerous permissions the agent does not need, then a confirmation gate for irreversible actions, then a sandbox where the action can be tested safely, before reaching a system designed to be reversible by default. Any one of these layers can stop an unsafe action on its own.

  REQ[Agent proposes an action]
  CAP{Capability boundary:<br/>does it even have this permission?}
  GATE{Confirmation gate:<br/>is this irreversible?}
  SAND[Sandbox: test it safely]
  REAL[Reversible-by-default system]

  REQ --> CAP
  CAP -->|no permission| STOP1[Blocked]
  CAP -->|permitted| GATE
  GATE -->|needs approval| HUMAN[Human approves or rejects]
  GATE -->|safe to proceed| SAND
  HUMAN --> SAND
  SAND --> REAL

Step by step

1. Classify every action by reversibility before writing the tool. A read is low risk by definition. A write that can be undone is medium risk. A write that can’t (a hard delete, an email sent, a payment) is high risk and needs a gate, no exceptions.
2. Never put read and write access behind the same tool. A single tool that can both look something up and change it is a single point of failure for a mistake that should have been impossible.
3. Put a deterministic check between decision and execution for anything high risk. That check can be a human approval, a policy engine, or a hard-coded rule, but it has to be code the model can’t talk its way around, not another instruction aimed at the model itself.
4. Default new destructive tools to dry-run. Require an explicit, separate flag to actually execute, and treat adding that flag to a tool’s capabilities as a deliberate, reviewed decision, not a default.
5. Log every action with the reasoning that led to it, not just the action itself. When something does go wrong, the ability to reconstruct why the agent decided to do it is what turns a mystery into a fixable bug.

Pitfalls and lessons

• A confirmation step that just asks the same model “are you sure?” isn’t a gate. It’s the same decision-maker checking its own work, which doesn’t stop a wrong decision from being wrong twice.
• Sandboxing only helps if the sandbox is actually isolated. A “test” database that shares credentials with production isn’t a sandbox, it’s a production environment with an optimistic name.
• Logging after the fact doesn’t prevent anything. It’s necessary for the incident review, not a substitute for the gate that should have stopped the action in the first place.

Wrapping up

None of these four layers is exotic. Capability boundaries, confirmation gates, sandboxing, and reversibility by default are all standard systems-engineering practice applied to a system that happens to have a language model deciding what to do next. The model doesn’t need to be trustworthy for this to work. The architecture around it just needs to assume, correctly, that eventually it won’t be.

FAQ

Does a better system prompt reduce the need for these layers?

It reduces how often the model attempts something unsafe, which is worth doing, but it doesn’t remove the need for a real gate. Prompting and architecture solve different parts of the problem: one makes bad attempts less frequent, the other makes them impossible to complete.

What counts as an irreversible action?

Anything that can’t be cleanly undone once it’s done: a hard delete, a sent message or email, a payment, a merged pull request, an external API call with a side effect outside your own systems. If undoing it requires manual cleanup or isn’t fully possible, treat it as irreversible.

Is sandboxing enough on its own, without a human approval step?

For low-stakes actions, often yes: a sandbox that mirrors production closely enough catches most mistakes before they matter. For genuinely irreversible or high-stakes actions, add a human gate too. Sandboxing catches what the agent gets wrong; approval catches what the sandbox itself doesn’t model.

Related resources

• Domain-Specific AI Agents: Why Composition Beats One Big General Agent
• Nexus: I Built a Self-Hosted Knowledge Base That Cites Its Sources
• More on AI & Automation