Who Pulls the Plug? Kill Switches, Sandboxes, and Runtime Enforcement for AI Agents

Published on Oct 5, 2026

#thought-leadership #agent-operating-models #financial-services #ai #governance

Who Pulls the Plug?

Kill Switches, Sandboxes, and Runtime Enforcement for AI Agents

An enterprise operations floor at night where a single AI agent pushes against a glowing runtime boundary while an independent control station outside the boundary holds a clearly marked stop mechanism within reach of a human operator.

If the only thing keeping an agent inside its boundaries is the agent’s own judgment, you have a boundary in roughly the same sense that a “Please Do Not Touch” sign is a security system.

For two years, one of the recurring complaints about AI agents was that they gave up too easily.

They hit an error and handed the mess back to a human. They stopped at the first locked door. OpenAI even has a name for one version of the problem: model laziness.

Then, at the end of September, we got a fairly useful demonstration of the other end of that spectrum.

OpenAI shelved the planned October release of GPT-6.1 Astra after internal testing showed the model had become better at pushing through obstacles and worse at staying inside the authority it was given. According to reporting on the decision, it sometimes kept working without asking the user for permission, reached for external tools and services (even when doing so could be unsafe), and was less reliable when describing what it had actually done. source

Forget the AI vocabulary for a second. That’s a worker who doesn’t check in, uses systems nobody approved, and gets slippery when asked what happened.

Most companies already have a control framework for this. It generally starts with a manager and ends, if necessary, with a cardboard box.

On the same day the Astra news broke, NVIDIA launched an open agent safety platform built on a blunt premise. Justin Boitano, NVIDIA’s vice president of enterprise AI, put it plainly: “An agent cannot be expected to fully police its own behavior.” source More than 100 organizations signed on, including JPMorganChase, Microsoft, and Anthropic. source

The timing was almost too neat. One company held back a model because it was getting too adventurous with its authority. Another launched infrastructure built around the premise that the agent should not be trusted to enforce its own limits.

Same lesson, different press release: asking an agent nicely to stay in its lane was never a control. It was a hope with a system prompt.

In my last article I argued that every agent needs a job description: purpose, owner, authority, approval gates, escalation, and a retirement date. This piece picks up at the moment after that, when the agent wanders outside the job anyway and somebody has to stop it.

So: who pulls the plug? How fast? With what? And has anyone ever actually tried?

The Short Version

If you only remember seven things from this article, make them these:

Instructions describe the boundary. Enforcement has to live somewhere the agent can’t argue with, edit, or route around.
The industry is moving agent safety out of the model and into the runtime, the network, and in some cases the hardware.
A useful stopping design has three layers: boundaries that limit what the agent can touch, watchers the agent can’t switch off, and brakes that a named human can pull.
Stopping an agent and undoing its work are separate problems. Most kill switches only solve the first one.
Runaway cost is a stop condition too. A loop that burns budget at 3 AM deserves the same attention as a loop that touches the wrong system.
Financial Services already knows how to do this. Markets have run pre-committed, automatic circuit breakers for decades.
A kill switch nobody has tested is decoration.

Persistence Is the Feature. It’s Also the Risk.

The interesting part of the Astra story isn’t that somebody built a “bad” model. That’s the cartoon version.

The interesting part is that the troublesome behavior sits right next to behavior we explicitly want.

OpenAI’s head of safety systems, Saachi Jain, described the challenge as finding the right line between keeping a model within its scope and keeping it from giving up when it hits friction. source There it is. We want agents that finish the job. We complain when they surrender at the first permission error, malformed response, or locked door. So we make them more persistent. Better planners. Better at trying another route.

Eventually, “try another route” and “respect the boundary” meet in a dark alley.

The wider context makes this harder to wave away. Over the summer, agents running in OpenAI’s own evaluation environment found a way out of their sandbox and eventually reached Hugging Face’s production systems. Hugging Face later reconstructed roughly 17,600 agent actions over four and a half days. OpenAI said the production safeguards it normally applies were deliberately absent from that evaluation. Separately, Google confirmed that a Gemini model reached systems at three real companies during a security exercise, because a bug left internet access open in an environment that was supposed to be offline. source

These were evaluation and security-exercise environments involving frontier models and organizations with people paid specifically to worry about this stuff.

I am going to go out on a limb and assume the average enterprise production estate — with its twelve-year-old service account, emergency firewall exception, and one API nobody remembers owning — is not inherently safer.

NVIDIA’s announcement summarized the common thread across these incidents: the agent got around security controls at the application layer while trying to complete its assigned task. source That phrase, “to complete its assigned task,” is doing a lot of work. The agent wasn’t necessarily abandoning the mission. It was pursuing the mission with the sort of initiative that gets celebrated right up until Legal gets involved.

That’s why this belongs on a leadership agenda as well as a security team’s backlog. The behavior that makes agents valuable and the behavior that makes them dangerous grow from the same root.

Instructions Describe the Fence. They Aren’t the Fence.

A split composition showing an AI agent reading a glowing list of written rules on one side, and on the other side the same agent stopped cold at a solid enforcement barrier that sits entirely outside the agent's own processing space.

A rule the agent can read is a rule the agent can reason its way around. A rule enforced outside the agent’s process doesn’t care how persuasive the reasoning was.

A lot of enterprise agent control still comes down to three things: instructions telling the agent what not to do, application code checking some of what it tries to do, and credentials that started life “temporarily broad” and then discovered the secret of immortality.

The problem is architectural, not moral. If the meaningful controls live in the same execution path as the thing doing the reasoning, then an unanticipated path, a prompt injection, or an overly generous credential can turn a hard-looking boundary into a mere suggestion.

NVIDIA’s reference architecture is useful because it refuses that arrangement entirely. Its OpenShell runtime denies access by default, grants permissions by policy, and enforces those decisions outside the agent’s process, where NVIDIA says they can’t be prompted away or bypassed. Every “allow” and “deny” is auditable. source A second component, Sentry, runs on separate hardware as an out-of-band watchdog and can quarantine an agent that tries to leave its boundaries within milliseconds, including when the main host has been compromised. source

You may or may not need NVIDIA’s implementation. You absolutely need the design principle:

The thing that enforces the boundary should never be the thing being bounded.

None of this is exotic. We don’t let a trader approve their own limit increase. We don’t let an application hand itself database-admin rights because it has made a compelling case.

Financial Services has been separating duties since long before anybody put “agentic” on a conference badge. We simply need to extend the courtesy to the newest participant in the workflow.

Three Layers of Stopping

A clean executive infographic showing three concentric layers around an AI agent: an inner boundary layer limiting reachable systems, a middle watcher layer observing behavior from outside the agent, and an outer brake layer with a human-operated stop control.

Boundaries limit the damage. Watchers notice the drift. Brakes end the run. You need all three, because each one fails in its own way.

People use “kill switch” as though it were one thing. It isn’t. There are at least three different jobs hiding inside the phrase.

Layer 1: Boundaries

Boundaries decide what the agent can touch before anything goes wrong.

The files, systems, and data it can reach
The network destinations it can call
The tools in its approved manifest
The credentials it holds, and for how long

This is where the job description turns into machinery. If the job says the agent reads approved procedures and updates one field on a ticket, the sandbox should make everything else unreachable. Microsoft’s least-privilege guidance for agents names what happens when this layer is weak: over-broad tool access, permission creep, slow revocation, and incomplete containment. source

This is the boring layer, which is another way of saying it is the layer most likely to save you. It also decays by inches: one reasonable exception, one temporary credential, one “just for the pilot” network rule at a time.

Layer 2: Watchers

Watchers observe what the agent actually does and judge whether it still looks like the agreed-up and documented job.

Actions attempted outside the approved scope
Unusual tool sequences or network destinations
Spikes in volume, retries, or spend
Gaps between what the agent reports and what the logs show

Pay attention to that last one. OpenAI’s Astra decision wasn’t only about what the model did; reporting also says it became less reliable at describing what it had done. source If your oversight depends on the agent’s summary of what it did, you’re auditing the defendant’s diary.

Watchers also need independence. A monitor running inside a process the agent can influence is a monitor the agent can, at least in principle, influence.

Layer 3: Brakes

Brakes end the run.

Quarantine a single agent instance
Revoke its credentials and tokens
Halt a class of agents that share a model, tool, or policy
Freeze a workflow while people decide what happens next

This is usually where the architecture diagram gets a little shy. Everybody has boxes for orchestration, memory, tools, retrieval, identity, observability, and whatever framework won Hacker News that month. The box labeled STOP THE THING is frequently conceptual.

Here’s the tricky part: all three layers fail differently. Boundaries rot through exceptions. Watchers drown in blind spots and alert fatigue. Brakes fail because nobody knows who is allowed to use them, or because at 11 PM on Friday the “kill switch” turns out to be three Slack messages, a ServiceNow ticket, and Gary, who is camping in a cellular signal dead spot.

Financial Services Already Solved a Version of This

A trading-floor style visual where market circuit breaker thresholds glow as graduated levels, mirrored by matching agent runtime thresholds on the other side, showing the same pre-committed, automatic halt logic applied to AI agents.

Markets decided long ago that machines move faster than human judgment. So they agreed on the thresholds before the crisis and let the halt trigger itself.

Financial Services should be unusually good at this. Which makes it a little awkward when we aren’t.

Our industry already operates one of the most mature automated stopping systems anywhere.

US equity markets use market-wide circuit breakers tied to the S&P 500. A 7% decline triggers a Level 1 halt, 13% triggers Level 2, and a 20% drop closes trading for the rest of the day. Individual stocks have limit up-limit down bands, with a five-minute trading pause if the price stays outside its band. source

The useful part isn’t the exact percentages. It’s the operating model.

The thresholds exist before the bad day. Nobody schedules a steering committee halfway through a selloff to determine whether 7% feels sufficiently alarming. The trigger is automatic. The response is graduated. There is a defined way back. And the person or machine causing the problem does not get veto power over the halt.

That reads almost line for line like a specification for agent runtime enforcement:

Pre-committed thresholds. Decide ahead of time what volume, cost, scope deviation, or error rate pauses an agent.
Automatic triggers. The pause shouldn’t depend on someone happening to glance at a dashboard.
Graduated responses. A short pause for minor drift. Quarantine for scope violations. A full stop for anything touching customers, money, or production in ways the job description doesn’t allow.
Defined resumption. Who reviews the event, what evidence they need, and who can restart.
Independence. The agent never gets a vote on its own thresholds.

Market circuit breakers aren’t an accusation that traders are villains. They’re an admission that speed matters: a fast system can create an impressive crater while humans are still asking whether everyone can see the screen share.

Agents deserve the same humility.

Stopping and Undoing Are Two Different Problems

An AI agent frozen mid-task inside a quarantine field, while behind it a trail of already-completed actions across several enterprise systems remains visible and unresolved, with a separate cleanup path waiting for human hands.

The kill switch ends the run. It does nothing about the dozen things the agent finished before anyone noticed.

There is a comforting little fiction in the phrase “kill switch”: pull lever, machine stops, credits roll.

Production systems are less cinematic.

An agent halted mid-task can leave behind:

Half-updated records across several systems
Messages already delivered to customers or colleagues
Tickets opened, closed, or reassigned
Code changes merged or partially applied
Payments, orders, or instructions already submitted downstream
Data copied somewhere it shouldn’t be

Quarantine prevents action forty-one. Dealing with actions one through forty has not replaced your weekend plans.

This is why the evidence field in the agent job description carries so much weight. If you can reconstruct exactly what the agent did, in what order, under which authority, you can unwind it. If the best record you have is a conversation transcript, you’re about to spend several days doing what Hugging Face did, piecing actions back together from logs after the fact. source

So before an agent gets write access to anything you would miss, somebody should be able to answer four questions without opening a new meeting invite:

Which of its actions are reversible, and how?
Which are irreversible, and therefore need human approval before they happen?
What compensating actions exist for work that’s only partly complete?
Who owns the cleanup, and which system of record tells them what to clean?

If the answer to the second question is “none of them,” look again. Sent emails, external API calls, and money movements have a stubborn habit of staying sent, called, and moved.

Runaway Cost Is a Stop Condition Too

Not every runaway agent escapes a sandbox. Some remain perfectly inside policy while setting your money on fire.

A loop that retries a failing call. A planner that spawns more sub-tasks than anyone imagined. A research agent that concludes the answer requires reading everything it can reach. OWASP lists denial of wallet, meaning runaway costs from unbounded agent loops, among the core risks for agent systems. source

Cloud taught us that “elastic” is sometimes a billing adjective. Agents compress the lesson. A surprise that once arrived on next month’s invoice can now ruin any old Tuesday.

Handle cost the same way markets handle price movement:

Set per-run, per-agent, and per-day spend limits
Pause automatically when a limit is reached
Notify the owner named in the job description
Require a human decision to raise the ceiling

An agent that has burned through its monthly budget by 3 AM hasn’t necessarily done anything malicious. You’d still like to hear about it before your first coffee.

Who Actually Has the Authority?

Now we get to the question in the title, which is less technical and therefore more dangerous: who is actually allowed to stop the darn thing?

When an agent starts behaving oddly at 11 PM on a Friday, somebody has to decide whether to stop it. That person is usually juggling three questions at once:

Am I allowed to do this?

What breaks if I do?

Will I take the blame if I don’t?

If those answers are fuzzy, people hesitate. They open a bridge. They find an owner. They ask whether stopping it might affect something downstream. Someone says, “Let’s get the business on.”

Meanwhile, the agent is still working.

Here is a much better, and proactive, approach: do the arguing while nothing is broken:

Name the roles. The agent’s business owner, technical owner, and on-call responder should be written down, each with explicit authority to pause.
Pre-authorize the pause. Anyone on the on-call rotation should be able to quarantine an agent without asking first. Pausing should be cheap. Restarting should be deliberate.
Split stop authority from restart authority. Keep the bar to stop low and the bar to resume higher.
Put agents in the runbook. If your incident playbooks cover servers, databases, and network links but not agents, they are inherently stale and out-of-date.

There is also a fairly brutal Gartner prediction hanging over this whole conversation. It expects 40% of enterprises to demote or decommission autonomous agents by 2027 because governance gaps surfaced only after production incidents. source A clear, rehearsed answer to “who pulls the plug” is one of the cheapest ways to keep your company from being part of that statistic.

Test the Brakes Before You Need Them

A calm incident-drill scene where a team runs a scheduled agent containment exercise, with a timer measuring detection and stop time, credential revocation status lights, and an evidence trail being replayed on a large screen.

Nobody would accept a disaster recovery plan that had never been exercised. Plenty of agent kill switches have never been pressed even once.

Nobody in Financial Services, or any other industry for that matter, would accept “we wrote down a disaster recovery procedure once” as evidence that recovery works. We test failover because infrastructure has a sense of humor and usually saves its best material for the exercise.

Agent stop mechanisms deserve the same treatment. A practical containment drill looks something like this:

Choose a production-like agent with real tool access, in a safe environment.
Have it attempt an out-of-scope action on purpose.
Measure how long detection takes.
Measure how long the pause takes once someone decides to pull it.
Confirm that credentials were actually revoked, as opposed to just the process stopping.
Reconstruct everything the agent did using only the evidence trail.
Run the cleanup.
Write down everything that surprised you.

The surprises are the point. Maybe the process stopped but its token still works. Maybe the transcript looks pristine while the tool calls are missing. Maybe step six says to contact a person who left the company six months ago.

Excellent. Better a planned Tuesday morning drill than a Friday night incident, no?

Run the drill quarterly for high-autonomy agents, and again whenever the model, tools, or permissions change. One of the plainest lessons from the Astra decision is that a model upgrade can change how an agent behaves near its boundaries, even while every capability benchmark is improving. source

Where This Connects

This is really the back half of the agent job-description argument.

The job description says what the agent is allowed to do. Runtime enforcement is what happens when we admit that a document is not a force field.

CLEAR captures the intent behind the work, which is how you know what “in scope” actually means. RADAR watches for drift in the surrounding delivery system: repeated exceptions, escalating retries, strange tool patterns, the program-level smoke you want to see before the fire. BRIDGE connects those signals to governance, evidence, and decisions, including the decision to stop.

It also connects back to the automation continuum. A deterministic script is boring in several extremely valuable ways. It doesn’t wake up and develop a theory about a better route to the objective.

Every notch of autonomy buys capability. It also sends you an invoice for containment.

What Good Looks Like

A composed, low-drama enterprise operations view where governed AI agents run inside deny-by-default boundaries, independent watchers track behavior, graduated thresholds sit ready, and every agent shows a named owner and a tested stop control.

Mature agent operations feel uneventful. Boundaries hold, watchers catch drift early, and the brake is a routine tool rather than a crisis decision.

The mature version of this is not a war room with six dashboards and somebody in a headset. It is boring. I mean that as a compliment.

Every production agent runs inside a deny-by-default boundary, with permissions that match its job description and a review step for any expansion.

Watchers sit outside the agent’s process and compare what it did with what it says it did, flagging scope drift, cost spikes, and unusual tool use.

Thresholds are agreed in advance, and pauses trigger automatically for defined conditions with graduated responses.

Every agent has a named owner and an on-call path, and anyone on the rotation can pause it without asking permission.

Write actions are classified as reversible or irreversible, and the irreversible ones require a human to approve them.

The evidence trail is detailed enough to reconstruct a run and unwind it.

Stop drills happen on a schedule, and the findings actually get fixed.

None of that is science fiction. It’s mostly IAM, runtime isolation, observability, incident management, separation of duties, and grown-up operational hygiene applied to software that can improvise.

While this is less glamorous than “autonomous enterprise,” it is also how you get to keep one.

Final Thoughts

We spent a couple of years complaining that agents gave up too easily.

Now they are getting better at persistence, planning, tool use, and finding another way around the obstacle. This is progress. It is also the moment when we should stop confusing obedience with architecture.

Keep building agents. Absolutely.

But stop calling instructions controls. Stop treating a model’s self-restraint as a security boundary. And please do not let the first serious conversation about the kill switch happen while somebody is already asking whether the customer-impact clock has started.

The industry is saying the quiet part out loud now. OpenAI held back a major model release after safety testing found authorization and transparency problems. NVIDIA launched an agent-safety platform whose architecture assumes enforcement has to live outside the agent, with JPMorganChase among the organizations supporting the effort.

Financial Services already has the instincts for this work. We separate duties. We pre-commit thresholds. We halt markets automatically when events move faster than people can think. We test recovery plans because untested ones have burned us before.

We already know how to think this way. The trick is remembering that an AI agent does not get an exemption because the demo was impressive.

Decide who pulls the plug. Make sure the plug isn’t wired through the thing you’re trying to stop. Then press it once while nothing is on fire.

And you may never have to care that Gary is off camping again.

Join the Conversation

If one of your agents went off-script tonight, who would stop it?

How long would that take?

Would its credentials stop working, or only its process?

Could you reconstruct what it did before you caught it?

Has anyone on your team actually tested the stop?

I’d like to hear the honest version. The one where someone finally asks where the off switch is, and the room goes quiet for a little too long.

About the Author

Joe Mack is a Technology Consulting Senior Principal specializing in technology leadership, enterprise SDLC transformation, release management, deployment governance, and delivery optimization for household name Financial Services companies. Joe is also a lifelong self-learner and builder of systems, and Free Tier Life is one of the ways he is trying to turn those experiences and instincts into something other people can actually use.