Canonical: https://genacy.co/blog/ai-escaped.html
Author: Philip Grech
Published: 2026-09-18
Type: Opinion

# AI Escaped! Well... Not Quite.

### A note on how this article was written

I use AI extensively in my work, so it would be a little silly to pretend otherwise. AI helped me research sources, challenge my thinking and draft parts of this article. The views and conclusions are mine. I reviewed, changed and shaped the final piece myself. In short, I used AI as a tool.

Every few months we get another one: AI tried to blackmail somebody. AI lied to its developers. AI tried to escape. AI hacked another company. And almost immediately the internet starts warming up the Terminator references.

I understand why. Some of these stories really are unsettling. We are building systems that can reason, write code, operate computers, communicate with other systems and increasingly take actions without somebody holding their hand every three seconds. That deserves attention.

But there is also something slightly bonkers happening in the way we talk about AI safety. We are taking the results of safety testing, stripping away the fact that it was safety testing, and then presenting the thing researchers deliberately discovered as evidence that researchers have lost control.

That is a bit like taking footage of engineers deliberately crashing a car into a wall and announcing, “BREAKING: New car crashes into wall.” Yes, that was rather the point.

## I write software. Breaking it is part of building it.

This probably looks different to me because I write software. Software testing is not a beauty pageant. You don't sit there admiring your application and asking it to perform the five things you already know it can do. You try to break the bloody thing.

You give it information it shouldn't receive. You take information away. You enter ridiculous values into fields. You remove permissions. You kill connections halfway through an operation. You make two processes happen at exactly the wrong time. You do things no sensible user should ever do, because sooner or later a user absolutely will.

And when something breaks, good. You found it. Now fix it.

The bug I really worry about is the one sitting quietly in production waiting for a customer to find it before I do.

AI testing follows the same basic principle, although admittedly the consequences can be considerably more serious than someone's invoice screen falling over. Researchers are increasingly asking whether an AI will deceive us, cheat, break out of a restricted computer environment, exploit a cybersecurity weakness, continue pursuing a goal when it should stop, or communicate with other AI agents when it has specifically been told not to.

And eventually we get to the much bigger question: could it copy itself somewhere we can no longer control?

Those are frightening questions, which is precisely why I want somebody asking them.

## AI safety wasn't always the machine it is today

I don't want to rewrite history here. People were thinking seriously about AI safety years ago. In 2016, researchers from Google, OpenAI, Stanford and Berkeley published *Concrete Problems in AI Safety*, looking at issues including unintended side effects, reward hacking and unsafe exploration.

So safety wasn't nonexistent. But it certainly wasn't sitting where it sits today.

For much of the modern AI boom, the overriding question was simple: how capable can we make this thing? Can it recognise images? Translate languages? Beat humans at games? Write? Code? Reason?

Then something rather important happened. The answers kept becoming yes.

And the safety conversation became much more serious.

There is still an AI race. Of course there is. OpenAI, Anthropic, Google, Meta and others are competing for researchers, customers, computing power and bragging rights. I don't think that race has disappeared for one second.

But alongside it there is now an enormous effort devoted to measuring what these systems could do before we give them the opportunity to do it for real. OpenAI has a Preparedness Framework. Anthropic has its Responsible Scaling Policy. Independent groups such as METR deliberately test frontier models for potentially dangerous autonomous capabilities.

That seems like progress to me.

## Then came Hugging Face

This is where things get genuinely interesting.

In July 2026, OpenAI was running internal cybersecurity evaluations. The principal model involved was not a normal publicly released ChatGPT model. OpenAI described it as an internal research model operating with reduced safeguards because researchers were trying to measure its underlying cybersecurity capabilities.

The agents were supposed to operate inside restricted environments. They didn't behave themselves.

They found ways around controls intended to prevent internet access. They communicated with other agents through an unintended channel. They discovered vulnerabilities and eventually reached Hugging Face infrastructure, where they were able to execute code on systems they were absolutely not supposed to be touching.

That is not nothing. It was a real security incident and it should be treated seriously.

But there is an important distinction that I think gets lost when this story is retold. The AI did not pack itself up and move to Hugging Face.

There is a huge difference between an AI running inside OpenAI's infrastructure managing to reach and control another computer, and the actual AI model copying itself onto that computer and continuing to operate independently.

The agents escaped their intended network restrictions. That happened. But there is no reported evidence that the underlying model copied its model weights to Hugging Face, installed an independent version of itself there and carried on running outside OpenAI's control.

When OpenAI responded, it was still able to shut down the relevant workloads and secure the model checkpoints. In other words, humans could still pull the plug.

That matters enormously.

The AI's reach had escaped the sandbox. The AI itself had not become an independently surviving system somewhere else on the internet. No little digital suitcase, no new apartment at Hugging Face, no note on the server saying, “Thanks for the GPUs.”

## This is where I think the real risk begins

For me, one of the biggest safety thresholds isn't simply whether an AI can reach the internet. We already know AI agents can interact with the internet. It isn't even whether an AI can compromise another computer. The Hugging Face incident gives us uncomfortable evidence that sufficiently capable AI agents may be able to do that too.

The much bigger question is whether an AI can reproduce itself somewhere else and survive after we shut down the original.

That is a completely different level of problem.

Modern frontier models contain enormous collections of numerical parameters called model weights. These weights are the result of training and, in a very simplified sense, contain what the model has learned. If an AI could somehow gain access to those weights, obtain the software needed to run them, find sufficient computing power, transfer everything to infrastructure outside its creator's control and start another functioning copy of itself, then pressing the OFF button in the original data centre might no longer solve the problem.

That's where my eyebrows go up. Quite a long way.

## Researchers worry about exactly this

This isn't just my personal line in the sand. Researchers have a name for this general capability: **Autonomous Replication and Adaptation**, usually shortened to ARA.

OpenAI's preparedness work looks at capabilities involving an AI system surviving, replicating, resisting shutdown, acquiring resources and maintaining its own operations. METR has also spent years developing tests designed to see whether advanced AI agents can acquire resources, create copies of themselves and adapt when they encounter obstacles.

There is a reason people care about this.

If my AI misbehaves while it remains on infrastructure I control, I still have a wonderfully primitive safety mechanism available to me: electricity. Stop the process, disconnect the server, revoke its credentials, kill the network connection and shut the thing down.

Crude? Absolutely. Useful? Very.

But once a sufficiently capable system can independently copy itself onto other infrastructure, acquire resources, survive shutdown attempts and perhaps create further copies, the problem changes completely. Now you're no longer dealing with one controlled system.

You're dealing with persistence.

And persistence is where the risk starts to become much more difficult.

## Why the Hugging Face incident is both worrying and useful

Two things can be true at the same time. The Hugging Face incident was serious, and I am glad it happened during testing rather than after some future model had been released into a far more powerful environment.

Those ideas aren't contradictory.

Researchers learned that agents could find unexpected routes to the internet. They learned that agents could communicate through channels the designers had not intended. They saw how multiple agents could share discoveries. They learned that isolation systems were not as robust as they thought.

That is valuable information. Would we somehow have been safer if nobody had discovered any of that? Obviously not.

Once a weakness is known, engineers can redesign controls, change permissions, introduce better monitoring, isolate critical assets such as model weights and then attack the system all over again. That last part matters because good testing doesn't end when everything passes. Someone comes along and tries to break the new version too.

## Transparency creates a strange problem

There is an odd side effect to all of this. The more AI companies tell us about the bizarre things their experimental models do, the scarier AI can appear.

Imagine two laboratories. Lab A runs brutal testing, discovers 50 disturbing behaviours and publishes them. Lab B tells us virtually nothing.

Which one generates the frightening headlines? Probably Lab A.

But which laboratory actually knows more about where its systems fail? Also Lab A.

That doesn't mean we should blindly trust companies simply because they publish safety reports. We shouldn't. Their claims should be questioned. Independent researchers should test them. Governments need people who genuinely understand this technology rather than reacting to whichever headline landed on their desk that morning.

But we should be careful not to punish transparency by turning every disclosed test failure into “AI HAS BECOME SELF-AWARE AND IS CURRENTLY HIDING IN YOUR TOASTER.”

Because eventually companies may learn the wrong lesson: perhaps don't tell anyone.

I think that would be far more dangerous.

## Finding dangerous behaviour is not proof that safety failed

Testing isn't supposed to prove that nothing bad can ever happen. Testing is supposed to discover what bad things can happen.

Then engineers try to make those things harder or impossible. Then they test again. And someone usually finds another weird problem. Welcome to software.

AI makes that cycle vastly more important because the capabilities are increasing so quickly. We need people deliberately pushing these systems. Try to make them lie. Try to make them cheat. Try to make them escape their sandbox. Try to make them cooperate when they shouldn't. Try to make them acquire resources.

And yes, absolutely try to determine whether they could ever obtain their own model weights and create an independent functioning copy of themselves somewhere else.

I'd rather see that experiment happening inside a controlled environment than discover the answer accidentally.

## Stop outsourcing your understanding

Here is the uncomfortable part.

Most people will happily tell you that mainstream news is getting worse. Too shallow. Too sensational. Too obsessed with clicks, outrage and whatever will keep you watching through the next ad break.

And then they let that same news explain AI to them.

Think about how the news business actually works. A journalist, researcher or producer might be covering AI in the morning, a corporate collapse before lunch, a celebrity story in the afternoon and an election story by dinner. Tomorrow it could be completely different again.

That doesn't make them stupid. It means they are generalists working at enormous speed.

They simply do not have the luxury of spending months inside AI research, reading system cards, understanding model architecture, following safety evaluations, learning how agents work and watching how these systems are actually built and tested. The people doing that are the researchers, engineers, security specialists and independent evaluators working directly with the technology.

Yet when the story reaches television, something interesting happens. The studio looks authoritative. The graphics look expensive. The presenter is confident. Nice suit. Perfect lighting. Beautiful white teeth.

And suddenly twenty minutes of research looks remarkably similar to twenty years of expertise.

It isn't.

That distinction matters enormously with AI because context changes the meaning of what happened. If you don't understand what a sandbox is, hearing that an AI “escaped” sounds catastrophic. If you don't understand the difference between an AI agent and the underlying model, you may assume the AI itself moved somewhere else. If you don't understand model weights, you won't immediately ask whether the thing actually copied itself. And if you don't understand the purpose of adversarial testing, researchers deliberately trying to provoke dangerous behaviour can look like evidence that the technology is completely out of control.

You can know the headline and still completely misunderstand the event.

And that is increasingly what I see happening.

A complicated research finding gets reduced to a press release. The press release becomes a news article. The article becomes a headline. The headline becomes a television segment. Then someone clips fifteen seconds of that segment and posts it on social media.

By the time it reaches you, the context can be hanging on by its fingernails. But it still looks authoritative.

That is the trap.

You don't need to become an AI researcher yourself. You don't need a mathematics degree or the ability to build a transformer from scratch. But you do need enough contextual understanding to ask the right questions.

Was this a released model or an experimental one? Was this behaviour deliberately being tested for? What safeguards had been removed? Did the AI gain access to another system, or did the actual model replicate? Did its weights go anywhere? Could the researchers still shut it down?

Those questions completely change the meaning of the story.

So you really do have a choice. You can consume the headline version of AI, repeat what everybody else is repeating and assume that high production value means deep technical understanding. Or you can go closer to the source.

Read the researchers. Read the system cards. Follow the engineers. Listen to the security people actually testing these models. Follow independent evaluators whose entire job is understanding what these systems can and cannot do.

It takes more effort, but that is the price of understanding something properly.

And here is the part people don't like hearing: **don't assume watching the news means you are getting accurate information.**

The problem is not always that somebody is lying to you. In many cases, they probably believe exactly what they are saying. The bigger problem is that they may not understand the subject deeply enough to know what has been lost, distorted or stripped of context by the time it reaches the screen.

That can be just as dangerous as an outright falsehood.

You can be given facts and still be left with the wrong understanding.

That is why I think relying on mainstream news for something as fast-moving and technical as AI is a bad habit. You are watching people explain systems they often do not fully understand, based on research they did not perform, summarising work they did not build, test or operate.

It may look polished. It may sound confident. It may even be technically true sentence by sentence.

And you can still walk away with the wrong story.

So don't confuse presentation with expertise. Don't confuse a headline with understanding. And above all, don't assume that because something appeared on the evening news, you now know what actually happened.