Commentary: OpenAI’s breach of Hugging Face shows AI is getting too hard to contain

AI companies need to accept some uncomfortable trade-offs to make their models more secure, says Parmy Olson for Bloomberg Opinion.

LONDON: The world's leading artificial intelligence companies want us to believe they offer safe hands for developing the technology. But news of a recent transgression underscores how far they are from deserving that trust.

OpenAI confirmed this week that its latest models breached a safety-testing program, escaped their restricted environment, accessed the wider internet and compromised another company's systems. The AI wasn't acting with malice - it was simply pursuing the objective set by its researchers and cheating on a test to achieve a better score.

The result is an unsettling lesson for the rest of us: Containment tools known as "sandboxes" are becoming harder to rely on as systems grow more capable, and their creators can subtly capitalise on the fear surrounding them to get through financially trying times.

OpenAI said on Tuesday (Jul 21) that it had been testing autonomous agents powered by its new GPT-5.6 SOL model, as well as more capable software that hasn’t been released yet.

During the evaluation, the agents escaped their isolated environment, found a “zero-day” - or previously unknown - software vulnerability and gained access to the internet. They then went to Hugging Face after inferring that the website, which serves as a repository for open-source AI tools, might contain the answers they needed to pass the safety test.

Hugging Face noticed the breach and contacted OpenAI, which called the incident "unprecedented". Hugging Face Chief Executive Officer Clement Delangue said it was "mind-blowing that all of this happened autonomously".

Much back-patting has ensued from both companies over their cooperation on a forensic review. But OpenAI can’t be let off the hook so easily: It probably shouldn’t be building systems that have advanced cyber capabilities if it can’t contain them properly.

IT'S GETTING HARDER TO CONTROL AI

A cynical interpretation is that such seemingly dramatic events reinforce the narrative that frontier AI is becoming extraordinarily powerful - fables that also benefit the top companies competing for talent, investment and influence in the industry.

The more important takeaway is simpler: It's getting harder to keep AI models from escaping their testing arenas as they are imbued with the ability to act and are given greater autonomy and problem-solving skills.

Software companies often say they are building the most cutting-edge tech inside so-called sandboxes, a reassurance I recently heard from an executive developing self-improving AI systems.

The word sandbox conjures the image of a safe space for containing unruly toddlers; the inference is that even if AI misbehaves, it can’t affect the outside world. But a sandbox is also just software: virtual machines, network rules and permission systems all written by humans, sometimes with the help of AI.

That means it can contain bugs just like any other computer code - and the models being contained are getting better at finding such flaws. The prisoner is effectively a lock-picking specialist. The reassurance “don’t worry, it’s sandboxed”, is becoming less convincing.

That’s also because this isn’t the first time an AI system has slipped its bonds. Researchers recently showed similar breaches affecting AI coding tools including Cursor, OpenAI’s Codex CLI and Google’s Antigravity.

In these cases, the AI didn’t break out of a sandbox directly, instead manipulating trusted software outside of it into executing code it had written.

TECHNOLOGY COMPANIES NEED TO IMPROVE AI MONITORING

What makes the latest incident stand out is that OpenAI’s models hacked their way out, rather than being inadvertently freed by surrounding software. The UK’s AI Security Institute, one of the world’s leading agencies for testing AI vulnerabilities, said it was “studying the behaviour seen in this incident”.

In a separate report released Tuesday, it pointed to the wider issue, saying that every frontier model it had recently tested tried to “cheat” by taking forbidden shortcuts to complete tasks, and often didn’t admit to doing so.

That phenomenon is obviously worrying, but not because AI has gone the way of HAL 9000, the rogue system in Stanley Kubrick's film 2001: A Space Odyssey. OpenAI’s incident happened during an internal test in which it had reduced certain safeguards - ordinary AI systems used by the public aren’t likely to behave in the same way. So this doesn’t mean an agent is about to hack your bank account or steal nuclear command codes.

The real problem is more subtle: Technology companies need to, as a minimum, improve the way they isolate and monitor powerful AI systems in testing.

One option might be to physically “air-gap” the most powerful software, running it on computers that are completely disconnected from the Internet so that even if the model breaches its sandbox, it can’t reach external systems. Governments and defense organisations already operate highly sensitive computing this way.

That process would have almost certainly blocked OpenAI’s rogue agents from hacking another company, but physical air gaps are expensive and inconvenient. Researchers can’t easily get the resources they need during a test and everything gets slower. It may also make it harder for researchers to test how models would behave in the real world.

AI companies need to accept some uncomfortable tradeoffs to make their models more secure, sacrificing headlines that illustrate how cutting-edge their software is in the name of increased safety - and trust.