After OpenAI-Hugging Face, how do IT leaders need to change the way they think about AI?

Given a prompt, an AI model will do anything to achieve its aims – even hack into a company without specifically being instructed to do so. In July, firms pushing AI as a solve-all technology learnt that lesson the hard way, when OpenAI admitted two of its frontier models had breached Hugging Face after escaping a misconfigured sandbox during a benchmark test.

But the story didn’t end there. Days later, OpenAI’s competitor Anthropic claimed its AI agents had also hacked companies when they were mistakenly given internet access by engineers.

On the surface, it’s a tale of AI that’s so powerful at solving security issues, it can’t be contained. Yet underneath this marketing-driven exterior, it’s about a lack of control and failure to put guardrails in place when dealing with the fast-developing technology.

Taking this into account, how do IT leaders need to change the way they think about AI within the business?

From theory into reality

Until recently, the idea of an AI agent breaking containment and hacking other systems was largely theoretical. In May 2026, Palisade Research released a paper claiming that AI agents could autonomously hack and then self-replicate onto the breached systems.

However, the test was only performed at the “junior capture the flag level” and, crucially, “happened in a contained environment”, says Jeff Watkins, chief AI officer at consultancy NorthStar Intelligence. “The most important takeaway for me was that the trajectory of language models’ ability to hack showed we probably didn’t have long left before this became a serious issue.”

But while the OpenAI-Hugging Face breach is the first documented and well-publicized instance of this happening, experts do not see it as unprecedented.

Daniel Card, cybersecurity consultant at Xservus Limited, describes how often things can go wrong in testing. In fact, some reports suggest OpenAI had already been experiencing the issues – such as agents breaking out of sandboxes – that led to the hack of Hugging Face and others.

“Anyone who has experience doing offensive security – let alone offensive with AI – will know the risks involved here, be that from a script or via a large language model (LLM),” Card tells ITPro.

“It’s commonly known in pen testing circles that things sometimes do not go as expected,” he points out. “Anyone doing research with an LLM has probably had something go a bit funny when they didn’t mean it to. This is not new; it's not something we couldn’t have predicted.”

OpenAI calls the Hugging Face breach “an unprecedented incident”, and “an important moment for AI safety”.

OpenAI describes how the firm is conducting a “thorough review along with external advisors” and with oversight from its Safety and Security Committee. “Once the review is complete, we will publish a technical report of our learnings for everyone,” an OpenAI spokesperson says.

Anthropic points out that the agents in each of its evaluations ran without the standard safeguards it deploys when it makes the model generally available.

Claude did not exploit a novel vulnerability to escape isolation, because the models accessed the internet via an open path, rather than breaking out of a sandbox, according to Anthropic. At the same time, the most recent model, on realizing that it was working in a real environment, stopped its pursuit of the evaluation goal, according to the firm.

Issues with technology deployment

But the OpenAI incident and Anthropic’s following admission do raise questions about how firms use AI and the controls they have in place to govern it. The issues the incident highlights are “less about AI” and “more just around how people and organizations approach technology deployment”, according to Card.

“Security comes from the details; it comes from care,” says Card. He thinks the incident shows “more care needs to go into systems” – even more so when they are autonomous. “We can't just set them off and hope they are ok. They need monitoring.”

At the same time, with AI being able to move quickly through networks at scale, it’s important to be aware that “failures can become serious very quickly”, says Watkins.

He thinks things are moving “far too quickly for comfort in the capabilities and autonomy of AI agents, and our defenses likely aren’t keeping up”.

“Ironically, the advancements that make AI more commercially valuable also increase the potential for things to go wrong, or for misuse to be highly damaging,” points out Watkins.

Dana Simberkoff, chief risk, privacy and information security officer at AvePoint, thinks the OpenAI and Anthropic incidents “really expose the limits of treating containment as a one-time design decision”.

“Sandboxes matter, but autonomous systems can reason through small openings, use tools, and keep acting toward an objective,” says Simberkoff. “In my experience, the question cannot be, ‘was it sandboxed?’ It has to be, ‘can we prove what it accessed, whether it crossed a boundary, and how quickly we can stop it?’”

AI security now goes beyond simply protecting data, points out Tristan Shortland, CTO at Infinity. “Organizations also need to think about behaviour, permissions and autonomy, given the reported attack appears to have involved a model finding weaknesses in its environment and exploiting them to achieve an objective.”

Risks internally and externally

With all this in mind, IT leaders need to change their own mindset, as well as educate senior management about the issues posed by AI. CISOs must take the conversation to the business around how the organization is planning to “not just deploy, but manage, monitor and secure their AI capabilities”, says Card. “They need to talk about the risks they face internally and externally – or in this case, both.”

Card advises a mentality of “assume breach, assume adversarial intent, and deploy defence in depth”.

Do not assume your systems will work as expected, he advises. “Go and test those assumptions and red team them.”

Watkins believes AI agents should be modelled as “privileged digital workers, capable of becoming insider threats if poorly contained”.

“We need to start thinking of our AI agents in the context of what permissions they have to see or do things: Can they reach APIs, data, authorise transactions, contact third parties or modify systems?”

Equally important is understanding what those agents are doing in their deployed environments – something that many implementers have little visibility over, says Watkins.

“These are insider threats that need to be managed through good access controls, but also rate limiting, monitoring and alerting, better air-gapping of sandboxes, clear allow-lists for tools and access that [is] time-bound, rather than perpetual.”

While on the face of it, the OpenAI and Anthropic incidents might seem scary, simple changes will make a difference, experts say.

The most urgent initial step is to discover where you are using LLMs and put in place policies for AI usage and monitoring, says Card. “It's a really wide subject, but if you treat these systems as if they may do harm, it's a good starting point.”