UK government probes OpenAI breach after ‘unprecedented’ hack

The UK government is probing the first known case of an artificial intelligence model breaking out of a controlled test by itself and hacking another company’s systems. Officials at the government-backed AI Security Institute (AISI) are investigating the security breach at OpenAI and whether similar incidents could occur at other top developers, a spokesperson told City AM. It comes after one of OpenAI’s models found a hidden flaw, broke out of its test environment, and targeted the AI platform Hugging Face to steal answers to a cyber test.

The incident, which OpenAI described as an “unprecedented cyber incident”, marks the first publicly disclosed case of a so-called frontier AI system autonomously hacking into systems outside its test environment on its own to finish a task it was given. A government spokesperson said: “The UK’s AI Security Institute is studying the behaviour seen in this incident – an AI system pursuing goals through unintended and unauthorised means – as part of its world leading efforts to make frontier AI safer. “As AI capabilities evolve, it’s important that everyone steps up their cyber defences, and organisations should take practical steps like Cyber Essentials in order to do so.

AISI continues to work with OpenAI and other labs to better understand AI capabilities and improve safeguards.” The Financial Conduct Authority (FCA) and the EU’s cybersecurity agency Enisa are both monitoring the wider situation to see how different industries could be affected. The AI Security Institute has already published research examining how advanced AI models like OpenAI can reach their goals in unwanted ways.

Officials now see the Hugging Face event as a key real-world example to help guide future work on AI safety. In a blog post, OpenAI chief executive Sam Altman admitted that the model escaped an internal cybersecurity evaluation after engineers deliberately disabled its normal safety guardrails to test its hacking capabilities. Rather than completing