"Unprecedented" AI Breach: OpenAI's New Model ‘Cheats’ on Benchmark Test by Gaining Access to Open Internet and Hacking Into Hugging Face System

AN OPENAI artificial intelligence model has raised serious cybersecurity concerns during a routine benchmark test. The tech company recently ran a test on its latest most advanced AI model to check its ability to solve cyber security challenges. Although, OpenAI claims that the model was placed in a secured test environment, the AI model broke out of it on its own without anyone giving it that command.

The new model found a hidden bug in the software, gained access to the real internet, and hacked into Hugging Face—an open-source platform that stores various AI models and data. The incident has raised fresh concerns over how advanced AI systems act when given complex tasks. According to OpenAI and Hugging Face, the AI models were able to bypass restrictions, exploit previously unknown software flaws, and carry out more than 17,000 actions against Hugging Face's network before the breach was contained.

Open AI Model Escaped Secured Test Environment in “Unprecedented” Incident

The incident happened during testing on ExploitGym, a cybersecurity benchmark designed to evaluate AI capabilities in offensive security tasks. OpenAI's GPT-5.6 Sol model and another pre-release model were placed inside an isolated environment without internet access, while some of their safety restrictions had been reduced for research purposes. However, instead of remaining confined, the AI models reportedly discovered a previously unknown bug in the testing stage and used it to breach into open internet access.

Once online, the AI models identified Hugging Face as a platform that could contain information relevant to the benchmark test. According to OpenAI, the systems then used stolen login credentials along with another vulnerability in Hugging Face's software to enter the company's production systems.

See Also: India Pushes Ahead in AI Race with Massive Data Centers, But Andhra Pradesh Pays Price with Water, Energy, and Uncertain Jobs

The models reportedly conducted over 17,000 automated attacks on the Hugging Face system from multiple IP addresses within a short period while searching for information that would help it ‘cheat’ in the benchmark test. Hugging Face detected unusual activity in mid-July and quickly contained the breach. OpenAI later informed the company that its AI models had been responsible for the attack.

The company has not yet released a full technical report explaining exactly how the breach occurred or identified the pre-release model involved. Researchers say the case is significant because the AI did not simply solve a cybersecurity challenge. It reportedly adapted its strategy, escaped its restricted environment, searched the internet, located a relevant external platform, and chained together multiple attack methods to achieve its assigned objective—all of it without any human command.

Both OpenAI and Hugging Face announced that they are now jointly investigating the incident and have begun sharing details of the vulnerabilities with cybersecurity researchers to help prevent similar attacks.

Hugging Face Co-Founder Calls the Breach "A Wake-Up Call" for the AI Industry

Speaking to BBC, Hugging Face co-founder and Chief Science Officer Thomas Wolf described the incident as "a wake-up call" for the AI industry. According to him, the attack looked very different from the cyberattacks his company normally faces. He said Hugging Face initially had no idea who was behind the intrusion until OpenAI contacted them.

Wolf warned that AI-powered cyberattacks could become one of the most common forms of digital threats, arguing that many companies still do not realise "the game has changed."

Hugging Face is one of the world's largest open-source AI platforms, hosting thousands of AI models and datasets used by researchers, developers, and companies across the world, making it a prime target for cyber threats. Despite the breach, there are currently no reports suggesting large-scale data loss or long-term compromise of user information.

Suggested Reading: