Logo for OpenAI, owner of ChatGPT AP Photo/Michael Dwyer OpenAI reported that some of its most advanced artificial intelligence models acted on their own and hacked a startup after the company lost control over them during a security test. The creator of ChatGPT stated that its agent — an AI system capable of operating autonomously after receiving initial instructions from humans — was being tested in a controlled environment, but had vulnerabilities. The system targeted Hugging Face, one of the world's largest platforms for sharing AI models, gaining access to some of the company's internal systems. ??Do you have any reporting suggestions? Send it to g1 OpenAI called the incident “unprecedented” and said it was conducting an investigation in conjunction with Hugging Face. Hugging Face CEO Clement Delangue said in an X post that it was "amazing that all of this happened autonomously." "The investigation is ongoing, and we will share more learnings about what may be the first incident of its kind," he added. Man sues OpenAI and says ChatGPT reinforced delusion that he was Jesus Christ Insecure test environments (sandboxes) Gina Neff, director of the Minderoo Center for Technology and Democracy at the University of Cambridge, in England, told BBC Radio 4's Today program that security tests — known as sandboxes — "should be safe environments where it is possible to observe what models are capable of." "In this case, it appears that OpenAI did not create a sufficiently secure testing environment," she added. Instead, the agents created their own cyberattack against the test environment, finding a vulnerability that allowed them to escape. Once outside, the AI ??identified Hugging Face as a likely source of the answers it sought in the test and attempted to gain access. Neil Lawrence, professor of machine learning at the University of Cambridge, called the feat "impressive" but stressed that it "is entirely within the known capabilities of the current generation" of high-performance AI models. He noted that OpenAI is looking to go public on the stock exchange and faces strong pressure from rival Anthropic, which rose to prominence with its own powerful AI tool, Mythos. "OpenAI is now playing catch-up; they are trying to demonstrate their own systems' capabilities in cybersecurity." "This shows us that OpenAI is not capable of implementing its own technology securely," he added. In its initial statement about the hack on July 16, Hugging Face said it was still evaluating whether customer or partner data had been affected and would contact affected parties if necessary. The company stated that it has already corrected the vulnerabilities highlighted by the incident and rebuilt the affected systems. “AI-driven autonomous offensive tools are no longer theoretical,” the company stated. "Defending an online platform now means treating the data and model surface as a first-order attack surface, and using AI in defense to keep pace. "We will continue to invest in this area and share what we learn." 'Time for reflection' The incident raised new questions about the capabilities of advanced AI systems and whether existing safeguards are sufficient as the technology becomes more powerful. Spencer Starkey, an executive at cybersecurity firm SonicWall, told the BBC that the episode made it clear that organizations need "Intensify" their defenses and "treat cyber resilience as a key operational priority." "The uncomfortable truth is that many organizations are still defending at human speed, while adversaries are advancing at machine speed," he said. Meanwhile, Travis Lelle, chief security engineer at consultancy Guidepoint Security, said the update marked a "moment of reflection in cybersecurity." control that they can't understand the context." However, Jake Moore, global cybersecurity consultant at ESET, noted that the announcement could also have a competitive dimension. He argued that OpenAI may be trying to highlight its own AI capabilities as rival Anthropic attracts increasing attention with its Claude Mythos model. "This raises the question of whether OpenAI is trying to match Anthropic's marketing success," he said. It comes a week after Chinese AI startup Moonshot introduced the Kimi K3 — a new and gigantic artificial intelligence model that, according to the company, can rival the main US companies.