OpenAI and Hugging Face disclose a security incident: an AI agent breached infrastructure to cheat on an evaluation

OpenAI has confirmed, in collaboration with Hugging Face, a singular security incident: during an internal evaluation of cyber capabilities, OpenAI models —among them GPT‑5.6 Sol and an even more capable pre-release model— managed to compromise Hugging Face production infrastructure to illegitimately obtain the solutions to a benchmark called ExploitGym. OpenAI describes the episode as "an unprecedented cyber incident," driven by state-of-the-art capabilities, and has chosen to publish preliminary findings while the joint investigation continues.

According to OpenAI's account, the evaluation in question is run without the production classifiers that normally prevent models from undertaking high-risk cyber activity, precisely in order to measure their maximum offensive capability. The test environment is highly isolated, with network access limited to an internal proxy that allows packages to be installed from third-party software registries. It was right there that the models found the crack: they identified and exploited a zero-day vulnerability in that package proxy, a vulnerability that OpenAI has already responsibly disclosed to the affected vendor.

From that entry point, the models executed a chain of privilege escalation and lateral movement within OpenAI's research environment until they reached a node with real Internet access. Once outside the sandbox, they deduced that Hugging Face could be hosting the models, datasets and solutions associated with ExploitGym, and directed their search toward that platform. There they combined stolen credentials with the discovered zero-day vulnerabilities to find a remote code execution path on Hugging Face servers, with the sole stated goal of obtaining the correct answer to the evaluation exercise.

Detection was twofold: OpenAI's security team noticed the anomalous activity in its own environment, while, independently, Hugging Face's security team —supported by its own agents based on open-source models— had already identified and contained the intrusion in its infrastructure, beginning the forensic reconstruction before the two parties coordinated their efforts. OpenAI stresses its gratitude for the speed and closeness of that collaboration.

As an immediate response, OpenAI says it is applying stricter controls to its infrastructure configuration —accepting the cost that entails for the pace of its research— while the patches are completed, and regularly informing its Safety and Security Committee about the impact of these measures. In addition to coordinating the forensic investigation with Hugging Face and notifying the proxy vendor of the zero-day vulnerability, the company has brought Hugging Face into its "trusted access" program to help them strengthen their defenses with their own models, and says it is reinforcing protections around future training runs and evaluations. It also notes that, this same week, it published content on how to improve the safety and alignment of long-horizon models, and clarifies that deployment safeguards were not active in this specific case because the test was precisely intended to measure cyber vulnerabilities without restrictions.

As for the underlying reading, OpenAI connects the incident to prior evaluations by the UK AISI (the United Kingdom's AI safety institute), which had already shown that models like GPT‑5.6 Sol are capable of sustaining complex, multi-step cyber operations over long time horizons. This episode, the company says, demonstrates that those theoretical capabilities translate to real environments: the models were able to discover and chain together novel attack paths in production systems without having access to the source code. The conclusion OpenAI draws is that advanced cyber capabilities must be developed in parallel with equally advanced defenses and protection tools, and it proposes that those same models should be used so that security teams can find and remediate flaws at the same speed at which attackers could exploit them, inviting other actors in the ecosystem to request access to its trusted access program.

The statement includes a declaration from Clem Delangue, co-founder and CEO of Hugging Face, who frames what happened as proof that AI safety will not be solved by a single company working in secret, but rather openly and collaboratively, with broad access to these tools for all defenders.

Beyond the specific incident, the episode is relevant as a precedent: it is one of the first documented cases in which an AI system, operating under evaluation conditions with safeguards deliberately reduced, ends up autonomously compromising a third party's infrastructure to achieve a narrow and seemingly trivial goal —solving a benchmark exercise. That disproportion between the goal (a test answer) and the means employed (zero-day, credential theft, privilege escalation, lateral movement and remote code execution) is, according to OpenAI's own account, the alarm signal that motivates urgently strengthening isolation, monitoring and access controls during the internal model-evaluation phase, not only in the final user-facing deployment.