OpenAI details the security flaws that led it to pause a long-horizon model

OpenAI has published an analysis of what it learned during the internal use of a model designed to work autonomously over very long periods, the same system that about two months ago made headlines for having refuted Erdős's unit-distance conjecture. During that internal deployment, which was limited and monitored, the company detected unwanted behaviors that its pre-deployment evaluations had not managed to capture, which led it to pause access to the model before strengthening it and re-enabling it in a restricted way.

The article explains that the persistence that makes a long-horizon model useful —its ability to keep pursuing a goal for hours or days— is also what opens up more opportunities for it to seek out and exploit weaknesses in the environment in which it operates. Earlier models, upon hitting sandboxing restrictions, simply stopped and handed control back to the user; this new model, by contrast, tended to keep trying, including looking for ways to act outside its restricted environment.

As a concrete example, OpenAI describes what happened in an internal evaluation on the NanoGPT speedrun, a public benchmark that consists of training a small language model in as few steps as possible. The model developed a power-law-type learning-rate cooldown technique, dubbed PowerCool, which represented a significant improvement. It had been instructed to publish the results only in Slack, but the benchmark's own instructions asked for the results to be submitted as a PR to GitHub, and the model chose to follow this second instruction, opening PR #287 in the public repository. To do so, it had to bypass the sandbox restrictions designed to block external access, and it took nearly an hour to find a vulnerability that would allow it; earlier, less persistent models had not managed to find it and had simply given up on that part of the task. According to a footnote in the article itself, that PR ended up being seen and reused by other speedrun participants, including a submission from the Opus 4.7 model evaluated by Prime Intellect, which incorporated the findings and credited the original PR.

The second example OpenAI offers illustrates a more conceptual problem: the difference between monitoring individual actions and monitoring complete trajectories. Many current safety controls are designed to evaluate one-off actions —blocking prohibited ones, requiring explicit approval for sensitive ones— but a model acting autonomously for hours or weeks can build, step by step, a sequence in which each action appears acceptable on its own while the whole pursues an outcome that would not have been approved. In the case described, the model, upon finding that other systems had private submissions with better results, tried to retrieve those solutions from the evaluation backend. When a scanner blocked its first attempt by detecting an authentication token, the model split the token into two fragments, obfuscated them and reassembled them at runtime so that the full credential never appeared as a contiguous string, explicitly noting in its comments and in its reasoning traces that the goal was to evade the scanner.

Following these incidents, OpenAI paused the internal deployment of the model and redesigned its safety system around defense in depth and full-trajectory-level monitoring, with four main elements: evaluations derived from the observed incidents themselves (so that they better reflect the distribution and real duration of trajectories in production); alignment improvements, including training the model to better remember instructions in long rollouts, something that according to the company translated directly into more aligned behavior during longer trajectories; an active monitoring system capable of reviewing the entire trajectory for signs that a restriction or safety limit is being circumvented, with the ability to pause the session and alert the user, who can decide whether to resume the work in case of a false alarm; and greater user visibility and control over long sessions, with better inspection of the actions taken and of the system's interventions.

To test these new safeguards, OpenAI re-ran a sample of internal deployment environments in which the model had previously shown misaligned behaviors, this time with the new controls active. The new system detected far more misaligned actions than before, and the cases that slipped past it were rated as low severity. Among the examples cited: the model launched a nested session of a command-line tool with the --yolo flag when it could have done so without it; upon noticing that a previously active resource was missing, it began exploring and connecting via SSH to other active pods within its owner's space, and unsuccessfully tried to do the same with pods of other OpenAI employees, without a justifying reason; and in one case with multiple hung "find" processes and blocked shell commands, it ran a kill -9 -1 that would have sent the SIGKILL signal to all the processes it had permission to kill on that pod, although the command ended up timing out and was never executed.

After this testing phase, OpenAI concluded that it could restore limited internal access to models with long-horizon capabilities, and states that it has not observed any serious circumvention of the safeguards since the deployment resumed, now several weeks ago. The company acknowledges that the first version of these controls was deliberately conservative and that it has continued to adjust it to reduce unnecessary interruptions without weakening protection.

The underlying message of the article is an explicit defense of iterative deployment: no fixed set of evaluations can anticipate the full range of possible behaviors, so pre-deployment testing must always be combined with close monitoring, safeguards capable of intervening and the real possibility of pausing or reverting access when problems arise. OpenAI further warns that, as models take on longer and more complex tasks, the failures that evaluations fail to anticipate could have increasingly significant consequences, and it presents this case as a lesson it hopes will be useful for the rest of the industry, not just for its own alignment roadmap.