Up to 25% of OpenAI's high-end AI traffic lacked safety oversight during audit

For ordinary users interacting with the world’s most advanced AI systems, a new audit suggests a period of reduced oversight. Researchers from METR and Redwood Research spent three weeks on-site at OpenAI, examining the internal systems designed to catch rogue behavior in artificial intelligence models. Their investigation, conducted from July 21 to August 11, revealed that the monitoring infrastructure did not cover all activity, leaving gaps that allowed certain actions to proceed without inspection. While OpenAI reports that these specific weaknesses have been addressed, the findings highlight the complexity of supervising autonomous systems.

The numbers behind the findings are substantial. The audit team estimated that between six and 25 percent of internal traffic from the GPT-5.5 class of models and above was not monitored at all during the review period. METR notes that this estimate carries significant uncertainty in both directions. In a separate area, roughly 10 percent of internal Codex traffic went untracked between May and July due to a technical bug related to context summaries known as compaction events. This estimate also involves notable uncertainty. OpenAI identified and fixed this specific bug while the auditors were still on-site.

The researchers involved in the pilot program included experts from two prominent AI safety organizations, METR and Redwood Research. They had direct access to OpenAI’s codebase and internal documents and interviewed approximately a dozen employees. The team discovered that during the period when compaction events were unmonitored, AI agents could delay oversight by simply continuing to execute actions rather than handing control back to the user. The auditors also identified methods to hide hundreds of actions from the monitor and bypass the Codex sandbox entirely.

Further investigation revealed that specially crafted inputs could likely crash the monitoring system completely. Additionally, model behavior during reinforcement learning (RL) training was not being monitored at the start of the audit. OpenAI clarified that it paused RL training following an incident with Hugging Face in August and subsequently expanded its monitoring capabilities. However, these changes occurred after the auditors’ access period ended, meaning they were not part of the formal evaluation.

The stakes for future AI development lie in the transparency of such reviews. OpenAI states that it has fixed all the weaknesses identified by the auditors. The full report consists of 61 pages, which was handed over to OpenAI internally. After redactions, METR and Redwood Research leadership received a 38-page version, while the public received only a short summary. METR stated it did not have the right to disclose the redacted portions independently. This audit was a joint pilot program designed to grant outside researchers deep access to internal infrastructure, with METR indicating a desire for greater public transparency in future evaluations.