Skip to content

Policy

Apollo to publish two reports from four-month OpenAI safety review

Edited and reviewed by AI Ping Editorial Team

3-line summary

  1. Six Apollo Research staff will examine model training and internal use at OpenAI. The pilot is part of OpenAI’s first round of on-site assessments with four outside groups.
  2. Apollo will publish its conclusions and account for access limits. OpenAI can request redactions of confidential evidence, and a committee of the foundation board will receive the reports in full.
  3. Transluce plans a separate six-week review starting in early December. It can inspect internal-only models, with its public findings limited to models already released.
PracticalOffice workersDevelopers
An open binder and a red file on a conference table seen through a doorway
AI-generated conceptual image: A doorway and binder in an imagined room represent outside researchers reviewing material on site. · AI Ping

Apollo Research said on November 16 that six of its staff would work inside OpenAI for four months and publish two reports on model safety. It is one of four outside groups taking part in OpenAI’s first round of on-site assessments. Apollo plans to examine whether models act against their safety controls during development or internal use, and whether OpenAI’s monitoring would detect that behavior.

The reports will give Apollo’s conclusions and describe limits on what it could examine. OpenAI can ask for confidential evidence to be removed, while the OpenAI Foundation’s Safety and Security Committee will receive the full reports. Teams assessing AI systems will be able to read the findings alongside an account of what their authors could see.

Looking at models during training

Apollo’s earlier evaluations generally took place after additional training had finished, shortly before models went public. The new pilot reaches back into training. For the first part, Apollo can examine saved versions of models and records from those stages. It will look at whether the way models were rewarded during training encouraged them to act against controls. Detailed access for the other parts still has to be agreed.

Access is tied to the models, dates and questions in the agreement. OpenAI can restrict records even within that scope when it considers that necessary for technical, security or intellectual-property reasons. Apollo will be told when information is withheld and will receive a written explanation. It can seek more access and escalate a refusal.

Apollo may suggest whom to interview, with OpenAI making the selection. Interviews can inform the work. To publish a claim learned from a staff conversation, Apollo has to substantiate it independently or get formal confirmation from OpenAI. The group can report that OpenAI declined to confirm a statement and explain how that affected its assessment.

Blue folders with abstract traces beside a notebook and glasses
AI-generated conceptual image: Blue files and a notebook represent training records and interview material that Apollo can review under the agreement. · AI Ping

The public findings and the underlying evidence

Apollo’s first report will cover the incentives created during training. The second will examine safety tests, internal use and whether signs of behavior that undermines controls could be detected. Both will describe the methods and limitations, questions the team could not assess, and supporting material that it can make public.

OpenAI will review the reports before publication. Apollo says the agreement protects its verdict on each claim and bars redactions based simply on an unfavorable result. Some supporting evidence can still be removed for confidentiality. Apollo can explain how such omissions affected the conclusions. It can also discuss the terms governing its access and disclosure without first submitting that account to OpenAI.

The unredacted reports go to the safety committee of the nonprofit foundation that controls OpenAI. The company says assessors and committee members can meet privately during the work. The committee can settle disagreements unresolved by the company and the evaluators. OpenAI can also request a delay to address a safety or security issue before findings go public.

A slim open report beside a thick file packet tied with a band
AI-generated conceptual image: An open report and a bound packet represent the published findings and the full reports sent to the foundation committee. · AI Ping

What the other teams plan to report

Transluce plans to spend six weeks starting in early December evaluating the tests OpenAI uses after training. It will be able to examine at least one internal-only model, with its public report confined to results about released models. Transluce says it will document its working conditions under an AI Evaluator Forum standard. The standard calls for reporting unmet conditions and the reasons alongside the findings.

The arrangements for paying the teams also differ. OpenAI says Apollo and Goodfire receive compensation that does not depend on their findings. Transluce says it accepts no fee, with OpenAI covering computing costs and providing equipment. Goodfire will study signals inside models during training. It will share its findings chiefly with OpenAI researchers and plans to publish some of what it learns as the work progresses.

OpenAI says METR will examine its largest reinforcement-learning run for problems existing tests may have missed. The scope of a separate study of OpenAI’s monitoring still needs to be agreed. OpenAI and Transluce intend to run a follow-up of roughly eight weeks in early 2027 with broader access. The terms of that next round are still being worked out.

Sources