Apollo to publish two reports from four-month OpenAI safety review
Edited and reviewed by AI Ping Editorial Team
3-line summary
- Six Apollo Research staff will examine model training and internal use at OpenAI. The pilot is part of OpenAI’s first round of on-site assessments with four outside groups.
- Apollo will publish its conclusions and account for access limits. OpenAI can request redactions of confidential evidence, and a committee of the foundation board will receive the reports in full.
- Transluce plans a separate six-week review starting in early December. It can inspect internal-only models, with its public findings limited to models already released.
Apollo Research said on November 16 that six of its staff would work inside OpenAI for four months and publish two reports on model safety. It is one of four outside groups taking part in OpenAI’s first round of on-site assessments. Apollo plans to examine whether models act against their safety controls during development or internal use, and whether OpenAI’s monitoring would detect that behavior.
The reports will give Apollo’s conclusions and describe limits on what it could examine. OpenAI can ask for confidential evidence to be removed, while the OpenAI Foundation’s Safety and Security Committee will receive the full reports. Teams assessing AI systems will be able to read the findings alongside an account of what their authors could see.
Looking at models during training
Apollo’s earlier evaluations generally took place after additional training had finished, shortly before models went public. The new pilot reaches back into training. For the first part, Apollo can examine saved versions of models and records from those stages. It will look at whether the way models were rewarded during training encouraged them to act against controls. Detailed access for the other parts still has to be agreed.
Access is tied to the models, dates and questions in the agreement. OpenAI can restrict records even within that scope when it considers that necessary for technical, security or intellectual-property reasons. Apollo will be told when information is withheld and will receive a written explanation. It can seek more access and escalate a refusal.
Apollo may suggest whom to interview, with OpenAI making the selection. Interviews can inform the work. To publish a claim learned from a staff conversation, Apollo has to substantiate it independently or get formal confirmation from OpenAI. The group can report that OpenAI declined to confirm a statement and explain how that affected its assessment.
The public findings and the underlying evidence
Apollo’s first report will cover the incentives created during training. The second will examine safety tests, internal use and whether signs of behavior that undermines controls could be detected. Both will describe the methods and limitations, questions the team could not assess, and supporting material that it can make public.
OpenAI will review the reports before publication. Apollo says the agreement protects its verdict on each claim and bars redactions based simply on an unfavorable result. Some supporting evidence can still be removed for confidentiality. Apollo can explain how such omissions affected the conclusions. It can also discuss the terms governing its access and disclosure without first submitting that account to OpenAI.
The unredacted reports go to the safety committee of the nonprofit foundation that controls OpenAI. The company says assessors and committee members can meet privately during the work. The committee can settle disagreements unresolved by the company and the evaluators. OpenAI can also request a delay to address a safety or security issue before findings go public.
What the other teams plan to report
Transluce plans to spend six weeks starting in early December evaluating the tests OpenAI uses after training. It will be able to examine at least one internal-only model, with its public report confined to results about released models. Transluce says it will document its working conditions under an AI Evaluator Forum standard. The standard calls for reporting unmet conditions and the reasons alongside the findings.
The arrangements for paying the teams also differ. OpenAI says Apollo and Goodfire receive compensation that does not depend on their findings. Transluce says it accepts no fee, with OpenAI covering computing costs and providing equipment. Goodfire will study signals inside models during training. It will share its findings chiefly with OpenAI researchers and plans to publish some of what it learns as the work progresses.
OpenAI says METR will examine its largest reinforcement-learning run for problems existing tests may have missed. The scope of a separate study of OpenAI’s monitoring still needs to be agreed. OpenAI and Transluce intend to run a follow-up of roughly eight weeks in early 2027 with broader access. The terms of that next round are still being worked out.
Sources
- 오픈AI 현장 평가 시범사업의 접근·공개 조건 · A Pilot Towards Embedded Evaluations — Apollo Research
- 외부 4곳의 1차 현장 평가 · Our first phase of embedded assessments — OpenAI
- 현장 안전성 평가의 검증 질문 · Towards embedded evaluations for scheming propensities — Apollo Research
- 재단과 이사회 감독 구조 · Our structure — OpenAI
- 6주 평가와 공개 조건 · Announcing an Embedded Assessment Pilot with OpenAI — Transluce
- 학습 중 모델 검토 협업 · Announcing our collaboration with OpenAI — Goodfire
- 평가 작업 조건을 공개하는 기준 · AEF-1 Minimum Operating Conditions — AI Evaluator Forum
