AI leaderboard changes its exam: 60% private or held-out, Opus 5.5 leads

Artificial Analysis changed the exam behind its AI leaderboard on October 27, releasing Intelligence Index v5. Private test sets or held-out answers now underpin 60% of the index. Claude Opus 5.5 at max reasoning effort leads with 58 points, ahead of GPT-6 Astra at max with 57.

Nine evaluations cover knowledge work, coding, science and general capability, each category carrying 25%. Documents, software changes and research outputs contribute alongside knowledge and reasoning questions. These weights determine how different kinds of work feed into the overall comparison.

Kimi K3.1 Pro scores 46, leading Artificial Analysis’s open-weight category despite its unreleased weights. Among downloadable models, DeepSeek V4.1 Flash 1017 and GLM-5.3 lead at a rounded 32 each at max effort. The announcement distinguishes promised releases from available weights.

Testing whether a code repair fits the project

Newly included FrontierCode 2.0 asks whether a repository maintainer would accept an agent’s change. Its creator, Cognition, evaluates how models interpret informal requests while respecting existing code conventions and design. The expanded suite covers security patches and performance work, with an emphasis on complete, maintainable repairs.

Failing a blocking criterion gives an attempt zero credit. A repair may need to fix the same bug in a sibling code path as well as the one mentioned in the request. Cognition also audited reference solutions: in 80% of the new tasks, these differ from upstream patches.

The original June 8 methodology, retained by the revision, explains how agent-written tests are checked against the original broken code: they must fail there. Language-model review also assesses readability and design conventions. The grading combines executable checks with maintainer-defined quality criteria.

The science addition is Terminal-Bench-Science 0.1, an existing benchmark released on August 27 by Stanford researchers and domain experts. Its 70 tasks span life, physical, earth, mathematical and engineering sciences. They assess research workflows through verifiable artifacts, including analyses, simulations, proofs and code, rather than relying entirely on written answers.

Tasks include sensor calibration, signal processing, statistical inference and model fitting. Researchers propose workflows, then domain and technical reviewers examine scientific validity, implementation and verification. The first release selected 70 tasks from 920 proposals. Creators emphasize objectively testable work grounded in scientific practice.

The Intelligence Index primarily evaluates English text. Image inputs, speech inputs and multilingual capability are measured separately and excluded from its composite score. The methodology discloses task counts, repetitions and tool use. Command-driven research workflows and individual question answers pass through different grading procedures within the same overall index.

Artificial Analysis also launched Progress Index v1 for comparisons over time. Intelligence Index composition changes as easier evaluations retire and harder ones arrive. Its methodology says scores across versions are not intended for long-run comparisons: the exam changes the basis of the number.

This index jointly estimates model capability and task difficulty, connecting systems through shared tasks. Older and newer models thus share one scale despite different test coverage. Models need results on four benchmarks to enter. Artificial Analysis assigns current comparisons to the Intelligence Index and longer-term analysis to the Progress Index.