Opus 5.5 leads new GDPx agent benchmark with a 12% overall score

Claude Opus 5.5 leads GDPx, a new evaluation of AI agents doing specialist work, with an overall score of 12.0%. SemiAnalysis released the first version on November 2, covering 206 tasks across 11 domains with 14 data partners. Gemini 4 Carbon scored 10.9% and GPT-6 Astra 9.6%. The release gives teams comparing agents a view of performance inside task environments alongside an overall ranking.

SemiAnalysis sets a default eight-hour limit and reports results across three attempts. Domains receive equal weight, while tasks use either binary or partial grading. The headline percentage combines these scores; it is not a simple count of successfully completed everyday jobs.

A chip design must survive verification

Aria’s public task gives an agent eight hours to design and physically implement a small neural-network accelerator. Correct calculations, timing, wiring and power constraints all matter. The task uses an academic 45-nanometer process and a flow taking roughly ten minutes on 16 CPUs, allowing repeated design iterations.

The published instruction requires preserving external ports and matching the reference’s outputs. The machine has no network; documentation is supplied locally. Deliverables include circuit code and a routed layout. Building, checking and energy measurements must finish within 100 minutes on 16 CPUs.

A failure at synthesis, functional testing, physical sign-off or simulation earns zero. Passing designs are compared with a reference through their energy-delay-area product. Aria reported 0.392 for Opus 5.5’s best design, whose product was 2.6 times the reference. No passing AI design matched the reference’s energy efficiency.

Transaction files and failed training runs

The public finance task instead asks an agent to finish an acquisition target’s quality-of-earnings workbook. The domain page describes a roughly 4,000-file replica of a real private-equity transaction. The agent has eight hours, two CPUs and 8 GiB of memory to correct existing work while preserving its accurate entries.

The assignment includes earnings adjustments, month-end working capital and net debt. It requires consistent accounting definitions, preservation of workbook labels, units and signs, and notes supporting each figure. The public run list records zero for each of Opus 5.5’s three attempts on this particular task.

For AI research, a published task asks an agent to audit a reinforcement-learning run whose scores have stopped improving. It must compare preserved rewards, evaluations, traces and historical code, reconstruct the timeline and test competing explanations. Demonstrable implementation defects belong in the final report.

That assignment allows five hours on four CPUs with 16 GiB of memory. Proposed repairs need validation results, with remaining uncertainty documented. Opus 5.5’s three listed attempts scored 0.4, 0.2 and 0.2. These partial scores illustrate another grading approach within the benchmark.

SemiAnalysis says excluding cybersecurity, biology and alignment—where safety filters can intervene—changes the ranking. The published results therefore incorporate the effects of execution tools and safeguards alongside task capability.

Google’s Sundar Pichai welcomed the benchmark’s headroom and Carbon’s cost result in a November 2 post. GDPx provides domain comparisons and public execution records as well as its overall table. SemiAnalysis plans to release new versions every three to six months.