SemiAnalysis releases GDPx benchmark; Opus 5.5 leads at 12%

• Opus 5.5 leads GDPx at 12.0%, ahead of Gemini 4 Carbon’s 10.9% in the launch leaderboard.
• GDPx targets sectors expected to generate high returns from frontier intelligence, following SemiAnalysis_’s explicitly selective economic thesis.
• Removing three safety-sensitive domains puts GPT-6 Astra first, according to SemiAnalysis_’s alternative ranking.
• Grading is already drawing scrutiny: a model-assisted review questions finance checks and the trading simulator’s cancellation latency.

SemiAnalysis followed a boxing teaser with the GDPx launch, whose finance environment contains roughly 4,000 anonymized deal files, a welcome dose of workplace mess. Its public rollout viewer releases one task’s traces per partner, while an early model-assisted review flags a possible trading-simulator timing error.

Carbon trails Opus by 1.1 points at roughly 32% of its average task cost. SemiAnalysis describes Carbon as not yet generally available.

Muse Wonder Full is absent from the cost-performance plot because it lacks a public price, a comparison gap eliebakouch also noted.

GDPx contains 206 tasks contributed by specialist partners. SemiAnalysis handled final QA and helped iterate the tasks.

The domain inventory breaks down as follows:

• Chip design, 34 tasks: verification, RTL and physical design. Aria, Arcverex and Ergodex.
• Cybersecurity, 18: proprietary zero-days in live, multi-service environments. Arithmetic.
• Biology, 13: experimental analysis and scientific decisions. LatchBio.
• Finance, 24: deal modeling and transaction accounting. Living Context.
• Legal, 20: eight corporate and transactional practice areas. Chakra.
• Quant trading, 10: IEX order-book strategies scored on held-out daily profit and loss. Third Moment.
• Recursive self-improvement, 10: ML research engineering. Extrapolate.
• Hardware design, 21: CAD, CAM, PCB and FPGA work. Beautiful Machines.
• Materials science, 23: simulated laboratory equipment and experimental data. ValenceForge and Emulated.
• Alignment, 18: honesty and authority boundaries under pressure. Fleet.
• Software engineering, 15: long-horizon features, debugging and migrations. Datacurve.

The selection deliberately excludes industries such as government, real estate and insurance:

Fireworks supplied inference for open-source models, while Modal supplied sandbox infrastructure, according to the partner acknowledgments.

The methodology defines the headline score through three choices:

• Attempts: single-attempt performance averaged across three runs.
• Grading: binary or partial-credit scoring, depending on the domain.
• Weighting: equal weight for every domain, then equal weight for partners within multi-partner domains.

Finance and legal produce near-universal zero scores, according to SemiAnalysis.

Ethan Mollick, a Wharton professor, called for human baselines comparable to GDPval’s and argued that zeros conceal economically meaningful differences in partial success.

Models ran in their native harnesses where available:

• Claude Code: Opus 5.5.
• Codex: GPT-6 Astra.
• Antigravity: Gemini 4 Carbon.
• OpenCode: fallback for models without a native harness.

The default task budget is eight hours, with shorter limits for some tasks.

An Antigravity trace showing repeated provider errors and retries.

eliebakouch attributed the retries to safeguards, while the displayed log labels the failures as empty model output. The screenshot records 186 model calls and 48 tool calls over 28 minutes.

SemiAnalysis attributes depressed scores for some models to safety classifiers in cybersecurity, biology and alignment. Excluding those domains moves Astra from third to first and Carbon from second to fifth.

These filtered scores describe a different eight-domain task mix. They do not measure the omitted tasks with safeguards disabled.

SemiAnalysis reports different winners across the task set:

• Opus 5.5: quant trading, hardware design and chip design.
• Gemini 4 Carbon: biology and software engineering.
• GPT-6 Astra: recursive self-improvement and materials science.

Cybersecurity has another leader:

The sample RSI task asks agents to diagnose a broken GRPO training run:

• Evidence: approximately 1,000 rollout traces containing 21.2 million tokens, alongside training results and historical harness code.
• Hidden failure: at 32K tokens, the harness injects a message telling the model to stop exploring and finalize its deliverables.
• Training effect: Extrapolate reports threefold lower mean reward when training on the resulting premature-finalization rollouts rather than clean rollouts.

The task recreates a failure encountered during frontier-model development, according to Extrapolate’s account in the launch post.

Each partner has open-sourced rollouts for one task. The remaining rollouts stay private to prevent hill climbing.

dejavucoder clarified in a follow-up that the benchmark’s lack of an open-source release was the serious objection within an otherwise joking complaint.

Mollick shared a GPT-6 Pro review that questioned two finance assessments:

• Opus’s third attempt: four customer-program adjustments marked missing by the verifier appeared on a supporting tab and flowed through formulas into a grouped adjustment in the main earnings walk.
• Astra’s first attempt: the workbook grouped $825,000 of transaction expenses with net debt, violating the requested classification, although its notes and final response explicitly distinguished the amounts.

The review identified other genuine problems in Opus’s workbook and did not establish that it should have passed overall. It distinguished four failure modes:

1. Never finding an issue.
2. Finding it but interpreting it incorrectly.
3. Understanding it but implementing it incorrectly.
4. Handling it correctly in a form the checker fails to recognize.

The same review alleged an execution-model discrepancy in the trading simulator:

• Order placement: 10 microseconds of network latency plus IEX’s 350-microsecond delay, totaling 360 microseconds.
• Cancellation: 10 microseconds of network latency plus 10 microseconds of processing, totaling 20 microseconds, with the IEX delay omitted.

IEX’s own explanation of its speed bump explicitly applies the 350-microsecond delay to cancellations.

The review says the winning agent added passive market-making, where faster cancellation can help avoid unfavorable fills. Whether that strategy would still pass after correcting cancellation latency remains unresolved in the review.