SWE-bench Pro

A more challenging benchmark for coding agents

A Scale AI coding-agent evaluation benchmark and public leaderboard based on expert-validated real-world software-engineering tasks.

Recent stories

1 linked story

newsPRIMARY2026-07-08

OpenAI audits SWE-Bench Pro and finds 30% of public tasks broken

OpenAI audited SWE-Bench Pro and found 30% of public tasks were broken. It retracted its earlier recommendation to use the benchmark as a leading coding eval.