SWE-bench Verified on Claude Opus 5.5
500 tasks, 1 attempt, allowlisted network. Updated 19 Oct 2026.
Real GitHub issues, resolved against the projects' own test suites. Comparable with published full-set numbers, but scores this high on a public set may reflect contamination: read them as harness differences on one model, not as the model's skill. About 5-7 hours and about $80 per run. Bring your own Anthropic key.
Runs
Every published run on this board, each with its own pass rate over the board's tasks, beside its configuration's pooled score.
Every run's pass rate
Pass rate, higher is better.
HarnessRuns, 0 to 100%PooledRuns