Skip to content
Hbench
hbench

SWE-bench Verified on Claude Opus 5.5

500 tasks, 1 attempt, allowlisted network. Updated 19 Oct 2026.

Real GitHub issues, resolved against the projects' own test suites. Comparable with published full-set numbers, but scores this high on a public set may reflect contamination: read them as harness differences on one model, not as the model's skill. About 5-7 hours and about $80 per run. Bring your own Anthropic key.

Build a run for this board

Compare

Pick two harnesses to compare them task by task.

Swap A and B

DeepSeek Harness 0.2.0-rc.2 against Claude Code 2.1.302, task by task

Over the 500 tasks both ran, 1 attempt each

Embed this chart

DeepSeek Harness 0.2.0-rc.2 minus Claude Code 2.1.302, pass rate

+2.6 points

95% interval +0.8 to +4.6

The interval leaves out zero, so the paired test tells the two apart.

Where the tasks landed

  • 19Only DeepSeek Harness 0.2.0-rc.2 solved
  • 448Both solved
  • 6Only Claude Code 2.1.302 solved
  • 27Neither solved

Per-task results of runs #17 and #18 stay private until their owner agrees, so only these totals show and the task list is hidden. What a board makes public

Of 500 tasks, 19 only DeepSeek Harness 0.2.0-rc.2 solved, 6 only Claude Code 2.1.302 solved, 448 both solved and 27 neither did.