humaneval-plus@1.0 (unreviewed) on GPT-6.1 Sol
164 tasks, 1 attempt, open network. Updated 24 Oct 2026.
164 tasks from humaneval-plus@1.0 on ghcr.io. Nobody has reviewed this dataset, so a score here says nothing about the benchmark it's named after.
Compare
Pick two harnesses to compare them task by task.
Claude Code 2.1.309 against DeepSeek Harness 0.2.1-rc.2, task by task
Over the 164 tasks both ran, 1 attempt each
Claude Code 2.1.309 minus DeepSeek Harness 0.2.1-rc.2, pass rate
−0.6 points
95% interval −1.8 to 0.0
The interval includes zero, so the paired test can't tell the two apart.
Where the tasks landed
- 0Only Claude Code 2.1.309 solved
- 157Both solved
- 1Only DeepSeek Harness 0.2.1-rc.2 solved
- 6Neither solved
Of 164 tasks, 0 only Claude Code 2.1.309 solved, 1 only DeepSeek Harness 0.2.1-rc.2 solved, 157 both solved and 6 neither did.
| Task | Reward | Tokens | Time | |||
|---|---|---|---|---|---|---|
| Claude Code 2.1.309 | DeepSeek Harness 0.2.1-rc.2 | Claude Code 2.1.309 | DeepSeek Harness 0.2.1-rc.2 | Claude Code 2.1.309 | DeepSeek Harness 0.2.1-rc.2 | |
humaneval-plus-34 | 0 | 1 | 40k | 19k | 22 s | 27 s |