humaneval-plus@1.0 (unreviewed) on GPT-5.6 Terra
164 tasks, 1 attempt, open network. Updated 24 Oct 2026.
164 tasks from humaneval-plus@1.0 on ghcr.io. Nobody has reviewed this dataset, so a score here says nothing about the benchmark it's named after.
Compare
Pick two harnesses to compare them task by task.
Claude Code 2.1.309 against Codex 0.164.0, task by task
Over the 164 tasks both ran, 1 attempt each
Claude Code 2.1.309 minus Codex 0.164.0, pass rate
−1.8 points
95% interval −4.9 to +0.6
The interval includes zero, so the paired test can't tell the two apart.
Where the tasks landed
- 1Only Claude Code 2.1.309 solved
- 153Both solved
- 4Only Codex 0.164.0 solved
- 6Neither solved
Of 164 tasks, 1 only Claude Code 2.1.309 solved, 4 only Codex 0.164.0 solved, 153 both solved and 6 neither did.
| Task | Reward | Tokens | Time | |||
|---|---|---|---|---|---|---|
| Claude Code 2.1.309 | Codex 0.164.0 | Claude Code 2.1.309 | Codex 0.164.0 | Claude Code 2.1.309 | Codex 0.164.0 | |
humaneval-plus-103 | 0 | 1 | 30k | 64k | 19 s | 28 s |
humaneval-plus-116 | 1 | 0 | 32k | 83k | 41 s | 1.4 min |
humaneval-plus-145 | 0 | 1 | 30k | 55k | 19 s | 1.0 min |
humaneval-plus-76 | 0 | 1 | 45k | 57k | 1.0 min | 1.5 min |
humaneval-plus-80 | 0 | 1 | 30k | 50k | 16 s | 15 s |