Benchmarks / all boards
Benchmarks
Each row is one benchmark on one board, usually its board with the most harnesses. A board fixes the model, task set, attempts and network, so only the harness changes. Every result is self-reported. Δ is a harness's gap to the board's reference in points, on the tasks both ran: how we measure it.
| Show board | Board | |||||
|---|---|---|---|---|---|---|
| Terminal-Bench 2.0terminal-bench@2.0 | Claude Opus 4.65 attempts, open network | 8 | 8 | 25 Oct 2026 | ||
| SWE-bench Verifiedswebench-verified@1.0 | Claude Opus 5.51 attempt, allowlisted network | 2 | 2 | 19 Oct 2026 | ||
| HumanEvalFixhumanevalfix@1.0 | Qwen2.5-Coder 7Bsample-40, 1 attempt, allowlisted network | 1 | 1 | 18 Oct 2026 | ||
| humaneval-plus@1.0 (unreviewed)ghcr.io | GPT-6.1 Sol1 attempt, open network | 3 | 3 | 24 Oct 2026 | ||
Latest uploads
| Run | Harness | Board | Pass rate | Finished |
|---|---|---|---|---|
| #37 | DeepSeek Harness0.2.1-rc.2 | humaneval-plus@1.0 (unreviewed) on GPT-6.1 Solopen network | 96.3%93.3–98.8 | 24 Oct 2026 |
| #36 | Codex0.164.0 | humaneval-plus@1.0 (unreviewed) on codex-auto-reviewopen network | 94.5%90.9–97.6 | 24 Oct 2026 |
| #35 | Codex0.164.0 | humaneval-plus@1.0 (unreviewed) on GPT-5.6 Terraopen network | 95.7%92.7–98.2 | 24 Oct 2026 |
| #34 | Codex0.164.0 | humaneval-plus@1.0 (unreviewed) on GPT-5.6 Lunaopen network | 95.7%92.7–98.8 | 24 Oct 2026 |
| #33 | Codex0.164.0 | humaneval-plus@1.0 (unreviewed) on GPT-6 Astraopen network | 95.7%92.7–98.2 | 24 Oct 2026 |