Terminal-Bench 2.0 on Claude Opus 4.6
89 tasks, 5 attempts, open network. Updated 25 Oct 2026.
Compare
Pick two harnesses to compare them task by task.
pilot against Terminus 2 2.0.0, task by task
Over the 89 tasks both ran, 5 attempts each
pilot minus Terminus 2 2.0.0, pass rate
+19.1 points
95% interval +7.9 to +29.4
The interval leaves out zero, so the paired test tells the two apart.
Where the tasks landed
- 25Only pilot solved
- 50Both solved
- 9Only Terminus 2 2.0.0 solved
- 5Neither solved
Of 89 tasks, 25 only pilot solved, 9 only Terminus 2 2.0.0 solved, 50 both solved and 5 neither did.
| Task | Reward | Tokens | Time | |||
|---|---|---|---|---|---|---|
| pilot | Terminus 2 2.0.0 | pilot | Terminus 2 2.0.0 | pilot | Terminus 2 2.0.0 | |
caffe-cifar-10 | 1 | 0 | 11k | 69k | 25.0 min | 20.0 min |
compile-compcert | 1 | 0 | 5,290 | 394k | 26.2 min | 40.0 min |
configure-git-webserver | 1 | 0 | 6,631 | 68k | 16.1 min | 2.2 min |
db-wal-recovery | 1 | 0 | 3,098 | 253k | 2.2 min | 15.0 min |
extract-moves-from-video | 1 | 0 | 1,937 | 1.6M | 1.4 min | 30.0 min |
filter-js-from-html | 1 | 0 | 45k | 164k | 32.2 min | 4.4 min |
git-multibranch | 0 | 1 | 16k | 43k | 12.7 min | 2.5 min |
gpt2-codegolf | 1 | 0 | 136k | 81k | 48.9 min | 15.0 min |
model-extraction-relu-logits | 1 | 0 | 47k | 75k | 15.9 min | 6.5 min |
mteb-leaderboard | 1 | 0 | 512 | 3.6M | 1.2 min | 42.3 min |
mteb-retrieve | 1 | 0 | 1,908 | 13k | 1.7 min | 1.4 min |
path-tracing | 1 | 0 | 55k | 71k | 20.3 min | 30.0 min |
prove-plus-comm | 0 | 1 | — | 7,143 | 1 s | 35 s |
pytorch-model-recovery | 0 | 1 | — | 104k | 1 s | 8.6 min |
qemu-alpine-ssh | 0 | 1 | — | 200k | 2 s | 9.8 min |
qemu-startup | 0 | 1 | — | 80k | 1 s | 7.6 min |
raman-fitting | 1 | 0 | 58k | 190k | 18.7 min | 15.0 min |
sam-cell-seg | 1 | 0 | 26k | 337k | 13.6 min | 14.8 min |
torch-tensor-parallelism | 1 | 0 | 17k | 81k | 9.2 min | 5.7 min |
adaptive-rejection-sampler | 0.8 | 0 | 27k | 239k | 7.8 min | 11.1 min |
feal-linear-cryptanalysis | 1 | 0.2 | 102k | 46k | 26.6 min | 29.7 min |
gcode-to-text | 1 | 0.2 | 778 | 695k | 1.0 min | 13.7 min |
install-windows-3.11 | 0.8 | 0 | 16k | 2.3M | 19.3 min | 31.2 min |
regex-chess | 0 | 0.8 | — | 3.0M | 15.6 min | 52.7 min |
sanitize-git-repo | 1 | 0.2 | 24k | 144k | 5.7 min | 2.5 min |
tune-mjcf | 1 | 0.2 | 76k | 156k | 42.7 min | 15.0 min |
video-processing | 1 | 0.2 | 39k | 187k | 13.8 min | 6.2 min |
dna-assembly | 1 | 0.4 | 108k | 238k | 30.9 min | 26.7 min |
dna-insert | 1 | 0.4 | 5,816 | 175k | 2.8 min | 6.5 min |
headless-terminal | 0 | 0.6 | 16k | 30k | 8.5 min | 1.6 min |
path-tracing-reverse | 0 | 0.6 | 89k | 986k | 29.8 min | 27.2 min |
build-cython-ext | 1 | 0.6 | 18k | 334k | 9.9 min | 8.9 min |
build-pmars | 1 | 0.6 | 2,427 | 85k | 2.2 min | 2.3 min |
count-dataset-tokens | 1 | 0.6 | 1,331 | 55k | 2.4 min | 4.2 min |
mailman | 0.4 | 0.8 | 40k | 166k | 21.2 min | 5.6 min |
make-mips-interpreter | 0.8 | 0.4 | 130k | 1.1M | 39.8 min | 19.2 min |
overfull-hbox | 1 | 0.6 | 17k | 137k | 8.2 min | 6.6 min |
pytorch-model-cli | 1 | 0.6 | 13k | 59k | 7.5 min | 4.1 min |
schemelike-metacircular-eval | 0.6 | 1 | 80k | 458k | 39.7 min | 20.3 min |
torch-pipeline-parallelism | 1 | 0.6 | 46k | 205k | 20.5 min | 13.9 min |
winning-avg-corewars | 0.6 | 0.2 | 119k | 1.6M | 42.6 min | 58.7 min |
query-optimize | 0.8 | 0.6 | 11k | 75k | 58.8 min | 8.3 min |
cancel-async-tasks | 0.2 | 0.4 | 2,923 | 23k | 2.0 min | 1.8 min |
bn-fit-modify | 1 | 0.8 | 13k | 54k | 6.2 min | 4.9 min |
build-pov-ray | 1 | 0.8 | 14k | 692k | 11.0 min | 12.1 min |
chess-best-move | 1 | 0.8 | 1,153 | 137k | 1.2 min | 7.5 min |
circuit-fibsqrt | 0.8 | 1 | 81k | 297k | 22.0 min | 17.5 min |
constraints-scheduling | 0.8 | 1 | 5,642 | 21k | 2.4 min | 1.9 min |
distribution-search | 1 | 0.8 | 69k | 483k | 27.9 min | 16.2 min |
extract-elf | 1 | 0.8 | 3,803 | 30k | 1.9 min | 1.3 min |
nginx-request-logging | 0.8 | 1 | 5,012 | 41k | 4.1 min | 1.7 min |
protein-assembly | 1 | 0.8 | 12k | 378k | 5.9 min | 17.5 min |
write-compressor | 1 | 0.8 | 36k | 119k | 11.3 min | 7.4 min |