Terminal-Bench 2.0 on Claude Opus 4.6
89 tasks, 5 attempts, open network. Updated 25 Oct 2026.
Compare
Pick two harnesses to compare them task by task.
Terminus 2 2.0.0 against pilot, task by task
Over the 89 tasks both ran, 5 attempts each
Terminus 2 2.0.0 minus pilot, pass rate
−19.1 points
95% interval −29.4 to −7.9
The interval leaves out zero, so the paired test tells the two apart.
Where the tasks landed
- 9Only Terminus 2 2.0.0 solved
- 50Both solved
- 25Only pilot solved
- 5Neither solved
Of 89 tasks, 9 only Terminus 2 2.0.0 solved, 25 only pilot solved, 50 both solved and 5 neither did.
| Task | Reward | Tokens | Time | |||
|---|---|---|---|---|---|---|
| Terminus 2 2.0.0 | pilot | Terminus 2 2.0.0 | pilot | Terminus 2 2.0.0 | pilot | |
caffe-cifar-10 | 0 | 1 | 69k | 11k | 20.0 min | 25.0 min |
compile-compcert | 0 | 1 | 394k | 5,290 | 40.0 min | 26.2 min |
configure-git-webserver | 0 | 1 | 68k | 6,631 | 2.2 min | 16.1 min |
db-wal-recovery | 0 | 1 | 253k | 3,098 | 15.0 min | 2.2 min |
extract-moves-from-video | 0 | 1 | 1.6M | 1,937 | 30.0 min | 1.4 min |
filter-js-from-html | 0 | 1 | 164k | 45k | 4.4 min | 32.2 min |
git-multibranch | 1 | 0 | 43k | 16k | 2.5 min | 12.7 min |
gpt2-codegolf | 0 | 1 | 81k | 136k | 15.0 min | 48.9 min |
model-extraction-relu-logits | 0 | 1 | 75k | 47k | 6.5 min | 15.9 min |
mteb-leaderboard | 0 | 1 | 3.6M | 512 | 42.3 min | 1.2 min |
mteb-retrieve | 0 | 1 | 13k | 1,908 | 1.4 min | 1.7 min |
path-tracing | 0 | 1 | 71k | 55k | 30.0 min | 20.3 min |
prove-plus-comm | 1 | 0 | 7,143 | — | 35 s | 1 s |
pytorch-model-recovery | 1 | 0 | 104k | — | 8.6 min | 1 s |
qemu-alpine-ssh | 1 | 0 | 200k | — | 9.8 min | 2 s |
qemu-startup | 1 | 0 | 80k | — | 7.6 min | 1 s |
raman-fitting | 0 | 1 | 190k | 58k | 15.0 min | 18.7 min |
sam-cell-seg | 0 | 1 | 337k | 26k | 14.8 min | 13.6 min |
torch-tensor-parallelism | 0 | 1 | 81k | 17k | 5.7 min | 9.2 min |
adaptive-rejection-sampler | 0 | 0.8 | 239k | 27k | 11.1 min | 7.8 min |
feal-linear-cryptanalysis | 0.2 | 1 | 46k | 102k | 29.7 min | 26.6 min |
gcode-to-text | 0.2 | 1 | 695k | 778 | 13.7 min | 1.0 min |
install-windows-3.11 | 0 | 0.8 | 2.3M | 16k | 31.2 min | 19.3 min |
regex-chess | 0.8 | 0 | 3.0M | — | 52.7 min | 15.6 min |
sanitize-git-repo | 0.2 | 1 | 144k | 24k | 2.5 min | 5.7 min |
tune-mjcf | 0.2 | 1 | 156k | 76k | 15.0 min | 42.7 min |
video-processing | 0.2 | 1 | 187k | 39k | 6.2 min | 13.8 min |
dna-assembly | 0.4 | 1 | 238k | 108k | 26.7 min | 30.9 min |
dna-insert | 0.4 | 1 | 175k | 5,816 | 6.5 min | 2.8 min |
headless-terminal | 0.6 | 0 | 30k | 16k | 1.6 min | 8.5 min |
path-tracing-reverse | 0.6 | 0 | 986k | 89k | 27.2 min | 29.8 min |
build-cython-ext | 0.6 | 1 | 334k | 18k | 8.9 min | 9.9 min |
build-pmars | 0.6 | 1 | 85k | 2,427 | 2.3 min | 2.2 min |
count-dataset-tokens | 0.6 | 1 | 55k | 1,331 | 4.2 min | 2.4 min |
mailman | 0.8 | 0.4 | 166k | 40k | 5.6 min | 21.2 min |
make-mips-interpreter | 0.4 | 0.8 | 1.1M | 130k | 19.2 min | 39.8 min |
overfull-hbox | 0.6 | 1 | 137k | 17k | 6.6 min | 8.2 min |
pytorch-model-cli | 0.6 | 1 | 59k | 13k | 4.1 min | 7.5 min |
schemelike-metacircular-eval | 1 | 0.6 | 458k | 80k | 20.3 min | 39.7 min |
torch-pipeline-parallelism | 0.6 | 1 | 205k | 46k | 13.9 min | 20.5 min |
winning-avg-corewars | 0.2 | 0.6 | 1.6M | 119k | 58.7 min | 42.6 min |
query-optimize | 0.6 | 0.8 | 75k | 11k | 8.3 min | 58.8 min |
cancel-async-tasks | 0.4 | 0.2 | 23k | 2,923 | 1.8 min | 2.0 min |
bn-fit-modify | 0.8 | 1 | 54k | 13k | 4.9 min | 6.2 min |
build-pov-ray | 0.8 | 1 | 692k | 14k | 12.1 min | 11.0 min |
chess-best-move | 0.8 | 1 | 137k | 1,153 | 7.5 min | 1.2 min |
circuit-fibsqrt | 1 | 0.8 | 297k | 81k | 17.5 min | 22.0 min |
constraints-scheduling | 1 | 0.8 | 21k | 5,642 | 1.9 min | 2.4 min |
distribution-search | 0.8 | 1 | 483k | 69k | 16.2 min | 27.9 min |
extract-elf | 0.8 | 1 | 30k | 3,803 | 1.3 min | 1.9 min |
nginx-request-logging | 1 | 0.8 | 41k | 5,012 | 1.7 min | 4.1 min |
protein-assembly | 0.8 | 1 | 378k | 12k | 17.5 min | 5.9 min |
write-compressor | 0.8 | 1 | 119k | 36k | 7.4 min | 11.3 min |