Terminal-Bench 2.0 on Gemini 3 Pro Preview
89 tasks, 5 attempts, open network. Updated 25 Oct 2026.
Compare
Pick two harnesses to compare them task by task.
Ante against sageagent, task by task
Over the 89 tasks both ran, 5 attempts each
Ante minus sageagent, pass rate
−0.4 points
95% interval −6.7 to +5.4
The interval includes zero, so the paired test can't tell the two apart.
Where the tasks landed
- 7Only Ante solved
- 51Both solved
- 7Only sageagent solved
- 24Neither solved
Of 89 tasks, 7 only Ante solved, 7 only sageagent solved, 51 both solved and 24 neither did.
| Task | Reward | Tokens | Time | |||
|---|---|---|---|---|---|---|
| Ante | sageagent | Ante | sageagent | Ante | sageagent | |
chess-best-move | 0.2 | 1 | — | — | 3.4 min | 3.1 min |
configure-git-webserver | 0.2 | 1 | — | — | 1.9 min | 1.6 min |
polyglot-c-py | 0.2 | 1 | — | — | 3.0 min | 2.7 min |
protein-assembly | 0.8 | 0 | — | — | 6.8 min | 4.4 min |
schemelike-metacircular-eval | 1 | 0.2 | — | — | 8.9 min | 8.2 min |
adaptive-rejection-sampler | 0.2 | 0.8 | — | — | 4.6 min | 3.0 min |
query-optimize | 0.8 | 0.2 | — | — | 4.4 min | 5.0 min |
crack-7z-hash | 0.4 | 1 | — | — | 13.3 min | 8.3 min |
mteb-retrieve | 0.6 | 0 | — | — | 5.0 min | 1.9 min |
path-tracing | 0.6 | 0 | — | — | 10.4 min | 3.9 min |
qemu-startup | 0.4 | 1 | — | — | 9.5 min | 3.0 min |
break-filter-js-from-html | 1 | 0.6 | — | — | 9.5 min | 7.2 min |
build-pmars | 0.4 | 0 | — | — | 4.3 min | 5.6 min |
circuit-fibsqrt | 0.8 | 0.4 | — | — | 11.4 min | 10.3 min |
qemu-alpine-ssh | 0.4 | 0 | — | — | 10.5 min | 4.8 min |
rstan-to-pystan | 0.6 | 1 | — | — | 14.0 min | 15.4 min |
sparql-university | 0.8 | 0.4 | — | — | 2.6 min | 3.0 min |
winning-avg-corewars | 0.2 | 0.6 | — | — | 11.9 min | 7.6 min |
sanitize-git-repo | 0.6 | 0.8 | — | — | 3.1 min | 4.3 min |
caffe-cifar-10 | 0.2 | 0 | — | — | 16.4 min | 6.0 min |
dna-assembly | 0.4 | 0.2 | — | — | 7.5 min | 12.0 min |
extract-moves-from-video | 0.2 | 0 | — | — | 13.8 min | 16.3 min |
financial-document-processor | 0.2 | 0 | — | — | 3.4 min | 8.3 min |
mteb-leaderboard | 0 | 0.2 | — | — | 2.0 min | 12.1 min |
raman-fitting | 0 | 0.2 | — | — | 5.4 min | 5.1 min |
torch-pipeline-parallelism | 0 | 0.2 | — | — | 4.3 min | 5.8 min |
torch-tensor-parallelism | 0 | 0.2 | — | — | 5.2 min | 5.9 min |
video-processing | 0.2 | 0 | — | — | 4.4 min | 4.7 min |
bn-fit-modify | 0.8 | 1 | — | — | 6.8 min | 7.2 min |
build-cython-ext | 0.8 | 1 | — | — | 6.2 min | 5.1 min |
build-pov-ray | 0.8 | 1 | — | — | 3.1 min | 6.3 min |
count-dataset-tokens | 0.8 | 1 | — | — | 5.5 min | 4.6 min |
extract-elf | 0.8 | 1 | — | — | 3.1 min | 3.1 min |
git-multibranch | 0.8 | 1 | — | — | 1.8 min | 1.9 min |
large-scale-text-editing | 1 | 0.8 | — | — | 3.6 min | 7.0 min |
largest-eigenval | 1 | 0.8 | — | — | 7.2 min | 7.6 min |
log-summary-date-ranges | 1 | 0.8 | — | — | 1.6 min | 1.9 min |
merge-diff-arc-agi-task | 1 | 0.8 | — | — | 2.4 min | 2.6 min |
polyglot-rust-c | 0.8 | 1 | — | — | 3.6 min | 3.0 min |
pytorch-model-cli | 1 | 0.8 | — | — | 4.3 min | 3.6 min |
reshard-c4-data | 0.8 | 1 | — | — | 4.3 min | 4.6 min |
sqlite-with-gcov | 1 | 0.8 | — | — | 3.0 min | 2.7 min |
tune-mjcf | 0.8 | 1 | — | — | 10.3 min | 5.1 min |
write-compressor | 0.8 | 1 | — | — | 10.9 min | 6.1 min |