Terminal-Bench 2.0 on Gemini 3.1 Pro Preview
89 tasks, 5 attempts, open network. Updated 25 Oct 2026.
Compare
Pick two harnesses to compare them task by task.
tongagents 0.8.0 against Ante, task by task
Over the 89 tasks both ran, 5 attempts each
tongagents 0.8.0 minus Ante, pass rate
+6.5 points
95% interval +1.3 to +12.4
The interval leaves out zero, so the paired test tells the two apart.
Where the tasks landed
- 10Only tongagents 0.8.0 solved
- 63Both solved
- 4Only Ante solved
- 12Neither solved
Of 89 tasks, 10 only tongagents 0.8.0 solved, 4 only Ante solved, 63 both solved and 12 neither did.
| Task | Reward | Tokens | Time | |||
|---|---|---|---|---|---|---|
| tongagents 0.8.0 | Ante | tongagents 0.8.0 | Ante | tongagents 0.8.0 | Ante | |
polyglot-c-py | 1 | 0 | — | — | 6.2 min | 8.2 min |
sam-cell-seg | 1 | 0 | — | — | 30.8 min | 36.8 min |
db-wal-recovery | 1 | 0.2 | — | — | 4.3 min | 12.5 min |
polyglot-rust-c | 0.8 | 0 | — | — | 8.3 min | 8.4 min |
qemu-startup | 0.6 | 0 | — | — | 12.5 min | 15.0 min |
query-optimize | 0.4 | 1 | — | — | 15.0 min | 13.8 min |
adaptive-rejection-sampler | 1 | 0.6 | — | — | 11.2 min | 9.6 min |
build-cython-ext | 0.6 | 1 | — | — | 9.6 min | 7.0 min |
compile-compcert | 0.8 | 0.4 | — | — | 25.3 min | 30.6 min |
configure-git-webserver | 0.8 | 0.4 | — | — | 7.2 min | 6.6 min |
extract-elf | 0.8 | 0.4 | — | — | 11.9 min | 10.4 min |
financial-document-processor | 0.6 | 1 | — | — | 10.1 min | 4.8 min |
path-tracing | 0.6 | 1 | — | — | 24.8 min | 15.6 min |
regex-chess | 0.8 | 0.4 | — | — | 31.0 min | 22.6 min |
winning-avg-corewars | 0.6 | 1 | — | — | 29.9 min | 20.1 min |
caffe-cifar-10 | 0.2 | 0 | — | — | 20.0 min | 20.0 min |
extract-moves-from-video | 0.2 | 0 | — | — | 30.0 min | 30.0 min |
gpt2-codegolf | 0 | 0.2 | — | — | 15.0 min | 15.0 min |
mteb-leaderboard | 0 | 0.2 | — | — | 50.8 min | 32.7 min |
path-tracing-reverse | 0.4 | 0.2 | — | — | 25.8 min | 27.7 min |
raman-fitting | 0.4 | 0.2 | — | — | 15.0 min | 12.8 min |
train-fasttext | 0 | 0.2 | — | — | 54.6 min | 60.0 min |
build-pmars | 1 | 0.8 | — | — | 5.7 min | 8.3 min |
cancel-async-tasks | 0.8 | 1 | — | — | 13.4 min | 13.2 min |
count-dataset-tokens | 1 | 0.8 | — | — | 12.5 min | 14.0 min |
crack-7z-hash | 0.6 | 0.4 | — | — | 12.4 min | 14.6 min |
custom-memory-heap-crash | 1 | 0.8 | — | — | 7.7 min | 10.7 min |
dna-assembly | 1 | 0.8 | — | — | 20.9 min | 15.8 min |
gcode-to-text | 1 | 0.8 | — | — | 9.7 min | 5.9 min |
git-multibranch | 1 | 0.8 | — | — | 9.7 min | 6.5 min |
large-scale-text-editing | 0.8 | 1 | — | — | 18.7 min | 20.0 min |
mcmc-sampling-stan | 1 | 0.8 | — | — | 17.1 min | 13.9 min |
merge-diff-arc-agi-task | 1 | 0.8 | — | — | 5.7 min | 8.1 min |
model-extraction-relu-logits | 0.4 | 0.6 | — | — | 10.9 min | 11.0 min |
mteb-retrieve | 1 | 0.8 | — | — | 18.3 min | 14.7 min |
pypi-server | 1 | 0.8 | — | — | 2.9 min | 3.0 min |
qemu-alpine-ssh | 0.4 | 0.6 | — | — | 15.0 min | 12.8 min |
rstan-to-pystan | 1 | 0.8 | — | — | 20.8 min | 24.3 min |
sanitize-git-repo | 1 | 0.8 | — | — | 9.5 min | 8.3 min |
video-processing | 0.4 | 0.6 | — | — | 14.1 min | 21.8 min |