Terminal-Bench 2.0 on GPT-5.3-Codex
89 tasks, 5 attempts, open network. Updated 25 Oct 2026.
Compare
Pick two harnesses to compare them task by task.
droid against mux, task by task
Over the 89 tasks both ran, 5 attempts each
droid minus mux, pass rate
+2.7 points
95% interval −2.7 to +8.3
The interval includes zero, so the paired test can't tell the two apart.
Where the tasks landed
- 7Only droid solved
- 60Both solved
- 7Only mux solved
- 15Neither solved
Of 89 tasks, 7 only droid solved, 7 only mux solved, 60 both solved and 15 neither did.
| Task | Reward | Tokens | Time | |||
|---|---|---|---|---|---|---|
| droid | mux | droid | mux | droid | mux | |
mcmc-sampling-stan | 0 | 1 | — | 1.0M | 10.0 min | 19.1 min |
polyglot-c-py | 1 | 0.2 | — | 444k | 5.8 min | 7.1 min |
polyglot-rust-c | 0.8 | 0 | — | 248k | 8.5 min | 7.0 min |
configure-git-webserver | 0.6 | 0 | — | 8,580 | 3.8 min | 1.1 min |
mailman | 1 | 0.4 | — | — | 11.9 min | 30.0 min |
winning-avg-corewars | 1 | 0.4 | — | 217k | 19.0 min | 20.1 min |
adaptive-rejection-sampler | 1 | 0.6 | — | 399k | 8.0 min | 7.6 min |
cancel-async-tasks | 1 | 0.6 | — | 13k | 2.0 min | 1.2 min |
compile-compcert | 0.8 | 0.4 | — | 1.7M | 15.9 min | 11.5 min |
dna-assembly | 0 | 0.4 | — | 637k | 8.9 min | 27.7 min |
dna-insert | 0.4 | 0.8 | — | 209k | 3.9 min | 18.7 min |
gcode-to-text | 0.4 | 0 | — | 1.7M | 11.4 min | 14.9 min |
make-mips-interpreter | 0.8 | 0.4 | — | 2.6M | 14.7 min | 15.5 min |
model-extraction-relu-logits | 0.4 | 0.8 | — | 282k | 10.1 min | 9.1 min |
mteb-leaderboard | 0.6 | 1 | — | 3.2M | 15.8 min | 15.6 min |
overfull-hbox | 0.4 | 0.8 | — | 242k | 3.1 min | 3.4 min |
protein-assembly | 1 | 0.6 | — | 611k | 7.0 min | 23.5 min |
qemu-alpine-ssh | 0 | 0.4 | — | 457k | 15.0 min | 14.5 min |
torch-tensor-parallelism | 0.4 | 0.8 | — | 199k | 5.4 min | 4.2 min |
extract-elf | 0.8 | 0.6 | — | 344k | 5.9 min | 6.7 min |
tune-mjcf | 0.8 | 0.6 | — | 93k | 9.3 min | 10.3 min |
extract-moves-from-video | 0.4 | 0.2 | — | 118k | 28.0 min | 24.6 min |
mteb-retrieve | 0 | 0.2 | — | 60k | 2.2 min | 1.9 min |
torch-pipeline-parallelism | 0 | 0.2 | — | 217k | 7.1 min | 6.8 min |
train-fasttext | 0.2 | 0 | — | — | 54.9 min | 30.1 min |
chess-best-move | 1 | 0.8 | — | 860k | 3.2 min | 9.0 min |
db-wal-recovery | 0.4 | 0.6 | — | 1.1M | 6.9 min | 10.0 min |
feal-differential-cryptanalysis | 0.8 | 1 | — | 315k | 6.0 min | 9.7 min |
path-tracing | 1 | 0.8 | — | 2.3M | 8.4 min | 16.0 min |
query-optimize | 0.4 | 0.6 | — | 255k | 13.0 min | 12.8 min |
schemelike-metacircular-eval | 1 | 0.8 | — | 860k | 14.4 min | 10.9 min |