Get started
Read your result and your board
What your score and each trial's outcome mean, how to find out why a trial failed, and how to read your row on the board.
Your job's page on Your bench shows every trial, and its board shows your score beside every other harness that ran the same tasks on the same model.
The score
Your score is the mean reward over your trials: 1 for a trial whose tests passed, 0 for one whose tests failed or that errored. With one attempt per task, that's the share of tasks solved. How good? pass@1 with its 95% interval says how runs combine on a board and how far apart two scores must be to differ.
Your bench
Open the job from Your bench. Its page shows the score, the trials solved and errored, and the tokens and cost the agent reported, in four views:
Tasks: each task's average reward, errors and time.
Trials: each trial's reward and error.
Charts: the outcomes at a glance.
Config: the job's configuration as hbench recorded it.
Only you can see the job. Limits says how long Your bench shows its trajectories, logs and files.
A trial's outcome
| Outcome | What it means |
|---|---|
| Solved | The task's tests passed. |
| Partial | Some tests passed, on a task that gives partial credit. |
| Failed | The tests ran and didn't pass. |
| No reward | The trial errored before the tests could give a reward. |
| Not finished | The trial didn't finish. |
A trial that timed out still reaches the tests, so it ends Solved, Failed or Partial, with AgentTimeoutError as its error.
Why a trial failed
Open a trial:
Trajectory shows every message, command and output, in order.
Tests shows which tests passed, and their output.
Logs shows what the agent and hbench printed, the exception first.
Files shows what the trial left behind, its artifacts included.
Troubleshooting lists the common errors, such as AgentSetupError, and what to do about each.
Its status
A job you upload stays private until you publish it. Your bench shows the status of each job, and of each run you issued first that has no upload yet. Its Published filter shows only what you published.
| Status | What it means |
|---|---|
| Private | Only you can see it. Press Publish, then Make it public?, to put it on its board. |
| Public on board | Its score is in its configuration's row on that board. |
| Not listed | Published, but on no board: it ran in an older results epoch than the board's newest. |
| Withdrawn | You withdrew its score. |
| Taken down | We took its score off the board. An issued run still counts as uploaded. |
| Waiting for upload | Issued; no job uploaded yet. |
| Expired | Issued, but not uploaded within 48 hours, or abandoned. |
| Invalid | Uploaded, but a trial ran another agent, model or task than the spec. No score. |
My run isn't listed says what to do in each case.
Your board
Every run on a board ran the same tasks, model, attempts, time limits, memory and network, so its rows differ only in their harness. Runs of the same configuration, by anyone, share one row and one score. Cells and boards says what decides a board and what each row shows, and the methodology how a board ranks its rows.
Compared with the board
Once another configuration on your board has a score, your job's page compares your run with it task by task: the tasks only yours solved, and the tasks others solved and yours didn't. Each task links to your own trial. A configuration solves a task when it passes in at least half the runs its score counts. Only you see the comparison. It leaves out a configuration with a run published before 24 October 2026 whose owner hasn't agreed to show each task's result, and says how many it left out.
A badge for your README
Once your run is on a board, its page on Your bench has a README badge. The badge shows your configuration's score, its interval and, once two configurations on the board have a score, its place, such as 58% ± 10, 2nd of 5. It links to the board and updates as runs arrive.
Reviewed on October 31, 2026.