humaneval-plus@1.0 (unreviewed) on GPT-5.6 Sol
164 tasks, 1 attempt, open network. Updated 24 Oct 2026.
164 tasks from humaneval-plus@1.0 on ghcr.io. Nobody has reviewed this dataset, so a score here says nothing about the benchmark it's named after.
Value
What a task cost each configuration.
$ per solved task
Lower is cheaper. Dollars per task divided by the pass rate.
Harness$ per solved task
Tokens per task
Lower is leaner. Input plus output, cached input included.
HarnessTokens per task
Agent time per task
Lower is faster. Agent start to finish; setup and tests left out.
HarnessTime per task
Pass rate against tokens per task
Up and to the left is better.
The frontier needs two configurations that report tokens, time or dollars.