SWE-bench Verified (50) on DeepSeek-Flash
sample-50, 50 tasks, 1 attempt, allowlisted network.
Real GitHub issues, resolved against the projects' own test suites. A subset, so compare within this board, not with published full-set numbers. About 3-5 hours per run. Bring your own DeepSeek key.