Open Issues Need Help
View All on GitHub Measure cost per solved task, not just pass rate about 7 hours ago
good first issue
How do you know if a harness is any good? Benchmarking and testing harnesses — the scoreboard, the cost, and the things one-run benchmarks never measure. Open problem.
Write an adapter for one harness about 7 hours ago
good first issue
How do you know if a harness is any good? Benchmarking and testing harnesses — the scoreboard, the cost, and the things one-run benchmarks never measure. Open problem.
Build a one-command runner about 7 hours ago
good first issue
How do you know if a harness is any good? Benchmarking and testing harnesses — the scoreboard, the cost, and the things one-run benchmarks never measure. Open problem.
Run the existing benchmark once and tell us what it cost about 7 hours ago
good first issue research
How do you know if a harness is any good? Benchmarking and testing harnesses — the scoreboard, the cost, and the things one-run benchmarks never measure. Open problem.