Open Issues Need Help
View All on GitHub ci: notify-on-red misses cancelled/skipped needed jobs about 2 months ago
bug good first issue ci
Add coverage for unsupported agent file types about 2 months ago
good first issue benchmark tests
tests: repeatability headline still fail-open via STABLE over N runs ⊂ UNSTABLE over N runs about 2 months ago
bug good first issue tests
fix(benchmark): verify_benchmark_seal raises TypeError on a non-ASCII receipt stdout instead of reporting a mismatch about 2 months ago
bug good first issue benchmark
fix(scripts): verify_attestation raises AttributeError on an evidence bundle whose inputs is not a mapping about 2 months ago
bug good first issue
docs(specs): spec 007 pins 7 plan kinds while the planner accepts and advertises 13 about 2 months ago
bug documentation good first issue
fix(benchmark): component_mix reports 0.0 component means for an unscored aggregate about 2 months ago
bug good first issue benchmark
fix(benchmark): gap_integrity reports CONSISTENT for a fully-unscored artifact its siblings reject about 2 months ago
bug good first issue benchmark
scripts/compare_eval.py: a symlink loop leaks a raw errno instead of a named error about 2 months ago
bug good first issue benchmark
planner: non-int/negative n crashes or silently truncates the wrong end of the plan about 2 months ago
bug good first issue tests
run_eval --held-out silently ignored or overrides an explicit --repo-set-partition about 2 months ago
bug good first issue benchmark
score: truthy non-list plan/revealed files aborts module recall scoring about 2 months ago
bug good first issue benchmark
judge: summarize_judge_orders crashes on truthy non-iterable categories about 2 months ago
bug good first issue benchmark
scripts/compare_eval.py: raw traceback instead of a clean error on a bad artifact about 2 months ago
bug good first issue ci
review_pr: fetch_pr crashes when gh returns truthy non-list files about 2 months ago
bug good first issue tests
Multi-repo unscored placeholder `composite_mean: 0.0` still treated as a real score in trend/regression/CI gates about 2 months ago
bug good first issue benchmark tests ci
report: render_report crashes on a non-finite (NaN/Infinity) numeric field instead of rendering n/a about 2 months ago
bug good first issue benchmark tests
judge_calibration: headline and failed_scenarios crash on truthy non-list failed/symmetry_checks about 2 months ago
bug good first issue benchmark tests
headline_score reports an unscored multi-repo run as a real 0.0 score about 2 months ago
bug good first issue benchmark
scripts/trend.py: raw traceback instead of a clean error on a bad artifact about 2 months ago
bug good first issue tests ci
runner: run_multi_replay crashes when freeze_window is a truthy non-dict about 2 months ago
bug good first issue benchmark tests
judge_gate: judge_headline and failed_checks crash on truthy non-list checks about 2 months ago
bug good first issue benchmark
report: multi-repo artifact with non-list per_repo renders wrong template about 2 months ago
bug good first issue benchmark tests
weight_sweep crashes on non-dict rows in the replay rows list about 2 months ago
bug good first issue benchmark
scripts/review_pr.py: raw traceback instead of a clean error on gh failure about 2 months ago
bug good first issue tests ci
scripts/run_eval.py: raw traceback instead of a clean error on a bad --repo/--repo-set about 2 months ago
bug good first issue benchmark tests ci
run_replay crashes when an agent solve() returns a non-dict output about 2 months ago
bug good first issue benchmark tests
leakage_audit: audit_headline crashes on truthy non-list findings about 2 months ago
bug good first issue leakage tests
sample_adequacy: headline and failed_checks crash on truthy non-list checks about 2 months ago
bug good first issue benchmark tests
Add judge tally integrity gate for replay win/loss accounting about 2 months ago
enhancement good first issue benchmark tests
bug enhancement good first issue benchmark
bug good first issue benchmark
Heuristic baseline crashes on a non-string commit subject in recent_commits about 2 months ago
bug good first issue benchmark
Add tests/test_spec_001_solve.py: offline solve() contract verifying tests about 2 months ago
documentation good first issue tests
promotion: checks row sanitization missing from headline and failed_checks about 2 months ago
bug good first issue benchmark tests
Add tests/test_spec_002_compose.py: specs/002 composition contract verifying tests about 2 months ago
enhancement good first issue tests
regression: checks row sanitization missing from headline and failed_checks about 2 months ago
bug good first issue benchmark tests
context_for_agent keeps labels when labels_as_of_t is missing about 2 months ago
bug good first issue agent leakage
judge_gate: checks row missing name/passed keys crashes with KeyError about 2 months ago
bug good first issue benchmark
skip_budget: checks row sanitization missing from headline and failed_checks about 2 months ago
bug good first issue benchmark tests
Add decisive-rate summary utility for replay tally outcomes about 2 months ago
enhancement good first issue benchmark tests
parse_owner_repo returns wrong owner/repo for URLs with trailing subpaths about 2 months ago
bug good first issue benchmark
bug good first issue benchmark tests
Add freeze coverage utility for per-repo freeze commit reporting about 2 months ago
enhancement help wanted benchmark tests
README probe lists diverge between freeze and git fallback about 2 months ago
bug good first issue benchmark agent
judge_gate CLI dumps a raw traceback on a missing/invalid artifact instead of a clean error about 2 months ago
bug good first issue ci
load_context crashes on a malformed frozen context file instead of falling back to git about 2 months ago
bug good first issue agent
margin_outlook reports 'unavailable' for every multi-repo run (decisive_margin can't read judge_report) about 2 months ago
bug good first issue benchmark
Offline judge: falsy scalar plan items (false/0/0.0) score substance, inflating a padded plan's rank about 2 months ago
bug good first issue benchmark tests
[bug] load_solve produces raw traceback on missing/invalid agent files about 2 months ago
bug good first issue benchmark
generalization gate helpers crash on a non-list checks field about 2 months ago
bug good first issue benchmark
weight_integrity skips generalization partitions missing scored_repos about 2 months ago
bug good first issue benchmark tests
repeatability: assess_repeatability raises StatisticsError on an unscored artifact set when min_runs <= 0 about 2 months ago
bug good first issue benchmark
bug good first issue benchmark
judge_gate trusts stale judge_report disagreement_rate about 2 months ago
bug enhancement good first issue benchmark ci
disagreement_outlook: a stale disagreement_rate can flip a slice's stable/unstable verdict about 2 months ago
bug good first issue benchmark
Close all parity gaps between the two git-only context builders about 2 months ago
bug good first issue benchmark agent leakage
composite_spread CLI dumps traceback on directory artifact paths about 2 months ago
bug good first issue tests
Harden all remaining artifact CLIs against unreadable-path errors (OSError), not just FileNotFoundError about 2 months ago
bug enhancement good first issue benchmark
decisive_rate: CLI dumps a traceback on a directory (or unreadable) artifact path instead of a clean error about 2 months ago
bug good first issue
judge_wlt: CLI dumps a traceback on a directory (or unreadable) artifact path instead of a clean error about 2 months ago
bug good first issue tests
comparability: CLI dumps a traceback on a directory (or unreadable) artifact path instead of a clean error about 2 months ago
bug good first issue tests
win_rate CLI: handle directory/unreadable artifact paths, not just FileNotFoundError about 2 months ago
bug good first issue tests
score_calibration: check_calibration crashes on a non-finite/oversized expected value instead of failing the scenario about 2 months ago
bug good first issue benchmark
decisive_rate: a zero-task generalization partition is summed into the overall, presenting one partition's rate as the whole-run rate about 2 months ago
bug good first issue benchmark
bug good first issue
scripts/score_integrity.py: raw errno message instead of a clean error on a directory/unreadable artifact path about 2 months ago
bug enhancement good first issue benchmark
benchmark/repo_score_spread: zero-task (skipped) repo's placeholder composite_mean of 0.0 is counted as a real score, fabricating a maximal spread about 2 months ago
bug good first issue benchmark
check_score_floor fails open: a non-finite composite_mean silently passes the --fail-under gate about 2 months ago
bug good first issue tests ci
dual_order_coverage / scored_fraction CLIs crash on an oversized-int JSON literal (loader misses ValueError) about 2 months ago
bug good first issue tests
report CLI crashes on an oversized-int literal or a non-UTF-8 file (loader misses ValueError) about 2 months ago
bug good first issue help wanted tests
error_repo_share / dual_order_coverage headlines crash or print nan% on non-finite/oversized shares about 2 months ago
bug good first issue benchmark tests
improvement / regression headlines crash on an oversized-int composite (weak _is_number) about 2 months ago
bug good first issue benchmark
dual_order_coverage / scored_fraction CLIs: raw errno on a directory/unreadable artifact path about 2 months ago
bug enhancement good first issue benchmark
scripts/repeatability_gate.py: raw errno on a directory/unreadable/broken-symlink artifact path about 2 months ago
bug enhancement good first issue
scripts/error_repo_share.py: raw errno on a directory/unreadable/broken-symlink artifact path about 2 months ago
bug enhancement good first issue benchmark
repo_set: a non-positive freeze_window.horizon_days loads but silently drops the repo about 2 months ago
bug good first issue benchmark tests
scripts/weight_integrity.py: dangling symlink reported as "not found"; symlink loops leak raw errno about 2 months ago
bug good first issue benchmark
scripts/agree_order_share.py: raw errno instead of an actionable error on a bad artifact path about 2 months ago
enhancement good first issue chore
scripts/disagree_order_share.py: raw errno instead of an actionable error on a bad artifact path about 2 months ago
enhancement good first issue benchmark
scripts/dual_order_share.py: raw errno instead of an actionable error on a bad artifact path about 2 months ago
bug enhancement good first issue benchmark
scripts/order_agree_rate.py: raw errno on a directory/unreadable/broken-symlink artifact path about 2 months ago
bug enhancement good first issue
scripts/offline_share.py: raw errno on a directory/unreadable/broken-symlink artifact path about 2 months ago
bug good first issue ci
scripts/partition_task_share.py: raw errno on a directory/unreadable/broken-symlink artifact path about 2 months ago
bug enhancement good first issue benchmark
scripts/repo_score_spread.py: raw errno on a directory/unreadable/broken-symlink artifact path about 2 months ago
bug enhancement good first issue ci
repo_coverage / audit_context CLIs print a bare exception instead of an actionable path error about 2 months ago
bug enhancement good first issue tests chore
scripts/margin_outlook.py: dangling symlink reported as "not found"; symlink loops leak raw errno about 2 months ago
bug good first issue benchmark
scripts/skip_share.py: dangling symlink reported as "not found"; symlink loops leak raw errno about 2 months ago
bug enhancement good first issue benchmark
decisive_rate CLI crashes on a symlink loop and misreports a broken symlink about 2 months ago
bug help wanted tests ci
scripts/acceptance.py: a broken symlink and a symlink loop are not reported cleanly about 2 months ago
bug enhancement good first issue tests
scripts/comparability.py: a broken symlink and a symlink loop are not reported cleanly about 2 months ago
bug good first issue benchmark tests
scripts/component_mix.py: raw errno on a directory/unreadable/broken-symlink artifact path about 2 months ago
bug enhancement good first issue
scripts/promotion.py: a broken symlink and a symlink loop are not reported cleanly about 2 months ago
bug enhancement good first issue benchmark
judge_calibration.validate_scenario crashes on a non-string expected_winner about 2 months ago
bug good first issue benchmark
scripts/artifact_snapshot.py: a broken symlink and a symlink loop are not reported cleanly about 2 months ago
bug good first issue benchmark
scripts/gap_integrity.py: dangling symlink misreported; symlink loop leaks errno; path failure exit 1 collides with --strict about 2 months ago
bug good first issue tests ci
dual_order_share / disagree_order_share CLIs: dangling symlink misreported as "not found"; symlink loops leak raw errno about 2 months ago
bug good first issue benchmark
aggregate_integrity / judge_report_integrity CLIs: broken-symlink misreport, unnamed ELOOP/not-a-directory, and load errors exit 1 (collide with the --strict gate) about 2 months ago
bug enhancement good first issue
specs: the knowable-at-T GitHub context producer (benchmark/github_context.py) has no contract spec about 2 months ago
bug documentation good first issue benchmark
specs: the freeze snapshot assembly (benchmark/freeze.py) has no contract spec beyond parse_path_list about 2 months ago
documentation help wanted