UKGovernmentBEIS/inspect_ai

bash_session sends the literal string `None` to the shell when `input` is omitted for `type_submit` 2 days ago

bug good first issue accepted

2.5K

Inspect: A framework for large language model evaluations

Python

`computer` tool: `coordinate` description names 2 actions; 9 require it and 2 more act on it 7 days ago

bug good first issue accepted

UKGovernmentBEIS/inspect_ai

2.5K

Inspect: A framework for large language model evaluations

Python

Lowercase `z` UTC ISO timestamps fail on Python 3.11+ despite the normalizer accepting `z` 8 days ago

bug good first issue accepted

UKGovernmentBEIS/inspect_ai

2.5K

Inspect: A framework for large language model evaluations

Python

HF task literal targets containing `:` are silently truncated 8 days ago

bug good first issue accepted

UKGovernmentBEIS/inspect_ai

2.5K

Inspect: A framework for large language model evaluations

Python

perplexity() and target_perplexity() scorers still raise OverflowError on a large NLL 8 days ago

bug good first issue accepted

UKGovernmentBEIS/inspect_ai

2.5K

Inspect: A framework for large language model evaluations

Python

csv_dataset() fails on UTF-8 files with a BOM (the default Excel on Windows writes) with a misleading "No input in dataset" 12 days ago

good first issue accepted

UKGovernmentBEIS/inspect_ai

2.5K

Inspect: A framework for large language model evaluations

Python

bug: `eval_config.approval` in a run-config file crashes with `AttributeError` 12 days ago

good first issue accepted

UKGovernmentBEIS/inspect_ai

2.5K

Inspect: A framework for large language model evaluations

Python

Apply the existing `/responses/input_tokens` 404 fallback to 405s caused by `/responses/{response_id}` route collision 12 days ago

good first issue accepted

UKGovernmentBEIS/inspect_ai

2.5K

Inspect: A framework for large language model evaluations

Python

Model-graded scorers should warn at runtime when no grader model or model-role is bound (silent self-grading) about 1 month ago

good first issue accepted

UKGovernmentBEIS/inspect_ai

2.5K

Inspect: A framework for large language model evaluations

Python

Improve display of grouped metrics 11 months ago

good first issue viewer

UKGovernmentBEIS/inspect_ai

2.5K

Inspect: A framework for large language model evaluations

Python

Open Issues Need Help