huggingface/lighteval

[FT] showing count in Markdown summary table 10 days ago

good first issue feature

2.5K

Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends

Python

#evaluation#evaluation-framework#evaluation-metrics#huggingface

[EVAL] Adding PHARE 20 days ago

good first issue help wanted new-task

huggingface/lighteval

2.5K

Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends

Python

#evaluation#evaluation-framework#evaluation-metrics#huggingface

[BUG] Optimize tokenization about 1 month ago

bug good first issue help wanted science-team

huggingface/lighteval

2.5K

Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends

Python

#evaluation#evaluation-framework#evaluation-metrics#huggingface

[FT] Build in a way to specify specific IDs/Lines in Dataset to use as few-shot examples in the same split about 2 months ago

good first issue help wanted feature

huggingface/lighteval

2.5K

Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends

Python

#evaluation#evaluation-framework#evaluation-metrics#huggingface

[FT] Manage script and language in the Language enum 2 months ago

good first issue feature

huggingface/lighteval

2.5K

Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends

Python

#evaluation#evaluation-framework#evaluation-metrics#huggingface

[EVAL] Long Horizon Execution 6 months ago

good first issue help wanted new-task

huggingface/lighteval

2.5K

Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends

Python

#evaluation#evaluation-framework#evaluation-metrics#huggingface

[EVAL] MultiChallenge 7 months ago

help wanted new-task

huggingface/lighteval

2.5K

Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends

Python

#evaluation#evaluation-framework#evaluation-metrics#huggingface

[FT] Add tests for nanotron 8 months ago

good first issue help wanted feature ignore-for-release

huggingface/lighteval

2.5K

Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends

Python

#evaluation#evaluation-framework#evaluation-metrics#huggingface

[EVAL] Big-Bench Extra Hard (BBEH) 8 months ago

good first issue help wanted new-task science-team

huggingface/lighteval

2.5K

Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends

Python

#evaluation#evaluation-framework#evaluation-metrics#huggingface

[EVAL] Add kyrgyzLLM benchmark 9 months ago

good first issue new-task

huggingface/lighteval

2.5K

Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends

Python

#evaluation#evaluation-framework#evaluation-metrics#huggingface

[EVAL] SciCode: reasearch coding benchmark 10 months ago

help wanted new-task

huggingface/lighteval

2.5K

Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends

Python

#evaluation#evaluation-framework#evaluation-metrics#huggingface

[EVAL] HELMET: long context evals 11 months ago

help wanted new-task

huggingface/lighteval

2.5K

Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends

Python

#evaluation#evaluation-framework#evaluation-metrics#huggingface

[FT] LiteLLM concurrency parameters hard-coded 11 months ago

good first issue help wanted feature/enhancement

huggingface/lighteval

2.5K

Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends

Python

#evaluation#evaluation-framework#evaluation-metrics#huggingface

Append revision to filepath in `--output_dir`? 11 months ago

good first issue feature/enhancement science-team

huggingface/lighteval

2.5K

Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends

Python

#evaluation#evaluation-framework#evaluation-metrics#huggingface

Open Issues Need Help