buildinghumanetech

buildinghumanetech/humanebench

This repo was used for HumaneBench v1 results

29 stars 6 forks 29 watchers Python Apache License 2.0

View on GitHub Website

10 Open Issues Need Help Last updated: Oct 8, 2026

Open Issues Need Help

View All on GitHub

Translate five scenarios into your language 27 days ago

good first issue

buildinghumanetech/humanebench

29

This repo was used for HumaneBench v1 results

Python

Run the 5-minute quickstart and report where you got stuck 27 days ago

good first issue

buildinghumanetech/humanebench

29

This repo was used for HumaneBench v1 results

Python

Rate a batch of model responses (no code needed) 27 days ago

good first issue

buildinghumanetech/humanebench

29

This repo was used for HumaneBench v1 results

Python

Manual dataset population about 1 year ago

enhancement good first issue accountable:Andalib

buildinghumanetech/humanebench

29

This repo was used for HumaneBench v1 results

Python

Research personas for app-specific benchmarking and prompting about 1 year ago

good first issue accountable:Erika

buildinghumanetech/humanebench

29

This repo was used for HumaneBench v1 results

Python

Switch from CSV to JSONL Format about 1 year ago

enhancement good first issue accountable:Andalib

buildinghumanetech/humanebench

29

This repo was used for HumaneBench v1 results

Python

Align generated data with desired input content about 1 year ago

AI Summary: The user wants to modify the HumaneBench data generation pipeline to produce `input` content where humans ask for personal advice, rather than advice on using other AIs. This change aims to enable a broader evaluation of the AI's understanding of general human situations, moving away from a primary focus on AI-specific interactions.

Complexity: 3/5

bug good first issue accountable:Andalib

buildinghumanetech/humanebench

29

This repo was used for HumaneBench v1 results

Python

Create automated test suite for data generation pipeline about 1 year ago

enhancement good first issue accountable:Andalib

buildinghumanetech/humanebench

29

This repo was used for HumaneBench v1 results

Python

Create automated test suite for benchmarking logic about 1 year ago

enhancement good first issue accountable:Jack

buildinghumanetech/humanebench

29

This repo was used for HumaneBench v1 results

Python

Implement Pydantic Schema Validation about 1 year ago

enhancement good first issue accountable:Andalib

buildinghumanetech/humanebench

29

This repo was used for HumaneBench v1 results

Python