junru-zhu/failure-transparent-agents

Open Issues Need Help

Validation wanted: annotate the frozen 270-response human sample 11 days ago

help wanted human-validation

Benchmarking whether LLM agents admit tool failure instead of hallucinating success — 100 tasks, 1,800 responses, deterministic simulator, reproducible harness.

Python

#ai-agents#ai-evaluation#benchmark#guardrails#hallucination#llm#reproducibility#tool-use

Replication wanted: evaluate an additional open-weight model 11 days ago

help wanted model-result replication

junru-zhu/failure-transparent-agents

Benchmarking whether LLM agents admit tool failure instead of hallucinating success — 100 tasks, 1,800 responses, deterministic simulator, reproducible harness.

Python

#ai-agents#ai-evaluation#benchmark#guardrails#hallucination#llm#reproducibility#tool-use