Open Issues Need Help
View All on GitHub Default `mp_context_type="fork"` segfaults all workers on macOS (supported platform); actionable hint was removed from the error message about 1 month ago
good first issue priority-high
Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs
Python
Benchmark hangs forever (scheduler/worker deadlock) when dataset has no mappable columns — should fail fast instead 3 months ago
good first issue bug escape
Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs
Python
TypeError: Object of type PosixPath is not JSON serializable when saving benchmark report with --output-dir 4 months ago
good first issue priority-high bug escape
Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs
Python
Define effective hierarchy of debug logging 8 months ago
good first issue priority-low internal
Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs
Python
good first issue internal
Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs
Python
good first issue internal needs followup
Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs
Python
documentation good first issue
Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs
Python
good first issue bug
Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs
Python
Add support for a Python vLLM backend about 1 year ago
good first issue priority-high internal
Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs
Python