homemade-software-inc

homemade-software-inc/completion-kit

Your prompts need tests too. Run prompts against real datasets, score outputs with LLM judges, version everything, and compare runs to see what got better.

1 stars 1 forks 1 watchers Ruby Other
anthropic evaluation-framework evaluation-metrics llm llm-as-judge llm-eval llm-evaluation llm-evaluation-framework llm-evaluation-metrics llmops mcp ollama openai prompt-engineering prompt-testing rails rails-engine ruby ruby-on-rails
View on GitHub Website
1 Open Issue Need Help Last updated: Jul 28, 2026

Open Issues Need Help

View All on GitHub
Default the judge to temperature 0 (and warn when it isn't) 8 days ago
enhancement good first issue
homemade-software-inc/completion-kit
1

Your prompts need tests too. Run prompts against real datasets, score outputs with LLM judges, version everything, and compare runs to see what got better.

Ruby
#anthropic#evaluation-framework#evaluation-metrics#llm#llm-as-judge#llm-eval#llm-evaluation#llm-evaluation-framework#llm-evaluation-metrics#llmops#mcp#ollama#openai#prompt-engineering#prompt-testing#rails#rails-engine#ruby#ruby-on-rails