hardikkaurani

hardikkaurani/Eval-Forge

Production-grade open-source LLM evaluation platform with G-Eval, LLM-as-a-Judge, RAG evaluation, and customizable AI evaluation pipelines.

1 stars 0 forks 1 watchers Python MIT License
ai ai-evaluation artificial-intelligence benchmarking developer-tools evaluation fastapi g-eval llm llm-as-a-judge llm-evaluation open-source postgresql python rag rag-evaluation react redis typescript
View on GitHub
27 Open Issues Need Help Last updated: Jul 28, 2026

Open Issues Need Help

View All on GitHub
Add Request Cancellation (AbortController) to React Query Hooks 11 days ago
help wanted good first issue frontend
hardikkaurani/Eval-Forge
1

Production-grade open-source LLM evaluation platform with G-Eval, LLM-as-a-Judge, RAG evaluation, and customizable AI evaluation pipelines.

Python
#ai#ai-evaluation#artificial-intelligence#benchmarking#developer-tools#evaluation#fastapi#g-eval#llm#llm-as-a-judge#llm-evaluation#open-source#postgresql#python#rag#rag-evaluation#react#redis#typescript
Implement Request Correlation ID Tracing (X-Request-ID) to Celery Worker Logs about 1 month ago
help wanted backend
hardikkaurani/Eval-Forge
1

Production-grade open-source LLM evaluation platform with G-Eval, LLM-as-a-Judge, RAG evaluation, and customizable AI evaluation pipelines.

Python
#ai#ai-evaluation#artificial-intelligence#benchmarking#developer-tools#evaluation#fastapi#g-eval#llm#llm-as-a-judge#llm-evaluation#open-source#postgresql#python#rag#rag-evaluation#react#redis#typescript
Add OpenAPI Schema Generation Test to CI Workflow about 1 month ago
help wanted good first issue testing
hardikkaurani/Eval-Forge
1

Production-grade open-source LLM evaluation platform with G-Eval, LLM-as-a-Judge, RAG evaluation, and customizable AI evaluation pipelines.

Python
#ai#ai-evaluation#artificial-intelligence#benchmarking#developer-tools#evaluation#fastapi#g-eval#llm#llm-as-a-judge#llm-evaluation#open-source#postgresql#python#rag#rag-evaluation#react#redis#typescript
Add Automated Visual Regression Tests with Playwright about 1 month ago
help wanted frontend testing
hardikkaurani/Eval-Forge
1

Production-grade open-source LLM evaluation platform with G-Eval, LLM-as-a-Judge, RAG evaluation, and customizable AI evaluation pipelines.

Python
#ai#ai-evaluation#artificial-intelligence#benchmarking#developer-tools#evaluation#fastapi#g-eval#llm#llm-as-a-judge#llm-evaluation#open-source#postgresql#python#rag#rag-evaluation#react#redis#typescript
Create Standalone Python CLI Tool (evalforge-cli) about 1 month ago
help wanted backend
hardikkaurani/Eval-Forge
1

Production-grade open-source LLM evaluation platform with G-Eval, LLM-as-a-Judge, RAG evaluation, and customizable AI evaluation pipelines.

Python
#ai#ai-evaluation#artificial-intelligence#benchmarking#developer-tools#evaluation#fastapi#g-eval#llm#llm-as-a-judge#llm-evaluation#open-source#postgresql#python#rag#rag-evaluation#react#redis#typescript
Implement Side-by-Side Pairwise LLM Response Diff Viewer Component about 1 month ago
help wanted frontend
hardikkaurani/Eval-Forge
1

Production-grade open-source LLM evaluation platform with G-Eval, LLM-as-a-Judge, RAG evaluation, and customizable AI evaluation pipelines.

Python
#ai#ai-evaluation#artificial-intelligence#benchmarking#developer-tools#evaluation#fastapi#g-eval#llm#llm-as-a-judge#llm-evaluation#open-source#postgresql#python#rag#rag-evaluation#react#redis#typescript
Add Jinja2 Template Syntax Error Validator Before Prompt Execution about 1 month ago
help wanted backend
hardikkaurani/Eval-Forge
1

Production-grade open-source LLM evaluation platform with G-Eval, LLM-as-a-Judge, RAG evaluation, and customizable AI evaluation pipelines.

Python
#ai#ai-evaluation#artificial-intelligence#benchmarking#developer-tools#evaluation#fastapi#g-eval#llm#llm-as-a-judge#llm-evaluation#open-source#postgresql#python#rag#rag-evaluation#react#redis#typescript
Add Prometheus Metrics Exporter Endpoint (/metrics) about 1 month ago
help wanted backend
hardikkaurani/Eval-Forge
1

Production-grade open-source LLM evaluation platform with G-Eval, LLM-as-a-Judge, RAG evaluation, and customizable AI evaluation pipelines.

Python
#ai#ai-evaluation#artificial-intelligence#benchmarking#developer-tools#evaluation#fastapi#g-eval#llm#llm-as-a-judge#llm-evaluation#open-source#postgresql#python#rag#rag-evaluation#react#redis#typescript
Add Virtualized Table Rendering for Large Dataset Record Lists about 1 month ago
help wanted frontend performance
hardikkaurani/Eval-Forge
1

Production-grade open-source LLM evaluation platform with G-Eval, LLM-as-a-Judge, RAG evaluation, and customizable AI evaluation pipelines.

Python
#ai#ai-evaluation#artificial-intelligence#benchmarking#developer-tools#evaluation#fastapi#g-eval#llm#llm-as-a-judge#llm-evaluation#open-source#postgresql#python#rag#rag-evaluation#react#redis#typescript
Implement Automatic WebSocket Reconnection with Exponential Backoff about 1 month ago
help wanted frontend
hardikkaurani/Eval-Forge
1

Production-grade open-source LLM evaluation platform with G-Eval, LLM-as-a-Judge, RAG evaluation, and customizable AI evaluation pipelines.

Python
#ai#ai-evaluation#artificial-intelligence#benchmarking#developer-tools#evaluation#fastapi#g-eval#llm#llm-as-a-judge#llm-evaluation#open-source#postgresql#python#rag#rag-evaluation#react#redis#typescript
Implement DeepSeek-V3 LLM Provider Driver about 1 month ago
help wanted backend
hardikkaurani/Eval-Forge
1

Production-grade open-source LLM evaluation platform with G-Eval, LLM-as-a-Judge, RAG evaluation, and customizable AI evaluation pipelines.

Python
#ai#ai-evaluation#artificial-intelligence#benchmarking#developer-tools#evaluation#fastapi#g-eval#llm#llm-as-a-judge#llm-evaluation#open-source#postgresql#python#rag#rag-evaluation#react#redis#typescript
Implement Sliding Window Rate Limit Header Exposure (X-RateLimit-Remaining) about 1 month ago
help wanted backend performance
hardikkaurani/Eval-Forge
1

Production-grade open-source LLM evaluation platform with G-Eval, LLM-as-a-Judge, RAG evaluation, and customizable AI evaluation pipelines.

Python
#ai#ai-evaluation#artificial-intelligence#benchmarking#developer-tools#evaluation#fastapi#g-eval#llm#llm-as-a-judge#llm-evaluation#open-source#postgresql#python#rag#rag-evaluation#react#redis#typescript
Add Docker Image Vulnerability Scanning (Trivy) to CI Pipeline about 1 month ago
help wanted good first issue
hardikkaurani/Eval-Forge
1

Production-grade open-source LLM evaluation platform with G-Eval, LLM-as-a-Judge, RAG evaluation, and customizable AI evaluation pipelines.

Python
#ai#ai-evaluation#artificial-intelligence#benchmarking#developer-tools#evaluation#fastapi#g-eval#llm#llm-as-a-judge#llm-evaluation#open-source#postgresql#python#rag#rag-evaluation#react#redis#typescript
Add Strict TypeScript Discriminated Union Types for WebSocket Progress Events about 1 month ago
help wanted good first issue frontend
hardikkaurani/Eval-Forge
1

Production-grade open-source LLM evaluation platform with G-Eval, LLM-as-a-Judge, RAG evaluation, and customizable AI evaluation pipelines.

Python
#ai#ai-evaluation#artificial-intelligence#benchmarking#developer-tools#evaluation#fastapi#g-eval#llm#llm-as-a-judge#llm-evaluation#open-source#postgresql#python#rag#rag-evaluation#react#redis#typescript
Add Dark Mode Contrast Ratio Compliance to ScheduledJobs Page about 1 month ago
help wanted good first issue frontend
hardikkaurani/Eval-Forge
1

Production-grade open-source LLM evaluation platform with G-Eval, LLM-as-a-Judge, RAG evaluation, and customizable AI evaluation pipelines.

Python
#ai#ai-evaluation#artificial-intelligence#benchmarking#developer-tools#evaluation#fastapi#g-eval#llm#llm-as-a-judge#llm-evaluation#open-source#postgresql#python#rag#rag-evaluation#react#redis#typescript
Implement Automated Pre-Commit Configuration (.pre-commit-config.yaml) about 1 month ago
help wanted good first issue
hardikkaurani/Eval-Forge
1

Production-grade open-source LLM evaluation platform with G-Eval, LLM-as-a-Judge, RAG evaluation, and customizable AI evaluation pipelines.

Python
#ai#ai-evaluation#artificial-intelligence#benchmarking#developer-tools#evaluation#fastapi#g-eval#llm#llm-as-a-judge#llm-evaluation#open-source#postgresql#python#rag#rag-evaluation#react#redis#typescript
Add Structured JSON Error Schema (RFC 7807) to Global Exception Handler about 1 month ago
help wanted good first issue backend
hardikkaurani/Eval-Forge
1

Production-grade open-source LLM evaluation platform with G-Eval, LLM-as-a-Judge, RAG evaluation, and customizable AI evaluation pipelines.

Python
#ai#ai-evaluation#artificial-intelligence#benchmarking#developer-tools#evaluation#fastapi#g-eval#llm#llm-as-a-judge#llm-evaluation#open-source#postgresql#python#rag#rag-evaluation#react#redis#typescript
Standardize Python Datetime UTC Usage across Enterprise Services about 1 month ago
help wanted good first issue backend
hardikkaurani/Eval-Forge
1

Production-grade open-source LLM evaluation platform with G-Eval, LLM-as-a-Judge, RAG evaluation, and customizable AI evaluation pipelines.

Python
#ai#ai-evaluation#artificial-intelligence#benchmarking#developer-tools#evaluation#fastapi#g-eval#llm#llm-as-a-judge#llm-evaluation#open-source#postgresql#python#rag#rag-evaluation#react#redis#typescript
Update Installation Guide for Docker-compose Deployment about 1 month ago
documentation good first issue
hardikkaurani/Eval-Forge
1

Production-grade open-source LLM evaluation platform with G-Eval, LLM-as-a-Judge, RAG evaluation, and customizable AI evaluation pipelines.

Python
#ai#ai-evaluation#artificial-intelligence#benchmarking#developer-tools#evaluation#fastapi#g-eval#llm#llm-as-a-judge#llm-evaluation#open-source#postgresql#python#rag#rag-evaluation#react#redis#typescript
Improve Test Coverage for Evaluator Run API Endpoint about 1 month ago
good first issue backend testing
hardikkaurani/Eval-Forge
1

Production-grade open-source LLM evaluation platform with G-Eval, LLM-as-a-Judge, RAG evaluation, and customizable AI evaluation pipelines.

Python
#ai#ai-evaluation#artificial-intelligence#benchmarking#developer-tools#evaluation#fastapi#g-eval#llm#llm-as-a-judge#llm-evaluation#open-source#postgresql#python#rag#rag-evaluation#react#redis#typescript
Create Pytest Fixture for Redis Connection Mocking in Offline Runs about 1 month ago
help wanted good first issue backend testing
hardikkaurani/Eval-Forge
1

Production-grade open-source LLM evaluation platform with G-Eval, LLM-as-a-Judge, RAG evaluation, and customizable AI evaluation pipelines.

Python
#ai#ai-evaluation#artificial-intelligence#benchmarking#developer-tools#evaluation#fastapi#g-eval#llm#llm-as-a-judge#llm-evaluation#open-source#postgresql#python#rag#rag-evaluation#react#redis#typescript
Add Strict Pydantic Response Model for Dataset Import Route about 1 month ago
help wanted good first issue backend
hardikkaurani/Eval-Forge
1

Production-grade open-source LLM evaluation platform with G-Eval, LLM-as-a-Judge, RAG evaluation, and customizable AI evaluation pipelines.

Python
#ai#ai-evaluation#artificial-intelligence#benchmarking#developer-tools#evaluation#fastapi#g-eval#llm#llm-as-a-judge#llm-evaluation#open-source#postgresql#python#rag#rag-evaluation#react#redis#typescript
Implement Automatic Database Seed Script for Development & Demonstration about 2 months ago
good first issue dx
hardikkaurani/Eval-Forge
1

Production-grade open-source LLM evaluation platform with G-Eval, LLM-as-a-Judge, RAG evaluation, and customizable AI evaluation pipelines.

Python
#ai#ai-evaluation#artificial-intelligence#benchmarking#developer-tools#evaluation#fastapi#g-eval#llm#llm-as-a-judge#llm-evaluation#open-source#postgresql#python#rag#rag-evaluation#react#redis#typescript
Audit and Synchronize ARCHITECTURE.md and README.md with Live API Capabilities about 2 months ago
documentation good first issue
hardikkaurani/Eval-Forge
1

Production-grade open-source LLM evaluation platform with G-Eval, LLM-as-a-Judge, RAG evaluation, and customizable AI evaluation pipelines.

Python
#ai#ai-evaluation#artificial-intelligence#benchmarking#developer-tools#evaluation#fastapi#g-eval#llm#llm-as-a-judge#llm-evaluation#open-source#postgresql#python#rag#rag-evaluation#react#redis#typescript
Add Global React Error Boundary and Toast Notification System about 2 months ago
good first issue frontend
hardikkaurani/Eval-Forge
1

Production-grade open-source LLM evaluation platform with G-Eval, LLM-as-a-Judge, RAG evaluation, and customizable AI evaluation pipelines.

Python
#ai#ai-evaluation#artificial-intelligence#benchmarking#developer-tools#evaluation#fastapi#g-eval#llm#llm-as-a-judge#llm-evaluation#open-source#postgresql#python#rag#rag-evaluation#react#redis#typescript
Standardize API Router Prefixes and Tag Declarations Across v1 API Router about 2 months ago
good first issue backend
hardikkaurani/Eval-Forge
1

Production-grade open-source LLM evaluation platform with G-Eval, LLM-as-a-Judge, RAG evaluation, and customizable AI evaluation pipelines.

Python
#ai#ai-evaluation#artificial-intelligence#benchmarking#developer-tools#evaluation#fastapi#g-eval#llm#llm-as-a-judge#llm-evaluation#open-source#postgresql#python#rag#rag-evaluation#react#redis#typescript
Audit and Synchronize ARCHITECTURE.md and README.md with Live API Capabilities about 2 months ago
documentation good first issue
hardikkaurani/Eval-Forge
1

Production-grade open-source LLM evaluation platform with G-Eval, LLM-as-a-Judge, RAG evaluation, and customizable AI evaluation pipelines.

Python
#ai#ai-evaluation#artificial-intelligence#benchmarking#developer-tools#evaluation#fastapi#g-eval#llm#llm-as-a-judge#llm-evaluation#open-source#postgresql#python#rag#rag-evaluation#react#redis#typescript