hidai25/eval-view - 72 Help Wanted Issues

Open Issues Need Help

evalview check --watch (flag alias for the existing watcher) 11 days ago

help wanted good first issue cli

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

Cookbook: 'prompt-as-migration' recipe 2 months ago

documentation help wanted

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

Discord webhook notifier (parallel to Slack) 2 months ago

enhancement help wanted good first issue

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

GitHub Action: post regression diff as PR comment 2 months ago

enhancement help wanted cli

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

Add Pydantic AI tool-call schema validator 2 months ago

enhancement help wanted

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

Add Vercel AI SDK adapter 2 months ago

enhancement help wanted

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

Add Discord webhook support to evalview monitor 4 months ago

help wanted good first issue

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

Show token cost breakdown in check output 4 months ago

enhancement help wanted good first issue

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

Add `--json` flag to `evalview snapshot` 4 months ago

enhancement help wanted good first issue

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

`evalview validate` — lint test files without running them 4 months ago

enhancement help wanted good first issue

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

Support TOML as alternative to YAML for test cases 4 months ago

enhancement help wanted good first issue

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

Add slow-agent warning before timeout fires 4 months ago

enhancement help wanted good first issue

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

`evalview diff` — pretty-print what changed between two runs 4 months ago

enhancement help wanted good first issue

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

Add cron schedule syntax to evalview monitor 4 months ago

help wanted good first issue

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

feat(importers): add CSV support for generate --from-log 4 months ago

enhancement help wanted good first issue

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

Add Ollama example — local agent with no API keys 4 months ago

documentation help wanted good first issue

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

Add AutoGen/AG2 integration guide 4 months ago

documentation help wanted

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

Add Vercel AI SDK integration guide 4 months ago

documentation help wanted

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

Community test registry — seed templates 4 months ago

documentation help wanted

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

Interactive HTML report — add filtering and search 4 months ago

enhancement help wanted

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

Watch mode — Textual TUI upgrade 4 months ago

enhancement help wanted

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

Badge — GitHub Actions step to auto-commit badge 4 months ago

enhancement help wanted

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

Add `evalview import --from pydantic-evals` 4 months ago

enhancement help wanted

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

Add `evalview export --format openai-evals` 4 months ago

enhancement help wanted

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

CrewAI — track cost per agent 4 months ago

enhancement help wanted

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

CrewAI adapter — streaming support 4 months ago

enhancement help wanted

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

OpenClaw — auto-research loop example 4 months ago

documentation help wanted

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

MCP contract drift — auto-detect schema changes 4 months ago

enhancement help wanted

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

Pydantic AI adapter — structured output validation 4 months ago

enhancement help wanted

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

CrewAI adapter — support hierarchical process type 4 months ago

enhancement help wanted

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

CrewAI adapter — capture per-agent reasoning steps 4 months ago

enhancement help wanted

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

Add test templates to `evalview init` 4 months ago

enhancement help wanted good first issue

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

Add `evalview doctor` command 4 months ago

enhancement help wanted good first issue

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

Improve error messages for common adapter failures 4 months ago

enhancement help wanted good first issue

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

Write "EvalView vs pydantic_evals" comparison doc 4 months ago

documentation help wanted good first issue

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

Write "EvalView vs crewai test" comparison doc 4 months ago

documentation help wanted good first issue

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

Add Pydantic AI example — customer support agent 4 months ago

documentation help wanted good first issue

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

Add CrewAI example — Trip Planner crew 4 months ago

documentation help wanted good first issue

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

docs: add end-to-end generate guides for LangGraph, MCP, and generic HTTP agents 5 months ago

documentation help wanted

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

ci: add example GitHub Actions workflow for generated suite review 5 months ago

documentation enhancement help wanted

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

feat(importers): add generic HTTP trace export format support 5 months ago

enhancement help wanted good first issue

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

feat: webhook adapter for testing event-driven agents (async/callback) 5 months ago

enhancement help wanted good first issue

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

feat: GitHub Actions reusable workflow for agent regression testing 5 months ago

enhancement help wanted good first issue

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

feat: cost and latency trend alerts in monitor 5 months ago

enhancement help wanted good first issue

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

feat: auto-generate edge case tests from existing test suite 5 months ago

enhancement help wanted good first issue

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

feat: A/B compare two agent versions side by side 5 months ago

enhancement help wanted good first issue

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

Export monitor history to JSONL for trend analysis 5 months ago

help wanted good first issue

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

Add live terminal dashboard to evalview monitor 5 months ago

help wanted good first issue

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

Add email alerts to evalview monitor (SMTP) 5 months ago

help wanted good first issue

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

feat: better error message when agent endpoint is unreachable 5 months ago

AI Summary: This GitHub issue proposes improving error messages when an agent endpoint is unreachable due to connection issues (refused, timeout, DNS failure). Instead of generic Python tracebacks, the system should catch specific HTTP exceptions like `httpx.ConnectError` and `httpx.TimeoutException` to display user-friendly messages, including the endpoint URL and helpful hints. The pattern should also be applied to other adapters beyond HTTP.

Complexity: 2/5

enhancement help wanted good first issue

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

feat: add --timeout flag to check command 5 months ago

AI Summary: This issue requests adding a `--timeout` flag to the `evalview check` command. This flag will allow users to specify a timeout duration per test, defaulting to 30 seconds, to handle slow agent responses. The implementation involves using `click.option` and passing the timeout value to the underlying adapter, following the pattern established by the `run` command.

Complexity: 1/5

enhancement help wanted good first issue cli

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

feat: export results to markdown 5 months ago

enhancement help wanted good first issue

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

feat: HTML report light/dark mode toggle 5 months ago

enhancement help wanted good first issue

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

feat: export check results to CSV 5 months ago

enhancement help wanted good first issue

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

feat: multi-turn golden baseline — save and diff per-turn tool sequences 5 months ago

enhancement help wanted good first issue

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

feat: retry and statistical mode support for multi-turn tests 5 months ago

enhancement help wanted good first issue statistics

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

feat: multi-turn conversation support in HTML visual report (Mermaid diagram + turn sections) 5 months ago

enhancement help wanted good first issue visualization

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

feat: multi-turn support in evalview capture (record conversation sessions as turns: YAML) 5 months ago

enhancement help wanted good first issue cli

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

feat: per-turn pass/fail display in multi-turn console output 5 months ago

enhancement help wanted good first issue cli

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

Add Cohere adapter 5 months ago

help wanted good first issue

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

Add Mistral adapter 5 months ago

help wanted good first issue

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

Add --eval no-hallucination named evaluator 5 months ago

help wanted good first issue

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

Add --eval no-pii named evaluator 5 months ago

help wanted good first issue

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

Add --eval factual named evaluator 5 months ago

help wanted good first issue

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

Add docstrings to undocumented ConsoleReporter helper methods 5 months ago

documentation good first issue

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

Add Pydantic field validators to TestCase model 5 months ago

enhancement help wanted good first issue

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

Add edge case tests for CostEvaluator and LatencyEvaluator 5 months ago

enhancement help wanted good first issue

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

Add health_check() method to OllamaAdapter 5 months ago

enhancement help wanted good first issue

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

Cache LLM judge responses during repeated test runs 8 months ago

enhancement help wanted performance

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

Use confidence intervals in pass/fail decisions 8 months ago

enhancement help wanted statistics

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

Add --statistical flag to CLI for explicit mode toggle 8 months ago

enhancement good first issue cli

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing

Add distribution visualization for variance reports 8 months ago

enhancement good first issue visualization

hidai25/eval-view

124

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python

#agent-benchmark#agent-evaluation#agentic-ai#ai-agents#anthropic#autogen#cli#crewai#evaluation#langchain-agent#langgraph#llm#mcp#openai-assistants#pytest#python#regression-testing#testing