AgentPostmortem

AgentPostmortem/Tracecase

CI for AI agents. Record agent runs, replay them against prompt and model changes, and catch regressions and unsafe tool calls before they ship. Diffs each suite against its previous run and returns shouldFail to gate the merge.

0 stars 2 forks 0 watchers TypeScript MIT License
ai-agents ci evaluation nextjs regression-testing supabase typescript
View on GitHub Website
14 Open Issues Need Help Last updated: Aug 6, 2026

Open Issues Need Help

View All on GitHub
Added/removed-case detection in the regression diff 8 days ago
help wanted good first issue
AgentPostmortem/Tracecase
0

CI for AI agents. Record agent runs, replay them against prompt and model changes, and catch regressions and unsafe tool calls before they ship. Diffs each suite against its previous run and returns shouldFail to gate the merge.

TypeScript
#ai-agents#ci#evaluation#nextjs#regression-testing#supabase#typescript
Latency-regression flag from latencyMs (ingested, never consumed) 8 days ago
help wanted good first issue
AgentPostmortem/Tracecase
0

CI for AI agents. Record agent runs, replay them against prompt and model changes, and catch regressions and unsafe tool calls before they ship. Diffs each suite against its previous run and returns shouldFail to gate the merge.

TypeScript
#ai-agents#ci#evaluation#nextjs#regression-testing#supabase#typescript
Flaky-case detection across recent runs 8 days ago
help wanted good first issue
AgentPostmortem/Tracecase
0

CI for AI agents. Record agent runs, replay them against prompt and model changes, and catch regressions and unsafe tool calls before they ship. Diffs each suite against its previous run and returns shouldFail to gate the merge.

TypeScript
#ai-agents#ci#evaluation#nextjs#regression-testing#supabase#typescript
Side-by-side run compare view (diff two arbitrary runs) 8 days ago
help wanted good first issue
AgentPostmortem/Tracecase
0

CI for AI agents. Record agent runs, replay them against prompt and model changes, and catch regressions and unsafe tool calls before they ship. Diffs each suite against its previous run and returns shouldFail to gate the merge.

TypeScript
#ai-agents#ci#evaluation#nextjs#regression-testing#supabase#typescript
/api/ai history entries are unvalidated 8 days ago
help wanted good first issue
AgentPostmortem/Tracecase
0

CI for AI agents. Record agent runs, replay them against prompt and model changes, and catch regressions and unsafe tool calls before they ship. Diffs each suite against its previous run and returns shouldFail to gate the merge.

TypeScript
#ai-agents#ci#evaluation#nextjs#regression-testing#supabase#typescript
POST /api/ai swallows upstream failure as HTTP 200 with empty reply 11 days ago
help wanted good first issue
AgentPostmortem/Tracecase
0

CI for AI agents. Record agent runs, replay them against prompt and model changes, and catch regressions and unsafe tool calls before they ship. Diffs each suite against its previous run and returns shouldFail to gate the merge.

TypeScript
#ai-agents#ci#evaluation#nextjs#regression-testing#supabase#typescript
Dashboard pagination can never leave page 1 on Next 15+ 11 days ago
help wanted good first issue
AgentPostmortem/Tracecase
0

CI for AI agents. Record agent runs, replay them against prompt and model changes, and catch regressions and unsafe tool calls before they ship. Diffs each suite against its previous run and returns shouldFail to gate the merge.

TypeScript
#ai-agents#ci#evaluation#nextjs#regression-testing#supabase#typescript
POST /api/ai: non-numeric max sends max_tokens null to Groq 11 days ago
help wanted good first issue
AgentPostmortem/Tracecase
0

CI for AI agents. Record agent runs, replay them against prompt and model changes, and catch regressions and unsafe tool calls before they ship. Diffs each suite against its previous run and returns shouldFail to gate the merge.

TypeScript
#ai-agents#ci#evaluation#nextjs#regression-testing#supabase#typescript
POST /api/ai: non-string prompt throws unhandled TypeError 11 days ago
help wanted good first issue
AgentPostmortem/Tracecase
0

CI for AI agents. Record agent runs, replay them against prompt and model changes, and catch regressions and unsafe tool calls before they ship. Diffs each suite against its previous run and returns shouldFail to gate the merge.

TypeScript
#ai-agents#ci#evaluation#nextjs#regression-testing#supabase#typescript
Public /api/demo can evict real history (no rate limit, unscoped prune) 14 days ago
help wanted good first issue
AgentPostmortem/Tracecase
0

CI for AI agents. Record agent runs, replay them against prompt and model changes, and catch regressions and unsafe tool calls before they ship. Diffs each suite against its previous run and returns shouldFail to gate the merge.

TypeScript
#ai-agents#ci#evaluation#nextjs#regression-testing#supabase#typescript
POST /api/demo has zero error handling 14 days ago
help wanted good first issue
AgentPostmortem/Tracecase
0

CI for AI agents. Record agent runs, replay them against prompt and model changes, and catch regressions and unsafe tool calls before they ship. Diffs each suite against its previous run and returns shouldFail to gate the merge.

TypeScript
#ai-agents#ci#evaluation#nextjs#regression-testing#supabase#typescript
No upper bound on results[] length 14 days ago
help wanted good first issue
AgentPostmortem/Tracecase
0

CI for AI agents. Record agent runs, replay them against prompt and model changes, and catch regressions and unsafe tool calls before they ship. Diffs each suite against its previous run and returns shouldFail to gate the merge.

TypeScript
#ai-agents#ci#evaluation#nextjs#regression-testing#supabase#typescript
Per-result items in POST /api/runs are unvalidated 14 days ago
help wanted good first issue
AgentPostmortem/Tracecase
0

CI for AI agents. Record agent runs, replay them against prompt and model changes, and catch regressions and unsafe tool calls before they ship. Diffs each suite against its previous run and returns shouldFail to gate the merge.

TypeScript
#ai-agents#ci#evaluation#nextjs#regression-testing#supabase#typescript
Reject empty result arrays so an empty eval run cannot pass CI 27 days ago
bug good first issue
AgentPostmortem/Tracecase
0

CI for AI agents. Record agent runs, replay them against prompt and model changes, and catch regressions and unsafe tool calls before they ship. Diffs each suite against its previous run and returns shouldFail to gate the merge.

TypeScript
#ai-agents#ci#evaluation#nextjs#regression-testing#supabase#typescript