Giskard-AI/giskard-oss

Add `contains_any` / `contains_all` check 7 days ago

good first issue

5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python

#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai

Add `hallucination` check 17 days ago

good first issue

Giskard-AI/giskard-oss

5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python

#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai

Add `readability` check about 1 month ago

enhancement good first issue

Giskard-AI/giskard-oss

5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python

#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai

Add `faithfulness` check about 1 month ago

good first issue

Giskard-AI/giskard-oss

5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python

#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai

CLI tool for running checks 2 months ago

Help wanted

Giskard-AI/giskard-oss

5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python

#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai

Add `bias` check 2 months ago

good first issue

Giskard-AI/giskard-oss

5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python

#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai

Add `sentiment` check 2 months ago

good first issue

Giskard-AI/giskard-oss

5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python

#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai

`giskard-llm` `__all__` contains phantom entry `ResponseOutputFunctionCall` 3 months ago

bug good first issue

Giskard-AI/giskard-oss

5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python

#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai

feat(giskard-checks): extend input_type support to all pydantic-compatible types 3 months ago

good first issue

Giskard-AI/giskard-oss

5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python

#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai

Adopt `TypeVar(default=...)` for generic base classes so callers can omit bounded type parameters 4 months ago

enhancement good first issue

Giskard-AI/giskard-oss

5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python

#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai

Add `pii_detection` check 4 months ago

good first issue

Giskard-AI/giskard-oss

5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python

#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai

Allow running a Scenario multiple times (`multiple_runs`) 4 months ago

good first issue

Giskard-AI/giskard-oss

5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python

#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai

Improve ergonomics for composing checks in scenarios 4 months ago

enhancement good first issue d:easy

Giskard-AI/giskard-oss

5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python

#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai

Observability integration (OpenTelemetry/OpenLLMmetry) 4 months ago

Help wanted

Giskard-AI/giskard-oss

5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python

#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai

Add JUnit XML export for SuiteResult 4 months ago

Help wanted

Giskard-AI/giskard-oss

5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python

#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai

Add `json_valid` check 4 months ago

good first issue

Giskard-AI/giskard-oss

5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python

#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai

Add multiple embedding providers for SemanticSimilarity 4 months ago

Help wanted

Giskard-AI/giskard-oss

5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python

#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai

Add check composition operators: `AllOf`, `AnyOf`, `Not` 4 months ago

Help wanted

Giskard-AI/giskard-oss

5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python

#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai

Add retry/majority-voting for LLM-based checks 4 months ago

Help wanted

Giskard-AI/giskard-oss

5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python

#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai

Add `prompt_injection` check 4 months ago

good first issue

Giskard-AI/giskard-oss

5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python

#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai

Add `toxicity` check 4 months ago

good first issue

Giskard-AI/giskard-oss

5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python

#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai

Add `latency` check 4 months ago

good first issue

Giskard-AI/giskard-oss

5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python

#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai

Add `AnswerRelevance` check 4 months ago

enhancement good first issue

Giskard-AI/giskard-oss

5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python

#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai

Add JUnit XML export for SuiteResult 4 months ago

enhancement good first issue

Giskard-AI/giskard-oss

5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python

#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai

Add `ContextRelevance` check 4 months ago

enhancement good first issue

Giskard-AI/giskard-oss

5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python

#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai

Include scenario errors details when displaying suite report 5 months ago

enhancement good first issue Easy

Giskard-AI/giskard-oss

5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python

#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai

Make `Suite.append()` chainable (fluent suite builder) 5 months ago

enhancement good first issue Easy

Giskard-AI/giskard-oss

5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python

#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai

Feature: reexport `resolve` at root level 5 months ago

enhancement good first issue Easy

Giskard-AI/giskard-oss

5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python

#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai

Testset generation lost when a single generation fail 6 months ago

enhancement good first issue

Giskard-AI/giskard-oss

5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python

#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai

Allow user to specify N retries 10 months ago

enhancement good first issue

Giskard-AI/giskard-oss

5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python

#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai

Generate synthetic questions with listing choices 10 months ago

enhancement good first issue

Giskard-AI/giskard-oss

5.7K

🐢 Open-Source Evaluation & Testing library for LLM Agents

Python

#agent-evaluation#ai-red-team#ai-security#ai-testing#fairness-ai#llm#llm-eval#llm-evaluation#llm-security#llmops#ml-testing#ml-validation#mlops#rag-evaluation#red-team-tools#responsible-ai#trustworthy-ai

Open Issues Need Help