awslabs/stickler

[FEATURE]: a comparator for normalized text equality 16 days ago

enhancement good first issue help wanted

35

A library for evaluating structured data and AI outputs with weighted field comparison and custom comparators

Python

Delete the superseded ConfusionMatrixCalculator nested and list methods (329 dead lines) and their StructuredModel wrappers 17 days ago

enhancement help wanted

awslabs/stickler

35

A library for evaluating structured data and AI outputs with weighted field comparison and custom comparators

Python

[BUG]: print_confusion_matrix raises ValueError when the per-field confusion matrix is empty 17 days ago

bug good first issue help wanted

awslabs/stickler

35

A library for evaluating structured data and AI outputs with weighted field comparison and custom comparators

Python

Remove the compare() -> _compare() deprecation shim 23 days ago

enhancement good first issue help wanted

awslabs/stickler

35

A library for evaluating structured data and AI outputs with weighted field comparison and custom comparators

Python

[DOCS]: nothing explains the two configuration paths, and the internal README documents the wrong extension prefix 23 days ago

documentation good first issue help wanted

awslabs/stickler

35

A library for evaluating structured data and AI outputs with weighted field comparison and custom comparators

Python

Delegate JSON Schema parsing to a library; keep only Stickler-specific semantics 25 days ago

enhancement good first issue help wanted

awslabs/stickler

35

A library for evaluating structured data and AI outputs with weighted field comparison and custom comparators

Python

Implement AUROC, Rand index, V-measure and Kendall tau in numpy; drop the scikit-learn dependency 25 days ago

enhancement good first issue help wanted

awslabs/stickler

35

A library for evaluating structured data and AI outputs with weighted field comparison and custom comparators

Python

[FEATURE]: Consolidate the *_helper.py modules in structured_object_evaluator/models/ about 2 months ago

enhancement good first issue

awslabs/stickler

35

A library for evaluating structured data and AI outputs with weighted field comparison and custom comparators

Python

Remove dead duplicate FieldHelper.is_structured_field_type (no union handling, no callers) about 2 months ago

enhancement good first issue

awslabs/stickler

35

A library for evaluating structured data and AI outputs with weighted field comparison and custom comparators

Python

[FEATURE]: Add non-LLM date comparator for canonical/non-canonical date equality 5 months ago

enhancement good first issue help wanted

awslabs/stickler

35

A library for evaluating structured data and AI outputs with weighted field comparison and custom comparators

Python

[FEATURE]: Implement the 8 placeholder LLMComparator unit tests 5 months ago

enhancement good first issue help wanted

awslabs/stickler

35

A library for evaluating structured data and AI outputs with weighted field comparison and custom comparators

Python

[BUG]: SemanticComparator.compare silently swallows embedding failures 5 months ago

bug good first issue

awslabs/stickler

35

A library for evaluating structured data and AI outputs with weighted field comparison and custom comparators

Python

[BUG]: SemanticComparator stores model_id as a 1-tuple instead of a string 5 months ago

bug good first issue

awslabs/stickler

35

A library for evaluating structured data and AI outputs with weighted field comparison and custom comparators

Python

[FEATURE]: Add an example script for BertComparator 5 months ago

documentation enhancement good first issue

awslabs/stickler

35

A library for evaluating structured data and AI outputs with weighted field comparison and custom comparators

Python

[FEATURE]: Export comparators from the top-level stickler package 5 months ago

enhancement good first issue

awslabs/stickler

35

A library for evaluating structured data and AI outputs with weighted field comparison and custom comparators

Python

Open Issues Need Help