Add deterministic eval suite (+ README/manifest refresh)#4

Closed

drakulavich wants to merge 16 commits into

mainfrom

Commits on May 30, 2026

docs: design spec for iago eval suite

drakulavich
and
claude
committed
chore(plugin): sync descriptions across manifests, add greptile/squawk keywords

drakulavich
and
claude
committed
docs(readme): restyle header in kesha-voice-kit style

drakulavich
and
claude
committed
docs: implementation plan for iago eval suite

drakulavich
and
claude
committed
test(evals): bootstrap import of production diagram functions

drakulavich
and
claude
committed
feat(evals): corpus loader with schema validation

drakulavich
and
claude
committed
feat(evals): scoring with confusion matrix and per-class metrics
drakulavich
committed
feat(evals): parse-only mermaid validator with python wrapper

drakulavich
and
claude
committed
feat(evals): reporters for tables, markers, badge, results.json

drakulavich
and
claude
committed
test(evals): synthetic corpus covering each rubric branch + abstain

drakulavich
and
claude
committed
feat(evals): gh-backed real-PR capture helper

drakulavich
and
claude
committed
feat(evals): driver with --check gate for thresholds and doc drift

drakulavich
and
claude
committed
docs: add BENCHMARKS.md, README benchmarks section and accuracy badge
drakulavich
committed
fix(action): strip numeric migration prefix from ER entity names
drakulavich
committed
chore(evals): generate baseline results and set accuracy threshold
drakulavich
committed
ci: run eval harness tests and --check gate
drakulavich
committed