Skip to content

Add eval fixture validator (CR-121)#95

Merged
coreytshaffer merged 1 commit into
mainfrom
codex/cr-121-eval-fixture-validator
Jul 10, 2026
Merged

Add eval fixture validator (CR-121)#95
coreytshaffer merged 1 commit into
mainfrom
codex/cr-121-eval-fixture-validator

Conversation

@coreytshaffer

Copy link
Copy Markdown
Owner

Scope

Adds a pure, line-aware JSONL validator for the CR-077 evaluation fixture contract.

Included

  • Required-field and closed-vocabulary checks
  • Non-empty and duplicate case_id detection
  • Malformed JSONL diagnostics
  • Synthetic unit coverage and fixture-contract documentation

Excluded

No c eval CLI changes, scoring, model/backend calls, routing/admission integration, ledger writes, or adversarial/tampering expansion.

Validation

  • python -m pytest -q tests/test_eval_fixture_validator.py → 12 passed
  • python -m pytest -q → 928 passed, 2 skipped
  • git diff --check clean

@coreytshaffer
coreytshaffer marked this pull request as ready for review July 10, 2026 08:05
@coreytshaffer
coreytshaffer merged commit 29a4404 into main Jul 10, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant