Skip to content

Add deterministic evaluation handoff bundle#100

Merged
coreytshaffer merged 1 commit into
mainfrom
codex/cr-127-evaluation-handoff-bundle
Jul 17, 2026
Merged

Add deterministic evaluation handoff bundle#100
coreytshaffer merged 1 commit into
mainfrom
codex/cr-127-evaluation-handoff-bundle

Conversation

@coreytshaffer

Copy link
Copy Markdown
Owner

Summary

  • add tc eval build-handoff --fixture <jsonl> --actuals-dir <dir> --out-dir <new-dir>
  • package validated fixtures and explicit actual outcomes into the fixed CR-123 handoff layout
  • write a deterministic evaluation_handoff_manifest.v0 with sorted entries and SHA-256 digests
  • preflight fixture and actual content with persistent privacy invariants before any output is created
  • stage to a temporary sibling and atomically publish without overwrite or --force
  • document the manifest contract and preserve external ownership of scoring

Why

CR-123 established the file-based evaluation handoff contract but deliberately did not materialize a bundle. CR-127 implements that bounded packaging step so operators can hand deterministic, inspectable static evidence to the external evaluator without adding scoring or evaluator execution inside TriageCore.

Impact

The builder requires explicit paths, copies source evidence byte-for-byte, rejects malformed, unsafe, unknown, sensitive, overlapping, or ambiguous inputs with closed reason codes, and leaves no final bundle after failure. Missing actuals remain allowed because the external evaluator—not TriageCore—owns pass/fail judgment.

Boundaries

No scoring, matching, verdicts, certification, evaluator execution/import, model/backend/network calls, routing, ledger, worker, approval behavior, automatic discovery, default paths, overwrite, compression, signing, upload, or publication is added.

Validation

  • focused eval/bundle regression surface — 60 passed
  • python -m pytest -q — 971 passed, 2 skipped
  • python -m triage_core.tc_cli audit --privacy-invariants — 698 records passed
  • git diff --check — clean

@coreytshaffer
coreytshaffer marked this pull request as ready for review July 17, 2026 08:04
@coreytshaffer
coreytshaffer merged commit e9ac2bf into main Jul 17, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant