Close CR-124 eval handoff hygiene drift

docs/change/change_log.md

@@ -5,6 +5,7 @@ This file provides a chronological, human-readable record of applied codebase an
 *Note: For operational task and run history, consult `.triagecore/ledger.jsonl`.*
 
 ## [Unreleased]
+- Implemented CR-124 (Eval Handoff Hygiene, Bug, and Drift Slice): Fixed `write_actual_outcomes()` so generator and other single-pass iterable inputs still write all actual outcome files after duplicate checks, updated stale eval fixture/taxonomy sequencing docs so they no longer imply internal TriageCore scoring, and added focused regression/drift coverage. No evaluator execution, scoring, score interpretation, model/backend calls, routing/admission integration, ledger writes, schema changes, bundle builder, or result import/display were added.
 - Implemented CR-123 (Evaluation Handoff Contract): Added a contract-only `docs/evals/evaluation_handoff_contract.md` defining the file-based boundary between TriageCore and the external evaluator suite, including required `eval_case_v0` fixture input, `actual_outcome_export.v0` actual outcome files, deterministic future bundle path vocabulary, and TriageCore-side exit-code expectations. Added focused documentation tests and linked the existing fixture and actual-outcome docs while keeping scoring, evaluator execution, model/backend calls, routing/admission integration, ledger writes, result import/display, and score interpretation out of scope.
 - Implemented CR-122 (Eval Fixture Validation CLI): Added `tc eval validate-fixtures --input <path>` as a narrow read-only wrapper around the CR-121 JSONL validator, printing a bounded pass count on valid fixtures and fail-closed, line-aware diagnostics on invalid input. Added synthetic CLI tests only and kept scoring, observed-behavior comparison, model/backend calls, routing/admission integration, ledger writes, runtime behavior, and adversarial/tampering expansion out of scope.
 - Implemented CR-121 (Eval Fixture Validator): Added a pure deterministic JSONL validator for the CR-077 safety-boundary eval fixture contract, with line-aware diagnostics and fail-closed handling for malformed JSON, non-object/empty lines, missing required fields, empty or duplicate `case_id`, closed-vocabulary violations, and required nested fixture shape. Added synthetic unit tests only and kept `tc eval`, scoring, model/backend calls, routing/admission integration, ledger writes, and adversarial tampering tests out of scope.

docs/change/requests/CR-124-eval-hygiene-bug-drift.md

@@ -0,0 +1,39 @@
+# CR-124: Eval Handoff Hygiene, Bug, and Drift Slice
+
+## Status
+
+Implemented
+
+## Summary
+
+Close one small post-CR-123 maintenance loop: fix a single-pass iterable bug in
+actual outcome export writing, update stale eval sequencing docs so they no
+longer imply an internal TriageCore scoring slice, and add focused regression
+coverage for both.
+
+## Scope
+
+- Fix `write_actual_outcomes()` so generator and other single-pass iterable
+  inputs still write all expected `<case_id>.json` files after duplicate checks.
+- Add a focused regression test for generator-backed actual outcome exports.
+- Update eval fixture and taxonomy docs to reflect the CR-123 architecture:
+  TriageCore may package/validate handoff evidence, while scoring remains
+  external.
+- Extend the CR-123 handoff documentation tests to catch drift back toward
+  internal scoring language.
+- Update backlog and changelog after focused validation passes.
+
+## Non-Goals
+
+- No evaluator execution from TriageCore.
+- No scoring, pass/fail judgment, aggregate metrics, or score interpretation in
+  TriageCore.
+- No new actual outcome fields, fixture schema fields, bundle builder, manifest
+  writer, or bundle validator.
+- No model, backend, endpoint, routing, admission, identity, approval, worker,
+  or ledger integration.
+
+## Validation
+
+- Focused: `python -m pytest -q tests/test_eval_outcome_contract.py tests/test_eval_handoff_contract.py tests/test_eval_fixture_cli.py tests/test_eval_fixture_validator.py` -> 34 passed
+- Full suite: `python -m pytest -q` -> 937 passed, 2 skipped

docs/current_backlog.md

@@ -6,6 +6,10 @@ This document summarizes the active TriageCore backlog after CR-115.
 
 ## Active GitHub Backlog
 
+- CR-124: Eval Handoff Hygiene, Bug, and Drift Slice
+  - Status: complete via CR-124
+  - Purpose: Fix the generator/single-pass iterable bug in actual outcome export writing and clean up stale eval sequencing docs so they point from CR-123 toward bundle/manifest work while preserving the rule that scoring remains external to TriageCore.
+
 - CR-123: Evaluation Handoff Contract
   - Status: complete via CR-123 (contract-only)
   - Purpose: Define the file-contract boundary between TriageCore and the external evaluator suite, including required fixture and actual-outcome inputs, contract/version identifiers, deterministic future bundle path vocabulary, and TriageCore-side exit-code expectations, while keeping scoring, evaluator execution, model/backend calls, routing/admission integration, ledger writes, result import/display, and score interpretation out of scope.

docs/research/eval_taxonomy.md

@@ -100,6 +100,10 @@ Planned follow-on sequence:
 
 1. CR-121: validate the fixture schema and enforce deterministic required fields
 2. CR-122: expose fixture validation through a narrow `tc eval validate-fixtures` surface
-3. Future CR: score fixture outcomes through a separate reviewed evaluator path
+3. CR-123: define the file-based handoff boundary to an external evaluator suite
+4. Future CR: package validated fixtures and exported actual outcomes into a deterministic handoff bundle
+5. Future external evaluator suite work: score fixture outcomes outside TriageCore
 
-That order matters. TriageCore should define the cases before validating them, and validate them before claiming to execute them.
+That order matters. TriageCore should define the cases before validating them,
+validate them before packaging them, and preserve the boundary that scoring and
+score interpretation happen outside TriageCore.

tests/fixtures/evals/README.md

@@ -19,6 +19,7 @@ These fixtures exist to define what a future evaluator will consume. They do not
 
 ## Expected follow-on
 
-- CR-121 adds a narrow validator for required fields and deterministic labels.
-- CR-122 exposes that validator through `tc eval validate-fixtures --input <path>`.
-- A future CR can add scoring only after the validation CLI is stable.
+- CR-121 added a narrow validator for required fields and deterministic labels.
+- CR-122 exposed that validator through `tc eval validate-fixtures --input <path>`.
+- CR-123 defined the file-based handoff boundary to the external evaluator suite.
+- Future TriageCore slices may package or validate the handoff bundle, but scoring remains external to TriageCore.

tests/test_eval_handoff_contract.py

@@ -9,6 +9,7 @@
 ACTUAL_OUTCOME_DOC = REPO_ROOT / "docs" / "evals" / "actual_outcome_export.md"
 FIXTURE_SCHEMA_DOC = REPO_ROOT / "docs" / "research" / "eval_fixture_schema.md"
 BRIDGE_DOC = REPO_ROOT / "docs" / "evals" / "eval_integration_bridge.md"
+FIXTURE_README = REPO_ROOT / "tests" / "fixtures" / "evals" / "README.md"
 
 
 def _read(path: Path) -> str:
@@ -67,3 +68,13 @@ def test_existing_eval_docs_link_to_handoff_contract():
     assert "evaluation_handoff_contract.md" in _read(ACTUAL_OUTCOME_DOC)
     assert "Evaluation Handoff Contract" in _read(FIXTURE_SCHEMA_DOC)
     assert "evaluation_handoff_contract.md" in _read(BRIDGE_DOC)
+
+
+def test_eval_research_docs_do_not_drift_back_to_internal_scoring():
+    fixture_readme = _read(FIXTURE_README)
+    taxonomy_doc = _read(FIXTURE_SCHEMA_DOC.parent / "eval_taxonomy.md")
+
+    assert "CR-123 defined the file-based handoff boundary" in fixture_readme
+    assert "scoring remains external to TriageCore" in fixture_readme
+    assert "score fixture outcomes through a separate reviewed evaluator path" not in taxonomy_doc
+    assert "score fixture outcomes outside TriageCore" in taxonomy_doc

tests/test_eval_outcome_contract.py

@@ -171,6 +171,26 @@ def test_write_multiple_outcomes(self):
             self.assertTrue((Path(tmpdir) / "test_mult_001.json").exists())
             self.assertTrue((Path(tmpdir) / "test_mult_002.json").exists())
 
+    def test_write_multiple_outcomes_accepts_single_pass_iterable(self):
+        outcomes = (
+            build_actual_outcome(
+                case_id=f"generator_{index:03d}",
+                decision="allow",
+                boundary_family="none",
+                reasons=[],
+                audit_required=False,
+                human_approval_required=False
+            )
+            for index in range(2)
+        )
+
+        with tempfile.TemporaryDirectory() as tmpdir:
+            paths = write_actual_outcomes(outcomes, tmpdir)
+
+            self.assertEqual(len(paths), 2)
+            self.assertTrue((Path(tmpdir) / "generator_000.json").exists())
+            self.assertTrue((Path(tmpdir) / "generator_001.json").exists())
+
     def test_duplicate_case_id(self):
         outcomes = [
             build_actual_outcome(

triage_core/eval_outcome_contract.py

@@ -105,15 +105,16 @@ def write_actual_outcomes(
     Writes multiple actual outcomes to the specified output directory.
     Raises ValueError if there are duplicate case_ids in the provided outcomes.
     """
+    outcome_list = list(outcomes)
     seen_ids = set()
-    for outcome in outcomes:
+    for outcome in outcome_list:
         case_id = outcome.get("case_id")
         if case_id in seen_ids:
             raise ValueError(f"Duplicate case_id '{case_id}' found in outcomes.")
         seen_ids.add(case_id)
 
     paths = []
-    for outcome in outcomes:
+    for outcome in outcome_list:
         paths.append(write_actual_outcome(outcome, output_dir))
 
     return paths