Skip to content

Use benchmark Python for checkpoint evaluation - #37

Merged
tmusser merged 1 commit into
mainfrom
fix/checkpoint-benchmark-python
Jul 10, 2026
Merged

Use benchmark Python for checkpoint evaluation#37
tmusser merged 1 commit into
mainfrom
fix/checkpoint-benchmark-python

Conversation

@tmusser

@tmusser tmusser commented Jul 10, 2026

Copy link
Copy Markdown
Owner

Summary

  • preserve the configured benchmark Python path, including virtual-environment paths;
  • use the same interpreter for checkpoint verification and hidden evaluators;
  • validate the evaluation environment once per phase;
  • distinguish invalid evaluation environments from agent functional failures;
  • preserve tri-state environment evidence for historical bundles;
  • keep setup and usage bootstrap-safe when the project venv does not yet exist;
  • surface interpreter and environment-validity metadata in traces and scorecards.

Validation

  • focused observer and scorecard tests: 39 passed
  • full benchmark harness suite: 264 passed
  • shell syntax checks: passed
  • git diff --check: passed

Evidence

The repaired Task 7 Codex pair produced complete checkpoint coverage and evaluator-backed first-functional and first-bench-ready provider-item boundaries without the prior missing-pytest failure.

Generated benchmark runs and reports remain local and are not included.

@tmusser
tmusser merged commit 6f18d77 into main Jul 10, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant