[Pentexa]

…ibility

Priority A — corrected difference-direction in Memory v0.3 reporting:

• paired_difference calls in memory_experiment.py and memory_merge.py now use (candidate, baseline); previously they computed baseline − candidate, so positive deltas suggested variant improvements when variants actually worsened (window +0.2543 -> -0.2543, ratio +0.2242 -> -0.2242, combined +0.1941 -> -0.1941).
• metrics.paired_difference docstring states the convention.
• Committed memory_v03_summary.json comparisons recomputed; raw per-config evidence unchanged.
• docs/MEMORY_V03.md delta table and CIs corrected.

Priority B — enforced public-observation boundary:

• New academy/eval/protocol.py: public_episode_spec hides .config; spec_for(agent, spec) gives the full spec only to privileged oracle.
• Runner boundaries (evaluator, learning loop, curriculum_experiment) now pass public specs to student policies.
• Adversarial tests: an evil student cannot read hidden config via the runner; evaluator passes public spec to students and full spec to oracle.

Priority C — benchmark integrity:

• cmd_eval no longer silently evaluates untrained agents: warning on stderr + 'untrained_fallback' marker persisted in eval artifacts.
• Documented hash-seed sensitivity of sample_substrate (sort key uses str(frozenset)); frozen protocol records the pinned-hash requirement.
• Verified committed memory_v03 grids reproduce exactly from current splits under PYTHONHASHSEED=0 (classic 16/16, OOD 12/12).
• Added regression tests that fail when committed summaries disagree with raw JSONL evidence or when recomputed comparisons differ.

Priority D — frozen Academy evaluation grid:

• generalization_audit/configs/academy_eval_grid_v01.json (classic 16 configs x 2 seeds x 4 episodes; OOD 12 x 1 x 3) matches the committed memory_v03 grid under pinned hash seed.

Tests: Academy 187/187, audit suite 37/37, Rust 32+9, hygiene checks pass.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.