Repository navigation
v0.5.26 — Deterministic regenerate-all pipeline (Loop 6) - #138
Merged
Merged
Conversation
Add PYTHONPATH=src .venv/bin/python scripts/regenerate_all.py === Regenerating all derived outputs === OK: make demo OK: make validate-scoring OK: make validate-scoring-phase3 OK: make validate-scoring-strict OK: make bench-leakage OK: make bench-cluster-split OK: make bench-expert-ablation OK: make bench-selectivity OK: make bench-triage OK: make bench-feature-decomp OK: make metrics-snapshot PASS: All tracked outputs unchanged — pipeline is deterministic target that runs all 11 pipeline + benchmark targets in sequence and verifies that tracked outputs (metrics_snapshot.json) are unchanged. New: - scripts/regenerate_all.py: standalone script that runs make demo, validate-scoring (3 variants), bench-leakage, bench-cluster-split, bench-expert-ablation, bench-selectivity, bench-triage, bench-feature-decomp, and metrics-snapshot. Uses to verify determinism. Supports --update flag for committing new baselines and --skip-targets for CI-only verification. - tests/test_regenerate_all.py: 11 tests covering sha256, run_targets, run_make, check_determinism (pass/fail), help flag, and --skip-targets with mocked snapshots. - Makefile regenerate-all target Total tests: 1700 passing (+11 from v0.5.25) docs/50_LOOP_PLAN.md: Loop 6 marked complete, position advanced to Loop 8
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Chosen bottleneck
Bottleneck: No script to regenerate all derived outputs deterministically. The committed baseline could drift from what the pipeline actually produces.
Class: Reproducibility / structural integrity
Why it matters
Reproducibility is a core project value. A
make regenerate-alltarget proves that running the full pipeline from versioned inputs produces exactly the committed outputs. This is the reproducibility check that every loop should pass before claiming progress.Changes
New:
scripts/regenerate_all.pygit diff --exit-codeon tracked outputs to verify determinism--updateflag to auto-stage new baselinesrun_make(),run_targets(),check_determinism()New:
tests/test_regenerate_all.py(11 tests)Updated:
Makefile,docs/50_LOOP_PLAN.mdmake regenerate-alltarget, help text, .PHONYVerification
make regenerate-allruns all 11 targets, pipeline deterministic (exit 0)Phase 0 completion status
make ciwith benchmark gate passesRecommended next loop
Loop 8: CONTRIBUTING.md with safety-first contribution checklist and PR template. This closes Phase 0.