Skip to content

Scan-side bundle: stop on an unresolved clone, require the history flags, file blobs per period - #26

Merged
mmcky merged 2 commits into
mainfrom
feat/scan-side-bundle
Sep 1, 2026
Merged

mmcky merged 2 commits into
mainfrom
feat/scan-side-bundle

Conversation

@mmcky

@mmcky mmcky commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Closes #17 and #24. Three changes to qestyle_scan.py that each move the checker digest, taken together so both periods are re-measured once.

What changes

  • git_snapshot stops the scan when a series' checkout cannot be resolved to a commit, instead of returning an empty one that every report header then quoted (The gate fails open: delete lectures/data/ and qestyle_check.py still passes #15 item 3). --unpinned is the explicit way past it — for a directory that is deliberately not a checkout, such as a candidate extracted with git archive (the VERIFICATION.md recovery snippet now passes it) — and writes no pin and no blob table.
  • --period and --append-history are required. A scan without them measured a period and recorded no pins for it, and left the previous run's reach rows under the period's label (A period's pinned commits are not recorded, so an earlier baseline cannot be reproduced #13, The gate cannot tell a stale history row from a current one #21). Every documented invocation already passed both; the scan itself now refuses.
  • lectures/data/blobs/<period>.csvlecture_blobs.csv filed by period beside the pins (lecture_blobs.csv is current-period-only, so churn must be re-derived every time #17). Churn between any two periods is a two-file diff. New gate check blob-tables: every period in snapshot_history.csv has a table, per-series row counts equal the pins' lecture counts, blobs are 40-hex, and the newest table is byte-identical to lecture_blobs.csv.
  • Deterministic tie order in rule_reach.csv / rule_reach_history.csv (reach desc, occurrences desc, rule id). Ties used to fall back to dict insertion order, which differed between two scans of the same corpus and made a value-identical re-measure show as 70 changed lines. One consequence: two tied qe-admon-* rows swap places in a generated details.md table.

Deliberately not in the bundle: making --history mandatory in qestyle_report (#21 mentioned it). --splice-only runs after prose edits are legitimate and frequent; the gate's score-history check already catches a stale score row.

The re-measure

Checker digest aef064f3b2606b5150d246fa. Both periods re-measured with the new scanner and recorded in tools/VERIFICATION.md:

period source result
2026-08 .corpus/ at the pins violations.csv, lecture_blobs.csv, snapshot.json, every per-lecture report byte-identical; rule_reach.csv and the 2026-08 history rows value-identical (tie reorder only); blobs/2026-08.csv = lecture_blobs.csv
2026-05 snapshot_history.csv pins as sparse worktrees under .corpus/.prev-2026-05/ 2026-05 history rows value-identical, 35 of 35 on both columns; snapshot_history.csv moved on checker only, basis stayed recovered; blobs/2026-05.csv written for the first time, 300 rows matching the pins

Churn from the two new blob tables reproduces the record exactly: 186 unchanged, 114 edited, 48 new, 0 removed → 162 of 348 (47 %), the figures ROADMAP § 2.4 already quotes (and now describes as a two-file diff).

Verified error paths

case result
scan without --period / --append-history argparse error, exit 2
corpus directory that is not a checkout stops with cannot resolve HEAD to a commit … pass --unpinned …, nothing written
same with --unpinned measures; warns per series that no pin row is written; no blobs/ table; snapshot_history.csv gets no row
gate passes: 10 pins at 6b5150d246fa, 35 reach rows held, 648 blobs across 2 periods, 12 score rows

No score, report, or overlay moved. history.csv, history_mechanical.csv, scores*.csv untouched.

🤖 Generated with Claude Code

…ags, file blobs per period

Three changes to qestyle_scan.py that each move the checker digest, taken
together so both periods are re-measured once (#24):

- git_snapshot stops the scan when a series' checkout cannot be resolved to
  a commit, instead of returning an empty one that every report header then
  quoted (#15 item 3). --unpinned is the explicit way past it, for a
  directory that is deliberately not a checkout (a candidate extracted with
  git archive); it measures and writes no pin and no blob table.
- --period and --append-history are required. A scan without them measured
  a period and recorded no pins for it, and left the previous run's reach
  rows under the period's label (#13, #21). Every documented invocation
  already passed both.
- lecture_blobs.csv is also filed by period as blobs/<period>.csv beside
  the pins (#17), so churn between any two periods is a two-file diff. The
  gate holds every recorded period to a table whose per-series counts are
  the pins', and the newest table byte-identical to lecture_blobs.csv.

Also: the reach tables' tie order now breaks on occurrences then rule id
rather than dict insertion order, which differed between two scans of the
same corpus and made a value-identical re-measure show as 70 changed lines.

Re-measured at the new digest a0a2fd049290 (tools/VERIFICATION.md): 2026-08
from .corpus at the pins, 2026-05 from snapshot_history.csv's pins as
worktrees. violations.csv, lecture_blobs.csv, snapshot.json and every report
byte-identical; both periods' reach rows value-identical (35 of 35 on both
columns for 2026-05); snapshot_history.csv moved on checker only, basis
stayed recovered. blobs/2026-05.csv gives the record's churn exactly:
186 unchanged, 114 edited, 48 new.

Closes #17
Closes #24

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings September 1, 2026 05:17

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR tightens the scan-side invariants and the gate around period reproducibility: the scanner now fails closed on unresolved clones, requires the period/history flags, and persists per-period lecture blob tables so churn becomes a simple file diff rather than a reconstruction step.

Changes:

  • Make tools/qestyle_scan.py fail closed on unresolved series clones (with --unpinned as the explicit escape hatch).
  • Require --period and --append-history in the scanner, and stabilize reach-table ordering with a deterministic tie-break.
  • Add per-period blob tables under lectures/data/blobs/<period>.csv plus a new blob-tables gate check in tools/qestyle_check.py.

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
UPDATE.md Updates the runbook to reflect the new required flags and per-period blob tables.
tools/VERIFICATION.md Documents the re-measure and adds an --unpinned example for archive extractions.
tools/qestyle_scan.py Implements fail-closed snapshot resolution, required CLI flags, per-period blob tables, and deterministic reach ordering.
tools/qestyle_check.py Adds the new blob-tables gate check to validate per-period blob tables and newest-table consistency.
ROADMAP.md Updates narrative to reference per-period blob tables as the churn measurement source.
README.md Mentions that blob tables are now kept per period for churn diffs.
lectures/details.md Reflects regenerated ordering from deterministic reach tie-breaks.
lectures/data/snapshot_history.csv Updates checker digest for recorded periods after scanner changes.
lectures/data/rule_reach.csv Reflects deterministic tie-break ordering.
lectures/data/rule_reach_history.csv Reflects deterministic tie-break ordering and re-measure outputs.
lectures/data/blobs/2026-08.csv Adds new per-period blob table for 2026-08.
lectures/data/blobs/2026-05.csv Adds new per-period blob table for 2026-05.
CLAUDE.md Updates pipeline layout documentation to include per-period blob tables.
.claude/skills/pass-publish/SKILL.md Updates the publish skill docs to include the additional blob-table write.
.claude/skills/pass-measure/SKILL.md Updates the measure skill docs to reflect the new required flags and blob-table invariants.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread tools/qestyle_scan.py Outdated
Comment thread tools/qestyle_check.py Outdated
… honest blob-tables note

write_snapshot_history was still called under --unpinned. It wrote no new
rows, but it replaces the period's rows, so an --unpinned scan with
--append-history pointed at the committed data would have deleted a
recorded period's pins under a candidate's label. The call is now guarded
like the blob-table write, and the run says so in one line. Verified against
a copy of the real data: snapshot_history.csv byte-identical afterwards.

check_blob_tables noted "<newest> matches lecture_blobs.csv" even when the
byte compare had just failed. The note now carries the verdict.

Digest re-stamped to 6b5150d246fa by re-running both scans; only `checker`
moved, reach rows and blob tables unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@mmcky
mmcky merged commit d32b12a into main Sep 1, 2026
2 checks passed
@mmcky
mmcky deleted the feat/scan-side-bundle branch September 1, 2026 05:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

lecture_blobs.csv is current-period-only, so churn must be re-derived every time

2 participants