VALIDATION: independent review of the 2026-08-31 ledger migration

[mmcky]

Everything below was verified during the work by the same agent that did it, in one session, on one machine. This issue exists to break that bias.

Bias to test for

The session's verification monoculture was gh CLI + curl + this repository's own tools/qestyle_check.py. Almost every claim — sites live, issues closed, numbers consistent, links resolving — was established with one of those three, and the third of them is part of the artifact under test. Two consequences worth aiming at: the gate cross-checks the reports against lectures/data/*.csv, but both sides come out of the same pipeline, so a systematically wrong detector satisfies it; and every "the site works" claim is an HTTP status plus a grep of raw HTML, never a rendered page seen by a human. One curl check in the session did produce a false 404 from a malformed URL before it was caught, which is the failure mode to assume rather than the exception.

Confirm reader-facing outcomes independently of that toolchain: open the pages in a browser, follow the links by hand, and re-derive every count from the bytes rather than from a number quoted here.

What landed

Repo	Merged	Other
compliance-lecture-style (new)	none — main was pushed directly, 90 commits, no PR and no CI run	Issues #1–#6 opened; PR #11 open (a correction); Dependabot PRs #7–#10 open and unreviewed
audit.2026-05.style-guide	#9 (d4a8fa6)	PR #8 closed unmerged; issues #1, #3, #4, #5, #6, #7 closed; repository archived
workspace-lectures	#60 (6f946d3)	—
project-style-guide	#12 (c0f4e51)	—
action-style-guide	—	Issues #18–#21: bodies replaced, #19 and #21 retitled, 4 comments posted

1. Reader-facing outcomes

• Open https://quantecon.github.io/compliance-lecture-style/ in a browser. The redirect to intro.html resolves, the page renders, the sidebar lists five series, and the five charts on charts.html display as images rather than broken placeholders.
• Open https://quantecon.github.io/audit.2026-05.style-guide/ in a browser. It still serves after archival, and its front page shows an {important} admonition rendered as an admonition (not as literal ```{important} text) linking to the ledger. Follow that link and confirm it lands.
• From the ledger's front page, follow a per-lecture drill-down (e.g. lecture-dp/lqcontrol.html) and confirm the report renders with its score table.
• The clearest known incident to confirm clean. The front page previously asserted "Math is the binding constraint on most of them". Confirm the published page now names Writing, then re-derive it yourself from lectures/data/scores.csv: over the 197 HIGH lectures, count how many have Writing ≤ 4 versus Math ≤ 4. The session's figures are 176 and 64 — do not confirm those, reproduce them. Note the definition is ambiguous (categories at the floor, sole cause, unique minimum); check the published sentence is defensible under more than one.
• Beyond what the session tested: view either site on a narrow/mobile viewport, and check one page in a browser with JavaScript disabled. Nothing in the session exercised rendering at all.

2. Artifact integrity

• The ledger's history genuinely descends from the audit repo's pass branch. In a fresh clone: git merge-base --is-ancestor 42cd094 main should succeed, and git rev-list --count main should be 90. Confirm 42cd094 is still reachable in the archived repo (refs/heads/claude/project-review-lecture-updates-6o9a2f) — the ledger's provenance depends on that branch surviving archival.
• The 348 review overlays were re-serialised during migration. The session claims content is byte-identical apart from an added source key, verified by its own script. Re-check independently: fetch an overlay from the archived branch and the same file from the ledger, strip source, and compare parsed JSON — not text. Do at least five, including one with non-ASCII in its prose.
• Every row of lectures/data/lecture_blobs.csv should equal git rev-parse <commit>:lectures/<stem>.md in the corresponding series at the commit lectures/data/snapshot.json pins. Check a sample from the lecture repos directly, not from a corpus clone the ledger's tooling produced.
• Re-derive the row counts rather than trusting them: lecture_blobs.csv should be 348 rows plus a header, matching snapshot.json's n_lectures and its per_series breakdown.

3. The highest-value claim to re-test

Reproducibility. Everything the ledger asserts rests on: the same pinned commits in produce the same numbers out. Every number published, and the whole case for a ledger rather than a frozen report, depends on it.

• In a clean clone, on a different machine if possible, clone the five series at exactly the commits snapshot.json pins, run tools/qestyle_scan.py, and diff the regenerated violations.csv, rule_reach.csv and series_rule_reach.csv against what is committed. Expect byte-identical. Report any drift, however small.
• Then the adversarial version, which the session never did: reproducibility is not correctness. Pick three rules from tools/qestyle_rules.py, read their definitions in QuantEcon/action-style-guide's style_checker/rules/, and hand-audit ten flagged occurrences each against the rule text. tools/VERIFICATION.md claims all 41 checks were sampled this way — but by the same lineage of agent that wrote the checks. A rule that is confidently and consistently wrong reproduces perfectly and passes the gate.
• Re-derive the churn measurement in PR Measure the churn saving instead of estimating it, and correct the estimate #11: comparing lecture blobs at the 2026-05 and 2026-08 snapshots should give 176 unchanged, 122 edited, 50 new. This number was itself a correction of an earlier overstatement, so it has been wrong once already.

4. Records written

• workspace-lectures: manifest.yml parses and bin/doctor resolves all 12 repos. The ledger clones to repos-infrastructure/, and bin/foreach --lectures does not include it.
• The ledger's label set matches QEP-2's normative qeps/qep-0002-labels.yml for scope: core plus refactor — re-derive from that file rather than trusting the count of 20. Confirm no non-QEP-2 label survives.
• project-style-guide: D13 is present, D12 carries a supersede note, and no file in that repo still points at the audit repo as the live home of the runbook. Grep the whole repo, not just the two files the PR touched — one such reference (PLAN.md) was outside the diff and was only caught by review.

5. Tracker consistency

• Published pages reference ledger issues by number. Confirm lectures/details.md and README.md's references to [qe-link-001/002] Question about the format of link #1–Revisit the audit weights now that rule reach is measured #4 land on the four migrated questions and not on a Dependabot PR. Issues and PRs share a number space; four Dependabot PRs (⬆️ Bump actions/upload-pages-artifact from 3 to 5 #7–⬆️ Bump actions/deploy-pages from 4 to 5 #10) were opened minutes after the repo was created.
• audit.2026-05.style-guide has 0 open issues and 0 open PRs, and [qe-link-001/002] Question about the format of link #1/Decide how lectures shared between lecture-dp and lecture-python.myst are counted #3/Revisit the audit weights now that rule reach is measured #4/No LICENSE on a public repo that republishes lecture prose verbatim #6 each carry a comment naming the correct successor (audit#1→ledger#1, Decide how lectures shared between lecture-dp and lecture-python.myst are counted #3→Spec §4: the MEDIUM priority band is structurally almost empty #2, Revisit the audit weights now that rule reach is measured #4→Decide how lectures shared between lecture-dp and lecture-python.myst are counted #3, No LICENSE on a public repo that republishes lecture prose verbatim #6→Revisit the audit weights now that rule reach is measured #4). Verify the mapping, not just that a comment exists — a transposed pair would look identical at a glance.
• action-style-guide issues Record a period's pins and the checker that measured them (#13) #18–The gate cannot tell a stale history row from a current one #21: each live body should match contributions/issues/0{1,2,3,4}-*.md in the ledger byte-for-byte. Fetch and diff. Confirm every link in those bodies resolves, and — the part the session could not judge — that a reader of that repo would find the rewritten bodies an improvement rather than an unexplained overwrite.

6. Known blind spots

• The gate cannot check a claim written in a sentence (recorded as The gate cannot check a claim written into a sentence — and one such claim was wrong #5). Verify the blind spot is real: on a scratch branch, edit a hand-written sentence in lectures/intro.md to state something the CSVs contradict, and confirm tools/qestyle_check.py still passes. Then ask whether the sweep that found the original instance shared the blind spot — it did; the inversion was found by re-deriving from scores.csv, not by any check.
• CI does not re-measure, by design. Confirm that a PR which changes a count in lectures/data/violations.csv and the matching per-lecture report together passes CI green. The workflow's own trailing comment claims this; test it rather than reading it.
• 366 recorded reviewer doubts across 229 lectures have no resolution marker. tools/qestyle_status.py reports them as recorded, not open, because the overlay schema cannot express adjudication. Confirm the count, and sample five against tools/VERIFICATION.md to see whether they were in fact adjudicated — the session assumed they were, from the claim that all 41 checks were sampled.
• The ledger's main received 90 commits with no PR and no CI run. Everything in section 3 is the check that should have been. Confirm CI passes on main today by dispatching it.

7. Decisions settled today

• Archived repositories keep serving GitHub Pages. Predicted from three QuantEcon archives, then confirmed on this repo after its own archival. Re-confirm now, and confirm raw.githubusercontent.com still serves from it.
• Archiving locks issues (no new comments, no reopening) — the premise for migrating the four open questions. Confirm by attempting a comment on a closed audit issue.
• Pages cannot be regenerated after archiving. This was load-bearing for the ordering (pointer merged and deployed before archiving) but was never directly tested — the plan was built to avoid needing to. It remains an assumption; say so if you can confirm or refute it.
• All five lecture repos still sit at their pinned commits, so the ledger is exactly current and the review queue is legitimately empty. Re-derive against the GitHub API; if any has moved since, the queue should become non-empty and tools/qestyle_status.py --queue should list precisely the changed lectures.

8. Deliberately not done

• PR ⬆️ Bump actions/setup-python from 5 to 7 #8 on the audit repo was closed unmerged, not merged — its closing comment records why (merging would have deployed the 2026-08 pass over the published May-2026 report and deleted the 2026-05 per-lecture prose from the only place it exists). Confirm that reasoning holds, and that the 2026-05 per-lecture pages are still reachable on the archived site. They exist nowhere else.
• The licence question is deferred, not resolved (No LICENSE on a public repo that republishes lecture prose verbatim #6). Confirm no licence claim was asserted anywhere in the migration.
• Four Dependabot PRs are open and unreviewed, one of which (actions/deploy-pages 4→5) is the Node-20 bump the runbook had tracked by hand.
• QEP-3 is still an open PR. Confirm nothing in the ledger or the archived repo states the compliance-{domain} type as accepted policy — a review caught exactly that overstatement once, in the audit repo's README.

For the validator

For the validator: work in a session that did not do this work. Do not use the tool named under "Bias to test for" except where a check explicitly says to run it. Re-derive counts rather than confirming them. Where a check can be run against a surface the original session did not exercise, do that too — the margin beyond the checklist is where regressions hide. Deliver: one comment on this issue with a per-item verdict (confirmed / confirmed with caveat / refuted / not completable, with evidence), a new issue for any regression found (do not bury findings in the comment), and leave the checkboxes to the issue owner unless told otherwise.

Where the reasoning lives

• README.md and ROADMAP.md in this repository — what the ledger is, and §1's record of why the rename was rejected and why not status-*.
• CLAUDE.md and UPDATE.md — the working rules and the reference runbook; .claude/skills/pass-*/SKILL.md are the procedures.
• lectures/spec.md §§8–10 — the rubric and the pass methodology; tools/VERIFICATION.md — the per-rule verdicts and the measurement traps.
• The decision record is Repo naming: no rename — assemble compliance-lecture-style, then archive this repo audit.2026-05.style-guide#2 and the execution tracker is Assemble compliance-lecture-style, then archive this repo audit.2026-05.style-guide#7, both readable on the archived repo. Programme-level context is D13 in the private QuantEcon/project-style-guide.