docs: the compliance data contract, measured against the ledger - #15
Merged
Merged
Conversation
Writes `docs/compliance-data.md`, the Phase 0 data contract the compliance plugin will read against. Part of #4, co-signed with the ledger under QuantEcon/compliance-lecture-style#29. Everything in it was measured against the files at the 2026-08 pass rather than read off the design brief, because the brief is wrong in several places and those errors are why the item exists. Corrections it records: - The brief says an empty cell means "category not in scope". No file uses an empty cell that way. There are three spellings of nothing and two meanings: `N/A` (263 occurrences, five columns) is "not applicable to this lecture"; `out-of-scope` (348, the `jax` column only) is "never in the audit's remit", and `spec.md` says explicitly that the two are distinct; an empty cell (5 occurrences, all in aggregates) is a mean over zero in-scope values and must never be coerced to 0, since 0 is the most severe score on the scale. - `rule_reach_history.csv` has six columns, not three. - `scores.csv` carries a `jax` column the brief omits. - Every file is CRLF-terminated. A reader that splits on `\n` turns `HIGH` into `HIGH\r` and breaks every priority lookup. The datavis reader already treats LF, CRLF and a lone CR as one break; this is recorded so the ledger is not later "fixed" to write LF. - `TOTAL` is a pseudo-series in three files, is a corpus-wide mean rather than the mean of the rows above it, and must never be averaged into a series chart. - `share_pct` is fully derived and the identity holds for all 70 rows, so the plugin asserts it rather than recomputing. The blocking finding is the issue cards' prose, and it splits in two. The reviewer half is recoverable: `reviews/<series>/<lecture>.json` — 348 files — carries `judgment[{rule, count, lines, detail}]`, and `qestyle_draft.py` drops `lines` and `detail` on the way to `judgment.csv`. So the plugin reads `reviews/`, and no ledger tooling changes for that half. The mechanical half is not recoverable: those lines and examples are produced live at draft time and persisted nowhere, so either the `--evidence` output is committed or mechanical cards carry a title and a count and no example. That is the first of five open questions put to the ledger maintainers. The directive syntax sheet (AC3) is included and is no longer blocked. The card kit keeps its container-and-item form as a stated exception, and takes the gated `{qe-…-start}` … `{qe-…-end}` form wherever a card body may hold a `code-cell`, following the ledger's own `qe-admon-001`. Both are specified in the plugins repository's CONTRACT.md. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…lly are Four facts measured after the first draft, each of which changes what the plugin does rather than only what the document says. `lecture` is not unique. 348 rows carry 281 distinct stems and 57 stems occur in more than one series, covering 124 rows — `amss`, `calvo` and `kalman_2` several times over. The key is `(series, lecture)`, and a join on `lecture` alone silently merges lectures from different series. Two series names contain a dot, so a series value must not be split on `.`; stems are mixed case, so a case-insensitive index is unsafe too. `jax` is a column-level fact, not a row-level one: `out-of-scope` in all 348 rows because the rules target a repository outside this corpus. It should be stated once rather than rendered as 348 identical cells, and `series_summary.csv` already drops it. `writing` and `links` can never be `N/A` — the scanner hard-codes them as applicable — so a renderer may rely on their being numeric. `admonitions` carries three distinct numeric values in the whole file: 7.5, 9.5 and 10.0. It is a three-level ordinal, and a continuous ramp over it asserts a precision the data does not have. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
🎭 Visual regression results
Details
|
There was a problem hiding this comment.
🟡 Changes recommended
The new contract doc has a few accuracy/clarity defects (wrong open-question cross-reference and copy/paste-unfriendly formula/token formatting) that should be corrected before it’s treated as authoritative.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Adds a new documentation contract (docs/compliance-data.md) that specifies the CSV/JSON inputs and null/sentinel semantics the compliance report plugin should follow, explicitly measured against the ledger outputs rather than the design brief.
Changes:
- Introduces a Phase 0 “compliance data contract” describing file locations, keys, row/column semantics, and sentinel meanings.
- Documents discrepancies between the design brief and measured ledger data (e.g.,
rule_reach_history.csvcolumns,jaxcolumn behavior, CRLF line endings). - Captures open questions that block parts of the intended report behavior (notably mechanical evidence/prose persistence).
File summaries
| File | Description |
|---|---|
| docs/compliance-data.md | New contract doc defining how the theme/plugin should interpret compliance ledger outputs and missing-value semantics. |
Review details
- Files reviewed: 1/1 changed files
- Comments generated: 3
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
This was referenced Sep 10, 2026
…e is editing safety The syntax sheet said the card kit takes the gated form wherever a body may hold a `code-cell`. Both forms are supported and both carry one — verified with a real kernel at any nesting depth — so the sheet now says so and gives the real basis for choosing. The asymmetry is not execution but editing. A mis-nested container exits 0 and takes the rest of the page with it: write the container and its item at the same colon depth and the parser pairs the first closing marker with the inner directive, so the card renders, the outer marker is left loose, and every line after it is swallowed into a `code` node — no error, no warning, under `--strict`. An unclosed gate exits 1, named and located. So: gated for executable content and for anything long or often-edited; nested for a set of prose cards, where it reads better and is the only form in which the container can reject a bad item with a line number. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The three-tier rule-universe paragraph pointed at open question 1 (mechanical issue-card prose); the item it means is 2, the checked column on rule_titles.csv, which is what makes "41 rules checked" statable. share_pct's formula used a typographic multiplication sign inside a code span whose round(..., 1) is literally Python's builtin, so a reader copying it gets a syntax error. Now an asterisk. The staleness note wrote a bare uppercase MERGED where pr_state is defined twelve lines earlier as the lowercase token set. Now the documented token. The uppercase priority values on line 108 are left alone: HIGH, MEDIUM, LOW and NONE are the real values in scores.csv. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Writes
docs/compliance-data.md, the Phase 0 data contract the compliance plugin reads against. Part of #4; the ledger's co-signing half is QuantEcon/compliance-lecture-style#29.Everything in it was measured against the files at the 2026-08 pass rather than read off the design brief. That was the point of the item: the brief is wrong in several places, and each error would have become a plugin bug.
What the brief got wrong
rule_reach_history.csvhas three columnsscores.csvcarries ajaxcolumn,out-of-scopein all 348 rowsThe sentinels.
N/A(263 occurrences across five columns) means "not applicable to this lecture".out-of-scope(348, thejaxcolumn only) means "never in the audit's remit", andspec.mdstates explicitly that the two are distinct. An empty cell (5 occurrences, all in aggregates) is a mean computed over zero in-scope values and must never be coerced to0— on a 0–10 scale,0is the most severe score, so coercing would paint a category red for having nothing to measure.Because the datavis contract has one vocabulary for a missing value, the
N/Aversusout-of-scopedistinction goes in a second property rather than a second spelling, which is what the family rule already requires.CRLF. A reader that splits on
\nleaves a trailing carriage return on the last field of every row, turningHIGHintoHIGH\rand breaking every priority lookup. The datavis reader already treats LF, CRLF and a lone CR as one record break, so nothing needs to change — but it is recorded so the ledger is not later "fixed" to write LF.The key is
(series, lecture). 348 rows carry 281 distinct stems, and 57 stems appear in more than one series across 124 rows. A join onlecturealone silently merges lectures from different series.The blocking finding
The brief says issue lines and example text exist only in the generated markdown. That is right about the CSVs and wrong about the ledger, and the difference splits the work in two.
The reviewer half is recoverable.
reviews/<series>/<lecture>.json— 348 files — carriesjudgment[{rule, count, lines, detail}]plus strengths, actions and a commit-and-blob provenance stamp.detailis exactly the prose an issue card needs.tools/qestyle_draft.pywrites only(series, stem, rule, count), solinesanddetailare dropped on the way tojudgment.csv. The plugin should readreviews/, notjudgment.csv— no ledger tooling changes for this half.The mechanical half is not recoverable. Those lines and examples are produced live by the lexer at draft time and persisted nowhere in the repository.
qestyle_scan.py --evidencecan write them and the runbook already passes the flag, but the output is deliberately kept outside the ledger. Either that evidence is committed, or mechanical issue cards carry a rule title and a count and no example. This is the one open question that blocks a designed region, and it is the first of five put to the ledger maintainers.AC3, the directive syntax sheet, is included and no longer blocked
The card kit keeps its container-and-item form as a stated exception to the datavis family rule, and every container offers two forms, both supported: nested in one fence, or gated as
{qe-…-start}…{qe-…-end}. Either carries acode-cell, verified with a real kernel at any nesting depth, so the choice is about editing rather than execution.Reach for the gated form when a body holds executable code, following
qe-admon-001, and for any long or often-edited card set. The reason is one asymmetry: a mis-nested container exits 0 and takes the rest of the page with it. Container and item at the same colon depth leaves the outer closing marker loose, every line after it is swallowed into acodenode, and the build reports nothing under--strict. An unclosed gate exits 1, named and located.Keep the nested form for a set of prose cards. It reads better, keeps the set in one fence, and is the only form in which the container can reject a bad item with a line number.
Both are specified in the plugins repository's
CONTRACT.md, in the companion change QuantEcon/quantecon-plugins.mystmd#22.Acceptance criteria
docs/compliance-data.mdwritten here — cross-linking from the ledger'sUPDATE.mdis the ledger's half, Phase 0: agree the data contract and add findings.csv to the pass compliance-lecture-style#29findings.csvincluded as a proposed schemaqe-*directives, in both the nested and the gated formsStatus
Marked draft, pending the ledger's sign-off in its own header. Five questions are put to the maintainers, and two of them — the mechanical evidence and a
checkedcolumn onrule_titles.csv— change what the report can truthfully say.🤖 Generated with Claude Code