Repository navigation
Spec errata: define block-level content, scope the transforms by enclosing field - #131
Merged
Merged
Conversation
Closes #122. §7.2 rules 6 and 11 state a MUST over "block-level content" and nothing defines the term — it appears four times in spec/ and is defined nowhere. An unenforceable MUST is the defect; the admissibility question the issue asks is downstream of it. The answer the taxonomy already gives is yes. §4.1 "Text Leaf Nodes" sits under "## 4. Core Block Types"; §1.4 calls a text node "the only block type that holds text"; `$defs/textNode` composes `blockBase`, whose description reads "shared by every block". Neither §7.1 nor §7.2 has a rule reaching the root array. The issue's two premises are both false: there is no sentence saying a text node is not a block object, and a text node does carry the base fields. It is also already assumed by published artifacts. The KAT vector `reject-unpaired-surrogate` ships `blocks:[{"type":"text","value": "\ud83d"}]` and rejects it for the surrogate, not the position; the fixture `reject-content-split-combining-sequence-mid-array` puts a run directly in the root array; and 09 §4.2 makes those members Merkle leaves. Twenty-one arrays resolve their items to `$defs/block` — seven core, fourteen from extensions — and every one admits a text node. So this states what is true rather than changing it. Four sites: the definition in 00 §1.4, a sentence at 03 §2.2 saying the root array holds block-level content while the content root is NOT a block (§7.1 rule 5), a note at §3.1 that a text node shares the base structure — which is why it may carry an `id` or `attributes` — and the 03a §8.1 glossary row, which said "block objects" and was the reading this corrects. Rules 6 and 11 are left exactly as they stand. The definition makes them checkable; what they require is unchanged.
Calling a text node a block makes §4.3.1 item 5's "the `id` of every block" read as though a text node's `id` were relabeled. It is not, and both implementations already agree: item 4 preserves a text node carrying an `id` unchanged, and `definedId` returns undefined for `type: "text"`. A reader following the new definition without this clause would relabel those ids and compute a different document ID. States what is already true rather than changing it.
Closes #126. PR #128 scoped §4.3.1 item 4 and §4.3.2 item 2 to block-content positions in both implementations and changed no spec file. The transform preamble admits exactly one carve-out — "uniformly, but for the two id-relabeling exceptions item 5 names" — and item 4 is not one of them, so the shipped behaviour had no licence. Measured: a `vendor:callout` carrying `body:[text,text]` with a split combining sequence rejected before #128 and loads clean after, and the two readers derive different canonical bytes for the same document. Item 5 is the template. It scoped by enclosing field, named its arrays exhaustively, and amended the preamble to admit the carve-out. Item 4 got the behaviour without either. This adds both. Three edits. The preamble now names item 4's scoping alongside item 5's exceptions. Item 4 states the exhaustive set of block-content arrays, keyed on the enclosing field rather than element shape, with `attributes` a barrier at every depth — §3.1 already calls it a closed set, which is why a text-shaped object there is data. Item 2's referent moves from "each block" to "each run of adjacent text nodes within a block-content array". That is a real widening and it is deliberate: the content root is NOT a block (Content Blocks §7.1 rule 5), so a run directly in the root `blocks` array had no referent at all, though both implementations reject one. It also decouples the rule from whether a text leaf is a root-level block, which is #122's question and should not decide this one. No code changes; all 378 conformance cases pass untouched. An extension receives the transform by NAMING its rich-text array with one of the six keys — a rule about property names, which nothing in the manifest's closed extension declaration can block.
Nine findings on nine added lines; two lenses agreed on three of them. 07-state-machine.md:304 and :358 and errors.json's summary still bound the REJECT to "the concatenated text content of a block". §4.3.2's referent had moved to a run in a block-content array precisely because the content root is not a block — so the row that ASSIGNS the REJECT had no referent for a root-level run, which a published fixture exercises. That is the gap #126 exists to close, left open one file over. Item 4 said "a block's `blocks` … `content`". Neither is ever a block's property: `blocks` is the content root's and a phantom content object's, and `content` is a `presentation:footnote` MARK's. An implementer told to look on blocks finds nothing. The 03a glossary row for `content` said only "plain text shorthand" and now names the array form too. Item 2's run is bare adjacency; item 4's is mark-identical and merge-eligible. Citing item 4 for the run itself imports that condition, and under that reading a document the published fixture REJECTS would be accepted. Now stated explicitly, in the direction the implementations already agree on. The two rules do not reach identical positions and the erratum implied they did: item 4 does not recurse into a text node's `marks`, so it never reaches a footnote mark's `content`, while item 2 does. Calling a text node a block pulls it into §3.2's id rules, which say ids SHOULD be present on all blocks and MUST be present under several extensions. Requiring one on every text node makes item 4's merge unreachable, since an id-bearing text node is a merge boundary. §3.2 now scopes those requirements away from text nodes; item 5's namespace sentence excludes them directly rather than relying on §1.4 to say so. They are excluded from MEMBERSHIP, not preserved as members, so item 5's "the only namespace members whose ids are left as authored" and its count of relabeling exceptions both stand unchanged. §4.12.1 leaves reader support for the simplified table-cell form OPTIONAL, so an unqualified "MUST NOT reject" overreached. The claim is about structural validity, not about supporting the form. And 03 §2.2's own table still read "Array of block objects" sixteen lines below the sentence added to correct that reading. Two measured id defects this surfaced are filed as #129 rather than fixed here: a text-node id may duplicate a block id, and may spell a canonical name, putting two `b0`s in hashed content.
Four more stale referents and two contradictions the second pass found. errors.json's NOTE still scoped the run arm to "a block's whole text content" — the summary was corrected but not the note beneath it. The consequence clause in 07 §5.4.3 still said "a non-NFC block"; it is about the text, not the container. 03a §4 carries the same uniqueness bullets as 03 §3.2 and needed the same carve-out; only the 03 copy had it. A text node's id is now stated as outside that namespace in both places. The reference's own comment read "A text node is typed but is NOT a content block" — the exact negation of the erratum, in the file the erratum describes. It now says what is actually true: a text node is a block type whose id sits outside the relabeled namespace. The definition was circular across documents: 00 §1.4 defined block-level content in terms of a block-content array, which only 06 §4.3.1 defined, by enumeration. 00 now defines the term on its own terms and 06 enumerates the positions.
Closes #129 defect 1. Item 5 already states the prohibition, and gives the reason: an id left as authored beside a generated `b0` puts two identifiers into the canonical output under one name, so a `#b0` reference has two candidate referents. But the prohibition is scoped to namespace MEMBERS, and a text node is not one — `definedId` returns undefined for it before any namespace test. So the reasoning applied and the rule did not. Measured before this change: {"blocks":[{"type":"paragraph","id":"real", "children":[{"type":"text","id":"b0","value":"x"}]}]} -> {"blocks":[{"children":[{"id":"b0",...}],"id":"b0",...}]} Two `b0`s in hashed content, no error. Item 5 now states the text-node case directly rather than leaving it to be inferred from a rule it is outside of, and the reference extends the existing check to it. The check runs over the CANONICAL content, so it reaches a nested text node, not only a root-level one. Mutation-verified both ways: removing the check fails the new test and only it; widening it to ban every text-node id fails the new test's preservation assertion plus two pre-existing ones, since a text-node id is content-significant and differing ids must yield differing document ids. #129's second defect — a text-node id may duplicate a block id — is NOT fixed here. It turns on whether text-node ids belong in the uniqueness namespace at all, which is a different question from whether they may collide with a generated name.
Three real defects, two of them mine from this session. The #129 check ignored the `attributes` barrier every other rule honours, so a schema-valid JSON-LD annotation carrying a text-shaped member with `id: "b0"` became a canonicalization error. That is a NEW false REJECT, the exact class this work exists to remove, introduced while fixing a different one. The check now tracks the barrier, with a control asserting the annotation case is accepted — mutation-verified: dropping the barrier fails it. Item 4 said the arrays are "a block's `caption`, `children`, `hints` or `preamble`". Four owner classes carry no `type` at all — `subfigure`, `exercisePart`, `solution`, a structured abstract's section — and the content root, which owns one, is not a block. An implementation keying on the owner rather than the field name skips a subfigure's `children` and computes a different document ID. BLOCK_CONTENT_KEYS' own comment warns of exactly this ("KEYS, NOT (type, key) PAIRS"); the erratum described that code and made the mistake it exists to prevent. Now keyed on the field name, with the untyped owners named. Five sites still quoted the referent the erratum replaced, two of them PUBLISHED: a fixture description and its generator quoted "the concatenated text content of each block" — a sentence `grep spec/` no longer finds. The reference and the oracle each carried the same stale quote, independently transcribed. And §5.4.2's consequence clause kept "a non-NFC block" after §5.4.3's was fixed, so two matching sentences diverged. §3.1's `id` row read "Unique block identifier within document", which now reaches text nodes and contradicts §3.2's carve-out four lines below. It points at §3.2 instead.
My own repo-wide grep after the previous commit found two the review missed: the raw-scan doc comment in the reference, and errors.json's consequence clause, which kept "a non-NFC block" after both its counterparts in 07 were fixed. A grep for every phrasing of the old referent now returns nothing across spec/, scripts/, conformance/ and the published artifacts.
The adversarial read caught it: the check tested `value.type === 'text'` anywhere in the tree, so a text-SHAPED member of a `semantic:bibliography` `entries` array was rejected — a data payload whose id item 5 explicitly leaves as authored, where nothing generated can collide with it. Shape-keying, written into the middle of a slice whose whole purpose is replacing shape-keying with position-keying. Now a member is only a text NODE where a text node can be: inside a block-content array, outside the `attributes` barrier. Both exemptions carry controls, and re-keying the check on shape fails the test.
Both review cycles raised it. "An extension block carrying rich text MUST name that array with one of these keys" binds no one a reader can check: a `vendor:callout` with `body:[text,text]` canonicalizes clean and unmerged, is a conformant document, and has no §5.4 row and no code path. A MUST whose violation produces a valid document is the defect #122 was filed about — "block-level content" was a MUST nothing defined. It was a statement of consequence dressed as a requirement, and now says so: name the array one of these keys and the transforms apply, name it anything else and they do not. That is what an extension author needs to know, and it is true.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #122, #126, and #129's first defect. Three spec decisions researched, decided, and written down, plus the one code fix that could not be separated from them.
#122 — "block-level content" was a MUST nothing defined
§7.2 rules 6 and 11 require table-cell and admonition children to be "block-level content". The term appears four times in
spec/and is defined nowhere, which makes both an unenforceable MUST. The admissibility question the issue asks is downstream of that.The answer the taxonomy already gives is yes: §4.1 "Text Leaf Nodes" sits under "## 4. Core Block Types", §1.4 calls a text node "the only block type that holds text", and
$defs/textNodecomposesblockBase, whose description reads "shared by every block". The issue's two premises are both false — there is no sentence saying a text node is not a block object, and a text node does carry the base fields.It is also already assumed by published artifacts: the KAT vector
reject-unpaired-surrogateshipsblocks:[{"type":"text","value":"\ud83d"}]and rejects it for the surrogate, not the position. Twenty-one arrays resolve their items to$defs/block— seven core, fourteen from extensions — and every one admits a text node.So the erratum states what is true rather than changing it, and carves text nodes out of the rules that would otherwise now reach them: item 5's relabeled namespace, §3.2's id requirements, and 03a §4's uniqueness bullets.
#126 — the narrowing in PR #128 had no licence in
spec/That PR scoped §4.3.1 item 4 and §4.3.2 item 2 to block-content positions in both implementations and changed no spec file. The transform preamble admits exactly one carve-out — "uniformly, but for the two id-relabeling exceptions item 5 names" — and item 4 is not one of them.
Item 5 is the template: it scoped by enclosing field, named its arrays exhaustively, and amended the preamble. Item 4 now does all three. Item 2's referent moves from "each block" to "each maximal run of adjacent text nodes within a block-content array", which licenses a run in the root array — the content root is not a block (§7.1 rule 5), so a run directly there had no referent at all, though both implementations reject one.
Membership is keyed on the field NAME, not the owner's type. Four owner classes carry no
type—subfigure,exercisePart,solution, a structured abstract's section — and an implementation keying on the owner computes a different document ID.#129 defect 1 — an authored
b0on a text nodeItem 5 already stated the prohibition and gave the reason: an id left as authored beside a generated
b0puts two identifiers into the canonical output under one name. But it is scoped to namespace MEMBERS, and a text node is not one. Measured before the fix: twob0s in hashed content, no error, so a#b0reference had two candidate referents.The check is keyed on position — a member is a text node only inside a block-content array, outside the
attributesbarrier — with controls for a JSON-LD annotation and a CSLentriesmember, both of which a shape test rejects.Review
Two
/review-fixcycles, the branch's cap. 19 findings, all closed or filed.Cycle 1 (citation audit + normative consistency) raised 11. Cycle 2 (closed-list verification + an adversarial implementer reading from the text alone) raised 8, three of which the first cycle's fixes had already closed.
Both cycles were overwhelmingly finding defects in this branch's own new prose, not in the codebase. Two are worth naming because they are the same mistake:
caption,children,hints" while describingBLOCK_CONTENT_KEYS, whose own comment reads "KEYS, NOT (type, key) PAIRS… several content arrays hang off sub-objects that carry notypeat all".entriesmember.The adversarial lens's verdict before those fixes was "no — a from-the-text implementation computes a different document ID for academic exercise content". After them, both readings converge.
Five sites still quoted the referent the erratum replaced, two of them published — a fixture description and its generator quoted a sentence
grep spec/no longer finds. A repo-wide grep for every phrasing now returns nothing.Filed, not fixed
isContentRefmatches ref paths by suffix rather than resolving them, so an array of a block-typed$defis invisible to the derivation.blockEntryrequiresidwhile §3.2 makes it optional, and a published fixture ships three id-less root members — andblockCountdoes not say whether it counts the stored or canonical form, which differ (3 → 2, measured).Verification
30/30 gates green, plus the CI steps that are not npm scripts (
npm audit --audit-level=high→ 0 vulnerabilities,generate:templateinto a fresh empty dir).check:conformance: 378/378 (215 vectors + 163 fixtures), 0 failed, 0 advisory, 0 skipped.errors.jsonunchanged at v0.14 / 86 codes — its note and summary were corrected, which its owncompatibilityPolicypermits without a bump.