Skip to content

§4.3.1 items 2 and 3 are keyed on element SHAPE, not position: a text-shaped object anywhere gets mark normalization — a false REJECT and a silent ID collapse #136

Open

Description

@gvonnessi

Measured on main at 8ce0b5d. Found by the #134 review panel.

The asymmetry

canon threads a blocked flag for the attributes barrier through every rule except one. The merge (mergeAdjacentText), the canonical-name prohibition (forbidCanonicalTextIds), the stored-byte scan (validateStoredByteInvariants) and the id walk (walkContentNodes) all consult it. Item 3's mark pipeline does not:

// scripts/lib/canonicalize.ts, canon()
case 'text':
  if (Array.isArray(obj.marks)) {
    const normalized = normalizeMarks(obj.marks, assetMap); // §4.3.1 item 3
    if (normalized.length === 0) delete obj.marks;
    else obj.marks = normalized;
  }
  break;

No blocked test. And the recursion below then skips the array entirely — if (type === 'text' && key === 'marks') continue; — so a barriered marks array is not even walked as ordinary data.

attributes.semantic is a JSON-LD annotation with additionalProperties: true, so an author may put {"type":"text","value":…,"marks":[…]} inside it. Membership here is keyed on element shape, not position — which is exactly the defect class #131 closed for item 4 and §4.3.2 item 2, one rule over.

Three measured consequences

  1. False REJECT (loudest). normalizeMarks resolves link-mark hrefs against the asset map, so a conformant document carrying
    attributes.semantic.sample = {"type":"text","value":"data","marks":[{"type":"link","href":"assets/images/not-registered.png"}]}
    throws asset reference "assets/images/not-registered.png" resolves to no registered asset. Content Blocks §3.1 closes attributes to authored values, so that string is data — a CSL or JSON-LD payload that merely looks like an archive path hard-fails canonicalization.
  2. Silent ID collapse. Two documents differing only in the order of such a marks array share one document ID.
  3. Silent data loss. A duplicate mark there is dropped; marks: [] has its key deleted.

What is NOT wrong

The reference is conformant to the specification as written, and the first proposed fix — threading blocked into canon's marks branch — would make it diverge. §4.3.1's preamble (spec/core/06-document-hashing.md, "Content transforms") says the transforms apply

uniformly, but for the position scoping item 4 names and the id-relabeling exceptions item 5 names

and item 4 names only itself and §4.3.2 item 2 as position-scoped. Items 2 and 3 are therefore uniform by the spec's own words, attributes included. So this is a specification question, not an implementation bug: should items 2 and 3 carry the barrier that items 4 and 5 carry?

Why it is worth answering

The rationale item 4 and item 5 already give for the barrier — that attributes members are data even when shaped like inline content, because §3.1 closes the field to authored values — applies word for word to items 2 and 3. If it is right there it is right here, and consequence 1 above is a conformant document rejected.

What a fix needs

Barrier work in this repository has an established standard, set by #121 and #125: enumerate every consumer of the premise before changing what enters the namespace, because widening or narrowing a transform's reach breaks consumers of its old boundary. That inventory has not been done for items 2 and 3.

Also note the near-miss guard: scripts/test-canonicalize.ts ("a `marks` array that is NOT a text node's is left unsorted") builds its annotation as {'@type':'X', marks} — never text-shaped, so it straddles the failing case without landing on it. Whatever is decided, that test needs the text-shaped arm.

Raised from the #134 panel; #134 itself only corrects a comment that asserted this protection holds.

Activity

  1. changed the title [-]§4.3.1 items 2 and 3 reach through the `attributes` barrier on a text-SHAPED member: a false REJECT and a silent ID collapse[/-] [+]§4.3.1 items 2 and 3 are keyed on element SHAPE, not position: a text-shaped object anywhere gets mark normalization — a false REJECT and a silent ID collapse[/+] on Sep 5, 2026
  2. gvonnessi commented on Sep 5, 2026

    @gvonnessi
    CollaboratorAuthor

    Correction: this is wider than attributes, and the original title was under-scoped

    Measured during the #134 cycle-2 review. The issue as filed described this as the attributes barrier failing to reach items 2 and 3. That is one instance, not the rule.

    canon's case 'text' is a switch (type) arm with no position test of any kind. It is not that the barrier fails to reach it — it is that membership here is keyed on element shape, so the normalization fires wherever a text-shaped object sits, whether or not attributes is anywhere in the document.

    The witness that makes the point, with no attributes involved

    {"type":"semantic:bibliography","id":"bib","entries":[
      {"id":"smith2020","type":"article-journal",
       "note":{"type":"text","value":"n","marks":["italic","bold"]}}]}

    Canonical output has note.marks as ["bold","italic"], and the two authored orders collapse onto one document ID. An extension block's own data array ({"type":"ext:thing","payload":[{"type":"text",…,"marks":[…]}]}) collapses identically.

    A CSL entry is precisely what canonicalize.ts's own comment at the rewriteIds marks branch names as a non-text vehicle whose order is left alone. It is not left alone.

    The repository already contains the correct definition

    alphaRenameIds' forbidCanonicalTextIds states it: "Only a member of a block-content array is a text NODE. A text-shaped object anywhere else is data." Items 4 and 5 are built on that definition. Items 2 and 3 are not, and the two comments therefore disagree with each other inside one module.

    This is the same defect class #131 closed for item 4 and §4.3.2 item 2 — shape-keyed where it should be position-keyed — surviving in the two rules that change was not scoped to.

    Scope correction for any fix

    The reach is not only canon. Item 5's post-rewrite pass calls sortDedupMarks on any object with type === 'text' as well, so a consumer inventory has to cover both passes, not just canon's case 'text'.

    Everything in the original report stands — the false REJECT on an unregistered assets/ href, the ID collapse, the dropped duplicate and deleted empty array, and the point that the reference is conformant to the specification as written because §4.3.1's preamble makes items 2 and 3 uniform. What changes is the size of the surface: every position a text-shaped object can occupy, not one field.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions