Smartphrase: document anchoring is coupled to the concept entity — half of phrase uses have no path back to the SAP text

[metadatadriven]

Parent: #9

Finding

Document anchoring in the smartphrase layer is carried by a single field — sapRef on study-graph concepts — and concepts are only one of the four binding kinds a phrase instance can use. Everything the prose asserts that does not route through a concept binding has no typed path back to the SAP text that grounds it.

This surfaced in a full-study encoding exercise against the methods_02 artefacts (Transformation Library v0.7, Method schema v0.9.1): a 2×2 crossover SAP encoded end-to-end as 96 phrase-template instances (85 analyses + 11 derivations), with a viewer that highlights each instance's sapRef quotes in the source document. Method phrases visibly never highlighted anything; a systematic audit showed the method case is just the most visible member of a class.

Audit (all 96 instances)

1. Bindings by kind — three of four have no document path

Binding kind	Uses	Path to SAP text
concept	400	✅ via the bound concept's sapRef
method	89	✗ resolves into the library; the library wall (correctly) forbids study text
output	85	✗ output classes are library entities too
value	74	✗ raw configuration values ("95") — yet "95% confidence intervals" is a sentence in the SAP

2. Fixed-text phrases — no bindings, so nothing to anchor through. Seven phrases, 156 uses, whose entire text is a claim about what the SAP says (e.g. "adjusting for period", "with a random participant effect", "with follow-up time as an offset"). Each mirrors a specific SAP sentence; none can cite it.

Together: 404 of ~800 phrase uses — about half the smartphrase spans in any generated sentence — have no corresponding anchor in the source document.

3. Entity classes beyond phrases

• Estimands: 0 of 40 carry a sapRef. Their ranks and ICE scopes are encoded straight from the SAP's estimands section — exactly the text a reviewing statistician wants to check — and nothing quotes it. (The existing demo study graphs embed section numbers informally in estimand labels, as strings.)
• Instances: 0 of 96. The section reference lives untyped inside the instance label ("… (SAP 15.10.3)"), so tooling has to regex it out.
• Trace templates anchor to datasets/variables — a different axis, by design; no issue there.

Root cause

sapRef is a property of the concept entity, when document grounding is really a property of the study-side use. The boundary is right in principle — library entities (methods, output classes, templates) are study-agnostic and must not carry study text (acdc_method.yaml: "Methods MUST NOT reference clinical or analysis concepts") — but the anchor field was attached to one entity class on the study side rather than to the use itself.

Evidence the gap forces bad workarounds: the encoding exercise ended up smuggling the SAP's model-choice sentence into the proposed method's description field — study text inside a library entity, the wrong side of the very wall the model draws — because the quote had nowhere typed to go.

Requirement (in the spirit of the parent issue: requirement, not solution)

Every assertion the generated prose makes about the study — the choice of method, a configuration value, a summary measure, a fixed methodological qualifier, an estimand's scope — should be traceable to the document text that grounds it, with the same fidelity sapRef already gives concepts. The traceability-to-data claim (#9 claim 2) has a missing twin: traceability to the source document.

Candidate direction (design input to weigh, not a constraint)

One change appears to close the whole class without touching the library wall: allow sapRef on the phrase instance (the {phrase, bindings} object in the study graph — already study-side). That covers method, value, output and fixed-text uses uniformly, and lets a concept-bound use carry a use-specific quote where the concept's own quote is too generic. Two smaller companions:

• sapRef on estimands (study entities, same footing as concepts);
• a typed sectionRef on instances, replacing the label-embedded "(SAP n.n)" convention.

Precedence when both exist (instance-level vs concept-level sapRef) is a design decision to record at that point.

[metadatadriven]

Confirmed on estimands_01, and a plan

The audit reproduces here. Same shape, smaller numbers:

Binding kind	Uses	Anchored
concept	38	18
method	8	0
value	7	0
output	6	0

Plus 0 of 6 estimands and 0 of 8 analysis instances. Plan committed at smartphrase/PLAN-anchoring.md; ten TDD-ordered tasks.

Two additions to the diagnosis

sapRef appears in zero pinned outputs. It is emitted by no projection — not constructModelView, not toJSONLD, nothing. So it is inert: nothing consumes it, nothing displays it, and no gate can detect it regressing. I'd offer that as a deeper statement of the root cause than "attached to the wrong entity class". Anchoring was never wired into the one-state/many-projections architecture at all, which is why three of four binding kinds could go unanchored without anyone noticing. The fix therefore has to make anchors a projection, not just add a field — otherwise the same blindness persists in a new location.

The label-embedded convention the issue objects to is ours, and recent. "Primary estimand — progression-free survival (SAP 3.1, 7.7.2)" was written into study-graph-pre0102.js during #11, days ago. Noting it because it shows how fast the workaround reappears when there is nowhere typed for the reference to go — which is the issue's own argument, demonstrated twice independently.

Design choices worth flagging before the work lands

One anchor type, four homes. { section, quote } on concepts, phrase instances, estimands and analysis instances. This folds the issue's suggested separate sectionRef into the same type with quote optional, rather than adding a second field — an instance anchor is usually section-only. One shape to learn, one resolver.

Quotes are verified against the source, not merely stored. Because the converted SAP is in the repo, the gate can assert that every quote appears verbatim in the section it cites, after normalising whitespace and the typographic punctuation the PDF conversion produced. This is the part that makes the fix worth more than a new field: it turns "traceable to the document" into a checked fact and makes a paraphrase a build failure.

It has already earned its place in the plan. TRT.PRE0102's existing sapRef reads "randomized 1:1 to everolimus or placebo, both with fulvestrant" — a paraphrase, not a quotation, sitting in the repo since #9 and looking exactly like every real quote around it. The gate catches it on first run.

Precedence: both retained, instance-level wins for display. The concept's anchor answers "what is this?"; the use's answers "why is it here?". A use-specific quote should not silently erase the definitional one.

Coverage is gated per study, keyed on a declaration. A study states sourceDocument, or null with a reason. PrE0102 is then held to a hard requirement — every phrase use anchored — while the CDISC Pilot, whose study layer is constructed and has no SAP behind it, is exempt by declaration rather than by silence.

The worked examples come from one sentence

SAP §7.7.2: "Median time and 90% confidence interval for PFS, TTP, and OS will be summarized for all eligible, treated subjects by treatment arm using Kaplan-Meier estimates." That single sentence grounds the method binding, the value binding (90) and the output binding (median_survival) — three of the four unanchorable kinds, in real source text rather than illustrative text.

Fixed-text phrases get two worked examples, also real. Our library has only one fixed-text phrase and no instance uses it, so the class was unexercised here. Two are being added from §7.7.2, both with no placeholders:

• SP_KM_CURVES — "In addition to the summary table, PFS and OS will be displayed by treatment arm using Kaplan-Meier survival curves." Bound to the PFS and OS analyses only, not TTP, because the SAP names those two. Getting that right is itself a check that the encoding follows the document rather than the pattern.
• SP_CENSOR_LTFU — and this is the sharpest demonstration available of why the anchor belongs on the use. One phrase, one library definition, identical rendered text, bound to three instances, each citing a different clause of the same sentence: "censored at the time of last tumor assessment for TTP and PFS" for two of them, "at the time of the last known contact for OS" for the third. A concept-level anchor cannot express that, and there is no concept to hang it on in the first place.

Related, smaller

We have a contained instance of the leak the issue describes. acdc-library-proposed.js carries study text in provenance.rationale ("The SAP's PFS analysis is descriptive…"). It is defensible where it sits — provenance is metadata about why a proposal exists, not a library semantic field — but it must not travel upstream when T.PFS_KaplanMeier is contributed, so it is being marked NOT FOR UPSTREAM explicitly.

Open questions carried into execution

1. Is method_qualifier the right role for SP_KM_CURVES? It is a reporting qualifier, not a methodological one. A dedicated reporting role would be more honest but is another library minor version. Flagging rather than deciding.
2. Should the tag dialect serialise anchors? The plan says no — a quotation is not an attribute value — and instead preserves anchors across a round-trip so an edit through the text surface cannot silently strip provenance. If the working group wants that surface to be provenance-complete, that changes.
3. Should a released library require an anchor for phrases whose text asserts something about a document? The fix keeps the library untouched by putting sapRef on the study side. But a fixed-text phrase like "adjusting for period" is a claim about a document by construction, and arguably ought to be unusable without a citation. That is a library-governance question, and it sits alongside the ownership question already open on Smartphrase: demonstrate SAP <-> AC/DC model linkage (PoC for the working group) #9.

Landing on estimands_01, since the estimand and instance halves of the fix only exist there.

[metadatadriven]

Reproducing the audit

The numbers in the issue body come from running the audit below over a full-study graph in a downstream encoding repo (private). It is generic — it runs against any study graph that follows the methods_02 demo data shape — so the same measurement can be taken on the in-repo demo graphs (study-graph.js, study-graph-pre0102.js) or any future encoding.

Method: enumerate every phrase-instance binding by kind (concept / method / value / output), flag phrase uses with no concept binding (whole phrase un-anchorable through the current mechanism), and check each study-graph entity class for a sapRef field.

// anchor-audit.mjs — node anchor-audit.mjs
// Point the three load() lines at the demo data and a study graph.
import { readFileSync } from "node:fs";
import vm from "node:vm";
const load = p => vm.runInThisContext(readFileSync(p, "utf8"), { filename: p });
globalThis.window = globalThis;
load("demo/data/acdc-library.js");
load("demo/data/acdc-library-proposed.js");
load("demo/data/study-graph-pre0102.js");   // ← the graph under audit
const G = Object.values(globalThis.STUDY_GRAPHS)[0];

const insts = [...(G.instances || []), ...(G.derivations || [])];
const byKind = { concept: 0, method: 0, value: 0, output: 0 };
const unanchored = {};   // phrase OID → uses with no concept binding
for (const inst of insts) for (const ph of inst.phrases || []) {
  const bs = Object.values(ph.bindings || {});
  for (const b of bs) {
    if (!b) continue;
    if (b.concept !== undefined) byKind.concept++;
    else if (b.method !== undefined) byKind.method++;
    else if (b.value !== undefined) byKind.value++;
    else if (b.output !== undefined) byKind.output++;
  }
  if (!bs.some(b => b && b.concept !== undefined))
    unanchored[ph.phrase] = (unanchored[ph.phrase] || 0) + 1;
}
console.log("bindings by kind:", byKind, "— only `concept` reaches a sapRef");
console.log("phrase uses with no concept binding:", unanchored);
console.log("concepts with sapRef:",
  Object.values(G.concepts).filter(c => c.sapRef).length, "/", Object.keys(G.concepts).length);
console.log("estimands with sapRef:",
  Object.values(G.estimands || {}).filter(e => e.sapRef).length, "/", Object.keys(G.estimands || {}).length);
console.log("instances with sapRef:",
  insts.filter(i => i.sapRef).length, "/", insts.length);

Two observations from running it at study scale that a single-passage demo would not surface:

1. The anchor-less share is not a corner case — it grows with encoding depth. Method qualifiers ("adjusting for period", "with a random participant effect", offsets, windowing rules) are fixed-text phrases, and a real SAP's methods sections are dense with exactly those.
2. The viewer consequence is immediate and visible: with sapRef quotes highlighted in the source document, every concept-bound span lights up and every method/value/output/fixed-text span silently doesn't — a reviewer notices within minutes, which is how this issue was found.

[metadatadriven]

Fixed on estimands_01

Ten commits, 9f39155..c78e790. PrE0102: 33 of 33 phrase uses anchored, every quote verified verbatim against the converted SAP. The CDISC Pilot is exempt by declaration. All three gates green.

Before: concept 18/38 anchored, method 0/8, value 0/7, output 0/6, estimands 0/6, instances 0/8.

What changed

One anchor type, four study-side homes. { section, quote? } on concepts, phrase instances, estimands and analysis instances. The suggested separate sectionRef is folded into the same type with quote optional rather than added as a second field. Precedence: the phrase instance's anchor wins for display ("why is this here?"), a bound concept's is the fallback ("what is this?"), and both survive — anchorSource records which applied. The library wall is untouched, which was the point.

Anchors are now projected. This turned out to matter more than the field placement. sapRef appeared in zero pinned outputs — emitted by no projection, so no surface could show it and no gate could detect it regressing. Adding a field alone would have reproduced that blindness in a new location. Anchors now appear in the model view (sapRef, documentAnchors[]) and in the graph as prov:wasQuotedFrom — the W3C term for an entity derived by quoting another, so the identifier policy is satisfied rather than an AC/DC term invented.

Quotes are verified, not merely stored. tools/verify.mjs asserts that every quote appears verbatim in the section it cites. An anchor is now a checked fact and a paraphrase is a build failure.

It earned its place on the first run. TRT.PRE0102's sapRef read "randomized 1:1 to everolimus or placebo, both with fulvestrant" — a paraphrase, in the repo since #9, sitting among real quotations and indistinguishable from them by eye. Replaced with real §4.1 text. The planted-fault block now pins that the gate rejects it, rejects invented text, and rejects a citation to a subsection that does not exist.

Coverage gated per study, keyed on a declaration. A study states sourceDocument or null with a reason, so the Pilot's exemption is declared rather than inferred from silence.

The fixed-text case, which is the sharpest demonstration

Two phrases added from §7.7.2, both with no placeholders at all — the class with no binding for any concept anchor to route through.

SP_CENSOR_LTFU is bound to all four time-to-event analyses. One phrase, one library definition, identical rendered text — and the overall-survival analysis cites a different clause of the same sentence:

• PFS / TTP → "Subjects who are lost to follow-up are censored at the time of last tumor assessment for TTP and PFS"
• OS → "at the time of the last known contact for OS"

No concept-level anchor could express that, and there is no concept to hang one on. SP_KM_CURVES is bound to PFS and OS but not TTP, because the SAP names only those two — following the document rather than the pattern.

The other three unanchorable kinds are all grounded by one sentence: "Median time and 90% confidence interval for PFS, TTP, and OS will be summarized … using Kaplan-Meier estimates" anchors the method binding, the value binding (90) and the output binding (median_survival).

Two things worth reporting back

The label-embedded convention was ours, and days old. "Primary estimand — progression-free survival (SAP 3.1, 7.7.2)" went into this repo during #11, before the issue was raised. Independent evidence for the issue's own argument: the workaround reappears immediately when there is nowhere typed for a reference to go. It is gone, and a gate now rejects it.

One case in quote verification is genuinely ambiguous. The PDF wrapped mid-word, so §7.7.2 contains "Kaplan-\nMeier" and the KM-curves quote failed on first run. A line-break hyphen may belong to the word (Kaplan-Meier) or be pure typesetting (random-/ised), and nothing in the text says which. Rather than guess, both readings are tried and either may match — that cannot admit a wrong quote, since the text must still equal one of two legitimate de-hyphenations of what the document actually contains. Anyone reusing this against a differently-converted document should know the case exists.

Also

method_qualifier became a repeating role in practice once the fixed-text qualifiers landed, and three of them ran together with bare spaces; given a ", " conjunction in all three language packs. The JSON-LD anchor lookup was wrong on first cut — it paired resolved phrases to instances by oid, which breaks for a repeating role using one oid twice — now fixed by a shared anchorForPhrase() used by both the prose and the graph, so the two cannot disagree about provenance.

The overlay's provenance block is marked NOT FOR UPSTREAM. It necessarily contains study text, which is fine as proposal metadata but must stay behind when the entities are contributed — the same leak the issue describes, contained rather than the wall relaxed.

Open questions

1. Is method_qualifier the right role for SP_KM_CURVES? It is a reporting qualifier, not a methodological one. A dedicated reporting role would be more honest but is another library minor version.
2. Should the tag dialect serialise anchors? It does not — a quotation is not an attribute value — and anchors are instead carried across a round-trip by matching phrase oid plus bindings, so an edit through the text surface cannot silently strip provenance. If that surface should be provenance-complete, this changes.
3. Should a released library require an anchor for phrases whose text is a claim about a document? A fixed-text phrase like "adjusting for period" asserts something the SAP says, by construction. The fix keeps the library untouched by putting sapRef on the study side, but arguably such a phrase should be unusable without a citation. A governance question, next to the library-ownership one already open on Smartphrase: demonstrate SAP <-> AC/DC model linkage (PoC for the working group) #9. Added as an ask in the walkthrough.

Design recorded as D17 and D18 in smartphrase/DESIGN.md; the anchor type and source-document declaration are documented in REFERENCE.md §3.4.

[metadatadriven]

Downstream adoption report — the fix holds at study scale

The estimands_01 anchoring fix (D17/D18, tip c78e790) has been adopted end-to-end in the full-study encoding that raised this issue. Results, as evidence the design works beyond the PrE0102 exemplar:

Scale. 44 concepts, 40 estimands, 96 analysis instances/derivations, and ~425 phrase-use anchors — roughly 600 typed anchors in total. The uses were anchored via a shared catalogue of ~23 verified quotes applied declaratively per transformation template: one SAP sentence licenses a method use and its fixed-text qualifiers ("adjusting for period", "with a random participant effect"), because it is the sentence that asserts all three. That catalogue-per-template shape kept the migration to a single pass rather than 400 hand edits — worth noting for anyone adopting at scale.

The quote gate earns its keep. Every quote verified verbatim on the gate's first run — and that is because writing them against a verifying gate forces the discipline; the encoding's previous free-text refs included paraphrases, editorial brackets and ellipsis-spliced fragments that the typed form simply refuses. Mutation test: corrupting one character of one quote produced 183 failures.

Two adoption findings:

1. The conversion must mark subsection headings. sectionsIn resolves cited sections through Markdown headings; our pdftotext conversion originally marked only top-level sections, so every subsection citation failed until the converter was taught to emit them (cross-checked against the SAP's own TOC — 129 subsections, plus camelCase instrument names like "iscorEB" and digit-led titles like "2x2 Crossover trial" that naive heading heuristics miss). Perhaps worth a line in the conversion standard.
2. Section-only anchors are the right escape valve. Two parameters are quoted from a table whose row text the -layout conversion interleaves with data columns — no contiguous verbatim quote exists. {section} without quote, with the situation recorded in a note, encodes that honestly; the optional-quote design choice was correct.

Viewer consequence, closing the loop on how this issue was found: with per-use anchors, every smartphrase span in the generated sentence now lights up in the source document — method, value, output and fixed-text uses included. The visible asymmetry that exposed the gap is gone.

The one residual from the original report: the study-side workaround that had smuggled a SAP sentence into a proposed method's description is removed — grounding moved to the typed anchors, with remaining study references in the proposed overlay marked as NOT-FOR-UPSTREAM provenance per the same convention this repo's overlay uses.