You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Smartphrase: single sapRef + first-match-wins anchoring sends reviewers to missing or less-relevant document text — support multiple references per smartphrase #13
Human review feedback from the full-study encoding exercise (the one audited in #12: a 2×2 crossover SAP, 96 phrase-template instances, viewer highlighting each instance's sapRef quotes in the source document). An independent statistical reviewer, working through the viewer analysis-by-analysis, reported that smartphrase→document linkage is flaky in a specific, repeatable way: for some analyses a phrase span links to nothing at all, and for others it links to a less relevant part of the SAP than the passage that actually specifies the output under review. Their suggested remedy, verbatim: "I think this could be fixed with multiple references within a smartphrase."
This is the relevance/multiplicity twin of #12. #12 asks that every phrase use have a path back to the document; this issue is that even where a path exists, one sapRef per entity plus first-match-wins resolution routinely produces the wrong one.
Mechanics
Two model properties combine to cause it:
1. sapRef is single-valued everywhere it exists. A concept carries one definitional quote; a phrase instance (per #12's candidate direction, prototyped in the demo engine) carries one use-specific quote. But a concept is designed to be reused — the same parameter/population/method-qualifier concept is bound by dozens of analyses whose specifying text lives in different SAP sections. One quote cannot be the right anchor for all of them.
2. Anchor resolution is first-match-wins. The demo engine's anchorForPhrase resolves: phrase-instance sapRef if present, else the first bound concept that has one — then stops. A phrase with several bindings discards every anchor after the first, and the binding order (an authoring accident) decides which passage the reviewer is sent to.
Observed failure modes
Generic-definition capture. A concept defined with its quote from the section where it first appears (e.g. the primary-analysis section) anchors every later use — so an analysis specified in a different section links back to the primary-analysis text instead of its own specifying passage. The reviewer reads the wrong justification for the right sentence.
Secondary anchors invisible. Where a use genuinely rests on two passages (a definition and a section-specific qualification — common for conditional analyses), the model can represent only one, so the viewer can show only one.
A smartphrase use should be able to carry all the document references that ground it, and a viewer/reviewer should be shown the reference(s) most relevant to the specific analysis being inspected — not whichever single quote happened to be attached first at authoring time.
Candidate directions (design input to weigh, not constraints)
Pluralise the field: sapRef → sapRefs[] on concepts and phrase instances (back-compatible: a single string reads as a one-element list). Viewer shows all, ordered.
Union resolution: replace first-match-wins in anchorForPhrase with collect-all — instance refs plus every bound concept's refs, deduplicated — so no anchor is silently discarded. Cheapest change; pairs naturally with either of the above.
Precedence/ordering when a use has instance-level and concept-level refs is the same open design decision #12 records; whatever is decided there should hold here.
Acceptance sketch
The reviewer scenario above stops reproducing: every phrase span in the exercise's 96 instances links to at least one passage, and spot-checks on reused concepts land on the section that specifies that analysis.
Parent: #9
Finding
Human review feedback from the full-study encoding exercise (the one audited in #12: a 2×2 crossover SAP, 96 phrase-template instances, viewer highlighting each instance's
sapRefquotes in the source document). An independent statistical reviewer, working through the viewer analysis-by-analysis, reported that smartphrase→document linkage is flaky in a specific, repeatable way: for some analyses a phrase span links to nothing at all, and for others it links to a less relevant part of the SAP than the passage that actually specifies the output under review. Their suggested remedy, verbatim: "I think this could be fixed with multiple references within a smartphrase."This is the relevance/multiplicity twin of #12. #12 asks that every phrase use have a path back to the document; this issue is that even where a path exists, one
sapRefper entity plus first-match-wins resolution routinely produces the wrong one.Mechanics
Two model properties combine to cause it:
1.
sapRefis single-valued everywhere it exists. A concept carries one definitional quote; a phrase instance (per #12's candidate direction, prototyped in the demo engine) carries one use-specific quote. But a concept is designed to be reused — the same parameter/population/method-qualifier concept is bound by dozens of analyses whose specifying text lives in different SAP sections. One quote cannot be the right anchor for all of them.2. Anchor resolution is first-match-wins. The demo engine's
anchorForPhraseresolves: phrase-instancesapRefif present, else the first bound concept that has one — then stops. A phrase with several bindings discards every anchor after the first, and the binding order (an authoring accident) decides which passage the reviewer is sent to.Observed failure modes
Requirement (requirement, not solution — per #9)
Candidate directions (design input to weigh, not constraints)
sapRef→sapRefs[]on concepts and phrase instances (back-compatible: a single string reads as a one-element list). Viewer shows all, ordered.sectionRefproposed in Smartphrase: document anchoring is coupled to the concept entity — half of phrase uses have no path back to the SAP text #12) — nearest-section wins for display, others remain one click away.anchorForPhrasewith collect-all — instance refs plus every bound concept's refs, deduplicated — so no anchor is silently discarded. Cheapest change; pairs naturally with either of the above.Precedence/ordering when a use has instance-level and concept-level refs is the same open design decision #12 records; whatever is decided there should hold here.
Acceptance sketch