From cd09ce9a4f3f5e3a03b674e4d4fffab83a1a66ab Mon Sep 17 00:00:00 2001 From: ejsyx Date: Mon, 17 Aug 2026 14:03:34 +0200 Subject: [PATCH 01/58] docs: design spec for inlined multivalued element identity Element identity per inlined multivalued slot comes from a key, a unique_keys composed key, or a diff.linkml.io/opaque annotation; an opt-in linter flags models that still allow ambiguous deltas. Co-Authored-By: Claude Fable 5 --- ...ned-multivalued-element-identity-design.md | 248 ++++++++++++++++++ 1 file changed, 248 insertions(+) create mode 100644 docs/superpowers/specs/2026-08-17-inlined-multivalued-element-identity-design.md diff --git a/docs/superpowers/specs/2026-08-17-inlined-multivalued-element-identity-design.md b/docs/superpowers/specs/2026-08-17-inlined-multivalued-element-identity-design.md new file mode 100644 index 0000000..11f4d20 --- /dev/null +++ b/docs/superpowers/specs/2026-08-17-inlined-multivalued-element-identity-design.md @@ -0,0 +1,248 @@ +# Element Identity for Inlined Multivalued Slots — Design + +**Status:** Draft, awaiting user review before plan-writing. +**Date:** 2026-08-17 +**Branch:** `feat/inlined-multivalued-element-identity` +**Supersedes:** the abandoned shapes work on branch `feat/container-shapes-and-verify-old` (see "Recyclable material" at the end). + +## Goal + +Answer the question, for each inlined multivalued shape: **where does element identity come from?** + +The offered options are: + +- a **key** (or identifier) — a slot on the element class, +- a **composed key** (content) — declared with the existing LinkML meta `unique_keys`, +- **opaque** — nowhere; the value is replaced as a whole. + +"Where does identity for an element come from" should be easy to answer for a data model author. + +## Non-goal + +The inferred semantics do **not** change for downstream projects not opting in to the linter. The positional index path deltas produced today are still valuable for projects not dealing with multiple sources producing deltas for the same object at the same time. + +## Proposed solution + +Inlined multivalued slots cause ambiguous deltas downstream. + +1. We add a **`diff.linkml.io/opaque` annotation**, with the meaning: **stop all recursion, replace the whole value.** +2. Together with **new support for the existing LinkML meta `unique_keys`**, allowing to declare composed keys. +3. An **opt-in linter** warns when your data model still allows for ambiguous deltas in multi-sourced operation. +4. We provide **examples explaining the nuanced options** a data model author has when the linter flags ambiguity (worked out below on the constituting examples). + +Consequence of this strict mode is that some LinkML schemas need to be reworked to be compliant, and more cases of patches (produced outside this diff lib) cannot be applied fully. + +## Problem statement + +When the element class of an inlined multivalued slot carries no key or identifier, the diff falls back to positional index paths (`src/runtime/src/diff.rs:267-296` — `for i in 0..max_len`, numeric path segments). A positional path is only meaningful against the exact list the producer saw. In multi-sourced operation the golden record has drifted by the time a patch arrives, and the index silently selects a different element — no error, wrong data. + +Both constituting examples are real, from the asset360 model at +`consolidator-server/components/py/asset360-model/asset360_model/schemas/asset360/repository/v1.0.0/`. + +### Example 1 — phone numbers: data loss and duplication under current inferred semantics + +`Service.hasPhoneNumber` is an inlined list of anonymous value objects (`tunnels.yaml:1954-1999`, locale annotations elided): + +```yaml + Service: + attributes: + hasPhoneNumber: + description: Phone number of the resource. + range: ServicePhoneNumber + multivalued: true + inlined_as_list: true + + ServicePhoneNumber: + description: >- + A phone number associated with a Service, classified by its function + (emergency, non-urgent, operator, etc.). Inlined into Service. + attributes: + phoneNumber: + range: string + hasNumberFunction: + range: NumberFunction + required: true +``` + +`ServicePhoneNumber` has no `identifier`, no `key`, and the schema declares no `unique_keys` anywhere. Elements have no stable identity at all — yet the model *does* have an identity rule in mind. It lives in hand-written SHACL (`constraints.shacl.ttl:546-612`), invisible to LinkML: + +```turtle +asset360:Service_OnePhoneNumberPerFunctionShape + a sh:NodeShape ; + sh:targetClass asset360:Service ; + sh:message "A service cannot have two phone numbers with the same function."@en ; + sh:property [ + sh:path asset360:hasPhoneNumber ; + sh:qualifiedValueShape [ sh:path asset360:hasNumberFunction ; sh:hasValue "Emergency_Number" ] ; + sh:qualifiedMaxCount 1 + ] ; + # ... one qualifiedMaxCount block per NumberFunction value ... +``` + +So `hasNumberFunction` *behaves* as a key — but LinkML sees an unkeyed list, and the diff sees positions. (The ingestion layer even carries an external identity per phone number — `source_id_key: new_uri` in `changeset-generator/.../ce_kwoa_a1552/service_phone_number.yaml` — which is dropped at the schema boundary.) + +**Data loss.** Two sources both derived their deltas from the same snapshot: + +```json +"hasPhoneNumber": [ + {"phoneNumber": "09/241.25.00", "hasNumberFunction": "Emergency_Number"}, + {"phoneNumber": "09/241.25.03", "hasNumberFunction": "Non_Urgent_Communication"} +] +``` + +- Source A corrects the non-urgent number: `Update hasPhoneNumber/1/phoneNumber` → `"09/241.25.99"`. +- Source B removes the emergency number. Positionally that is a cascade: `Update hasPhoneNumber/0/*` (the non-urgent content shifts into index 0, with the *stale* phone number `09/241.25.03`) plus `Remove hasPhoneNumber/1`. + +Apply A then B: B's cascade overwrites index 0 with the stale snapshot value — A's correction is silently reverted. Apply B then A: A's path `hasPhoneNumber/1` now points at nothing (or at whatever drifted in) — the edit is lost or lands on the wrong element. Either order corrupts; neither reports an error. + +**Duplication.** Two ingest sources independently discover the same new operator number and both emit `Add hasPhoneNumber/2 = {"phoneNumber": "09/241.25.10", "hasNumberFunction": "Operator"}`. Both apply; the golden record now holds the same phone number twice. Under key-based identity the second add would have been recognised as the same element. + +### Example 2 — `PositioningSystemCoordinate`: one class, two identity shapes + +`PositioningSystemCoordinate` declares a key — `typeURI`, the type designator, which plays the role of the coordinate-system discriminator (`rsm.yaml:610-625`, `asset360.yaml:541-550`, elided): + +```yaml + PositioningSystemCoordinate: + is_a: ObservableProperty + description: A tuple of coordinates in a given positioning system. + slots: + - typeURI # range: uri, designates_type: true + attributes: + PositioningSystemCoordinate_positioningSystem: + range: PositioningSystem + slot_usage: + typeURI: + key: true +``` + +**Shape B — keyed dict (the key is real).** `SpotLocation` holds at most one coordinate per coordinate-system type (`rsm.yaml:678-696`): + +```yaml + SpotLocation: + is_a: BaseLocation + attributes: + SpotLocation_coordinates: + multivalued: true + inlined: true + inlined_as_list: false # JSON object keyed by typeURI + range: PositioningSystemCoordinate +``` + +Real committed data (`asset360-model/tests/data/signal-obj-with-track.json:38-96`, trimmed) — one `LinearCoordinate`, one `GeographicCoordinate`, keyed by class URI: + +```json +"SpotLocation_coordinates": { + "https://data.infrabel.be/asset360-rsm-subset/LinearCoordinate": { + "measure": {"Quantity_unit": ".../unit/Kilometer", "NumericQuantity_value": 526.0}, + "typeURI": "http://rsm.uic.org/RSM12#EAID_CB107995_3610_4622_824B_708281B24CEA" + }, + "https://data.infrabel.be/asset360-rsm-subset/GeographicCoordinate": { + "typeURI": "https://data.infrabel.be/asset360-rsm-subset/GeographicCoordinate", + "latitude": 50.820240734349845, + "longitude": 4.316083008990688 + } +} +``` + +~25k records rely on this keyed behaviour. (The same pattern recurs one level up: `locations` is a dict of `BaseLocation` keyed by the `locationrole` enum slot, `rsm.yaml:734-742`.) + +**Shape A — vertex ring (the key is constant).** `Polyline` and `Polygon` hold the *same* class as an ordered ring (`rsm.yaml:567-608`): + +```yaml + Polyline: + is_a: NamedResource + attributes: + PolyLine_coordinates: + range: PositioningSystemCoordinate + multivalued: true + inlined_as_list: true + + Polygon: + is_a: NamedResource + attributes: + Polygon_coordinates: + range: PositioningSystemCoordinate + multivalued: true + inlined_as_list: true +``` + +Every vertex of a ring is the same coordinate subclass, so the declared key (`typeURI`) is **constant across elements**. Position is the only identity, ring data legitimately repeats the key value (a closed ring may even repeat a whole vertex), and vertex order is meaningful. + +One class, two identity shapes. That ring data violates any `unique_keys` the class would declare — so the class can't declare one. **Remodelling into different classes is the only solution**: the keyed lookup usage keeps a class whose key/`unique_keys` declaration is truthful for all of its data, and the ring usage gets its own class (an un-keyed vertex) whose containers are declared opaque. We do not compromise on the goal by letting one class carry contradictory identity declarations per slot. + +## The options when the linter flags a slot + +The linter's question is always the same — *where does element identity come from?* — and the author has exactly three answers, plus a rework escape hatch. Worked out on the constituting examples: + +### Option 1 — a key (or identifier): give elements the identity they already have + +`hasNumberFunction` is already the de-facto key (the SHACL shape says so). Declare it: + +```yaml + ServicePhoneNumber: + attributes: + phoneNumber: + range: string + hasNumberFunction: + range: NumberFunction + key: true + required: true +``` + +Deltas become key-addressed (`hasPhoneNumber/Emergency_Number/phoneNumber`), immune to drift and reorder; the duplicate-add collapses into an update of the same element. The SHACL uniqueness rule is now also expressed in the schema itself. + +### Option 2 — a composed key (content): declare `unique_keys` + +When no single slot identifies an element but a combination does, declare the existing LinkML meta `unique_keys` on the element class: + +```yaml + ServicePhoneNumber: + unique_keys: + phone_identity: + unique_key_slots: + - hasNumberFunction + - phoneNumber + attributes: + ... +``` + +Element identity is the composed key — here effectively the element's content. Elements are matched across versions by that identity; an edit of a key-constituent is a remove-plus-add of the whole element, an edit of a non-key slot is addressed at the element. Reorder is not a change. + +### Option 3 — opaque: identity comes from nowhere, say so + +For the vertex ring, elements genuinely have no identity — a moved vertex, an inserted vertex, a reversed ring are all edits *of the geometry*, not of a vertex: + +```yaml + RingVertex_coordinates: + range: RingVertex + multivalued: true + inlined_as_list: true + annotations: + diff.linkml.io/opaque: true +``` + +All recursion stops at the slot: any change below it is exactly one `Update` at the slot path carrying the whole old and new value. Two sources editing the same ring conflict visibly at the ring level — whole value against whole value — instead of interleaving vertex indices into a corrupt geometry. + +### Option 4 — remodel: when one class needs two answers + +`PositioningSystemCoordinate` needs option "key" in `SpotLocation` and option "opaque" in `Polyline`/`Polygon`, and its ring data violates the very declaration the keyed usage needs. Split the class: the keyed coordinate lookup keeps `PositioningSystemCoordinate` (with its `typeURI` key), the rings move to a dedicated vertex class with no key, held by opaque slots. Identity declarations stay class-level truths; slots choose only *whether* to recurse, never *what identity means*. + +## Consequence of strict mode + +- Some LinkML schemas need to be reworked to be compliant — in asset360 concretely: `ServicePhoneNumber` gains a key or `unique_keys`, and the `PositioningSystemCoordinate` ring/lookup dual use is split into two classes. +- More cases of patches produced outside this diff lib cannot be applied fully: a patch that addresses elements positionally has no meaning against an opaque slot (only whole-value updates apply) and no reliable meaning against a keyed/composed-key container. Such deltas are reported as failed rather than guessed at. + +## Recyclable material + +The abandoned branch `feat/container-shapes-and-verify-old` (local, single commit `312e6d8`) and the follow-up spike `spike/unique-keys-vs-opaque` contain fixtures and tests that carry over; the `DiffShape::Set`/`ShapeConfig` machinery itself is superseded by this design. + +Recycled (near-verbatim, renamed to `opaque` where the branch says `array`): + +- `diff_shapes.rs:158-219` — `array_slot_emits_one_whole_slot_update` (move / insert / drop / reverse a vertex ring, each exactly one whole-slot `Update`), the scalar-list variant, and `array_slot_unchanged_emits_nothing`. +- `diff_shapes.rs:419-463` — `keyed_slot_keeps_minimal_field_level_deltas`, as the regression guard that keyed containers keep minimal field-level deltas. +- `diff_shapes.rs:467-492` — `undeclared_slots_keep_positional_behaviour`, the non-goal's compatibility guard. +- `load_duplicate_keys.rs:113-181` — missing `key` is an error / missing `identifier` warns / type-designator carve-out, as linter severity fixtures (the designator carve-out exists precisely because of coordinate classes like Example 2). +- Multiplicity guards (`set_diff_respects_multiplicity`, the `["a","a"]` load assertion): a repeated element is data, never deduped. +- From the spike: `opaque_*` tests, `coordinates_match_by_unique_keys_derived_key`, and the `unique_keys` ambiguity-warning pair. + +Not recycled: the `Set` content-matching diff/patch (`diff_set`, `apply_set_leaf_delta`, the drift-location trio) and the single-slot `shape_key` override — both replaced by `unique_keys`-declared identity. From be409795ea7654a3d4d488ddb6537f9b3e0f08f6 Mon Sep 17 00:00:00 2001 From: ejsyx Date: Tue, 18 Aug 2026 13:38:23 +0200 Subject: [PATCH 02/58] docs: correct per-example resolutions in identity design spec Phone numbers resolve via a unique_keys declaration of the existing SHACL rule; the coordinate rings resolve by lifting the positioning system identity up a layer and holding opaque lists of a bare Vertex class, leaving SpotLocation and the coordinate hierarchy untouched. Co-Authored-By: Claude Fable 5 --- ...ned-multivalued-element-identity-design.md | 105 +++++++++++++----- 1 file changed, 78 insertions(+), 27 deletions(-) diff --git a/docs/superpowers/specs/2026-08-17-inlined-multivalued-element-identity-design.md b/docs/superpowers/specs/2026-08-17-inlined-multivalued-element-identity-design.md index 11f4d20..aa22288 100644 --- a/docs/superpowers/specs/2026-08-17-inlined-multivalued-element-identity-design.md +++ b/docs/superpowers/specs/2026-08-17-inlined-multivalued-element-identity-design.md @@ -97,6 +97,8 @@ Apply A then B: B's cascade overwrites index 0 with the stale snapshot value — **Duplication.** Two ingest sources independently discover the same new operator number and both emit `Add hasPhoneNumber/2 = {"phoneNumber": "09/241.25.10", "hasNumberFunction": "Operator"}`. Both apply; the golden record now holds the same phone number twice. Under key-based identity the second add would have been recognised as the same element. +The correct resolution for this slot is Option 2 below: declare the SHACL rule as `unique_keys`. + ### Example 2 — `PositioningSystemCoordinate`: one class, two identity shapes `PositioningSystemCoordinate` declares a key — `typeURI`, the type designator, which plays the role of the coordinate-system discriminator (`rsm.yaml:610-625`, `asset360.yaml:541-550`, elided): @@ -166,70 +168,119 @@ Real committed data (`asset360-model/tests/data/signal-obj-with-track.json:38-96 inlined_as_list: true ``` -Every vertex of a ring is the same coordinate subclass, so the declared key (`typeURI`) is **constant across elements**. Position is the only identity, ring data legitimately repeats the key value (a closed ring may even repeat a whole vertex), and vertex order is meaningful. +Every vertex of a ring is the same coordinate subclass, so the declared key (`typeURI`) is **constant across elements**. Real committed data shows it plainly (`consolidator_api/goldenrecords/tests/data/a_trail.json`, trimmed to two of four vertices — all four repeat the identical `typeURI` and the identical positioning system): + +```json +"polylines": [{ + "PolyLine_coordinates": [ + {"x": 214888.6, "y": 97029.71, + "typeURI": "https://data.infrabel.be/asset360/MicroCoordinate", + "PositioningSystemCoordinate_positioningSystem": {"typeURI": "http://rsm.uic.org/RSM12#EAID_4160AA98_..."}}, + {"x": 214819.42, "y": 97029.71, + "typeURI": "https://data.infrabel.be/asset360/MicroCoordinate", + "PositioningSystemCoordinate_positioningSystem": {"typeURI": "http://rsm.uic.org/RSM12#EAID_4160AA98_..."}} + ] +}] +``` + +Position is the only identity, ring data legitimately repeats the key value (a closed ring may even repeat a whole vertex), and vertex order is meaningful. The per-vertex `typeURI` and positioning system are pure redundancy: the system is a property of the ring, not of the vertex. -One class, two identity shapes. That ring data violates any `unique_keys` the class would declare — so the class can't declare one. **Remodelling into different classes is the only solution**: the keyed lookup usage keeps a class whose key/`unique_keys` declaration is truthful for all of its data, and the ring usage gets its own class (an un-keyed vertex) whose containers are declared opaque. We do not compromise on the goal by letting one class carry contradictory identity declarations per slot. +One class, two identity shapes. That ring data violates any `unique_keys` the class would declare — so the class can't declare one, and letting the opaque annotation suppress a schema constraint would be a layering inversion we do not accept. The resolution is to rework the model: move the positioning-system identity up a layer and give rings a bare vertex class that never inherits the key. One valid remodeling is worked out in Option 4 below; `SpotLocation_coordinates` and the coordinate hierarchy stay untouched. ## The options when the linter flags a slot -The linter's question is always the same — *where does element identity come from?* — and the author has exactly three answers, plus a rework escape hatch. Worked out on the constituting examples: +The linter's question is always the same — *where does element identity come from?* — and the author has exactly three answers, plus a rework escape hatch. Each constituting example gets its correct resolution below. + +### Option 1 — a key (or identifier): identity a single slot already provides -### Option 1 — a key (or identifier): give elements the identity they already have +When the element class declares a `key` or `identifier` slot that is truthful for all of its data, nothing needs to change — the diff already matches elements by it. `SpotLocation_coordinates` is the example: a dict keyed by `typeURI`, at most one coordinate per positioning system, exactly what the data means. -`hasNumberFunction` is already the de-facto key (the SHACL shape says so). Declare it: +### Option 2 — a composed key (content): declare `unique_keys` — the phone number solution + +`hasNumberFunction` is already the de-facto identity — the SHACL shape says so. Declare exactly that rule with the existing LinkML meta `unique_keys`: ```yaml ServicePhoneNumber: + unique_keys: + one_number_per_function: + unique_key_slots: + - hasNumberFunction attributes: phoneNumber: range: string hasNumberFunction: range: NumberFunction - key: true required: true ``` -Deltas become key-addressed (`hasPhoneNumber/Emergency_Number/phoneNumber`), immune to drift and reorder; the duplicate-add collapses into an update of the same element. The SHACL uniqueness rule is now also expressed in the schema itself. +This is the SHACL constraint expressed verbatim in the schema, and it is purely additive: `unique_keys` changes neither serialization nor loading of the deployed list data (unlike promoting the slot to `key: true`, which changes the container's serialization contract). Element identity is the function: deltas are addressed by it (`hasPhoneNumber/Emergency_Number/phoneNumber`), immune to drift and reorder, and the duplicate-add from Example 1 collides into the same element instead of duplicating it. -### Option 2 — a composed key (content): declare `unique_keys` +`unique_key_slots` composes: had the model allowed several numbers per function, `[hasNumberFunction, phoneNumber]` would make the full content the identity — at the price that every correction becomes a remove-plus-add of the whole element. Here that composition would be wrong: it would permit two numbers for the same function, contradicting the SHACL rule. -When no single slot identifies an element but a combination does, declare the existing LinkML meta `unique_keys` on the element class: +### Option 3 — opaque: identity comes from nowhere, say so + +For the vertex ring, elements genuinely have no identity — a moved vertex, an inserted vertex, a reversed ring are all edits *of the geometry*, not of a vertex: ```yaml - ServicePhoneNumber: - unique_keys: - phone_identity: - unique_key_slots: - - hasNumberFunction - - phoneNumber - attributes: - ... + Polygon_coordinates: + range: Vertex + multivalued: true + inlined_as_list: true + annotations: + diff.linkml.io/opaque: true ``` -Element identity is the composed key — here effectively the element's content. Elements are matched across versions by that identity; an edit of a key-constituent is a remove-plus-add of the whole element, an edit of a non-key slot is addressed at the element. Reorder is not a change. +All recursion stops at the slot: any change below it is exactly one `Update` at the slot path carrying the whole old and new value. Two sources editing the same ring conflict visibly at the ring level — whole value against whole value — instead of interleaving vertex indices into a corrupt geometry. -### Option 3 — opaque: identity comes from nowhere, say so +### Option 4 — remodel: when one class needs two answers — the coordinate solution -For the vertex ring, elements genuinely have no identity — a moved vertex, an inserted vertex, a reversed ring are all edits *of the geometry*, not of a vertex: +`PositioningSystemCoordinate` needs Option 1 in `SpotLocation` and Option 3 in `Polyline`/`Polygon`, and its ring data violates the very declaration the keyed usage needs. No slot-level override can fix that — identity declarations stay class-level truths; slots choose only *whether* to recurse, never *what identity means*. Rework the model instead. One valid remodeling (shown for `Polygon`; `Polyline` is symmetric): ```yaml - RingVertex_coordinates: - range: RingVertex + AreaLocation: + attributes: + polygons: + range: Polygon + multivalued: true + inlined_as_list: true + + Polygon: + is_a: NamedResource + unique_keys: + one_polygon_per_positioning_system: + unique_key_slots: + - positioningSystemType + attributes: + positioningSystemType: # the identity the vertices used to repeat, + range: uri # lifted up to the ring layer + required: true + Polygon_positioningSystem: + range: PositioningSystem + inlined: true + Polygon_coordinates: + range: Vertex multivalued: true inlined_as_list: true annotations: diff.linkml.io/opaque: true -``` -All recursion stops at the slot: any change below it is exactly one `Update` at the slot path carrying the whole old and new value. Two sources editing the same ring conflict visibly at the ring level — whole value against whole value — instead of interleaving vertex indices into a corrupt geometry. + Vertex: + attributes: + x: {range: float} + y: {range: float} + z: {range: float} +``` -### Option 4 — remodel: when one class needs two answers +The move is to introduce a layer so the key slot no longer sits on the elements holding the geometry data: -`PositioningSystemCoordinate` needs option "key" in `SpotLocation` and option "opaque" in `Polyline`/`Polygon`, and its ring data violates the very declaration the keyed usage needs. Split the class: the keyed coordinate lookup keeps `PositioningSystemCoordinate` (with its `typeURI` key), the rings move to a dedicated vertex class with no key, held by opaque slots. Identity declarations stay class-level truths; slots choose only *whether* to recurse, never *what identity means*. +- `locations` is already keyed by `locationrole` — that layer exists. +- A polygon's identity within `polygons` is its positioning system, declared as a `unique_keys` composed key on `Polygon`. The schema's own description already says the list holds the same shape once per positioning system; the redundant per-vertex `typeURI` / positioning system move up to the ring layer, which is what they always described. +- The ring becomes an opaque list of a bare `Vertex` class that never inherits `key: typeURI`, so no declaration is violated by repeated vertices. +- `SpotLocation_coordinates` and the `PositioningSystemCoordinate` hierarchy are untouched; the ~25k keyed lookup records keep their behaviour. ## Consequence of strict mode -- Some LinkML schemas need to be reworked to be compliant — in asset360 concretely: `ServicePhoneNumber` gains a key or `unique_keys`, and the `PositioningSystemCoordinate` ring/lookup dual use is split into two classes. +- Some LinkML schemas need to be reworked to be compliant — in asset360 concretely: `ServicePhoneNumber` gains a `unique_keys` declaration, and the polygon/polyline rings move their positioning-system identity up a layer and become opaque lists of a bare `Vertex` class. - More cases of patches produced outside this diff lib cannot be applied fully: a patch that addresses elements positionally has no meaning against an opaque slot (only whole-value updates apply) and no reliable meaning against a keyed/composed-key container. Such deltas are reported as failed rather than guessed at. ## Recyclable material From 0391a94a813a53f714700a8fedfecdeae7a27600 Mon Sep 17 00:00:00 2001 From: ejsyx Date: Tue, 18 Aug 2026 14:05:13 +0200 Subject: [PATCH 03/58] docs: implementation plan for inlined multivalued element identity Co-Authored-By: Claude Fable 5 --- ...18-inlined-multivalued-element-identity.md | 1564 +++++++++++++++++ 1 file changed, 1564 insertions(+) create mode 100644 docs/superpowers/plans/2026-08-18-inlined-multivalued-element-identity.md diff --git a/docs/superpowers/plans/2026-08-18-inlined-multivalued-element-identity.md b/docs/superpowers/plans/2026-08-18-inlined-multivalued-element-identity.md new file mode 100644 index 0000000..abf31c2 --- /dev/null +++ b/docs/superpowers/plans/2026-08-18-inlined-multivalued-element-identity.md @@ -0,0 +1,1564 @@ +# Inlined Multivalued Element Identity Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Make element identity for inlined multivalued slots declarable — key/identifier, `unique_keys` composed key, or `diff.linkml.io/opaque` (nowhere) — and add an opt-in linter that flags slots where identity comes from nowhere. + +**Architecture:** Three additive features in the existing crates. (1) A `diff.linkml.io/opaque` slot annotation makes `diff` stop all recursion and emit one whole-value `Update`, and makes `patch` refuse to descend below the slot. (2) `diff`'s keyed-list matching learns a fallback identity derived from the range class's (inheritance-merged) `unique_keys`; `patch` learns to resolve those labels. (3) A new `identity_lint` module offers two opt-in entry points (schema-level ambiguity lint, data-level duplicate-identity lint) that are **not** wired into default validation. No changes to `DiffOptions`/`PatchOptions` shapes; behaviour only changes for schemas that declare the annotation or `unique_keys` (no current fixture or consumer does). + +**Tech Stack:** Rust workspace (`linkml_runtime`, `schemaview`, `linkml_tools`, `linkml_runtime_python` via PyO3), cargo integration tests. + +**Spec:** `docs/superpowers/specs/2026-08-17-inlined-multivalued-element-identity-design.md` + +## Global Constraints + +- **Non-goal (spec):** inferred semantics must not change for schemas that declare nothing. Every existing test passes unchanged; the positional list branch's output (including its path segments) stays byte-identical. Never add a uniqueness guard to key/identifier-labelled matching — only to `unique_keys`-derived matching. +- **"Report, never guess" (spec):** a patch that cannot locate its target unambiguously returns `Ok(false)` so the path lands in `PatchTrace::failed`. No fuzzy fallbacks. +- **Layering (spec):** validation/lint code never reads `diff.linkml.io/*` to *suppress* a schema-constraint finding. The schema-level linter may skip opaque slots (identity is declared: nowhere); the data-level duplicate check never consults the annotation. +- `PatchTrace::failed` stays `Vec>` — do not import the abandoned branch's `PatchFailure` struct. +- Every task ends green: `cargo test --workspace` (or the named `-p` subset), `cargo fmt --all`, `cargo clippy --workspace --all-targets -- -D warnings` at the final task. +- Commit after every task; message style from `git log`: `feat(runtime): ...`, `test(runtime): ...`, `feat(schemaview): ...`. + +--- + +### Task 1: `ClassView::unique_keys()` — inheritance-merged accessor + +The metamodel already parses `unique_keys` (`src/metamodel/src/lib.rs:11520`, `ClassDefinition.unique_keys: Option>>`) but `ClassView::def()` returns the raw definition: keys declared on an `is_a` parent or mixin are invisible (ClassView merges slots, not unique_keys). Everything downstream (diff matching, linter) needs a merged, deterministic view. + +**Files:** +- Modify: `src/schemaview/src/classview.rs` (new method near `key_or_identifier_slot()`, ~line 606) +- Create: `src/schemaview/tests/unique_keys.rs` +- Create: `src/schemaview/tests/data/unique_keys.yaml` + +**Interfaces:** +- Consumes: `ClassView::def()`, `ClassView::parent_class()` (classview.rs:590), the mixin-resolution pattern of `collect_ancestors_map` (classview.rs:624), `linkml_meta::UniqueKey { unique_key_name, unique_key_slots: Vec, consider_nulls_inequal, .. }`. +- Produces: `pub fn unique_keys(&self) -> Vec<(String, UniqueKey)>` on `ClassView` — merged across `is_a` and mixins (nearest definition wins per name), **sorted by name** (HashMap order is nondeterministic and diff paths must be stable). Tasks 4, 5, 6 rely on exactly this signature. + +- [ ] **Step 1: Write the fixture schema** + +`src/schemaview/tests/data/unique_keys.yaml` (copy the header — `id`, `name`, `prefixes`, `default_range`, types/imports — from an existing fixture in `src/schemaview/tests/data/` so types resolve the same way): + +```yaml +classes: + Base: + unique_keys: + by_code: + unique_key_slots: [code] + shared_name: + unique_key_slots: [base_field] + attributes: + code: {range: string} + base_field: {range: string} + MixinCls: + mixin: true + unique_keys: + by_tag: + unique_key_slots: [tag] + attributes: + tag: {range: string} + Child: + is_a: Base + mixins: [MixinCls] + unique_keys: + shared_name: # overrides Base's entry of the same name + unique_key_slots: [child_field] + attributes: + child_field: {range: string} + Plain: + attributes: + whatever: {range: string} +``` + +- [ ] **Step 2: Write the failing test** + +`src/schemaview/tests/unique_keys.rs` (mirror the SchemaView setup of a neighbouring test, e.g. `class_lookup.rs`): + +```rust +use linkml_schemaview::identifier::{converter_from_schema, Identifier}; +use linkml_schemaview::io::from_yaml; +use linkml_schemaview::schemaview::SchemaView; +use std::path::PathBuf; + +fn fixture() -> SchemaView { + let mut p = PathBuf::from(env!("CARGO_MANIFEST_DIR")); + p.push("tests/data/unique_keys.yaml"); + let schema = from_yaml(&p).unwrap(); + let mut sv = SchemaView::new(); + sv.add_schema(schema).unwrap(); + sv +} + +#[test] +fn unique_keys_merge_across_is_a_and_mixins_nearest_wins() { + let sv = fixture(); + let conv = sv.converter(); + let child = sv + .get_class(&Identifier::new("Child"), &conv) + .unwrap() + .expect("class not found"); + let uks = child.unique_keys(); + let names: Vec<&str> = uks.iter().map(|(n, _)| n.as_str()).collect(); + // name-sorted, merged from Base (by_code), MixinCls (by_tag), Child (shared_name override) + assert_eq!(names, vec!["by_code", "by_tag", "shared_name"]); + let shared = &uks.iter().find(|(n, _)| n == "shared_name").unwrap().1; + assert_eq!( + shared.unique_key_slots, + vec!["child_field".to_string()], + "the nearest declaration must win" + ); +} + +#[test] +fn class_without_unique_keys_yields_empty() { + let sv = fixture(); + let conv = sv.converter(); + let plain = sv + .get_class(&Identifier::new("Plain"), &conv) + .unwrap() + .expect("class not found"); + assert!(plain.unique_keys().is_empty()); +} +``` + +- [ ] **Step 3: Run tests to verify they fail** + +Run: `cargo test -p schemaview --test unique_keys` +Expected: FAIL to compile — `unique_keys` method not found on `ClassView`. + +- [ ] **Step 4: Implement the accessor** + +In `src/schemaview/src/classview.rs`, near `key_or_identifier_slot()`. BFS from `self` through `is_a` parents and mixins so nearer declarations claim a name first (`or_insert`). The sketch below uses `parent_class()` for the `is_a` chain; resolve mixin names to `ClassView`s the same way `collect_ancestors_map` (classview.rs:624) does — adjust to that private API rather than inventing a new resolution path. + +```rust + /// The class's `unique_keys`, merged across the inheritance chain + /// (`is_a` parents and mixins), nearest declaration winning per name. + /// Name-sorted: declaration order is lost in the underlying map and + /// consumers (diff path segments) need a deterministic order. + pub fn unique_keys(&self) -> Vec<(String, UniqueKey)> { + use std::collections::{HashMap, VecDeque}; + let mut merged: HashMap = HashMap::new(); + let mut queue: VecDeque = VecDeque::from([self.clone()]); + let mut seen: std::collections::HashSet = Default::default(); + while let Some(cv) = queue.pop_front() { + if !seen.insert(cv.name().to_string()) { + continue; + } + if let Some(uks) = cv.def().unique_keys.as_ref() { + for (name, uk) in uks { + merged + .entry(name.clone()) + .or_insert_with(|| (**uk).clone()); + } + } + if let Ok(Some(parent)) = cv.parent_class() { + queue.push_back(parent); + } + // + push each resolved mixin ClassView (resolution as in + // collect_ancestors_map, classview.rs:624) + } + let mut out: Vec<(String, UniqueKey)> = merged.into_iter().collect(); + out.sort_by(|a, b| a.0.cmp(&b.0)); + out + } +``` + +Import `linkml_meta::UniqueKey` at the top of the file (check the existing `linkml_meta` imports there for the path). + +- [ ] **Step 5: Run tests to verify they pass** + +Run: `cargo test -p schemaview --test unique_keys` +Expected: PASS (both tests). + +- [ ] **Step 6: Run the schemaview suite for regressions** + +Run: `cargo test -p schemaview` +Expected: PASS, no other test touched. + +- [ ] **Step 7: Commit** + +```bash +git add src/schemaview/src/classview.rs src/schemaview/tests/unique_keys.rs src/schemaview/tests/data/unique_keys.yaml +git commit -m "feat(schemaview): ClassView::unique_keys merged across is_a and mixins" +``` + +--- + +### Task 2: shared fixture + `diff.linkml.io/opaque` on the diff side + +**Files:** +- Create: `src/runtime/tests/data/identity.yaml` +- Create: `src/runtime/tests/diff_opaque.rs` +- Modify: `src/runtime/src/diff.rs` (annotation const + check + early-return in `inner`, near lines 10–21 and 121–125) +- Modify: `src/runtime/src/lib.rs` (re-export `OPAQUE_ANNOTATION` in the `pub use diff::{...}` list, line 49) + +**Interfaces:** +- Consumes: `slot_is_ignored` pattern (`diff.rs:12-21`), `LinkMLInstance::equals(other, treat_missing_as_null)`. +- Produces: `pub const OPAQUE_ANNOTATION: &str = "diff.linkml.io/opaque"` and `pub(crate) fn slot_is_opaque(slot: &SlotView) -> bool` in `diff.rs` (Tasks 3 and 6 use both); the fixture `identity.yaml` (all later runtime test files load it). + +- [ ] **Step 1: Write the fixture schema** + +`src/runtime/tests/data/identity.yaml`. Copy the header (`id`, `name`, `prefixes`, `default_range`, and the types/imports mechanism) from `src/runtime/tests/data/personinfo.yaml` so `string`/`float`/`uri` resolve identically. Classes (this is the spec's asset360 material, condensed): + +```yaml +classes: + Service: + attributes: + name: {range: string} + # spec Example 1 with its correct resolution: identity = the function + hasPhoneNumber: + range: ServicePhoneNumber + multivalued: true + inlined_as_list: true + # same shape, nothing declared: must keep positional behaviour; linter flags it + plainPhoneNumber: + range: PlainPhoneNumber + multivalued: true + inlined_as_list: true + # inherited unique_keys (via is_a) must also drive matching + escalation: + range: EmergencyPhoneNumber + multivalued: true + inlined_as_list: true + # composite two-slot unique key + contacts: + range: Contact + multivalued: true + inlined_as_list: true + # opaque + a keyed element class: data lint must still check unique_keys + archivedContacts: + range: Contact + multivalued: true + inlined_as_list: true + annotations: + diff.linkml.io/opaque: true + # scalar list, nothing declared: positional; linter flags it + tags: + range: string + multivalued: true + # scalar list declared opaque + opaqueTags: + range: string + multivalued: true + annotations: + diff.linkml.io/opaque: true + # spec Example 2's ring: opaque list of bare vertices + outline: + range: Vertex + multivalued: true + inlined_as_list: true + annotations: + diff.linkml.io/opaque: true + # single-valued opaque object: "stop all recursion" is not list-specific + profile: + range: Profile + inlined: true + annotations: + diff.linkml.io/opaque: true + # keyed class inlined as dict: already unambiguous, linter stays silent + labels: + range: Label + multivalued: true + inlined: true + # reference list (not inlined): out of the linter's scope + operators: + range: Operator + multivalued: true + inlined: false + area: + range: AreaLocation + inlined: true + + ServicePhoneNumber: + unique_keys: + one_number_per_function: + unique_key_slots: [hasNumberFunction] + attributes: + phoneNumber: {range: string} + hasNumberFunction: {range: NumberFunction, required: true} + + PlainPhoneNumber: + attributes: + phoneNumber: {range: string} + hasNumberFunction: {range: NumberFunction, required: true} + + EmergencyPhoneNumber: + is_a: ServicePhoneNumber + attributes: + note: {range: string} + + Contact: + unique_keys: + contact_identity: + unique_key_slots: [kind, phone] + attributes: + kind: {range: string, required: true} + phone: {range: string, required: true} + note: {range: string} + + Label: + attributes: + lang: {range: string, key: true, required: true} + text: {range: string} + + Profile: + attributes: + bio: {range: string} + motto: {range: string} + + Operator: + attributes: + opId: {range: string, identifier: true} + opName: {range: string} + + AreaLocation: + attributes: + polygons: + range: Polygon + multivalued: true + inlined_as_list: true + + Polygon: + unique_keys: + one_polygon_per_positioning_system: + unique_key_slots: [positioningSystemType] + attributes: + positioningSystemType: {range: uri, required: true} + coordinates: + range: Vertex + multivalued: true + inlined_as_list: true + annotations: + diff.linkml.io/opaque: true + + Vertex: + attributes: + x: {range: float} + y: {range: float} + +enums: + NumberFunction: + permissible_values: + Emergency_Number: + Non_Urgent_Communication: + Operator: +``` + +- [ ] **Step 2: Write the failing tests** + +`src/runtime/tests/diff_opaque.rs`. The move/insert/drop/reverse loop is recycled near-verbatim from the abandoned branch's `array_slot_emits_one_whole_slot_update` (its `feat/container-shapes-and-verify-old` branch, `src/runtime/tests/diff_shapes.rs:158-197`) — same assertions, annotation renamed. + +```rust +use linkml_runtime::{diff, load_json_str, patch, DiffOptions, LinkMLInstance, PatchOptions}; +use linkml_schemaview::identifier::{converter_from_schema, Identifier}; +use linkml_schemaview::io::from_yaml; +use linkml_schemaview::schemaview::{ClassView, SchemaView}; +use linkml_schemaview::Converter; +use linkml_runtime::{Delta, DeltaOp}; +use serde_json::{json, Value as JsonValue}; +use std::path::PathBuf; + +struct Fixture { + sv: SchemaView, + conv: Converter, + service: ClassView, +} + +fn fixture() -> Fixture { + let mut p = PathBuf::from(env!("CARGO_MANIFEST_DIR")); + p.push("tests/data/identity.yaml"); + let schema = from_yaml(&p).unwrap(); + let mut sv = SchemaView::new(); + sv.add_schema(schema.clone()).unwrap(); + let conv = converter_from_schema(&schema); + let service = sv + .get_class(&Identifier::new("Service"), &conv) + .unwrap() + .expect("class not found"); + Fixture { sv, conv, service } +} + +impl Fixture { + fn load(&self, v: JsonValue) -> LinkMLInstance { + load_json_str(&v.to_string(), &self.sv, &self.service, &self.conv) + .unwrap() + .into_instance() + .unwrap() + } +} + +fn diff2(f: &Fixture, before: JsonValue, after: JsonValue) -> Vec { + diff(&f.load(before), &f.load(after), DiffOptions::new(true)) +} + +fn only(deltas: &[Delta]) -> &Delta { + assert_eq!(deltas.len(), 1, "expected exactly one delta: {deltas:#?}"); + &deltas[0] +} + +fn square() -> Vec { + vec![ + json!({"x": 4.35, "y": 50.85}), + json!({"x": 4.36, "y": 50.85}), + json!({"x": 4.36, "y": 50.86}), + json!({"x": 4.35, "y": 50.86}), + ] +} + +fn outline(items: Vec) -> JsonValue { + json!({"name": "svc", "outline": items}) +} + +#[test] +fn opaque_ring_edit_is_one_whole_slot_update() { + let f = fixture(); + + let mut moved = square(); + moved[1] = json!({"x": 4.37, "y": 50.85}); + let mut inserted = square(); + inserted.insert(2, json!({"x": 4.365, "y": 50.855})); + let dropped_first = square()[1..].to_vec(); + let mut reversed = square(); + reversed.reverse(); + + for (label, after) in [ + ("move one vertex", moved), + ("insert a vertex mid-ring", inserted), + ("drop the first vertex", dropped_first), + ("reverse the ring", reversed), + ] { + let deltas = diff2(&f, outline(square()), outline(after)); + let delta = only(&deltas); + assert_eq!(delta.path, vec!["outline".to_string()], "{label}"); + assert_eq!(delta.op, DeltaOp::Update, "{label}"); + assert_eq!( + delta.old.as_ref().and_then(|v| v.as_array()).map(|a| a.len()), + Some(4), + "{label}: old must be the whole slot" + ); + assert!(delta.new.as_ref().is_some_and(|v| v.is_array()), "{label}"); + } +} + +#[test] +fn opaque_slot_unchanged_emits_nothing() { + let f = fixture(); + let deltas = diff2(&f, outline(square()), outline(square())); + assert!(deltas.is_empty(), "{deltas:#?}"); +} + +#[test] +fn opaque_scalar_list_is_one_whole_slot_update() { + let f = fixture(); + let before = json!({"name": "svc", "opaqueTags": ["a", "b"]}); + let after = json!({"name": "svc", "opaqueTags": ["b", "c", "d"]}); + let deltas = diff2(&f, before, after); + let delta = only(&deltas); + assert_eq!(delta.path, vec!["opaqueTags".to_string()]); + assert_eq!(delta.op, DeltaOp::Update); +} + +#[test] +fn opaque_single_valued_object_is_one_whole_value_update() { + let f = fixture(); + let before = json!({"name": "svc", "profile": {"bio": "b", "motto": "old"}}); + let after = json!({"name": "svc", "profile": {"bio": "b", "motto": "new"}}); + let deltas = diff2(&f, before, after); + let delta = only(&deltas); + assert_eq!(delta.path, vec!["profile".to_string()]); + assert_eq!(delta.op, DeltaOp::Update); + assert_eq!(delta.old, Some(json!({"bio": "b", "motto": "old"}))); + assert_eq!(delta.new, Some(json!({"bio": "b", "motto": "new"}))); +} + +#[test] +fn opaque_whole_slot_update_round_trips_through_patch() { + let f = fixture(); + let before = outline(square()); + let mut moved = square(); + moved[1] = json!({"x": 4.37, "y": 50.85}); + let after = outline(moved); + let deltas = diff2(&f, before.clone(), after.clone()); + let (patched, trace) = patch(&f.load(before), &deltas, PatchOptions::default()).unwrap(); + assert!(trace.failed.is_empty(), "{:?}", trace.failed); + assert!( + patched.equals(&f.load(after), true), + "round-trip mismatch: {}", + patched.to_json() + ); +} +``` + +(If `Delta`/`DeltaOp`/`PatchOptions` import paths differ, copy the exact imports from `src/runtime/tests/diff.rs`.) + +- [ ] **Step 3: Run tests to verify they fail** + +Run: `cargo test -p linkml_runtime --test diff_opaque` +Expected: FAIL — `opaque_ring_edit_is_one_whole_slot_update` sees many positional deltas instead of one; the single-valued and scalar cases see field-level/element-level deltas. (The round-trip test may pass already; that's fine.) + +- [ ] **Step 4: Implement the diff side** + +In `src/runtime/src/diff.rs`, next to `IGNORE_ANNOTATION` (line 10): + +```rust +/// Slot annotation declaring that element identity comes from nowhere: stop +/// all recursion, the slot's value is one atomic unit. Any change below the +/// slot is described as a single whole-value `Update` at the slot path, and +/// `patch` refuses paths that descend below it. +pub const OPAQUE_ANNOTATION: &str = "diff.linkml.io/opaque"; + +pub(crate) fn slot_is_opaque(slot: &SlotView) -> bool { + if slot.definitions().is_empty() { + return false; + } + slot.definition() + .annotations + .as_ref() + .map(|a| a.contains_key(OPAQUE_ANNOTATION)) + .unwrap_or(false) +} +``` + +(Presence-based, exactly like `slot_is_ignored` — the annotation's *presence* is the declaration, mirroring `diff.linkml.io/ignore`.) + +In `diff`'s `inner`, extend the existing slot guard (lines 121–125): + +```rust + if let Some(sl) = slot { + if slot_is_ignored(sl) { + return; + } + if slot_is_opaque(sl) { + if !s.equals(t, opts.treat_missing_as_null) { + out.push(Delta { + path: path.clone(), + op: DeltaOp::Update, + old: Some(s.to_json()), + new: Some(t.to_json()), + }); + } + return; + } + } +``` + +Note: `inner` receives `slot: Some(..)` only for direct slot values (the object arm); list/mapping elements recurse with `None`, so the check fires exactly at slot level. A slot appearing/disappearing entirely is handled by the object arm's missing-slot branches, which already emit whole-value deltas without recursing. + +In `src/runtime/src/lib.rs:49`, add `OPAQUE_ANNOTATION` to the diff re-export list. + +- [ ] **Step 5: Run tests to verify they pass** + +Run: `cargo test -p linkml_runtime --test diff_opaque` +Expected: PASS (all 5). + +- [ ] **Step 6: Run the full runtime suite for regressions** + +Run: `cargo test -p linkml_runtime` +Expected: PASS — no existing fixture carries the annotation, so nothing else may change. + +- [ ] **Step 7: Commit** + +```bash +git add src/runtime/tests/data/identity.yaml src/runtime/tests/diff_opaque.rs src/runtime/src/diff.rs src/runtime/src/lib.rs +git commit -m "feat(runtime): diff.linkml.io/opaque stops recursion, replaces whole value" +``` + +--- + +### Task 3: opaque on the patch side — refuse to descend + +A patch produced outside this diff lib may address structure below an opaque slot (e.g. positional vertex paths). Locating anything below the slot would be a guess; the spec says such deltas are reported as failed. + +**Files:** +- Modify: `src/runtime/src/diff.rs` (`apply_delta_object` descent at ~line 873, `apply_delta_mapping` at ~880, `apply_delta_list` at ~928) +- Test: `src/runtime/tests/diff_opaque.rs` (extend) + +**Interfaces:** +- Consumes: `slot_is_opaque` from Task 2. +- Produces: patch behaviour only — no new names. + +- [ ] **Step 1: Write the failing tests** + +Append to `src/runtime/tests/diff_opaque.rs`: + +```rust +#[test] +fn patch_below_opaque_slot_is_reported_failed_not_guessed() { + let f = fixture(); + let golden = f.load(outline(square())); + let delta = Delta { + path: vec!["outline".to_string(), "1".to_string(), "x".to_string()], + op: DeltaOp::Update, + old: Some(json!(4.36)), + new: Some(json!(9.99)), + }; + let (patched, trace) = patch(&golden, &[delta.clone()], PatchOptions::default()).unwrap(); + assert_eq!(trace.failed, vec![delta.path.clone()]); + assert!( + patched.equals(&golden, true), + "nothing may change: {}", + patched.to_json() + ); +} + +#[test] +fn patch_at_opaque_slot_path_still_applies_whole_value() { + let f = fixture(); + let golden = f.load(outline(square())); + let mut moved = square(); + moved[1] = json!({"x": 4.37, "y": 50.85}); + let delta = Delta { + path: vec!["outline".to_string()], + op: DeltaOp::Update, + old: Some(json!(square())), + new: Some(json!(moved.clone())), + }; + let (patched, trace) = patch(&golden, &[delta], PatchOptions::default()).unwrap(); + assert!(trace.failed.is_empty(), "{:?}", trace.failed); + assert!(patched.equals(&f.load(outline(moved)), true)); +} +``` + +- [ ] **Step 2: Run tests to verify the new one fails** + +Run: `cargo test -p linkml_runtime --test diff_opaque` +Expected: `patch_below_opaque_slot_is_reported_failed_not_guessed` FAILS (the positional path currently applies); the whole-value test passes. + +- [ ] **Step 3: Implement the refusal** + +In `apply_delta_object` (diff.rs, the descent after the `path.len() == 1` early return, ~line 873): + +```rust + if let Some(child) = values.get_mut(key) { + let slot = class.slots().iter().find(|s| s.name == *key); + if slot.is_some_and(slot_is_opaque) { + // The path descends below an opaque slot: it addresses structure + // the slot does not expose. Report, never guess. + return Ok(false); + } + return apply_delta_linkml_inner(child, &path[1..], op, newv, trace, opts); + } +``` + +And defensively at the top of `apply_delta_mapping` and `apply_delta_list` (reached with a non-empty path only when the delta addresses *inside* the container — which is below the container's slot). This covers patches whose root instance is itself a list/mapping: + +```rust + if slot_is_opaque(slot) { + return Ok(false); + } +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `cargo test -p linkml_runtime --test diff_opaque` +Expected: PASS (all 7). + +- [ ] **Step 5: Full runtime suite** + +Run: `cargo test -p linkml_runtime` +Expected: PASS. + +- [ ] **Step 6: Commit** + +```bash +git add src/runtime/src/diff.rs src/runtime/tests/diff_opaque.rs +git commit -m "feat(runtime): patch refuses to descend below an opaque slot" +``` + +--- + +### Task 4: `unique_keys`-derived identity in diff list matching + +**Files:** +- Modify: `src/runtime/src/diff.rs` (extract label helpers, extend the `(List, List)` arm at lines 211–296) +- Create: `src/runtime/tests/diff_unique_keys.rs` + +**Interfaces:** +- Consumes: `ClassView::unique_keys()` (Task 1), the fixture (Task 2). +- Produces (all in `diff.rs`, used by Tasks 5 and 6): + - `pub(crate) fn element_key_label(v: &LinkMLInstance) -> Option` — the existing key/identifier label, extracted from the inline closure. + - `pub(crate) fn element_unique_key_label(v: &LinkMLInstance) -> Option` — label from the first (name-sorted) non-empty `unique_keys` entry; single-slot keys yield the bare scalar string, composite keys yield the JSON-array encoding of the values in `unique_key_slots` order (e.g. `["Emergency","02/111.11.11"]`). + - `pub(crate) fn element_identity_label(v: &LinkMLInstance) -> Option` — `element_key_label` first, `element_unique_key_label` as fallback. + - `fn scalar_slot_string(values: &HashMap, slot_name: &str) -> Option`. + +**Precedence and guard rules (from the spec + non-goal):** +1. opaque > key/identifier > unique_keys > positional. +2. `unique_keys` labels participate in *matching only*, never in positional path segments — the positional branch keeps using `element_key_label`, so its output stays byte-identical to today. +3. A `unique_keys` label is a class-level claim, weaker than a key: if any element's label would come from `unique_keys` and the labels are not unique within both lists, fall back to positional matching (dirty data must keep today's behaviour). Key/identifier-labelled lists get **no** new uniqueness guard. + +- [ ] **Step 1: Write the failing tests** + +`src/runtime/tests/diff_unique_keys.rs` — reuse the exact `Fixture`/`diff2`/`only` harness from `diff_opaque.rs` (copy it; integration tests don't share modules). Data builders: + +```rust +fn e() -> JsonValue { + json!({"phoneNumber": "09/241.25.00", "hasNumberFunction": "Emergency_Number"}) +} +fn n() -> JsonValue { + json!({"phoneNumber": "09/241.25.03", "hasNumberFunction": "Non_Urgent_Communication"}) +} +fn o() -> JsonValue { + json!({"phoneNumber": "09/241.25.10", "hasNumberFunction": "Operator"}) +} +fn phones(items: Vec) -> JsonValue { + json!({"name": "svc", "hasPhoneNumber": items}) +} +``` + +Tests: + +```rust +#[test] +fn unique_key_matching_targets_field_edits_by_key() { + let f = fixture(); + let mut n2 = n(); + n2["phoneNumber"] = json!("09/241.25.99"); + let deltas = diff2(&f, phones(vec![e(), n()]), phones(vec![e(), n2])); + let delta = only(&deltas); + assert_eq!( + delta.path, + vec![ + "hasPhoneNumber".to_string(), + "Non_Urgent_Communication".to_string(), + "phoneNumber".to_string() + ] + ); + assert_eq!(delta.op, DeltaOp::Update); +} + +#[test] +fn unique_key_matching_ignores_reorder() { + let f = fixture(); + let deltas = diff2(&f, phones(vec![e(), n(), o()]), phones(vec![o(), n(), e()])); + assert!(deltas.is_empty(), "reorder must be invisible: {deltas:#?}"); +} + +#[test] +fn unique_key_remove_and_add_are_key_addressed() { + let f = fixture(); + let deltas = diff2(&f, phones(vec![e(), n()]), phones(vec![n()])); + let delta = only(&deltas); + assert_eq!(delta.op, DeltaOp::Remove); + assert_eq!( + delta.path, + vec!["hasPhoneNumber".to_string(), "Emergency_Number".to_string()] + ); + assert_eq!(delta.old, Some(e())); + + let deltas = diff2(&f, phones(vec![e(), n()]), phones(vec![e(), n(), o()])); + let delta = only(&deltas); + assert_eq!(delta.op, DeltaOp::Add); + assert_eq!( + delta.path, + vec!["hasPhoneNumber".to_string(), "Operator".to_string()] + ); + assert_eq!(delta.new, Some(o())); +} + +#[test] +fn changing_the_key_slot_is_remove_plus_add() { + let f = fixture(); + let mut moved = e(); + moved["hasNumberFunction"] = json!("Operator"); + let deltas = diff2(&f, phones(vec![e(), n()]), phones(vec![moved.clone(), n()])); + assert_eq!(deltas.len(), 2, "{deltas:#?}"); + assert!(deltas + .iter() + .any(|d| d.op == DeltaOp::Remove && d.old.as_ref() == Some(&e()))); + assert!(deltas + .iter() + .any(|d| d.op == DeltaOp::Add && d.new.as_ref() == Some(&moved))); +} + +#[test] +fn duplicate_unique_key_data_falls_back_to_positional() { + let f = fixture(); + // two Emergency numbers: violates the class claim; data must keep + // today's positional behaviour, with numeric path segments + let e2 = json!({"phoneNumber": "09/000.00.00", "hasNumberFunction": "Emergency_Number"}); + let mut e2_edit = e2.clone(); + e2_edit["phoneNumber"] = json!("09/111.11.11"); + let deltas = diff2( + &f, + phones(vec![e(), e2]), + phones(vec![e(), e2_edit]), + ); + let delta = only(&deltas); + assert_eq!( + delta.path, + vec![ + "hasPhoneNumber".to_string(), + "1".to_string(), + "phoneNumber".to_string() + ], + "positional fallback must use numeric segments, never the duplicate label" + ); +} + +#[test] +fn undeclared_class_keeps_positional_cascade() { + let f = fixture(); + // PlainPhoneNumber has no unique_keys: removal still cascades as today + let before = json!({"name": "svc", "plainPhoneNumber": [e(), n(), o()]}); + let after = json!({"name": "svc", "plainPhoneNumber": [n(), o()]}); + let deltas = diff2(&f, before, after); + assert_eq!(deltas.len(), 3, "{deltas:#?}"); +} + +#[test] +fn composite_unique_key_uses_json_array_segment() { + let f = fixture(); + let a = json!({"kind": "Emergency", "phone": "02/111.11.11", "note": "old"}); + let mut a2 = a.clone(); + a2["note"] = json!("new"); + let b = json!({"kind": "Operator", "phone": "02/333.33.33"}); + let before = json!({"name": "svc", "contacts": [a, b.clone()]}); + let after = json!({"name": "svc", "contacts": [a2, b]}); + let deltas = diff2(&f, before, after); + let delta = only(&deltas); + assert_eq!( + delta.path, + vec![ + "contacts".to_string(), + r#"["Emergency","02/111.11.11"]"#.to_string(), + "note".to_string() + ] + ); +} + +#[test] +fn inherited_unique_keys_drive_matching() { + let f = fixture(); + // EmergencyPhoneNumber inherits one_number_per_function via is_a + let x = json!({"phoneNumber": "1", "hasNumberFunction": "Emergency_Number", "note": "a"}); + let mut x2 = x.clone(); + x2["note"] = json!("b"); + let y = json!({"phoneNumber": "2", "hasNumberFunction": "Operator", "note": "c"}); + let before = json!({"name": "svc", "escalation": [x, y.clone()]}); + let after = json!({"name": "svc", "escalation": [x2, y]}); + let deltas = diff2(&f, before, after); + let delta = only(&deltas); + assert_eq!( + delta.path, + vec![ + "escalation".to_string(), + "Emergency_Number".to_string(), + "note".to_string() + ] + ); +} +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `cargo test -p linkml_runtime --test diff_unique_keys` +Expected: the matching tests FAIL (positional deltas today); `duplicate_unique_key_data_falls_back_to_positional` and `undeclared_class_keeps_positional_cascade` PASS already (they assert today's behaviour and must stay green throughout). + +- [ ] **Step 3: Implement** + +In `src/runtime/src/diff.rs`, replace the inline `label` closure of the `(List, List)` arm (lines 212–226) with module-level helpers, then rework the arm: + +```rust +pub(crate) fn scalar_slot_string( + values: &std::collections::HashMap, + slot_name: &str, +) -> Option { + if let Some(LinkMLInstance::Scalar { value, .. }) = values.get(slot_name) { + return match value { + JsonValue::String(s) => Some(s.clone()), + other => Some(other.to_string()), + }; + } + None +} + +/// The key/identifier value identifying `v` among its list siblings, if any. +pub(crate) fn element_key_label(v: &LinkMLInstance) -> Option { + if let LinkMLInstance::Object { values, class, .. } = v { + let id_slot = class.key_or_identifier_slot()?; + return scalar_slot_string(values, &id_slot.name); + } + None +} + +/// A matching label derived from the range class's merged `unique_keys`. +/// +/// The name-sorted first entry with a non-empty slot list is the matching +/// identity (declaration order is not preserved by the metamodel, and diff +/// paths must be stable). Single-slot keys use the bare scalar value as the +/// label and path segment; composite keys use the JSON array encoding of the +/// values in `unique_key_slots` order (`["Emergency","02/111.11.11"]`) — +/// unambiguous, parseable, and displayable. +pub(crate) fn element_unique_key_label(v: &LinkMLInstance) -> Option { + if let LinkMLInstance::Object { values, class, .. } = v { + let uks = class.unique_keys(); + let (_, uk) = uks.iter().find(|(_, uk)| !uk.unique_key_slots.is_empty())?; + let parts: Option> = uk + .unique_key_slots + .iter() + .map(|s| scalar_slot_string(values, s)) + .collect(); + let mut parts = parts?; + return Some(if parts.len() == 1 { + parts.remove(0) + } else { + serde_json::to_string(&parts).expect("Vec serializes") + }); + } + None +} + +/// Identity for keyed list matching: a key/identifier slot outranks a +/// `unique_keys` claim. +pub(crate) fn element_identity_label(v: &LinkMLInstance) -> Option { + element_key_label(v).or_else(|| element_unique_key_label(v)) +} + +/// Whether any element's matching label would come from `unique_keys` rather +/// than a key/identifier slot. +fn labels_from_unique_keys(elements: &[LinkMLInstance]) -> bool { + elements.iter().any(|v| { + if let LinkMLInstance::Object { class, .. } = v { + class.key_or_identifier_slot().is_none() && !class.unique_keys().is_empty() + } else { + false + } + }) +} + +fn labels_are_unique(elements: &[LinkMLInstance], label: F) -> bool +where + F: Fn(&LinkMLInstance) -> Option, +{ + let mut seen = std::collections::HashSet::new(); + elements.iter().filter_map(label).all(|l| seen.insert(l)) +} +``` + +The `(List, List)` arm becomes: + +```rust +(LinkMLInstance::List { values: sl, .. }, LinkMLInstance::List { values: tl, .. }) => { + // Positional path segments keep using key/identifier labels ONLY, so the + // positional branch's output is byte-identical to before unique_keys + // support existed. + let label = |v: &LinkMLInstance| -> Option { element_key_label(v) }; + let identity = |v: &LinkMLInstance| -> Option { element_identity_label(v) }; + let mut keyed = sl.iter().all(|v| identity(v).is_some()) + && tl.iter().all(|v| identity(v).is_some()); + // A unique_keys label is a class-level claim, not a per-element identity: + // data may legitimately violate it (it must still load and round-trip). + // Matching duplicates by such a label would silently collapse elements, + // so refuse to guess and fall back to positional. + if keyed + && (labels_from_unique_keys(sl) || labels_from_unique_keys(tl)) + && !(labels_are_unique(sl, &identity) && labels_are_unique(tl, &identity)) + { + keyed = false; + } + if keyed { + // ... identical to the current keyed block (lines 233-266), + // with every `label(..)` call replaced by `identity(..)` ... + } else { + // ... the current positional block (lines 267-296), UNCHANGED, + // still using `label` for opportunistic segments ... + } +} +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `cargo test -p linkml_runtime --test diff_unique_keys` +Expected: PASS (all 8). + +- [ ] **Step 5: Full runtime suite — the non-goal gate** + +Run: `cargo test -p linkml_runtime && cargo test -p schemaview` +Expected: PASS. Any change in `diff.rs`/`diff_identifier.rs`/`trace.rs` test output means the positional branch or key-labelled matching drifted — fix the implementation, not the tests. + +- [ ] **Step 6: Commit** + +```bash +git add src/runtime/src/diff.rs src/runtime/tests/diff_unique_keys.rs +git commit -m "feat(runtime): match inlined list elements by unique_keys-derived identity" +``` + +--- + +### Task 5: `unique_keys` labels on the patch side + +**Files:** +- Modify: `src/runtime/src/diff.rs` (`resolve_list_index`, lines 530–558) +- Test: `src/runtime/tests/diff_unique_keys.rs` (extend) + +**Interfaces:** +- Consumes: `element_key_label`, `element_unique_key_label` (Task 4). +- Produces: patch behaviour only. + +- [ ] **Step 1: Write the failing tests** + +Append to `diff_unique_keys.rs`: + +```rust +#[test] +fn patch_locates_element_by_unique_key_under_drift() { + let f = fixture(); + // producer saw [E, N]; golden drifted to [N, E, O] + let golden = f.load(phones(vec![n(), e(), o()])); + let delta = Delta { + path: vec![ + "hasPhoneNumber".to_string(), + "Emergency_Number".to_string(), + "phoneNumber".to_string(), + ], + op: DeltaOp::Update, + old: Some(json!("09/241.25.00")), + new: Some(json!("09/999.99.99")), + }; + let (patched, trace) = patch(&golden, &[delta], PatchOptions::default()).unwrap(); + assert!(trace.failed.is_empty(), "{:?}", trace.failed); + let mut e2 = e(); + e2["phoneNumber"] = json!("09/999.99.99"); + assert!( + patched.equals(&f.load(phones(vec![n(), e2, o()])), true), + "the edit must land on E wherever it sits: {}", + patched.to_json() + ); +} + +#[test] +fn patch_reports_ambiguous_unique_key_instead_of_guessing() { + let f = fixture(); + // golden drifted into two Emergency elements: locating "the" one is a guess + let e2 = json!({"phoneNumber": "09/000.00.00", "hasNumberFunction": "Emergency_Number"}); + let golden = f.load(phones(vec![e(), e2])); + let delta = Delta { + path: vec![ + "hasPhoneNumber".to_string(), + "Emergency_Number".to_string(), + "phoneNumber".to_string(), + ], + op: DeltaOp::Update, + old: Some(json!("09/241.25.00")), + new: Some(json!("09/999.99.99")), + }; + let (patched, trace) = patch(&golden, &[delta.clone()], PatchOptions::default()).unwrap(); + assert_eq!(trace.failed, vec![delta.path.clone()]); + assert!(patched.equals(&golden, true), "nothing may change"); +} + +#[test] +fn unique_key_deltas_round_trip_through_patch() { + let f = fixture(); + let mut n2 = n(); + n2["phoneNumber"] = json!("09/241.25.99"); + for (before, after) in [ + (phones(vec![e(), n()]), phones(vec![e(), n2])), // field edit + (phones(vec![e(), n()]), phones(vec![n()])), // remove + (phones(vec![e(), n()]), phones(vec![e(), n(), o()])), // add + ] { + let deltas = diff2(&f, before.clone(), after.clone()); + let (patched, trace) = patch(&f.load(before), &deltas, PatchOptions::default()).unwrap(); + assert!(trace.failed.is_empty(), "{:?}", trace.failed); + assert!(patched.equals(&f.load(after), true), "{}", patched.to_json()); + } +} +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `cargo test -p linkml_runtime --test diff_unique_keys` +Expected: the two locate tests FAIL (segment `Emergency_Number` resolves to no index today); the remove/add legs of the round-trip may also fail. + +- [ ] **Step 3: Implement** + +Extend `resolve_list_index` (diff.rs:530). Keep the existing numeric-index attempt and the existing key/identifier `find_map` block **byte-identical** (do not rewrite it in terms of `element_key_label` — its scalar comparison semantics differ subtly for non-string keys). Append after it: + +```rust + // unique_keys-derived location. Only an unambiguous hit counts: if the + // golden record drifted into duplicate labels, locating "the" element + // would be a guess — return None so the delta is reported as failed. + let mut hit: Option = None; + for (i, v) in values.iter().enumerate() { + if element_key_label(v).is_some() { + continue; // this element is addressed by its key, handled above + } + if element_unique_key_label(v).as_deref() == Some(key) { + if hit.is_some() { + return None; + } + hit = Some(i); + } + } + hit +``` + +(The existing key block ends with a `find_map(...)` expression; restructure to `let by_key = ...; if by_key.is_some() { return by_key; }` followed by the code above.) + +Note on `Add` deltas: an `Add` whose unique-key segment resolves to no element takes the existing `idx_opt = None` append path in `apply_list_leaf_delta` (line 687) — that is correct and needs no change. + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `cargo test -p linkml_runtime --test diff_unique_keys` +Expected: PASS (all 11). + +- [ ] **Step 5: Full runtime suite** + +Run: `cargo test -p linkml_runtime` +Expected: PASS. + +- [ ] **Step 6: Commit** + +```bash +git add src/runtime/src/diff.rs src/runtime/tests/diff_unique_keys.rs +git commit -m "feat(runtime): patch resolves unique_keys-derived path segments" +``` + +--- + +### Task 6: the opt-in identity linter + +Two opt-in entry points in a new module. Neither is called from `validate_issues` or any load path — that is the spec's non-goal. + +**Files:** +- Create: `src/runtime/src/identity_lint.rs` +- Modify: `src/runtime/src/lib.rs` (add `pub mod identity_lint;` next to `pub mod diff;` at line 28; add two variants to `ValidationProblemType` at lines 150–157; re-export `identity_lint::{lint_element_identity, lint_instance_identity}`) +- Create: `src/runtime/tests/identity_lint.rs` + +**Interfaces:** +- Consumes: `ClassView::unique_keys()` (Task 1), `slot_is_opaque`/`OPAQUE_ANNOTATION` (Task 2), `element_identity_label` (Task 4), `SlotView::determine_slot_container_mode()` / `determine_slot_inline_mode()` / `get_range_class()` / `is_range_scalar()` (slotview.rs:500–564), `ValidationResultSink` (`push_warning`, `into_vec`), `SchemaView::{get_class_ids, get_class, converter}`. +- Produces: + - `ValidationProblemType::AmbiguousElementIdentity` and `ValidationProblemType::DuplicateElementIdentity` (new enum variants — the compiler will flag every `match` that needs extending, including the Python binding's problem-type stringification; extend them all). + - `pub fn lint_element_identity(sv: &SchemaView) -> Vec` — schema-level. + - `pub fn lint_instance_identity(value: &LinkMLInstance) -> Vec` — data-level. + +**Lint rules (schema-level), per class × slot:** +- Only `SlotContainerMode::List` slots are candidates (`Mapping` is keyed by construction, `SingleValue` has no elements). +- Skip `SlotInlineMode::Reference` (elements are references, not inlined — out of the spec's scope). +- Skip slots carrying `diff.linkml.io/ignore` or `diff.linkml.io/opaque` (identity declared: nowhere). +- Object range: OK if the range class has `key_or_identifier_slot()` or a non-empty `unique_keys()`. Otherwise warn. +- Scalar range (`is_range_scalar`): always warn (positional identity; the only declaration available is opaque). +- Severity: **Warning**, never error. `subject` = `vec![class_name, slot_name]`. + +**Lint rules (data-level), per `List` node in the instance tree:** +- Elements whose class declares a key/identifier or `unique_keys` and whose `element_identity_label` repeats within the container → one `DuplicateElementIdentity` warning per repeated label, at the container's path. +- Deliberately does **not** consult the opaque annotation (layering: `diff.linkml.io/*` never suppresses a schema-constraint finding). The remodel in the spec is what makes rings silent — their `Vertex` class declares nothing. +- Elements with no declared identity (no key, no unique_keys) are never flagged — repeated content is data, not an error. + +- [ ] **Step 1: Write the failing tests** + +`src/runtime/tests/identity_lint.rs` (same `fixture()` harness copied from `diff_opaque.rs`, plus these): + +```rust +use linkml_runtime::{lint_element_identity, lint_instance_identity, ValidationProblemType}; + +#[test] +fn schema_lint_flags_exactly_the_undeclared_positional_slots() { + let f = fixture(); + let warnings = lint_element_identity(&f.sv); + let mut flagged: Vec<(String, String)> = warnings + .iter() + .map(|w| (w.subject[0].clone(), w.subject[1].clone())) + .collect(); + flagged.sort(); + assert_eq!( + flagged, + vec![ + ("Service".to_string(), "plainPhoneNumber".to_string()), + ("Service".to_string(), "tags".to_string()), + ], + "everything else declares its identity source: {warnings:#?}" + ); + for w in &warnings { + assert_eq!(w.problem_type, ValidationProblemType::AmbiguousElementIdentity); + assert!(!w.severity.is_error(), "the linter warns, never errors"); + assert!( + w.detail.contains("unique_keys") && w.detail.contains("diff.linkml.io/opaque"), + "the warning must name the author's options: {}", + w.detail + ); + } +} + +#[test] +fn data_lint_flags_duplicate_declared_identities() { + let f = fixture(); + let dup = json!({"phoneNumber": "09/000.00.00", "hasNumberFunction": "Emergency_Number"}); + let inst = f.load(phones(vec![e(), dup])); + let warnings = lint_instance_identity(&inst); + assert_eq!(warnings.len(), 1, "{warnings:#?}"); + assert_eq!(warnings[0].problem_type, ValidationProblemType::DuplicateElementIdentity); + assert_eq!(warnings[0].subject, vec!["hasPhoneNumber".to_string()]); + assert!(!warnings[0].severity.is_error()); +} + +#[test] +fn data_lint_is_silent_on_clean_and_undeclared_data() { + let f = fixture(); + // unique phone functions, repeated scalar tags, repeated identity-less vertices + let inst = f.load(json!({ + "name": "svc", + "hasPhoneNumber": [e(), n()], + "tags": ["a", "a"], + "outline": [{"x": 1.0, "y": 2.0}, {"x": 1.0, "y": 2.0}] + })); + let warnings = lint_instance_identity(&inst); + assert!(warnings.is_empty(), "{warnings:#?}"); +} + +#[test] +fn data_lint_does_not_let_opaque_suppress_a_schema_constraint() { + let f = fixture(); + // archivedContacts is opaque, but Contact declares unique_keys: duplicates + // still violate the class's claim. diff vocabulary never silences schema truth. + let c = json!({"kind": "Emergency", "phone": "02/111.11.11"}); + let inst = f.load(json!({"name": "svc", "archivedContacts": [c.clone(), c]})); + let warnings = lint_instance_identity(&inst); + assert_eq!(warnings.len(), 1, "{warnings:#?}"); + assert_eq!(warnings[0].subject, vec!["archivedContacts".to_string()]); +} +``` + +(Reuse `e()`, `n()`, `phones()` builders from `diff_unique_keys.rs` — copy them in.) + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `cargo test -p linkml_runtime --test identity_lint` +Expected: FAIL to compile — module and variants don't exist. + +- [ ] **Step 3: Implement the module** + +`src/runtime/src/identity_lint.rs`: + +```rust +//! Opt-in element-identity linter. +//! +//! Answers, per multivalued inlined slot: **where does element identity come +//! from?** A key (or identifier) on the element class, a composed key +//! declared with `unique_keys`, or the `diff.linkml.io/opaque` annotation +//! (nowhere — the slot's value is replaced as a whole). A slot with none of +//! these produces positional deltas, which are ambiguous when several sources +//! produce deltas for the same object concurrently. +//! +//! Neither entry point is wired into loading or [`crate::validate_issues`]: +//! projects that do not opt in keep today's inferred semantics. +//! +//! When a slot is flagged, the data-model author has four options (worked +//! examples in `docs/superpowers/specs/2026-08-17-inlined-multivalued-element-identity-design.md`): +//! 1. declare a `key`/`identifier` slot on the element class; +//! 2. declare the identity as `unique_keys` (composed keys supported); +//! 3. annotate the slot `diff.linkml.io/opaque` — replace the value as a whole; +//! 4. remodel, when one class would need two identity answers. + +use crate::diff::{element_identity_label, slot_is_opaque, OPAQUE_ANNOTATION}; +use crate::{ + LinkMLInstance, ValidationProblemType, ValidationResult, ValidationResultSink, +}; +use linkml_schemaview::identifier::Identifier; +use linkml_schemaview::schemaview::SchemaView; +use std::collections::HashMap; + +/// Schema-level lint: warn for every multivalued inlined slot whose element +/// identity comes from nowhere. Warnings only — the schema stays usable. +pub fn lint_element_identity(sv: &SchemaView) -> Vec { + use linkml_schemaview::slotview::{SlotContainerMode, SlotInlineMode}; + let mut sink = ValidationResultSink::default(); + let conv = sv.converter(); + let mut class_ids = sv.get_class_ids(); + class_ids.sort(); + for class_id in class_ids { + let Ok(Some(class)) = sv.get_class(&Identifier::new(&class_id), &conv) else { + continue; + }; + for slot in class.slots() { + if slot.determine_slot_container_mode() != SlotContainerMode::List { + continue; + } + if slot.determine_slot_inline_mode() == SlotInlineMode::Reference { + continue; // elements are references, not inlined + } + if slot_is_opaque(slot) { + continue; // identity declared: nowhere, replace as a whole + } + if let Some(rc) = slot.get_range_class() { + if rc.key_or_identifier_slot().is_some() || !rc.unique_keys().is_empty() { + continue; + } + } + sink.push_warning( + ValidationProblemType::AmbiguousElementIdentity, + vec![class.name().to_string(), slot.name.clone()], + format!( + "elements of '{}.{}' have no declared identity: deltas are \ + positional and ambiguous under multi-sourced operation. \ + Declare a key/identifier or unique_keys on the element \ + class, annotate the slot with {} to replace the value as \ + a whole, or remodel.", + class.name(), + slot.name, + OPAQUE_ANNOTATION + ), + ); + } + } + sink.into_vec() +} + +/// Data-level lint: warn for every list container whose elements repeat a +/// declared identity (key/identifier or unique_keys value). +/// +/// Deliberately does NOT consult `diff.linkml.io/opaque`: a schema constraint +/// is class-level truth, and diff vocabulary never suppresses it. +pub fn lint_instance_identity(value: &LinkMLInstance) -> Vec { + let mut sink = ValidationResultSink::default(); + let mut path = Vec::new(); + walk(value, &mut path, &mut sink); + sink.into_vec() +} + +fn walk(v: &LinkMLInstance, path: &mut Vec, sink: &mut ValidationResultSink) { + match v { + LinkMLInstance::List { values, .. } => { + check_duplicates(values, path, sink); + for (i, child) in values.iter().enumerate() { + path.push(i.to_string()); + walk(child, path, sink); + path.pop(); + } + } + LinkMLInstance::Object { values, .. } | LinkMLInstance::Mapping { values, .. } => { + for (k, child) in values { + path.push(k.clone()); + walk(child, path, sink); + path.pop(); + } + } + LinkMLInstance::Scalar { .. } | LinkMLInstance::Null { .. } => {} + } +} + +fn check_duplicates( + values: &[LinkMLInstance], + path: &[String], + sink: &mut ValidationResultSink, +) { + let mut seen: HashMap = HashMap::new(); + for v in values { + if let Some(label) = element_identity_label(v) { + *seen.entry(label).or_insert(0) += 1; + } + } + let mut dups: Vec<(String, usize)> = + seen.into_iter().filter(|(_, n)| *n > 1).collect(); + dups.sort(); + for (label, n) in dups { + sink.push_warning( + ValidationProblemType::DuplicateElementIdentity, + path.to_vec(), + format!( + "{n} elements share the declared identity '{label}'; deltas \ + addressing it are ambiguous" + ), + ); + } +} +``` + +In `src/runtime/src/lib.rs`: +- add the two `ValidationProblemType` variants, +- `pub mod identity_lint;`, +- `pub use identity_lint::{lint_element_identity, lint_instance_identity};`. + +Then chase compile errors: every exhaustive `match` on `ValidationProblemType` (runtime, python crate) needs the two new arms — stringify them as `"ambiguous_element_identity"` / `"duplicate_element_identity"` following the existing naming style found at those match sites. + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `cargo test -p linkml_runtime --test identity_lint` +Expected: PASS (all 4). + +- [ ] **Step 5: Workspace build + runtime suite** + +Run: `cargo build --workspace && cargo test -p linkml_runtime` +Expected: PASS (workspace build catches the python-crate match arms). + +- [ ] **Step 6: Commit** + +```bash +git add src/runtime/src/identity_lint.rs src/runtime/src/lib.rs src/runtime/tests/identity_lint.rs src/python/src/lib.rs +git commit -m "feat(runtime): opt-in element-identity linter (schema + instance)" +``` + +--- + +### Task 7: Python bindings + +The library's real consumer (consolidator-server) is Python. Expose the two lint functions; `diff`/`patch` signatures are unchanged, so no other binding moves. + +**Files:** +- Modify: `src/python/src/lib.rs` (two new `#[pyfunction]`s + registration in the `_native` module at lines 739–756) +- Modify: `src/python/python/linkml_runtime_rust/_native.pyi` (two stubs) + +**Interfaces:** +- Consumes: `linkml_runtime::{lint_element_identity, lint_instance_identity}` (Task 6), the existing `validation_results_to_py` helper (python lib.rs:1070) and `PyValidationResult`. +- Produces: Python functions `lint_element_identity(schema_view) -> list[ValidationResult]` and `lint_instance_identity(instance) -> list[ValidationResult]`. + +- [ ] **Step 1: Implement the bindings** + +Follow `py_diff` (registered at python lib.rs:745) for how the wrapper types expose their inner `SchemaView`/`LinkMLInstance` — use the same accessor pattern, then: + +```rust +#[cfg_attr(feature = "stubgen", gen_stub_pyfunction)] +#[pyfunction] +#[pyo3(name = "lint_element_identity")] +fn py_lint_element_identity( + py: Python<'_>, + schema_view: &PySchemaView, +) -> PyResult>> { + validation_results_to_py( + py, + linkml_runtime::lint_element_identity(schema_view_inner(schema_view)), + ) +} + +#[cfg_attr(feature = "stubgen", gen_stub_pyfunction)] +#[pyfunction] +#[pyo3(name = "lint_instance_identity")] +fn py_lint_instance_identity( + py: Python<'_>, + instance: &PyLinkMLInstance, +) -> PyResult>> { + validation_results_to_py( + py, + linkml_runtime::lint_instance_identity(instance_inner(instance)), + ) +} +``` + +(`schema_view_inner`/`instance_inner` stand for however `py_diff`/`py_patch` reach the wrapped Rust values — reuse that exact mechanism, whatever it is named; also mirror the `gen_stub_pyfunction` usage of neighbouring functions, including whether they gate it behind the `stubgen` feature.) Register both with `m.add_function(wrap_pyfunction!(...))?;` next to `py_diff`. + +- [ ] **Step 2: Add the stubs** + +In `_native.pyi`, next to the `diff`/`patch` stubs: + +```python +def lint_element_identity(schema_view: SchemaView) -> list[ValidationResult]: ... +def lint_instance_identity(instance: LinkMLInstance) -> list[ValidationResult]: ... +``` + +(Match the actual parameter/class names used by the neighbouring stubs in that file.) + +- [ ] **Step 3: Verify it compiles** + +Run: `cargo check -p linkml_runtime_python` +Expected: clean. If the repo has a Python test suite under `src/python/`, also run it the way its README/CI does; otherwise compilation plus stub review is the gate here. + +- [ ] **Step 4: Commit** + +```bash +git add src/python/src/lib.rs src/python/python/linkml_runtime_rust/_native.pyi +git commit -m "feat(python): expose element-identity lint functions" +``` + +--- + +### Task 8: CLI flag on `linkml-schema-validate` + +**Files:** +- Modify: `src/tools/src/bin/linkml_schema_validate.rs` (new `--lint-identity` flag) +- Modify: `src/tools/Cargo.toml` only if the tools crate does not already depend on `linkml_runtime` (the diff/patch bins suggest it does). + +**Interfaces:** +- Consumes: `linkml_runtime::lint_element_identity` (Task 6). +- Produces: `linkml-schema-validate --lint-identity` prints one line per warning and (like warnings elsewhere in the tool) does not change the exit code. + +- [ ] **Step 1: Implement** + +Add to the `Args` struct: + +```rust + /// Opt-in: warn for multivalued inlined slots whose element identity + /// comes from nowhere (positional, ambiguous deltas in multi-sourced use). + #[arg(long, default_value_t = false)] + lint_identity: bool, +``` + +After the existing validation logic completes (both output formats), run and print: + +```rust + if args.lint_identity { + for w in linkml_runtime::lint_element_identity(&sv) { + println!("warning[{}]: {}", w.subject.join("."), w.detail); + } + } +``` + +(Adapt the printing to the tool's existing `OutputFormat` handling: in `Json` mode, emit the warnings as a JSON array the same way existing results are serialized there — follow the surrounding code.) + +- [ ] **Step 2: Verify manually against the fixture** + +Run: `cargo run -p linkml_tools --bin linkml-schema-validate -- src/runtime/tests/data/identity.yaml --lint-identity` +Expected: exactly two warning lines — `Service.plainPhoneNumber` and `Service.tags` — plus whatever the normal validation prints. Then run without the flag and confirm no identity warnings appear (opt-in). + +- [ ] **Step 3: Build check** + +Run: `cargo build -p linkml_tools` +Expected: clean. + +- [ ] **Step 4: Commit** + +```bash +git add src/tools/src/bin/linkml_schema_validate.rs src/tools/Cargo.toml +git commit -m "feat(tools): --lint-identity flag on linkml-schema-validate" +``` + +--- + +### Task 9: docs polish + final verification + +**Files:** +- Modify: `src/runtime/src/diff.rs` (rustdoc only: extend the `Delta` doc comment's path-segment sentence at lines 35–50 to mention unique_keys-derived segments and the composite JSON-array encoding; extend the `diff` doc comment at lines 103–111 with one paragraph on opaque slots) +- Modify: `src/runtime/src/identity_lint.rs` (confirm the module doc lists the four author options and links the spec — it is the authoring-guide surface the spec promises) + +**Interfaces:** none — documentation and the final gate. + +- [ ] **Step 1: Write the rustdoc additions** + +On `Delta` (after the existing segment sentence, line 38–39): + +``` +/// For inlined lists whose element class declares `unique_keys`, the segment is +/// the unique-key value (single-slot keys), or the JSON array encoding of the +/// values in `unique_key_slots` order (composite keys), e.g. +/// `["Emergency","02/111.11.11"]`. +``` + +On `diff` (after the null/missing semantics list): + +``` +/// Slots annotated `diff.linkml.io/opaque` stop all recursion: any change at or +/// below the slot is described as a single whole-value `Update` at the slot +/// path. See [`OPAQUE_ANNOTATION`]. +``` + +- [ ] **Step 2: Full workspace gate** + +Run: + +```bash +cargo fmt --all +cargo clippy --workspace --all-targets -- -D warnings +cargo test --workspace +``` + +Expected: all green. Fix anything clippy raises in the new code only. + +- [ ] **Step 3: Spot-check the spec's headline scenarios end to end** + +Run: `cargo test -p linkml_runtime --test diff_unique_keys --test diff_opaque --test identity_lint` +Expected: PASS — this is the spec's Example 1 (phones via unique_keys), Example 2 (opaque ring + bare Vertex + polygon unique key), and the linter contract. + +- [ ] **Step 4: Commit** + +```bash +git add src/runtime/src/diff.rs src/runtime/src/identity_lint.rs +git commit -m "docs(runtime): document opaque annotation and unique_keys path segments" +``` From a9f6760c0cefba14bdece5a381f379a4b30e1022 Mon Sep 17 00:00:00 2001 From: ejsyx Date: Tue, 18 Aug 2026 14:21:34 +0200 Subject: [PATCH 04/58] feat(schemaview): ClassView::unique_keys merged across is_a and mixins Co-Authored-By: Claude Fable 5 --- src/schemaview/src/classview.rs | 45 +++++++++++++++++++++- src/schemaview/tests/data/unique_keys.yaml | 35 +++++++++++++++++ src/schemaview/tests/unique_keys.rs | 44 +++++++++++++++++++++ 3 files changed, 122 insertions(+), 2 deletions(-) create mode 100644 src/schemaview/tests/data/unique_keys.yaml create mode 100644 src/schemaview/tests/unique_keys.rs diff --git a/src/schemaview/src/classview.rs b/src/schemaview/src/classview.rs index ca23d95..7454a0c 100644 --- a/src/schemaview/src/classview.rs +++ b/src/schemaview/src/classview.rs @@ -1,8 +1,8 @@ -use std::collections::{HashMap, HashSet}; +use std::collections::{HashMap, HashSet, VecDeque}; use std::sync::{Arc, OnceLock}; use crate::converter::Converter; -use linkml_meta::{ClassDefinition, SchemaDefinition, SlotDefinition}; +use linkml_meta::{ClassDefinition, SchemaDefinition, SlotDefinition, UniqueKey}; use crate::identifier::{Identifier, IdentifierError}; use crate::schemaview::{CanonicalIds, SchemaView, SchemaViewError}; @@ -621,6 +621,47 @@ impl ClassView { .find(|s| s.definition().identifier.unwrap_or(false)) } + /// Returns the class's `unique_keys`, merged across the inheritance chain + /// (`is_a` parents and mixins), with the nearest declaration winning per name. + /// + /// The result is sorted by unique key name: declaration order is lost in the + /// underlying map, and consumers (such as diff path segments) need a + /// deterministic order. + pub fn unique_keys(&self) -> Vec<(String, UniqueKey)> { + let mut merged: HashMap = HashMap::new(); + let mut queue: VecDeque = VecDeque::from([self.clone()]); + let mut seen: HashSet = HashSet::new(); + + while let Some(cv) = queue.pop_front() { + if !seen.insert(cv.canonical_uri().to_string()) { + continue; + } + if let Some(uks) = cv.def().unique_keys.as_ref() { + for (name, uk) in uks { + merged.entry(name.clone()).or_insert_with(|| (**uk).clone()); + } + } + if let Ok(Some(parent)) = cv.parent_class() { + queue.push_back(parent); + } + if let Some(mixins) = &cv.data.class.mixins { + if let Some(conv) = cv.data.sv.converter_for_schema(&cv.data.schema_uri) { + for mixin in mixins { + if let Ok(Some(mixin_view)) = + cv.data.sv.get_class(&Identifier::new(mixin), &conv) + { + queue.push_back(mixin_view); + } + } + } + } + } + + let mut out: Vec<(String, UniqueKey)> = merged.into_iter().collect(); + out.sort_by(|a, b| a.0.cmp(&b.0)); + out + } + fn collect_ancestors_map( class_view: &ClassView, include_mixins: bool, diff --git a/src/schemaview/tests/data/unique_keys.yaml b/src/schemaview/tests/data/unique_keys.yaml new file mode 100644 index 0000000..37cdcf1 --- /dev/null +++ b/src/schemaview/tests/data/unique_keys.yaml @@ -0,0 +1,35 @@ +id: http://example.org/unique_keys +name: unique_keys +prefixes: + ex: http://example.org/unique_keys/ +default_prefix: ex +default_range: string + +classes: + Base: + unique_keys: + by_code: + unique_key_slots: [code] + shared_name: + unique_key_slots: [base_field] + attributes: + code: {range: string} + base_field: {range: string} + MixinCls: + mixin: true + unique_keys: + by_tag: + unique_key_slots: [tag] + attributes: + tag: {range: string} + Child: + is_a: Base + mixins: [MixinCls] + unique_keys: + shared_name: + unique_key_slots: [child_field] + attributes: + child_field: {range: string} + Plain: + attributes: + whatever: {range: string} diff --git a/src/schemaview/tests/unique_keys.rs b/src/schemaview/tests/unique_keys.rs new file mode 100644 index 0000000..94b102d --- /dev/null +++ b/src/schemaview/tests/unique_keys.rs @@ -0,0 +1,44 @@ +use linkml_schemaview::identifier::Identifier; +use linkml_schemaview::io::from_yaml; +use linkml_schemaview::schemaview::SchemaView; +use std::path::PathBuf; + +fn fixture() -> SchemaView { + let mut p = PathBuf::from(env!("CARGO_MANIFEST_DIR")); + p.push("tests/data/unique_keys.yaml"); + let schema = from_yaml(&p).unwrap(); + let mut sv = SchemaView::new(); + sv.add_schema(schema).unwrap(); + sv +} + +#[test] +fn unique_keys_merge_across_is_a_and_mixins_nearest_wins() { + let sv = fixture(); + let conv = sv.converter(); + let child = sv + .get_class(&Identifier::new("Child"), &conv) + .unwrap() + .expect("class not found"); + let uks = child.unique_keys(); + let names: Vec<&str> = uks.iter().map(|(n, _)| n.as_str()).collect(); + // name-sorted, merged from Base (by_code), MixinCls (by_tag), Child (shared_name override) + assert_eq!(names, vec!["by_code", "by_tag", "shared_name"]); + let shared = &uks.iter().find(|(n, _)| n == "shared_name").unwrap().1; + assert_eq!( + shared.unique_key_slots, + vec!["child_field".to_string()], + "the nearest declaration must win" + ); +} + +#[test] +fn class_without_unique_keys_yields_empty() { + let sv = fixture(); + let conv = sv.converter(); + let plain = sv + .get_class(&Identifier::new("Plain"), &conv) + .unwrap() + .expect("class not found"); + assert!(plain.unique_keys().is_empty()); +} From 673db0463630d2915f99c9349585c86094974e7f Mon Sep 17 00:00:00 2001 From: ejsyx Date: Tue, 18 Aug 2026 14:28:48 +0200 Subject: [PATCH 05/58] =?UTF-8?q?docs:=20uniform=20keyed=20matching=20?= =?UTF-8?q?=E2=80=94=20deliberate=20compat=20break=20per=20spike=20finding?= =?UTF-8?q?s?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Fable 5 --- ...18-inlined-multivalued-element-identity.md | 135 +++++++++++------- ...ned-multivalued-element-identity-design.md | 4 +- 2 files changed, 84 insertions(+), 55 deletions(-) diff --git a/docs/superpowers/plans/2026-08-18-inlined-multivalued-element-identity.md b/docs/superpowers/plans/2026-08-18-inlined-multivalued-element-identity.md index abf31c2..acae782 100644 --- a/docs/superpowers/plans/2026-08-18-inlined-multivalued-element-identity.md +++ b/docs/superpowers/plans/2026-08-18-inlined-multivalued-element-identity.md @@ -12,7 +12,7 @@ ## Global Constraints -- **Non-goal (spec):** inferred semantics must not change for schemas that declare nothing. Every existing test passes unchanged; the positional list branch's output (including its path segments) stays byte-identical. Never add a uniqueness guard to key/identifier-labelled matching — only to `unique_keys`-derived matching. +- **Non-goal + deliberate break (spec, Non-goal section):** slots that declare nothing keep positional semantics, but keyed matching becomes uniform: a list matches by identity iff every element on both sides yields an identity label (key/identifier first, else `unique_keys`) AND the labels are unique within each side; every other case is positional with plain numeric path segments. This deliberately removes (a) opportunistic key labels mixed into positional paths and (b) the silent collapse of duplicate key values under keyed matching. Existing tests that assert those two removed behaviours are updated — the implementer's report must list each updated test with its old assertion and why it changed. All other existing tests pass unchanged. - **"Report, never guess" (spec):** a patch that cannot locate its target unambiguously returns `Ok(false)` so the path lands in `PatchTrace::failed`. No fuzzy fallbacks. - **Layering (spec):** validation/lint code never reads `diff.linkml.io/*` to *suppress* a schema-constraint finding. The schema-level linter may skip opaque slots (identity is declared: nowhere); the data-level duplicate check never consults the annotation. - `PatchTrace::failed` stays `Vec>` — do not import the abandoned branch's `PatchFailure` struct. @@ -259,6 +259,11 @@ classes: range: Label multivalued: true inlined: true + # keyed class inlined as list: the uniform guard applies to key labels too + labelList: + range: Label + multivalued: true + inlined_as_list: true # reference list (not inlined): out of the linter's scope operators: range: Operator @@ -680,10 +685,10 @@ git commit -m "feat(runtime): patch refuses to descend below an opaque slot" - `pub(crate) fn element_identity_label(v: &LinkMLInstance) -> Option` — `element_key_label` first, `element_unique_key_label` as fallback. - `fn scalar_slot_string(values: &HashMap, slot_name: &str) -> Option`. -**Precedence and guard rules (from the spec + non-goal):** +**Precedence and guard rules (from the spec's Non-goal section — the uniform rule):** 1. opaque > key/identifier > unique_keys > positional. -2. `unique_keys` labels participate in *matching only*, never in positional path segments — the positional branch keeps using `element_key_label`, so its output stays byte-identical to today. -3. A `unique_keys` label is a class-level claim, weaker than a key: if any element's label would come from `unique_keys` and the labels are not unique within both lists, fall back to positional matching (dirty data must keep today's behaviour). Key/identifier-labelled lists get **no** new uniqueness guard. +2. A list is matched by identity iff every element on both sides yields an identity label (`element_identity_label`) AND the labels are unique within each side. The guard is uniform — it applies to key/identifier labels exactly as to `unique_keys` labels (removing today's silent collapse of duplicate keys). +3. The positional branch uses plain numeric segments (`i.to_string()`) only — the old opportunistic mixing of key values into positional paths is removed. - [ ] **Step 1: Write the failing tests** @@ -794,6 +799,24 @@ fn duplicate_unique_key_data_falls_back_to_positional() { ); } +#[test] +fn duplicate_key_data_falls_back_to_positional_not_collapse() { + let f = fixture(); + // Label declares `lang` as key; a list that repeats the key must not be + // silently collapsed by keyed matching — uniform guard, positional fallback. + let before = json!({"name": "svc", "labelList": [ + {"lang": "nl", "text": "a"}, {"lang": "nl", "text": "b"}]}); + let after = json!({"name": "svc", "labelList": [ + {"lang": "nl", "text": "a"}, {"lang": "nl", "text": "B"}]}); + let deltas = diff2(&f, before, after); + let delta = only(&deltas); + assert_eq!( + delta.path, + vec!["labelList".to_string(), "1".to_string(), "text".to_string()], + "duplicate key labels must fall back to plain numeric segments" + ); +} + #[test] fn undeclared_class_keeps_positional_cascade() { let f = fixture(); @@ -851,7 +874,7 @@ fn inherited_unique_keys_drive_matching() { - [ ] **Step 2: Run tests to verify they fail** Run: `cargo test -p linkml_runtime --test diff_unique_keys` -Expected: the matching tests FAIL (positional deltas today); `duplicate_unique_key_data_falls_back_to_positional` and `undeclared_class_keeps_positional_cascade` PASS already (they assert today's behaviour and must stay green throughout). +Expected: the matching tests FAIL (positional deltas today); `duplicate_key_data_falls_back_to_positional_not_collapse` also FAILS today (current keyed matching silently collapses the duplicates — the defect the uniform guard removes). `duplicate_unique_key_data_falls_back_to_positional` and `undeclared_class_keeps_positional_cascade` PASS already and must stay green throughout. - [ ] **Step 3: Implement** @@ -913,18 +936,6 @@ pub(crate) fn element_identity_label(v: &LinkMLInstance) -> Option { element_key_label(v).or_else(|| element_unique_key_label(v)) } -/// Whether any element's matching label would come from `unique_keys` rather -/// than a key/identifier slot. -fn labels_from_unique_keys(elements: &[LinkMLInstance]) -> bool { - elements.iter().any(|v| { - if let LinkMLInstance::Object { class, .. } = v { - class.key_or_identifier_slot().is_none() && !class.unique_keys().is_empty() - } else { - false - } - }) -} - fn labels_are_unique(elements: &[LinkMLInstance], label: F) -> bool where F: Fn(&LinkMLInstance) -> Option, @@ -938,29 +949,23 @@ The `(List, List)` arm becomes: ```rust (LinkMLInstance::List { values: sl, .. }, LinkMLInstance::List { values: tl, .. }) => { - // Positional path segments keep using key/identifier labels ONLY, so the - // positional branch's output is byte-identical to before unique_keys - // support existed. - let label = |v: &LinkMLInstance| -> Option { element_key_label(v) }; let identity = |v: &LinkMLInstance| -> Option { element_identity_label(v) }; - let mut keyed = sl.iter().all(|v| identity(v).is_some()) - && tl.iter().all(|v| identity(v).is_some()); - // A unique_keys label is a class-level claim, not a per-element identity: - // data may legitimately violate it (it must still load and round-trip). - // Matching duplicates by such a label would silently collapse elements, - // so refuse to guess and fall back to positional. - if keyed - && (labels_from_unique_keys(sl) || labels_from_unique_keys(tl)) - && !(labels_are_unique(sl, &identity) && labels_are_unique(tl, &identity)) - { - keyed = false; - } + // Uniform rule (spec, Non-goal section): keyed matching iff every element + // on both sides carries an identity label and the labels are unique + // within each side. Duplicate labels (a list repeating a key, or data + // violating a unique_keys claim) fall back to positional — matching + // duplicates by label would silently collapse elements. + let keyed = sl.iter().all(|v| identity(v).is_some()) + && tl.iter().all(|v| identity(v).is_some()) + && labels_are_unique(sl, &identity) + && labels_are_unique(tl, &identity); if keyed { // ... identical to the current keyed block (lines 233-266), // with every `label(..)` call replaced by `identity(..)` ... } else { - // ... the current positional block (lines 267-296), UNCHANGED, - // still using `label` for opportunistic segments ... + // ... the current positional block (lines 267-296), with the + // opportunistic label chain REPLACED by plain numeric segments: + // every path segment is `i.to_string()`. } } ``` @@ -968,12 +973,12 @@ The `(List, List)` arm becomes: - [ ] **Step 4: Run tests to verify they pass** Run: `cargo test -p linkml_runtime --test diff_unique_keys` -Expected: PASS (all 8). +Expected: PASS (all 9). -- [ ] **Step 5: Full runtime suite — the non-goal gate** +- [ ] **Step 5: Full runtime suite — the compatibility gate** Run: `cargo test -p linkml_runtime && cargo test -p schemaview` -Expected: PASS. Any change in `diff.rs`/`diff_identifier.rs`/`trace.rs` test output means the positional branch or key-labelled matching drifted — fix the implementation, not the tests. +Expected: mostly PASS. Existing tests that assert the two deliberately removed behaviours — opportunistic key labels inside positional paths, or keyed matching of lists with duplicate labels — may fail; update each such test to the uniform rule and list every updated test in your report with its old assertion and why it changed. Any other failure means the implementation drifted — fix the implementation, not the test. - [ ] **Step 6: Commit** @@ -991,8 +996,8 @@ git commit -m "feat(runtime): match inlined list elements by unique_keys-derived - Test: `src/runtime/tests/diff_unique_keys.rs` (extend) **Interfaces:** -- Consumes: `element_key_label`, `element_unique_key_label` (Task 4). -- Produces: patch behaviour only. +- Consumes: `element_identity_label` (Task 4). +- Produces: patch behaviour only. Note the deliberate break (Global Constraints): duplicate labels in the current list now refuse (`Ok(false)` → `trace.failed`) instead of first-match; this applies to key/identifier labels exactly as to `unique_keys` labels. - [ ] **Step 1: Write the failing tests** @@ -1046,6 +1051,24 @@ fn patch_reports_ambiguous_unique_key_instead_of_guessing() { assert!(patched.equals(&golden, true), "nothing may change"); } +#[test] +fn patch_refuses_ambiguous_duplicate_key_labels() { + let f = fixture(); + // Duplicate key/identifier labels refuse exactly like duplicate + // unique_keys labels — the uniform rule on the patch side. + let golden = f.load(json!({"name": "svc", "labelList": [ + {"lang": "nl", "text": "a"}, {"lang": "nl", "text": "b"}]})); + let delta = Delta { + path: vec!["labelList".to_string(), "nl".to_string(), "text".to_string()], + op: DeltaOp::Update, + old: Some(json!("a")), + new: Some(json!("z")), + }; + let (patched, trace) = patch(&golden, &[delta.clone()], PatchOptions::default()).unwrap(); + assert_eq!(trace.failed, vec![delta.path.clone()]); + assert!(patched.equals(&golden, true), "nothing may change"); +} + #[test] fn unique_key_deltas_round_trip_through_patch() { let f = fixture(); @@ -1071,18 +1094,23 @@ Expected: the two locate tests FAIL (segment `Emergency_Number` resolves to no i - [ ] **Step 3: Implement** -Extend `resolve_list_index` (diff.rs:530). Keep the existing numeric-index attempt and the existing key/identifier `find_map` block **byte-identical** (do not rewrite it in terms of `element_key_label` — its scalar comparison semantics differ subtly for non-string keys). Append after it: +Rewrite `resolve_list_index` (diff.rs:530) as one unified resolver. The numeric-index attempt stays first and unchanged; the old key/identifier `find_map` block is **replaced** by an identity-label pass that uses the same precedence and stringification diff uses to build segments (making diff→patch symmetric), and that refuses ambiguity: ```rust - // unique_keys-derived location. Only an unambiguous hit counts: if the - // golden record drifted into duplicate labels, locating "the" element - // would be a guess — return None so the delta is reported as failed. +fn resolve_list_index(values: &[LinkMLInstance], key: &str) -> Option { + if let Ok(idx) = key.parse::() { + if idx < values.len() { + return Some(idx); + } + } + // Identity-label location (key/identifier first, else unique_keys) — the + // same precedence and stringification diff uses to build the segment. + // Only an unambiguous hit counts: if the current list holds duplicate + // labels, locating "the" element would be a guess — return None so the + // delta is reported as failed. let mut hit: Option = None; for (i, v) in values.iter().enumerate() { - if element_key_label(v).is_some() { - continue; // this element is addressed by its key, handled above - } - if element_unique_key_label(v).as_deref() == Some(key) { + if element_identity_label(v).as_deref() == Some(key) { if hit.is_some() { return None; } @@ -1090,27 +1118,26 @@ Extend `resolve_list_index` (diff.rs:530). Keep the existing numeric-index attem } } hit +} ``` -(The existing key block ends with a `find_map(...)` expression; restructure to `let by_key = ...; if by_key.is_some() { return by_key; }` followed by the code above.) - Note on `Add` deltas: an `Add` whose unique-key segment resolves to no element takes the existing `idx_opt = None` append path in `apply_list_leaf_delta` (line 687) — that is correct and needs no change. - [ ] **Step 4: Run tests to verify they pass** Run: `cargo test -p linkml_runtime --test diff_unique_keys` -Expected: PASS (all 11). +Expected: PASS (all 12). -- [ ] **Step 5: Full runtime suite** +- [ ] **Step 5: Full runtime suite — the compatibility gate** Run: `cargo test -p linkml_runtime` -Expected: PASS. +Expected: mostly PASS. Existing tests that assert the removed first-match-on-duplicate-labels patch behaviour may fail; update each such test to the uniform refuse-ambiguity rule and list every updated test in your report with its old assertion and why it changed. Any other failure means the implementation drifted — fix the implementation, not the test. - [ ] **Step 6: Commit** ```bash git add src/runtime/src/diff.rs src/runtime/tests/diff_unique_keys.rs -git commit -m "feat(runtime): patch resolves unique_keys-derived path segments" +git commit -m "feat(runtime): patch resolves identity-label path segments, refuses ambiguity" ``` --- diff --git a/docs/superpowers/specs/2026-08-17-inlined-multivalued-element-identity-design.md b/docs/superpowers/specs/2026-08-17-inlined-multivalued-element-identity-design.md index aa22288..19d6a40 100644 --- a/docs/superpowers/specs/2026-08-17-inlined-multivalued-element-identity-design.md +++ b/docs/superpowers/specs/2026-08-17-inlined-multivalued-element-identity-design.md @@ -19,7 +19,9 @@ The offered options are: ## Non-goal -The inferred semantics do **not** change for downstream projects not opting in to the linter. The positional index path deltas produced today are still valuable for projects not dealing with multiple sources producing deltas for the same object at the same time. +Positional index deltas remain the inferred semantics for slots that declare nothing — they are still valuable for projects not dealing with multiple sources producing deltas for the same object at the same time, and the opt-in linter is the only place that complains about them. + +One deliberate compatibility break ships with this design (a spike finding): **keyed matching becomes uniform**. A list is matched by element identity only when every element on both sides yields an identity label (key/identifier first, else the class's `unique_keys`) and the labels are unique within each side; path segments are then the labels. In every other case matching is positional and path segments are plain numeric indices. This removes two behaviours of the old fallback: key values were opportunistically mixed into positional paths, and duplicate key values within a "keyed" list were silently collapsed by the matcher. Consumers see cleaner, uniform delta paths; `patch` keeps accepting both numeric and label segments, and reports a delta as failed instead of guessing when a label matches more than one element. ## Proposed solution From fdcd37380a81558c766fab6a9d9ab2a1e338a988 Mon Sep 17 00:00:00 2001 From: ejsyx Date: Tue, 18 Aug 2026 14:31:38 +0200 Subject: [PATCH 06/58] feat(runtime): diff.linkml.io/opaque stops recursion, replaces whole value --- src/runtime/src/diff.rs | 28 +++++ src/runtime/src/lib.rs | 4 +- src/runtime/tests/data/identity.yaml | 160 +++++++++++++++++++++++++++ src/runtime/tests/diff_opaque.rs | 143 ++++++++++++++++++++++++ 4 files changed, 334 insertions(+), 1 deletion(-) create mode 100644 src/runtime/tests/data/identity.yaml create mode 100644 src/runtime/tests/diff_opaque.rs diff --git a/src/runtime/src/diff.rs b/src/runtime/src/diff.rs index 9c186ab..936a5dd 100644 --- a/src/runtime/src/diff.rs +++ b/src/runtime/src/diff.rs @@ -20,6 +20,23 @@ fn slot_is_ignored(slot: &SlotView) -> bool { .unwrap_or(false) } +/// Slot annotation declaring that element identity comes from nowhere: stop +/// all recursion, the slot's value is one atomic unit. Any change below the +/// slot is described as a single whole-value `Update` at the slot path, and +/// `patch` refuses paths that descend below it. +pub const OPAQUE_ANNOTATION: &str = "diff.linkml.io/opaque"; + +pub(crate) fn slot_is_opaque(slot: &SlotView) -> bool { + if slot.definitions().is_empty() { + return false; + } + slot.definition() + .annotations + .as_ref() + .map(|a| a.contains_key(OPAQUE_ANNOTATION)) + .unwrap_or(false) +} + /// Operation applied by a [`Delta`]. #[derive(Clone, Debug, PartialEq, Serialize, Deserialize)] #[serde(rename_all = "lowercase")] @@ -122,6 +139,17 @@ pub fn diff(source: &LinkMLInstance, target: &LinkMLInstance, opts: DiffOptions) if slot_is_ignored(sl) { return; } + if slot_is_opaque(sl) { + if !s.equals(t, opts.treat_missing_as_null) { + out.push(Delta { + path: path.clone(), + op: DeltaOp::Update, + old: Some(s.to_json()), + new: Some(t.to_json()), + }); + } + return; + } } match (s, t) { ( diff --git a/src/runtime/src/lib.rs b/src/runtime/src/lib.rs index 65dea99..aa5fc29 100644 --- a/src/runtime/src/lib.rs +++ b/src/runtime/src/lib.rs @@ -46,7 +46,9 @@ pub use blame::{ blame_map_to_paths, format_blame_map, format_blame_map_with, get_blame_info, patch_with_blame, record_blame_from_trace, }; -pub use diff::{diff, patch, Delta, DeltaOp, DiffOptions, PatchOptions, PatchTrace}; +pub use diff::{ + diff, patch, Delta, DeltaOp, DiffOptions, PatchOptions, PatchTrace, OPAQUE_ANNOTATION, +}; #[derive(Debug)] pub struct LinkMLError { validation_issues: Vec, diff --git a/src/runtime/tests/data/identity.yaml b/src/runtime/tests/data/identity.yaml new file mode 100644 index 0000000..caeafb6 --- /dev/null +++ b/src/runtime/tests/data/identity.yaml @@ -0,0 +1,160 @@ +id: https://w3id.org/linkml/examples/identity +name: identity +description: |- + Shared fixture for element-identity features: unique_keys, the + `diff.linkml.io/opaque` annotation, and the identity linter. +license: https://creativecommons.org/publicdomain/zero/1.0/ +imports: + - linkml:types +prefixes: + identity: https://w3id.org/linkml/examples/identity/ + linkml: https://w3id.org/linkml/ +default_prefix: identity +default_range: string + +classes: + Service: + attributes: + name: {range: string} + # spec Example 1 with its correct resolution: identity = the function + hasPhoneNumber: + range: ServicePhoneNumber + multivalued: true + inlined_as_list: true + # same shape, nothing declared: must keep positional behaviour; linter flags it + plainPhoneNumber: + range: PlainPhoneNumber + multivalued: true + inlined_as_list: true + # inherited unique_keys (via is_a) must also drive matching + escalation: + range: EmergencyPhoneNumber + multivalued: true + inlined_as_list: true + # composite two-slot unique key + contacts: + range: Contact + multivalued: true + inlined_as_list: true + # opaque + a keyed element class: data lint must still check unique_keys + archivedContacts: + range: Contact + multivalued: true + inlined_as_list: true + annotations: + diff.linkml.io/opaque: true + # scalar list, nothing declared: positional; linter flags it + tags: + range: string + multivalued: true + # scalar list declared opaque + opaqueTags: + range: string + multivalued: true + annotations: + diff.linkml.io/opaque: true + # spec Example 2's ring: opaque list of bare vertices + outline: + range: Vertex + multivalued: true + inlined_as_list: true + annotations: + diff.linkml.io/opaque: true + # single-valued opaque object: "stop all recursion" is not list-specific + profile: + range: Profile + inlined: true + annotations: + diff.linkml.io/opaque: true + # keyed class inlined as dict: already unambiguous, linter stays silent + labels: + range: Label + multivalued: true + inlined: true + # keyed class inlined as list: the uniform guard applies to key labels too + labelList: + range: Label + multivalued: true + inlined_as_list: true + # reference list (not inlined): out of the linter's scope + operators: + range: Operator + multivalued: true + inlined: false + area: + range: AreaLocation + inlined: true + + ServicePhoneNumber: + unique_keys: + one_number_per_function: + unique_key_slots: [hasNumberFunction] + attributes: + phoneNumber: {range: string} + hasNumberFunction: {range: NumberFunction, required: true} + + PlainPhoneNumber: + attributes: + phoneNumber: {range: string} + hasNumberFunction: {range: NumberFunction, required: true} + + EmergencyPhoneNumber: + is_a: ServicePhoneNumber + attributes: + note: {range: string} + + Contact: + unique_keys: + contact_identity: + unique_key_slots: [kind, phone] + attributes: + kind: {range: string, required: true} + phone: {range: string, required: true} + note: {range: string} + + Label: + attributes: + lang: {range: string, key: true, required: true} + text: {range: string} + + Profile: + attributes: + bio: {range: string} + motto: {range: string} + + Operator: + attributes: + opId: {range: string, identifier: true} + opName: {range: string} + + AreaLocation: + attributes: + polygons: + range: Polygon + multivalued: true + inlined_as_list: true + + Polygon: + unique_keys: + one_polygon_per_positioning_system: + unique_key_slots: [positioningSystemType] + attributes: + positioningSystemType: {range: uri, required: true} + coordinates: + range: Vertex + multivalued: true + inlined_as_list: true + annotations: + diff.linkml.io/opaque: true + + Vertex: + attributes: + x: {range: float} + y: {range: float} + +enums: + NumberFunction: + permissible_values: + Emergency_Number: + Non_Urgent_Communication: + Operator: diff --git a/src/runtime/tests/diff_opaque.rs b/src/runtime/tests/diff_opaque.rs new file mode 100644 index 0000000..f82f7e1 --- /dev/null +++ b/src/runtime/tests/diff_opaque.rs @@ -0,0 +1,143 @@ +use linkml_runtime::{ + diff, load_json_str, patch, Delta, DeltaOp, DiffOptions, LinkMLInstance, PatchOptions, +}; +use linkml_schemaview::identifier::{converter_from_schema, Identifier}; +use linkml_schemaview::io::from_yaml; +use linkml_schemaview::schemaview::{ClassView, SchemaView}; +use linkml_schemaview::Converter; +use serde_json::{json, Value as JsonValue}; +use std::path::PathBuf; + +struct Fixture { + sv: SchemaView, + conv: Converter, + service: ClassView, +} + +fn fixture() -> Fixture { + let mut p = PathBuf::from(env!("CARGO_MANIFEST_DIR")); + p.push("tests/data/identity.yaml"); + let schema = from_yaml(&p).unwrap(); + let mut sv = SchemaView::new(); + sv.add_schema(schema.clone()).unwrap(); + let conv = converter_from_schema(&schema); + let service = sv + .get_class(&Identifier::new("Service"), &conv) + .unwrap() + .expect("class not found"); + Fixture { sv, conv, service } +} + +impl Fixture { + fn load(&self, v: JsonValue) -> LinkMLInstance { + load_json_str(&v.to_string(), &self.sv, &self.service, &self.conv) + .unwrap() + .into_instance() + .unwrap() + } +} + +fn diff2(f: &Fixture, before: JsonValue, after: JsonValue) -> Vec { + diff(&f.load(before), &f.load(after), DiffOptions::new(true)) +} + +fn only(deltas: &[Delta]) -> &Delta { + assert_eq!(deltas.len(), 1, "expected exactly one delta: {deltas:#?}"); + &deltas[0] +} + +fn square() -> Vec { + vec![ + json!({"x": 4.35, "y": 50.85}), + json!({"x": 4.36, "y": 50.85}), + json!({"x": 4.36, "y": 50.86}), + json!({"x": 4.35, "y": 50.86}), + ] +} + +fn outline(items: Vec) -> JsonValue { + json!({"name": "svc", "outline": items}) +} + +#[test] +fn opaque_ring_edit_is_one_whole_slot_update() { + let f = fixture(); + + let mut moved = square(); + moved[1] = json!({"x": 4.37, "y": 50.85}); + let mut inserted = square(); + inserted.insert(2, json!({"x": 4.365, "y": 50.855})); + let dropped_first = square()[1..].to_vec(); + let mut reversed = square(); + reversed.reverse(); + + for (label, after) in [ + ("move one vertex", moved), + ("insert a vertex mid-ring", inserted), + ("drop the first vertex", dropped_first), + ("reverse the ring", reversed), + ] { + let deltas = diff2(&f, outline(square()), outline(after)); + let delta = only(&deltas); + assert_eq!(delta.path, vec!["outline".to_string()], "{label}"); + assert_eq!(delta.op, DeltaOp::Update, "{label}"); + assert_eq!( + delta + .old + .as_ref() + .and_then(|v| v.as_array()) + .map(|a| a.len()), + Some(4), + "{label}: old must be the whole slot" + ); + assert!(delta.new.as_ref().is_some_and(|v| v.is_array()), "{label}"); + } +} + +#[test] +fn opaque_slot_unchanged_emits_nothing() { + let f = fixture(); + let deltas = diff2(&f, outline(square()), outline(square())); + assert!(deltas.is_empty(), "{deltas:#?}"); +} + +#[test] +fn opaque_scalar_list_is_one_whole_slot_update() { + let f = fixture(); + let before = json!({"name": "svc", "opaqueTags": ["a", "b"]}); + let after = json!({"name": "svc", "opaqueTags": ["b", "c", "d"]}); + let deltas = diff2(&f, before, after); + let delta = only(&deltas); + assert_eq!(delta.path, vec!["opaqueTags".to_string()]); + assert_eq!(delta.op, DeltaOp::Update); +} + +#[test] +fn opaque_single_valued_object_is_one_whole_value_update() { + let f = fixture(); + let before = json!({"name": "svc", "profile": {"bio": "b", "motto": "old"}}); + let after = json!({"name": "svc", "profile": {"bio": "b", "motto": "new"}}); + let deltas = diff2(&f, before, after); + let delta = only(&deltas); + assert_eq!(delta.path, vec!["profile".to_string()]); + assert_eq!(delta.op, DeltaOp::Update); + assert_eq!(delta.old, Some(json!({"bio": "b", "motto": "old"}))); + assert_eq!(delta.new, Some(json!({"bio": "b", "motto": "new"}))); +} + +#[test] +fn opaque_whole_slot_update_round_trips_through_patch() { + let f = fixture(); + let before = outline(square()); + let mut moved = square(); + moved[1] = json!({"x": 4.37, "y": 50.85}); + let after = outline(moved); + let deltas = diff2(&f, before.clone(), after.clone()); + let (patched, trace) = patch(&f.load(before), &deltas, PatchOptions::default()).unwrap(); + assert!(trace.failed.is_empty(), "{:?}", trace.failed); + assert!( + patched.equals(&f.load(after), true), + "round-trip mismatch: {}", + patched.to_json() + ); +} From b228b4be51a754caa05787529740a7583af6fb7b Mon Sep 17 00:00:00 2001 From: ejsyx Date: Tue, 18 Aug 2026 14:40:35 +0200 Subject: [PATCH 07/58] feat(runtime): patch refuses to descend below an opaque slot Co-Authored-By: Claude Fable 5 --- src/runtime/src/diff.rs | 16 +++++++++++++ src/runtime/tests/diff_opaque.rs | 41 ++++++++++++++++++++++++++++++++ 2 files changed, 57 insertions(+) diff --git a/src/runtime/src/diff.rs b/src/runtime/src/diff.rs index 936a5dd..8f0437e 100644 --- a/src/runtime/src/diff.rs +++ b/src/runtime/src/diff.rs @@ -899,6 +899,12 @@ fn apply_delta_object( ); } if let Some(child) = values.get_mut(key) { + let slot = class.slots().iter().find(|s| s.name == *key); + if slot.is_some_and(slot_is_opaque) { + // The path descends below an opaque slot: it addresses structure + // the slot does not expose. Report, never guess. + return Ok(false); + } return apply_delta_linkml_inner(child, &path[1..], op, newv, trace, opts); } Ok(false) @@ -916,6 +922,11 @@ fn apply_delta_mapping( trace: &mut PatchTrace, opts: PatchOptions, ) -> LResult { + // A non-empty path here addresses *inside* the mapping, which is below the + // mapping's own slot. Reached when the patched root is itself a mapping. + if slot_is_opaque(slot) { + return Ok(false); + } let key = &path[0]; if path.len() == 1 { let value = newv.cloned().unwrap_or(JsonValue::Null); @@ -965,6 +976,11 @@ fn apply_delta_list( trace: &mut PatchTrace, opts: PatchOptions, ) -> LResult { + // A non-empty path here addresses *inside* the list, which is below the + // list's own slot. Reached when the patched root is itself a list. + if slot_is_opaque(slot) { + return Ok(false); + } let key = &path[0]; let idx_opt = resolve_list_index(values, key); if path.len() == 1 { diff --git a/src/runtime/tests/diff_opaque.rs b/src/runtime/tests/diff_opaque.rs index f82f7e1..7938dfd 100644 --- a/src/runtime/tests/diff_opaque.rs +++ b/src/runtime/tests/diff_opaque.rs @@ -141,3 +141,44 @@ fn opaque_whole_slot_update_round_trips_through_patch() { patched.to_json() ); } + +#[test] +fn patch_below_opaque_slot_is_reported_failed_not_guessed() { + let f = fixture(); + let golden = f.load(outline(square())); + let delta = Delta { + path: vec!["outline".to_string(), "1".to_string(), "x".to_string()], + op: DeltaOp::Update, + old: Some(json!(4.36)), + new: Some(json!(9.99)), + }; + let (patched, trace) = patch( + &golden, + std::slice::from_ref(&delta), + PatchOptions::default(), + ) + .unwrap(); + assert_eq!(trace.failed, vec![delta.path.clone()]); + assert!( + patched.equals(&golden, true), + "nothing may change: {}", + patched.to_json() + ); +} + +#[test] +fn patch_at_opaque_slot_path_still_applies_whole_value() { + let f = fixture(); + let golden = f.load(outline(square())); + let mut moved = square(); + moved[1] = json!({"x": 4.37, "y": 50.85}); + let delta = Delta { + path: vec!["outline".to_string()], + op: DeltaOp::Update, + old: Some(json!(square())), + new: Some(json!(moved.clone())), + }; + let (patched, trace) = patch(&golden, &[delta], PatchOptions::default()).unwrap(); + assert!(trace.failed.is_empty(), "{:?}", trace.failed); + assert!(patched.equals(&f.load(outline(moved)), true)); +} From 32342e73c923339a46da945e32b12b52f26ef94b Mon Sep 17 00:00:00 2001 From: ejsyx Date: Tue, 18 Aug 2026 14:50:43 +0200 Subject: [PATCH 08/58] feat(runtime): match inlined list elements by unique_keys-derived identity MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Diff's (List, List) arm now derives element identity uniformly: a key/identifier slot first, else the range class's merged `unique_keys` (single-slot keys use the bare scalar, composite keys the JSON array encoding of the values in `unique_key_slots` order). Keyed matching applies iff every element on both sides yields an identity label AND the labels are unique within each side. The guard is uniform: key/identifier labels are checked for duplicates exactly as unique_keys labels are, removing the silent collapse of lists that repeat a key. Lists that fail the guard fall back to positional matching with plain numeric segments — the old opportunistic mixing of key values into positional paths is gone, since a label that failed the guard cannot address an element unambiguously. The label helpers are extracted from the inline closure as element_key_label / element_unique_key_label / element_identity_label / scalar_slot_string for reuse by the patch resolver and the linter. Co-Authored-By: Claude Fable 5 --- src/runtime/src/diff.rs | 122 ++++++++++---- src/runtime/tests/diff_unique_keys.rs | 219 ++++++++++++++++++++++++++ 2 files changed, 307 insertions(+), 34 deletions(-) create mode 100644 src/runtime/tests/diff_unique_keys.rs diff --git a/src/runtime/src/diff.rs b/src/runtime/src/diff.rs index 8f0437e..a4b71a3 100644 --- a/src/runtime/src/diff.rs +++ b/src/runtime/src/diff.rs @@ -37,6 +37,70 @@ pub(crate) fn slot_is_opaque(slot: &SlotView) -> bool { .unwrap_or(false) } +pub(crate) fn scalar_slot_string( + values: &std::collections::HashMap, + slot_name: &str, +) -> Option { + if let Some(LinkMLInstance::Scalar { value, .. }) = values.get(slot_name) { + return match value { + JsonValue::String(s) => Some(s.clone()), + other => Some(other.to_string()), + }; + } + None +} + +/// The key/identifier value identifying `v` among its list siblings, if any. +pub(crate) fn element_key_label(v: &LinkMLInstance) -> Option { + if let LinkMLInstance::Object { values, class, .. } = v { + let id_slot = class.key_or_identifier_slot()?; + return scalar_slot_string(values, &id_slot.name); + } + None +} + +/// A matching label derived from the range class's merged `unique_keys`. +/// +/// The name-sorted first entry with a non-empty slot list is the matching +/// identity (declaration order is not preserved by the metamodel, and diff +/// paths must be stable). Single-slot keys use the bare scalar value as the +/// label and path segment; composite keys use the JSON array encoding of the +/// values in `unique_key_slots` order (`["Emergency","02/111.11.11"]`) — +/// unambiguous, parseable, and displayable. +pub(crate) fn element_unique_key_label(v: &LinkMLInstance) -> Option { + if let LinkMLInstance::Object { values, class, .. } = v { + let uks = class.unique_keys(); + let (_, uk) = uks.iter().find(|(_, uk)| !uk.unique_key_slots.is_empty())?; + let parts: Option> = uk + .unique_key_slots + .iter() + .map(|s| scalar_slot_string(values, s)) + .collect(); + let mut parts = parts?; + return Some(if parts.len() == 1 { + parts.remove(0) + } else { + // Infallible JSON array encoding (the crate denies `expect`). + JsonValue::Array(parts.into_iter().map(JsonValue::String).collect()).to_string() + }); + } + None +} + +/// Identity for keyed list matching: a key/identifier slot outranks a +/// `unique_keys` claim. +pub(crate) fn element_identity_label(v: &LinkMLInstance) -> Option { + element_key_label(v).or_else(|| element_unique_key_label(v)) +} + +fn labels_are_unique(elements: &[LinkMLInstance], label: F) -> bool +where + F: Fn(&LinkMLInstance) -> Option, +{ + let mut seen = std::collections::HashSet::new(); + elements.iter().filter_map(label).all(|l| seen.insert(l)) +} + /// Operation applied by a [`Delta`]. #[derive(Clone, Debug, PartialEq, Serialize, Deserialize)] #[serde(rename_all = "lowercase")] @@ -52,8 +116,11 @@ pub enum DeltaOp { /// Semantic delta emitted by [`diff`] and consumed by [`patch`]. /// /// The `path` identifies the location within the instance tree. Each segment is a -/// slot name, mapping key, list index, or (when available) the identifier/key slot -/// value for inlined objects in lists. +/// slot name, mapping key, list index, or — for inlined objects in lists matched +/// by identity — the element's identity label: its identifier/key slot value, or +/// failing that a value derived from the range class's `unique_keys`. Lists whose +/// elements do not all carry a *unique* identity label are addressed by numeric +/// index instead. /// /// Operations are expressed jointly via [`Delta::op`], `old`, and `new`: /// @@ -237,36 +304,28 @@ pub fn diff(source: &LinkMLInstance, target: &LinkMLInstance, opts: DiffOptions) } } (LinkMLInstance::List { values: sl, .. }, LinkMLInstance::List { values: tl, .. }) => { - let label = |v: &LinkMLInstance| -> Option { - if let LinkMLInstance::Object { values, class, .. } = v { - if let Some(id_slot) = class.key_or_identifier_slot() { - if let Some(LinkMLInstance::Scalar { value, .. }) = - values.get(&id_slot.name) - { - return match value { - JsonValue::String(s) => Some(s.clone()), - other => Some(other.to_string()), - }; - } - } - } - None - }; - // If every item in both lists carries an identifier, match by id - // rather than by position. Positional diff corrupts mid-list + let identity = |v: &LinkMLInstance| -> Option { element_identity_label(v) }; + // Uniform rule (spec, Non-goal section): keyed matching iff every + // element on both sides carries an identity label and the labels + // are unique within each side. Positional diff corrupts mid-list // removes/inserts into shifted Updates, and on patch the label - // resolver can remove the wrong duplicate. - let keyed = - sl.iter().all(|v| label(v).is_some()) && tl.iter().all(|v| label(v).is_some()); + // resolver can remove the wrong duplicate. Duplicate labels (a + // list repeating a key, or data violating a unique_keys claim) + // fall back to positional — matching duplicates by label would + // silently collapse elements. + let keyed = sl.iter().all(|v| identity(v).is_some()) + && tl.iter().all(|v| identity(v).is_some()) + && labels_are_unique(sl, identity) + && labels_are_unique(tl, identity); if keyed { use std::collections::HashSet; - let src_ids: HashSet = sl.iter().filter_map(&label).collect(); + let src_ids: HashSet = sl.iter().filter_map(&identity).collect(); let tgt_by_id: std::collections::HashMap = tl .iter() - .filter_map(|v| label(v).map(|id| (id, v))) + .filter_map(|v| identity(v).map(|id| (id, v))) .collect(); for sv in sl { - let Some(id) = label(sv) else { continue }; + let Some(id) = identity(sv) else { continue }; path.push(id.clone()); match tgt_by_id.get(&id) { Some(tv) => inner(path, None, sv, tv, opts, out), @@ -280,7 +339,7 @@ pub fn diff(source: &LinkMLInstance, target: &LinkMLInstance, opts: DiffOptions) path.pop(); } for tv in tl { - let Some(id) = label(tv) else { continue }; + let Some(id) = identity(tv) else { continue }; if !src_ids.contains(&id) { path.push(id); out.push(Delta { @@ -295,14 +354,9 @@ pub fn diff(source: &LinkMLInstance, target: &LinkMLInstance, opts: DiffOptions) } else { let max_len = std::cmp::max(sl.len(), tl.len()); for i in 0..max_len { - let step = if let Some(sv) = sl.get(i) { - label(sv) - .or_else(|| tl.get(i).and_then(&label)) - .unwrap_or_else(|| i.to_string()) - } else { - tl.get(i).and_then(&label).unwrap_or_else(|| i.to_string()) - }; - path.push(step); + // Plain numeric segments only: a label that failed the + // keyed guard cannot address an element unambiguously. + path.push(i.to_string()); match (sl.get(i), tl.get(i)) { (Some(sv), Some(tv)) => inner(path, None, sv, tv, opts, out), (Some(sv), None) => out.push(Delta { diff --git a/src/runtime/tests/diff_unique_keys.rs b/src/runtime/tests/diff_unique_keys.rs new file mode 100644 index 0000000..18d75dd --- /dev/null +++ b/src/runtime/tests/diff_unique_keys.rs @@ -0,0 +1,219 @@ +use linkml_runtime::{diff, load_json_str, Delta, DeltaOp, DiffOptions, LinkMLInstance}; +use linkml_schemaview::identifier::{converter_from_schema, Identifier}; +use linkml_schemaview::io::from_yaml; +use linkml_schemaview::schemaview::{ClassView, SchemaView}; +use linkml_schemaview::Converter; +use serde_json::{json, Value as JsonValue}; +use std::path::PathBuf; + +struct Fixture { + sv: SchemaView, + conv: Converter, + service: ClassView, +} + +fn fixture() -> Fixture { + let mut p = PathBuf::from(env!("CARGO_MANIFEST_DIR")); + p.push("tests/data/identity.yaml"); + let schema = from_yaml(&p).unwrap(); + let mut sv = SchemaView::new(); + sv.add_schema(schema.clone()).unwrap(); + let conv = converter_from_schema(&schema); + let service = sv + .get_class(&Identifier::new("Service"), &conv) + .unwrap() + .expect("class not found"); + Fixture { sv, conv, service } +} + +impl Fixture { + fn load(&self, v: JsonValue) -> LinkMLInstance { + load_json_str(&v.to_string(), &self.sv, &self.service, &self.conv) + .unwrap() + .into_instance() + .unwrap() + } +} + +fn diff2(f: &Fixture, before: JsonValue, after: JsonValue) -> Vec { + diff(&f.load(before), &f.load(after), DiffOptions::new(true)) +} + +fn only(deltas: &[Delta]) -> &Delta { + assert_eq!(deltas.len(), 1, "expected exactly one delta: {deltas:#?}"); + &deltas[0] +} + +fn e() -> JsonValue { + json!({"phoneNumber": "09/241.25.00", "hasNumberFunction": "Emergency_Number"}) +} +fn n() -> JsonValue { + json!({"phoneNumber": "09/241.25.03", "hasNumberFunction": "Non_Urgent_Communication"}) +} +fn o() -> JsonValue { + json!({"phoneNumber": "09/241.25.10", "hasNumberFunction": "Operator"}) +} +fn phones(items: Vec) -> JsonValue { + json!({"name": "svc", "hasPhoneNumber": items}) +} + +#[test] +fn unique_key_matching_targets_field_edits_by_key() { + let f = fixture(); + let mut n2 = n(); + n2["phoneNumber"] = json!("09/241.25.99"); + let deltas = diff2(&f, phones(vec![e(), n()]), phones(vec![e(), n2])); + let delta = only(&deltas); + assert_eq!( + delta.path, + vec![ + "hasPhoneNumber".to_string(), + "Non_Urgent_Communication".to_string(), + "phoneNumber".to_string() + ] + ); + assert_eq!(delta.op, DeltaOp::Update); +} + +#[test] +fn unique_key_matching_ignores_reorder() { + let f = fixture(); + let deltas = diff2(&f, phones(vec![e(), n(), o()]), phones(vec![o(), n(), e()])); + assert!(deltas.is_empty(), "reorder must be invisible: {deltas:#?}"); +} + +#[test] +fn unique_key_remove_and_add_are_key_addressed() { + let f = fixture(); + let deltas = diff2(&f, phones(vec![e(), n()]), phones(vec![n()])); + let delta = only(&deltas); + assert_eq!(delta.op, DeltaOp::Remove); + assert_eq!( + delta.path, + vec!["hasPhoneNumber".to_string(), "Emergency_Number".to_string()] + ); + assert_eq!(delta.old, Some(e())); + + let deltas = diff2(&f, phones(vec![e(), n()]), phones(vec![e(), n(), o()])); + let delta = only(&deltas); + assert_eq!(delta.op, DeltaOp::Add); + assert_eq!( + delta.path, + vec!["hasPhoneNumber".to_string(), "Operator".to_string()] + ); + assert_eq!(delta.new, Some(o())); +} + +#[test] +fn changing_the_key_slot_is_remove_plus_add() { + let f = fixture(); + let mut moved = e(); + moved["hasNumberFunction"] = json!("Operator"); + let deltas = diff2(&f, phones(vec![e(), n()]), phones(vec![moved.clone(), n()])); + assert_eq!(deltas.len(), 2, "{deltas:#?}"); + assert!(deltas + .iter() + .any(|d| d.op == DeltaOp::Remove && d.old.as_ref() == Some(&e()))); + assert!(deltas + .iter() + .any(|d| d.op == DeltaOp::Add && d.new.as_ref() == Some(&moved))); +} + +#[test] +fn duplicate_unique_key_data_falls_back_to_positional() { + let f = fixture(); + // two Emergency numbers: violates the class claim; data must keep + // today's positional behaviour, with numeric path segments + let e2 = json!({"phoneNumber": "09/000.00.00", "hasNumberFunction": "Emergency_Number"}); + let mut e2_edit = e2.clone(); + e2_edit["phoneNumber"] = json!("09/111.11.11"); + let deltas = diff2(&f, phones(vec![e(), e2]), phones(vec![e(), e2_edit])); + let delta = only(&deltas); + assert_eq!( + delta.path, + vec![ + "hasPhoneNumber".to_string(), + "1".to_string(), + "phoneNumber".to_string() + ], + "positional fallback must use numeric segments, never the duplicate label" + ); +} + +#[test] +fn duplicate_key_data_falls_back_to_positional_not_collapse() { + let f = fixture(); + // Label declares `lang` as key; a list that repeats the key must not be + // silently collapsed by keyed matching — uniform guard, positional fallback. + let before = json!({"name": "svc", "labelList": [ + {"lang": "nl", "text": "a"}, {"lang": "nl", "text": "b"}]}); + let after = json!({"name": "svc", "labelList": [ + {"lang": "nl", "text": "a"}, {"lang": "nl", "text": "B"}]}); + let deltas = diff2(&f, before, after); + let delta = only(&deltas); + assert_eq!( + delta.path, + vec!["labelList".to_string(), "1".to_string(), "text".to_string()], + "duplicate key labels must fall back to plain numeric segments" + ); +} + +#[test] +fn undeclared_class_keeps_positional_cascade() { + let f = fixture(); + // PlainPhoneNumber has no unique_keys: removal still cascades as today. + // Both surviving elements shift up, and each differs from the element that + // used to sit at its index in both slots: 2 x 2 shifted-slot Updates plus + // the trailing Remove. + let before = json!({"name": "svc", "plainPhoneNumber": [e(), n(), o()]}); + let after = json!({"name": "svc", "plainPhoneNumber": [n(), o()]}); + let deltas = diff2(&f, before, after); + assert_eq!(deltas.len(), 5, "{deltas:#?}"); + assert!( + deltas.iter().all(|d| d.path[1].parse::().is_ok()), + "positional cascade must use numeric segments: {deltas:#?}" + ); +} + +#[test] +fn composite_unique_key_uses_json_array_segment() { + let f = fixture(); + let a = json!({"kind": "Emergency", "phone": "02/111.11.11", "note": "old"}); + let mut a2 = a.clone(); + a2["note"] = json!("new"); + let b = json!({"kind": "Operator", "phone": "02/333.33.33"}); + let before = json!({"name": "svc", "contacts": [a, b.clone()]}); + let after = json!({"name": "svc", "contacts": [a2, b]}); + let deltas = diff2(&f, before, after); + let delta = only(&deltas); + assert_eq!( + delta.path, + vec![ + "contacts".to_string(), + r#"["Emergency","02/111.11.11"]"#.to_string(), + "note".to_string() + ] + ); +} + +#[test] +fn inherited_unique_keys_drive_matching() { + let f = fixture(); + // EmergencyPhoneNumber inherits one_number_per_function via is_a + let x = json!({"phoneNumber": "1", "hasNumberFunction": "Emergency_Number", "note": "a"}); + let mut x2 = x.clone(); + x2["note"] = json!("b"); + let y = json!({"phoneNumber": "2", "hasNumberFunction": "Operator", "note": "c"}); + let before = json!({"name": "svc", "escalation": [x, y.clone()]}); + let after = json!({"name": "svc", "escalation": [x2, y]}); + let deltas = diff2(&f, before, after); + let delta = only(&deltas); + assert_eq!( + delta.path, + vec![ + "escalation".to_string(), + "Emergency_Number".to_string(), + "note".to_string() + ] + ); +} From c120110a0ae31b1dcd1fb7492685f01633368e15 Mon Sep 17 00:00:00 2001 From: ejsyx Date: Tue, 18 Aug 2026 14:57:21 +0200 Subject: [PATCH 09/58] docs(plan): identity-addressed lists resolve by label only on patch side Co-Authored-By: Claude Fable 5 --- ...18-inlined-multivalued-element-identity.md | 50 +++++++++++++++---- 1 file changed, 41 insertions(+), 9 deletions(-) diff --git a/docs/superpowers/plans/2026-08-18-inlined-multivalued-element-identity.md b/docs/superpowers/plans/2026-08-18-inlined-multivalued-element-identity.md index acae782..52b5497 100644 --- a/docs/superpowers/plans/2026-08-18-inlined-multivalued-element-identity.md +++ b/docs/superpowers/plans/2026-08-18-inlined-multivalued-element-identity.md @@ -1051,6 +1051,25 @@ fn patch_reports_ambiguous_unique_key_instead_of_guessing() { assert!(patched.equals(&golden, true), "nothing may change"); } +#[test] +fn patch_refuses_positional_segment_into_identity_addressed_list() { + let f = fixture(); + let golden = f.load(phones(vec![e(), n()])); + // A stale positional patch aimed at a list whose elements all carry + // unique identity labels: applying "index 0" would be a guess, and for + // numeric-valued identity labels it would silently hit the wrong element. + let delta = Delta { + path: vec!["hasPhoneNumber".to_string(), "0".to_string()], + op: DeltaOp::Remove, + old: Some(e()), + new: None, + }; + let (patched, trace) = + patch(&golden, std::slice::from_ref(&delta), PatchOptions::default()).unwrap(); + assert_eq!(trace.failed, vec![delta.path.clone()]); + assert!(patched.equals(&golden, true), "nothing may be removed"); +} + #[test] fn patch_refuses_ambiguous_duplicate_key_labels() { let f = fixture(); @@ -1094,23 +1113,34 @@ Expected: the two locate tests FAIL (segment `Emergency_Number` resolves to no i - [ ] **Step 3: Implement** -Rewrite `resolve_list_index` (diff.rs:530) as one unified resolver. The numeric-index attempt stays first and unchanged; the old key/identifier `find_map` block is **replaced** by an identity-label pass that uses the same precedence and stringification diff uses to build segments (making diff→patch symmetric), and that refuses ambiguity: +Rewrite `resolve_list_index` (diff.rs:530) as one unified resolver that mirrors how diff produces segments. A list whose elements all carry unique identity labels ("keyed-shaped") is addressed by label ONLY — diff emits label segments for exactly these lists, and resolving a numeric segment positionally against one would silently write the wrong element whenever an identity label is itself numeric (integer-ranged key slots). Positional lists keep numeric-first, with a single unambiguous label hit as drift tolerance. The old key/identifier `find_map` block is replaced entirely: ```rust fn resolve_list_index(values: &[LinkMLInstance], key: &str) -> Option { + // A list whose elements all carry unique identity labels is addressed by + // label ONLY: diff emits label segments for exactly these lists, and a + // numeric segment aimed at one (a stale positional patch) would be a + // guess — report, never guess. This also keeps integer-valued identity + // labels (e.g. a year as unique key) unambiguous: they resolve as + // labels, never as positions. + let labels: Vec> = values.iter().map(element_identity_label).collect(); + let keyed_shaped = !values.is_empty() && labels.iter().all(|l| l.is_some()) && { + let mut seen = std::collections::HashSet::new(); + labels.iter().flatten().all(|l| seen.insert(l.clone())) + }; + if keyed_shaped { + return labels.iter().position(|l| l.as_deref() == Some(key)); + } + // Positional list: numeric index first (the segments diff produces for + // these lists), then a single unambiguous label hit for drift tolerance. if let Ok(idx) = key.parse::() { if idx < values.len() { return Some(idx); } } - // Identity-label location (key/identifier first, else unique_keys) — the - // same precedence and stringification diff uses to build the segment. - // Only an unambiguous hit counts: if the current list holds duplicate - // labels, locating "the" element would be a guess — return None so the - // delta is reported as failed. let mut hit: Option = None; - for (i, v) in values.iter().enumerate() { - if element_identity_label(v).as_deref() == Some(key) { + for (i, l) in labels.iter().enumerate() { + if l.as_deref() == Some(key) { if hit.is_some() { return None; } @@ -1121,12 +1151,14 @@ fn resolve_list_index(values: &[LinkMLInstance], key: &str) -> Option { } ``` +(An `Add` whose label segment matches nothing returns `None`, which takes the existing append path in `apply_list_leaf_delta` — correct for both list shapes.) + Note on `Add` deltas: an `Add` whose unique-key segment resolves to no element takes the existing `idx_opt = None` append path in `apply_list_leaf_delta` (line 687) — that is correct and needs no change. - [ ] **Step 4: Run tests to verify they pass** Run: `cargo test -p linkml_runtime --test diff_unique_keys` -Expected: PASS (all 12). +Expected: PASS (all 13). - [ ] **Step 5: Full runtime suite — the compatibility gate** From c4bffea1820063ca2eb5765f5d7d34e2e6114f72 Mon Sep 17 00:00:00 2001 From: ejsyx Date: Tue, 18 Aug 2026 15:01:24 +0200 Subject: [PATCH 10/58] feat(runtime): patch resolves identity-label path segments, refuses ambiguity Co-Authored-By: Claude Fable 5 --- src/runtime/src/diff.rs | 46 +++++----- src/runtime/tests/diff_unique_keys.rs | 127 +++++++++++++++++++++++++- 2 files changed, 151 insertions(+), 22 deletions(-) diff --git a/src/runtime/src/diff.rs b/src/runtime/src/diff.rs index a4b71a3..f974210 100644 --- a/src/runtime/src/diff.rs +++ b/src/runtime/src/diff.rs @@ -610,33 +610,37 @@ fn replace_child_subtree( } fn resolve_list_index(values: &[LinkMLInstance], key: &str) -> Option { + // A list whose elements all carry unique identity labels is addressed by + // label ONLY: diff emits label segments for exactly these lists, and a + // numeric segment aimed at one (a stale positional patch) would be a + // guess — report, never guess. This also keeps integer-valued identity + // labels (e.g. a year as unique key) unambiguous: they resolve as + // labels, never as positions. + let labels: Vec> = values.iter().map(element_identity_label).collect(); + let keyed_shaped = !values.is_empty() && labels.iter().all(|l| l.is_some()) && { + let mut seen = std::collections::HashSet::new(); + labels.iter().flatten().all(|l| seen.insert(l.clone())) + }; + if keyed_shaped { + return labels.iter().position(|l| l.as_deref() == Some(key)); + } + // Positional list: numeric index first (the segments diff produces for + // these lists), then a single unambiguous label hit for drift tolerance. if let Ok(idx) = key.parse::() { if idx < values.len() { return Some(idx); } } - values.iter().enumerate().find_map(|(i, v)| { - if let LinkMLInstance::Object { - values: mv, class, .. - } = v - { - class - .key_or_identifier_slot() - .and_then(|id_slot| mv.get(&id_slot.name)) - .and_then(|child| match child { - LinkMLInstance::Scalar { value, .. } => match value { - JsonValue::String(s) => (s == key).then_some(i), - other => { - let key_json = JsonValue::String(key.to_string()); - (other == &key_json).then_some(i) - } - }, - _ => None, - }) - } else { - None + let mut hit: Option = None; + for (i, l) in labels.iter().enumerate() { + if l.as_deref() == Some(key) { + if hit.is_some() { + return None; + } + hit = Some(i); } - }) + } + hit } fn try_update_scalar_in_place( diff --git a/src/runtime/tests/diff_unique_keys.rs b/src/runtime/tests/diff_unique_keys.rs index 18d75dd..ade2823 100644 --- a/src/runtime/tests/diff_unique_keys.rs +++ b/src/runtime/tests/diff_unique_keys.rs @@ -1,4 +1,6 @@ -use linkml_runtime::{diff, load_json_str, Delta, DeltaOp, DiffOptions, LinkMLInstance}; +use linkml_runtime::{ + diff, load_json_str, patch, Delta, DeltaOp, DiffOptions, LinkMLInstance, PatchOptions, +}; use linkml_schemaview::identifier::{converter_from_schema, Identifier}; use linkml_schemaview::io::from_yaml; use linkml_schemaview::schemaview::{ClassView, SchemaView}; @@ -217,3 +219,126 @@ fn inherited_unique_keys_drive_matching() { ] ); } + +#[test] +fn patch_locates_element_by_unique_key_under_drift() { + let f = fixture(); + // producer saw [E, N]; golden drifted to [N, E, O] + let golden = f.load(phones(vec![n(), e(), o()])); + let delta = Delta { + path: vec![ + "hasPhoneNumber".to_string(), + "Emergency_Number".to_string(), + "phoneNumber".to_string(), + ], + op: DeltaOp::Update, + old: Some(json!("09/241.25.00")), + new: Some(json!("09/999.99.99")), + }; + let (patched, trace) = patch(&golden, &[delta], PatchOptions::default()).unwrap(); + assert!(trace.failed.is_empty(), "{:?}", trace.failed); + let mut e2 = e(); + e2["phoneNumber"] = json!("09/999.99.99"); + assert!( + patched.equals(&f.load(phones(vec![n(), e2, o()])), true), + "the edit must land on E wherever it sits: {}", + patched.to_json() + ); +} + +#[test] +fn patch_reports_ambiguous_unique_key_instead_of_guessing() { + let f = fixture(); + // golden drifted into two Emergency elements: locating "the" one is a guess + let e2 = json!({"phoneNumber": "09/000.00.00", "hasNumberFunction": "Emergency_Number"}); + let golden = f.load(phones(vec![e(), e2])); + let delta = Delta { + path: vec![ + "hasPhoneNumber".to_string(), + "Emergency_Number".to_string(), + "phoneNumber".to_string(), + ], + op: DeltaOp::Update, + old: Some(json!("09/241.25.00")), + new: Some(json!("09/999.99.99")), + }; + let (patched, trace) = patch( + &golden, + std::slice::from_ref(&delta), + PatchOptions::default(), + ) + .unwrap(); + assert_eq!(trace.failed, vec![delta.path.clone()]); + assert!(patched.equals(&golden, true), "nothing may change"); +} + +#[test] +fn patch_refuses_positional_segment_into_identity_addressed_list() { + let f = fixture(); + let golden = f.load(phones(vec![e(), n()])); + // A stale positional patch aimed at a list whose elements all carry + // unique identity labels: applying "index 0" would be a guess, and for + // numeric-valued identity labels it would silently hit the wrong element. + let delta = Delta { + path: vec!["hasPhoneNumber".to_string(), "0".to_string()], + op: DeltaOp::Remove, + old: Some(e()), + new: None, + }; + let (patched, trace) = patch( + &golden, + std::slice::from_ref(&delta), + PatchOptions::default(), + ) + .unwrap(); + assert_eq!(trace.failed, vec![delta.path.clone()]); + assert!(patched.equals(&golden, true), "nothing may be removed"); +} + +#[test] +fn patch_refuses_ambiguous_duplicate_key_labels() { + let f = fixture(); + // Duplicate key/identifier labels refuse exactly like duplicate + // unique_keys labels — the uniform rule on the patch side. + let golden = f.load(json!({"name": "svc", "labelList": [ + {"lang": "nl", "text": "a"}, {"lang": "nl", "text": "b"}]})); + let delta = Delta { + path: vec![ + "labelList".to_string(), + "nl".to_string(), + "text".to_string(), + ], + op: DeltaOp::Update, + old: Some(json!("a")), + new: Some(json!("z")), + }; + let (patched, trace) = patch( + &golden, + std::slice::from_ref(&delta), + PatchOptions::default(), + ) + .unwrap(); + assert_eq!(trace.failed, vec![delta.path.clone()]); + assert!(patched.equals(&golden, true), "nothing may change"); +} + +#[test] +fn unique_key_deltas_round_trip_through_patch() { + let f = fixture(); + let mut n2 = n(); + n2["phoneNumber"] = json!("09/241.25.99"); + for (before, after) in [ + (phones(vec![e(), n()]), phones(vec![e(), n2])), // field edit + (phones(vec![e(), n()]), phones(vec![n()])), // remove + (phones(vec![e(), n()]), phones(vec![e(), n(), o()])), // add + ] { + let deltas = diff2(&f, before.clone(), after.clone()); + let (patched, trace) = patch(&f.load(before), &deltas, PatchOptions::default()).unwrap(); + assert!(trace.failed.is_empty(), "{:?}", trace.failed); + assert!( + patched.equals(&f.load(after), true), + "{}", + patched.to_json() + ); + } +} From ae8e8d05ce7b3b6d46459dcc6db5816ff6852105 Mon Sep 17 00:00:00 2001 From: ejsyx Date: Tue, 18 Aug 2026 15:08:56 +0200 Subject: [PATCH 11/58] fix(runtime): unresolved list Update reports failed instead of appending Co-Authored-By: Claude Fable 5 --- src/runtime/src/diff.rs | 8 +++++ src/runtime/tests/diff_unique_keys.rs | 48 +++++++++++++++++++++++++++ 2 files changed, 56 insertions(+) diff --git a/src/runtime/src/diff.rs b/src/runtime/src/diff.rs index f974210..954015b 100644 --- a/src/runtime/src/diff.rs +++ b/src/runtime/src/diff.rs @@ -758,6 +758,14 @@ where { match op { DeltaOp::Add | DeltaOp::Update => { + // An `Update` addressing an element that is not there is a stale + // address, not an invitation to append: report, never guess. Only + // `Add` treats "no such index" as "put it at the end". Checked + // before `build_child` so a failed address reports rather than + // surfacing a build error for a value that is never applied. + if idx_opt.is_none() && matches!(op, DeltaOp::Update) { + return Ok(false); + } let new_child = build_child()?; if let Some(idx) = idx_opt { let existing = &mut values[idx]; diff --git a/src/runtime/tests/diff_unique_keys.rs b/src/runtime/tests/diff_unique_keys.rs index ade2823..58aba98 100644 --- a/src/runtime/tests/diff_unique_keys.rs +++ b/src/runtime/tests/diff_unique_keys.rs @@ -342,3 +342,51 @@ fn unique_key_deltas_round_trip_through_patch() { ); } } + +#[test] +fn patch_refuses_positional_update_into_identity_addressed_list() { + let f = fixture(); + let golden = f.load(phones(vec![e(), n()])); + // A stale positional Update against a keyed-shaped list resolves to no + // element. It must report, not append: an unresolved address is never an + // invitation to grow the list. + let mut e2 = e(); + e2["phoneNumber"] = json!("09/999.99.99"); + let delta = Delta { + path: vec!["hasPhoneNumber".to_string(), "0".to_string()], + op: DeltaOp::Update, + old: Some(e()), + new: Some(e2), + }; + let (patched, trace) = patch( + &golden, + std::slice::from_ref(&delta), + PatchOptions::default(), + ) + .unwrap(); + assert_eq!(trace.failed, vec![delta.path.clone()]); + assert!(patched.equals(&golden, true), "nothing may be appended"); +} + +#[test] +fn patch_refuses_update_whose_label_matches_nothing() { + let f = fixture(); + let golden = f.load(phones(vec![e(), n()])); + // An Update addressing an element that is not there: the producer meant to + // edit an existing Operator entry, and the golden has none. Reporting is + // the only honest answer — appending would invent an edit as a creation. + let delta = Delta { + path: vec!["hasPhoneNumber".to_string(), "Operator".to_string()], + op: DeltaOp::Update, + old: Some(o()), + new: Some(o()), + }; + let (patched, trace) = patch( + &golden, + std::slice::from_ref(&delta), + PatchOptions::default(), + ) + .unwrap(); + assert_eq!(trace.failed, vec![delta.path.clone()]); + assert!(patched.equals(&golden, true), "nothing may be appended"); +} From 6d9f398502832f339ce22a70e3108924b0574151 Mon Sep 17 00:00:00 2001 From: ejsyx Date: Tue, 18 Aug 2026 15:15:44 +0200 Subject: [PATCH 12/58] feat(runtime): opt-in element-identity linter (schema + instance) Co-Authored-By: Claude Fable 5 --- src/python/src/lib.rs | 2 + src/runtime/src/diff.rs | 2 +- src/runtime/src/identity_lint.rs | 124 +++++++++++++++++++++++++++++ src/runtime/src/lib.rs | 8 ++ src/runtime/tests/identity_lint.rs | 121 ++++++++++++++++++++++++++++ 5 files changed, 256 insertions(+), 1 deletion(-) create mode 100644 src/runtime/src/identity_lint.rs create mode 100644 src/runtime/tests/identity_lint.rs diff --git a/src/python/src/lib.rs b/src/python/src/lib.rs index aa2f14e..5fac483 100644 --- a/src/python/src/lib.rs +++ b/src/python/src/lib.rs @@ -1053,6 +1053,8 @@ fn validation_problem_type_label(problem_type: &ValidationProblemType) -> &'stat ValidationProblemType::SlotRangeViolation => "slot_range_violation", ValidationProblemType::MaxCountViolation => "max_count_violation", ValidationProblemType::ParsingError => "parsing_error", + ValidationProblemType::AmbiguousElementIdentity => "ambiguous_element_identity", + ValidationProblemType::DuplicateElementIdentity => "duplicate_element_identity", } } diff --git a/src/runtime/src/diff.rs b/src/runtime/src/diff.rs index 954015b..6fe079a 100644 --- a/src/runtime/src/diff.rs +++ b/src/runtime/src/diff.rs @@ -9,7 +9,7 @@ use std::collections::hash_map::Entry; const IGNORE_ANNOTATION: &str = "diff.linkml.io/ignore"; -fn slot_is_ignored(slot: &SlotView) -> bool { +pub(crate) fn slot_is_ignored(slot: &SlotView) -> bool { if slot.definitions().is_empty() { return false; } diff --git a/src/runtime/src/identity_lint.rs b/src/runtime/src/identity_lint.rs new file mode 100644 index 0000000..c600d4f --- /dev/null +++ b/src/runtime/src/identity_lint.rs @@ -0,0 +1,124 @@ +//! Opt-in element-identity linter. +//! +//! Answers, per multivalued inlined slot: **where does element identity come +//! from?** A key (or identifier) on the element class, a composed key +//! declared with `unique_keys`, or the `diff.linkml.io/opaque` annotation +//! (nowhere — the slot's value is replaced as a whole). A slot with none of +//! these produces positional deltas, which are ambiguous when several sources +//! produce deltas for the same object concurrently. +//! +//! Neither entry point is wired into loading or [`crate::validate_issues`]: +//! projects that do not opt in keep today's inferred semantics. +//! +//! When a slot is flagged, the data-model author has four options (worked +//! examples in `docs/superpowers/specs/2026-08-17-inlined-multivalued-element-identity-design.md`): +//! 1. declare a `key`/`identifier` slot on the element class; +//! 2. declare the identity as `unique_keys` (composed keys supported); +//! 3. annotate the slot `diff.linkml.io/opaque` — replace the value as a whole; +//! 4. remodel, when one class would need two identity answers. + +use crate::diff::{element_identity_label, slot_is_ignored, slot_is_opaque, OPAQUE_ANNOTATION}; +use crate::{LinkMLInstance, ValidationProblemType, ValidationResult, ValidationResultSink}; +use linkml_schemaview::identifier::Identifier; +use linkml_schemaview::schemaview::SchemaView; +use std::collections::HashMap; + +/// Schema-level lint: warn for every multivalued inlined slot whose element +/// identity comes from nowhere. Warnings only — the schema stays usable. +pub fn lint_element_identity(sv: &SchemaView) -> Vec { + use linkml_schemaview::slotview::{SlotContainerMode, SlotInlineMode}; + let mut sink = ValidationResultSink::default(); + let conv = sv.converter(); + let mut class_ids = sv.get_class_ids(); + class_ids.sort(); + for class_id in class_ids { + let Ok(Some(class)) = sv.get_class(&Identifier::new(&class_id), &conv) else { + continue; + }; + for slot in class.slots() { + if slot.determine_slot_container_mode() != SlotContainerMode::List { + continue; + } + if slot.determine_slot_inline_mode() == SlotInlineMode::Reference { + continue; // elements are references, not inlined + } + if slot_is_opaque(slot) || slot_is_ignored(slot) { + continue; // identity declared: nowhere, replace as a whole + } + if let Some(rc) = slot.get_range_class() { + if rc.key_or_identifier_slot().is_some() || !rc.unique_keys().is_empty() { + continue; + } + } + sink.push_warning( + ValidationProblemType::AmbiguousElementIdentity, + vec![class.name().to_string(), slot.name.clone()], + format!( + "elements of '{}.{}' have no declared identity: deltas are \ + positional and ambiguous under multi-sourced operation. \ + Declare a key/identifier or unique_keys on the element \ + class, annotate the slot with {} to replace the value as \ + a whole, or remodel.", + class.name(), + slot.name, + OPAQUE_ANNOTATION + ), + ); + } + } + sink.into_vec() +} + +/// Data-level lint: warn for every list container whose elements repeat a +/// declared identity (key/identifier or unique_keys value). +/// +/// Deliberately does NOT consult `diff.linkml.io/opaque`: a schema constraint +/// is class-level truth, and diff vocabulary never suppresses it. +pub fn lint_instance_identity(value: &LinkMLInstance) -> Vec { + let mut sink = ValidationResultSink::default(); + let mut path = Vec::new(); + walk(value, &mut path, &mut sink); + sink.into_vec() +} + +fn walk(v: &LinkMLInstance, path: &mut Vec, sink: &mut ValidationResultSink) { + match v { + LinkMLInstance::List { values, .. } => { + check_duplicates(values, path, sink); + for (i, child) in values.iter().enumerate() { + path.push(i.to_string()); + walk(child, path, sink); + path.pop(); + } + } + LinkMLInstance::Object { values, .. } | LinkMLInstance::Mapping { values, .. } => { + for (k, child) in values { + path.push(k.clone()); + walk(child, path, sink); + path.pop(); + } + } + LinkMLInstance::Scalar { .. } | LinkMLInstance::Null { .. } => {} + } +} + +fn check_duplicates(values: &[LinkMLInstance], path: &[String], sink: &mut ValidationResultSink) { + let mut seen: HashMap = HashMap::new(); + for v in values { + if let Some(label) = element_identity_label(v) { + *seen.entry(label).or_insert(0) += 1; + } + } + let mut dups: Vec<(String, usize)> = seen.into_iter().filter(|(_, n)| *n > 1).collect(); + dups.sort(); + for (label, n) in dups { + sink.push_warning( + ValidationProblemType::DuplicateElementIdentity, + path.to_vec(), + format!( + "{n} elements share the declared identity '{label}'; deltas \ + addressing it are ambiguous" + ), + ); + } +} diff --git a/src/runtime/src/lib.rs b/src/runtime/src/lib.rs index aa5fc29..31fd5e3 100644 --- a/src/runtime/src/lib.rs +++ b/src/runtime/src/lib.rs @@ -26,6 +26,7 @@ use constraints::{run_object_constraints, run_slot_constraints}; pub mod blame; pub mod diff; +pub mod identity_lint; #[cfg(feature = "ttl")] pub mod rdf_export; #[cfg(feature = "ttl")] @@ -49,6 +50,7 @@ pub use blame::{ pub use diff::{ diff, patch, Delta, DeltaOp, DiffOptions, PatchOptions, PatchTrace, OPAQUE_ANNOTATION, }; +pub use identity_lint::{lint_element_identity, lint_instance_identity}; #[derive(Debug)] pub struct LinkMLError { validation_issues: Vec, @@ -156,6 +158,12 @@ pub enum ValidationProblemType { SlotRangeViolation, MaxCountViolation, ParsingError, + /// A multivalued inlined slot whose elements have no declared identity + /// (opt-in, [`crate::lint_element_identity`]). + AmbiguousElementIdentity, + /// Elements of one list repeat a declared identity (opt-in, + /// [`crate::lint_instance_identity`]). + DuplicateElementIdentity, } pub type InstancePath = Vec; diff --git a/src/runtime/tests/identity_lint.rs b/src/runtime/tests/identity_lint.rs new file mode 100644 index 0000000..4f328f2 --- /dev/null +++ b/src/runtime/tests/identity_lint.rs @@ -0,0 +1,121 @@ +use linkml_runtime::{ + lint_element_identity, lint_instance_identity, load_json_str, LinkMLInstance, + ValidationProblemType, +}; +use linkml_schemaview::identifier::{converter_from_schema, Identifier}; +use linkml_schemaview::io::from_yaml; +use linkml_schemaview::schemaview::{ClassView, SchemaView}; +use linkml_schemaview::Converter; +use serde_json::{json, Value as JsonValue}; +use std::path::PathBuf; + +struct Fixture { + sv: SchemaView, + conv: Converter, + service: ClassView, +} + +fn fixture() -> Fixture { + let mut p = PathBuf::from(env!("CARGO_MANIFEST_DIR")); + p.push("tests/data/identity.yaml"); + let schema = from_yaml(&p).unwrap(); + let mut sv = SchemaView::new(); + sv.add_schema(schema.clone()).unwrap(); + let conv = converter_from_schema(&schema); + let service = sv + .get_class(&Identifier::new("Service"), &conv) + .unwrap() + .expect("class not found"); + Fixture { sv, conv, service } +} + +impl Fixture { + fn load(&self, v: JsonValue) -> LinkMLInstance { + load_json_str(&v.to_string(), &self.sv, &self.service, &self.conv) + .unwrap() + .into_instance() + .unwrap() + } +} + +fn e() -> JsonValue { + json!({"phoneNumber": "09/241.25.00", "hasNumberFunction": "Emergency_Number"}) +} +fn n() -> JsonValue { + json!({"phoneNumber": "09/241.25.03", "hasNumberFunction": "Non_Urgent_Communication"}) +} +fn phones(items: Vec) -> JsonValue { + json!({"name": "svc", "hasPhoneNumber": items}) +} + +#[test] +fn schema_lint_flags_exactly_the_undeclared_positional_slots() { + let f = fixture(); + let warnings = lint_element_identity(&f.sv); + let mut flagged: Vec<(String, String)> = warnings + .iter() + .map(|w| (w.subject[0].clone(), w.subject[1].clone())) + .collect(); + flagged.sort(); + assert_eq!( + flagged, + vec![ + ("Service".to_string(), "plainPhoneNumber".to_string()), + ("Service".to_string(), "tags".to_string()), + ], + "everything else declares its identity source: {warnings:#?}" + ); + for w in &warnings { + assert_eq!( + w.problem_type, + ValidationProblemType::AmbiguousElementIdentity + ); + assert!(!w.severity.is_error(), "the linter warns, never errors"); + assert!( + w.detail.contains("unique_keys") && w.detail.contains("diff.linkml.io/opaque"), + "the warning must name the author's options: {}", + w.detail + ); + } +} + +#[test] +fn data_lint_flags_duplicate_declared_identities() { + let f = fixture(); + let dup = json!({"phoneNumber": "09/000.00.00", "hasNumberFunction": "Emergency_Number"}); + let inst = f.load(phones(vec![e(), dup])); + let warnings = lint_instance_identity(&inst); + assert_eq!(warnings.len(), 1, "{warnings:#?}"); + assert_eq!( + warnings[0].problem_type, + ValidationProblemType::DuplicateElementIdentity + ); + assert_eq!(warnings[0].subject, vec!["hasPhoneNumber".to_string()]); + assert!(!warnings[0].severity.is_error()); +} + +#[test] +fn data_lint_is_silent_on_clean_and_undeclared_data() { + let f = fixture(); + // unique phone functions, repeated scalar tags, repeated identity-less vertices + let inst = f.load(json!({ + "name": "svc", + "hasPhoneNumber": [e(), n()], + "tags": ["a", "a"], + "outline": [{"x": 1.0, "y": 2.0}, {"x": 1.0, "y": 2.0}] + })); + let warnings = lint_instance_identity(&inst); + assert!(warnings.is_empty(), "{warnings:#?}"); +} + +#[test] +fn data_lint_does_not_let_opaque_suppress_a_schema_constraint() { + let f = fixture(); + // archivedContacts is opaque, but Contact declares unique_keys: duplicates + // still violate the class's claim. diff vocabulary never silences schema truth. + let c = json!({"kind": "Emergency", "phone": "02/111.11.11"}); + let inst = f.load(json!({"name": "svc", "archivedContacts": [c.clone(), c]})); + let warnings = lint_instance_identity(&inst); + assert_eq!(warnings.len(), 1, "{warnings:#?}"); + assert_eq!(warnings[0].subject, vec!["archivedContacts".to_string()]); +} From b11781298d2b92148e37e7fb50ea3fcd384b47d8 Mon Sep 17 00:00:00 2001 From: ejsyx Date: Tue, 18 Aug 2026 15:23:15 +0200 Subject: [PATCH 13/58] fix(runtime): deterministic instance lint order, sharper lint messages Co-Authored-By: Claude Fable 5 --- src/runtime/src/identity_lint.rs | 52 +++++++++++++++++++++++------- src/runtime/tests/identity_lint.rs | 46 ++++++++++++++++++++++++++ 2 files changed, 87 insertions(+), 11 deletions(-) diff --git a/src/runtime/src/identity_lint.rs b/src/runtime/src/identity_lint.rs index c600d4f..a703b30 100644 --- a/src/runtime/src/identity_lint.rs +++ b/src/runtime/src/identity_lint.rs @@ -16,12 +16,18 @@ //! 2. declare the identity as `unique_keys` (composed keys supported); //! 3. annotate the slot `diff.linkml.io/opaque` — replace the value as a whole; //! 4. remodel, when one class would need two identity answers. +//! +//! A slot annotated `diff.linkml.io/ignore` is never flagged either, for a +//! different reason: diff skips such a slot entirely, so it produces no deltas +//! and has no element identity to declare. `ignore` silences the lint by +//! removing the slot from diff's scope; `opaque` silences it by answering the +//! question with "nowhere — replace the value as a whole". use crate::diff::{element_identity_label, slot_is_ignored, slot_is_opaque, OPAQUE_ANNOTATION}; use crate::{LinkMLInstance, ValidationProblemType, ValidationResult, ValidationResultSink}; use linkml_schemaview::identifier::Identifier; use linkml_schemaview::schemaview::SchemaView; -use std::collections::HashMap; +use std::collections::{BTreeMap, HashMap}; /// Schema-level lint: warn for every multivalued inlined slot whose element /// identity comes from nowhere. Warnings only — the schema stays usable. @@ -42,27 +48,48 @@ pub fn lint_element_identity(sv: &SchemaView) -> Vec { if slot.determine_slot_inline_mode() == SlotInlineMode::Reference { continue; // elements are references, not inlined } - if slot_is_opaque(slot) || slot_is_ignored(slot) { - continue; // identity declared: nowhere, replace as a whole + if slot_is_opaque(slot) { + continue; // identity declared: nowhere, replace the value as a whole + } + if slot_is_ignored(slot) { + continue; // outside diff's scope entirely: no deltas, no identity } - if let Some(rc) = slot.get_range_class() { + let range_class = slot.get_range_class(); + if let Some(rc) = &range_class { if rc.key_or_identifier_slot().is_some() || !rc.unique_keys().is_empty() { continue; } } - sink.push_warning( - ValidationProblemType::AmbiguousElementIdentity, - vec![class.name().to_string(), slot.name.clone()], - format!( + // The advice has to fit the range: a scalar- or enum-ranged slot has + // no element class on which a key could be declared. + let detail = match &range_class { + Some(rc) => format!( "elements of '{}.{}' have no declared identity: deltas are \ positional and ambiguous under multi-sourced operation. \ Declare a key/identifier or unique_keys on the element \ - class, annotate the slot with {} to replace the value as \ - a whole, or remodel.", + class '{}', annotate the slot with {} to replace the value \ + as a whole, or remodel.", + class.name(), + slot.name, + rc.name(), + OPAQUE_ANNOTATION + ), + None => format!( + "elements of '{}.{}' have no declared identity: the range is \ + not a class, so deltas can only be positional, and they are \ + ambiguous under multi-sourced operation. Annotate the slot \ + with {} to replace the value as a whole, or remodel the \ + range into a class that declares a key/identifier or \ + unique_keys.", class.name(), slot.name, OPAQUE_ANNOTATION ), + }; + sink.push_warning( + ValidationProblemType::AmbiguousElementIdentity, + vec![class.name().to_string(), slot.name.clone()], + detail, ); } } @@ -92,7 +119,10 @@ fn walk(v: &LinkMLInstance, path: &mut Vec, sink: &mut ValidationResultS } } LinkMLInstance::Object { values, .. } | LinkMLInstance::Mapping { values, .. } => { - for (k, child) in values { + // Name-sorted, so the warning order is stable across runs: the + // values are a `HashMap`, whose iteration order is not. + let ordered: BTreeMap<&String, &LinkMLInstance> = values.iter().collect(); + for (k, child) in ordered { path.push(k.clone()); walk(child, path, sink); path.pop(); diff --git a/src/runtime/tests/identity_lint.rs b/src/runtime/tests/identity_lint.rs index 4f328f2..f1c31a1 100644 --- a/src/runtime/tests/identity_lint.rs +++ b/src/runtime/tests/identity_lint.rs @@ -77,6 +77,24 @@ fn schema_lint_flags_exactly_the_undeclared_positional_slots() { w.detail ); } + + let detail = |slot: &str| { + warnings + .iter() + .find(|w| w.subject[1] == slot) + .map(|w| w.detail.clone()) + .unwrap_or_default() + }; + assert!( + detail("plainPhoneNumber").contains("element class 'PlainPhoneNumber'"), + "an object range must be told which class to declare the identity on: {}", + detail("plainPhoneNumber") + ); + assert!( + !detail("tags").contains("element class"), + "a scalar range has no element class to declare keys on: {}", + detail("tags") + ); } #[test] @@ -94,6 +112,34 @@ fn data_lint_flags_duplicate_declared_identities() { assert!(!warnings[0].severity.is_error()); } +#[test] +fn data_lint_warning_order_is_deterministic_across_sibling_containers() { + let f = fixture(); + // Three sibling containers, each with a duplicated declared identity. The + // instance's slots live in a HashMap, so the warnings must be emitted in a + // stable (name-sorted) order rather than in process-dependent hash order. + let c = json!({"kind": "Emergency", "phone": "02/111.11.11"}); + let dup = json!({"phoneNumber": "09/000.00.00", "hasNumberFunction": "Emergency_Number"}); + let data = json!({ + "name": "svc", + "archivedContacts": [c.clone(), c.clone()], + "contacts": [c.clone(), c], + "hasPhoneNumber": [e(), dup], + }); + let expected = vec![ + vec!["archivedContacts".to_string()], + vec!["contacts".to_string()], + vec!["hasPhoneNumber".to_string()], + ]; + // Repeated on freshly built instances: each `HashMap` gets its own hash + // seed, so an unordered walk would eventually disagree with itself. + for _ in 0..10 { + let warnings = lint_instance_identity(&f.load(data.clone())); + let subjects: Vec> = warnings.iter().map(|w| w.subject.clone()).collect(); + assert_eq!(subjects, expected, "{warnings:#?}"); + } +} + #[test] fn data_lint_is_silent_on_clean_and_undeclared_data() { let f = fixture(); From 19dbbd9b5e0be89dd52e1820a404eef3d177e503 Mon Sep 17 00:00:00 2001 From: ejsyx Date: Tue, 18 Aug 2026 15:31:36 +0200 Subject: [PATCH 14/58] feat(tools): --lint-identity flag on linkml-schema-validate Opt-in schema lint surfacing multivalued inlined slots whose element identity comes from nowhere. Without the flag output is unchanged; with it, warnings print but never affect the exit code. Text mode prints one `warning[Class.slot]: ` line per finding. JSON mode wraps the existing document so the result stays a single parseable object: `{"status":"valid","identity_warnings":[...]}`. Two behaviours worth calling out: - The lint is skipped when the schema does not validate. Its answers would be wrong on an incomplete graph (an unresolved import makes a class range look like a non-class), and running it on a class with a dangling `slots:` reference panics in SlotView::definition, which indexes an empty definition list. - Warnings are sorted by subject. ClassView::slots() is backed by a HashMap, so the linter's per-class slot order varies between runs. The lint reads the same SchemaView the validation used, so classes reached through `imports:` and resolve_schemas are linted too. Co-Authored-By: Claude Fable 5 --- src/tools/src/bin/linkml_schema_validate.rs | 78 ++++++++++++++++++++- 1 file changed, 77 insertions(+), 1 deletion(-) diff --git a/src/tools/src/bin/linkml_schema_validate.rs b/src/tools/src/bin/linkml_schema_validate.rs index 53aa457..9edd160 100644 --- a/src/tools/src/bin/linkml_schema_validate.rs +++ b/src/tools/src/bin/linkml_schema_validate.rs @@ -1,4 +1,5 @@ use clap::{Parser, ValueEnum}; +use linkml_runtime::{ValidationResult, ValidationSeverity}; #[cfg(feature = "resolve")] use linkml_schemaview::resolve::resolve_schemas; use linkml_schemaview::{identifier::Identifier, io::from_yaml, schemaview::SchemaView, Converter}; @@ -12,6 +13,11 @@ struct Args { /// Output format #[arg(long, value_enum, default_value_t = OutputFormat::Text)] output: OutputFormat, + /// Opt-in: warn for multivalued inlined slots whose element identity comes + /// from nowhere (positional, ambiguous deltas in multi-sourced use). + /// Warnings never change the exit code. + #[arg(long, default_value_t = false)] + lint_identity: bool, } #[derive(ValueEnum, Clone)] @@ -102,6 +108,31 @@ fn enum_exists( } } +fn severity_label(severity: &ValidationSeverity) -> &'static str { + match severity { + ValidationSeverity::Fatal => "fatal", + ValidationSeverity::Error => "error", + ValidationSeverity::Warning => "warning", + ValidationSeverity::Info => "info", + } +} + +fn identity_warnings_json(warnings: &[ValidationResult]) -> serde_json::Value { + serde_json::Value::Array( + warnings + .iter() + .map(|w| { + serde_json::json!({ + "type": format!("{:?}", w.problem_type), + "severity": severity_label(&w.severity), + "subject": w.subject, + "detail": w.detail, + }) + }) + .collect(), + ) +} + fn main() -> Result<(), Box> { let args = Args::parse(); let schema = from_yaml(&args.schema)?; @@ -177,17 +208,62 @@ fn main() -> Result<(), Box> { })?; if errors.is_empty() { + // Opt-in identity lint. It runs against the same SchemaView the + // validation above used, so classes pulled in by `resolve_schemas` + // from `imports:` are linted too. Warnings only — the exit code is + // whatever the validation produced. + let mut identity_warnings = if args.lint_identity { + linkml_runtime::lint_element_identity(&sv) + } else { + Vec::new() + }; + // `ClassView::slots()` is backed by a HashMap, so the linter emits a + // class's slots in an order that varies between runs. Sort by subject + // (class, then slot) so repeated runs over the same schema produce + // identical, diffable output. + identity_warnings.sort_by(|a, b| a.subject.cmp(&b.subject)); match args.output { - OutputFormat::Text => println!("schema valid"), + OutputFormat::Text => { + println!("schema valid"); + for w in &identity_warnings { + println!("warning[{}]: {}", w.subject.join("."), w.detail); + } + } + OutputFormat::Json if args.lint_identity => { + println!( + "{}", + serde_json::to_string_pretty(&serde_json::json!({ + "status": "valid", + "identity_warnings": identity_warnings_json(&identity_warnings), + }))? + ); + } OutputFormat::Json => println!("{}", serde_json::json!({"status":"valid"})), } Ok(()) } else { + // The lint is deliberately skipped when the schema does not validate: + // it asks "where does this slot's element identity come from?" of a + // schema graph that is known to be incomplete, so its answers would be + // wrong (an unresolved import turns a class range into "not a class"). + // Reporting the errors first is also the only useful output here. match args.output { OutputFormat::Text => { for e in &errors { println!("{e}"); } + if args.lint_identity { + println!("note: --lint-identity skipped: fix the schema errors above first"); + } + } + OutputFormat::Json if args.lint_identity => { + println!( + "{}", + serde_json::to_string_pretty(&serde_json::json!({ + "errors": errors, + "identity_lint_skipped": "schema has errors; fix them and re-run", + }))? + ); } OutputFormat::Json => { println!("{}", serde_json::to_string_pretty(&errors)?); From 00a0dd5211b99dedf4faeb27a88660db5b4b2338 Mon Sep 17 00:00:00 2001 From: ejsyx Date: Tue, 18 Aug 2026 15:40:14 +0200 Subject: [PATCH 15/58] fix(tools): stable machine-readable JSON shape for --lint-identity The flagged JSON surface varied by case: identity_warnings was absent when the lint was skipped, and the skip was signalled by a prose string under the flag-named key. Both flagged cases now emit the same five keys, so a consumer reads one shape unconditionally: status, errors, identity_warnings, identity_lint_skipped, identity_lint_skipped_reason identity_lint_skipped is a real boolean with the reason in its own field, and identity_warnings distinguishes null (skipped, unknown) from [] (ran, found nothing). Output without the flag is unchanged, verified byte-for-byte against the pre-flag build in all four cases. Also drop exact duplicate warnings. lint_element_identity iterates get_class_ids(), which holds one id per class URI, so a class declaring an explicit class_uri is visited under both that and its default URI and every one of its warnings is emitted twice (personinfo.yaml: 10 warnings, 6 distinct). Correct the sort comment: the library does sort the classes it visits; only a single class's slot order is HashMap-derived. The sort now also lives inside the flag branch, so nothing runs when the flag is off. Co-Authored-By: Claude Fable 5 --- src/tools/src/bin/linkml_schema_validate.rs | 31 +++++++++++++++------ 1 file changed, 23 insertions(+), 8 deletions(-) diff --git a/src/tools/src/bin/linkml_schema_validate.rs b/src/tools/src/bin/linkml_schema_validate.rs index 9edd160..6e747dc 100644 --- a/src/tools/src/bin/linkml_schema_validate.rs +++ b/src/tools/src/bin/linkml_schema_validate.rs @@ -212,16 +212,24 @@ fn main() -> Result<(), Box> { // validation above used, so classes pulled in by `resolve_schemas` // from `imports:` are linted too. Warnings only — the exit code is // whatever the validation produced. - let mut identity_warnings = if args.lint_identity { - linkml_runtime::lint_element_identity(&sv) + let identity_warnings = if args.lint_identity { + let mut warnings = linkml_runtime::lint_element_identity(&sv); + // `lint_element_identity` visits classes in sorted order, but a + // class's own slots come from `ClassView::slots()`, which is backed + // by a HashMap — so the relative order of one class's warnings + // varies between runs. Sort by subject (class, then slot) to keep + // repeated runs over the same schema diffable. + warnings.sort_by(|a, b| a.subject.cmp(&b.subject).then(a.detail.cmp(&b.detail))); + // `lint_element_identity` iterates `get_class_ids()`, which holds one + // id per class *URI* — a class declaring an explicit `class_uri` is + // indexed under both that and its default URI, so it is visited + // twice and every one of its warnings is emitted twice. Drop the + // exact duplicates rather than reporting a slot twice. + warnings.dedup_by(|a, b| a.subject == b.subject && a.detail == b.detail); + warnings } else { Vec::new() }; - // `ClassView::slots()` is backed by a HashMap, so the linter emits a - // class's slots in an order that varies between runs. Sort by subject - // (class, then slot) so repeated runs over the same schema produce - // identical, diffable output. - identity_warnings.sort_by(|a, b| a.subject.cmp(&b.subject)); match args.output { OutputFormat::Text => { println!("schema valid"); @@ -234,7 +242,10 @@ fn main() -> Result<(), Box> { "{}", serde_json::to_string_pretty(&serde_json::json!({ "status": "valid", + "errors": errors, "identity_warnings": identity_warnings_json(&identity_warnings), + "identity_lint_skipped": false, + "identity_lint_skipped_reason": serde_json::Value::Null, }))? ); } @@ -260,8 +271,12 @@ fn main() -> Result<(), Box> { println!( "{}", serde_json::to_string_pretty(&serde_json::json!({ + "status": "invalid", "errors": errors, - "identity_lint_skipped": "schema has errors; fix them and re-run", + "identity_warnings": serde_json::Value::Null, + "identity_lint_skipped": true, + "identity_lint_skipped_reason": + "schema has errors; fix them and re-run", }))? ); } From 2f4c6ac5ecb22637a30f4a63df7ed16f684eb9a5 Mon Sep 17 00:00:00 2001 From: ejsyx Date: Tue, 18 Aug 2026 15:44:47 +0200 Subject: [PATCH 16/58] fix(runtime): lint_element_identity visits each class once, returns sorted MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two defects, both surfaced by running the linter over a real schema through linkml-schema-validate. get_class_ids() yields one id per class URI, and a class declaring an explicit class_uri is indexed under both that URI and its default one. Walking those ids visited such a class twice and reported every one of its slots twice (personinfo.yaml: 10 warnings for 6 distinct findings). Key a seen-set on the canonical URI, which is the same for both ids — the idiom ClassView::unique_keys already uses to walk a hierarchy. The class ids were sorted, but a class's own slots come from ClassView::slots(), which is HashMap-backed, so per-class warning order varied between runs. Sort the returned Vec by subject once, in the library, so every consumer inherits a stable order rather than each re-sorting. Service in the test fixture now declares class_uri, which is what exercises the first defect; the exact-set assert in schema_lint_flags_exactly_the_undeclared_positional_slots fails on the duplicates without the fix. diff_opaque and diff_unique_keys share the fixture and are unaffected. Co-Authored-By: Claude Fable 5 --- src/runtime/src/identity_lint.rs | 19 +++++++++++++-- src/runtime/tests/data/identity.yaml | 3 +++ src/runtime/tests/identity_lint.rs | 35 ++++++++++++++++++++++++++++ 3 files changed, 55 insertions(+), 2 deletions(-) diff --git a/src/runtime/src/identity_lint.rs b/src/runtime/src/identity_lint.rs index a703b30..df27cb8 100644 --- a/src/runtime/src/identity_lint.rs +++ b/src/runtime/src/identity_lint.rs @@ -27,7 +27,7 @@ use crate::diff::{element_identity_label, slot_is_ignored, slot_is_opaque, OPAQU use crate::{LinkMLInstance, ValidationProblemType, ValidationResult, ValidationResultSink}; use linkml_schemaview::identifier::Identifier; use linkml_schemaview::schemaview::SchemaView; -use std::collections::{BTreeMap, HashMap}; +use std::collections::{BTreeMap, HashMap, HashSet}; /// Schema-level lint: warn for every multivalued inlined slot whose element /// identity comes from nowhere. Warnings only — the schema stays usable. @@ -37,10 +37,19 @@ pub fn lint_element_identity(sv: &SchemaView) -> Vec { let conv = sv.converter(); let mut class_ids = sv.get_class_ids(); class_ids.sort(); + let mut seen: HashSet = HashSet::new(); for class_id in class_ids { let Ok(Some(class)) = sv.get_class(&Identifier::new(&class_id), &conv) else { continue; }; + // `get_class_ids` yields one id per class *URI*: a class declaring an + // explicit `class_uri` is indexed under both that and its default URI, + // so walking the ids naively reports each of its slots twice. Key the + // seen-set on the canonical URI, which is identical for both ids, the + // same idiom `ClassView::unique_keys` uses to walk a class hierarchy. + if !seen.insert(class.canonical_uri().to_string()) { + continue; + } for slot in class.slots() { if slot.determine_slot_container_mode() != SlotContainerMode::List { continue; @@ -93,7 +102,13 @@ pub fn lint_element_identity(sv: &SchemaView) -> Vec { ); } } - sink.into_vec() + let mut warnings = sink.into_vec(); + // The classes are visited in sorted id order, but a class's own slots come + // from `ClassView::slots()`, which is HashMap-backed, so the warnings for a + // single class arrive in an order that varies between runs. Sort here, once, + // so every consumer inherits a stable, diffable order. + warnings.sort_by(|a, b| a.subject.cmp(&b.subject)); + warnings } /// Data-level lint: warn for every list container whose elements repeat a diff --git a/src/runtime/tests/data/identity.yaml b/src/runtime/tests/data/identity.yaml index caeafb6..ed0bc87 100644 --- a/src/runtime/tests/data/identity.yaml +++ b/src/runtime/tests/data/identity.yaml @@ -14,6 +14,9 @@ default_range: string classes: Service: + # An explicit class_uri indexes the class under BOTH this URI and its + # default one, so a linter walking `get_class_ids()` sees it twice. + class_uri: identity:ServiceEndpoint attributes: name: {range: string} # spec Example 1 with its correct resolution: identity = the function diff --git a/src/runtime/tests/identity_lint.rs b/src/runtime/tests/identity_lint.rs index f1c31a1..e3e1b50 100644 --- a/src/runtime/tests/identity_lint.rs +++ b/src/runtime/tests/identity_lint.rs @@ -97,6 +97,41 @@ fn schema_lint_flags_exactly_the_undeclared_positional_slots() { ); } +#[test] +fn schema_lint_visits_a_class_with_an_explicit_class_uri_exactly_once() { + // `Service` declares `class_uri: identity:ServiceEndpoint`, so schemaview + // indexes it under both that URI and its default one and `get_class_ids()` + // yields it twice. Walking those ids naively reports every one of the + // class's slots twice. + let f = fixture(); + let warnings = lint_element_identity(&f.sv); + let mut subjects: Vec> = warnings.iter().map(|w| w.subject.clone()).collect(); + subjects.sort(); + let mut unique = subjects.clone(); + unique.dedup(); + assert_eq!( + subjects, unique, + "each slot must be reported once, not once per class URI: {warnings:#?}" + ); +} + +#[test] +fn schema_lint_warning_order_is_deterministic() { + // A class's slots come from `ClassView::slots()`, which is HashMap-backed, + // so the per-class warning order is process-dependent. Each fresh + // `SchemaView` gets its own hash seed, so an unsorted walk eventually + // disagrees with itself. + let expected = vec![ + vec!["Service".to_string(), "plainPhoneNumber".to_string()], + vec!["Service".to_string(), "tags".to_string()], + ]; + for _ in 0..20 { + let warnings = lint_element_identity(&fixture().sv); + let subjects: Vec> = warnings.iter().map(|w| w.subject.clone()).collect(); + assert_eq!(subjects, expected, "{warnings:#?}"); + } +} + #[test] fn data_lint_flags_duplicate_declared_identities() { let f = fixture(); From 774b825ef155a718cc339e322f5b891feafff71b Mon Sep 17 00:00:00 2001 From: ejsyx Date: Tue, 18 Aug 2026 15:44:47 +0200 Subject: [PATCH 17/58] refactor(tools): drop CLI-side lint dedupe/sort lint_element_identity now dedupes visits and returns a sorted Vec, so the CLI can render what the library hands it. Output is unchanged. Co-Authored-By: Claude Fable 5 --- src/tools/src/bin/linkml_schema_validate.rs | 15 +-------------- 1 file changed, 1 insertion(+), 14 deletions(-) diff --git a/src/tools/src/bin/linkml_schema_validate.rs b/src/tools/src/bin/linkml_schema_validate.rs index 6e747dc..62738bc 100644 --- a/src/tools/src/bin/linkml_schema_validate.rs +++ b/src/tools/src/bin/linkml_schema_validate.rs @@ -213,20 +213,7 @@ fn main() -> Result<(), Box> { // from `imports:` are linted too. Warnings only — the exit code is // whatever the validation produced. let identity_warnings = if args.lint_identity { - let mut warnings = linkml_runtime::lint_element_identity(&sv); - // `lint_element_identity` visits classes in sorted order, but a - // class's own slots come from `ClassView::slots()`, which is backed - // by a HashMap — so the relative order of one class's warnings - // varies between runs. Sort by subject (class, then slot) to keep - // repeated runs over the same schema diffable. - warnings.sort_by(|a, b| a.subject.cmp(&b.subject).then(a.detail.cmp(&b.detail))); - // `lint_element_identity` iterates `get_class_ids()`, which holds one - // id per class *URI* — a class declaring an explicit `class_uri` is - // indexed under both that and its default URI, so it is visited - // twice and every one of its warnings is emitted twice. Drop the - // exact duplicates rather than reporting a slot twice. - warnings.dedup_by(|a, b| a.subject == b.subject && a.detail == b.detail); - warnings + linkml_runtime::lint_element_identity(&sv) } else { Vec::new() }; From 15641b207d62196ffcddbe2a019d1d301b6cd446 Mon Sep 17 00:00:00 2001 From: ejsyx Date: Tue, 18 Aug 2026 15:55:20 +0200 Subject: [PATCH 18/58] fix(runtime): dedupe lint class visits by schema+name, not class_uri The round-2 dedupe keyed its seen-set on ClassView::canonical_uri(), which returns the explicit class_uri when one is declared. LinkML lets distinct classes declare the same class_uri -- meta.yaml's Anything and extensions.yaml's AnyValue both declare linkml:Any -- and both stay reachable through their own default URIs. The second such class hit seen.insert == false and had every one of its warnings dropped, with which class survived depending on hash order. A silent false negative is worse in a lint than the duplicate reporting the dedupe was added to fix. Key on (schema_id, name) instead, which is unique per class by construction and still collapses the two URIs of a single class. New fixture identity_shared_class_uri.yaml pulls in both directions at once: SharedUriA and SharedUriB share one class_uri and must both be reported, while TwoUris is indexed under two URIs of its own and must be reported once. Keeping them in one schema means neither fix can be made by breaking the other. The regression lives in its own fixture rather than in identity.yaml so that identity.yaml keeps yielding exactly the two documented Service warnings, which is the CLI's acceptance criterion. Co-Authored-By: Claude Fable 5 --- src/runtime/src/identity_lint.rs | 15 +++-- .../tests/data/identity_shared_class_uri.yaml | 56 +++++++++++++++++++ src/runtime/tests/identity_lint.rs | 39 +++++++++++++ 3 files changed, 105 insertions(+), 5 deletions(-) create mode 100644 src/runtime/tests/data/identity_shared_class_uri.yaml diff --git a/src/runtime/src/identity_lint.rs b/src/runtime/src/identity_lint.rs index df27cb8..33a36ba 100644 --- a/src/runtime/src/identity_lint.rs +++ b/src/runtime/src/identity_lint.rs @@ -37,17 +37,22 @@ pub fn lint_element_identity(sv: &SchemaView) -> Vec { let conv = sv.converter(); let mut class_ids = sv.get_class_ids(); class_ids.sort(); - let mut seen: HashSet = HashSet::new(); + let mut seen: HashSet<(String, String)> = HashSet::new(); for class_id in class_ids { let Ok(Some(class)) = sv.get_class(&Identifier::new(&class_id), &conv) else { continue; }; // `get_class_ids` yields one id per class *URI*: a class declaring an // explicit `class_uri` is indexed under both that and its default URI, - // so walking the ids naively reports each of its slots twice. Key the - // seen-set on the canonical URI, which is identical for both ids, the - // same idiom `ClassView::unique_keys` uses to walk a class hierarchy. - if !seen.insert(class.canonical_uri().to_string()) { + // so walking the ids naively reports each of its slots twice. + // + // Key the seen-set on (schema, name), which is unique per class by + // construction. Keying on the class URI would be wrong in the other + // direction: LinkML lets distinct classes declare the same `class_uri` + // (meta.yaml's `Anything` and extensions.yaml's `AnyValue` both declare + // `linkml:Any`), and that would silently drop the second class's + // warnings — a false negative, worse in a lint than a duplicate. + if !seen.insert((class.schema_id().to_string(), class.name().to_string())) { continue; } for slot in class.slots() { diff --git a/src/runtime/tests/data/identity_shared_class_uri.yaml b/src/runtime/tests/data/identity_shared_class_uri.yaml new file mode 100644 index 0000000..718f605 --- /dev/null +++ b/src/runtime/tests/data/identity_shared_class_uri.yaml @@ -0,0 +1,56 @@ +id: https://w3id.org/linkml/examples/identity-shared-class-uri +name: identity_shared_class_uri +description: |- + Regression fixture for the identity linter's class-visit dedupe. + + LinkML permits several distinct classes to declare the SAME `class_uri` + (in this repo, `meta.yaml`'s `Anything` and `extensions.yaml`'s `AnyValue` + both declare `linkml:Any`). A linter that dedupes its class visits by class + URI treats the second such class as already-seen and silently drops all of + its warnings — a false negative, which is worse than the duplicate + reporting the dedupe exists to prevent. + + `SharedUriA` and `SharedUriB` declare one shared `class_uri` and each holds + one multivalued inlined slot with no declared element identity, so both must + be reported. `TwoUris` additionally declares an explicit `class_uri`, which + indexes it under both that and its default URI: it must be reported once, + not twice. The two requirements pull in opposite directions, which is the + point of keeping them in one fixture. +license: https://creativecommons.org/publicdomain/zero/1.0/ +imports: + - linkml:types +prefixes: + shared: https://w3id.org/linkml/examples/identity-shared-class-uri/ + linkml: https://w3id.org/linkml/ +default_prefix: shared +default_range: string + +classes: + SharedUriA: + class_uri: shared:SharedThing + attributes: + itemsA: + range: Vertex + multivalued: true + inlined_as_list: true + + SharedUriB: + class_uri: shared:SharedThing + attributes: + itemsB: + range: Vertex + multivalued: true + inlined_as_list: true + + TwoUris: + class_uri: shared:SomethingElse + attributes: + itemsC: + range: Vertex + multivalued: true + inlined_as_list: true + + Vertex: + attributes: + x: {range: float} + y: {range: float} diff --git a/src/runtime/tests/identity_lint.rs b/src/runtime/tests/identity_lint.rs index e3e1b50..5cc5a0e 100644 --- a/src/runtime/tests/identity_lint.rs +++ b/src/runtime/tests/identity_lint.rs @@ -15,6 +15,17 @@ struct Fixture { service: ClassView, } +/// Loads a schema from `tests/data` into its own `SchemaView`. +fn schema_view(file: &str) -> SchemaView { + let mut p = PathBuf::from(env!("CARGO_MANIFEST_DIR")); + p.push("tests/data"); + p.push(file); + let schema = from_yaml(&p).unwrap(); + let mut sv = SchemaView::new(); + sv.add_schema(schema).unwrap(); + sv +} + fn fixture() -> Fixture { let mut p = PathBuf::from(env!("CARGO_MANIFEST_DIR")); p.push("tests/data/identity.yaml"); @@ -115,6 +126,34 @@ fn schema_lint_visits_a_class_with_an_explicit_class_uri_exactly_once() { ); } +#[test] +fn schema_lint_reports_both_classes_that_share_one_class_uri() { + // LinkML lets distinct classes declare the same `class_uri` (meta.yaml's + // `Anything` and extensions.yaml's `AnyValue` both use `linkml:Any`). + // Deduping visits by class_uri makes the second class look already-seen and + // silently drops every one of its warnings — a false negative, which is + // worse in a lint than the duplicate reporting it was meant to fix. + // + // The fixture pulls in both directions at once: `SharedUriA`/`SharedUriB` + // share one `class_uri` and must BOTH be reported, while `TwoUris` is + // indexed under two URIs of its own and must be reported only once. + let sv = schema_view("identity_shared_class_uri.yaml"); + let subjects: Vec> = lint_element_identity(&sv) + .iter() + .map(|w| w.subject.clone()) + .collect(); + assert_eq!( + subjects, + vec![ + vec!["SharedUriA".to_string(), "itemsA".to_string()], + vec!["SharedUriB".to_string(), "itemsB".to_string()], + vec!["TwoUris".to_string(), "itemsC".to_string()], + ], + "classes sharing a class_uri must each be linted, and a class indexed \ + under two URIs must be linted once" + ); +} + #[test] fn schema_lint_warning_order_is_deterministic() { // A class's slots come from `ClassView::slots()`, which is HashMap-backed, From 2cfe8f2cdf902efc4ee648ea02e1dba064032bf7 Mon Sep 17 00:00:00 2001 From: ejsyx Date: Tue, 18 Aug 2026 16:07:58 +0200 Subject: [PATCH 19/58] fix(schemaview): resolve_schemas loops to fixpoint, resolves imports relative to source files Two defects kept a multi-file schema with sibling imports from ever loading, found while linting a real asset360 model. resolve_schemas snapshotted get_unresolved_schemas() once and iterated that snapshot. An import only becomes visible after the schema declaring it is loaded, so imports-of-imports were never attempted: a root importing ./rsm resolved rsm, but rsm's own ./types stayed unresolved. Resolution now repeats until nothing is unresolved or a round achieves nothing. A single failure also returned Err immediately, abandoning the rest of the pass; failures are now collected per round and only become the error when a round resolves nothing, so an import that failed while other work was still progressing gets retried. Nothing recorded where a schema was read from, so a relative import was joined onto parent(get_resolution_uri_of_schema(S)), which is None for the root (degenerating to the process CWD) and the original relative import string for everything else ("./types" -> parent "." -> CWD again). Resolution now tracks schema id -> canonicalized source directory for every schema it loads from a file, and tries bases in order: recorded source dir, then the old resolution-uri parent, then CWD. Keeping CWD last preserves schemas that import CWD-relative paths. New resolve_schemas_from(sv, root_source) seeds the root's own directory, which is the only thing the caller knows and the view cannot. Plain resolve_schemas delegates unseeded, so existing callers are unaffected. Co-Authored-By: Claude Fable 5 --- src/schemaview/src/resolve.rs | 217 +++++++++++++----- .../tests/data/missing_imports.yaml | 15 ++ src/schemaview/tests/data/nested_a.yaml | 17 ++ src/schemaview/tests/data/nested_b.yaml | 12 + src/schemaview/tests/data/nested_root.yaml | 17 ++ src/schemaview/tests/nested_local_import.rs | 91 ++++++++ 6 files changed, 311 insertions(+), 58 deletions(-) create mode 100644 src/schemaview/tests/data/missing_imports.yaml create mode 100644 src/schemaview/tests/data/nested_a.yaml create mode 100644 src/schemaview/tests/data/nested_b.yaml create mode 100644 src/schemaview/tests/data/nested_root.yaml create mode 100644 src/schemaview/tests/nested_local_import.rs diff --git a/src/schemaview/src/resolve.rs b/src/schemaview/src/resolve.rs index b920e09..c29bb52 100644 --- a/src/schemaview/src/resolve.rs +++ b/src/schemaview/src/resolve.rs @@ -2,7 +2,8 @@ use crate::{ io::{from_uri, from_yaml}, schemaview::SchemaView, }; -use std::path::Path; +use std::collections::HashMap; +use std::path::{Path, PathBuf}; fn get_uri_for_id(id: &str) -> Option<&'static str> { match id { @@ -15,68 +16,168 @@ fn get_uri_for_id(id: &str) -> Option<&'static str> { } } -pub fn resolve_schemas(sv: &mut SchemaView) -> Result<(), String> { - let unresolved = sv.get_unresolved_schemas(); - if unresolved.is_empty() { +/// Directory a schema's relative imports should be resolved against, given the +/// path the schema itself was loaded from. Accepts either the schema file or a +/// directory, so callers can pass whichever they have. +fn source_dir_of(path: &Path) -> Option { + let canonical = std::fs::canonicalize(path).ok()?; + if canonical.is_dir() { + Some(canonical) + } else { + canonical.parent().map(|p| p.to_path_buf()) + } +} + +/// Returns `path` if it names an existing file, retrying with a `.yaml`/`.yml` +/// extension — LinkML imports habitually omit it (`imports: [./types]`). +fn existing_schema_file(path: PathBuf) -> Option { + if path.is_file() { + return Some(path); + } + for ext in ["yaml", "yml"] { + let candidate = path.with_extension(ext); + if candidate.is_file() { + return Some(candidate); + } + } + None +} + +/// Locates the file behind a relative import of `schema_id`. +/// +/// Bases are tried in order of how much we actually know: +/// 1. the directory of the file `schema_id` was itself loaded from, +/// 2. the directory of the import URI that pulled `schema_id` in, +/// 3. the process working directory — the historical behaviour, kept last so +/// that schemas importing CWD-relative paths keep resolving. +fn locate_import( + sv: &SchemaView, + schema_id: &str, + uri: &str, + source_dirs: &HashMap, +) -> Option { + let raw = Path::new(uri); + if raw.is_absolute() { + return existing_schema_file(raw.to_path_buf()); + } + let mut bases: Vec = Vec::new(); + if let Some(dir) = source_dirs.get(schema_id) { + bases.push(dir.clone()); + } + if let Some(resolution_uri) = sv.get_resolution_uri_of_schema(schema_id) { + if let Some(parent) = Path::new(&resolution_uri).parent() { + bases.push(parent.to_path_buf()); + } + } + bases.push(PathBuf::new()); + for base in bases { + if let Some(found) = existing_schema_file(base.join(raw)) { + return Some(found); + } + } + None +} + +/// Loads the schema behind one unresolved import and records where it came +/// from, so that schema's own relative imports can be resolved in a later round. +fn resolve_one( + sv: &mut SchemaView, + schema_id: &str, + uri: &str, + source_dirs: &mut HashMap, +) -> Result<(), String> { + let import_ref = Some((schema_id.to_string(), uri.to_string())); + + if let Some(resolved_uri) = get_uri_for_id(uri) { + let schema = from_uri(resolved_uri) + .map_err(|e| format!("Failed to load schema from {}: {}", resolved_uri, e))?; + sv.add_schema_with_import_ref(schema, import_ref)?; return Ok(()); } - for (schema_id, uri) in unresolved { - let schema_source_uri = sv.get_resolution_uri_of_schema(&schema_id); - if let Some(resolved_uri) = get_uri_for_id(&uri) { - // Load the schema from the resolved URI - let schema = match from_uri(resolved_uri) { - Ok(s) => s, - Err(e) => { - return Err(format!( - "Failed to load schema from {}: {}", - resolved_uri, e - )) - } - }; - sv.add_schema_with_import_ref(schema, Some((schema_id, uri)))?; - } else { - // Attempt to treat the unresolved entry as a local file path - let mut path = Path::new(&uri).to_owned(); - if !path.is_absolute() { - // imported_from_dir = parent from schema_source_uri - let imported_from_dir = - Path::new(schema_source_uri.as_deref().unwrap_or("unknown")) - .parent() - .map(|p| p.to_path_buf()); - if let Some(dir) = imported_from_dir { - path = dir.join(path); - } - if !path.exists() && path.with_extension("yaml").exists() { - path.set_extension("yaml"); - } - if !path.exists() && path.with_extension("yml").exists() { - path.set_extension("yml"); - } + let Some(path) = locate_import(sv, schema_id, uri, source_dirs) else { + return Err(format!( + "No resolution found for URI: {} imported from {}", + uri, + source_dirs + .get(schema_id) + .map(|d| d.display().to_string()) + .or_else(|| sv.get_resolution_uri_of_schema(schema_id)) + .unwrap_or_else(|| schema_id.to_owned()) + )); + }; + + let schema = from_yaml(&path) + .map_err(|e| format!("Failed to load schema from {}: {}", path.display(), e))?; + let loaded_id = schema.id.clone(); + sv.add_schema_with_import_ref(schema, import_ref)?; + if let Some(dir) = source_dir_of(&path) { + source_dirs.insert(loaded_id, dir); + } + Ok(()) +} + +/// Resolves imports repeatedly until nothing is left or a round achieves +/// nothing, so that imports-of-imports are reached. +/// +/// A failure no longer abandons the rest of the pass: every import is attempted +/// each round, and only when a round resolves nothing do the accumulated +/// messages become the error. An import that failed while other work was still +/// progressing gets retried, because that later work may be exactly what makes +/// it resolvable. +fn resolve_to_fixpoint( + sv: &mut SchemaView, + mut source_dirs: HashMap, +) -> Result<(), String> { + loop { + let unresolved = sv.get_unresolved_schemas(); + if unresolved.is_empty() { + return Ok(()); + } + let mut progressed = false; + let mut failures: Vec = Vec::new(); + for (schema_id, uri) in unresolved { + match resolve_one(sv, &schema_id, &uri, &mut source_dirs) { + Ok(()) => progressed = true, + Err(e) => failures.push(e), } - if path.exists() { - let schema = match from_yaml(&path) { - Ok(s) => s, - Err(e) => { - return Err(format!( - "Failed to load schema from {}: {}", - path.display(), - e - )) - } - }; - sv.add_schema_with_import_ref( - schema.clone(), - Some((schema_id.clone(), uri.clone())), - )?; + } + if !progressed { + return Err(if failures.is_empty() { + "import resolution stalled with imports still unresolved".to_string() } else { - return Err(format!( - "No resolution found for URI: {} imported from {}", - uri, - schema_source_uri.unwrap_or("unknown".to_owned()) - )); - } + failures.join("\n") + }); } } - Ok(()) +} + +/// Resolves every import reachable from the schemas already in `sv`. +/// +/// Relative imports of those schemas are resolved against the process working +/// directory, because nothing records where they were loaded from. Prefer +/// [`resolve_schemas_from`], which removes that guess. +pub fn resolve_schemas(sv: &mut SchemaView) -> Result<(), String> { + resolve_to_fixpoint(sv, HashMap::new()) +} + +/// Resolves every import reachable from the schemas already in `sv`, treating +/// `root_source` — the path they were loaded from — as the base for their +/// relative imports. Schemas pulled in during resolution get the same treatment +/// from their own file, so `a.yaml` importing `./b` works at any depth and from +/// any working directory. +/// +/// `root_source` may be the schema file or its directory. +pub fn resolve_schemas_from(sv: &mut SchemaView, root_source: &Path) -> Result<(), String> { + let mut source_dirs = HashMap::new(); + if let Some(dir) = source_dir_of(root_source) { + // Everything currently in the view came from `root_source`: the callers + // load exactly one schema before resolving. + sv.with_schema_definitions(|schemas| { + for id in schemas.keys() { + source_dirs.insert(id.clone(), dir.clone()); + } + }); + } + resolve_to_fixpoint(sv, source_dirs) } diff --git a/src/schemaview/tests/data/missing_imports.yaml b/src/schemaview/tests/data/missing_imports.yaml new file mode 100644 index 0000000..834b564 --- /dev/null +++ b/src/schemaview/tests/data/missing_imports.yaml @@ -0,0 +1,15 @@ +id: http://example.com/missing_imports +name: missing_imports +prefixes: + missing_imports: http://example.com/missing_imports/ +default_prefix: missing_imports +default_range: string +# Neither file exists: resolution must report BOTH, not stop at the first. +imports: + - ./definitely_not_here + - ./also_not_here +classes: + Stub: + attributes: + id: + identifier: true diff --git a/src/schemaview/tests/data/nested_a.yaml b/src/schemaview/tests/data/nested_a.yaml new file mode 100644 index 0000000..bb46b4d --- /dev/null +++ b/src/schemaview/tests/data/nested_a.yaml @@ -0,0 +1,17 @@ +id: http://example.com/nested_a +name: nested_a +prefixes: + nested_a: http://example.com/nested_a/ + nested_b: http://example.com/nested_b/ +default_prefix: nested_a +default_range: string +# An import of an import: only reachable if resolution loops to a fixpoint. +imports: + - ./nested_b +classes: + MiddleThing: + attributes: + id: + identifier: true + leaf: + range: nested_b:LeafThing diff --git a/src/schemaview/tests/data/nested_b.yaml b/src/schemaview/tests/data/nested_b.yaml new file mode 100644 index 0000000..df12948 --- /dev/null +++ b/src/schemaview/tests/data/nested_b.yaml @@ -0,0 +1,12 @@ +id: http://example.com/nested_b +name: nested_b +prefixes: + nested_b: http://example.com/nested_b/ +default_prefix: nested_b +default_range: string +classes: + LeafThing: + attributes: + id: + identifier: true + label: {} diff --git a/src/schemaview/tests/data/nested_root.yaml b/src/schemaview/tests/data/nested_root.yaml new file mode 100644 index 0000000..aa9d474 --- /dev/null +++ b/src/schemaview/tests/data/nested_root.yaml @@ -0,0 +1,17 @@ +id: http://example.com/nested_root +name: nested_root +prefixes: + nested_root: http://example.com/nested_root/ + nested_a: http://example.com/nested_a/ +default_prefix: nested_root +default_range: string +# Relative to THIS file's directory, not the process CWD. +imports: + - ./nested_a +classes: + RootThing: + attributes: + id: + identifier: true + middle: + range: nested_a:MiddleThing diff --git a/src/schemaview/tests/nested_local_import.rs b/src/schemaview/tests/nested_local_import.rs new file mode 100644 index 0000000..a36f2eb --- /dev/null +++ b/src/schemaview/tests/nested_local_import.rs @@ -0,0 +1,91 @@ +#![cfg(feature = "resolve")] +//! Relative imports must resolve against the importing schema's own file, and +//! resolution must keep going until nothing is left to resolve. +//! +//! `nested_root.yaml` imports `./nested_a`, which in turn imports `./nested_b`. +//! Two independent things have to work for `nested_b` to arrive: +//! +//! 1. **Fixpoint.** `nested_a`'s import only becomes visible after `nested_a` +//! itself is loaded, so a single pass over the initially-unresolved list can +//! never reach `nested_b`. +//! 2. **Source-relative bases.** `./nested_a` is meaningless relative to the +//! process CWD (these tests run from the crate root, not `tests/data`), so +//! the base directory has to come from the file each schema was loaded from. + +use linkml_schemaview::io::from_yaml; +use linkml_schemaview::resolve::{resolve_schemas, resolve_schemas_from}; +use linkml_schemaview::schemaview::SchemaView; +use std::path::PathBuf; + +fn data_path(name: &str) -> PathBuf { + let mut p = PathBuf::from(env!("CARGO_MANIFEST_DIR")); + p.push("tests"); + p.push("data"); + p.push(name); + p +} + +fn view_of(name: &str) -> (SchemaView, PathBuf) { + let path = data_path(name); + let schema = from_yaml(&path).unwrap(); + let mut sv = SchemaView::new(); + sv.add_schema(schema).unwrap(); + (sv, path) +} + +#[test] +fn resolves_an_import_of_an_import_relative_to_each_source_file() { + let (mut sv, path) = view_of("nested_root.yaml"); + // The process CWD is the crate root, so `./nested_a` does not exist + // relative to it: only the seeded source directory can resolve this. + assert!( + !PathBuf::from("./nested_a.yaml").exists(), + "precondition: the fixture must not be reachable from the CWD, \ + otherwise this test would pass for the wrong reason" + ); + + resolve_schemas_from(&mut sv, &path).unwrap(); + + assert!( + sv.get_unresolved_schemas().is_empty(), + "left unresolved: {:?}", + sv.get_unresolved_schemas() + ); + assert!( + sv.get_schema("http://example.com/nested_a").is_some(), + "the root's own import must resolve against the root's directory" + ); + assert!( + sv.get_schema("http://example.com/nested_b").is_some(), + "the import of an import must resolve against ITS importer's directory, \ + which requires both the fixpoint loop and per-schema source tracking" + ); +} + +#[test] +fn unseeded_resolution_still_honours_cwd_relative_imports() { + // `local_main.yaml` imports `tests/data/local_target.yaml`, a CWD-relative + // path. Seeding a source directory must not break that older style: the + // source directory is the first base tried, not the only one. + let (mut sv, path) = view_of("local_main.yaml"); + resolve_schemas_from(&mut sv, &path).unwrap(); + assert!(sv.get_unresolved_schemas().is_empty()); + assert!(sv.get_schema("http://example.com/local_target").is_some()); + + // ...and the unseeded entry point behaves exactly as before. + let (mut sv, _) = view_of("local_main.yaml"); + resolve_schemas(&mut sv).unwrap(); + assert!(sv.get_schema("http://example.com/local_target").is_some()); +} + +#[test] +fn reports_every_unresolvable_import_rather_than_only_the_first() { + // Two bad imports in one schema: the error must mention both, since a pass + // that returns on the first failure hides the rest of the work. + let (mut sv, path) = view_of("missing_imports.yaml"); + let err = resolve_schemas_from(&mut sv, &path).unwrap_err(); + assert!( + err.contains("definitely_not_here") && err.contains("also_not_here"), + "both failures must be reported, got: {err}" + ); +} From 6a3999a1b0bafc0a0fac0726f38b2e4883bd0d29 Mon Sep 17 00:00:00 2001 From: ejsyx Date: Tue, 18 Aug 2026 16:07:58 +0200 Subject: [PATCH 20/58] fix(tools): seed schema source path into import resolution Every bin takes a schema path and then resolved imports without telling the resolver where that file lives, so a schema importing ./sibling only loaded when the process happened to run from the schema's directory. All five now call resolve_schemas_from with args.schema. Co-Authored-By: Claude Fable 5 --- src/tools/src/bin/linkml_convert.rs | 4 ++-- src/tools/src/bin/linkml_diff.rs | 4 ++-- src/tools/src/bin/linkml_patch.rs | 4 ++-- src/tools/src/bin/linkml_schema_validate.rs | 4 ++-- src/tools/src/bin/linkml_validate.rs | 4 ++-- 5 files changed, 10 insertions(+), 10 deletions(-) diff --git a/src/tools/src/bin/linkml_convert.rs b/src/tools/src/bin/linkml_convert.rs index 4cd1b33..da33697 100644 --- a/src/tools/src/bin/linkml_convert.rs +++ b/src/tools/src/bin/linkml_convert.rs @@ -10,7 +10,7 @@ use linkml_runtime::{ #[cfg(feature = "ttl")] use linkml_schemaview::io::from_yaml; #[cfg(all(feature = "ttl", feature = "resolve"))] -use linkml_schemaview::resolve::resolve_schemas; +use linkml_schemaview::resolve::resolve_schemas_from; #[cfg(feature = "ttl")] use linkml_schemaview::schemaview::SchemaView; #[cfg(feature = "ttl")] @@ -99,7 +99,7 @@ fn main() -> Result<(), Box> { #[cfg(feature = "resolve")] { eprintln!("Resolving schemas..."); - resolve_schemas(&mut sv).map_err(|e| e.to_string())?; + resolve_schemas_from(&mut sv, &args.schema).map_err(|e| e.to_string())?; eprintln!("Schemas resolved"); } let conv = sv.converter(); diff --git a/src/tools/src/bin/linkml_diff.rs b/src/tools/src/bin/linkml_diff.rs index ec4a85b..6cdc3ab 100644 --- a/src/tools/src/bin/linkml_diff.rs +++ b/src/tools/src/bin/linkml_diff.rs @@ -2,7 +2,7 @@ use clap::Parser; use linkml_runtime::{diff, load_json_file, load_yaml_file, DiffOptions}; use linkml_schemaview::io::from_yaml; #[cfg(feature = "resolve")] -use linkml_schemaview::resolve::resolve_schemas; +use linkml_schemaview::resolve::resolve_schemas_from; use linkml_schemaview::schemaview::{ClassView, SchemaView}; use linkml_schemaview::Converter; use std::fs::File; @@ -61,7 +61,7 @@ fn main() -> Result<(), Box> { let mut sv = SchemaView::new(); sv.add_schema(schema.clone()).map_err(|e| e.to_string())?; #[cfg(feature = "resolve")] - resolve_schemas(&mut sv).map_err(|e| e.to_string())?; + resolve_schemas_from(&mut sv, &args.schema).map_err(|e| e.to_string())?; let conv = sv.converter(); let class_view = sv.get_tree_root_or(args.class.as_deref()).ok_or_else(|| { format!( diff --git a/src/tools/src/bin/linkml_patch.rs b/src/tools/src/bin/linkml_patch.rs index 3405c41..cf3e199 100644 --- a/src/tools/src/bin/linkml_patch.rs +++ b/src/tools/src/bin/linkml_patch.rs @@ -2,7 +2,7 @@ use clap::Parser; use linkml_runtime::{load_json_file, load_yaml_file, patch, Delta}; use linkml_schemaview::io::from_yaml; #[cfg(feature = "resolve")] -use linkml_schemaview::resolve::resolve_schemas; +use linkml_schemaview::resolve::resolve_schemas_from; use linkml_schemaview::schemaview::{ClassView, SchemaView}; use linkml_schemaview::Converter; use std::fs::File; @@ -84,7 +84,7 @@ fn main() -> Result<(), Box> { let mut sv = SchemaView::new(); sv.add_schema(schema.clone()).map_err(|e| e.to_string())?; #[cfg(feature = "resolve")] - resolve_schemas(&mut sv).map_err(|e| e.to_string())?; + resolve_schemas_from(&mut sv, &args.schema).map_err(|e| e.to_string())?; let conv = sv.converter(); let class_view = sv.get_tree_root_or(args.class.as_deref()).ok_or_else(|| { format!( diff --git a/src/tools/src/bin/linkml_schema_validate.rs b/src/tools/src/bin/linkml_schema_validate.rs index 62738bc..dadd72d 100644 --- a/src/tools/src/bin/linkml_schema_validate.rs +++ b/src/tools/src/bin/linkml_schema_validate.rs @@ -1,7 +1,7 @@ use clap::{Parser, ValueEnum}; use linkml_runtime::{ValidationResult, ValidationSeverity}; #[cfg(feature = "resolve")] -use linkml_schemaview::resolve::resolve_schemas; +use linkml_schemaview::resolve::resolve_schemas_from; use linkml_schemaview::{identifier::Identifier, io::from_yaml, schemaview::SchemaView, Converter}; use std::path::PathBuf; @@ -139,7 +139,7 @@ fn main() -> Result<(), Box> { let mut sv = SchemaView::new(); sv.add_schema(schema.clone()).map_err(|e| e.to_string())?; #[cfg(feature = "resolve")] - if let Err(e) = resolve_schemas(&mut sv) { + if let Err(e) = resolve_schemas_from(&mut sv, &args.schema) { eprintln!("{e}"); } let conv = sv.converter(); diff --git a/src/tools/src/bin/linkml_validate.rs b/src/tools/src/bin/linkml_validate.rs index bc3da77..5b9ae4e 100644 --- a/src/tools/src/bin/linkml_validate.rs +++ b/src/tools/src/bin/linkml_validate.rs @@ -5,7 +5,7 @@ use linkml_runtime::{ use linkml_schemaview::identifier::Identifier; use linkml_schemaview::io::from_yaml; #[cfg(feature = "resolve")] -use linkml_schemaview::resolve::resolve_schemas; +use linkml_schemaview::resolve::resolve_schemas_from; use linkml_schemaview::schemaview::SchemaView; use serde_json::json; use std::path::PathBuf; @@ -34,7 +34,7 @@ fn main() -> Result<(), Box> { ) .map_err(|e| e.to_string())?; #[cfg(feature = "resolve")] - resolve_schemas(&mut sv).map_err(|e| e.to_string())?; + resolve_schemas_from(&mut sv, &args.schema).map_err(|e| e.to_string())?; let conv = sv.converter(); let class_view = sv .get_class(&Identifier::new(&args.class), &conv) From 0215dbd9d834f6edbbeeebfc0817184aa9b96cee Mon Sep 17 00:00:00 2001 From: ejsyx Date: Tue, 18 Aug 2026 16:15:05 +0200 Subject: [PATCH 21/58] feat(runtime): lint reports inherited slots once, at the introducing class A flagged slot is inherited by every descendant, so the lint reported it once per class and buried the single declaration the author would edit. On the asset360 model that meant 48 warnings covering 25 distinct (slot, element class) pairs: quantity 7 times via ObservableProperty's coordinate subclasses, elementCollections 8, and pictures / secondaryImages / hasCoveredSections 4 each. A flagged slot is now reported only where it is introduced: skipped when the direct is_a parent carries a slot of the same name that is flagged for the same reason. Applied at every level this leaves the topmost flagged declarer. The "same reason" half matters. A subclass whose slot_usage changes the identity answer is judged on its own merits in both directions: narrowing a range to a keyed class makes the subclass silent while its parent stays reported, and widening a clean slot to an identity-less range makes the subclass the introducer and reports it there. Only the is_a chain is walked. A slot arriving from a mixin is still reported on the class using the mixin as well as on the mixin itself: a mixin can be applied to unrelated classes, so there is no single owning declaration to point at. Documented in the rustdoc rather than guessed at. asset360 goes from 48 warnings to 25 with all 25 distinct pairs intact. Co-Authored-By: Claude Fable 5 --- src/runtime/src/identity_lint.rs | 79 +++++++++++++++---- .../tests/data/identity_inheritance.yaml | 79 +++++++++++++++++++ src/runtime/tests/identity_lint.rs | 33 ++++++++ 3 files changed, 175 insertions(+), 16 deletions(-) create mode 100644 src/runtime/tests/data/identity_inheritance.yaml diff --git a/src/runtime/src/identity_lint.rs b/src/runtime/src/identity_lint.rs index 33a36ba..1753349 100644 --- a/src/runtime/src/identity_lint.rs +++ b/src/runtime/src/identity_lint.rs @@ -22,17 +22,72 @@ //! and has no element identity to declare. `ignore` silences the lint by //! removing the slot from diff's scope; `opaque` silences it by answering the //! question with "nowhere — replace the value as a whole". +//! +//! Warnings are reported at the class that **introduces** the slot: a flagged +//! slot inherited unchanged by a descendant is not repeated there, since the +//! declaration the author would edit lives on the ancestor. use crate::diff::{element_identity_label, slot_is_ignored, slot_is_opaque, OPAQUE_ANNOTATION}; use crate::{LinkMLInstance, ValidationProblemType, ValidationResult, ValidationResultSink}; use linkml_schemaview::identifier::Identifier; -use linkml_schemaview::schemaview::SchemaView; +use linkml_schemaview::schemaview::{ClassView, SchemaView}; +use linkml_schemaview::slotview::SlotView; use std::collections::{BTreeMap, HashMap, HashSet}; +/// Whether this slot is one the lint flags: a multivalued inlined slot whose +/// element identity comes from nowhere. +/// +/// Split out from the reporting loop because the same question has to be asked +/// of an inherited slot on its parent class, to decide which class introduced +/// the problem. +fn slot_lacks_element_identity(slot: &SlotView) -> bool { + use linkml_schemaview::slotview::{SlotContainerMode, SlotInlineMode}; + if slot.determine_slot_container_mode() != SlotContainerMode::List { + return false; + } + if slot.determine_slot_inline_mode() == SlotInlineMode::Reference { + return false; // elements are references, not inlined + } + if slot_is_opaque(slot) { + return false; // identity declared: nowhere, replace the value as a whole + } + if slot_is_ignored(slot) { + return false; // outside diff's scope entirely: no deltas, no identity + } + if let Some(rc) = slot.get_range_class() { + if rc.key_or_identifier_slot().is_some() || !rc.unique_keys().is_empty() { + return false; + } + } + true +} + +/// Whether `class` is where a flagged slot should be reported, rather than an +/// ancestor it merely inherits the problem from. +/// +/// Answers no only when the direct `is_a` parent carries a slot of the same +/// name that is flagged *for the same reason*. Applied at every level this +/// leaves exactly the topmost flagged declarer, and it keeps a subclass whose +/// `slot_usage` changes the answer — narrowing the range to a keyed class, or +/// widening it away from one — judged on its own merits in both directions. +/// +/// Only the `is_a` chain is walked. A slot arriving from a mixin is reported on +/// the class using the mixin as well as on the mixin itself: a mixin can be +/// applied to unrelated classes, so there is no single owning declaration to +/// point at, and chasing one is not worth the complexity. +fn introduces_flagged_slot(class: &ClassView, slot_name: &str) -> bool { + let Ok(Some(parent)) = class.parent_class() else { + return true; // no is_a parent: this class is the declarer + }; + let Some(parent_slot) = parent.slot(&Identifier::new(slot_name)) else { + return true; // the parent does not have it: introduced here + }; + !slot_lacks_element_identity(&parent_slot) +} + /// Schema-level lint: warn for every multivalued inlined slot whose element /// identity comes from nowhere. Warnings only — the schema stays usable. pub fn lint_element_identity(sv: &SchemaView) -> Vec { - use linkml_schemaview::slotview::{SlotContainerMode, SlotInlineMode}; let mut sink = ValidationResultSink::default(); let conv = sv.converter(); let mut class_ids = sv.get_class_ids(); @@ -56,24 +111,16 @@ pub fn lint_element_identity(sv: &SchemaView) -> Vec { continue; } for slot in class.slots() { - if slot.determine_slot_container_mode() != SlotContainerMode::List { + if !slot_lacks_element_identity(slot) { continue; } - if slot.determine_slot_inline_mode() == SlotInlineMode::Reference { - continue; // elements are references, not inlined - } - if slot_is_opaque(slot) { - continue; // identity declared: nowhere, replace the value as a whole - } - if slot_is_ignored(slot) { - continue; // outside diff's scope entirely: no deltas, no identity + // Report at the class that introduces the slot: repeating an + // inherited warning on every descendant buries the one declaration + // the author would actually edit. + if !introduces_flagged_slot(&class, &slot.name) { + continue; } let range_class = slot.get_range_class(); - if let Some(rc) = &range_class { - if rc.key_or_identifier_slot().is_some() || !rc.unique_keys().is_empty() { - continue; - } - } // The advice has to fit the range: a scalar- or enum-ranged slot has // no element class on which a key could be declared. let detail = match &range_class { diff --git a/src/runtime/tests/data/identity_inheritance.yaml b/src/runtime/tests/data/identity_inheritance.yaml new file mode 100644 index 0000000..9626d05 --- /dev/null +++ b/src/runtime/tests/data/identity_inheritance.yaml @@ -0,0 +1,79 @@ +id: https://w3id.org/linkml/examples/identity-inheritance +name: identity_inheritance +description: |- + Regression fixture for "report a flagged slot once, at the class that + introduces it". + + A flagged slot is inherited by every descendant, so reporting it per class + buries the one place the author can fix it under a pile of copies. The lint + reports only the topmost class whose direct `is_a` parent does not already + carry the same slot in the same flagged state. + + The classes below cover the cases that pull against each other: + Base introduces the flagged `outline` -> reported + Middle inherits it unchanged -> silent + Leaf inherits it via Middle -> silent (recursion) + Fixed narrows `outline` to a keyed range -> silent, on its own merits + Broken widens a clean slot to an ambiguous + range, so IT introduces the problem -> reported + Tagged gets `tags` from a mixin, not is_a -> reported (see rustdoc) +license: https://creativecommons.org/publicdomain/zero/1.0/ +imports: + - linkml:types +prefixes: + inherit: https://w3id.org/linkml/examples/identity-inheritance/ + linkml: https://w3id.org/linkml/ +default_prefix: inherit +default_range: string + +classes: + Base: + attributes: + # Vertex declares no key, identifier or unique_keys: flagged here. + outline: + range: Vertex + multivalued: true + inlined_as_list: true + # Keyed declares a key: not flagged. + keyed: + range: Keyed + multivalued: true + inlined_as_list: true + + Middle: + is_a: Base + + Leaf: + is_a: Middle + + Fixed: + is_a: Base + slot_usage: + outline: + range: Keyed + + Broken: + is_a: Base + slot_usage: + keyed: + range: Vertex + + HasTagsMixin: + mixin: true + attributes: + tags: + range: string + multivalued: true + + Tagged: + mixins: + - HasTagsMixin + + Keyed: + attributes: + k: {range: string, key: true} + + Vertex: + attributes: + x: {range: float} + y: {range: float} diff --git a/src/runtime/tests/identity_lint.rs b/src/runtime/tests/identity_lint.rs index 5cc5a0e..da624c8 100644 --- a/src/runtime/tests/identity_lint.rs +++ b/src/runtime/tests/identity_lint.rs @@ -154,6 +154,39 @@ fn schema_lint_reports_both_classes_that_share_one_class_uri() { ); } +#[test] +fn schema_lint_reports_an_inherited_slot_only_where_it_is_introduced() { + // A flagged slot is inherited by every descendant, so reporting it per + // class buries the one place the author can fix it. Only the topmost + // flagged declarer is reported — but a subclass whose `slot_usage` changes + // the identity answer is judged on its own merits, in both directions. + let sv = schema_view("identity_inheritance.yaml"); + let subjects: Vec> = lint_element_identity(&sv) + .iter() + .map(|w| w.subject.clone()) + .collect(); + assert_eq!( + subjects, + vec![ + // introduces `outline`; Middle, Leaf and Broken inherit it unchanged + vec!["Base".to_string(), "outline".to_string()], + // widens an inherited clean slot into an ambiguous one: introduced here + vec!["Broken".to_string(), "keyed".to_string()], + // mixin-provided, reported both on the mixin and on its user + vec!["HasTagsMixin".to_string(), "tags".to_string()], + vec!["Tagged".to_string(), "tags".to_string()], + ], + "an inherited flagged slot must be reported once, at its introducing class" + ); + // Spelled out, because these are the cases a naive fix gets wrong: + for silent in ["Middle", "Leaf", "Fixed"] { + assert!( + !subjects.iter().any(|s| s[0] == silent), + "{silent} must stay silent, got {subjects:?}" + ); + } +} + #[test] fn schema_lint_warning_order_is_deterministic() { // A class's slots come from `ClassView::slots()`, which is HashMap-backed, From 2420aa0d71ab1a145827dd7b17b546abcee4568d Mon Sep 17 00:00:00 2001 From: ejsyx Date: Tue, 18 Aug 2026 16:56:00 +0200 Subject: [PATCH 22/58] feat(python): expose element-identity lint functions Add `lint_element_identity(schema_view)` and `lint_instance_identity(instance)` to the `_native` module, wrapping the runtime lints and reusing `validation_results_to_py`; add matching stubs to `_native.pyi`. Co-Authored-By: Claude Fable 5 --- .../python/linkml_runtime_rust/_native.pyi | 16 +++++++++ src/python/src/lib.rs | 35 +++++++++++++++++-- 2 files changed, 49 insertions(+), 2 deletions(-) diff --git a/src/python/python/linkml_runtime_rust/_native.pyi b/src/python/python/linkml_runtime_rust/_native.pyi index 6897745..87be927 100644 --- a/src/python/python/linkml_runtime_rust/_native.pyi +++ b/src/python/python/linkml_runtime_rust/_native.pyi @@ -5478,6 +5478,22 @@ def import_turtle(reader:typing.Any, schema_view:SchemaView, root_classes:typing Import RDF/Turtle into a streaming iterator of LinkML instances. """ +def lint_element_identity(schema_view:SchemaView) -> builtins.list[ValidationResult]: + r""" + Schema-level lint: warn for every multivalued inlined slot whose element + identity comes from nowhere. + + Warnings only — the schema stays usable. Results are deterministic: sorted + by subject, deduplicated across class URIs, and an inherited slot is + reported once, at the class that introduces the problem. + """ + +def lint_instance_identity(instance:LinkMLInstance) -> builtins.list[ValidationResult]: + r""" + Data-level lint: warn for every list whose elements repeat a declared + identity (key/identifier or ``unique_keys`` value). + """ + def load_json(source:typing.Any, sv:SchemaView, class_view:ClassView) -> tuple[typing.Optional[LinkMLInstance], builtins.list[ValidationResult]]: ... def load_yaml(source:typing.Any, sv:SchemaView, class_view:ClassView) -> tuple[typing.Optional[LinkMLInstance], builtins.list[ValidationResult]]: ... diff --git a/src/python/src/lib.rs b/src/python/src/lib.rs index 5fac483..e692741 100644 --- a/src/python/src/lib.rs +++ b/src/python/src/lib.rs @@ -4,8 +4,9 @@ use linkml_runtime::diff::{ }; use linkml_runtime::turtle::{turtle_to_string, TurtleOptions}; use linkml_runtime::{ - load_json_str, load_yaml_str, validate_issues, LinkMLInstance, LoadResult, NodeId, - ValidationProblemType, ValidationResult, ValidationSeverity, ValidationValue, + lint_element_identity, lint_instance_identity, load_json_str, load_yaml_str, validate_issues, + LinkMLInstance, LoadResult, NodeId, ValidationProblemType, ValidationResult, + ValidationSeverity, ValidationValue, }; use linkml_schemaview::identifier::Identifier; use linkml_schemaview::io; @@ -744,6 +745,8 @@ pub fn runtime_module(m: &Bound<'_, PyModule>) -> PyResult<()> { m.add_function(wrap_pyfunction!(load_json, m)?)?; m.add_function(wrap_pyfunction!(py_diff, m)?)?; m.add_function(wrap_pyfunction!(py_patch, m)?)?; + m.add_function(wrap_pyfunction!(py_lint_element_identity, m)?)?; + m.add_function(wrap_pyfunction!(py_lint_instance_identity, m)?)?; m.add_function(wrap_pyfunction!(py_import_turtle, m)?)?; m.add_function(wrap_pyfunction!(py_import_ntriples, m)?)?; m.add_function(wrap_pyfunction!(py_export_turtle, m)?)?; @@ -1571,6 +1574,34 @@ fn py_patch( Py::new(py, result) } +// ── Identity lints ────────────────────────────────────────────────────────── + +/// Schema-level lint: warn for every multivalued inlined slot whose element +/// identity comes from nowhere. +/// +/// Warnings only — the schema stays usable. Results are deterministic: sorted +/// by subject, deduplicated across class URIs, and an inherited slot is +/// reported once, at the class that introduces the problem. +#[cfg_attr(feature = "stubgen", gen_stub_pyfunction)] +#[pyfunction(name = "lint_element_identity")] +fn py_lint_element_identity( + py: Python<'_>, + schema_view: &PySchemaView, +) -> PyResult>> { + validation_results_to_py(py, lint_element_identity(schema_view.as_rust())) +} + +/// Data-level lint: warn for every list whose elements repeat a declared +/// identity (key/identifier or ``unique_keys`` value). +#[cfg_attr(feature = "stubgen", gen_stub_pyfunction)] +#[pyfunction(name = "lint_instance_identity")] +fn py_lint_instance_identity( + py: Python<'_>, + instance: &PyLinkMLInstance, +) -> PyResult>> { + validation_results_to_py(py, lint_instance_identity(&instance.value)) +} + // ── RDF import/export ─────────────────────────────────────────────────────── /// Streaming iterator over harvested LinkML instances. From 181c343a4f2bd1544bb1c232b328fe638d72eb9b Mon Sep 17 00:00:00 2001 From: ejsyx Date: Tue, 18 Aug 2026 17:13:42 +0200 Subject: [PATCH 23/58] docs(runtime): document opaque annotation and unique_keys path segments Delta's path rustdoc now spells out how a unique_keys-derived segment is encoded (bare value for a single-slot key, JSON array in unique_key_slots order for a composite key), and diff's rustdoc states that an opaque slot stops recursion and yields one whole-value Update at the slot path. Co-Authored-By: Claude Fable 5 --- src/runtime/src/diff.rs | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/src/runtime/src/diff.rs b/src/runtime/src/diff.rs index 6fe079a..11c19d0 100644 --- a/src/runtime/src/diff.rs +++ b/src/runtime/src/diff.rs @@ -122,6 +122,10 @@ pub enum DeltaOp { /// elements do not all carry a *unique* identity label are addressed by numeric /// index instead. /// +/// For a `unique_keys`-derived segment, a single-slot key contributes the bare +/// value of that slot, while a composite key contributes the JSON array encoding +/// of the values in `unique_key_slots` order, e.g. `["Emergency","02/111.11.11"]`. +/// /// Operations are expressed jointly via [`Delta::op`], `old`, and `new`: /// /// | `op` | `old` | `new` | Description | @@ -193,6 +197,10 @@ impl DiffOptions { /// - X → missing (object slot): ignored by default; `Update` to null when `treat_missing_as_null`. /// - X → missing (mapping key): ignored by default; `Remove` when `treat_missing_as_null`. /// - X → missing (list element): always `Remove` (lists are positional/complete). +/// +/// Slots annotated `diff.linkml.io/opaque` stop all recursion: any change at or +/// below the slot is described as a single whole-value `Update` at the slot +/// path. See [`OPAQUE_ANNOTATION`]. pub fn diff(source: &LinkMLInstance, target: &LinkMLInstance, opts: DiffOptions) -> Vec { fn inner( path: &mut Vec, From 3b2273d451f97e50b506341ab5ef5a1d76dcd1e2 Mon Sep 17 00:00:00 2001 From: ejsyx Date: Wed, 19 Aug 2026 08:37:39 +0200 Subject: [PATCH 24/58] =?UTF-8?q?fix(runtime):=20keep=20diff=E2=86=92patch?= =?UTF-8?q?=20round=20trip=20when=20only=20the=20target=20loses=20identity?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `diff` decided keyed matching from BOTH lists, while `patch` decides how to address a list from the ONE list in front of it. When the source list was label-addressed but the target repeated (or lacked) a label, diff emitted positional segments that patch, resolving the source by label only, refused: `patch(a, diff(a, b)) == b` broke and the edit was dropped. Emit one whole-slot `Update` for that quadrant instead — "this list stopped having coherent element identity" is honestly a whole-value change. All other quadrants keep today's behaviour. The one-list predicate is now shared: `list_is_keyed_shaped` backs both diff's fallback and `resolve_list_index`, so they cannot drift apart. Co-Authored-By: Claude Fable 5 --- src/runtime/src/diff.rs | 44 ++++++++++++++++-- src/runtime/tests/diff_unique_keys.rs | 67 +++++++++++++++++++++++++++ 2 files changed, 106 insertions(+), 5 deletions(-) diff --git a/src/runtime/src/diff.rs b/src/runtime/src/diff.rs index 11c19d0..ea2239a 100644 --- a/src/runtime/src/diff.rs +++ b/src/runtime/src/diff.rs @@ -101,6 +101,21 @@ where elements.iter().filter_map(label).all(|l| seen.insert(l)) } +/// Whether this one list is addressed by identity label: it is non-empty, +/// every element carries an identity label, and the labels are unique. +/// +/// This is the predicate that decides how a list is *addressed*, asked of a +/// single list. `diff` needs the same answer of both sides at once (a keyed +/// match needs identity on both), but every consumer that has only one list in +/// front of it — `patch`'s segment resolver, `navigate_path`, and diff's +/// keyed-source fallback — must agree, or a path one of them emits is a path +/// another cannot resolve. +pub(crate) fn list_is_keyed_shaped(values: &[LinkMLInstance]) -> bool { + !values.is_empty() + && values.iter().all(|v| element_identity_label(v).is_some()) + && labels_are_unique(values, element_identity_label) +} + /// Operation applied by a [`Delta`]. #[derive(Clone, Debug, PartialEq, Serialize, Deserialize)] #[serde(rename_all = "lowercase")] @@ -201,6 +216,13 @@ impl DiffOptions { /// Slots annotated `diff.linkml.io/opaque` stop all recursion: any change at or /// below the slot is described as a single whole-value `Update` at the slot /// path. See [`OPAQUE_ANNOTATION`]. +/// +/// Lists are matched by element identity when both sides carry unique identity +/// labels, and positionally otherwise — with one exception: when the *source* +/// list alone is label-addressed (the target repeats or lacks a label), the +/// change is described as a single whole-value `Update` at the list's path. +/// `patch` addresses a label-addressed list by label only, so positional +/// segments aimed at one could never be applied. pub fn diff(source: &LinkMLInstance, target: &LinkMLInstance, opts: DiffOptions) -> Vec { fn inner( path: &mut Vec, @@ -359,6 +381,22 @@ pub fn diff(source: &LinkMLInstance, target: &LinkMLInstance, opts: DiffOptions) path.pop(); } } + } else if list_is_keyed_shaped(sl) { + // The source alone is keyed-shaped: `patch` resolves such a + // list by label ONLY, so positional segments aimed at it are + // unappliable by design and `patch(a, diff(a, b))` would + // refuse the very deltas we just emitted. What actually + // happened is honestly a whole-value change — this list + // stopped having coherent element identity — so say that, + // once, at the slot. + if !s.equals(t, opts.treat_missing_as_null) { + out.push(Delta { + path: path.clone(), + op: DeltaOp::Update, + old: Some(s.to_json()), + new: Some(t.to_json()), + }); + } } else { let max_len = std::cmp::max(sl.len(), tl.len()); for i in 0..max_len { @@ -625,11 +663,7 @@ fn resolve_list_index(values: &[LinkMLInstance], key: &str) -> Option { // labels (e.g. a year as unique key) unambiguous: they resolve as // labels, never as positions. let labels: Vec> = values.iter().map(element_identity_label).collect(); - let keyed_shaped = !values.is_empty() && labels.iter().all(|l| l.is_some()) && { - let mut seen = std::collections::HashSet::new(); - labels.iter().flatten().all(|l| seen.insert(l.clone())) - }; - if keyed_shaped { + if list_is_keyed_shaped(values) { return labels.iter().position(|l| l.as_deref() == Some(key)); } // Positional list: numeric index first (the segments diff produces for diff --git a/src/runtime/tests/diff_unique_keys.rs b/src/runtime/tests/diff_unique_keys.rs index 58aba98..709ed3e 100644 --- a/src/runtime/tests/diff_unique_keys.rs +++ b/src/runtime/tests/diff_unique_keys.rs @@ -390,3 +390,70 @@ fn patch_refuses_update_whose_label_matches_nothing() { assert_eq!(trace.failed, vec![delta.path.clone()]); assert!(patched.equals(&golden, true), "nothing may be appended"); } + +/// A second element carrying the same identity label as [`e`]. +fn e2() -> JsonValue { + json!({"phoneNumber": "09/000.00.00", "hasNumberFunction": "Emergency_Number"}) +} + +#[test] +fn keyed_source_to_duplicated_target_is_one_whole_slot_update() { + let f = fixture(); + // The source list is keyed-shaped, the target repeats a label. Positional + // segments against a keyed-shaped source are unappliable by design (patch + // resolves such a list by label only), so the honest description is: this + // list stopped having coherent element identity — one whole-slot Update. + let before = phones(vec![e(), n()]); + let after = phones(vec![e(), e2()]); + let deltas = diff2(&f, before.clone(), after.clone()); + let delta = only(&deltas); + assert_eq!(delta.path, vec!["hasPhoneNumber".to_string()]); + assert_eq!(delta.op, DeltaOp::Update); + assert_eq!(delta.old, Some(json!([e(), n()]))); + assert_eq!(delta.new, Some(json!([e(), e2()]))); + + let (patched, trace) = patch(&f.load(before), &deltas, PatchOptions::default()).unwrap(); + assert!(trace.failed.is_empty(), "{:?}", trace.failed); + assert!( + patched.equals(&f.load(after), true), + "patch(a, diff(a,b)) must equal b: {}", + patched.to_json() + ); +} + +#[test] +fn duplicated_source_to_keyed_target_stays_positional_and_round_trips() { + let f = fixture(); + // The mirror image: the source is NOT keyed-shaped, so numeric segments + // are exactly what patch resolves against it. Keep positional deltas. + // + // The second element differs from its target only in the key-bearing slot, + // so the whole edit is one delta. A multi-delta variant of this case is + // order-dependent for reasons unrelated to the source-keyed fallback (see + // the branch report): once the key edit lands the list becomes + // keyed-shaped, and any numeric segment still queued stops resolving. + let dup = json!({"phoneNumber": "09/241.25.03", "hasNumberFunction": "Emergency_Number"}); + let before = phones(vec![e(), dup]); + let after = phones(vec![e(), n()]); + let deltas = diff2(&f, before.clone(), after.clone()); + assert!(!deltas.is_empty(), "expected positional deltas"); + for d in &deltas { + assert_eq!( + d.path[0], "hasPhoneNumber", + "unexpected delta path: {:?}", + d.path + ); + assert!( + d.path.len() > 1 && d.path[1].parse::().is_ok(), + "a non-keyed-shaped source keeps numeric segments: {:?}", + d.path + ); + } + let (patched, trace) = patch(&f.load(before), &deltas, PatchOptions::default()).unwrap(); + assert!(trace.failed.is_empty(), "{:?}", trace.failed); + assert!( + patched.equals(&f.load(after), true), + "patch(a, diff(a,b)) must equal b: {}", + patched.to_json() + ); +} From 8548fb5d662aa07d68096f3992ce0977c951b758 Mon Sep 17 00:00:00 2001 From: ejsyx Date: Wed, 19 Aug 2026 08:40:07 +0200 Subject: [PATCH 25/58] fix(runtime): navigate_path resolves list segments by the shared rule MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `navigate_path` ignored identity labels entirely and tried the numeric index first, so a `unique_keys` delta segment was invisible to it and a numeric segment silently returned the element at that *position* even when a sibling's identity label was that same number — the wrong element, reported as success. Both list resolvers are now one function, `diff::resolve_list_segment`: what diff emits, patch applies and navigate finds by construction, not by three implementations agreeing. Co-Authored-By: Claude Fable 5 --- .../python/linkml_runtime_rust/_native.pyi | 7 +- src/python/src/lib.rs | 7 +- src/runtime/src/diff.rs | 11 ++- src/runtime/src/lib.rs | 52 +++-------- src/runtime/tests/equality.rs | 18 ++-- src/runtime/tests/navigate.rs | 89 ++++++++++++++++++- 6 files changed, 134 insertions(+), 50 deletions(-) diff --git a/src/python/python/linkml_runtime_rust/_native.pyi b/src/python/python/linkml_runtime_rust/_native.pyi index 87be927..68b7869 100644 --- a/src/python/python/linkml_runtime_rust/_native.pyi +++ b/src/python/python/linkml_runtime_rust/_native.pyi @@ -2789,7 +2789,12 @@ class LinkMLInstance: def __getitem__(self, key:typing.Any) -> LinkMLInstance: ... def navigate(self, path:typing.Any) -> typing.Optional[LinkMLInstance]: r""" - Navigate by a path of strings (map keys or list indices). + Navigate by a path of strings: slot names, mapping keys, and — for + lists — element identity labels (identifier/key or `unique_keys` value) + when the list carries unique labels, numeric indices otherwise. This is + the same addressing `diff` emits and `patch` applies, so delta paths are + navigable; a numeric segment aimed at a label-addressed list resolves to + nothing rather than to that position. Returns a new LinkMLInstance if found, otherwise None. """ def keys(self) -> builtins.list[builtins.str]: ... diff --git a/src/python/src/lib.rs b/src/python/src/lib.rs index e692741..5704eaa 100644 --- a/src/python/src/lib.rs +++ b/src/python/src/lib.rs @@ -1228,7 +1228,12 @@ impl PyLinkMLInstance { } } - /// Navigate by a path of strings (map keys or list indices). + /// Navigate by a path of strings: slot names, mapping keys, and — for + /// lists — element identity labels (identifier/key or `unique_keys` value) + /// when the list carries unique labels, numeric indices otherwise. This is + /// the same addressing `diff` emits and `patch` applies, so delta paths are + /// navigable; a numeric segment aimed at a label-addressed list resolves to + /// nothing rather than to that position. /// Returns a new LinkMLInstance if found, otherwise None. #[pyo3(name = "navigate")] fn py_navigate<'py>( diff --git a/src/runtime/src/diff.rs b/src/runtime/src/diff.rs index ea2239a..426d5a1 100644 --- a/src/runtime/src/diff.rs +++ b/src/runtime/src/diff.rs @@ -655,7 +655,14 @@ fn replace_child_subtree( } } -fn resolve_list_index(values: &[LinkMLInstance], key: &str) -> Option { +/// Resolve one path segment against a list, to the index of the element it +/// addresses. +/// +/// The single rule every consumer of a delta path shares — `patch` when it +/// applies one, [`crate::LinkMLInstance::navigate_path`] when it follows one. +/// Keeping it in one function is what makes "diff emits it, patch applies it, +/// navigate finds it" true by construction rather than by three coincidences. +pub(crate) fn resolve_list_segment(values: &[LinkMLInstance], key: &str) -> Option { // A list whose elements all carry unique identity labels is addressed by // label ONLY: diff emits label segments for exactly these lists, and a // numeric segment aimed at one (a stale positional patch) would be a @@ -1090,7 +1097,7 @@ fn apply_delta_list( return Ok(false); } let key = &path[0]; - let idx_opt = resolve_list_index(values, key); + let idx_opt = resolve_list_segment(values, key); if path.len() == 1 { let value = newv.cloned().unwrap_or(JsonValue::Null); let slot_clone = slot.clone(); diff --git a/src/runtime/src/lib.rs b/src/runtime/src/lib.rs index 31fd5e3..5a4da7f 100644 --- a/src/runtime/src/lib.rs +++ b/src/runtime/src/lib.rs @@ -544,8 +544,19 @@ impl LinkMLInstance { } } - /// Navigate the value by a path of strings, where each element is either - /// a dictionary key (for maps) or a list index (for lists). + /// Navigate the value by a path of strings, where each segment is a slot + /// name, a mapping key, or — for lists — whatever addresses an element: + /// its identity label (identifier/key slot value, or a value derived from + /// the range class's `unique_keys`) for a list whose elements all carry + /// unique labels, and a numeric index otherwise. + /// + /// List segments resolve through the same rule [`crate::diff`] emits and + /// [`crate::patch`] applies, so a delta path is navigable by construction. + /// In particular a numeric segment aimed at a label-addressed list resolves + /// to nothing rather than to that position: when a label happens to be + /// `"0"`, position and label name different elements, and guessing which + /// one was meant is exactly the wrong-element hazard. + /// /// Returns `Some(&LinkMLInstance)` if the full path can be resolved, otherwise `None`. pub fn navigate_path(&self, path: I) -> Option<&LinkMLInstance> where @@ -560,42 +571,7 @@ impl LinkMLInstance { current = values.get(key)?; } LinkMLInstance::List { values, .. } => { - // Support either numeric index or identifier/key-based selection - if let Ok(idx) = key.parse::() { - current = values.get(idx)?; - } else { - // Attempt identifier-based lookup for object elements - let mut found: Option<&LinkMLInstance> = None; - for v in values.iter() { - if let LinkMLInstance::Object { - values: mv, class, .. - } = v - { - if let Some(id_slot) = class.key_or_identifier_slot() { - if let Some(LinkMLInstance::Scalar { value, .. }) = - mv.get(&id_slot.name) - { - match value { - JsonValue::String(sv) => { - if sv == key { - found = Some(v); - break; - } - } - other => { - let sv = other.to_string(); - if sv == key { - found = Some(v); - break; - } - } - } - } - } - } - } - current = found?; - } + current = values.get(diff::resolve_list_segment(values, key)?)?; } LinkMLInstance::Mapping { values, .. } => { current = values.get(key)?; diff --git a/src/runtime/tests/equality.rs b/src/runtime/tests/equality.rs index 3418a63..6aa6d2e 100644 --- a/src/runtime/tests/equality.rs +++ b/src/runtime/tests/equality.rs @@ -66,8 +66,10 @@ objects: "#; let v1 = load_yaml_doc(doc_with_null, &sv, &container, &conv); let v2 = load_yaml_doc(doc_without_slot, &sv, &container, &conv); - let p1 = v1.navigate_path(["objects", "0"]).unwrap(); - let p2 = v2.navigate_path(["objects", "0"]).unwrap(); + // `objects` is a list of NamedThing, which declares an `id` identifier: + // it is addressed by that label, never by position. + let p1 = v1.navigate_path(["objects", "P:1"]).unwrap(); + let p2 = v2.navigate_path(["objects", "P:1"]).unwrap(); assert!( p1.equals(p2, true), "Person with null assignment should equal omission" @@ -112,8 +114,10 @@ objects: "#; let v1 = load_yaml_doc(doc_a, &sv, &container, &conv); let v2 = load_yaml_doc(doc_b, &sv, &container, &conv); - let p1 = v1.navigate_path(["objects", "0"]).unwrap(); - let p2 = v2.navigate_path(["objects", "0"]).unwrap(); + // `objects` is a list of NamedThing, which declares an `id` identifier: + // it is addressed by that label, never by position. + let p1 = v1.navigate_path(["objects", "P:1"]).unwrap(); + let p2 = v2.navigate_path(["objects", "P:1"]).unwrap(); assert!(matches!(p1, LinkMLInstance::Object { .. })); assert!(matches!(p2, LinkMLInstance::Object { .. })); assert!(!p1.equals(p2, true), "List order must affect equality"); @@ -196,9 +200,9 @@ objects: let v1 = load_yaml_doc(doc1, &sv, &container, &conv); let v2 = load_yaml_doc(doc2, &sv, &container, &conv); let v3 = load_yaml_doc(doc3, &sv, &container, &conv); - let g1 = v1.navigate_path(["objects", "0", "gender"]).unwrap(); - let g2 = v2.navigate_path(["objects", "0", "gender"]).unwrap(); - let g3 = v3.navigate_path(["objects", "0", "gender"]).unwrap(); + let g1 = v1.navigate_path(["objects", "P:1", "gender"]).unwrap(); + let g2 = v2.navigate_path(["objects", "P:2", "gender"]).unwrap(); + let g3 = v3.navigate_path(["objects", "P:3", "gender"]).unwrap(); assert!(g1.equals(g2, true)); assert!(!g1.equals(g3, true)); } diff --git a/src/runtime/tests/navigate.rs b/src/runtime/tests/navigate.rs index 5c04155..0ee14a0 100644 --- a/src/runtime/tests/navigate.rs +++ b/src/runtime/tests/navigate.rs @@ -35,16 +35,103 @@ fn navigate_basic() { match &v { linkml_runtime::LinkMLInstance::Object { values, .. } => { assert!(values.contains_key("objects")); + // `objects` is inlined as a list of NamedThing, which declares an + // `id` identifier: it is addressed by that label, not by position. + // `has_medical_history` declares no identity, so it stays numeric. let inner = v.navigate_path([ "objects", - "2", + "P:002", "has_medical_history", "0", "diagnosis", "name", ]); assert!(inner.is_some()); + assert!( + v.navigate_path(["objects", "2"]).is_none(), + "a numeric segment must not address a label-addressed list" + ); } _ => panic!("expected map at root"), } } + +/// The identity fixture, loaded as a `Service` instance. +fn service(data: serde_json::Value) -> linkml_runtime::LinkMLInstance { + let schema = from_yaml(Path::new(&info_path("identity.yaml"))).unwrap(); + let mut sv = SchemaView::new(); + sv.add_schema(schema.clone()).unwrap(); + let conv = converter_from_schema(&schema); + let class = sv + .get_class(&Identifier::new("Service"), &conv) + .unwrap() + .expect("class not found"); + linkml_runtime::load_json_str(&data.to_string(), &sv, &class, &conv) + .unwrap() + .into_instance() + .unwrap() +} + +fn scalar(v: Option<&linkml_runtime::LinkMLInstance>) -> Option { + v.map(|v| v.to_json()) +} + +#[test] +fn navigate_resolves_a_unique_keys_segment() { + // `diff` addresses this list by its `unique_keys`-derived label, so a + // delta path that names one has to be navigable. + let v = service(serde_json::json!({"name": "svc", "hasPhoneNumber": [ + {"phoneNumber": "09/241.25.00", "hasNumberFunction": "Emergency_Number"}, + {"phoneNumber": "09/241.25.03", "hasNumberFunction": "Non_Urgent_Communication"}]})); + assert_eq!( + scalar(v.navigate_path(["hasPhoneNumber", "Non_Urgent_Communication", "phoneNumber"])), + Some(serde_json::json!("09/241.25.03")) + ); +} + +#[test] +fn navigate_resolves_a_composite_unique_key_segment() { + let v = service(serde_json::json!({"name": "svc", "contacts": [ + {"kind": "Emergency", "phone": "02/111.11.11", "note": "first"}, + {"kind": "Operator", "phone": "02/333.33.33", "note": "second"}]})); + assert_eq!( + scalar(v.navigate_path(["contacts", r#"["Emergency","02/111.11.11"]"#, "note"])), + Some(serde_json::json!("first")) + ); +} + +#[test] +fn navigate_refuses_a_numeric_segment_into_a_label_addressed_list() { + // `patch` refuses this ("report, never guess"); navigating must not + // silently hand back a different element than the one the path names. + let v = service(serde_json::json!({"name": "svc", "hasPhoneNumber": [ + {"phoneNumber": "09/241.25.00", "hasNumberFunction": "Emergency_Number"}, + {"phoneNumber": "09/241.25.03", "hasNumberFunction": "Non_Urgent_Communication"}]})); + assert!(v.navigate_path(["hasPhoneNumber", "0"]).is_none()); +} + +#[test] +fn navigate_prefers_the_label_over_the_index_when_a_label_is_numeric() { + // `lang` is a key slot, and one element's key IS "0". Index-first + // navigation hands back the element at position 0 — the wrong one. + let v = service(serde_json::json!({"name": "svc", "labelList": [ + {"lang": "nl", "text": "dutch"}, + {"lang": "0", "text": "zero"}]})); + assert_eq!( + scalar(v.navigate_path(["labelList", "0", "text"])), + Some(serde_json::json!("zero")) + ); +} + +#[test] +fn navigate_still_indexes_a_positional_list_numerically() { + // PlainPhoneNumber declares no identity: diff emits numeric segments for + // this list, and they must keep resolving. + let v = service(serde_json::json!({"name": "svc", "plainPhoneNumber": [ + {"phoneNumber": "09/241.25.00", "hasNumberFunction": "Emergency_Number"}, + {"phoneNumber": "09/241.25.03", "hasNumberFunction": "Non_Urgent_Communication"}]})); + assert_eq!( + scalar(v.navigate_path(["plainPhoneNumber", "1", "phoneNumber"])), + Some(serde_json::json!("09/241.25.03")) + ); +} From fc4a092b818c44586cd14f4b9a9393401700caab Mon Sep 17 00:00:00 2001 From: ejsyx Date: Wed, 19 Aug 2026 08:42:49 +0200 Subject: [PATCH 26/58] feat(runtime): warn when a class offers several unique_keys as identity MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Element identity derived from `unique_keys` comes from the name-sorted first entry: the metamodel does not preserve declaration order and delta paths have to be stable. A class declaring two entries therefore has an identity chosen alphabetically, and adding an earlier-sorting one silently re-addresses every delta path for every slot ranged on it — schema evolution with no signal. `lint_element_identity` now warns for such a class, naming the load-bearing entry. It flags slots the identity-less rule *passes*, so no existing warning changes; a class with a key/identifier slot is exempt (the key outranks every `unique_keys`, so none of them is load-bearing). The introducing-class rule now applies to both lint rules. `Delta`'s path documentation states the selection rule and the hazard. Co-Authored-By: Claude Fable 5 --- src/runtime/src/diff.rs | 5 + src/runtime/src/identity_lint.rs | 114 ++++++++++++++++-- .../data/identity_multiple_unique_keys.yaml | 79 ++++++++++++ src/runtime/tests/identity_lint.rs | 59 +++++++++ 4 files changed, 250 insertions(+), 7 deletions(-) create mode 100644 src/runtime/tests/data/identity_multiple_unique_keys.yaml diff --git a/src/runtime/src/diff.rs b/src/runtime/src/diff.rs index 426d5a1..8050699 100644 --- a/src/runtime/src/diff.rs +++ b/src/runtime/src/diff.rs @@ -140,6 +140,11 @@ pub enum DeltaOp { /// For a `unique_keys`-derived segment, a single-slot key contributes the bare /// value of that slot, while a composite key contributes the JSON array encoding /// of the values in `unique_key_slots` order, e.g. `["Emergency","02/111.11.11"]`. +/// When the range class declares several `unique_keys`, the identity comes from +/// the name-sorted first entry with a non-empty slot list — the metamodel does +/// not preserve declaration order, and paths have to be stable. The hazard is +/// schema evolution: adding an earlier-sorting entry re-addresses every path for +/// that slot, so [`crate::lint_element_identity`] warns about any such class. /// /// Operations are expressed jointly via [`Delta::op`], `old`, and `new`: /// diff --git a/src/runtime/src/identity_lint.rs b/src/runtime/src/identity_lint.rs index 1753349..a25986a 100644 --- a/src/runtime/src/identity_lint.rs +++ b/src/runtime/src/identity_lint.rs @@ -23,6 +23,13 @@ //! removing the slot from diff's scope; `opaque` silences it by answering the //! question with "nowhere — replace the value as a whole". //! +//! A second, narrower question is asked of slots that pass: **which** of the +//! range class's `unique_keys` provides the identity? Only the name-sorted +//! first entry does (declaration order is not preserved by the metamodel), so a +//! class declaring several has an alphabetically-decided identity and adding an +//! earlier-sorting entry silently re-addresses every delta path for every slot +//! ranged on it. That is warned about too, naming the load-bearing entry. +//! //! Warnings are reported at the class that **introduces** the slot: a flagged //! slot inherited unchanged by a descendant is not repeated there, since the //! declaration the author would edit lives on the ancestor. @@ -62,6 +69,52 @@ fn slot_lacks_element_identity(slot: &SlotView) -> bool { true } +/// The `unique_keys` entries a class offers as element identity, name-sorted. +/// +/// An entry with no slots names nothing and can never be load-bearing, so it is +/// not a candidate and does not make an otherwise-single-entry class ambiguous. +fn identity_unique_key_names(rc: &ClassView) -> Vec { + rc.unique_keys() + .into_iter() + .filter(|(_, uk)| !uk.unique_key_slots.is_empty()) + .map(|(name, _)| name) + .collect() +} + +/// Whether this slot's element identity is *ambiguous* rather than absent: its +/// range class offers several `unique_keys` entries to derive it from, and only +/// the name-sorted first is load-bearing. +/// +/// Returns the range class name and the candidate entry names (sorted, so the +/// load-bearing one is first). This flags slots the identity-less rule passes: +/// the identity exists, but which of the declarations provides it was decided +/// alphabetically rather than by the author. +/// +/// A class with a `key`/`identifier` slot is not ambiguous however many +/// `unique_keys` it declares: the key outranks them all, so none of them is +/// load-bearing and adding one changes nothing. +fn slot_has_ambiguous_unique_keys(slot: &SlotView) -> Option<(String, Vec)> { + use linkml_schemaview::slotview::{SlotContainerMode, SlotInlineMode}; + if slot.determine_slot_container_mode() != SlotContainerMode::List { + return None; + } + if slot.determine_slot_inline_mode() == SlotInlineMode::Reference { + return None; // elements are references, not inlined + } + if slot_is_opaque(slot) || slot_is_ignored(slot) { + return None; // no per-element delta paths to re-address + } + let rc = slot.get_range_class()?; + if rc.key_or_identifier_slot().is_some() { + return None; // the key outranks unique_keys entirely + } + let names = identity_unique_key_names(&rc); + if names.len() < 2 { + return None; + } + Some((rc.name().to_string(), names)) +} + /// Whether `class` is where a flagged slot should be reported, rather than an /// ancestor it merely inherits the problem from. /// @@ -75,18 +128,49 @@ fn slot_lacks_element_identity(slot: &SlotView) -> bool { /// the class using the mixin as well as on the mixin itself: a mixin can be /// applied to unrelated classes, so there is no single owning declaration to /// point at, and chasing one is not worth the complexity. -fn introduces_flagged_slot(class: &ClassView, slot_name: &str) -> bool { +fn introduces_flagged_slot(class: &ClassView, slot_name: &str, flagged: F) -> bool +where + F: Fn(&SlotView) -> bool, +{ let Ok(Some(parent)) = class.parent_class() else { return true; // no is_a parent: this class is the declarer }; let Some(parent_slot) = parent.slot(&Identifier::new(slot_name)) else { return true; // the parent does not have it: introduced here }; - !slot_lacks_element_identity(&parent_slot) + !flagged(&parent_slot) +} + +/// The warning text for a range class offering several `unique_keys`. +/// +/// `names` is name-sorted, so `names[0]` is the load-bearing entry. +fn ambiguous_unique_keys_detail( + class_name: &str, + slot_name: &str, + range_class: &str, + names: &[String], +) -> String { + let quoted: Vec = names.iter().map(|n| format!("'{n}'")).collect(); + format!( + "elements of '{}.{}' take their identity from the unique_keys of \ + element class '{}', which declares {}: {}. Only {} is load-bearing — \ + the metamodel does not preserve declaration order, so the name-sorted \ + first entry is used, and every delta path for this slot is addressed \ + by it. Adding an earlier-sorting entry silently re-addresses them all. \ + Keep one entry, or rename deliberately.", + class_name, + slot_name, + range_class, + names.len(), + quoted.join(", "), + quoted.first().map(String::as_str).unwrap_or("none"), + ) } /// Schema-level lint: warn for every multivalued inlined slot whose element -/// identity comes from nowhere. Warnings only — the schema stays usable. +/// identity comes from nowhere, and for every one whose identity is derived +/// from a class offering more than one `unique_keys` entry to derive it from. +/// Warnings only — the schema stays usable. pub fn lint_element_identity(sv: &SchemaView) -> Vec { let mut sink = ValidationResultSink::default(); let conv = sv.converter(); @@ -111,13 +195,29 @@ pub fn lint_element_identity(sv: &SchemaView) -> Vec { continue; } for slot in class.slots() { + // Report at the class that introduces the slot: repeating an + // inherited warning on every descendant buries the one declaration + // the author would actually edit. Applies to both rules below. if !slot_lacks_element_identity(slot) { + if let Some((rc_name, names)) = slot_has_ambiguous_unique_keys(slot) { + if introduces_flagged_slot(&class, &slot.name, |s| { + slot_has_ambiguous_unique_keys(s).is_some() + }) { + sink.push_warning( + ValidationProblemType::AmbiguousElementIdentity, + vec![class.name().to_string(), slot.name.clone()], + ambiguous_unique_keys_detail( + class.name(), + &slot.name, + &rc_name, + &names, + ), + ); + } + } continue; } - // Report at the class that introduces the slot: repeating an - // inherited warning on every descendant buries the one declaration - // the author would actually edit. - if !introduces_flagged_slot(&class, &slot.name) { + if !introduces_flagged_slot(&class, &slot.name, slot_lacks_element_identity) { continue; } let range_class = slot.get_range_class(); diff --git a/src/runtime/tests/data/identity_multiple_unique_keys.yaml b/src/runtime/tests/data/identity_multiple_unique_keys.yaml new file mode 100644 index 0000000..a7d01ac --- /dev/null +++ b/src/runtime/tests/data/identity_multiple_unique_keys.yaml @@ -0,0 +1,79 @@ +id: https://w3id.org/linkml/examples/identity_multiple_unique_keys +name: identity_multiple_unique_keys +description: |- + Element identity derived from `unique_keys` when a class declares more than + one entry: only the name-sorted first is load-bearing, so the schema author + has to be told which one every delta path is addressed by. +license: https://creativecommons.org/publicdomain/zero/1.0/ +imports: + - linkml:types +prefixes: + identity: https://w3id.org/linkml/examples/identity_multiple_unique_keys/ + linkml: https://w3id.org/linkml/ +default_prefix: identity +default_range: string + +classes: + Catalog: + attributes: + # two candidate identities: the lint must name the load-bearing one + badges: + range: Badge + multivalued: true + inlined_as_list: true + # exactly one entry: unambiguous, stays silent + tickets: + range: Ticket + multivalued: true + inlined_as_list: true + # a key slot outranks unique_keys entirely: nothing to disambiguate + seats: + range: Seat + multivalued: true + inlined_as_list: true + # identity declared as "nowhere": no per-element paths to re-address + archivedBadges: + range: Badge + multivalued: true + inlined_as_list: true + annotations: + diff.linkml.io/opaque: true + # outside diff's scope entirely + draftBadges: + range: Badge + multivalued: true + inlined_as_list: true + annotations: + diff.linkml.io/ignore: true + + # inherits `badges` unchanged: reported at Catalog, not here + SubCatalog: + is_a: Catalog + + Badge: + unique_keys: + by_code: + unique_key_slots: [code] + zz_by_label: + unique_key_slots: [label] + attributes: + code: {range: string, required: true} + label: {range: string, required: true} + + Ticket: + unique_keys: + by_serial: + unique_key_slots: [serial] + attributes: + serial: {range: string, required: true} + + Seat: + unique_keys: + by_number: + unique_key_slots: [number] + by_row: + unique_key_slots: [row] + attributes: + seatId: {range: string, key: true} + row: {range: string, required: true} + number: {range: string, required: true} diff --git a/src/runtime/tests/identity_lint.rs b/src/runtime/tests/identity_lint.rs index da624c8..81e4559 100644 --- a/src/runtime/tests/identity_lint.rs +++ b/src/runtime/tests/identity_lint.rs @@ -272,3 +272,62 @@ fn data_lint_does_not_let_opaque_suppress_a_schema_constraint() { assert_eq!(warnings.len(), 1, "{warnings:#?}"); assert_eq!(warnings[0].subject, vec!["archivedContacts".to_string()]); } + +#[test] +fn schema_lint_names_the_load_bearing_unique_key_when_a_class_declares_several() { + // Declaration order is not preserved by the metamodel, so element identity + // is derived from the name-sorted first `unique_keys` entry. A class with + // two entries therefore has a silent, alphabetically-decided identity: + // adding an earlier-sorting entry re-addresses every delta path for every + // slot ranged on it, with nothing to notice it by. + let sv = schema_view("identity_multiple_unique_keys.yaml"); + let warnings = lint_element_identity(&sv); + let subjects: Vec> = warnings.iter().map(|w| w.subject.clone()).collect(); + assert_eq!( + subjects, + vec![vec!["Catalog".to_string(), "badges".to_string()]], + "only the ambiguous slot is flagged, once, at its introducing class: \ + {warnings:#?}" + ); + let w = &warnings[0]; + assert_eq!( + w.problem_type, + ValidationProblemType::AmbiguousElementIdentity + ); + assert!(!w.severity.is_error(), "the linter warns, never errors"); + assert!( + w.detail.contains("'by_code'") && w.detail.contains("'zz_by_label'"), + "the warning must name every candidate: {}", + w.detail + ); + assert!( + w.detail.contains("Badge"), + "the warning must name the element class the entries live on: {}", + w.detail + ); + let winner = w.detail.find("'by_code'").unwrap_or(usize::MAX); + let other = w.detail.find("'zz_by_label'").unwrap_or(0); + assert!( + winner < other, + "the load-bearing entry must be named first, and identified as such: {}", + w.detail + ); +} + +#[test] +fn schema_lint_leaves_single_entry_and_keyed_classes_alone() { + // Guard rails for the ambiguity rule: one entry is unambiguous, a key slot + // outranks `unique_keys` so the entries are not load-bearing at all, and + // `opaque` / `ignore` mean there are no per-element paths to re-address. + let sv = schema_view("identity_multiple_unique_keys.yaml"); + let flagged: Vec = lint_element_identity(&sv) + .iter() + .map(|w| w.subject[1].clone()) + .collect(); + for silent in ["tickets", "seats", "archivedBadges", "draftBadges"] { + assert!( + !flagged.contains(&silent.to_string()), + "{silent} must stay silent, got {flagged:?}" + ); + } +} From 58c9e5de1a84780dcd2d39ec627a18869a069ee0 Mon Sep 17 00:00:00 2001 From: ejsyx Date: Wed, 19 Aug 2026 08:45:23 +0200 Subject: [PATCH 27/58] fix(tools): report the shared problem-type label in the CLI's JSON MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The `--lint-identity --output json` payload spelled `type` with `Debug` formatting (`AmbiguousElementIdentity`) while the Python binding reported `ambiguous_element_identity` for the same value: two machine surfaces, two spellings, and a variant rename would have silently changed the CLI contract. `ValidationProblemType::label()` now lives next to the enum and both surfaces call it. The CLI's `type` becomes snake_case — the flag shipped inside this branch, so no external consumer exists. Co-Authored-By: Claude Fable 5 --- src/python/src/lib.rs | 20 +----- src/runtime/src/lib.rs | 22 ++++++ src/tools/src/bin/linkml_schema_validate.rs | 5 +- src/tools/tests/schema_validate.rs | 75 +++++++++++++++++++++ 4 files changed, 104 insertions(+), 18 deletions(-) diff --git a/src/python/src/lib.rs b/src/python/src/lib.rs index 5704eaa..6168c73 100644 --- a/src/python/src/lib.rs +++ b/src/python/src/lib.rs @@ -5,8 +5,7 @@ use linkml_runtime::diff::{ use linkml_runtime::turtle::{turtle_to_string, TurtleOptions}; use linkml_runtime::{ lint_element_identity, lint_instance_identity, load_json_str, load_yaml_str, validate_issues, - LinkMLInstance, LoadResult, NodeId, ValidationProblemType, ValidationResult, - ValidationSeverity, ValidationValue, + LinkMLInstance, LoadResult, NodeId, ValidationResult, ValidationSeverity, ValidationValue, }; use linkml_schemaview::identifier::Identifier; use linkml_schemaview::io; @@ -877,7 +876,7 @@ impl From for PyValidationResult { impl PyValidationResult { #[getter] fn r#type(&self) -> String { - validation_problem_type_label(&self.inner.problem_type).to_string() + self.inner.problem_type.label().to_string() } #[getter] @@ -918,7 +917,7 @@ impl PyValidationResult { fn __repr__(&self) -> PyResult { Ok(format!( "ValidationResult(type='{}', severity='{}', subject={:?}, detail={})", - validation_problem_type_label(&self.inner.problem_type), + self.inner.problem_type.label(), severity_label(&self.inner.severity), self.inner.subject, self.inner.detail @@ -1048,19 +1047,6 @@ fn validation_value_to_py(py: Python<'_>, value: &ValidationValue) -> PyResult

&'static str { - match problem_type { - ValidationProblemType::UndeclaredSlot => "undeclared_slot", - ValidationProblemType::InapplicableSlot => "inapplicable_slot", - ValidationProblemType::MissingSlotValue => "missing_slot_value", - ValidationProblemType::SlotRangeViolation => "slot_range_violation", - ValidationProblemType::MaxCountViolation => "max_count_violation", - ValidationProblemType::ParsingError => "parsing_error", - ValidationProblemType::AmbiguousElementIdentity => "ambiguous_element_identity", - ValidationProblemType::DuplicateElementIdentity => "duplicate_element_identity", - } -} - fn severity_label(severity: &ValidationSeverity) -> &'static str { match severity { ValidationSeverity::Fatal => "fatal", diff --git a/src/runtime/src/lib.rs b/src/runtime/src/lib.rs index 5a4da7f..abddb0c 100644 --- a/src/runtime/src/lib.rs +++ b/src/runtime/src/lib.rs @@ -166,6 +166,28 @@ pub enum ValidationProblemType { DuplicateElementIdentity, } +impl ValidationProblemType { + /// The stable machine-readable name of this variant. + /// + /// Every machine surface — the Python binding's `problem_type`, the CLI's + /// JSON `type` — reports the value through this one function, so the two + /// cannot spell the same variant differently and a variant rename cannot + /// silently change either contract. `Debug` formatting is a Rust + /// implementation detail and is deliberately not it. + pub fn label(&self) -> &'static str { + match self { + Self::UndeclaredSlot => "undeclared_slot", + Self::InapplicableSlot => "inapplicable_slot", + Self::MissingSlotValue => "missing_slot_value", + Self::SlotRangeViolation => "slot_range_violation", + Self::MaxCountViolation => "max_count_violation", + Self::ParsingError => "parsing_error", + Self::AmbiguousElementIdentity => "ambiguous_element_identity", + Self::DuplicateElementIdentity => "duplicate_element_identity", + } + } +} + pub type InstancePath = Vec; #[derive(Clone, Debug, PartialEq, Eq, Default)] diff --git a/src/tools/src/bin/linkml_schema_validate.rs b/src/tools/src/bin/linkml_schema_validate.rs index dadd72d..b24cecf 100644 --- a/src/tools/src/bin/linkml_schema_validate.rs +++ b/src/tools/src/bin/linkml_schema_validate.rs @@ -123,7 +123,10 @@ fn identity_warnings_json(warnings: &[ValidationResult]) -> serde_json::Value { .iter() .map(|w| { serde_json::json!({ - "type": format!("{:?}", w.problem_type), + // The shared machine label, never `Debug` formatting: it + // is the same spelling the Python binding reports, and a + // variant rename cannot change it behind the CLI's back. + "type": w.problem_type.label(), "severity": severity_label(&w.severity), "subject": w.subject, "detail": w.detail, diff --git a/src/tools/tests/schema_validate.rs b/src/tools/tests/schema_validate.rs index 1d2dee2..3c34b57 100644 --- a/src/tools/tests/schema_validate.rs +++ b/src/tools/tests/schema_validate.rs @@ -27,3 +27,78 @@ fn person_schema_missing_slot() { cmd.arg(&schema); cmd.assert().success(); } + +/// The JSON `--lint-identity` payload, and the process's success. +fn lint_identity_json(schema: &str) -> (bool, serde_json::Value) { + let mut cmd = Command::cargo_bin("linkml-schema-validate").unwrap(); + cmd.arg(data_path(schema)) + .arg("--lint-identity") + .arg("--output") + .arg("json"); + let out = cmd.output().unwrap(); + let stdout = String::from_utf8(out.stdout).unwrap(); + let parsed = + serde_json::from_str(&stdout).unwrap_or_else(|e| panic!("not JSON ({e}): {stdout}")); + (out.status.success(), parsed) +} + +/// The five keys the `--lint-identity --output json` contract promises, on +/// every outcome. Consumers branch on `identity_lint_skipped`, so a key that +/// appears only on one path is a key they cannot rely on. +const LINT_JSON_KEYS: [&str; 5] = [ + "errors", + "identity_lint_skipped", + "identity_lint_skipped_reason", + "identity_warnings", + "status", +]; + +fn keys(v: &serde_json::Value) -> Vec { + v.as_object() + .expect("a JSON object") + .keys() + .cloned() + .collect() +} + +#[test] +fn lint_identity_json_shape_when_the_schema_is_valid() { + let (ok, v) = lint_identity_json("identity.yaml"); + assert!(ok, "the identity fixture must validate: {v}"); + assert_eq!(keys(&v), LINT_JSON_KEYS); + assert_eq!(v["status"], "valid"); + assert_eq!(v["errors"], serde_json::json!([])); + assert_eq!(v["identity_lint_skipped"], false); + assert!(v["identity_lint_skipped_reason"].is_null()); + + let warnings = v["identity_warnings"].as_array().expect("an array"); + assert!(!warnings.is_empty(), "the fixture has flagged slots: {v}"); + for w in warnings { + assert_eq!(keys(w), ["detail", "severity", "subject", "type"]); + // The machine-readable spelling, shared with the Python binding's + // `problem_type`. `Debug` formatting would make a variant rename a + // silent change of this contract, and would spell it differently from + // the other binding for the same value. + assert_eq!( + w["type"], "ambiguous_element_identity", + "the JSON type must be the shared snake_case label: {w}" + ); + assert_eq!(w["severity"], "warning"); + assert!(w["subject"].is_array()); + assert!(w["detail"].as_str().is_some_and(|d| !d.is_empty())); + } +} + +#[test] +fn lint_identity_json_shape_when_the_lint_is_skipped() { + let (ok, v) = lint_identity_json("invalid_schema.yaml"); + assert!(!ok, "an invalid schema must still exit non-zero: {v}"); + assert_eq!(keys(&v), LINT_JSON_KEYS); + assert_eq!(v["status"], "invalid"); + assert!(!v["errors"].as_array().expect("an array").is_empty()); + assert!(v["identity_warnings"].is_null()); + assert_eq!(v["identity_lint_skipped"], true); + assert!(v["identity_lint_skipped_reason"] + .as_str() + .is_some_and(|r| !r.is_empty())); +} From db41c054422b8f4f20a12b2a52adaec5212034cf Mon Sep 17 00:00:00 2001 From: ejsyx Date: Wed, 19 Aug 2026 08:49:28 +0200 Subject: [PATCH 28/58] docs(runtime): document order-dependent list resolution within one patch `patch` resolves each list segment against the list's current state as the deltas apply, not against a snapshot of the list they were produced from. On a list whose elements carry duplicate identity labels that is observable: a delta removing the duplication flips the list to identity-addressed mid-sequence, and numeric segments still queued in the same patch are then reported failed. It fails loudly and never guesses, and it is confined to lists the identity linters exist to flag, so document the behaviour rather than build snapshot resolution for a zone the design tells authors to model their way out of. Co-Authored-By: Claude Fable 5 --- src/runtime/src/diff.rs | 18 ++++++++++++++++++ src/runtime/tests/diff_unique_keys.rs | 8 ++++---- 2 files changed, 22 insertions(+), 4 deletions(-) diff --git a/src/runtime/src/diff.rs b/src/runtime/src/diff.rs index 8050699..e448e92 100644 --- a/src/runtime/src/diff.rs +++ b/src/runtime/src/diff.rs @@ -525,6 +525,24 @@ impl Default for PatchOptions { } } +/// Apply `deltas` to a clone of `source`, returning the result and a +/// [`PatchTrace`]. A delta whose path cannot be resolved is reported in +/// [`PatchTrace::failed`] rather than guessed at. +/// +/// **List segments are resolved against the list's CURRENT state**, as the +/// deltas are applied in order — not against a snapshot of the list the deltas +/// were produced from. On a list whose elements carry *duplicate* identity +/// labels this is observable: such a list is addressed numerically, but a delta +/// that removes the duplication flips it to identity-addressed mid-sequence, +/// and every numeric segment still queued in the same patch then resolves to +/// nothing and is reported in `failed`. The patch stops short; it never lands +/// an edit on a guessed element. +/// +/// This is confined to lists whose element identity is already degenerate — +/// exactly what [`crate::lint_element_identity`] (the schema declares no +/// identity) and [`crate::lint_instance_identity`] (the data repeats one) exist +/// to flag. Declaring an identity, or `diff.linkml.io/opaque`, removes the +/// situation rather than working around it. pub fn patch( source: &LinkMLInstance, deltas: &[Delta], diff --git a/src/runtime/tests/diff_unique_keys.rs b/src/runtime/tests/diff_unique_keys.rs index 709ed3e..5f4ff53 100644 --- a/src/runtime/tests/diff_unique_keys.rs +++ b/src/runtime/tests/diff_unique_keys.rs @@ -428,10 +428,10 @@ fn duplicated_source_to_keyed_target_stays_positional_and_round_trips() { // are exactly what patch resolves against it. Keep positional deltas. // // The second element differs from its target only in the key-bearing slot, - // so the whole edit is one delta. A multi-delta variant of this case is - // order-dependent for reasons unrelated to the source-keyed fallback (see - // the branch report): once the key edit lands the list becomes - // keyed-shaped, and any numeric segment still queued stops resolving. + // so the whole edit is one delta. A multi-delta variant is order-dependent + // for a reason unrelated to the source-keyed fallback, documented on + // `patch`: once the key edit lands the list becomes keyed-shaped, and any + // numeric segment still queued is reported failed rather than guessed. let dup = json!({"phoneNumber": "09/241.25.03", "hasNumberFunction": "Emergency_Number"}); let before = phones(vec![e(), dup]); let after = phones(vec![e(), n()]); From 6965b367fb249cf29021e7fd81996ceb62a91ad4 Mon Sep 17 00:00:00 2001 From: ejsyx Date: Wed, 19 Aug 2026 08:58:56 +0200 Subject: [PATCH 29/58] fix(python): navigate test uses identifier segment under label-addressed lists MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `python_navigate.rs` still walked `['objects','2',…]`. `objects` ranges on NamedThing, which declares an `id` identifier, so the shared list-segment rule addresses it by label and the numeric segment no longer resolves — CI's bare `cargo test --workspace` would have gone red where the local gates could not, the python crate not linking here. Uses 'P:002', and asserts the numeric segment now resolves to nothing. The assertions live in a `py_run!` string that cannot execute without a linking python crate, so `navigate_basic` mirrors them one-to-one against `navigate_path`, which `PyLinkMLInstance::navigate` delegates straight to. Also fixes the crate's one rustdoc warning: `[`crate::diff`]` was ambiguous between the module and the re-exported function. Co-Authored-By: Claude Fable 5 --- src/python/tests/python_navigate.rs | 13 ++++++++++--- src/runtime/src/lib.rs | 5 +++-- src/runtime/tests/navigate.rs | 13 ++++++++++++- 3 files changed, 25 insertions(+), 6 deletions(-) diff --git a/src/python/tests/python_navigate.rs b/src/python/tests/python_navigate.rs index d25fb9e..788e3bc 100644 --- a/src/python/tests/python_navigate.rs +++ b/src/python/tests/python_navigate.rs @@ -62,13 +62,20 @@ value = load_no_errors(lr, lr.load_yaml, data_path, sv, cls) assert 'objects' in value.keys() assert value.navigate(['objects']) is not None -# Navigate list index then nested keys to scalar -name = value.navigate(['objects','2','has_medical_history','0','diagnosis','name']) +# Navigate a list by element identity label, then nested keys to a scalar. +# `objects` ranges on NamedThing, which declares an `id` identifier, so the +# list is addressed by that label, never by position. `has_medical_history` +# declares no identity and stays numeric. +name = value.navigate(['objects','P:002','has_medical_history','0','diagnosis','name']) assert name is not None assert name.as_python() == 'headache' # Non-existent path -> None -assert value.navigate(['objects','1000']) is None +assert value.navigate(['objects','P:404']) is None + +# A numeric segment aimed at a label-addressed list resolves to nothing +# rather than to that position: it never guesses an element. +assert value.navigate(['objects','2']) is None "# ); }); diff --git a/src/runtime/src/lib.rs b/src/runtime/src/lib.rs index abddb0c..4abd490 100644 --- a/src/runtime/src/lib.rs +++ b/src/runtime/src/lib.rs @@ -572,8 +572,9 @@ impl LinkMLInstance { /// the range class's `unique_keys`) for a list whose elements all carry /// unique labels, and a numeric index otherwise. /// - /// List segments resolve through the same rule [`crate::diff`] emits and - /// [`crate::patch`] applies, so a delta path is navigable by construction. + /// List segments resolve through the same rule [`diff()`](crate::diff()) + /// emits and [`patch()`](crate::patch()) applies, so a delta path is + /// navigable by construction. /// In particular a numeric segment aimed at a label-addressed list resolves /// to nothing rather than to that position: when a label happens to be /// `"0"`, position and label name different elements, and guessing which diff --git a/src/runtime/tests/navigate.rs b/src/runtime/tests/navigate.rs index 0ee14a0..9900022 100644 --- a/src/runtime/tests/navigate.rs +++ b/src/runtime/tests/navigate.rs @@ -38,6 +38,10 @@ fn navigate_basic() { // `objects` is inlined as a list of NamedThing, which declares an // `id` identifier: it is addressed by that label, not by position. // `has_medical_history` declares no identity, so it stays numeric. + // Mirrors src/python/tests/python_navigate.rs one-to-one: + // `PyLinkMLInstance::navigate` delegates straight to this method, + // and that test's assertions live inside a `py_run!` string that + // cannot be executed where the python crate does not link. let inner = v.navigate_path([ "objects", "P:002", @@ -46,11 +50,18 @@ fn navigate_basic() { "diagnosis", "name", ]); - assert!(inner.is_some()); + assert_eq!( + inner.map(|n| n.to_json()), + Some(serde_json::json!("headache")) + ); assert!( v.navigate_path(["objects", "2"]).is_none(), "a numeric segment must not address a label-addressed list" ); + assert!( + v.navigate_path(["objects", "P:404"]).is_none(), + "a label matching no element resolves to nothing" + ); } _ => panic!("expected map at root"), } From e6a5fbfd33699e8bed4c739bc7dd34561fd33056 Mon Sep 17 00:00:00 2001 From: ejsyx Date: Wed, 19 Aug 2026 09:03:15 +0200 Subject: [PATCH 30/58] feat(runtime): lint flags list identity derived from a type designator key MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A class keyed by its own `designates_type` slot looks, to the identity-less rule, like a class that declares its identity. In a homogeneous inlined list it does not: the designator's value is fixed per class, so an N-vertex ring yields N identical labels, the diff engine's uniqueness guard falls back to positional addressing, and the declaration is misleading rather than load-bearing. Flag the list form only. The dict form keyed by the same designator is a different and legitimate model — at-most-one-element-per-subtype — and stays silent, as do `opaque`/`ignore` slots and ordinary discriminating keys. The key outranks `unique_keys` in `element_identity_label`, so this fires (and the several-unique_keys rule does not) when a class declares both. Reported once, at the class introducing the slot. On the asset360 model this catches `Polyline.PolyLine_coordinates` and `Polygon.Polygon_coordinates`, both ranged on `PositioningSystemCoordinate`, whose key is the `typeURI` discriminator. Co-Authored-By: Claude Fable 5 --- src/runtime/src/identity_lint.rs | 80 ++++++++++++++++- .../data/identity_type_designator_key.yaml | 90 +++++++++++++++++++ src/runtime/tests/identity_lint.rs | 67 ++++++++++++++ 3 files changed, 236 insertions(+), 1 deletion(-) create mode 100644 src/runtime/tests/data/identity_type_designator_key.yaml diff --git a/src/runtime/src/identity_lint.rs b/src/runtime/src/identity_lint.rs index a25986a..9a389e5 100644 --- a/src/runtime/src/identity_lint.rs +++ b/src/runtime/src/identity_lint.rs @@ -30,6 +30,12 @@ //! earlier-sorting entry silently re-addresses every delta path for every slot //! ranged on it. That is warned about too, naming the load-bearing entry. //! +//! A third rule catches a declared identity that cannot discriminate: a list +//! whose element class is keyed by its own type designator, whose value is +//! constant across a homogeneous list, so the key labels every element alike +//! (the dict form of the same class, meaning at-most-one-per-subtype, is left +//! alone). +//! //! Warnings are reported at the class that **introduces** the slot: a flagged //! slot inherited unchanged by a descendant is not repeated there, since the //! declaration the author would edit lives on the ancestor. @@ -115,6 +121,58 @@ fn slot_has_ambiguous_unique_keys(slot: &SlotView) -> Option<(String, Vec Option<(String, String)> { + use linkml_schemaview::slotview::{SlotContainerMode, SlotInlineMode}; + if slot.determine_slot_container_mode() != SlotContainerMode::List { + return None; // the dict form keyed by the designator is legitimate + } + if slot.determine_slot_inline_mode() == SlotInlineMode::Reference { + return None; // elements are references, not inlined + } + if slot_is_opaque(slot) || slot_is_ignored(slot) { + return None; // no per-element identity is being claimed + } + let rc = slot.get_range_class()?; + // The key outranks `unique_keys` in `element_identity_label`, so it is the + // key that diff would use however many unique_keys the class also declares. + let key = rc.key_or_identifier_slot()?; + if key.definition().designates_type != Some(true) { + return None; + } + Some((rc.name().to_string(), key.name.clone())) +} + +/// The warning text for a list whose identity is its element class's type +/// designator. +fn type_designator_identity_detail( + class_name: &str, + slot_name: &str, + range_class: &str, + designator: &str, +) -> String { + format!( + "elements of '{class_name}.{slot_name}' take their identity from \ + '{range_class}.{designator}', which is the type designator \ + (designates_type). Its value is fixed per class, so it is constant \ + across a homogeneous list: N same-type elements yield N identical \ + labels, the diff engine's uniqueness guard falls back to positional \ + addressing, and the declared key is misleading rather than \ + load-bearing. Remodel: move the discriminating identity a layer up and \ + range the list on a bare element class that does not declare the \ + designator as its key — or use the dict form instead, if \ + at-most-one-element-per-subtype is what the key really means." + ) +} + /// Whether `class` is where a flagged slot should be reported, rather than an /// ancestor it merely inherits the problem from. /// @@ -199,7 +257,27 @@ pub fn lint_element_identity(sv: &SchemaView) -> Vec { // inherited warning on every descendant buries the one declaration // the author would actually edit. Applies to both rules below. if !slot_lacks_element_identity(slot) { - if let Some((rc_name, names)) = slot_has_ambiguous_unique_keys(slot) { + // Precedence mirrors `element_identity_label`: a key outranks + // `unique_keys`, so a designator key is what diff would use and + // the several-unique_keys rule has nothing to say about the + // slot. The two are mutually exclusive by construction; the + // `else` states the order rather than relying on it. + if let Some((rc_name, designator)) = slot_identity_is_type_designator(slot) { + if introduces_flagged_slot(&class, &slot.name, |s| { + slot_identity_is_type_designator(s).is_some() + }) { + sink.push_warning( + ValidationProblemType::AmbiguousElementIdentity, + vec![class.name().to_string(), slot.name.clone()], + type_designator_identity_detail( + class.name(), + &slot.name, + &rc_name, + &designator, + ), + ); + } + } else if let Some((rc_name, names)) = slot_has_ambiguous_unique_keys(slot) { if introduces_flagged_slot(&class, &slot.name, |s| { slot_has_ambiguous_unique_keys(s).is_some() }) { diff --git a/src/runtime/tests/data/identity_type_designator_key.yaml b/src/runtime/tests/data/identity_type_designator_key.yaml new file mode 100644 index 0000000..a94246c --- /dev/null +++ b/src/runtime/tests/data/identity_type_designator_key.yaml @@ -0,0 +1,90 @@ +id: https://w3id.org/linkml/examples/identity_type_designator_key +name: identity_type_designator_key +description: |- + Element identity derived from a key that is also the class's type designator. + In a homogeneous inlined list (a vertex ring) every element carries the same + designator value, so the declared key labels every element identically and the + diff engine falls back to positional addressing. The dict (mapping) form of + the same class is legitimate: it means at-most-one-element-per-subtype. +license: https://creativecommons.org/publicdomain/zero/1.0/ +imports: + - linkml:types +prefixes: + identity: https://w3id.org/linkml/examples/identity_type_designator_key/ + linkml: https://w3id.org/linkml/ +default_prefix: identity +default_range: string + +classes: + Ring: + attributes: + # a homogeneous vertex ring: every element's key is the same designator + # value, so the declared identity collapses the ring to one element + vertices: + range: Coordinate + multivalued: true + inlined_as_list: true + # the designator key outranks unique_keys, so this is flagged by the + # designator rule and NOT by the several-unique_keys rule + markers: + range: Marker + multivalued: true + inlined_as_list: true + # dict form: keyed by the designator means at-most-one-per-subtype, which + # is a legitimate model + byType: + range: Coordinate + multivalued: true + inlined: true + # identity declared as "nowhere": the value is replaced as a whole + archivedVertices: + range: Coordinate + multivalued: true + inlined_as_list: true + annotations: + diff.linkml.io/opaque: true + # outside diff's scope entirely + draftVertices: + range: Coordinate + multivalued: true + inlined_as_list: true + annotations: + diff.linkml.io/ignore: true + # an ordinary key, discriminating per element: nothing to warn about + points: + range: Point + multivalued: true + inlined_as_list: true + + # inherits `vertices` unchanged: reported at Ring, not here + SubRing: + is_a: Ring + + Coordinate: + attributes: + typeURI: + range: uriorcurie + key: true + designates_type: true + x: {range: float, required: true} + y: {range: float, required: true} + + Marker: + unique_keys: + by_code: + unique_key_slots: [code] + zz_by_label: + unique_key_slots: [label] + attributes: + typeURI: + range: uriorcurie + key: true + designates_type: true + code: {range: string, required: true} + label: {range: string, required: true} + + Point: + attributes: + pointId: {range: string, key: true} + x: {range: float, required: true} + y: {range: float, required: true} diff --git a/src/runtime/tests/identity_lint.rs b/src/runtime/tests/identity_lint.rs index 81e4559..af7f1de 100644 --- a/src/runtime/tests/identity_lint.rs +++ b/src/runtime/tests/identity_lint.rs @@ -204,6 +204,73 @@ fn schema_lint_warning_order_is_deterministic() { } } +#[test] +fn schema_lint_flags_a_list_whose_identity_is_the_type_designator() { + // A key that is also the type designator is constant across a homogeneous + // list: every vertex of a ring carries the same value, so keyed matching + // would collapse an N-vertex ring to one element. The class "has a key", so + // the identity-less rule passes it — this rule is what sees it. + let sv = schema_view("identity_type_designator_key.yaml"); + let warnings = lint_element_identity(&sv); + let subjects: Vec> = warnings.iter().map(|w| w.subject.clone()).collect(); + assert_eq!( + subjects, + vec![ + // the designator key outranks `unique_keys`, so `markers` is flagged + // by this rule, once, and not by the several-unique_keys rule + vec!["Ring".to_string(), "markers".to_string()], + vec!["Ring".to_string(), "vertices".to_string()], + ], + "only the designator-keyed list slots are flagged, once each, at their \ + introducing class: {warnings:#?}" + ); + let w = warnings + .iter() + .find(|w| w.subject[1] == "vertices") + .expect("the ring slot must be flagged"); + assert_eq!( + w.problem_type, + ValidationProblemType::AmbiguousElementIdentity + ); + assert!(!w.severity.is_error(), "the linter warns, never errors"); + assert!( + w.detail.contains("typeURI") && w.detail.contains("Coordinate"), + "the warning must name the designator slot and its class: {}", + w.detail + ); + assert!( + w.detail.contains("designates_type"), + "the warning must say why the key is not discriminating: {}", + w.detail + ); + assert!( + w.detail.contains("positional"), + "the warning must say what the diff engine actually does: {}", + w.detail + ); +} + +#[test] +fn schema_lint_leaves_designator_dicts_and_ordinary_keys_alone() { + // Guard rails for the designator rule: the dict form keyed by the designator + // means at-most-one-element-per-subtype and is legitimate; an ordinary key + // discriminates per element; `opaque` / `ignore` answer the question already; + // and an unchanged inherited slot belongs to its introducing class. + let sv = schema_view("identity_type_designator_key.yaml"); + let warnings = lint_element_identity(&sv); + let flagged: Vec = warnings.iter().map(|w| w.subject[1].clone()).collect(); + for silent in ["byType", "points", "archivedVertices", "draftVertices"] { + assert!( + !flagged.contains(&silent.to_string()), + "{silent} must stay silent, got {warnings:#?}" + ); + } + assert!( + !warnings.iter().any(|w| w.subject[0] == "SubRing"), + "an unchanged inherited slot is reported at Ring only: {warnings:#?}" + ); +} + #[test] fn data_lint_flags_duplicate_declared_identities() { let f = fixture(); From c1402a6787fd830ef7bc172f2eddeb5a2917f37e Mon Sep 17 00:00:00 2001 From: ejsyx Date: Wed, 19 Aug 2026 09:11:01 +0200 Subject: [PATCH 31/58] test(runtime): pin designator-key rule on inherited slot_usage shape MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The fixture only covered `key: true` and `designates_type: true` on the same attribute. The shape in the wild — asset360's PositioningSystemCoordinate — declares the designator on a base class and promotes it to the key with `slot_usage` on the subclass. The rule already handles that through `SlotView::definition()`'s is_a chain merge, but nothing pinned it: a regression in that merging would have produced a silent false negative with every test still green. Add `TypedThing` (designator, no key) and `KeyedTypedThing` (`slot_usage: typeURI: {key: true}`), range a ring-style list slot on the subclass, and extend the exact-subject assertion plus a check that the message names 'KeyedTypedThing.typeURI'. Inverting the designator check drops `inheritedVertices` from the flagged set, so the new assertion is not vacuous. Also correct two stale docs: `lint_element_identity`'s rustdoc described two rules, and the introducing-class comment said "both rules below". There are three. Co-Authored-By: Claude Fable 5 --- src/runtime/src/identity_lint.rs | 8 ++++--- .../data/identity_type_designator_key.yaml | 24 +++++++++++++++++++ src/runtime/tests/identity_lint.rs | 14 +++++++++++ 3 files changed, 43 insertions(+), 3 deletions(-) diff --git a/src/runtime/src/identity_lint.rs b/src/runtime/src/identity_lint.rs index 9a389e5..152b7c0 100644 --- a/src/runtime/src/identity_lint.rs +++ b/src/runtime/src/identity_lint.rs @@ -226,8 +226,10 @@ fn ambiguous_unique_keys_detail( } /// Schema-level lint: warn for every multivalued inlined slot whose element -/// identity comes from nowhere, and for every one whose identity is derived -/// from a class offering more than one `unique_keys` entry to derive it from. +/// identity comes from nowhere; for every list whose identity is the range +/// class's type designator, which cannot tell the elements of a homogeneous +/// list apart; and for every one whose identity is derived from a class +/// offering more than one `unique_keys` entry to derive it from. /// Warnings only — the schema stays usable. pub fn lint_element_identity(sv: &SchemaView) -> Vec { let mut sink = ValidationResultSink::default(); @@ -255,7 +257,7 @@ pub fn lint_element_identity(sv: &SchemaView) -> Vec { for slot in class.slots() { // Report at the class that introduces the slot: repeating an // inherited warning on every descendant buries the one declaration - // the author would actually edit. Applies to both rules below. + // the author would actually edit. Applies to all three rules below. if !slot_lacks_element_identity(slot) { // Precedence mirrors `element_identity_label`: a key outranks // `unique_keys`, so a designator key is what diff would use and diff --git a/src/runtime/tests/data/identity_type_designator_key.yaml b/src/runtime/tests/data/identity_type_designator_key.yaml index a94246c..2ff9aef 100644 --- a/src/runtime/tests/data/identity_type_designator_key.yaml +++ b/src/runtime/tests/data/identity_type_designator_key.yaml @@ -50,6 +50,14 @@ classes: inlined_as_list: true annotations: diff.linkml.io/ignore: true + # the wild shape (asset360's PositioningSystemCoordinate): the designator + # is declared on a base class and a subclass promotes it to the key with + # `slot_usage`. The rule only sees this because `SlotView::definition()` + # merges the is_a chain; without that it is a silent false negative. + inheritedVertices: + range: KeyedTypedThing + multivalued: true + inlined_as_list: true # an ordinary key, discriminating per element: nothing to warn about points: range: Point @@ -83,6 +91,22 @@ classes: code: {range: string, required: true} label: {range: string, required: true} + # declares the designator but no key: nothing ranges a list on it directly + TypedThing: + attributes: + typeURI: + range: uri + designates_type: true + label: {range: string} + + # promotes the inherited designator to the key, exactly as + # PositioningSystemCoordinate does + KeyedTypedThing: + is_a: TypedThing + slot_usage: + typeURI: + key: true + Point: attributes: pointId: {range: string, key: true} diff --git a/src/runtime/tests/identity_lint.rs b/src/runtime/tests/identity_lint.rs index af7f1de..e20e2cd 100644 --- a/src/runtime/tests/identity_lint.rs +++ b/src/runtime/tests/identity_lint.rs @@ -216,6 +216,10 @@ fn schema_lint_flags_a_list_whose_identity_is_the_type_designator() { assert_eq!( subjects, vec![ + // the wild shape: the designator is declared on a base class and + // the subclass promotes it to the key with `slot_usage`, so the + // rule sees it only through `SlotView::definition()`'s chain merge + vec!["Ring".to_string(), "inheritedVertices".to_string()], // the designator key outranks `unique_keys`, so `markers` is flagged // by this rule, once, and not by the several-unique_keys rule vec!["Ring".to_string(), "markers".to_string()], @@ -248,6 +252,16 @@ fn schema_lint_flags_a_list_whose_identity_is_the_type_designator() { "the warning must say what the diff engine actually does: {}", w.detail ); + let inherited = warnings + .iter() + .find(|w| w.subject[1] == "inheritedVertices") + .expect("a designator promoted to key by slot_usage must be flagged too"); + assert!( + inherited.detail.contains("'KeyedTypedThing.typeURI'"), + "the warning must name the subclass that declares the key, and the \ + inherited slot it declares it on: {}", + inherited.detail + ); } #[test] From 8639bdf1167b55c343ebf24996e02c5781880052 Mon Sep 17 00:00:00 2001 From: ejsyx Date: Wed, 19 Aug 2026 10:59:04 +0200 Subject: [PATCH 32/58] =?UTF-8?q?docs(spec):=20addendum=20=E2=80=94=20desi?= =?UTF-8?q?gnator=20and=20canonicalization=20hardening=20rules?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Fable 5 --- ...ned-multivalued-element-identity-design.md | 25 +++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/docs/superpowers/specs/2026-08-17-inlined-multivalued-element-identity-design.md b/docs/superpowers/specs/2026-08-17-inlined-multivalued-element-identity-design.md index 19d6a40..c8bafb8 100644 --- a/docs/superpowers/specs/2026-08-17-inlined-multivalued-element-identity-design.md +++ b/docs/superpowers/specs/2026-08-17-inlined-multivalued-element-identity-design.md @@ -299,3 +299,28 @@ Recycled (near-verbatim, renamed to `opaque` where the branch says `array`): - From the spike: `opaque_*` tests, `coordinates_match_by_unique_keys_derived_key`, and the `unique_keys` ambiguity-warning pair. Not recycled: the `Set` content-matching diff/patch (`diff_set`, `apply_set_leaf_delta`, the drift-location trio) and the single-slot `shape_key` override — both replaced by `unique_keys`-declared identity. + +## Addendum (2026-08-19): designator and canonicalization hardening + +An empirical spike (typeURI × polymorphism × induced URIs, findings D1–D9) showed the shipped identity machinery honours *spellings* and *declarations* where it must honour *meaning*. Since this branch already carries a sanctioned compatibility break, the following rules are added to the design. Every behavioural change below must be attributable, output-line by output-line, in the downstream differential harness (consolidator-server corpus) before it merges. + +### Rules + +1. **A type designator is never an element identity (D3).** `element_key_label` skips a key/identifier slot whose merged definition has `designates_type: true`; identity falls through to `unique_keys`, else the list is positional. Rationale: a designator's value is a function of the element's class — constant across any homogeneous list by construction. The schema-level designator-key lint rule remains as the author-facing voice; this rule makes the engine agree with it. (Blast radius: homogeneous rings already fell back to positional via the uniqueness guard; the change moves polymorphic designator-keyed lists and classes whose real `unique_keys` was shadowed.) +2. **Identity compares meaning, not spelling (D1 + D6).** + - Designator values are canonicalized at load (the boxing chokepoint, mirroring int/float coercion): once the element's class is selected, the stored designator value is the class's canonical designator value. JSON and RDF loaders then agree; `to_json` emits the canonical form. + - Identity-label components whose slot range descends from `uri`/`uriorcurie` are IRI-expanded before comparison, in **all** resolve sites at once: diff emission, `resolve_list_segment` (patch + navigate), and the instance lint. A curie and its expansion are one identity. +3. **A class change is a whole-element replacement (D2a).** When diff pairs two objects whose classes differ, it emits a single whole-element `Update` — never field-level recursion across classes. +4. **`patch` never hard-errors on an unappliable delta (D2b).** A delta whose value cannot be built or applied at its resolved location records its path in `trace.failed` and leaves the tree untouched; `Err` is reserved for infrastructure failures. One bad delta must not void a batch. +5. **The inlined-dict key is real data (D5).** `build_mapping_entry_for_slot` injects the dict key into the element's key/identifier slot when the payload omits it (the LinkML `inlined` contract); a payload value that disagrees with the dict key is a load-time validation warning. For designator-keyed dicts, a dict key that is not an accepted designator value is a load-time validation warning. +6. **Lint extensions (D7, D4d, D8, D9).** + - Instance lint: a list that is addressed positionally *despite* a declared identity (some element yields no label) is warned — missing labels get a voice, not only duplicates. + - Schema lint: the multi-`unique_keys` ambiguity check unions entries across the range class's descendants and warns when descendants resolve different load-bearing entries (split label space). + - Schema lint: warn when two classes within one `is_a` hierarchy share a `class_uri` and the hierarchy carries a designator (stable-but-arbitrary class selection); the loader behaviour itself is unchanged (deliberately out of scope — too hot for this branch). + - `slot_usage: designates_type: false` leaving an unfillable `key` is documented in the linter rustdoc, not specially detected. + +### Explicitly out of scope (recorded, not fixed here) + +- Loader preference of native-URI matches over shared `class_uri` matches (D8's fix half). +- Namespacing identity labels by the `unique_keys` entry that produced them (D4's deep fix). +- `key: true` implying `required: true` at validation time. From 4e54fc410edfb1285503bdac76d03cb8d541263e Mon Sep 17 00:00:00 2001 From: ejsyx Date: Wed, 19 Aug 2026 11:00:46 +0200 Subject: [PATCH 33/58] =?UTF-8?q?docs(plan):=20addendum=20tasks=2010-16=20?= =?UTF-8?q?=E2=80=94=20designator/canonicalization=20hardening?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Fable 5 --- ...18-inlined-multivalued-element-identity.md | 52 +++++++++++++++++++ 1 file changed, 52 insertions(+) diff --git a/docs/superpowers/plans/2026-08-18-inlined-multivalued-element-identity.md b/docs/superpowers/plans/2026-08-18-inlined-multivalued-element-identity.md index 52b5497..b66da00 100644 --- a/docs/superpowers/plans/2026-08-18-inlined-multivalued-element-identity.md +++ b/docs/superpowers/plans/2026-08-18-inlined-multivalued-element-identity.md @@ -1621,3 +1621,55 @@ Expected: PASS — this is the spec's Example 1 (phones via unique_keys), Exampl git add src/runtime/src/diff.rs src/runtime/src/identity_lint.rs git commit -m "docs(runtime): document opaque annotation and unique_keys path segments" ``` + +--- + +# Addendum tasks (2026-08-19): designator and canonicalization hardening + +> Argued from the spec's "Addendum (2026-08-19)" section — its numbered rules are the binding requirements. Detailed empirical findings: `.superpowers/sdd/2026-08-18-inlined-multivalued-element-identity/spike-findings.md` (D1–D9). These tasks are specified rule-and-test-first rather than code-verbatim: each names its binding spec rule, its exact observable behaviour changes, and its test cases; implementers derive the code. **Every task's verification includes the differential harness gate (Task 10): re-run, and attribute every changed output line to a spec-addendum rule; unattributable changes fail the task.** + +### Task 10: downstream differential harness + baseline + +**Files:** Create scripts + baselines under `.superpowers/sdd/2026-08-18-inlined-multivalued-element-identity/harness/` (gitignored workspace — nothing committed to the repo). + +Build a script (bash or python, your choice) that, given a repo checkout, produces a deterministic text report over a fixed corpus, using the repo's CLIs (`linkml-validate`, `linkml-diff`, `linkml-patch`, `linkml-schema-validate --lint-identity`, `linkml-convert` where useful): +- Corpus A (real downstream): every committed instance JSON under /home/ejsyx/projects/consolidator-server/components/py (skip .worktrees), each validated against the asset360 schema root and its class where derivable (signal/goldenrecords/deltas test files; record per-file: validate output, load→convert normal form, self-diff = must be empty). +- Corpus B (in-repo): the runtime test fixtures' data files + meta.yaml; the asset360 schema lint output (already known: 27 warnings). +- Pairwise diffs: for the consolidator delta fixtures directory, diff meaningful before/after pairs if identifiable; otherwise diff each file against a jq-mutated variant (deterministic mutation) to exercise list matching. +- Output: one sorted, stable report file per corpus. Capture BASELINE at current HEAD into `harness/baseline/`. The gate for later tasks: `diff -u baseline/ current/` — reviewed line by line. + +**Verification:** run twice at HEAD — byte-identical reports (determinism). Report corpus size (file counts) and any files that fail to load at baseline (they are baseline facts, not defects). + +### Task 11: D3 — a type designator is never an element identity (spec rule 1) + +**Files:** Modify `src/runtime/src/diff.rs` (`element_key_label`); `src/runtime/src/identity_lint.rs` only if gating needs alignment; tests `src/runtime/tests/diff_unique_keys.rs` + `identity_lint.rs` + fixture(s). + +`element_key_label` returns `None` when the key/identifier slot's merged definition has `designates_type == Some(true)`. Behaviour deltas to pin with tests: (1) a class with a designator key AND `unique_keys` now matches by the unique_keys label (the D3 shadow case: reorder of `{x,y}`-keyed elements emits zero deltas); (2) a designator-keyed class WITHOUT unique_keys is positional (and: no double lint fire — the designator-key rule remains the only voice for it; check `slot_lacks_element_identity`'s key gate and keep exactly one warning per such slot); (3) polymorphic designator-keyed list (one element per subtype) is now positional — pin the new behaviour. Harness gate: expected attributable changes only in designator-keyed polymorphic/shadowed cases. + +### Task 12: D1 + D6 — identity compares meaning, not spelling (spec rule 2) + +**Files:** Modify `src/runtime/src/lib.rs` (boxing chokepoints: `parse_object_fixed_class`, `parse_object_value`, `build_mapping_entry_for_slot` — designator canonicalization after class selection, mirroring `coerce_scalar_to_range`), `src/runtime/src/diff.rs` (`scalar_slot_string` IRI expansion for uri/uriorcurie-descended ranges — hits all four resolve sites through the existing shared helpers; thread a Converter via the class's schemaview handle), tests + fixtures. + +Tests to pin: designator spelled as curie / full URI / implicit all load to ONE canonical stored value and `to_json` emits it; junk designator value (matches no accepted value) becomes a load-time validation warning + canonical fill (decide error-vs-warning by the loader-tolerance precedent: warning); curie-vs-expanded uri unique_keys values are ONE identity across diff, patch, navigate, instance lint (one test each); RDF-vs-JSON agreement (load the same content both ways where the harness corpus allows, or a focused turtle_import test). Harness gate: attributable changes = canonical designator spellings in convert output for files that spelled them differently; self-diffs stay empty. + +### Task 13: D2 — class change is whole-element replacement; patch never hard-errors (spec rules 3+4) + +**Files:** Modify `src/runtime/src/diff.rs` (object arm class comparison; apply paths' builder-error handling), tests. + +Tests: Bolt-vs-Nut same-key diff = ONE whole-element Update (both directions); patch of a hand-built bad delta (unbuildable value at a resolved location) → `Ok`, path in `trace.failed`, tree untouched, and OTHER deltas in the same batch still apply; existing suites' Err expectations updated only where they asserted the old contract (list each). Check `src/runtime/src/blame/` compiles and its semantics commentary still holds. Harness gate: no expected corpus changes (defensive: attribute any). + +### Task 14: D5 — the inlined-dict key is real data (spec rule 5) + +**Files:** Modify `src/runtime/src/lib.rs` (`build_mapping_entry_for_slot` + validation sink usage), tests. + +Tests: plain keyed class dict entry without payload key → key slot filled from dict key, NO MissingSlotValue error; payload key present and equal → clean; payload key disagreeing with dict key → validation WARNING naming both; designator-keyed dict whose key is not an accepted designator value → validation warning; designator-keyed dict key that IS accepted → designator canonicalized per Task 12 and consistent. Harness gate: expected attributable changes = disappearing MissingSlotValue errors and new divergence warnings on real dict data (the known asset360 LinearCoordinate divergence should surface as a warning — cite it in the report). + +### Task 15: lint extensions + docs (spec rule 6) + +**Files:** Modify `src/runtime/src/identity_lint.rs` (+ tests/fixtures), rustdoc. + +(1) Instance lint: "positional despite declared identity" warning when keyed-shaped fails due to MISSING labels while the range class declares key/unique_keys (D7; also gives half-labelled lists a voice). (2) Schema lint: multi-unique_keys check unions entries across `rc.get_descendants(true,false)`; additional warning when descendants resolve different load-bearing entries (D4d — the split-label-space case). (3) Schema lint: warn when two classes in one is_a hierarchy share a `class_uri` and the hierarchy carries a designator (D8, warn-only). (4) Rustdoc: D9 note (`slot_usage: designates_type: false` leaves an unfillable key) + module doc lists all rules. Each rule: RED/GREEN with its own fixture case; existing exact-set tests stay green or are extended deliberately (list changes). Harness gate: asset360 lint delta reported and attributed (new warnings expected; count them). + +### Task 16: hardening close-out + +Full gates (fmt, scoped clippy, `cargo test --workspace --exclude linkml_runtime_python`, `cargo check -p linkml_runtime_python`), final harness run with the complete attribution table (baseline → final), asset360 lint before/after summary, and one rustdoc/spec cross-check that every addendum rule is implemented or explicitly listed as out-of-scope. Docs-only fixes allowed; anything behavioural found here is reported, not fixed. From a6c69f5c720a7f6d42620ccf9bcdad913a8745df Mon Sep 17 00:00:00 2001 From: ejsyx Date: Wed, 19 Aug 2026 11:36:25 +0200 Subject: [PATCH 34/58] fix(runtime): a type designator is never an element identity MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `element_key_label` skipped nothing: a key (or identifier) that is also the class's type designator labelled every element of a homogeneous list alike, and shadowed the `unique_keys` such a class may really be identified by. A designator's value is a function of the element's *class*, so it can never tell two elements apart — spec addendum rule 1 (D3). The engine now looks past a designator key (`identity_key_slot`): identity falls through to `unique_keys`, else the list is positional. Behaviour deltas: a designator-keyed class WITH `unique_keys` is now matched by them (reorder is a no-op, edits are addressed by the real label, for diff, patch and navigate alike); a polymorphic designator-keyed list, which used to pass as one-element-per-subtype, is positional. The schema lint keeps exactly one voice per designator-keyed slot. Its identity-less and several-`unique_keys` rules now use the engine's notion of a key, which the designator-keyed shape newly satisfies, so the reporting loop asks the designator question first and alone — the sharpest diagnosis, and the only one naming the declaration the author would edit. Its wording is updated to describe what the engine now does with such a key. Co-Authored-By: Claude Fable 5 --- src/runtime/src/diff.rs | 27 ++- src/runtime/src/identity_lint.rs | 118 ++++++++----- .../data/identity_type_designator_key.yaml | 5 + src/runtime/tests/diff_unique_keys.rs | 159 ++++++++++++++++++ src/runtime/tests/identity_lint.rs | 35 ++++ 5 files changed, 300 insertions(+), 44 deletions(-) diff --git a/src/runtime/src/diff.rs b/src/runtime/src/diff.rs index e448e92..6d2e0c1 100644 --- a/src/runtime/src/diff.rs +++ b/src/runtime/src/diff.rs @@ -50,10 +50,28 @@ pub(crate) fn scalar_slot_string( None } +/// The class's key/identifier slot, when it can identify an *element*. +/// +/// A key (or identifier) that is also the class's type designator is skipped: a +/// designator's value is a function of the element's *class*, not of the +/// element, so it is constant across any homogeneous list by construction and +/// says "one element per subtype" across a polymorphic one. Neither is element +/// identity. Identity then falls through to `unique_keys` — which such a class +/// may well declare, and which the designator key used to shadow — else the +/// list is positional. [`crate::lint_element_identity`] is the author-facing +/// voice for the same shape; this is the engine agreeing with it. +pub(crate) fn identity_key_slot(class: &ClassView) -> Option<&SlotView> { + let slot = class.key_or_identifier_slot()?; + if slot.definition().designates_type == Some(true) { + return None; + } + Some(slot) +} + /// The key/identifier value identifying `v` among its list siblings, if any. pub(crate) fn element_key_label(v: &LinkMLInstance) -> Option { if let LinkMLInstance::Object { values, class, .. } = v { - let id_slot = class.key_or_identifier_slot()?; + let id_slot = identity_key_slot(class)?; return scalar_slot_string(values, &id_slot.name); } None @@ -88,7 +106,8 @@ pub(crate) fn element_unique_key_label(v: &LinkMLInstance) -> Option { } /// Identity for keyed list matching: a key/identifier slot outranks a -/// `unique_keys` claim. +/// `unique_keys` claim — unless that key is the class's type designator, which +/// is never element identity (see [`identity_key_slot`]). pub(crate) fn element_identity_label(v: &LinkMLInstance) -> Option { element_key_label(v).or_else(|| element_unique_key_label(v)) } @@ -135,7 +154,9 @@ pub enum DeltaOp { /// by identity — the element's identity label: its identifier/key slot value, or /// failing that a value derived from the range class's `unique_keys`. Lists whose /// elements do not all carry a *unique* identity label are addressed by numeric -/// index instead. +/// index instead. A key/identifier that is the class's type designator does not +/// count: it labels the class, not the element, so identity falls through to +/// `unique_keys` or to the index (see [`identity_key_slot`]). /// /// For a `unique_keys`-derived segment, a single-slot key contributes the bare /// value of that slot, while a composite key contributes the JSON array encoding diff --git a/src/runtime/src/identity_lint.rs b/src/runtime/src/identity_lint.rs index 152b7c0..f7f6e52 100644 --- a/src/runtime/src/identity_lint.rs +++ b/src/runtime/src/identity_lint.rs @@ -31,24 +31,30 @@ //! ranged on it. That is warned about too, naming the load-bearing entry. //! //! A third rule catches a declared identity that cannot discriminate: a list -//! whose element class is keyed by its own type designator, whose value is -//! constant across a homogeneous list, so the key labels every element alike -//! (the dict form of the same class, meaning at-most-one-per-subtype, is left -//! alone). +//! whose element class is keyed by its own type designator, whose value +//! describes the class rather than the element (the dict form of the same +//! class, meaning at-most-one-per-subtype, is left alone). The engine ignores +//! such a key entirely, so the slot also matches the first rule's shape (no +//! other identity is declared) or the second's (the `unique_keys` the key used +//! to shadow); this rule is the sharpest diagnosis and is asked first, so a +//! designator-keyed slot yields exactly one warning — this one. //! //! Warnings are reported at the class that **introduces** the slot: a flagged //! slot inherited unchanged by a descendant is not repeated there, since the //! declaration the author would edit lives on the ancestor. -use crate::diff::{element_identity_label, slot_is_ignored, slot_is_opaque, OPAQUE_ANNOTATION}; +use crate::diff::{ + element_identity_label, identity_key_slot, slot_is_ignored, slot_is_opaque, OPAQUE_ANNOTATION, +}; use crate::{LinkMLInstance, ValidationProblemType, ValidationResult, ValidationResultSink}; use linkml_schemaview::identifier::Identifier; use linkml_schemaview::schemaview::{ClassView, SchemaView}; use linkml_schemaview::slotview::SlotView; use std::collections::{BTreeMap, HashMap, HashSet}; -/// Whether this slot is one the lint flags: a multivalued inlined slot whose -/// element identity comes from nowhere. +/// Whether this is a multivalued inlined slot whose element identity comes from +/// nowhere — the engine's answer, from which the reporting loop subtracts the +/// designator case (see [`slot_is_identity_less_only`]). /// /// Split out from the reporting loop because the same question has to be asked /// of an inherited slot on its parent class, to decide which class introduced @@ -68,7 +74,13 @@ fn slot_lacks_element_identity(slot: &SlotView) -> bool { return false; // outside diff's scope entirely: no deltas, no identity } if let Some(rc) = slot.get_range_class() { - if rc.key_or_identifier_slot().is_some() || !rc.unique_keys().is_empty() { + // The engine's notion of a key, not the metamodel's: a key that is the + // class's type designator identifies the class, never the element, and + // `element_identity_label` looks straight past it (spec addendum rule + // 1). Such a class does lack element identity — but the designator rule + // below diagnoses it more precisely and is its only voice, so the + // reporting loop asks that question first. + if identity_key_slot(&rc).is_some() || !rc.unique_keys().is_empty() { return false; } } @@ -98,7 +110,10 @@ fn identity_unique_key_names(rc: &ClassView) -> Vec { /// /// A class with a `key`/`identifier` slot is not ambiguous however many /// `unique_keys` it declares: the key outranks them all, so none of them is -/// load-bearing and adding one changes nothing. +/// load-bearing and adding one changes nothing. A key that is the class's type +/// designator outranks nothing — the engine looks past it — so such a class is +/// judged on its `unique_keys` like any other; the designator rule speaks for +/// it first regardless, since that is the defect worth reporting. fn slot_has_ambiguous_unique_keys(slot: &SlotView) -> Option<(String, Vec)> { use linkml_schemaview::slotview::{SlotContainerMode, SlotInlineMode}; if slot.determine_slot_container_mode() != SlotContainerMode::List { @@ -111,7 +126,7 @@ fn slot_has_ambiguous_unique_keys(slot: &SlotView) -> Option<(String, Vec Option<(String, Vec Option<(String, String)> { use linkml_schemaview::slotview::{SlotContainerMode, SlotInlineMode}; if slot.determine_slot_container_mode() != SlotContainerMode::List { @@ -142,8 +163,8 @@ fn slot_identity_is_type_designator(slot: &SlotView) -> Option<(String, String)> return None; // no per-element identity is being claimed } let rc = slot.get_range_class()?; - // The key outranks `unique_keys` in `element_identity_label`, so it is the - // key that diff would use however many unique_keys the class also declares. + // Asked of the metamodel's key, not the engine's: this rule exists to + // report the key `element_identity_label` deliberately looks past. let key = rc.key_or_identifier_slot()?; if key.definition().designates_type != Some(true) { return None; @@ -151,6 +172,18 @@ fn slot_identity_is_type_designator(slot: &SlotView) -> Option<(String, String)> Some((rc.name().to_string(), key.name.clone())) } +/// The identity-less rule as the reporting loop applies it. +/// +/// A designator-keyed class declaring no `unique_keys` does lack element +/// identity, but the designator rule diagnoses it precisely and is its only +/// voice. Subtracting that case here keeps the "flagged for the same reason" +/// contract of [`introduces_flagged_slot`] honest: a subclass whose `slot_usage` +/// swaps a designator-keyed range for a bare one is still judged on its own +/// merits. +fn slot_is_identity_less_only(slot: &SlotView) -> bool { + slot_lacks_element_identity(slot) && slot_identity_is_type_designator(slot).is_none() +} + /// The warning text for a list whose identity is its element class's type /// designator. fn type_designator_identity_detail( @@ -160,13 +193,15 @@ fn type_designator_identity_detail( designator: &str, ) -> String { format!( - "elements of '{class_name}.{slot_name}' take their identity from \ + "elements of '{class_name}.{slot_name}' declare their identity as \ '{range_class}.{designator}', which is the type designator \ - (designates_type). Its value is fixed per class, so it is constant \ - across a homogeneous list: N same-type elements yield N identical \ - labels, the diff engine's uniqueness guard falls back to positional \ - addressing, and the declared key is misleading rather than \ - load-bearing. Remodel: move the discriminating identity a layer up and \ + (designates_type). Its value is a function of the element's class, not \ + of the element: constant across a homogeneous list, and one value per \ + subtype across a polymorphic one. It is therefore never element \ + identity, and the diff engine ignores the key outright — the list is \ + addressed by the element class's unique_keys if it declares any, \ + positionally otherwise. Declare an identity that varies per element: a \ + discriminating key/identifier or unique_keys on '{range_class}', or \ range the list on a bare element class that does not declare the \ designator as its key — or use the dict form instead, if \ at-most-one-element-per-subtype is what the key really means." @@ -258,28 +293,29 @@ pub fn lint_element_identity(sv: &SchemaView) -> Vec { // Report at the class that introduces the slot: repeating an // inherited warning on every descendant buries the one declaration // the author would actually edit. Applies to all three rules below. - if !slot_lacks_element_identity(slot) { - // Precedence mirrors `element_identity_label`: a key outranks - // `unique_keys`, so a designator key is what diff would use and - // the several-unique_keys rule has nothing to say about the - // slot. The two are mutually exclusive by construction; the - // `else` states the order rather than relying on it. - if let Some((rc_name, designator)) = slot_identity_is_type_designator(slot) { - if introduces_flagged_slot(&class, &slot.name, |s| { - slot_identity_is_type_designator(s).is_some() - }) { - sink.push_warning( - ValidationProblemType::AmbiguousElementIdentity, - vec![class.name().to_string(), slot.name.clone()], - type_designator_identity_detail( - class.name(), - &slot.name, - &rc_name, - &designator, - ), - ); - } - } else if let Some((rc_name, names)) = slot_has_ambiguous_unique_keys(slot) { + // The designator rule speaks first, and alone. Since the engine + // stopped accepting a designator as element identity, such a slot + // also matches the identity-less shape or the several-unique_keys + // shape, and one warning per slot means the sharpest one wins. + if let Some((rc_name, designator)) = slot_identity_is_type_designator(slot) { + if introduces_flagged_slot(&class, &slot.name, |s| { + slot_identity_is_type_designator(s).is_some() + }) { + sink.push_warning( + ValidationProblemType::AmbiguousElementIdentity, + vec![class.name().to_string(), slot.name.clone()], + type_designator_identity_detail( + class.name(), + &slot.name, + &rc_name, + &designator, + ), + ); + } + continue; + } + if !slot_is_identity_less_only(slot) { + if let Some((rc_name, names)) = slot_has_ambiguous_unique_keys(slot) { if introduces_flagged_slot(&class, &slot.name, |s| { slot_has_ambiguous_unique_keys(s).is_some() }) { @@ -297,7 +333,7 @@ pub fn lint_element_identity(sv: &SchemaView) -> Vec { } continue; } - if !introduces_flagged_slot(&class, &slot.name, slot_lacks_element_identity) { + if !introduces_flagged_slot(&class, &slot.name, slot_is_identity_less_only) { continue; } let range_class = slot.get_range_class(); diff --git a/src/runtime/tests/data/identity_type_designator_key.yaml b/src/runtime/tests/data/identity_type_designator_key.yaml index 2ff9aef..99ff7ae 100644 --- a/src/runtime/tests/data/identity_type_designator_key.yaml +++ b/src/runtime/tests/data/identity_type_designator_key.yaml @@ -107,6 +107,11 @@ classes: typeURI: key: true + # a second concrete subtype, so a list ranged on `KeyedTypedThing` can be + # polymorphic: one element per subtype, each carrying its own designator value + SpecialKeyedTypedThing: + is_a: KeyedTypedThing + Point: attributes: pointId: {range: string, key: true} diff --git a/src/runtime/tests/diff_unique_keys.rs b/src/runtime/tests/diff_unique_keys.rs index 5f4ff53..4acdc6e 100644 --- a/src/runtime/tests/diff_unique_keys.rs +++ b/src/runtime/tests/diff_unique_keys.rs @@ -457,3 +457,162 @@ fn duplicated_source_to_keyed_target_stays_positional_and_round_trips() { patched.to_json() ); } + +// --------------------------------------------------------------------------- +// D3 — a type designator is never an element identity (spec addendum rule 1). +// +// A designator's value is a function of the element's class, so it cannot tell +// two elements of one class apart. The engine skips a designator key entirely: +// identity falls through to `unique_keys`, else the list is positional. + +struct DesignatorFixture { + sv: SchemaView, + conv: Converter, + ring: ClassView, +} + +fn designator_fixture() -> DesignatorFixture { + let mut p = PathBuf::from(env!("CARGO_MANIFEST_DIR")); + p.push("tests/data/identity_type_designator_key.yaml"); + let schema = from_yaml(&p).unwrap(); + let mut sv = SchemaView::new(); + sv.add_schema(schema.clone()).unwrap(); + let conv = converter_from_schema(&schema); + let ring = sv + .get_class(&Identifier::new("Ring"), &conv) + .unwrap() + .expect("class not found"); + DesignatorFixture { sv, conv, ring } +} + +impl DesignatorFixture { + fn load(&self, v: JsonValue) -> LinkMLInstance { + load_json_str(&v.to_string(), &self.sv, &self.ring, &self.conv) + .unwrap() + .into_instance() + .unwrap() + } + + fn diff2(&self, before: JsonValue, after: JsonValue) -> Vec { + diff( + &self.load(before), + &self.load(after), + DiffOptions::new(true), + ) + } +} + +fn m1() -> JsonValue { + json!({"code": "M1", "label": "one"}) +} +fn m2() -> JsonValue { + json!({"code": "M2", "label": "two"}) +} + +#[test] +fn designator_key_does_not_shadow_unique_keys() { + // `Marker` declares BOTH a designator key (`typeURI`) and `unique_keys`. + // The designator labels every marker alike; the real identity is `by_code` + // (the name-sorted first entry). Reordering the list must be a no-op. + let f = designator_fixture(); + let deltas = f.diff2( + json!({"markers": [m1(), m2()]}), + json!({"markers": [m2(), m1()]}), + ); + assert!( + deltas.is_empty(), + "a reorder of unique_keys-identified markers is not a change: {deltas:#?}" + ); +} + +#[test] +fn designator_key_shadow_field_edit_is_addressed_by_the_unique_key_label() { + let f = designator_fixture(); + let mut edited = m2(); + edited["label"] = json!("TWO"); + let before = json!({"markers": [m1(), m2()]}); + let after = json!({"markers": [m1(), edited]}); + let deltas = f.diff2(before.clone(), after.clone()); + let delta = only(&deltas); + assert_eq!( + delta.path, + vec!["markers".to_string(), "M2".to_string(), "label".to_string()], + "the unique_keys label addresses the edit, not the constant designator" + ); + assert_eq!(delta.op, DeltaOp::Update); + + // diff ↔ patch ↔ navigate stay symmetric: the emitted label resolves + // through `resolve_list_segment` for both consumers. + let loaded = f.load(before); + assert!( + loaded + .navigate_path(["markers", "M2", "label"]) + .is_some_and(|v| v.to_json() == json!("two")), + "navigate must resolve the label diff emitted" + ); + let (patched, trace) = patch(&loaded, &deltas, PatchOptions::default()).unwrap(); + assert!(trace.failed.is_empty(), "{:?}", trace.failed); + assert!( + patched.equals(&f.load(after), true), + "patch(a, diff(a,b)) must equal b: {}", + patched.to_json() + ); +} + +#[test] +fn designator_keyed_class_without_unique_keys_is_positional() { + // `Coordinate` is keyed by its designator and declares no `unique_keys`, so + // it has no element identity at all: even a one-element list — where the + // constant designator would pass the uniqueness guard — is positional. + let f = designator_fixture(); + let before = json!({"vertices": [{"x": 1.0, "y": 2.0}]}); + let after = json!({"vertices": [{"x": 1.0, "y": 9.0}]}); + let deltas = f.diff2(before.clone(), after.clone()); + let delta = only(&deltas); + assert_eq!( + delta.path, + vec!["vertices".to_string(), "0".to_string(), "y".to_string()], + "a designator key yields no label, so the list is addressed by index" + ); + + let (patched, trace) = patch(&f.load(before), &deltas, PatchOptions::default()).unwrap(); + assert!(trace.failed.is_empty(), "{:?}", trace.failed); + assert!( + patched.equals(&f.load(after), true), + "{}", + patched.to_json() + ); +} + +#[test] +fn polymorphic_designator_keyed_list_is_positional() { + // One element per subtype: the designator values differ, so they used to + // pass as a per-element identity ("at most one element per subtype"). + // A designator still describes the class, never the element — positional. + let f = designator_fixture(); + let base = json!({"typeURI": "identity:KeyedTypedThing", "label": "base"}); + let special = json!({"typeURI": "identity:SpecialKeyedTypedThing", "label": "special"}); + let mut edited = base.clone(); + edited["label"] = json!("BASE"); + let before = json!({"inheritedVertices": [base, special.clone()]}); + let after = json!({"inheritedVertices": [edited, special]}); + let deltas = f.diff2(before.clone(), after.clone()); + let delta = only(&deltas); + assert_eq!( + delta.path, + vec![ + "inheritedVertices".to_string(), + "0".to_string(), + "label".to_string() + ], + "a polymorphic designator-keyed list is addressed by index too" + ); + + let (patched, trace) = patch(&f.load(before), &deltas, PatchOptions::default()).unwrap(); + assert!(trace.failed.is_empty(), "{:?}", trace.failed); + assert!( + patched.equals(&f.load(after), true), + "{}", + patched.to_json() + ); +} diff --git a/src/runtime/tests/identity_lint.rs b/src/runtime/tests/identity_lint.rs index e20e2cd..2e0a758 100644 --- a/src/runtime/tests/identity_lint.rs +++ b/src/runtime/tests/identity_lint.rs @@ -264,6 +264,41 @@ fn schema_lint_flags_a_list_whose_identity_is_the_type_designator() { ); } +#[test] +fn schema_lint_fires_once_per_designator_keyed_slot_and_names_the_designator() { + // The engine no longer accepts a designator as element identity (spec + // addendum rule 1), so a designator-keyed class declaring no `unique_keys` + // is *also* the shape the identity-less rule looks for, and one whose real + // `unique_keys` were shadowed is the shape the several-unique_keys rule + // looks for. The designator rule is the sharper diagnosis of both and + // stays their only voice: exactly one warning per slot, naming the + // designator that cannot discriminate. + let sv = schema_view("identity_type_designator_key.yaml"); + let warnings = lint_element_identity(&sv); + for (slot, why) in [ + ("vertices", "a designator-keyed class with no unique_keys"), + ( + "markers", + "a designator-keyed class whose unique_keys it shadowed", + ), + ] { + let fired: Vec<_> = warnings + .iter() + .filter(|w| w.subject == vec!["Ring".to_string(), slot.to_string()]) + .collect(); + assert_eq!( + fired.len(), + 1, + "{why} must produce exactly one warning for '{slot}': {warnings:#?}" + ); + assert!( + fired[0].detail.contains("designates_type") && fired[0].detail.contains("typeURI"), + "the designator rule must be the voice for '{slot}': {}", + fired[0].detail + ); + } +} + #[test] fn schema_lint_leaves_designator_dicts_and_ordinary_keys_alone() { // Guard rails for the designator rule: the dict form keyed by the designator From e5bb15af8c941586ab6beacd76bd224a8c2cf0df Mon Sep 17 00:00:00 2001 From: ejsyx Date: Wed, 19 Aug 2026 11:44:24 +0200 Subject: [PATCH 35/58] docs(runtime): correct chokepoint scope, stale fixture comments, lint hedge MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `identity_key_slot` is the chokepoint for the *labelling* question only. The `treat_changed_identifier_as_new_object` branch deliberately keeps asking the metamodel's key: it asks whether this is still the same thing, and a changed designator value means a different class — a whole-element replacement, which spec addendum rule 3 will make general. Routing it through `identity_key_slot` would drop that replacement and recurse across two classes. Comment at the site, and scope the rustdoc that implied otherwise. The designator fixture's comments still described the pre-rule-1 engine: the ring "collapsing to one element", the designator key "outranking unique_keys" (now exactly backwards), and positional fallback stated unconditionally. The lint's new wording over-claimed past the uniqueness guard: unique_keys address the list when their labels are unique within it. Co-Authored-By: Claude Fable 5 --- src/runtime/src/diff.rs | 14 ++++++++++ src/runtime/src/identity_lint.rs | 5 ++-- .../data/identity_type_designator_key.yaml | 26 ++++++++++++------- 3 files changed, 34 insertions(+), 11 deletions(-) diff --git a/src/runtime/src/diff.rs b/src/runtime/src/diff.rs index 6d2e0c1..7b142a2 100644 --- a/src/runtime/src/diff.rs +++ b/src/runtime/src/diff.rs @@ -60,6 +60,12 @@ pub(crate) fn scalar_slot_string( /// may well declare, and which the designator key used to shadow — else the /// list is positional. [`crate::lint_element_identity`] is the author-facing /// voice for the same shape; this is the engine agreeing with it. +/// +/// This governs *labelling* — how an element is addressed among its siblings — +/// and so is used by every resolve site (diff emission, `resolve_list_segment` +/// for patch and navigate, the instance lint). It is deliberately **not** used +/// by `diff`'s changed-key check, which asks a different question: see the +/// comment at `treat_changed_identifier_as_new_object`. pub(crate) fn identity_key_slot(class: &ClassView) -> Option<&SlotView> { let slot = class.key_or_identifier_slot()?; if slot.definition().designates_type == Some(true) { @@ -290,6 +296,14 @@ pub fn diff(source: &LinkMLInstance, target: &LinkMLInstance, opts: DiffOptions) // If objects have an identifier or key slot and it changed, treat as whole-object replacement // This applies for single-valued and list-valued inlined objects. if opts.treat_changed_identifier_as_new_object { + // Deliberately the metamodel's key, not [`identity_key_slot`]: + // this asks "is this still the same thing?", not "how is it + // labelled among its siblings". A changed key value means a + // different element, and for a *designator* key a changed + // value means a different class — which is a whole-element + // replacement too (spec addendum rule 3). Routing this + // through `identity_key_slot` would drop that replacement + // and recurse field-by-field across two classes. let key_slot_name = sc .key_or_identifier_slot() .or_else(|| tc.key_or_identifier_slot()) diff --git a/src/runtime/src/identity_lint.rs b/src/runtime/src/identity_lint.rs index f7f6e52..9c8e364 100644 --- a/src/runtime/src/identity_lint.rs +++ b/src/runtime/src/identity_lint.rs @@ -199,8 +199,9 @@ fn type_designator_identity_detail( of the element: constant across a homogeneous list, and one value per \ subtype across a polymorphic one. It is therefore never element \ identity, and the diff engine ignores the key outright — the list is \ - addressed by the element class's unique_keys if it declares any, \ - positionally otherwise. Declare an identity that varies per element: a \ + addressed by the element class's unique_keys if it declares any and \ + those labels are unique within the list, positionally otherwise. \ + Declare an identity that varies per element: a \ discriminating key/identifier or unique_keys on '{range_class}', or \ range the list on a bare element class that does not declare the \ designator as its key — or use the dict form instead, if \ diff --git a/src/runtime/tests/data/identity_type_designator_key.yaml b/src/runtime/tests/data/identity_type_designator_key.yaml index 99ff7ae..f18a910 100644 --- a/src/runtime/tests/data/identity_type_designator_key.yaml +++ b/src/runtime/tests/data/identity_type_designator_key.yaml @@ -1,11 +1,13 @@ id: https://w3id.org/linkml/examples/identity_type_designator_key name: identity_type_designator_key description: |- - Element identity derived from a key that is also the class's type designator. - In a homogeneous inlined list (a vertex ring) every element carries the same - designator value, so the declared key labels every element identically and the - diff engine falls back to positional addressing. The dict (mapping) form of - the same class is legitimate: it means at-most-one-element-per-subtype. + Element identity declared as a key that is also the class's type designator. + A designator's value describes the element's class, never the element, so it + is never element identity: the diff engine ignores such a key outright and + addresses the list by the element class's unique_keys when it declares any + (and their labels are unique within the list), positionally otherwise. The + dict (mapping) form of the same class is legitimate: it means + at-most-one-element-per-subtype. license: https://creativecommons.org/publicdomain/zero/1.0/ imports: - linkml:types @@ -18,14 +20,17 @@ default_range: string classes: Ring: attributes: - # a homogeneous vertex ring: every element's key is the same designator - # value, so the declared identity collapses the ring to one element + # a homogeneous vertex ring whose element class declares nothing but the + # designator key: the engine ignores that key, no identity is left, and + # the ring is addressed positionally (even at length one) vertices: range: Coordinate multivalued: true inlined_as_list: true - # the designator key outranks unique_keys, so this is flagged by the - # designator rule and NOT by the several-unique_keys rule + # the shadow case: the designator key used to outrank unique_keys and no + # longer does, so `by_code` (name-sorted first) is the real identity the + # engine matches on. Still flagged by the designator rule only — the + # sharpest diagnosis, and the sole voice for a designator-keyed slot markers: range: Marker multivalued: true @@ -54,6 +59,9 @@ classes: # is declared on a base class and a subclass promotes it to the key with # `slot_usage`. The rule only sees this because `SlotView::definition()` # merges the is_a chain; without that it is a silent false negative. + # Also the polymorphic case: with `SpecialKeyedTypedThing` in the list the + # designator values differ per element, which is a fact about the classes + # present, not an element identity — positional too. inheritedVertices: range: KeyedTypedThing multivalued: true From 2b3f91872bb50b4dc178392458822ea7d0c412b7 Mon Sep 17 00:00:00 2001 From: ejsyx Date: Wed, 19 Aug 2026 12:16:38 +0200 Subject: [PATCH 36/58] fix(schemaview): native class URI survives expansion in get_uri MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `ClassView::get_uri(native: true, expand: true)` returned the *canonical* URI, so for a class that declares a `class_uri` the schema-native URI was unreachable through the public API. Its only caller is `get_accepted_type_designator_values`, whose rustdoc promises "the canonical URI, native URI, and CURIE forms so that data produced with different prefix settings can be round-tripped" — it delivered three of the four, and the missing one was exactly the spelling a schema's own generator produces. The consequence was silent: a designator spelled `https://w3id.org/linkml/examples/personinfo/Organization` matched no accepted value, so `select_class` fell through to slot-shape matching and loaded the element as the base class `NamedThing`. Spec addendum rule 2 makes that set load-bearing — a spelling missing from it is not just ignored, it now decides which class's designator value gets stored — so the gap has to close before designator canonicalisation lands. `validation_issue_paths_include_list_indices_once` filtered for `UndeclaredSlot` at `objects.1.primary_email`, which only existed because the object had been loaded as `NamedThing`. The fixture is named `container_person_bad_email.yaml` and now behaves like it: the bad email is a pattern violation on a declared slot. The test is about the path carrying the list index exactly once, so it no longer filters by problem type. Co-Authored-By: Claude Fable 5 --- src/runtime/tests/validation.rs | 9 ++++----- src/schemaview/src/classview.rs | 13 +++++++++---- src/schemaview/tests/class_uri.rs | 12 ++++++++++++ 3 files changed, 25 insertions(+), 9 deletions(-) diff --git a/src/runtime/tests/validation.rs b/src/runtime/tests/validation.rs index 399798e..dc2a89a 100644 --- a/src/runtime/tests/validation.rs +++ b/src/runtime/tests/validation.rs @@ -155,11 +155,10 @@ fn validation_issue_paths_include_list_indices_once() { "1".to_string(), "primary_email".to_string(), ]; - let diag_paths: Vec<_> = diags - .iter() - .filter(|d| matches!(d.problem_type, ValidationProblemType::UndeclaredSlot)) - .map(|d| d.subject.clone()) - .collect(); + // The fixture's second object declares `objecttype` as Person, so the bad + // email is a pattern violation on a declared slot — this test is about the + // *path* carrying the list index exactly once, whatever the problem type. + let diag_paths: Vec<_> = diags.iter().map(|d| d.subject.clone()).collect(); assert!( diag_paths.iter().any(|p| p == &expected_path), "diagnostics: {:?}", diff --git a/src/schemaview/src/classview.rs b/src/schemaview/src/classview.rs index 7454a0c..2aa6b0d 100644 --- a/src/schemaview/src/classview.rs +++ b/src/schemaview/src/classview.rs @@ -254,6 +254,14 @@ impl ClassView { /// - `native=true` uses the schema's default prefix; `native=false` prefers /// an explicit `class_uri` when set. /// - `expand=true` returns a full URI; `expand=false` returns a CURIE. + /// + /// All four combinations are distinct for a class that declares a + /// `class_uri`. In particular `(native: true, expand: true)` is the + /// *schema-native* URI (``) and not the + /// `class_uri` — that is what `(native: false, expand: true)` returns. + /// [`Self::get_accepted_type_designator_values`] depends on the difference: + /// data that spells a designator with the schema-native URI of a class that + /// also declares a `class_uri` still *means* that class. pub fn get_uri( &self, conv: &Converter, @@ -279,7 +287,7 @@ impl ClassView { expand: bool, ) -> Result { match (native, expand) { - (true, true) => Ok(ids.canonical_uri()), + (true, true) => Ok(ids.native_uri()), (true, false) => { if let Some(curie) = ids.native_curie() { Ok(curie) @@ -306,9 +314,6 @@ impl ClassView { native: bool, expand: bool, ) -> Result { - if native && expand { - return Ok(self.canonical_uri()); - } let schema = self .data .sv diff --git a/src/schemaview/tests/class_uri.rs b/src/schemaview/tests/class_uri.rs index c615504..47de95c 100644 --- a/src/schemaview/tests/class_uri.rs +++ b/src/schemaview/tests/class_uri.rs @@ -29,6 +29,18 @@ fn class_get_uri() { cv.get_uri(&conv, true, false).unwrap().to_string(), "personinfo:Person" ); + assert_eq!( + cv.get_uri(&conv, false, true).unwrap().to_string(), + "https://w3id.org/linkml/Person" + ); + // `native` must survive expansion: this is the schema-native URI, not the + // `class_uri`. `get_accepted_type_designator_values` relies on the two being + // distinct — data spelling a designator with the native URI of a class that + // also declares a `class_uri` still means that class. + assert_eq!( + cv.get_uri(&conv, true, true).unwrap().to_string(), + "https://w3id.org/linkml/examples/personinfo/Person" + ); assert_eq!( cv.canonical_uri().to_string(), "https://w3id.org/linkml/Person" From 11cf89d173e7ac3eddaf0b49701f1e343918d672 Mon Sep 17 00:00:00 2001 From: ejsyx Date: Wed, 19 Aug 2026 12:17:36 +0200 Subject: [PATCH 37/58] fix(runtime): canonicalize designator values and IRI-expand identity labels MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Spec addendum rule 2 (findings D1 + D6): identity compares meaning, not spelling. Two halves that only work together, so they land together. **Designator values are canonicalised at the boxing chokepoint.** A `uriorcurie`-ranged designator can say "this is a `Circle`" as `canon:Circle`, as the expanded IRI, or by saying nothing at all; stored verbatim those were three different strings, so the same element authored by two producers diffed as a whole-element Remove + Add, and the RDF loader — which never harvests the designator predicate and always fills it from the class — disagreed with the JSON loader on every document. `canonicalize_type_designator` runs at all four `Object`-building sites (`parse_object_fixed_class`, `parse_object_value`, both arms of `build_mapping_entry_for_slot`), mirroring what `coerce_scalar_to_range` does for the int/float ambiguity. A supplied value that is no accepted designator value of the selected class is a load-time **warning**, not an error, and the slot then carries the canonical value — the loader-tolerance precedent, and the same posture rule 5 takes for a disagreeing dict key. This is also what makes diff's changed-key check ("a changed designator means a different class", spec rule 3) unconditionally true instead of true only for consistently-spelled data; no change was needed there. **Identity-label components that denote IRIs are expanded before comparison.** `ex:WGS84` and `https://example.org/canon/WGS84` are one IRI. The expansion lives in `scalar_slot_string`, the single function every identity label is built from, so all resolve sites move at once: diff emission, `resolve_list_segment` for patch and navigate, and the instance lint. The segments diff emits are then exactly what the resolver computes, by construction rather than by three coincidences; a round-trip test over a mixed-spelling pair pins it. A bare name and a CURIE with an unregistered prefix are left verbatim — inventing an expansion would rename identities the schema never claimed were IRIs. Only the identity *comparison* is normalised: the stored value of an ordinary `uri`-ranged slot is the author's data, unlike a designator, whose value is a function of the class. `example_personinfo_data{,_2}.yaml` spelled `objecttype` with the schema-native URI of classes that declare a `class_uri`; canonicalisation rewrites those on load, so the round-trip fixture pair now spells the canonical value it round-trips to. The other personinfo fixtures are left as they are, as live evidence of the canonicalisation. Downstream differential harness: corpus A byte-identical, all 95 self-diffs empty, and the `designator-to-curie` mutation — re-spelling every `typeURI` as a CURIE — drops from 46 non-zero diffs to zero across all 59 derivable instances. The 13 instances whose classes declare neither `id` nor `typeURI` were already at zero and did not move. Co-Authored-By: Claude Fable 5 --- src/runtime/src/diff.rs | 38 +- src/runtime/src/lib.rs | 95 ++++ .../tests/data/example_personinfo_data.yaml | 6 +- .../tests/data/example_personinfo_data_2.yaml | 8 +- .../tests/data/identity_canonical.yaml | 99 ++++ .../tests/identity_canonicalization.rs | 487 ++++++++++++++++++ src/schemaview/src/slotview.rs | 12 + 7 files changed, 736 insertions(+), 9 deletions(-) create mode 100644 src/runtime/tests/data/identity_canonical.yaml create mode 100644 src/runtime/tests/identity_canonicalization.rs diff --git a/src/runtime/src/diff.rs b/src/runtime/src/diff.rs index 7b142a2..e293857 100644 --- a/src/runtime/src/diff.rs +++ b/src/runtime/src/diff.rs @@ -1,6 +1,7 @@ use crate::{LResult, LinkMLInstance, NodeId, ValidationResultSink}; use linkml_schemaview::{ converter::Converter, + identifier::Identifier, schemaview::{ClassView, SchemaView, SlotView}, }; use serde::{Deserialize, Serialize}; @@ -37,13 +38,46 @@ pub(crate) fn slot_is_opaque(slot: &SlotView) -> bool { .unwrap_or(false) } +/// One component of an element's identity label, canonicalised so that +/// identity compares meaning and not spelling (spec addendum rule 2, D6). +/// +/// A slot whose range descends from `uri`/`uriorcurie` holds an IRI, and `ex:WGS84` +/// and `https://example.org/canon/WGS84` are the same IRI. Compared as raw +/// strings they are two identities: the same element re-spelled diffs as a +/// Remove + Add, `navigate_path` by the expansion misses the CURIE-spelled +/// element, and the instance lint calls a genuine duplicate unique. +/// +/// The expansion lives here, in the one function every identity label is built +/// from, so all four resolve sites move together: diff emission (via +/// [`element_key_label`] / [`element_unique_key_label`]), [`resolve_list_segment`] +/// for `patch` and `navigate_path`, and the instance lint. That is what keeps +/// "the segments diff emits are exactly what the resolver computes" true by +/// construction rather than by three coincidences. +/// +/// A bare name (no `:` at all) and a CURIE with an unregistered prefix are left +/// verbatim: `Identifier::to_uri` refuses them, and inventing an expansion +/// against the default prefix would rename identities the schema never claimed +/// were IRIs. +fn canonical_identity_component(raw: &str, slot: &SlotView) -> String { + if !slot.is_range_iri() { + return raw.to_string(); + } + let Some(conv) = slot.sv.converter_for_schema(slot.schema_id()) else { + return raw.to_string(); + }; + match Identifier::new(raw).to_uri(&conv) { + Ok(uri) => uri.0, + Err(_) => raw.to_string(), + } +} + pub(crate) fn scalar_slot_string( values: &std::collections::HashMap, slot_name: &str, ) -> Option { - if let Some(LinkMLInstance::Scalar { value, .. }) = values.get(slot_name) { + if let Some(LinkMLInstance::Scalar { value, slot, .. }) = values.get(slot_name) { return match value { - JsonValue::String(s) => Some(s.clone()), + JsonValue::String(s) => Some(canonical_identity_component(s, slot)), other => Some(other.to_string()), }; } diff --git a/src/runtime/src/lib.rs b/src/runtime/src/lib.rs index 4abd490..df62b9a 100644 --- a/src/runtime/src/lib.rs +++ b/src/runtime/src/lib.rs @@ -865,10 +865,87 @@ impl LinkMLInstance { .unwrap_or_else(|| base.clone()) } + /// Rewrite an already-present type designator value to the canonical value + /// of the class that was actually selected — identity compares meaning, not + /// spelling (spec addendum rule 2, finding D1). + /// + /// A designator says "this element is a `Circle`", and a `uriorcurie`-ranged + /// one can say it as `canon:Circle`, as the expanded IRI, or not at all. + /// Stored verbatim, those are three different strings: the same element + /// authored by two producers diffs as a whole-element Remove + Add, and the + /// RDF loader — which never harvests the designator predicate and always + /// fills it from the class ([`Self::populate_type_designator`], called from + /// `turtle_import`) — disagrees with the JSON loader on every document. + /// Canonicalising at the boxing chokepoint, exactly as + /// [`Self::coerce_scalar_to_range`] does for the int/float ambiguity, makes + /// the representation canonical before any diff/equals/patch/`to_json` sees + /// it, and makes the two loaders agree by construction. + /// + /// It is also what makes `diff`'s changed-key check ("a changed designator + /// means a different class") unconditionally true rather than true only for + /// consistently-spelled data. + /// + /// A supplied value that is *no* accepted designator value of the selected + /// class (a bare class name against a `uri` range, a typo, a stale class + /// name) is data the loader cannot honour. Following the loader-tolerance + /// precedent it is a **warning**, not an error: the instance still loads and + /// the slot carries the canonical value of the class `select_class` picked. + /// + /// Only a `Scalar` is rewritten. An explicit `null` and a structurally wrong + /// value (list, object) are left for the range checks that already ran to + /// describe; replacing them would hide the shape of the offending data. + fn canonicalize_type_designator( + values: &mut HashMap, + class: &ClassView, + conv: &Converter, + path: &[String], + validation_issues: &mut ValidationResultSink, + ) { + let Some(type_slot_def) = class.get_type_designator_slot() else { + return; + }; + let slot_name = type_slot_def.name.clone(); + let Ok(canonical) = class.get_type_designator_value(type_slot_def, conv) else { + return; + }; + let canonical = canonical.to_string(); + let accepted = class + .get_accepted_type_designator_values(type_slot_def, conv) + .unwrap_or_default(); + let Some(LinkMLInstance::Scalar { value, .. }) = values.get_mut(&slot_name) else { + return; + }; + let supplied = match &*value { + JsonValue::String(s) => s.clone(), + other => other.to_string(), + }; + if supplied == canonical { + return; + } + if !accepted.iter().any(|v| v.to_string() == supplied) { + let mut p = path.to_vec(); + p.push(slot_name); + validation_issues.push_warning( + ValidationProblemType::SlotRangeViolation, + p, + format!( + "type designator value `{supplied}` is not an accepted designator \ + value for class `{}`; stored as `{canonical}`", + class.name() + ), + ); + } + *value = JsonValue::String(canonical); + } + /// If the class has a `designates_type` slot and the values map does not /// already contain it, insert a Scalar value with the class's type /// designator value. This ensures round-trip fidelity for formats like /// JSON that lack an intrinsic typing mechanism (unlike RDF's rdf:type). + /// + /// The complement of [`Self::canonicalize_type_designator`], which settles + /// the case where the data *did* supply a value; JSON boxing runs both, RDF + /// harvesting only ever needs this one (it skips the designator predicate). fn populate_type_designator( values: &mut HashMap, class: &ClassView, @@ -943,6 +1020,7 @@ impl LinkMLInstance { path.clone(), validation_issues, ); + Self::canonicalize_type_designator(&mut values, class, conv, &path, validation_issues); Self::populate_type_designator(&mut values, class, sv, conv); Ok(LinkMLInstance::Object { node_id: new_node_id(), @@ -1180,6 +1258,7 @@ impl LinkMLInstance { path.clone(), validation_issues, ); + Self::canonicalize_type_designator(&mut values, &chosen, conv, &path, validation_issues); Self::populate_type_designator(&mut values, &chosen, sv, conv); Ok(LinkMLInstance::Object { node_id: new_node_id(), @@ -1464,6 +1543,13 @@ impl LinkMLInstance { path.clone(), validation_issues, ); + Self::canonicalize_type_designator( + &mut child_values, + &selected, + conv, + &path, + validation_issues, + ); Self::populate_type_designator(&mut child_values, &selected, sv, conv); Ok(LinkMLInstance::Object { node_id: new_node_id(), @@ -1507,6 +1593,15 @@ impl LinkMLInstance { path.clone(), validation_issues, ); + // `find_scalar_slot_for_inlined_map` picks the first non-key + // scalar slot, which can be the designator itself. + Self::canonicalize_type_designator( + &mut child_values, + &range_cv, + conv, + &path, + validation_issues, + ); Self::populate_type_designator(&mut child_values, &range_cv, sv, conv); Ok(LinkMLInstance::Object { node_id: new_node_id(), diff --git a/src/runtime/tests/data/example_personinfo_data.yaml b/src/runtime/tests/data/example_personinfo_data.yaml index 7c847a1..cff2701 100644 --- a/src/runtime/tests/data/example_personinfo_data.yaml +++ b/src/runtime/tests/data/example_personinfo_data.yaml @@ -1,14 +1,14 @@ objects: - id: T - objecttype: https://w3id.org/linkml/examples/personinfo/Organization + objecttype: http://schema.org/Organization name: club - id: P:001 name: fred bloggs - objecttype: https://w3id.org/linkml/examples/personinfo/Person + objecttype: http://schema.org/Person primary_email: fred.bloggs@example.com age_in_years: 33 - id: P:002 - objecttype: https://w3id.org/linkml/examples/personinfo/Person + objecttype: http://schema.org/Person name: joe schmoe primary_email: joe.schmoe@example.com has_employment_history: diff --git a/src/runtime/tests/data/example_personinfo_data_2.yaml b/src/runtime/tests/data/example_personinfo_data_2.yaml index c2451c0..8e41988 100644 --- a/src/runtime/tests/data/example_personinfo_data_2.yaml +++ b/src/runtime/tests/data/example_personinfo_data_2.yaml @@ -1,14 +1,14 @@ objects: - id: T - objecttype: https://w3id.org/linkml/examples/personinfo/Organization + objecttype: http://schema.org/Organization name: club - id: P:001 name: fred bloggs - objecttype: https://w3id.org/linkml/examples/personinfo/Person + objecttype: http://schema.org/Person primary_email: fred.bloggs@example.com age_in_years: 33 - id: P:002 - objecttype: https://w3id.org/linkml/examples/personinfo/Person + objecttype: http://schema.org/Person name: joe schmoe primary_email: joe.schmoe@example.com has_employment_history: @@ -34,7 +34,7 @@ objects: name: trepanation objecttype: https://w3id.org/linkml/examples/personinfo/ProcedureConcept - id: P:003 - objecttype: https://w3id.org/linkml/examples/personinfo/Person + objecttype: http://schema.org/Person name: alice smith primary_email: alice.smith@example.com age_in_years: 34 diff --git a/src/runtime/tests/data/identity_canonical.yaml b/src/runtime/tests/data/identity_canonical.yaml new file mode 100644 index 0000000..63d4b87 --- /dev/null +++ b/src/runtime/tests/data/identity_canonical.yaml @@ -0,0 +1,99 @@ +id: https://w3id.org/linkml/examples/identity_canonical +name: identity_canonical +description: |- + Fixture for spec addendum rule 2 — identity compares meaning, not spelling. + + Two shapes are exercised: + + * a type designator (`Shape.typeURI`, `Node.kind`, `Widget.typeURI`) whose + value data may spell as a CURIE, as a full URI, or omit entirely — all + three must box to the one canonical value the class declares, and a value + that is no accepted designator value at all must load with a warning. + * an identity component whose range descends from `uri`/`uriorcurie` + (`System.systemType`), where a CURIE and its expansion are one identity. +license: https://creativecommons.org/publicdomain/zero/1.0/ +imports: + - linkml:types +prefixes: + canon: https://w3id.org/linkml/examples/identity_canonical/ + ex: https://example.org/canon/ + linkml: https://w3id.org/linkml/ +default_prefix: canon +default_range: string + +classes: + Container: + attributes: + # polymorphic list; identity is unique_keys[label], the designator is not + shapes: + range: Shape + multivalued: true + inlined_as_list: true + # designator on a `uri`-ranged slot: canonical form is the expanded URI + nodes: + range: Node + multivalued: true + inlined_as_list: true + # dict form, so `build_mapping_entry_for_slot` is exercised too + widgets: + range: Widget + multivalued: true + inlined: true + # D6: identity component whose range descends from `uri` + systems: + range: System + multivalued: true + inlined_as_list: true + + Shape: + unique_keys: + by_label: + unique_key_slots: [label] + attributes: + typeURI: + range: uriorcurie + designates_type: true + label: {range: string, required: true} + + Circle: + is_a: Shape + attributes: + radius: {range: float} + + Node: + unique_keys: + by_name: + unique_key_slots: [name] + attributes: + kind: + range: uri + designates_type: true + name: {range: string, required: true} + + # declares a `class_uri`, so its canonical designator value and its + # schema-native URI differ — data may legitimately spell either + LeafNode: + is_a: Node + class_uri: ex:Leaf + attributes: + depth: {range: integer} + + Widget: + attributes: + wid: {range: string, key: true, required: true} + typeURI: + range: uriorcurie + designates_type: true + + FancyWidget: + is_a: Widget + attributes: + sparkle: {range: string} + + System: + unique_keys: + by_system_type: + unique_key_slots: [systemType] + attributes: + systemType: {range: uri, required: true} + value: {range: string} diff --git a/src/runtime/tests/identity_canonicalization.rs b/src/runtime/tests/identity_canonicalization.rs new file mode 100644 index 0000000..d455e4d --- /dev/null +++ b/src/runtime/tests/identity_canonicalization.rs @@ -0,0 +1,487 @@ +//! Spec addendum rule 2 — identity compares meaning, not spelling (D1 + D6). +//! +//! Two halves, pinned together because they only hold together: +//! +//! * designator values are canonicalised at the boxing chokepoint, so every +//! spelling of "this element is a `Circle`" becomes the one value the class +//! declares, and `to_json` emits it; +//! * identity-label components whose range descends from `uri`/`uriorcurie` +//! are IRI-expanded before comparison at *every* resolve site, so a CURIE and +//! its expansion are one identity in diff, patch, navigate and the lint. + +use linkml_runtime::{ + diff, lint_instance_identity, load_json_str, patch, Delta, DeltaOp, DiffOptions, + LinkMLInstance, PatchOptions, ValidationProblemType, ValidationResult, +}; +use linkml_schemaview::identifier::{converter_from_schema, Identifier}; +use linkml_schemaview::io::from_yaml; +use linkml_schemaview::schemaview::{ClassView, SchemaView}; +use linkml_schemaview::Converter; +use serde_json::{json, Value as JsonValue}; +use std::path::PathBuf; + +const CIRCLE_CURIE: &str = "canon:Circle"; +/// `LeafNode`'s canonical designator value: its `class_uri`, expanded (the slot +/// range is `uri`). +const LEAF_URI: &str = "https://example.org/canon/Leaf"; +/// The same class's *schema-native* URI — a different spelling of the same +/// meaning, and one the accepted-value set must recognise. +const LEAF_NATIVE_URI: &str = "https://w3id.org/linkml/examples/identity_canonical/LeafNode"; +const FANCY_CURIE: &str = "canon:FancyWidget"; +const WGS84_CURIE: &str = "ex:WGS84"; +const WGS84_URI: &str = "https://example.org/canon/WGS84"; +const ETRS89_CURIE: &str = "ex:ETRS89"; +const ETRS89_URI: &str = "https://example.org/canon/ETRS89"; + +struct Fixture { + sv: SchemaView, + conv: Converter, + container: ClassView, +} + +fn fixture() -> Fixture { + let mut p = PathBuf::from(env!("CARGO_MANIFEST_DIR")); + p.push("tests/data/identity_canonical.yaml"); + let schema = from_yaml(&p).unwrap(); + let mut sv = SchemaView::new(); + sv.add_schema(schema.clone()).unwrap(); + let conv = converter_from_schema(&schema); + let container = sv + .get_class(&Identifier::new("Container"), &conv) + .unwrap() + .expect("class not found"); + Fixture { + sv, + conv, + container, + } +} + +impl Fixture { + fn load_result(&self, v: JsonValue) -> (LinkMLInstance, Vec) { + let r = load_json_str(&v.to_string(), &self.sv, &self.container, &self.conv).unwrap(); + let issues = r.validation_issues.clone(); + (r.into_instance().expect("instance must load"), issues) + } + fn load(&self, v: JsonValue) -> LinkMLInstance { + self.load_result(v).0 + } + fn json(&self, v: JsonValue) -> JsonValue { + self.load(v).to_json() + } +} + +fn diff2(f: &Fixture, before: JsonValue, after: JsonValue) -> Vec { + diff(&f.load(before), &f.load(after), DiffOptions::new(true)) +} + +fn only(deltas: &[Delta]) -> &Delta { + assert_eq!(deltas.len(), 1, "expected exactly one delta: {deltas:#?}"); + &deltas[0] +} + +fn systems(items: Vec) -> JsonValue { + json!({ "systems": items }) +} + +// --------------------------------------------------------------------------- +// D1 — designator values canonicalised at load +// --------------------------------------------------------------------------- + +/// CURIE, full URI and omitted-entirely are three spellings of one fact. All +/// three must box to the class's canonical designator value, and `to_json` +/// must emit that value. +#[test] +fn every_designator_spelling_boxes_to_the_one_canonical_value() { + let f = fixture(); + let out = f.json(json!({ + "shapes": [ + {"typeURI": CIRCLE_CURIE, "label": "a", "radius": 1.0}, + {"typeURI": "https://w3id.org/linkml/examples/identity_canonical/Circle", + "label": "b", "radius": 2.0}, + {"label": "c", "radius": 3.0} + ] + })); + let shapes = out.get("shapes").and_then(JsonValue::as_array).unwrap(); + let spellings: Vec<&JsonValue> = shapes.iter().map(|s| &s["typeURI"]).collect(); + assert_eq!( + spellings, + vec![ + &json!(CIRCLE_CURIE), + &json!(CIRCLE_CURIE), + &json!(CIRCLE_CURIE) + ], + "one canonical designator value, whatever the data spelled: {out:#?}" + ); +} + +/// The canonical form follows the designator slot's *range*: a `uri`-ranged +/// designator canonicalises to the expanded URI, not to a CURIE. +/// +/// `LeafNode` declares a `class_uri`, so it has two legitimate URI spellings and +/// two CURIE spellings. All four — and the omitted case — mean the same class +/// and must land on the one canonical value, silently. +#[test] +fn uri_ranged_designator_canonicalises_to_the_expanded_class_uri() { + let f = fixture(); + let doc = json!({ + "nodes": [ + {"kind": "canon:LeafNode", "name": "n1", "depth": 1}, + {"kind": LEAF_NATIVE_URI, "name": "n2", "depth": 2}, + {"kind": "ex:Leaf", "name": "n3", "depth": 3}, + {"kind": LEAF_URI, "name": "n4", "depth": 4}, + {"name": "n5", "depth": 5} + ] + }); + let (inst, issues) = f.load_result(doc); + assert!( + issues.is_empty(), + "every spelling above is an accepted designator value: {issues:#?}" + ); + let out = inst.to_json(); + let nodes = out.get("nodes").and_then(JsonValue::as_array).unwrap(); + assert_eq!(nodes.len(), 5); + for n in nodes { + assert_eq!(n["kind"], json!(LEAF_URI), "in {out:#?}"); + } +} + +/// The dict (mapping) boxing chokepoint canonicalises too — `build_mapping_ +/// entry_for_slot` builds objects the list path never sees. +#[test] +fn dict_form_designator_is_canonicalised() { + let f = fixture(); + let out = f.json(json!({ + "widgets": { + "w1": {"wid": "w1", "typeURI": + "https://w3id.org/linkml/examples/identity_canonical/FancyWidget", + "sparkle": "yes"}, + "w2": {"wid": "w2", "typeURI": FANCY_CURIE, "sparkle": "no"} + } + })); + let widgets = out.get("widgets").unwrap(); + assert_eq!(widgets["w1"]["typeURI"], json!(FANCY_CURIE), "in {out:#?}"); + assert_eq!(widgets["w2"]["typeURI"], json!(FANCY_CURIE), "in {out:#?}"); +} + +/// A designator value matching *no* accepted designator value is data the +/// loader cannot honour. Following the loader-tolerance precedent it is a +/// warning, not an error: the instance still loads, and the slot is filled +/// with the canonical value of the class that was actually selected. +#[test] +fn junk_designator_value_warns_and_is_replaced_by_the_canonical_value() { + let f = fixture(); + let (inst, issues) = f.load_result(json!({ + "shapes": [{"typeURI": "Circle", "label": "a", "radius": 1.0}] + })); + let out = inst.to_json(); + assert_eq!( + out["shapes"][0]["typeURI"], + json!(CIRCLE_CURIE), + "junk designator must be replaced by the canonical value: {out:#?}" + ); + assert!( + !issues.iter().any(|i| i.severity.is_error()), + "junk designator must not be an error: {issues:#?}" + ); + let warned: Vec<&ValidationResult> = issues + .iter() + .filter(|i| { + i.problem_type == ValidationProblemType::SlotRangeViolation + && i.subject.last().map(String::as_str) == Some("typeURI") + }) + .collect(); + assert_eq!( + warned.len(), + 1, + "exactly one designator warning expected: {issues:#?}" + ); + assert!( + warned[0].detail.contains("Circle"), + "the warning must name the rejected value: {:?}", + warned[0].detail + ); +} + +/// The canonical spelling is what makes a *matching* spelling silent: nothing +/// warns when the data already spelled an accepted value. +#[test] +fn accepted_designator_spellings_do_not_warn() { + let f = fixture(); + let (_, issues) = f.load_result(json!({ + "shapes": [ + {"typeURI": CIRCLE_CURIE, "label": "a", "radius": 1.0}, + {"typeURI": "https://w3id.org/linkml/examples/identity_canonical/Circle", + "label": "b", "radius": 2.0}, + {"label": "c", "radius": 3.0} + ] + })); + assert!(issues.is_empty(), "no diagnostics expected: {issues:#?}"); +} + +// --------------------------------------------------------------------------- +// D6 — uri/uriorcurie identity components compared as IRIs +// --------------------------------------------------------------------------- + +/// A reorder is invisible even when the list is addressed by IRI labels and the +/// document mixes spellings — the labels are stable, the order is not identity. +#[test] +fn mixed_spelling_reorder_is_a_zero_delta_diff() { + let f = fixture(); + let deltas = diff2( + &f, + systems(vec![ + json!({"systemType": WGS84_CURIE, "value": "one"}), + json!({"systemType": ETRS89_URI, "value": "two"}), + ]), + systems(vec![ + json!({"systemType": ETRS89_URI, "value": "two"}), + json!({"systemType": WGS84_CURIE, "value": "one"}), + ]), + ); + assert!(deltas.is_empty(), "reorder must be invisible: {deltas:#?}"); +} + +/// The label a keyed list is addressed by is the *expanded* IRI, even when the +/// document spells the CURIE. This is what every other site then agrees with. +#[test] +fn identity_path_segments_are_expanded_iris() { + let f = fixture(); + let deltas = diff2( + &f, + systems(vec![json!({"systemType": WGS84_CURIE, "value": "one"})]), + systems(vec![json!({"systemType": WGS84_CURIE, "value": "two"})]), + ); + let d = only(&deltas); + assert_eq!(d.op, DeltaOp::Update); + assert_eq!( + d.path, + vec![ + "systems".to_string(), + WGS84_URI.to_string(), + "value".to_string() + ], + "path segments are IRIs: {deltas:#?}" + ); +} + +/// The two sides spell one element's identity differently. Rule 2 makes them +/// one identity, so the element is *matched* and the change is described +/// field-by-field under its IRI — never a Remove + Add of the whole element. +/// +/// The re-spelling itself stays a delta: only the identity *comparison* is +/// IRI-normalised. The stored value of an ordinary `uri`-ranged slot is the +/// author's data, unlike a type designator, whose value is a function of the +/// class and is therefore rewritten at load. +#[test] +fn respelled_element_is_matched_never_replaced() { + let f = fixture(); + let deltas = diff2( + &f, + systems(vec![json!({"systemType": WGS84_CURIE, "value": "one"})]), + systems(vec![json!({"systemType": WGS84_URI, "value": "two"})]), + ); + assert!( + deltas.iter().all(|d| d.op == DeltaOp::Update), + "no element churn: {deltas:#?}" + ); + let mut paths: Vec> = deltas.iter().map(|d| d.path.clone()).collect(); + paths.sort(); + assert_eq!( + paths, + vec![ + vec![ + "systems".to_string(), + WGS84_URI.to_string(), + "systemType".to_string() + ], + vec![ + "systems".to_string(), + WGS84_URI.to_string(), + "value".to_string() + ], + ], + "one element, addressed by its IRI: {deltas:#?}" + ); +} + +/// The invariant the two halves share: whatever segment diff emits, +/// `resolve_list_segment` must find — even when the document patch is applied +/// to spells the identity the other way round. +#[test] +fn patch_locates_the_element_under_spelling_drift() { + let f = fixture(); + let deltas = diff2( + &f, + systems(vec![json!({"systemType": WGS84_URI, "value": "one"})]), + systems(vec![json!({"systemType": WGS84_URI, "value": "two"})]), + ); + // The document being patched spells the same system as a CURIE. + let drifted = f.load(systems(vec![ + json!({"systemType": WGS84_CURIE, "value": "one"}), + ])); + let (patched, trace) = patch(&drifted, &deltas, PatchOptions::default()).unwrap(); + assert!(trace.failed.is_empty(), "delta must locate: {trace:#?}"); + assert_eq!(patched.to_json()["systems"][0]["value"], json!("two")); +} + +/// `navigate_path` shares the resolver, so either spelling addresses the +/// element regardless of how the document spells it. +#[test] +fn navigate_finds_the_element_by_the_expanded_iri() { + let f = fixture(); + let inst = f.load(systems(vec![ + json!({"systemType": WGS84_CURIE, "value": "one"}), + json!({"systemType": ETRS89_CURIE, "value": "two"}), + ])); + let hit = inst + .navigate_path(["systems", WGS84_URI, "value"]) + .expect("expanded IRI must address the CURIE-spelled element"); + assert_eq!(hit.to_json(), json!("one")); +} + +/// Two spellings that expand to the same IRI are a duplicate identity, and the +/// instance lint is the voice that says so. +#[test] +fn instance_lint_sees_a_duplicate_across_spellings() { + let f = fixture(); + let inst = f.load(systems(vec![ + json!({"systemType": WGS84_CURIE, "value": "one"}), + json!({"systemType": WGS84_URI, "value": "two"}), + ])); + let warnings = lint_instance_identity(&inst); + let dups: Vec<&ValidationResult> = warnings + .iter() + .filter(|w| w.problem_type == ValidationProblemType::DuplicateElementIdentity) + .collect(); + assert_eq!( + dups.len(), + 1, + "a curie and its expansion collide: {warnings:#?}" + ); + assert!( + dups[0].detail.contains(WGS84_URI), + "the duplicate is reported by its expanded IRI: {:?}", + dups[0].detail + ); +} + +/// Explicit round-trip pin for the shared-rule invariant: the segments diff +/// emits over a mixed-spelling pair are exactly what the resolver computes, so +/// every one of them addresses a real node and re-applying the diff reproduces +/// the target (list order is not identity, so the comparison is order-free). +#[test] +fn mixed_spelling_diff_segments_round_trip_through_the_resolver() { + let f = fixture(); + let before = systems(vec![ + json!({"systemType": WGS84_CURIE, "value": "one"}), + json!({"systemType": ETRS89_URI, "value": "two"}), + ]); + let after = systems(vec![ + json!({"systemType": ETRS89_CURIE, "value": "TWO"}), + json!({"systemType": WGS84_URI, "value": "ONE"}), + json!({"systemType": "ex:LAMBERT", "value": "three"}), + ]); + let deltas = diff2(&f, before.clone(), after.clone()); + assert!(!deltas.is_empty(), "there is real change here"); + let src = f.load(before); + for d in &deltas { + // Every emitted path but that of an Add must address an existing node. + if d.op != DeltaOp::Add { + assert!( + src.navigate_path(d.path.iter()).is_some(), + "diff emitted a segment its own resolver cannot follow: {:?}", + d.path + ); + } + } + let (patched, trace) = patch(&src, &deltas, PatchOptions::default()).unwrap(); + assert!(trace.failed.is_empty(), "{trace:#?}"); + assert_eq!( + sorted_systems(&patched.to_json()), + sorted_systems(&f.json(after)) + ); +} + +/// `systems` as a set: keyed patching edits in place, so the surviving element +/// order is the source's, and order is not what a keyed list is compared by. +fn sorted_systems(v: &JsonValue) -> Vec { + let mut out: Vec = v["systems"] + .as_array() + .unwrap() + .iter() + .map(JsonValue::to_string) + .collect(); + out.sort(); + out +} + +// --------------------------------------------------------------------------- +// JSON / RDF agreement +// --------------------------------------------------------------------------- + +/// The RDF loader has always written the canonical designator (it never +/// harvests the designator predicate, it fills it from the class). Once JSON +/// canonicalises too, the two loaders agree on a document that spelled the +/// designator non-canonically. +#[cfg(feature = "ttl")] +#[test] +fn json_and_rdf_loaders_agree_on_the_designator_spelling() { + use linkml_runtime::rdf_import::{import_turtle, ImportOptions}; + use linkml_runtime::turtle::{turtle_to_string, TurtleOptions}; + + let mut p = PathBuf::from(env!("CARGO_MANIFEST_DIR")); + p.push("tests/data/identity_canonical.yaml"); + let schema = from_yaml(&p).unwrap(); + let mut sv = SchemaView::new(); + sv.add_schema(schema.clone()).unwrap(); + let conv = converter_from_schema(&schema); + let container = sv + .get_class(&Identifier::new("Container"), &conv) + .unwrap() + .expect("class not found"); + + let doc = json!({ + "shapes": [{ + "typeURI": "https://w3id.org/linkml/examples/identity_canonical/Circle", + "label": "a", + "radius": 1.0 + }] + }); + let from_json = load_json_str(&doc.to_string(), &sv, &container, &conv) + .unwrap() + .into_instance() + .unwrap(); + + let ttl = turtle_to_string( + &from_json, + &sv, + &schema, + &conv, + TurtleOptions { skolem: false }, + ) + .unwrap(); + let stream = import_turtle( + std::io::Cursor::new(ttl.as_bytes()), + sv.clone(), + conv.clone(), + &["Container"], + ImportOptions::default(), + ) + .unwrap(); + let from_rdf = stream + .filter_map(|r| r.ok()) + .find_map(|(c, i)| if c == "Container" { Some(i) } else { None }) + .expect("one Container"); + + assert_eq!( + from_rdf.to_json()["shapes"][0]["typeURI"], + json!(CIRCLE_CURIE), + "the RDF loader's canonical designator" + ); + assert_eq!( + from_json.to_json()["shapes"][0]["typeURI"], + from_rdf.to_json()["shapes"][0]["typeURI"], + "JSON and RDF loaders must agree" + ); +} diff --git a/src/schemaview/src/slotview.rs b/src/schemaview/src/slotview.rs index 7beadc3..0100bd5 100644 --- a/src/schemaview/src/slotview.rs +++ b/src/schemaview/src/slotview.rs @@ -538,6 +538,18 @@ impl SlotView { .is_some_and(|ri| ri.is_integer()) } + /// Returns `true` when the primary range's type hierarchy contains `uri` or + /// `uriorcurie` — the value denotes an IRI, whatever it is spelled as. + /// + /// RDF serialization uses this to emit a named node rather than a literal; + /// the runtime's identity machinery uses it to know that a CURIE and its + /// expansion are the same value and must compare equal. + pub fn is_range_iri(&self) -> bool { + self.get_range_info() + .first() + .is_some_and(|ri| ri.is_range_iri) + } + /// Returns the resolved container shape for this slot. /// /// This resolves the interacting `multivalued`, `inlined`, and From b2f2cb6c7ee7b61f63ebfb49afffd9f6d9c2bf0e Mon Sep 17 00:00:00 2001 From: ejsyx Date: Wed, 19 Aug 2026 12:36:30 +0200 Subject: [PATCH 38/58] fix(runtime): canonicalization cost, dict-arm class selection, segment symmetry MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Review follow-up on 11cf89d. **Cost.** `canonicalize_type_designator` computed the canonical value *and* the full accepted set before it had even looked for the slot, so every object of a designator-carrying class paid two uncached `type_ancestors` walks and up to five `get_uri` calls — including the two commonest cases, an absent designator and an already-canonical one, where the answer is "do nothing". Now: find the scalar, read it, compare against the canonical value; compute the accepted set only when the two differ. An `Err` from `get_accepted_type_designator_values` also no longer collapses to "nothing is accepted" — an unknown accepted set is not an empty one, and treating it as empty warned about, and rewrote, data that may well have been right. It returns early, the same posture the canonical-value line takes. **Dict scalar arm.** `build_mapping_entry_for_slot`'s compact arm hardwires the slot's range class, but `find_scalar_slot_for_inlined_map` picks the first non-key scalar slot — which can be the designator. `{"w1": "canon:FancyWidget"}` then names its own class exactly as the object form does, and canonicalising against the range class rewrote it to `Widget` and warned: the misclassification 2b3f918 exists to prevent, reintroduced one arm over. That arm now selects the class from the scalar, scoped to the designator shape only — a compact entry whose scalar is an ordinary slot names no class and keeps the range class exactly as before. **Segment symmetry.** Labels were IRI-expanded on the way out but `resolve_list_segment` compared the caller's segment raw, so a stored delta or hand-written patch spelling a CURIE landed in `trace.failed` against expanded labels. Spec rule 2 says a curie and its expansion are one identity, in both directions. `segment_matches_label` normalises the incoming segment through the *same* slot the label came from (`identity_label_slot`, mirroring `element_identity_label`'s precedence), so the comparison is symmetric. Segments diff emits equal the label outright and never reach the expansion. Also: `identity_canonical.yaml` gains `BareLeaf` — a subclass with a `class_uri` and no distinguishing slot, so its schema-native URI is the only thing that can name it. `a_subclass_is_selected_by_its_native_uri_alone` fails loudly if `ClassView::get_uri(native, expand)` ever regresses; verified by reverting that line. And `validation_issue_paths_include_list_indices_once` pins `SlotRangeViolation` again rather than filtering nothing. Harness output is byte-identical to the 11cf89d capture: none of these five changes moves the downstream corpus. Co-Authored-By: Claude Fable 5 --- src/runtime/src/diff.rs | 58 ++++++++++++- src/runtime/src/lib.rs | 61 +++++++++---- .../tests/data/identity_canonical.yaml | 15 +++- .../tests/identity_canonicalization.rs | 87 +++++++++++++++++++ src/runtime/tests/validation.rs | 11 ++- 5 files changed, 208 insertions(+), 24 deletions(-) diff --git a/src/runtime/src/diff.rs b/src/runtime/src/diff.rs index e293857..10c7e59 100644 --- a/src/runtime/src/diff.rs +++ b/src/runtime/src/diff.rs @@ -152,6 +152,57 @@ pub(crate) fn element_identity_label(v: &LinkMLInstance) -> Option { element_key_label(v).or_else(|| element_unique_key_label(v)) } +/// The single slot `v`'s identity label was read from, when the label *is* one +/// scalar: a key/identifier, or a one-slot `unique_keys` entry. A composite +/// `unique_keys` entry encodes a JSON array and has no single source slot. +/// +/// Mirrors [`element_identity_label`]'s precedence exactly, including its +/// fall-through when the key slot carries no value — the point is to name the +/// slot that produced the label, so that a path segment can be normalised the +/// same way the label was. +fn identity_label_slot(v: &LinkMLInstance) -> Option<&SlotView> { + let LinkMLInstance::Object { values, class, .. } = v else { + return None; + }; + if let Some(slot) = identity_key_slot(class) { + if scalar_slot_string(values, &slot.name).is_some() { + return Some(slot); + } + } + let uks = class.unique_keys(); + let (_, uk) = uks.iter().find(|(_, uk)| !uk.unique_key_slots.is_empty())?; + let [only] = uk.unique_key_slots.as_slice() else { + return None; + }; + class.slots().iter().find(|s| s.name == *only) +} + +/// Does the path segment `key` address the element `v`, whose identity label is +/// `label`? +/// +/// The other half of spec addendum rule 2's "a curie and its expansion are one +/// identity". Labels are already IRI-expanded on the way out +/// ([`canonical_identity_component`]); a segment arriving from outside — a +/// stored delta, a hand-written patch, a caller's `navigate_path` — has been +/// through no such thing. Normalising it through the *same* slot the label came +/// from makes the comparison symmetric, so `ex:WGS84` addresses an element +/// whose label expanded to `https://example.org/canon/WGS84` and vice versa. +/// +/// Segments diff itself emits already equal the label outright and never reach +/// the expansion. +fn segment_matches_label(v: &LinkMLInstance, label: Option<&str>, key: &str) -> bool { + let Some(label) = label else { + return false; + }; + if label == key { + return true; + } + match identity_label_slot(v) { + Some(slot) => canonical_identity_component(key, slot) == label, + None => false, + } +} + fn labels_are_unique(elements: &[LinkMLInstance], label: F) -> bool where F: Fn(&LinkMLInstance) -> Option, @@ -762,8 +813,9 @@ pub(crate) fn resolve_list_segment(values: &[LinkMLInstance], key: &str) -> Opti // labels (e.g. a year as unique key) unambiguous: they resolve as // labels, never as positions. let labels: Vec> = values.iter().map(element_identity_label).collect(); + let matches = |i: usize| segment_matches_label(&values[i], labels[i].as_deref(), key); if list_is_keyed_shaped(values) { - return labels.iter().position(|l| l.as_deref() == Some(key)); + return (0..values.len()).find(|i| matches(*i)); } // Positional list: numeric index first (the segments diff produces for // these lists), then a single unambiguous label hit for drift tolerance. @@ -773,8 +825,8 @@ pub(crate) fn resolve_list_segment(values: &[LinkMLInstance], key: &str) -> Opti } } let mut hit: Option = None; - for (i, l) in labels.iter().enumerate() { - if l.as_deref() == Some(key) { + for i in 0..values.len() { + if matches(i) { if hit.is_some() { return None; } diff --git a/src/runtime/src/lib.rs b/src/runtime/src/lib.rs index df62b9a..3b09e27 100644 --- a/src/runtime/src/lib.rs +++ b/src/runtime/src/lib.rs @@ -904,27 +904,38 @@ impl LinkMLInstance { let Some(type_slot_def) = class.get_type_designator_slot() else { return; }; - let slot_name = type_slot_def.name.clone(); - let Ok(canonical) = class.get_type_designator_value(type_slot_def, conv) else { - return; - }; - let canonical = canonical.to_string(); - let accepted = class - .get_accepted_type_designator_values(type_slot_def, conv) - .unwrap_or_default(); - let Some(LinkMLInstance::Scalar { value, .. }) = values.get_mut(&slot_name) else { + // Cheapest checks first: the two commonest cases — the designator is + // absent (the RDF-shaped and the omit-it-and-let-the-loader-fill-it + // documents), or it is already canonical (every document a previous + // load emitted) — must not pay for a URI computation. Neither + // `get_type_designator_value` nor `get_accepted_type_designator_values` + // is cached: each walks `type_ancestors` and expands URIs through the + // converter, and this runs once per object built. + let Some(LinkMLInstance::Scalar { value, .. }) = values.get_mut(&type_slot_def.name) else { return; }; let supplied = match &*value { JsonValue::String(s) => s.clone(), other => other.to_string(), }; + let Ok(canonical) = class.get_type_designator_value(type_slot_def, conv) else { + return; + }; + let canonical = canonical.to_string(); if supplied == canonical { return; } + // Only a value that differs from the canonical one is worth the second + // walk. An `Err` here means the accepted set is unknown, not empty: + // warning-and-rewriting on it would punish the data for a schema the + // view cannot resolve, so leave the value alone — the same posture the + // canonical-value line above takes. + let Ok(accepted) = class.get_accepted_type_designator_values(type_slot_def, conv) else { + return; + }; if !accepted.iter().any(|v| v.to_string() == supplied) { let mut p = path.to_vec(); - p.push(slot_name); + p.push(type_slot_def.name.clone()); validation_issues.push_warning( ValidationProblemType::SlotRangeViolation, p, @@ -1575,6 +1586,24 @@ impl LinkMLInstance { ), ) })?; + // `find_scalar_slot_for_inlined_map` picks the first non-key + // scalar slot, which can be the class's *designator*: then + // `{"w1": "canon:FancyWidget"}` says "widget w1 is a + // FancyWidget", and the entry names its own class exactly as + // the object form's `{"typeURI": ...}` does. This arm otherwise + // hardwires the slot's range class, so without selecting here + // canonicalisation would rewrite that designator to the *range* + // class's value and warn about data that was right — the + // misclassification spec rule 2 exists to prevent. Selection is + // scoped to this one shape: a compact entry whose scalar is an + // ordinary slot names no class and keeps the range class. + let entry_class = if scalar_slot.definition().designates_type.unwrap_or(false) { + let mut named = serde_json::Map::new(); + named.insert(scalar_slot.name.clone(), other.clone()); + Self::select_class(&named, &range_cv, sv, conv) + } else { + range_cv.clone() + }; let mut child_values = HashMap::new(); child_values.insert( scalar_slot.name.clone(), @@ -1582,31 +1611,29 @@ impl LinkMLInstance { node_id: new_node_id(), value: Self::coerce_scalar_to_range(other, Some(&scalar_slot)), slot: scalar_slot.clone(), - class: Some(range_cv.clone()), + class: Some(entry_class.clone()), sv: sv.clone(), }, ); run_object_constraints( - &range_cv, + &entry_class, &child_values, &HashMap::new(), path.clone(), validation_issues, ); - // `find_scalar_slot_for_inlined_map` picks the first non-key - // scalar slot, which can be the designator itself. Self::canonicalize_type_designator( &mut child_values, - &range_cv, + &entry_class, conv, &path, validation_issues, ); - Self::populate_type_designator(&mut child_values, &range_cv, sv, conv); + Self::populate_type_designator(&mut child_values, &entry_class, sv, conv); Ok(LinkMLInstance::Object { node_id: new_node_id(), values: child_values, - class: range_cv, + class: entry_class, sv: sv.clone(), unknown_fields: HashMap::new(), }) diff --git a/src/runtime/tests/data/identity_canonical.yaml b/src/runtime/tests/data/identity_canonical.yaml index 63d4b87..a09c1b0 100644 --- a/src/runtime/tests/data/identity_canonical.yaml +++ b/src/runtime/tests/data/identity_canonical.yaml @@ -78,9 +78,22 @@ classes: attributes: depth: {range: integer} + # the guard for `ClassView::get_uri(native, expand)`: a `class_uri` and no + # distinguishing slot at all, so the ONLY thing that can select this class is + # its designator value. Spelled with its schema-native URI it is reachable + # only if that spelling is in the accepted set; if `get_uri(true, true)` ever + # returns the `class_uri` again, this class silently becomes `Node` and the + # designator gets rewritten to `Node`'s value. + BareLeaf: + is_a: Node + class_uri: ex:BareLeaf + + # `find_scalar_slot_for_inlined_map` picks the first non-key scalar slot — + # here the designator — so `{"w1": "canon:FancyWidget"}` is a legal compact + # dict entry that names its own class. Widget: attributes: - wid: {range: string, key: true, required: true} + wid: {range: string, key: true} typeURI: range: uriorcurie designates_type: true diff --git a/src/runtime/tests/identity_canonicalization.rs b/src/runtime/tests/identity_canonicalization.rs index d455e4d..e15c045 100644 --- a/src/runtime/tests/identity_canonicalization.rs +++ b/src/runtime/tests/identity_canonicalization.rs @@ -164,6 +164,57 @@ fn dict_form_designator_is_canonicalised() { assert_eq!(widgets["w2"]["typeURI"], json!(FANCY_CURIE), "in {out:#?}"); } +/// The *compact* dict entry — a bare scalar rather than an object — is built by +/// a second arm of `build_mapping_entry_for_slot` that hardwires the slot's +/// range class. When the scalar slot it fills is the class's designator, the +/// entry names its own class, and canonicalising against the range class would +/// rewrite `FancyWidget` to `Widget` and warn about data that was right. +#[test] +fn compact_dict_entry_naming_a_subclass_is_not_rewritten_to_the_range_class() { + let f = fixture(); + let (inst, issues) = f.load_result(json!({ + "widgets": { + "w1": "https://w3id.org/linkml/examples/identity_canonical/FancyWidget", + "w2": FANCY_CURIE + } + })); + assert!( + issues.is_empty(), + "the entry names an accepted designator value of a real subclass: {issues:#?}" + ); + let out = inst.to_json(); + let widgets = out.get("widgets").unwrap(); + assert_eq!( + widgets["w1"]["typeURI"], + json!(FANCY_CURIE), + "canonicalised, and to the *subclass* the entry named: {out:#?}" + ); + assert_eq!(widgets["w2"]["typeURI"], json!(FANCY_CURIE), "in {out:#?}"); +} + +/// The guard for `ClassView::get_uri(native, expand)`: `BareLeaf` declares a +/// `class_uri` and no distinguishing slot, so its schema-native URI is the only +/// thing that can name it. If that spelling ever drops out of the accepted set +/// the element silently becomes a `Node` and its designator is rewritten to +/// `Node`'s value — this test is what makes that loud. +#[test] +fn a_subclass_is_selected_by_its_native_uri_alone() { + let f = fixture(); + let (inst, issues) = f.load_result(json!({ + "nodes": [{ + "kind": "https://w3id.org/linkml/examples/identity_canonical/BareLeaf", + "name": "b1" + }] + })); + assert!(issues.is_empty(), "an accepted spelling: {issues:#?}"); + let out = inst.to_json(); + assert_eq!( + out["nodes"][0]["kind"], + json!("https://example.org/canon/BareLeaf"), + "selected as BareLeaf and canonicalised to its class_uri: {out:#?}" + ); +} + /// A designator value matching *no* accepted designator value is data the /// loader cannot honour. Following the loader-tolerance precedent it is a /// warning, not an error: the instance still loads, and the slot is filled @@ -325,6 +376,42 @@ fn patch_locates_the_element_under_spelling_drift() { assert_eq!(patched.to_json()["systems"][0]["value"], json!("two")); } +/// The symmetric half: a segment that arrives already spelled as a CURIE — a +/// stored delta, a hand-written patch — must address the element whose label +/// expanded to the IRI. Labels are normalised on the way out; the incoming +/// segment is normalised through the same slot on the way in. +#[test] +fn a_curie_spelled_segment_applies_against_expanded_labels() { + let f = fixture(); + let doc = systems(vec![ + json!({"systemType": WGS84_URI, "value": "one"}), + json!({"systemType": ETRS89_URI, "value": "two"}), + ]); + let hand_written = vec![Delta { + path: vec![ + "systems".to_string(), + WGS84_CURIE.to_string(), + "value".to_string(), + ], + op: DeltaOp::Update, + old: Some(json!("one")), + new: Some(json!("two")), + }]; + let src = f.load(doc); + let (patched, trace) = patch(&src, &hand_written, PatchOptions::default()).unwrap(); + assert!( + trace.failed.is_empty(), + "a curie segment and an expanded label are one identity: {trace:#?}" + ); + assert_eq!(patched.to_json()["systems"][0]["value"], json!("two")); + // …and the same segment navigates. + assert_eq!( + src.navigate_path(["systems", WGS84_CURIE, "value"]) + .map(LinkMLInstance::to_json), + Some(json!("one")) + ); +} + /// `navigate_path` shares the resolver, so either spelling addresses the /// element regardless of how the document spells it. #[test] diff --git a/src/runtime/tests/validation.rs b/src/runtime/tests/validation.rs index dc2a89a..cab56d7 100644 --- a/src/runtime/tests/validation.rs +++ b/src/runtime/tests/validation.rs @@ -156,9 +156,14 @@ fn validation_issue_paths_include_list_indices_once() { "primary_email".to_string(), ]; // The fixture's second object declares `objecttype` as Person, so the bad - // email is a pattern violation on a declared slot — this test is about the - // *path* carrying the list index exactly once, whatever the problem type. - let diag_paths: Vec<_> = diags.iter().map(|d| d.subject.clone()).collect(); + // email is a pattern violation on a declared slot rather than an unknown + // slot. This test is about the *path* carrying the list index exactly once, + // but it still pins a problem type so the filter stays a real assertion. + let diag_paths: Vec<_> = diags + .iter() + .filter(|d| matches!(d.problem_type, ValidationProblemType::SlotRangeViolation)) + .map(|d| d.subject.clone()) + .collect(); assert!( diag_paths.iter().any(|p| p == &expected_path), "diagnostics: {:?}", From b82d6295b69fed14254d90653123e890526b34f9 Mon Sep 17 00:00:00 2001 From: ejsyx Date: Wed, 19 Aug 2026 12:54:40 +0200 Subject: [PATCH 39/58] fix(runtime): class change is whole-element replacement, patch fails soft MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Spec addendum rules 3 and 4 (spike findings D2). Rule 3: when diff pairs two objects of different classes, the element was not edited, it was replaced. Recursing field-by-field across the two class definitions produced deltas describing one class's slots on the other's element (`thread` on a `Nut`) — a diff that no patch could apply, by construction. It now emits one whole-element Update, guarded by `equals` so the "objects the crate considers equal produce no delta" invariant survives. Class identity is the schema-qualified name; `equals`'s canonical-URI comparison is deliberately the looser test. The changed-key branch stays: it still owns a changed key value *within* one class, and its designator case is now subsumed (a canonicalised designator value is a function of the class). Rule 4: a delta whose payload cannot be built at the location it addresses records its path in `PatchTrace::failed` and leaves the tree untouched, instead of propagating the builder's Err and voiding the whole batch. Err is now reserved for infrastructure failure. Builds happen before any mutation, so "failed" always means "nothing happened". Also: `resolve_list_segment` tries exact label equality before the normalising scan — an element addressed by its own label is never shadowed by a sibling whose differently-spelled label normalises to the same string (heterogeneous lists only), and the common case stops re-deriving each element's label slot through its merged `unique_keys`. Co-Authored-By: Claude Fable 5 --- src/runtime/src/blame/mod.rs | 6 + src/runtime/src/diff.rs | 150 ++++++++++++-- .../tests/data/identity_class_change.yaml | 63 ++++++ src/runtime/tests/diff_class_change.rs | 194 ++++++++++++++++++ 4 files changed, 395 insertions(+), 18 deletions(-) create mode 100644 src/runtime/tests/data/identity_class_change.yaml create mode 100644 src/runtime/tests/diff_class_change.rs diff --git a/src/runtime/src/blame/mod.rs b/src/runtime/src/blame/mod.rs index d8d832f..14634a6 100644 --- a/src/runtime/src/blame/mod.rs +++ b/src/runtime/src/blame/mod.rs @@ -22,6 +22,12 @@ use std::fmt; /// strategy: every node touched by the patch (added or updated according to the /// [`PatchTrace`]) will have the supplied metadata cloned into the provided /// `blame` map. +/// +/// Deltas that did not apply are in [`PatchTrace::failed`] and touch nothing, +/// so they leave no blame entry — and, since [`patch`] never hard-errors on +/// one (spec addendum rule 4), an `Ok` here does not mean the whole batch +/// landed. Callers that treat a patch as atomic must check `trace.failed` +/// themselves; the blame map records what actually happened either way. pub fn patch_with_blame( value: &LinkMLInstance, deltas: &[Delta], diff --git a/src/runtime/src/diff.rs b/src/runtime/src/diff.rs index 10c7e59..9fe7e98 100644 --- a/src/runtime/src/diff.rs +++ b/src/runtime/src/diff.rs @@ -84,6 +84,19 @@ pub(crate) fn scalar_slot_string( None } +/// Are these two `ClassView`s the same class? +/// +/// Schema-qualified name, not pointer equality: a `ClassView` is a derived, +/// freshly built view, so two views of one class are routinely distinct values, +/// and a name alone is only unique within its schema. This is the question +/// "did diff pair two objects of different classes?" (spec addendum rule 3), +/// which is about the *declaration* the element instantiates — deliberately +/// finer than [`crate::LinkMLInstance::equals`]'s canonical-URI comparison, +/// which two classes sharing a `class_uri` (spike D8) also satisfy. +fn class_identity_equal(a: &ClassView, b: &ClassView) -> bool { + a.name() == b.name() && a.schema_id() == b.schema_id() +} + /// The class's key/identifier slot, when it can identify an *element*. /// /// A key (or identifier) that is also the class's type designator is skipped: a @@ -334,6 +347,10 @@ impl DiffOptions { /// below the slot is described as a single whole-value `Update` at the slot /// path. See [`OPAQUE_ANNOTATION`]. /// +/// Two paired objects of *different classes* are likewise one whole-element +/// `Update`, never a field-by-field recursion across the two class definitions: +/// the element did not change, it was replaced. +/// /// Lists are matched by element identity when both sides carry unique identity /// labels, and positionally otherwise — with one exception: when the *source* /// list alone is label-addressed (the target repeats or lacks a label), the @@ -378,17 +395,47 @@ pub fn diff(source: &LinkMLInstance, target: &LinkMLInstance, opts: DiffOptions) .. }, ) => { + // Spec addendum rule 3 (D2a): the two sides are different + // *kinds* of thing, so the change is one whole-element + // replacement — never field recursion across two classes. + // Recursing produced deltas describing one class's slots on + // the other's element (`thread` on a `Nut`), which no builder + // can apply: the diff was unpatchable by construction. + // + // Guarded by `equals` so the invariant "objects the crate + // considers equal produce no delta" survives: `equals` compares + // class identity by canonical URI, so two *differently named* + // classes sharing a `class_uri` (spike D8) with matching + // assignments stay silent, exactly as before. + // + // Not governed by `treat_changed_identifier_as_new_object`: + // that flag chooses how to describe a changed *key* within one + // class, while a cross-class recursion is unpatchable whatever + // the caller prefers. + if !class_identity_equal(sc, tc) { + if !s.equals(t, opts.treat_missing_as_null) { + out.push(Delta { + path: path.clone(), + op: DeltaOp::Update, + old: Some(s.to_json()), + new: Some(t.to_json()), + }); + } + return; + } // If objects have an identifier or key slot and it changed, treat as whole-object replacement // This applies for single-valued and list-valued inlined objects. if opts.treat_changed_identifier_as_new_object { // Deliberately the metamodel's key, not [`identity_key_slot`]: // this asks "is this still the same thing?", not "how is it - // labelled among its siblings". A changed key value means a - // different element, and for a *designator* key a changed - // value means a different class — which is a whole-element - // replacement too (spec addendum rule 3). Routing this - // through `identity_key_slot` would drop that replacement - // and recurse field-by-field across two classes. + // labelled among its siblings". + // + // Rule 3 above now owns the *designator* case: a designator + // value is canonicalised at load (rule 2), so a changed + // designator means a changed class and the whole-element + // Update has already been emitted. What is left for this + // branch — and why it stays — is a changed key value within + // ONE class: same class, different element. let key_slot_name = sc .key_or_identifier_slot() .or_else(|| tc.key_or_identifier_slot()) @@ -626,7 +673,11 @@ pub struct PatchTrace { pub deleted: Vec, /// Node IDs of nodes that were directly updated (e.g., parent containers, scalars). pub updated: Vec, - /// Paths of deltas that could not be applied (missing targets, etc.). + /// Paths of deltas that could not be applied: an address that resolves to + /// nothing or resolves ambiguously, a path descending below an opaque slot, + /// or a payload that cannot be built at the location it addresses (spec + /// addendum rule 4). Each such delta leaves the tree untouched and the rest + /// of the batch still applies. pub failed: Vec>, } @@ -649,6 +700,13 @@ impl Default for PatchOptions { /// [`PatchTrace`]. A delta whose path cannot be resolved is reported in /// [`PatchTrace::failed`] rather than guessed at. /// +/// One bad delta never voids the batch (spec addendum rule 4): a delta whose +/// payload cannot be built at the location it addresses — a scalar where the +/// range is a class, a slot the resolved element's class does not declare — is +/// reported the same way, with the tree untouched, and the remaining deltas +/// still apply. `Err` is reserved for infrastructure failure; callers wanting +/// "all or nothing" check `trace.failed` and discard the result themselves. +/// /// **List segments are resolved against the list's CURRENT state**, as the /// deltas are applied in order — not against a snapshot of the list the deltas /// were produced from. On a list whose elements carry *duplicate* identity @@ -753,6 +811,27 @@ where builder(value, schema_view, &conv) } +/// Build a delta's replacement value, turning a build failure into "this delta +/// did not apply" (spec addendum rule 4, D2b). +/// +/// A delta carries a JSON payload that may be nonsense at the location it +/// addresses: a scalar where the slot's range is a class, a slot the resolved +/// element's class does not declare, an enum value outside the permissible set. +/// Propagating the builder's `Err` voided the entire batch — one stale or +/// hand-written delta and every other delta in the same patch was lost, with no +/// record of which one was at fault. The delta's path goes to +/// [`PatchTrace::failed`] instead and the tree is left untouched; `Err` from +/// `patch` is reserved for infrastructure failure. +/// +/// Called before any mutation at every apply site, so "failed" always means +/// "nothing happened", never "half happened". +fn build_or_fail(build: F) -> Option +where + F: FnOnce() -> LResult, +{ + build().ok() +} + fn current_class_and_slot(current: &LinkMLInstance) -> (Option, Option) { match current { LinkMLInstance::Object { class, .. } => (Some(class.clone()), None), @@ -813,9 +892,24 @@ pub(crate) fn resolve_list_segment(values: &[LinkMLInstance], key: &str) -> Opti // labels (e.g. a year as unique key) unambiguous: they resolve as // labels, never as positions. let labels: Vec> = values.iter().map(element_identity_label).collect(); + // Exact label equality first, everywhere. Two reasons, one line: + // *correctness* — a segment that IS an element's label must address that + // element, never a sibling whose differently-spelled label happens to + // normalise to the same string (only reachable in a heterogeneous list, + // where siblings draw their labels from different slots); and *cost* — the + // normalising comparison re-derives the label slot per element, walking the + // class's merged `unique_keys`, so the common case (segments diff emitted, + // which equal the labels outright) should never pay for it. + let exact = |from: usize| { + labels[from..] + .iter() + .position(|l| l.as_deref() == Some(key)) + .map(|i| i + from) + }; let matches = |i: usize| segment_matches_label(&values[i], labels[i].as_deref(), key); if list_is_keyed_shaped(values) { - return (0..values.len()).find(|i| matches(*i)); + // Labels are unique here, so an exact hit is the only exact hit. + return exact(0).or_else(|| (0..values.len()).find(|i| matches(*i))); } // Positional list: numeric index first (the segments diff produces for // these lists), then a single unambiguous label hit for drift tolerance. @@ -824,6 +918,14 @@ pub(crate) fn resolve_list_segment(values: &[LinkMLInstance], key: &str) -> Opti return Some(idx); } } + if let Some(first) = exact(0) { + // Ambiguity is refused, not guessed at — but only exact hits compete + // with an exact hit. + return match exact(first + 1) { + Some(_) => None, + None => Some(first), + }; + } let mut hit: Option = None; for i in 0..values.len() { if matches(i) { @@ -894,7 +996,9 @@ where { match op { DeltaOp::Add | DeltaOp::Update => { - let new_child = build_child()?; + let Some(new_child) = build_or_fail(build_child) else { + return Ok(false); + }; match values.entry(key.to_string()) { Entry::Occupied(mut entry) => { let existing = entry.get_mut(); @@ -959,7 +1063,9 @@ where if idx_opt.is_none() && matches!(op, DeltaOp::Update) { return Ok(false); } - let new_child = build_child()?; + let Some(new_child) = build_or_fail(build_child) else { + return Ok(false); + }; if let Some(idx) = idx_opt { let existing = &mut values[idx]; if should_skip_update(existing, &new_child, opts) { @@ -1088,10 +1194,14 @@ fn apply_delta_root( let (class_opt, slot_opt) = current_class_and_slot(current); if let Some(cls) = class_opt { let slot_clone = slot_opt.clone(); - let new_node = with_converter(schema_view, v, move |value, sv, conv| { - LinkMLInstance::from_json(value, cls, slot_clone, sv, conv, false) - .into_instance_tolerate_errors() - })?; + let Some(new_node) = build_or_fail(|| { + with_converter(schema_view, v, move |value, sv, conv| { + LinkMLInstance::from_json(value, cls, slot_clone, sv, conv, false) + .into_instance_tolerate_errors() + }) + }) else { + return Ok(false); + }; mark_added_subtree(&new_node, trace); *current = new_node; Ok(true) @@ -1105,10 +1215,14 @@ fn apply_delta_root( let (class_opt, slot_opt) = current_class_and_slot(current); if let Some(cls) = class_opt { let slot_clone = slot_opt.clone(); - let new_node = with_converter(schema_view, v, move |value, sv, conv| { - LinkMLInstance::from_json(value, cls, slot_clone, sv, conv, false) - .into_instance_tolerate_errors() - })?; + let Some(new_node) = build_or_fail(|| { + with_converter(schema_view, v, move |value, sv, conv| { + LinkMLInstance::from_json(value, cls, slot_clone, sv, conv, false) + .into_instance_tolerate_errors() + }) + }) else { + return Ok(false); + }; if should_skip_update(current, &new_node, opts) { return Ok(true); } diff --git a/src/runtime/tests/data/identity_class_change.yaml b/src/runtime/tests/data/identity_class_change.yaml new file mode 100644 index 0000000..bbc8b09 --- /dev/null +++ b/src/runtime/tests/data/identity_class_change.yaml @@ -0,0 +1,63 @@ +id: https://w3id.org/linkml/examples/identity_class_change +name: identity_class_change +description: |- + Fixture for spec addendum rule 3 — a class change is a whole-element + replacement — and rule 4 — `patch` never hard-errors on an unappliable delta. + + `Part` declares the element identity (`unique_keys: [code]`) and the type + designator; `Bolt` and `Nut` are siblings carrying *disjoint* extra slots. + Two elements can therefore share one identity label while being different + things: diff must say so once, at the element, instead of recursing + field-by-field across two classes and emitting deltas (`thread` on a `Nut`) + that no builder can apply. +license: https://creativecommons.org/publicdomain/zero/1.0/ +imports: + - linkml:types +prefixes: + part: https://w3id.org/linkml/examples/identity_class_change/ + linkml: https://w3id.org/linkml/ +default_prefix: part +default_range: string + +classes: + Inventory: + attributes: + parts: + range: Part + multivalued: true + inlined_as_list: true + # single-valued inlined object: rule 3 is not list-specific + featured: + range: Part + inlined: true + # a class with a real key slot: the case diff's changed-key branch still + # owns after rule 3 (same class, changed key value) + owner: + range: Owner + inlined: true + + Part: + unique_keys: + by_code: + unique_key_slots: [code] + attributes: + typeURI: + range: uriorcurie + designates_type: true + code: {range: string, required: true} + note: {range: string} + + Bolt: + is_a: Part + attributes: + thread: {range: string} + + Nut: + is_a: Part + attributes: + pitch: {range: string} + + Owner: + attributes: + oid: {range: string, key: true} + oname: {range: string} diff --git a/src/runtime/tests/diff_class_change.rs b/src/runtime/tests/diff_class_change.rs new file mode 100644 index 0000000..fb70f13 --- /dev/null +++ b/src/runtime/tests/diff_class_change.rs @@ -0,0 +1,194 @@ +//! Spec addendum rules 3 and 4 (spike findings D2). +//! +//! Rule 3: when `diff` pairs two objects whose classes differ, the change is a +//! whole-element `Update` — never field-level recursion across two classes. +//! Rule 4: a delta whose value cannot be built or applied at its resolved +//! location records its path in `PatchTrace::failed`; it never voids the batch. + +use linkml_runtime::{ + diff, load_json_str, patch, Delta, DeltaOp, DiffOptions, LinkMLInstance, PatchOptions, +}; +use linkml_schemaview::identifier::{converter_from_schema, Identifier}; +use linkml_schemaview::io::from_yaml; +use linkml_schemaview::schemaview::{ClassView, SchemaView}; +use linkml_schemaview::Converter; +use serde_json::{json, Value as JsonValue}; +use std::path::PathBuf; + +struct Fixture { + sv: SchemaView, + conv: Converter, + inventory: ClassView, +} + +fn fixture() -> Fixture { + let mut p = PathBuf::from(env!("CARGO_MANIFEST_DIR")); + p.push("tests/data/identity_class_change.yaml"); + let schema = from_yaml(&p).unwrap(); + let mut sv = SchemaView::new(); + sv.add_schema(schema.clone()).unwrap(); + let conv = converter_from_schema(&schema); + let inventory = sv + .get_class(&Identifier::new("Inventory"), &conv) + .unwrap() + .expect("class not found"); + Fixture { + sv, + conv, + inventory, + } +} + +impl Fixture { + fn load(&self, v: JsonValue) -> LinkMLInstance { + load_json_str(&v.to_string(), &self.sv, &self.inventory, &self.conv) + .unwrap() + .into_instance() + .unwrap() + } +} + +fn bolt() -> JsonValue { + json!({"typeURI": "part:Bolt", "code": "B1", "thread": "M8"}) +} +fn nut() -> JsonValue { + json!({"typeURI": "part:Nut", "code": "B1", "pitch": "1.25"}) +} + +fn class_name_at(v: &LinkMLInstance, path: &[&str]) -> String { + let segs: Vec = path.iter().map(|s| s.to_string()).collect(); + match v.navigate_path(&segs).expect("path must resolve") { + LinkMLInstance::Object { class, .. } => class.name().to_string(), + other => panic!("expected an object, got {}", other.to_json()), + } +} + +/// Rule 3, both directions: same identity label, different class. +#[test] +fn class_change_is_one_whole_element_update() { + let f = fixture(); + for (before, after, want_class) in [(bolt(), nut(), "Nut"), (nut(), bolt(), "Bolt")] { + let src = f.load(json!({"parts": [before.clone()]})); + let tgt = f.load(json!({"parts": [after.clone()]})); + let deltas = diff(&src, &tgt, DiffOptions::new(true)); + assert_eq!( + deltas.len(), + 1, + "a class change is ONE delta, not field recursion: {deltas:#?}" + ); + let d = &deltas[0]; + assert_eq!(d.path, vec!["parts".to_string(), "B1".to_string()]); + assert_eq!(d.op, DeltaOp::Update); + assert_eq!( + d.new.as_ref().and_then(|v| v.get("code")), + Some(&json!("B1")) + ); + + // Rule 3 round trip: patch rebuilds the element as the NEW class, + // polymorphically, through the slot. + let (patched, trace) = patch(&src, &deltas, PatchOptions::default()).unwrap(); + assert!(trace.failed.is_empty(), "{:?}", trace.failed); + assert!(patched.equals(&tgt, true), "patched: {}", patched.to_json()); + assert_eq!(class_name_at(&patched, &["parts", "B1"]), want_class); + } +} + +/// Rule 3 is not list-specific: a single-valued inlined object changing class +/// is the same whole-element replacement. +#[test] +fn class_change_of_single_valued_object_is_one_update() { + let f = fixture(); + let src = f.load(json!({"featured": bolt()})); + let tgt = f.load(json!({"featured": nut()})); + let deltas = diff(&src, &tgt, DiffOptions::new(true)); + assert_eq!(deltas.len(), 1, "{deltas:#?}"); + assert_eq!(deltas[0].path, vec!["featured".to_string()]); + assert_eq!(deltas[0].op, DeltaOp::Update); + let (patched, trace) = patch(&src, &deltas, PatchOptions::default()).unwrap(); + assert!(trace.failed.is_empty(), "{:?}", trace.failed); + assert!(patched.equals(&tgt, true), "{}", patched.to_json()); + assert_eq!(class_name_at(&patched, &["featured"]), "Nut"); +} + +/// Same class, changed key value: still a whole-element replacement, and the +/// key-value branch that produces it is untouched by rule 3. +#[test] +fn same_class_key_change_still_replaces_whole_element() { + let f = fixture(); + let src = f.load(json!({"owner": {"oid": "o1", "oname": "Ada"}})); + let tgt = f.load(json!({"owner": {"oid": "o2", "oname": "Ada"}})); + let deltas = diff( + &src, + &tgt, + DiffOptions { + treat_changed_identifier_as_new_object: true, + ..DiffOptions::new(true) + }, + ); + assert_eq!(deltas.len(), 1, "{deltas:#?}"); + assert_eq!(deltas[0].path, vec!["owner".to_string()]); + assert_eq!(deltas[0].op, DeltaOp::Update); +} + +/// Rule 4: a hand-built delta whose value cannot be built at its resolved +/// location (`thread` is a `Bolt` slot; the element is a `Nut`) fails soft — +/// `Ok`, path in `trace.failed`, tree untouched — and the OTHER deltas in the +/// same batch still apply. This is the spike's exact probe. +#[test] +fn unbuildable_delta_fails_soft_without_voiding_the_batch() { + let f = fixture(); + let golden = f.load(json!({"parts": [nut()]})); + let bad = Delta { + path: vec!["parts".to_string(), "B1".to_string(), "thread".to_string()], + op: DeltaOp::Update, + old: Some(json!("M8")), + new: Some(json!("M10")), + }; + let good = Delta { + path: vec!["parts".to_string(), "B1".to_string(), "note".to_string()], + op: DeltaOp::Add, + old: None, + new: Some(json!("stocked")), + }; + let (patched, trace) = patch( + &golden, + &[bad.clone(), good.clone()], + PatchOptions::default(), + ) + .unwrap(); + assert_eq!(trace.failed, vec![bad.path.clone()]); + assert!( + patched.equals( + &f.load(json!({"parts": [{"typeURI": "part:Nut", "code": "B1", + "pitch": "1.25", "note": "stocked"}]})), + true + ), + "the good delta must still land, the bad one must change nothing: {}", + patched.to_json() + ); + assert!( + patched + .navigate_path(&["parts".to_string(), "B1".to_string(), "thread".to_string()]) + .is_none(), + "the unbuildable delta must leave no trace in the tree" + ); +} + +/// Rule 4 at the whole-element level: an `Update` carrying a payload no class +/// in the slot's range can accept fails soft too. +#[test] +fn unbuildable_whole_element_update_fails_soft() { + let f = fixture(); + let golden = f.load(json!({"parts": [nut()]})); + let bad = Delta { + path: vec!["parts".to_string(), "B1".to_string()], + op: DeltaOp::Update, + old: Some(nut()), + // a scalar where the slot's range is a class: unbuildable + new: Some(json!("not-an-object")), + }; + let (patched, trace) = patch(&golden, std::slice::from_ref(&bad), PatchOptions::default()) + .expect("a bad delta must not void the batch"); + assert_eq!(trace.failed, vec![bad.path.clone()]); + assert!(patched.equals(&golden, true), "nothing may change"); +} From 7d4933a4e0e8060cc498a0716b2bddd5b3035a92 Mon Sep 17 00:00:00 2001 From: ejsyx Date: Wed, 19 Aug 2026 12:54:44 +0200 Subject: [PATCH 40/58] feat(tools): linkml-patch surfaces failed delta paths `patch` never hard-errors on an unappliable delta: it records the path and applies the rest of the batch. The CLI dropped the trace, so a patch that silently skipped half its deltas exited 0 and wrote a file that looked clean. The paths now go to stderr, leaving stdout byte-identical for a fully applied patch. Co-Authored-By: Claude Fable 5 --- src/tools/src/bin/linkml_patch.rs | 31 ++++++++++++++++++++- src/tools/tests/diff_cli.rs | 45 +++++++++++++++++++++++++++++++ 2 files changed, 75 insertions(+), 1 deletion(-) diff --git a/src/tools/src/bin/linkml_patch.rs b/src/tools/src/bin/linkml_patch.rs index cf3e199..84d4f84 100644 --- a/src/tools/src/bin/linkml_patch.rs +++ b/src/tools/src/bin/linkml_patch.rs @@ -78,6 +78,34 @@ fn write_value( Ok(()) } +/// Report the deltas the patch could not apply. +/// +/// `patch` never hard-errors on an unappliable delta: it records the delta's +/// path and applies the rest of the batch. Dropping the trace made that +/// invisible here — a patch that silently skipped half its deltas exited 0 and +/// wrote a file that looked clean. The lines go to stderr, so the patched +/// document on stdout is byte-identical to before for a fully applied patch. +fn report_failed_deltas(path: &Path, failed: &[Vec]) { + if failed.is_empty() { + return; + } + eprintln!( + "{} of the deltas in '{}' could not be applied; the rest were applied.", + failed.len(), + path.display() + ); + for delta_path in failed { + eprintln!( + " - {}", + if delta_path.is_empty() { + "".to_string() + } else { + delta_path.join(".") + } + ); + } +} + fn main() -> Result<(), Box> { let args = Args::parse(); let schema = from_yaml(&args.schema)?; @@ -105,7 +133,7 @@ fn main() -> Result<(), Box> { } else { serde_yaml::from_str(&delta_text)? }; - let (patched, _trace) = patch( + let (patched, trace) = patch( &src, &deltas, linkml_runtime::diff::PatchOptions { @@ -113,6 +141,7 @@ fn main() -> Result<(), Box> { treat_missing_as_null: args.treat_missing_as_null, }, )?; + report_failed_deltas(&args.delta, &trace.failed); write_value(args.output.as_deref(), &patched)?; Ok(()) } diff --git a/src/tools/tests/diff_cli.rs b/src/tools/tests/diff_cli.rs index 765ae8a..6b2b749 100644 --- a/src/tools/tests/diff_cli.rs +++ b/src/tools/tests/diff_cli.rs @@ -45,3 +45,48 @@ fn cli_diff_and_patch_personinfo() { serde_yaml::from_str(&std::fs::read_to_string(&tgt).unwrap()).unwrap(); assert_eq!(out_data, tgt_data); } + +/// `patch` records an unappliable delta instead of voiding the batch, so the +/// CLI has to say which delta was dropped — otherwise a half-applied patch +/// exits 0 and looks clean. +#[test] +fn cli_patch_reports_failed_delta_paths_on_stderr() { + let schema = info_path("personinfo.yaml"); + let src = info_path("example_personinfo_data.yaml"); + let tmp = tempfile::tempdir().unwrap(); + let delta = tmp.path().join("delta.json"); + let out = tmp.path().join("out.yaml"); + // One delta addressing an object that is not there, one that applies. + std::fs::write( + &delta, + r#"[{"path": ["objects", "P:404", "name"], "op": "update", + "old": "nobody", "new": "somebody"}, + {"path": ["objects", "P:001", "name"], "op": "update", + "old": "fred bloggs", "new": "fred b."}]"#, + ) + .unwrap(); + + let mut cmd = Command::cargo_bin("linkml-patch").unwrap(); + cmd.arg(&schema) + .arg("-c") + .arg("Container") + .arg(&src) + .arg(&delta) + .arg("-o") + .arg(&out); + let assert = cmd.assert().success(); + let stderr = String::from_utf8(assert.get_output().stderr.clone()).unwrap(); + assert!( + stderr.contains("could not be applied") && stderr.contains("objects.P:404.name"), + "the dropped delta's path must be named: {stderr}" + ); + assert!( + !stderr.contains("objects.P:001.name"), + "the applied delta must not be reported: {stderr}" + ); + let patched = std::fs::read_to_string(&out).unwrap(); + assert!( + patched.contains("fred b."), + "the other delta must still land: {patched}" + ); +} From b1adf84e3ddb08e5539c879a2675613d6d2c5f73 Mon Sep 17 00:00:00 2001 From: ejsyx Date: Wed, 19 Aug 2026 13:08:51 +0200 Subject: [PATCH 41/58] fix(tools): three-way exit contract for linkml-patch MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 0 = every delta applied, 2 = partial application, 1 = hard failure (bad arguments, unreadable files, schema or parse errors). Partial application is designed operation under drift, so it is not an error exit — but the removed builder-error `Err` did at least give it a non-zero status, and a script should not have to parse stderr to notice that half its deltas were skipped. A code of its own restores the signal without conflating the two. The codes are in the `--help` text, the patched document is written before the exit, and the writer is flushed explicitly since `process::exit` runs no destructors. The CLI test now asserts the codes, and covers a genuine BUILD-failure delta (a payload with no slot to box it against) alongside the missing-target one: that shape used to propagate as an `Err` and void the batch, so it is what pins rule 4's soft path at the CLI surface. Co-Authored-By: Claude Fable 5 --- src/tools/src/bin/linkml_patch.rs | 37 ++++++++++++++++-- src/tools/tests/diff_cli.rs | 63 +++++++++++++++++++++++++++---- 2 files changed, 88 insertions(+), 12 deletions(-) diff --git a/src/tools/src/bin/linkml_patch.rs b/src/tools/src/bin/linkml_patch.rs index 84d4f84..d41d3cd 100644 --- a/src/tools/src/bin/linkml_patch.rs +++ b/src/tools/src/bin/linkml_patch.rs @@ -11,8 +11,28 @@ use std::path::{Path, PathBuf}; use linkml_tools::validation_utils::report_validation_issues; +/// Exit code for a patch that applied some, but not all, of its deltas. +/// +/// Partial application is designed behaviour, not an error: a delta whose +/// target has drifted away is recorded and skipped so the rest of the batch +/// still lands. It is also not something a script should have to parse stderr +/// to notice, and the old builder-error `Err` at least gave it a non-zero +/// status. A code of its own restores the machine signal without claiming the +/// run failed. +const EXIT_PARTIAL: i32 = 2; + #[derive(Parser)] -#[command(name = "linkml-patch")] +#[command( + name = "linkml-patch", + about = "Apply a delta file to a LinkML instance document", + long_about = "Apply a delta file to a LinkML instance document. + +Exit codes: + 0 every delta applied + 2 partial application: some deltas could not be applied, and their paths are + listed on stderr; the patched document is still written + 1 hard failure: bad arguments, unreadable files, schema or parse errors" +)] struct Args { /// LinkML schema YAML file schema: PathBuf, @@ -75,6 +95,9 @@ fn write_value( serde_yaml::to_writer(&mut writer, &json)?; } writer.write_all(b"\n")?; + // Explicit: the partial-application path leaves via `process::exit`, which + // runs no destructors. + writer.flush()?; Ok(()) } @@ -82,9 +105,10 @@ fn write_value( /// /// `patch` never hard-errors on an unappliable delta: it records the delta's /// path and applies the rest of the batch. Dropping the trace made that -/// invisible here — a patch that silently skipped half its deltas exited 0 and -/// wrote a file that looked clean. The lines go to stderr, so the patched -/// document on stdout is byte-identical to before for a fully applied patch. +/// invisible here — a patch that skipped half its deltas wrote a file that +/// looked clean. The lines go to stderr, so the patched document on stdout is +/// byte-identical to before for a fully applied patch; [`EXIT_PARTIAL`] carries +/// the same news to a caller that does not read prose. fn report_failed_deltas(path: &Path, failed: &[Vec]) { if failed.is_empty() { return; @@ -143,5 +167,10 @@ fn main() -> Result<(), Box> { )?; report_failed_deltas(&args.delta, &trace.failed); write_value(args.output.as_deref(), &patched)?; + if !trace.failed.is_empty() { + // The document is written first: a partial patch is a result, not a + // discarded run. + std::process::exit(EXIT_PARTIAL); + } Ok(()) } diff --git a/src/tools/tests/diff_cli.rs b/src/tools/tests/diff_cli.rs index 6b2b749..b5bb592 100644 --- a/src/tools/tests/diff_cli.rs +++ b/src/tools/tests/diff_cli.rs @@ -48,19 +48,28 @@ fn cli_diff_and_patch_personinfo() { /// `patch` records an unappliable delta instead of voiding the batch, so the /// CLI has to say which delta was dropped — otherwise a half-applied patch -/// exits 0 and looks clean. +/// writes a file that looks clean — and has to signal it in the exit status, +/// which is `2`: partial application, not a failed run. +/// +/// Both ways a delta can be unappliable are exercised: an address that resolves +/// to nothing (soft before rule 4 as well) and a payload that cannot be BUILT +/// at the location it addresses — `no_such_slot` is declared by no class, so +/// the builder has no slot to box the scalar against. That second one used to +/// propagate as an `Err` and void the whole batch; this test is what pins rule +/// 4's soft path all the way out to the CLI surface. #[test] -fn cli_patch_reports_failed_delta_paths_on_stderr() { +fn cli_patch_reports_failed_delta_paths_and_exits_2() { let schema = info_path("personinfo.yaml"); let src = info_path("example_personinfo_data.yaml"); let tmp = tempfile::tempdir().unwrap(); let delta = tmp.path().join("delta.json"); let out = tmp.path().join("out.yaml"); - // One delta addressing an object that is not there, one that applies. std::fs::write( &delta, r#"[{"path": ["objects", "P:404", "name"], "op": "update", "old": "nobody", "new": "somebody"}, + {"path": ["objects", "P:001", "no_such_slot"], "op": "update", + "old": "x", "new": "y"}, {"path": ["objects", "P:001", "name"], "op": "update", "old": "fred bloggs", "new": "fred b."}]"#, ) @@ -74,19 +83,57 @@ fn cli_patch_reports_failed_delta_paths_on_stderr() { .arg(&delta) .arg("-o") .arg(&out); - let assert = cmd.assert().success(); + let assert = cmd.assert().code(2); let stderr = String::from_utf8(assert.get_output().stderr.clone()).unwrap(); assert!( - stderr.contains("could not be applied") && stderr.contains("objects.P:404.name"), - "the dropped delta's path must be named: {stderr}" + stderr.contains("could not be applied") + && stderr.contains("objects.P:404.name") + && stderr.contains("objects.P:001.no_such_slot"), + "both dropped deltas must be named: {stderr}" ); assert!( - !stderr.contains("objects.P:001.name"), + !stderr.contains("objects.P:001.name\n"), "the applied delta must not be reported: {stderr}" ); let patched = std::fs::read_to_string(&out).unwrap(); assert!( patched.contains("fred b."), - "the other delta must still land: {patched}" + "the good delta must still land: {patched}" + ); + assert!( + !patched.contains("no_such_slot"), + "the unbuildable delta must leave no trace: {patched}" + ); +} + +/// The other half of the exit contract: a patch that applies every delta exits +/// `0`, and says nothing on stderr. +#[test] +fn cli_patch_exits_0_when_every_delta_applies() { + let schema = info_path("personinfo.yaml"); + let src = info_path("example_personinfo_data.yaml"); + let tmp = tempfile::tempdir().unwrap(); + let delta = tmp.path().join("delta.json"); + let out = tmp.path().join("out.yaml"); + std::fs::write( + &delta, + r#"[{"path": ["objects", "P:001", "name"], "op": "update", + "old": "fred bloggs", "new": "fred b."}]"#, + ) + .unwrap(); + + let mut cmd = Command::cargo_bin("linkml-patch").unwrap(); + cmd.arg(&schema) + .arg("-c") + .arg("Container") + .arg(&src) + .arg(&delta) + .arg("-o") + .arg(&out); + let assert = cmd.assert().code(0); + let stderr = String::from_utf8(assert.get_output().stderr.clone()).unwrap(); + assert!( + !stderr.contains("could not be applied"), + "a clean patch reports nothing: {stderr}" ); } From 894e279fd3a1b04c3cd2ccea721b33fdf7c0c1dc Mon Sep 17 00:00:00 2001 From: ejsyx Date: Wed, 19 Aug 2026 13:08:51 +0200 Subject: [PATCH 42/58] docs(runtime): patch totality, rule-3 guard, cross-schema note MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three rustdoc corrections at the sites task 13 touched: - `patch` claimed `Err` was "reserved for infrastructure failure", but with rule 4 it no longer fails at all — `LinkMLError` carries validation problems, not infrastructure ones. The `LResult` return is retained for API stability; an `Err` is unreachable, not the place to look for a rejected delta. - `diff` stated rule 3 unconditionally; it is `equals`-guarded, so two classes sharing a `class_uri` with identical content emit nothing. - Rule 3 qualifies class identity by schema *id*, so a genuinely cross-schema pairing (v1 against v2, or two schemas declaring one class name) coarsens every paired object to a whole-element Update — correct and patchable, just coarse. Two `SchemaView`s over the SAME schema do not: they qualify identically and diff as finely as ever, which the new test pins. Also completes the segment-resolution perf fix: `resolve_list_segment` built the labels and then called `list_is_keyed_shaped`, which derived every label again to answer a question about the labels already in hand. A `list_is_keyed_shaped_from_labels` sibling takes the labels; the existing predicate delegates to it for callers that have only the elements. Co-Authored-By: Claude Fable 5 --- src/runtime/src/diff.rs | 44 ++++++++++++++++++++++---- src/runtime/tests/diff_class_change.rs | 21 ++++++++++++ 2 files changed, 58 insertions(+), 7 deletions(-) diff --git a/src/runtime/src/diff.rs b/src/runtime/src/diff.rs index 9fe7e98..e9f8f37 100644 --- a/src/runtime/src/diff.rs +++ b/src/runtime/src/diff.rs @@ -234,9 +234,22 @@ where /// keyed-source fallback — must agree, or a path one of them emits is a path /// another cannot resolve. pub(crate) fn list_is_keyed_shaped(values: &[LinkMLInstance]) -> bool { - !values.is_empty() - && values.iter().all(|v| element_identity_label(v).is_some()) - && labels_are_unique(values, element_identity_label) + let labels: Vec> = values.iter().map(element_identity_label).collect(); + list_is_keyed_shaped_from_labels(&labels) +} + +/// [`list_is_keyed_shaped`] for a caller that already has the labels. +/// +/// The predicate itself, over labels rather than elements. Deriving a label is +/// not free — it walks the class's merged `unique_keys` and IRI-expands the +/// components — and [`resolve_list_segment`] needs both the labels and this +/// answer, so it must not pay for them twice. +fn list_is_keyed_shaped_from_labels(labels: &[Option]) -> bool { + let mut seen = std::collections::HashSet::new(); + !labels.is_empty() + && labels + .iter() + .all(|l| matches!(l, Some(l) if seen.insert(l.as_str()))) } /// Operation applied by a [`Delta`]. @@ -349,7 +362,18 @@ impl DiffOptions { /// /// Two paired objects of *different classes* are likewise one whole-element /// `Update`, never a field-by-field recursion across the two class definitions: -/// the element did not change, it was replaced. +/// the element did not change, it was replaced. "Different class" is the +/// schema-qualified name, but the `Update` is still suppressed when the two +/// objects compare equal, so two classes sharing a `class_uri` with identical +/// content emit nothing. +/// +/// The qualification is by schema *id*, not by `SchemaView` instance: two views +/// built separately over the same schema qualify their classes identically and +/// diff as finely as ever. Where it bites is a genuinely cross-schema pairing — +/// one tree typed by `…/schema/v1`, the other by `…/schema/v2`, or by two +/// schemas that merely declare the same class name. Then no paired object is +/// ever "the same class" and every one of them coarsens to a whole-element +/// `Update`: correct and patchable, just coarse. /// /// Lists are matched by element identity when both sides carry unique identity /// labels, and positionally otherwise — with one exception: when the *source* @@ -704,8 +728,14 @@ impl Default for PatchOptions { /// payload cannot be built at the location it addresses — a scalar where the /// range is a class, a slot the resolved element's class does not declare — is /// reported the same way, with the tree untouched, and the remaining deltas -/// still apply. `Err` is reserved for infrastructure failure; callers wanting -/// "all or nothing" check `trace.failed` and discard the result themselves. +/// still apply. Callers wanting "all or nothing" check `trace.failed` and +/// discard the result themselves. +/// +/// With that, `patch` no longer fails: every way a delta can go wrong is a +/// `trace.failed` entry, and `LinkMLError` carries validation problems, not +/// infrastructure ones. The `LResult` return is retained for API stability — +/// treat an `Err` as unreachable rather than as the place to look for a +/// rejected delta. /// /// **List segments are resolved against the list's CURRENT state**, as the /// deltas are applied in order — not against a snapshot of the list the deltas @@ -907,7 +937,7 @@ pub(crate) fn resolve_list_segment(values: &[LinkMLInstance], key: &str) -> Opti .map(|i| i + from) }; let matches = |i: usize| segment_matches_label(&values[i], labels[i].as_deref(), key); - if list_is_keyed_shaped(values) { + if list_is_keyed_shaped_from_labels(&labels) { // Labels are unique here, so an exact hit is the only exact hit. return exact(0).or_else(|| (0..values.len()).find(|i| matches(*i))); } diff --git a/src/runtime/tests/diff_class_change.rs b/src/runtime/tests/diff_class_change.rs index fb70f13..397705f 100644 --- a/src/runtime/tests/diff_class_change.rs +++ b/src/runtime/tests/diff_class_change.rs @@ -130,6 +130,27 @@ fn same_class_key_change_still_replaces_whole_element() { assert_eq!(deltas[0].op, DeltaOp::Update); } +/// Rule 3 qualifies class identity by schema *id*, not by `SchemaView` +/// instance: two views built separately over the same schema must still diff +/// field-by-field. (Cross-*schema* pairings do coarsen to whole-element +/// Updates — that is the documented, intended consequence.) +#[test] +fn separate_schema_views_over_one_schema_still_diff_finely() { + let a = fixture(); + let b = fixture(); + let src = a.load(json!({"parts": [bolt()]})); + let mut edited = bolt(); + edited["thread"] = json!("M10"); + let tgt = b.load(json!({"parts": [edited]})); + let deltas = diff(&src, &tgt, DiffOptions::new(true)); + assert_eq!(deltas.len(), 1, "{deltas:#?}"); + assert_eq!( + deltas[0].path, + vec!["parts".to_string(), "B1".to_string(), "thread".to_string()], + "a second SchemaView over the same schema is not a class change" + ); +} + /// Rule 4: a hand-built delta whose value cannot be built at its resolved /// location (`thread` is a `Bolt` slot; the element is a `Nut`) fails soft — /// `Ok`, path in `trace.failed`, tree untouched — and the OTHER deltas in the From 0e5dfd7591bee8909580987e385a7d3514ae7fc9 Mon Sep 17 00:00:00 2001 From: ejsyx Date: Wed, 19 Aug 2026 13:18:13 +0200 Subject: [PATCH 43/58] fix(tools): partial-application exit code moves off clap's usage code MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `2` was already taken: clap exits 2 on an unknown flag or a missing argument (verified for both), so the documented contract made a typo'd flag indistinguishable from a partially applied patch — the one confusion a script following the contract could least afford — and the help line claiming `1` for bad arguments was simply false. Partial application is now `3`. The help text lists all four: 0 every delta applied, 1 hard failure from this tool, 2 argument/usage error (the parser's convention, not ours to choose), 3 partial application with the document still written. A new test pins the distinction by asserting that a usage error exits 2, not 3. Co-Authored-By: Claude Fable 5 --- src/tools/src/bin/linkml_patch.rs | 14 ++++++++++---- src/tools/tests/diff_cli.rs | 19 ++++++++++++++++--- 2 files changed, 26 insertions(+), 7 deletions(-) diff --git a/src/tools/src/bin/linkml_patch.rs b/src/tools/src/bin/linkml_patch.rs index d41d3cd..7a748ae 100644 --- a/src/tools/src/bin/linkml_patch.rs +++ b/src/tools/src/bin/linkml_patch.rs @@ -19,7 +19,12 @@ use linkml_tools::validation_utils::report_validation_issues; /// to notice, and the old builder-error `Err` at least gave it a non-zero /// status. A code of its own restores the machine signal without claiming the /// run failed. -const EXIT_PARTIAL: i32 = 2; +/// +/// **3, not 2**: clap exits `2` on a usage error (an unknown flag, a missing +/// argument), which this tool does not get to choose. Claiming 2 would make a +/// typo'd flag indistinguishable from a partially applied patch — the one +/// reading a script could least afford to confuse. +const EXIT_PARTIAL: i32 = 3; #[derive(Parser)] #[command( @@ -29,9 +34,10 @@ const EXIT_PARTIAL: i32 = 2; Exit codes: 0 every delta applied - 2 partial application: some deltas could not be applied, and their paths are - listed on stderr; the patched document is still written - 1 hard failure: bad arguments, unreadable files, schema or parse errors" + 1 hard failure: unreadable files, schema, parse or write errors + 2 argument or usage error (the command-line parser's convention) + 3 partial application: some deltas could not be applied, and their paths are + listed on stderr; the patched document is still written" )] struct Args { /// LinkML schema YAML file diff --git a/src/tools/tests/diff_cli.rs b/src/tools/tests/diff_cli.rs index b5bb592..a3c139b 100644 --- a/src/tools/tests/diff_cli.rs +++ b/src/tools/tests/diff_cli.rs @@ -49,7 +49,8 @@ fn cli_diff_and_patch_personinfo() { /// `patch` records an unappliable delta instead of voiding the batch, so the /// CLI has to say which delta was dropped — otherwise a half-applied patch /// writes a file that looks clean — and has to signal it in the exit status, -/// which is `2`: partial application, not a failed run. +/// which is `3`: partial application, not a failed run, and not `2`, which the +/// argument parser has already claimed for usage errors. /// /// Both ways a delta can be unappliable are exercised: an address that resolves /// to nothing (soft before rule 4 as well) and a payload that cannot be BUILT @@ -58,7 +59,7 @@ fn cli_diff_and_patch_personinfo() { /// propagate as an `Err` and void the whole batch; this test is what pins rule /// 4's soft path all the way out to the CLI surface. #[test] -fn cli_patch_reports_failed_delta_paths_and_exits_2() { +fn cli_patch_reports_failed_delta_paths_and_exits_3() { let schema = info_path("personinfo.yaml"); let src = info_path("example_personinfo_data.yaml"); let tmp = tempfile::tempdir().unwrap(); @@ -83,7 +84,7 @@ fn cli_patch_reports_failed_delta_paths_and_exits_2() { .arg(&delta) .arg("-o") .arg(&out); - let assert = cmd.assert().code(2); + let assert = cmd.assert().code(3); let stderr = String::from_utf8(assert.get_output().stderr.clone()).unwrap(); assert!( stderr.contains("could not be applied") @@ -137,3 +138,15 @@ fn cli_patch_exits_0_when_every_delta_applies() { "a clean patch reports nothing: {stderr}" ); } + +/// A usage error is `2`, the argument parser's own code — which is why partial +/// application is `3`. A script that cannot tell a typo'd flag from a +/// half-applied patch has no contract at all. +#[test] +fn cli_patch_usage_error_exits_2_not_the_partial_code() { + Command::cargo_bin("linkml-patch") + .unwrap() + .arg("--no-such-flag") + .assert() + .code(2); +} From 357c307e499299664f72b256a988bfb5f32c3c66 Mon Sep 17 00:00:00 2001 From: ejsyx Date: Wed, 19 Aug 2026 13:18:13 +0200 Subject: [PATCH 44/58] fix(runtime): restore keyed-shape short-circuit, note cross-schema build failures MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `list_is_keyed_shaped(values)` had been reduced to "materialise every label, then ask the labels", which threw away `all()`'s short-circuit: a long list whose FIRST element carries no label used to settle the question in one derivation and had started paying for all of them. It gets its lazy body back; `list_is_keyed_shaped_from_labels` remains for the resolver, which genuinely holds the labels already. Also completes the cross-schema note: coarsening every paired object to a whole-element Update is patchable only where the two schemas still agree about the element's shape. Where they genuinely disagree the payload fails to build and lands in `trace.failed` — still better than the field-level recursion it replaced, which produced deltas that could not apply and had no single path to report. Co-Authored-By: Claude Fable 5 --- src/runtime/src/diff.rs | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git a/src/runtime/src/diff.rs b/src/runtime/src/diff.rs index e9f8f37..88c52f9 100644 --- a/src/runtime/src/diff.rs +++ b/src/runtime/src/diff.rs @@ -233,9 +233,15 @@ where /// front of it — `patch`'s segment resolver, `navigate_path`, and diff's /// keyed-source fallback — must agree, or a path one of them emits is a path /// another cannot resolve. +/// +/// Derives labels lazily and stops at the first element that has none — the +/// answer for an unlabelled list is settled by its first bare element, however +/// long the list is. [`list_is_keyed_shaped_from_labels`] is the variant for a +/// caller that has already paid for the labels. pub(crate) fn list_is_keyed_shaped(values: &[LinkMLInstance]) -> bool { - let labels: Vec> = values.iter().map(element_identity_label).collect(); - list_is_keyed_shaped_from_labels(&labels) + !values.is_empty() + && values.iter().all(|v| element_identity_label(v).is_some()) + && labels_are_unique(values, element_identity_label) } /// [`list_is_keyed_shaped`] for a caller that already has the labels. @@ -373,7 +379,11 @@ impl DiffOptions { /// one tree typed by `…/schema/v1`, the other by `…/schema/v2`, or by two /// schemas that merely declare the same class name. Then no paired object is /// ever "the same class" and every one of them coarsens to a whole-element -/// `Update`: correct and patchable, just coarse. +/// `Update`: correct, and patchable where the two schemas still agree about the +/// element's shape. Where they genuinely disagree, the `Update`'s payload will +/// not build against the source schema and lands in [`PatchTrace::failed`] — +/// still an improvement on the field-level recursion this replaced, which +/// produced deltas that could not apply *and* had no single path to report. /// /// Lists are matched by element identity when both sides carry unique identity /// labels, and positionally otherwise — with one exception: when the *source* From 0d4a44e77f6016f26ecf4983cde8562ff35d38c2 Mon Sep 17 00:00:00 2001 From: ejsyx Date: Wed, 19 Aug 2026 13:38:10 +0200 Subject: [PATCH 45/58] fix(runtime): inlined-dict keys are injected and validated against payloads MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Spec addendum rule 5 (finding D5). The LinkML `inlined` dict form says the mapping key *is* the element's key/identifier value, but the loader never wrote it back: a `required` key slot produced a MissingSlotValue error on legal data, and the loaded object did not carry its own key. `build_mapping_entry_for_slot` now takes the entry's dict key and injects it into the key slot when the payload omits it — before class selection (a designator-keyed dict selects the class its key names), before the object constraints (the required key is satisfied), and before canonicalization (an injected designator value is canonicalized like any other). When the payload does supply the key slot, two warnings can fire, never errors: a value that disagrees with the dict key (compared through `canonical_identity_component`, so a CURIE key and its expanded payload stay one identity), and — for a designator key — a dict key that is no accepted designator value of the selected class. The payload value is stored as the element's data; the mapping stays addressed by its dict key. `example_personinfo_data_2.yaml` gains the `role` key it always implied: it is the expected side of the linkml-patch CLI round-trip, so it must state its own normal form. The source file still omits `role`. Co-Authored-By: Claude Fable 5 --- src/runtime/src/diff.rs | 8 +- src/runtime/src/lib.rs | 201 +++++++++++-- .../tests/data/example_personinfo_data_2.yaml | 7 + .../tests/data/identity_canonical.yaml | 5 +- src/runtime/tests/data/inlined_dict_key.yaml | 72 +++++ .../tests/identity_canonicalization.rs | 5 + src/runtime/tests/inlined_dict_key.rs | 277 ++++++++++++++++++ 7 files changed, 556 insertions(+), 19 deletions(-) create mode 100644 src/runtime/tests/data/inlined_dict_key.yaml create mode 100644 src/runtime/tests/inlined_dict_key.rs diff --git a/src/runtime/src/diff.rs b/src/runtime/src/diff.rs index 88c52f9..d8dabbe 100644 --- a/src/runtime/src/diff.rs +++ b/src/runtime/src/diff.rs @@ -58,7 +58,7 @@ pub(crate) fn slot_is_opaque(slot: &SlotView) -> bool { /// verbatim: `Identifier::to_uri` refuses them, and inventing an expansion /// against the default prefix would rename identities the schema never claimed /// were IRIs. -fn canonical_identity_component(raw: &str, slot: &SlotView) -> String { +pub(crate) fn canonical_identity_component(raw: &str, slot: &SlotView) -> String { if !slot.is_range_iri() { return raw.to_string(); } @@ -1344,6 +1344,11 @@ fn apply_delta_mapping( if path.len() == 1 { let value = newv.cloned().unwrap_or(JsonValue::Null); let slot_clone = slot.clone(); + // The delta's own path segment is the entry's dict key, and rule 5 + // makes that key the element's key-slot value: a delta that adds an + // entry must build the same object the loader would have built for the + // same key. + let key_clone = key.clone(); return apply_hashmap_leaf_delta( values, key, @@ -1356,6 +1361,7 @@ fn apply_delta_mapping( let mut diags = ValidationResultSink::default(); let value = LinkMLInstance::build_mapping_entry_for_slot( &slot_clone, + &key_clone, val, sv, conv, diff --git a/src/runtime/src/lib.rs b/src/runtime/src/lib.rs index 3b09e27..b99c0fd 100644 --- a/src/runtime/src/lib.rs +++ b/src/runtime/src/lib.rs @@ -949,6 +949,110 @@ impl LinkMLInstance { *value = JsonValue::String(canonical); } + /// Hear what an inlined-dict entry's key says when the payload also states + /// it (spec addendum rule 5, finding D5). + /// + /// Called only when the payload *did* supply the key slot — the omitted + /// case is filled from the dict key by the caller, and any complaint about + /// that injected value is then [`Self::canonicalize_type_designator`]'s to + /// make, in its own words, on the value that was actually stored. + /// + /// Two things are checked, and both are warnings, never errors: the + /// document loads, because the LinkML `inlined` contract makes both halves + /// legal on their own and only their *combination* is contradictory. + /// + /// 1. **Divergence.** The dict key and the payload value say different + /// things about the same slot. The payload value is the element's data + /// and is what gets stored; the dict key is the address and is what the + /// mapping stays keyed by (`parse_mapping_slot` inserts under the raw + /// key). Neither can be dropped without losing information the document + /// contains, so both are named and the author decides. + /// + /// Compared through [`crate::diff::canonical_identity_component`], the + /// same function every identity label is built from, so a CURIE key and + /// its expanded payload are one identity (rule 2) rather than a + /// divergence. Class-level equivalence is deliberately *not* applied to + /// a designator key: `{"": {"typeURI": ""}}` names + /// one class twice, but the payload is canonicalised at load and the dict + /// key never is, so the loaded document still contradicts itself. That is + /// asset360's committed `SpotLocation_coordinates` shape. + /// + /// 2. **An unaccepted designator key.** When the key slot *is* the class's + /// type designator the dict key names the entry's class, and a key that + /// is no accepted designator value of the class actually selected names + /// nothing. Before this rule the case was silent twice over: nothing read + /// the key, and `populate_type_designator` filled the slot from the range + /// class, so the entry looked complete. + fn reconcile_dict_key_with_payload( + entry_key: &str, + key_slot: &SlotView, + payload: &JsonValue, + selected: &ClassView, + conv: &Converter, + path: &[String], + validation_issues: &mut ValidationResultSink, + ) { + let mut p = path.to_vec(); + p.push(key_slot.name.clone()); + + if key_slot.definition().designates_type == Some(true) { + // `Err` means the accepted set is unknown, not empty — the same + // posture `canonicalize_type_designator` takes: do not punish the + // data for a schema the view cannot resolve. + if let Some(td) = selected.get_type_designator_slot() { + if let Ok(accepted) = selected.get_accepted_type_designator_values(td, conv) { + if !accepted.iter().any(|v| v.to_string() == entry_key) { + validation_issues.push_warning( + ValidationProblemType::SlotRangeViolation, + p.clone(), + format!( + "inlined-dict key `{entry_key}` is not an accepted designator \ + value for class `{}`", + selected.name() + ), + ); + } + } + } + } + + // An explicit `null` and a structurally wrong value (list, object) are + // left for the range checks that already ran to describe; calling them + // a divergence would say the wrong thing about the wrong problem. + let supplied = match payload { + JsonValue::String(s) => s.clone(), + JsonValue::Null | JsonValue::Array(_) | JsonValue::Object(_) => return, + other => other.to_string(), + }; + if crate::diff::canonical_identity_component(entry_key, key_slot) + == crate::diff::canonical_identity_component(&supplied, key_slot) + { + return; + } + validation_issues.push_warning( + ValidationProblemType::SlotRangeViolation, + p, + format!( + "inlined-dict key `{entry_key}` disagrees with the payload value `{supplied}` of \ + key slot `{}`; the payload value is stored as the element's data and the entry \ + stays addressed by its dict key", + key_slot.name + ), + ); + } + + /// The dict key of an inlined mapping entry, as the value its key slot + /// takes when the payload omitted it (spec addendum rule 5). + /// + /// A JSON object key is a string and is injected as one. A numerically + /// ranged key slot therefore receives the string spelling of its key: the + /// loader runs no scalar type check (see [`crate::constraints`] — enum, + /// regex and min/max only), and inventing a second, untested numeric parse + /// here would be a coercion path of its own rather than this rule. + fn dict_key_value(entry_key: &str) -> JsonValue { + JsonValue::String(entry_key.to_string()) + } + /// If the class has a `designates_type` slot and the values map does not /// already contain it, insert a Scalar value with the class's type /// designator value. This ensures round-trip fidelity for formats like @@ -1122,6 +1226,7 @@ impl LinkMLInstance { for (k, v) in map.into_iter() { let child = Self::build_mapping_entry_for_slot( sl, + &k, v, sv, conv, @@ -1494,8 +1599,21 @@ impl LinkMLInstance { ) } + /// Build one entry of an inlined mapping. + /// + /// `entry_key` is the entry's dict key, which the LinkML `inlined` contract + /// makes the element's key/identifier *value* — real data, not just an + /// address (spec addendum rule 5, finding D5). It is injected into the key + /// slot when the payload omits it, which happens before class selection (so + /// a designator-keyed dict selects the class its key names), before the + /// object constraints run (so a `required` key slot is satisfied by the key + /// the document did supply, instead of erroring on legal data), and before + /// [`Self::canonicalize_type_designator`] (so an injected designator value + /// is canonicalised exactly as a payload-supplied one is). When the payload + /// does supply it, [`Self::reconcile_dict_key_with_payload`] speaks. pub(crate) fn build_mapping_entry_for_slot( map_slot: &SlotView, + entry_key: &str, value: JsonValue, sv: &SchemaView, conv: &Converter, @@ -1518,8 +1636,34 @@ impl LinkMLInstance { ) })?; match value { - JsonValue::Object(m) => { + JsonValue::Object(mut m) => { + let key_slot = range_cv.key_or_identifier_slot().cloned(); + // Alias-aware, because the payload may spell the slot by any + // name `slot_matches_key` accepts — the same lookup the child + // loop below uses, so "did the payload supply it?" and "which + // entry is it?" cannot disagree. + let payload_key: Option = key_slot.as_ref().and_then(|ks| { + m.iter() + .find(|(ck, _)| slot_matches_key(ks, ck)) + .map(|(_, cv)| cv.clone()) + }); + if let Some(ks) = &key_slot { + if payload_key.is_none() { + m.insert(ks.name.clone(), Self::dict_key_value(entry_key)); + } + } let selected = Self::select_class(&m, &range_cv, sv, conv); + if let (Some(ks), Some(pv)) = (key_slot.as_ref(), payload_key.as_ref()) { + Self::reconcile_dict_key_with_payload( + entry_key, + ks, + pv, + &selected, + conv, + &path, + validation_issues, + ); + } let mut child_values = HashMap::new(); for (ck, cv) in m.into_iter() { let slot_tmp = selected @@ -1571,10 +1715,8 @@ impl LinkMLInstance { }) } other => { - let key_slot_name = range_cv - .key_or_identifier_slot() - .map(|s| s.name.as_str()) - .unwrap_or(""); + let key_slot = range_cv.key_or_identifier_slot().cloned(); + let key_slot_name = key_slot.as_ref().map(|s| s.name.as_str()).unwrap_or(""); let scalar_slot = Self::find_scalar_slot_for_inlined_map(&range_cv, key_slot_name) .ok_or_else(|| { LinkMLError::single( @@ -1586,23 +1728,33 @@ impl LinkMLInstance { ), ) })?; + // The compact form states exactly two things: the dict key and + // one scalar. Both are named here, because either can be the + // class's *designator* and so name the entry's class. + // // `find_scalar_slot_for_inlined_map` picks the first non-key - // scalar slot, which can be the class's *designator*: then + // scalar slot, which can be the designator: then // `{"w1": "canon:FancyWidget"}` says "widget w1 is a - // FancyWidget", and the entry names its own class exactly as - // the object form's `{"typeURI": ...}` does. This arm otherwise + // FancyWidget", exactly as the object form's `{"typeURI": ...}` + // does. The dict key is the designator whenever the key slot + // and the designator slot are one — the asset360 + // `PositioningSystemCoordinate` shape. This arm otherwise // hardwires the slot's range class, so without selecting here // canonicalisation would rewrite that designator to the *range* // class's value and warn about data that was right — the - // misclassification spec rule 2 exists to prevent. Selection is - // scoped to this one shape: a compact entry whose scalar is an - // ordinary slot names no class and keeps the range class. - let entry_class = if scalar_slot.definition().designates_type.unwrap_or(false) { - let mut named = serde_json::Map::new(); - named.insert(scalar_slot.name.clone(), other.clone()); - Self::select_class(&named, &range_cv, sv, conv) - } else { - range_cv.clone() + // misclassification spec rule 2 exists to prevent. Selection + // stays scoped to that one question: a compact entry that names + // no designator at all names no class and keeps the range class. + let mut named = serde_json::Map::new(); + named.insert(scalar_slot.name.clone(), other.clone()); + if let Some(ks) = &key_slot { + named.insert(ks.name.clone(), Self::dict_key_value(entry_key)); + } + let entry_class = match range_cv.get_type_designator_slot() { + Some(td) if named.contains_key(&td.name) => { + Self::select_class(&named, &range_cv, sv, conv) + } + _ => range_cv.clone(), }; let mut child_values = HashMap::new(); child_values.insert( @@ -1615,6 +1767,21 @@ impl LinkMLInstance { sv: sv.clone(), }, ); + // Rule 5: the compact form has no payload for the key slot at + // all, so the dict key is the only thing that can fill it — + // there is nothing here to diverge from. + if let Some(ks) = &key_slot { + child_values.insert( + ks.name.clone(), + LinkMLInstance::Scalar { + node_id: new_node_id(), + value: Self::dict_key_value(entry_key), + slot: ks.clone(), + class: Some(entry_class.clone()), + sv: sv.clone(), + }, + ); + } run_object_constraints( &entry_class, &child_values, diff --git a/src/runtime/tests/data/example_personinfo_data_2.yaml b/src/runtime/tests/data/example_personinfo_data_2.yaml index 8e41988..2500902 100644 --- a/src/runtime/tests/data/example_personinfo_data_2.yaml +++ b/src/runtime/tests/data/example_personinfo_data_2.yaml @@ -15,11 +15,18 @@ objects: - employed_at: ROR:1 started_at_time: 2019-01-01 is_current: true + # `role` is the key slot of this inlined dict. Spec addendum rule 5 makes + # the dict key real data, so the loader fills `role` from it — this file is + # the *expected* side of the `linkml-patch` CLI round-trip and therefore + # states its own normal form. `example_personinfo_data.yaml`, the source + # side, still omits it, so the omitted case stays exercised end to end. has_familial_relationships: brother: + role: brother related_to: P:001 type: SIBLING_OF mother: + role: mother related_to: P:003 type: PARENT_OF has_medical_history: diff --git a/src/runtime/tests/data/identity_canonical.yaml b/src/runtime/tests/data/identity_canonical.yaml index a09c1b0..de0f29b 100644 --- a/src/runtime/tests/data/identity_canonical.yaml +++ b/src/runtime/tests/data/identity_canonical.yaml @@ -91,9 +91,12 @@ classes: # `find_scalar_slot_for_inlined_map` picks the first non-key scalar slot — # here the designator — so `{"w1": "canon:FancyWidget"}` is a legal compact # dict entry that names its own class. + # `wid` is `required` on purpose: the dict-key injection of spec addendum + # rule 5 is what satisfies it for both the object and the compact form, so a + # loader that stopped injecting fails here rather than quietly. Widget: attributes: - wid: {range: string, key: true} + wid: {range: string, key: true, required: true} typeURI: range: uriorcurie designates_type: true diff --git a/src/runtime/tests/data/inlined_dict_key.yaml b/src/runtime/tests/data/inlined_dict_key.yaml new file mode 100644 index 0000000..1890120 --- /dev/null +++ b/src/runtime/tests/data/inlined_dict_key.yaml @@ -0,0 +1,72 @@ +id: https://w3id.org/linkml/examples/inlined_dict_key +name: inlined_dict_key +description: |- + Fixture for spec addendum rule 5 — the inlined-dict key is real data (D5). + + Three shapes: + + * `Container.people` — an ordinary keyed class in dict form. The dict key is + the element's `pid`; a payload that omits it must be filled from the key, + and a payload that contradicts it must be heard. + * `Container.coords` — a dict whose range class's key slot *is* its type + designator, so the dict key names the entry's class. The asset360 shape. + * `Container.systems` — a key slot whose range descends from `uri`, where a + CURIE dict key and its expansion in the payload are one identity. +license: https://creativecommons.org/publicdomain/zero/1.0/ +imports: + - linkml:types +prefixes: + dk: https://w3id.org/linkml/examples/inlined_dict_key/ + ex: https://example.org/dk/ + linkml: https://w3id.org/linkml/ +default_prefix: dk +default_range: string + +classes: + Container: + attributes: + people: + range: Person + multivalued: true + inlined: true + coords: + range: Coordinate + multivalued: true + inlined: true + systems: + range: System + multivalued: true + inlined: true + + # `pid` is `required` on purpose: the injection is what satisfies it, and a + # loader that stopped injecting would fail loudly here rather than quietly. + Person: + attributes: + pid: {range: string, key: true, required: true} + name: {range: string} + + # key and designator on one slot — the asset360 `PositioningSystemCoordinate` + # shape (`slot_usage: typeURI: {key: true}` over a `designates_type` slot). + Coordinate: + attributes: + typeURI: + range: uriorcurie + designates_type: true + key: true + value: {range: string} + + # declares a `class_uri`, so its canonical designator value and its + # schema-native URI are two different accepted spellings — exactly the + # asset360 `LinearCoordinate` divergence. + LinearCoordinate: + is_a: Coordinate + class_uri: ex:Linear + attributes: + measure: {range: float} + + # a key slot whose range descends from `uri`: a CURIE dict key and the + # expanded payload are one identity (rule 2), so they must not diverge. + System: + attributes: + systemType: {range: uri, key: true, required: true} + label: {range: string} diff --git a/src/runtime/tests/identity_canonicalization.rs b/src/runtime/tests/identity_canonicalization.rs index e15c045..9a89697 100644 --- a/src/runtime/tests/identity_canonicalization.rs +++ b/src/runtime/tests/identity_canonicalization.rs @@ -190,6 +190,11 @@ fn compact_dict_entry_naming_a_subclass_is_not_rewritten_to_the_range_class() { "canonicalised, and to the *subclass* the entry named: {out:#?}" ); assert_eq!(widgets["w2"]["typeURI"], json!(FANCY_CURIE), "in {out:#?}"); + // `Widget.wid` is `required`, and the compact form supplies no payload for + // it at all — the dict-key injection of rule 5 is the only thing that can + // satisfy it. The `issues.is_empty()` assertion above is the guard. + assert_eq!(widgets["w1"]["wid"], json!("w1"), "in {out:#?}"); + assert_eq!(widgets["w2"]["wid"], json!("w2"), "in {out:#?}"); } /// The guard for `ClassView::get_uri(native, expand)`: `BareLeaf` declares a diff --git a/src/runtime/tests/inlined_dict_key.rs b/src/runtime/tests/inlined_dict_key.rs new file mode 100644 index 0000000..3a01744 --- /dev/null +++ b/src/runtime/tests/inlined_dict_key.rs @@ -0,0 +1,277 @@ +//! Spec addendum rule 5 — the inlined-dict key is real data (D5). +//! +//! LinkML's `inlined` dict form says the mapping key *is* the element's +//! key/identifier value; the payload is allowed to leave it out. The loader +//! used never to write it back, so a `required` key slot produced a +//! `MissingSlotValue` error on perfectly legal data, and the object that came +//! out of the load did not carry its own key. +//! +//! Three facts are pinned here: +//! +//! * the dict key is **injected** into the key slot when the payload omits it, +//! before the required/cardinality constraints run and before the type +//! designator is filled, so it flows through the same canonicalisation every +//! other designator value does (rule 2); +//! * a payload value that **disagrees** with the dict key is a warning naming +//! both. The payload value is data and is what gets *stored*; the dict key is +//! the address and is what the mapping stays keyed by; +//! * for a dict whose key slot **is** the type designator, a dict key that is +//! no accepted designator value is a warning. + +use linkml_runtime::{ + load_json_str, LinkMLInstance, ValidationProblemType, ValidationResult, ValidationSeverity, +}; +use linkml_schemaview::identifier::{converter_from_schema, Identifier}; +use linkml_schemaview::io::from_yaml; +use linkml_schemaview::schemaview::{ClassView, SchemaView}; +use linkml_schemaview::Converter; +use serde_json::{json, Value as JsonValue}; +use std::path::PathBuf; + +/// `LinearCoordinate`'s canonical designator value: its `class_uri` (the slot +/// range is `uriorcurie`, so the canonical spelling is the CURIE). +const LINEAR_CANONICAL: &str = "ex:Linear"; +/// The same class's *schema-native* URI — a second accepted spelling, and the +/// one asset360's committed data uses as the dict key while spelling the +/// payload with the `class_uri`. +const LINEAR_NATIVE: &str = "https://w3id.org/linkml/examples/inlined_dict_key/LinearCoordinate"; +const WGS84_CURIE: &str = "ex:WGS84"; +const WGS84_URI: &str = "https://example.org/dk/WGS84"; + +struct Fixture { + sv: SchemaView, + conv: Converter, + container: ClassView, +} + +fn fixture() -> Fixture { + let mut p = PathBuf::from(env!("CARGO_MANIFEST_DIR")); + p.push("tests/data/inlined_dict_key.yaml"); + let schema = from_yaml(&p).unwrap(); + let mut sv = SchemaView::new(); + sv.add_schema(schema.clone()).unwrap(); + let conv = converter_from_schema(&schema); + let container = sv + .get_class(&Identifier::new("Container"), &conv) + .unwrap() + .expect("class not found"); + Fixture { + sv, + conv, + container, + } +} + +impl Fixture { + fn load_result(&self, v: JsonValue) -> (LinkMLInstance, Vec) { + let r = load_json_str(&v.to_string(), &self.sv, &self.container, &self.conv).unwrap(); + let issues = r.validation_issues.clone(); + (r.into_instance().expect("instance must load"), issues) + } +} + +fn errors(issues: &[ValidationResult]) -> Vec<&ValidationResult> { + issues.iter().filter(|i| i.severity.is_error()).collect() +} + +// --------------------------------------------------------------------------- +// Injection +// --------------------------------------------------------------------------- + +/// The payload omits the key slot, as the `inlined` contract allows. The dict +/// key fills it: no `MissingSlotValue` for the `required` key, and the object +/// carries its own key out through `to_json`. +#[test] +fn dict_key_fills_the_key_slot_the_payload_omitted() { + let f = fixture(); + let (inst, issues) = f.load_result(json!({ + "people": {"p1": {"name": "Ann"}, "p2": {"name": "Bo"}} + })); + assert!( + errors(&issues).is_empty(), + "the dict key satisfies the required key slot: {issues:#?}" + ); + assert!( + !issues + .iter() + .any(|i| i.problem_type == ValidationProblemType::MissingSlotValue), + "no missing-value diagnostic of any severity: {issues:#?}" + ); + let out = inst.to_json(); + assert_eq!(out["people"]["p1"]["pid"], json!("p1"), "in {out:#?}"); + assert_eq!(out["people"]["p2"]["pid"], json!("p2"), "in {out:#?}"); +} + +/// The payload states the same key the dict does. Nothing to inject, nothing +/// to say. +#[test] +fn payload_key_equal_to_the_dict_key_is_silent() { + let f = fixture(); + let (inst, issues) = f.load_result(json!({ + "people": {"p1": {"pid": "p1", "name": "Ann"}} + })); + assert!(issues.is_empty(), "no diagnostics expected: {issues:#?}"); + assert_eq!(inst.to_json()["people"]["p1"]["pid"], json!("p1")); +} + +/// The payload contradicts the dict key. Rule 5 makes this a **warning**, not +/// an error, and the ruling on which value survives is recorded here: the +/// payload is the element's data and is stored; the dict key is the address and +/// the mapping stays keyed by it. Both values are named in the message so the +/// author can see which one to change. +#[test] +fn payload_key_disagreeing_with_the_dict_key_warns_and_the_payload_is_stored() { + let f = fixture(); + let (inst, issues) = f.load_result(json!({ + "people": {"p1": {"pid": "p2", "name": "Ann"}} + })); + assert!( + errors(&issues).is_empty(), + "a divergence is a warning, never an error: {issues:#?}" + ); + let warned: Vec<&ValidationResult> = issues + .iter() + .filter(|i| i.severity == ValidationSeverity::Warning) + .collect(); + assert_eq!(warned.len(), 1, "exactly one warning: {issues:#?}"); + let detail = &warned[0].detail; + assert!( + detail.contains("p1") && detail.contains("p2"), + "the warning must name both values: {detail:?}" + ); + assert_eq!( + warned[0].subject, + vec!["people".to_string(), "p1".to_string(), "pid".to_string()], + "reported at the key slot of the entry the dict key addresses" + ); + let out = inst.to_json(); + assert_eq!( + out["people"]["p1"]["pid"], + json!("p2"), + "the payload value is the stored data: {out:#?}" + ); + assert!( + out["people"].get("p2").is_none(), + "the mapping stays addressed by its dict key: {out:#?}" + ); +} + +/// A key slot whose range descends from `uri` compares as an IRI, not as a +/// string (rule 2): a CURIE dict key and its expansion in the payload are one +/// identity and must not be reported as a divergence. +#[test] +fn a_curie_dict_key_and_its_expanded_payload_are_one_identity() { + let f = fixture(); + let (_, issues) = f.load_result(json!({ + "systems": {WGS84_CURIE: {"systemType": WGS84_URI, "label": "one"}} + })); + assert!( + issues.is_empty(), + "a curie and its expansion are one identity: {issues:#?}" + ); +} + +// --------------------------------------------------------------------------- +// Designator-keyed dicts +// --------------------------------------------------------------------------- + +/// The dict key names the entry's class. An accepted spelling selects that +/// class, and the injected value is canonicalised on the way in exactly as a +/// payload-supplied designator is — the stored value stays canonical. +#[test] +fn an_accepted_designator_dict_key_selects_the_class_and_is_canonicalised() { + let f = fixture(); + let (inst, issues) = f.load_result(json!({ + "coords": {LINEAR_NATIVE: {"measure": 1.5}} + })); + assert!( + issues.is_empty(), + "the native URI is an accepted designator value: {issues:#?}" + ); + let out = inst.to_json(); + let entry = &out["coords"][LINEAR_NATIVE]; + assert_eq!( + entry["typeURI"], + json!(LINEAR_CANONICAL), + "injected from the dict key and canonicalised: {out:#?}" + ); + assert_eq!( + entry["measure"], + json!(1.5), + "the subclass's own slot survives, so the subclass was selected: {out:#?}" + ); +} + +/// A dict key that is no accepted designator value at all is a warning — the +/// case D5 found silently masked, because `populate_type_designator` filled the +/// slot from the range class and nothing ever looked at the key. +#[test] +fn a_designator_dict_key_that_is_not_accepted_warns() { + let f = fixture(); + let (inst, issues) = f.load_result(json!({ + "coords": {"Whatever": {"value": "v"}} + })); + assert!( + errors(&issues).is_empty(), + "an unaccepted key is a warning, never an error: {issues:#?}" + ); + assert_eq!(issues.len(), 1, "exactly one warning: {issues:#?}"); + assert!( + issues[0].detail.contains("Whatever"), + "the warning must name the rejected key: {:?}", + issues[0].detail + ); + let out = inst.to_json(); + assert_eq!( + out["coords"]["Whatever"]["typeURI"], + json!("dk:Coordinate"), + "the range class's canonical value is what is stored: {out:#?}" + ); +} + +/// The payload is fine on its own terms but the dict key is junk. The key's own +/// rejection has to be heard even when nothing else is wrong with the entry. +#[test] +fn an_unaccepted_designator_key_is_heard_even_when_the_payload_is_accepted() { + let f = fixture(); + let (_, issues) = f.load_result(json!({ + "coords": {"Whatever": {"typeURI": LINEAR_CANONICAL, "measure": 2.0}} + })); + assert!(errors(&issues).is_empty(), "warnings only: {issues:#?}"); + assert!( + issues.iter().any(|i| i.detail.contains("Whatever") + && i.detail.contains("not an accepted designator value")), + "the unaccepted dict key must be named: {issues:#?}" + ); +} + +/// The asset360 shape, reduced: the dict key spells the class with its +/// schema-native URI and the payload spells it with the `class_uri`. Both are +/// accepted designator values of the same class, but the two strings are two +/// different IRIs, and the mapping is addressed by the raw key while the +/// payload is canonicalised — so the document contradicts itself and says so. +#[test] +fn native_uri_key_against_class_uri_payload_is_a_divergence() { + let f = fixture(); + let (inst, issues) = f.load_result(json!({ + "coords": {LINEAR_NATIVE: {"typeURI": LINEAR_CANONICAL, "measure": 3.0}} + })); + assert!(errors(&issues).is_empty(), "warnings only: {issues:#?}"); + let divergences: Vec<&ValidationResult> = issues + .iter() + .filter(|i| i.detail.contains("disagrees")) + .collect(); + assert_eq!(divergences.len(), 1, "one divergence: {issues:#?}"); + assert!( + divergences[0].detail.contains(LINEAR_NATIVE) + && divergences[0].detail.contains(LINEAR_CANONICAL), + "both spellings named: {:?}", + divergences[0].detail + ); + let out = inst.to_json(); + assert_eq!( + out["coords"][LINEAR_NATIVE]["typeURI"], + json!(LINEAR_CANONICAL), + "the payload value, canonicalised, is the stored data: {out:#?}" + ); +} From 5d50f77bea5e4dc00d8f6e8a98e54b3457c627ef Mon Sep 17 00:00:00 2001 From: ejsyx Date: Wed, 19 Aug 2026 13:51:33 +0200 Subject: [PATCH 46/58] fix(runtime): dict-key divergence compares post-canonical stored value MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Review fix. `reconcile_dict_key_with_payload` compared the dict key against the RAW payload, before `canonicalize_type_designator` ran. For a designator key that was a false positive in one direction: key `ex:Linear` + payload `` are two accepted spellings of one class, so canonicalization rewrote the payload onto the key's own spelling — yet the load warned, and the message said "the payload value is stored" while naming a value that was not what got stored. Reloading the emitted document was silent. The divergence half now runs after both designator passes and reads the key slot out of `child_values`: warn when the entry, as it will be written back out, contradicts the key it is written under. The message names the stored value. The injected case is still skipped — it agrees by construction. The asset360 divergence (native-URI key vs canonical `RSM:#EAID_CB107995…`) is unaffected: canonicalization moves that payload away from the key, not onto it. Both directions are now tests, plus the round-trip invariant that makes the silence correct. Also folded in from the review: a comment on why the accepted-designator half stays a raw string compare (it must match what canonicalization matches, or one fact gets two voices); a test for the compact arm's key-is-designator path; and a rustdoc note that the key slot is read from the RANGE class only, so a subclass-only `slot_usage` key gets neither injection nor checks. Co-Authored-By: Claude Fable 5 --- src/runtime/src/lib.rs | 111 +++++++++++++++++--------- src/runtime/tests/inlined_dict_key.rs | 80 ++++++++++++++++++- 2 files changed, 150 insertions(+), 41 deletions(-) diff --git a/src/runtime/src/lib.rs b/src/runtime/src/lib.rs index b99c0fd..49171cc 100644 --- a/src/runtime/src/lib.rs +++ b/src/runtime/src/lib.rs @@ -961,20 +961,31 @@ impl LinkMLInstance { /// document loads, because the LinkML `inlined` contract makes both halves /// legal on their own and only their *combination* is contradictory. /// - /// 1. **Divergence.** The dict key and the payload value say different - /// things about the same slot. The payload value is the element's data - /// and is what gets stored; the dict key is the address and is what the - /// mapping stays keyed by (`parse_mapping_slot` inserts under the raw + /// 1. **Divergence.** The dict key and the element's key-slot value say + /// different things about the same slot. The payload is the element's + /// data and is what gets stored; the dict key is the address and is what + /// the mapping stays keyed by (`parse_mapping_slot` inserts under the raw /// key). Neither can be dropped without losing information the document /// contains, so both are named and the author decides. /// - /// Compared through [`crate::diff::canonical_identity_component`], the - /// same function every identity label is built from, so a CURIE key and - /// its expanded payload are one identity (rule 2) rather than a - /// divergence. Class-level equivalence is deliberately *not* applied to - /// a designator key: `{"": {"typeURI": ""}}` names - /// one class twice, but the payload is canonicalised at load and the dict - /// key never is, so the loaded document still contradicts itself. That is + /// The comparison is against the value **as stored**, which is why this + /// runs after [`Self::canonicalize_type_designator`] and + /// [`Self::populate_type_designator`] rather than against the raw + /// payload. A designator payload is rewritten to its class's canonical + /// spelling at load, so comparing the raw payload would call + /// `{"": {"typeURI": ""}}` a divergence while the + /// two ends of the message — key and "the stored value" — had in fact + /// already been made to agree, and reloading the emitted document would + /// be silent. The rule is therefore: warn when the entry, *as it will be + /// written back out*, contradicts the key it is written under. + /// + /// Both sides go through [`crate::diff::canonical_identity_component`], + /// the same function every identity label is built from, so a CURIE key + /// and an expanded stored value are one identity (rule 2) rather than a + /// divergence. What survives is the genuinely contradictory case: + /// `{"": {"typeURI": ""}}`, where both spellings + /// name one class but only the payload is canonicalised, so the loaded + /// entry really is addressed by one IRI and carries another. That is /// asset360's committed `SpotLocation_coordinates` shape. /// /// 2. **An unaccepted designator key.** When the key slot *is* the class's @@ -983,10 +994,16 @@ impl LinkMLInstance { /// nothing. Before this rule the case was silent twice over: nothing read /// the key, and `populate_type_designator` filled the slot from the range /// class, so the entry looked complete. + /// + /// The key slot itself is [`ClassView::key_or_identifier_slot`] on the + /// slot's **range** class, not on the class finally selected: the key has to + /// be read (and injected) before `select_class` can run, so a key declared + /// only by a subclass — via `slot_usage` on a descendant — is chicken-and-egg + /// and gets neither the injection nor these checks. fn reconcile_dict_key_with_payload( entry_key: &str, key_slot: &SlotView, - payload: &JsonValue, + values: &HashMap, selected: &ClassView, conv: &Converter, path: &[String], @@ -996,6 +1013,13 @@ impl LinkMLInstance { p.push(key_slot.name.clone()); if key_slot.definition().designates_type == Some(true) { + // Deliberately a raw string compare, unlike the divergence half + // below: the accepted set is what `canonicalize_type_designator` + // (Task 12, rule 2) matches against, verbatim, and a key that this + // check waved through on an IRI-equal-but-differently-spelled basis + // would then be rewritten by that function and reported by *it* + // instead — two voices for one fact. + // // `Err` means the accepted set is unknown, not empty — the same // posture `canonicalize_type_designator` takes: do not punish the // data for a schema the view cannot resolve. @@ -1016,16 +1040,18 @@ impl LinkMLInstance { } } - // An explicit `null` and a structurally wrong value (list, object) are - // left for the range checks that already ran to describe; calling them - // a divergence would say the wrong thing about the wrong problem. - let supplied = match payload { + // A `Null` and a structurally wrong value (list, object) are left for + // the range checks that already ran to describe; calling them a + // divergence would say the wrong thing about the wrong problem. + let Some(LinkMLInstance::Scalar { value, .. }) = values.get(&key_slot.name) else { + return; + }; + let stored = match value { JsonValue::String(s) => s.clone(), - JsonValue::Null | JsonValue::Array(_) | JsonValue::Object(_) => return, other => other.to_string(), }; if crate::diff::canonical_identity_component(entry_key, key_slot) - == crate::diff::canonical_identity_component(&supplied, key_slot) + == crate::diff::canonical_identity_component(&stored, key_slot) { return; } @@ -1033,9 +1059,9 @@ impl LinkMLInstance { ValidationProblemType::SlotRangeViolation, p, format!( - "inlined-dict key `{entry_key}` disagrees with the payload value `{supplied}` of \ - key slot `{}`; the payload value is stored as the element's data and the entry \ - stays addressed by its dict key", + "inlined-dict key `{entry_key}` disagrees with `{stored}`, the value stored in \ + key slot `{}`; the payload is the element's data and the dict key is its \ + address, so the entry keeps both and stays addressed by its dict key", key_slot.name ), ); @@ -1611,6 +1637,12 @@ impl LinkMLInstance { /// [`Self::canonicalize_type_designator`] (so an injected designator value /// is canonicalised exactly as a payload-supplied one is). When the payload /// does supply it, [`Self::reconcile_dict_key_with_payload`] speaks. + /// + /// The key slot is read from the slot's **range** class only. It has to be: + /// the key is injected *before* `select_class` runs, so consulting the + /// selected class would be circular. A key declared solely by a descendant + /// — `slot_usage: {k: {key: true}}` on a subclass, the range class having no + /// key of its own — therefore gets neither the injection nor the checks. pub(crate) fn build_mapping_entry_for_slot( map_slot: &SlotView, entry_key: &str, @@ -1642,28 +1674,15 @@ impl LinkMLInstance { // name `slot_matches_key` accepts — the same lookup the child // loop below uses, so "did the payload supply it?" and "which // entry is it?" cannot disagree. - let payload_key: Option = key_slot.as_ref().and_then(|ks| { - m.iter() - .find(|(ck, _)| slot_matches_key(ks, ck)) - .map(|(_, cv)| cv.clone()) - }); + let payload_states_key = key_slot + .as_ref() + .is_some_and(|ks| m.keys().any(|ck| slot_matches_key(ks, ck))); if let Some(ks) = &key_slot { - if payload_key.is_none() { + if !payload_states_key { m.insert(ks.name.clone(), Self::dict_key_value(entry_key)); } } let selected = Self::select_class(&m, &range_cv, sv, conv); - if let (Some(ks), Some(pv)) = (key_slot.as_ref(), payload_key.as_ref()) { - Self::reconcile_dict_key_with_payload( - entry_key, - ks, - pv, - &selected, - conv, - &path, - validation_issues, - ); - } let mut child_values = HashMap::new(); for (ck, cv) in m.into_iter() { let slot_tmp = selected @@ -1706,6 +1725,22 @@ impl LinkMLInstance { validation_issues, ); Self::populate_type_designator(&mut child_values, &selected, sv, conv); + // After both designator passes, so the comparison is against + // the value the entry will actually carry out. Skipped when the + // key was injected: the entry then agrees with its key by + // construction, and a rejected *injected* designator value is + // `canonicalize_type_designator`'s to report. + if let (Some(ks), true) = (key_slot.as_ref(), payload_states_key) { + Self::reconcile_dict_key_with_payload( + entry_key, + ks, + &child_values, + &selected, + conv, + &path, + validation_issues, + ); + } Ok(LinkMLInstance::Object { node_id: new_node_id(), values: child_values, diff --git a/src/runtime/tests/inlined_dict_key.rs b/src/runtime/tests/inlined_dict_key.rs index 3a01744..94b2dfa 100644 --- a/src/runtime/tests/inlined_dict_key.rs +++ b/src/runtime/tests/inlined_dict_key.rs @@ -245,11 +245,36 @@ fn an_unaccepted_designator_key_is_heard_even_when_the_payload_is_accepted() { ); } +/// The compact form of a designator-*keyed* dict: the dict key is the only +/// thing that names the class, and the bare scalar fills the first ordinary +/// slot. Selection, injection and canonicalisation all have to come off the key +/// alone. +#[test] +fn a_compact_entry_takes_its_class_from_a_designator_dict_key() { + let f = fixture(); + let (inst, issues) = f.load_result(json!({ + "coords": {LINEAR_NATIVE: "somevalue"} + })); + assert!(issues.is_empty(), "no diagnostics expected: {issues:#?}"); + let out = inst.to_json(); + let entry = &out["coords"][LINEAR_NATIVE]; + assert_eq!( + entry["typeURI"], + json!(LINEAR_CANONICAL), + "the class the key named, canonicalised: {out:#?}" + ); + assert_eq!( + entry["value"], + json!("somevalue"), + "the compact scalar lands in the first ordinary slot: {out:#?}" + ); +} + /// The asset360 shape, reduced: the dict key spells the class with its /// schema-native URI and the payload spells it with the `class_uri`. Both are -/// accepted designator values of the same class, but the two strings are two -/// different IRIs, and the mapping is addressed by the raw key while the -/// payload is canonicalised — so the document contradicts itself and says so. +/// accepted designator values of the same class, but the payload is +/// canonicalised at load and the key is not, so the *loaded* entry really is +/// addressed by one IRI and carries another. That contradiction is the warning. #[test] fn native_uri_key_against_class_uri_payload_is_a_divergence() { let f = fixture(); @@ -275,3 +300,52 @@ fn native_uri_key_against_class_uri_payload_is_a_divergence() { "the payload value, canonicalised, is the stored data: {out:#?}" ); } + +/// The mirror image of the test above, and the reason the divergence check +/// compares the value **as stored** rather than the raw payload: here the key is +/// the canonical spelling and the payload is the native URI, so rule 2 rewrites +/// the payload to the key's own spelling. The entry that comes out agrees with +/// the key it is written under, and reloading that output is silent — so the +/// load must be silent too. Comparing raw payloads would warn here, and the +/// message would name a "stored" value that is not what was stored. +#[test] +fn a_payload_canonicalised_onto_the_dict_key_is_not_a_divergence() { + let f = fixture(); + let (inst, issues) = f.load_result(json!({ + "coords": {LINEAR_CANONICAL: {"typeURI": LINEAR_NATIVE, "measure": 4.0}} + })); + assert!( + issues.is_empty(), + "canonicalisation made the two agree before anything was stored: {issues:#?}" + ); + let out = inst.to_json(); + assert_eq!( + out["coords"][LINEAR_CANONICAL]["typeURI"], + json!(LINEAR_CANONICAL), + "in {out:#?}" + ); + // The invariant that makes the silence correct: the emitted document reloads + // without a word. + let (_, reloaded) = f.load_result(out.clone()); + assert!(reloaded.is_empty(), "round-trip must stay silent: {out:#?}"); +} + +/// The same invariant from the other side: the asset360 divergence is real +/// precisely because it *survives* the round-trip — the emitted document is +/// still addressed by one IRI and still carries another. +#[test] +fn a_real_divergence_survives_the_round_trip() { + let f = fixture(); + let (inst, _) = f.load_result(json!({ + "coords": {LINEAR_NATIVE: {"typeURI": LINEAR_CANONICAL, "measure": 3.0}} + })); + let (_, reloaded) = f.load_result(inst.to_json()); + assert_eq!( + reloaded + .iter() + .filter(|i| i.detail.contains("disagrees")) + .count(), + 1, + "the contradiction is in the data, not in the load: {reloaded:#?}" + ); +} From fd69e76d64780e35b6dc6d62de5ba97aa4e210f1 Mon Sep 17 00:00:00 2001 From: ejsyx Date: Wed, 19 Aug 2026 14:17:53 +0200 Subject: [PATCH 47/58] feat(runtime): identity lint covers missing labels, split label spaces, shared class_uri MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Spec addendum rule 6 (D7, D4d, D8, D9). Three new lint rules and the rustdoc that lists all of them. Instance lint, "positional despite a declared identity" (D7): an inlined list some of whose elements yield no identity label, although the element class declares one, is addressed positionally after all. An optional key left empty is valid data, so neither the schema lint nor validation can see it. One warning per container, counting the unlabelled elements; half-labelled lists are the same defect and are counted the same way. Unlike the duplicate rule it honours `opaque`/`ignore` and skips reference lists: it claims only that the list is addressed positionally, and for those slots that claim is false. The unguarded version fired on exactly one list in the downstream corpus — asset360's `NetElement.ports`, a reference list — and on nothing else. Schema lint, `unique_keys` across descendants (D4d): a list ranged on a class holds elements of every class descending from it, each labelled by its own merged `unique_keys`, so the several-entries rule now unions candidates over the whole family. A new, additional warning fires when the family resolves DIFFERENT load-bearing entries — one list, two label spaces, where a path written against one cannot address an element of the other and two elements can collide on a label without violating either class's constraint. Each rule gates on its own introduces-predicate. Schema lint, shared `class_uri` under a designator (D8): two classes of one `is_a` hierarchy answering to one class URI while the hierarchy designates its type. The loader resolves such a value to one class, stably but by an ordering the schema does not state. Warning only; the loader is deliberately unchanged. Class-level, so it is emitted once per (hierarchy, shared URI) instead of being gated on an introducing slot. D9 (`slot_usage: designates_type: false` leaving a key nothing fills) is documented as a sharp edge that collapses into the instance rule, not detected separately. Harness: asset360 27 warnings before and after, rinf 19 before and after, self diffs empty. The only downstream-visible movement is the reworded several-entries message on an in-repo fixture, plus the three new fixtures. Co-Authored-By: Claude Fable 5 --- src/runtime/src/identity_lint.rs | 588 +++++++++++++++--- src/runtime/src/lib.rs | 7 +- .../data/identity_descendant_unique_keys.yaml | 115 ++++ .../tests/data/identity_missing_labels.yaml | 121 ++++ .../data/identity_shared_uri_designator.yaml | 77 +++ src/runtime/tests/identity_lint.rs | 278 ++++++++- 6 files changed, 1098 insertions(+), 88 deletions(-) create mode 100644 src/runtime/tests/data/identity_descendant_unique_keys.yaml create mode 100644 src/runtime/tests/data/identity_missing_labels.yaml create mode 100644 src/runtime/tests/data/identity_shared_uri_designator.yaml diff --git a/src/runtime/src/identity_lint.rs b/src/runtime/src/identity_lint.rs index 9c8e364..d1d3f1e 100644 --- a/src/runtime/src/identity_lint.rs +++ b/src/runtime/src/identity_lint.rs @@ -23,25 +23,87 @@ //! removing the slot from diff's scope; `opaque` silences it by answering the //! question with "nowhere — replace the value as a whole". //! -//! A second, narrower question is asked of slots that pass: **which** of the -//! range class's `unique_keys` provides the identity? Only the name-sorted -//! first entry does (declaration order is not preserved by the metamodel), so a -//! class declaring several has an alphabetically-decided identity and adding an -//! earlier-sorting entry silently re-addresses every delta path for every slot -//! ranged on it. That is warned about too, naming the load-bearing entry. +//! # The rules //! -//! A third rule catches a declared identity that cannot discriminate: a list -//! whose element class is keyed by its own type designator, whose value -//! describes the class rather than the element (the dict form of the same -//! class, meaning at-most-one-per-subtype, is left alone). The engine ignores -//! such a key entirely, so the slot also matches the first rule's shape (no -//! other identity is declared) or the second's (the `unique_keys` the key used -//! to shadow); this rule is the sharpest diagnosis and is asked first, so a -//! designator-keyed slot yields exactly one warning — this one. +//! [`lint_element_identity`] asks four questions of a schema, and +//! [`lint_instance_identity`] two of a loaded instance. All six warn; none +//! errors, and none changes what the engine does. //! -//! Warnings are reported at the class that **introduces** the slot: a flagged -//! slot inherited unchanged by a descendant is not repeated there, since the -//! declaration the author would edit lives on the ancestor. +//! ## Schema rules +//! +//! 1. **No declared identity.** A multivalued inlined list whose element class +//! declares no key/identifier and no `unique_keys` (or whose range is not a +//! class at all). Its deltas are positional, and positional deltas are +//! ambiguous when several sources produce deltas for one object +//! concurrently. This is the rule the four options above answer. +//! 2. **The identity is the type designator.** The element class's key (or +//! identifier) is its own `designates_type` slot, whose value describes the +//! class and not the element: constant across a homogeneous list, one value +//! per subtype across a polymorphic one. The engine ignores such a key +//! outright (spec addendum rule 1), so the slot also matches rule 1's shape +//! (nothing else is declared) or rule 3's (the `unique_keys` the key used to +//! shadow). This is the sharpest diagnosis and is asked first, so a +//! designator-keyed slot yields exactly one warning — this one. The dict +//! form of the same class is left alone: a mapping keyed by the designator +//! legitimately says at-most-one-element-per-subtype. +//! 3. **Several `unique_keys` to choose from.** Which of them provides the +//! identity? Only the name-sorted first does (declaration order is not +//! preserved by the metamodel), so a class offering several has an +//! alphabetically-decided identity, and adding an earlier-sorting entry +//! silently re-addresses every delta path for every slot ranged on it. The +//! candidates are counted across the range class **and every class +//! descending from it**, since a list ranged on a class holds elements of +//! all of them. +//! 4. **A split label space.** Those same classes resolve *different* +//! load-bearing entries: `Gadget` elements labelled by one entry and +//! `Widget is_a Gadget` elements by another, in one list. A path written +//! against one label space cannot address an element of the other, and two +//! elements resolving different entries can carry the same label without +//! violating either class's uniqueness constraint. Reported in addition to +//! rule 3, which such a family always also matches. +//! 5. **A shared `class_uri` under a designator.** Two classes of one `is_a` +//! hierarchy declare the same `class_uri` while the hierarchy designates its +//! type. A designator value is a class URI, so it names both classes at +//! once and the loader resolves it to one of them — stably, but by an +//! ordering the schema does not state. Warning only: the loader's choice is +//! deliberately unchanged. +//! +//! Slot warnings are reported at the class that **introduces** the slot: a +//! flagged slot inherited unchanged by a descendant is not repeated there, +//! since the declaration the author would edit lives on the ancestor. Each rule +//! gates on its own predicate, so a subclass whose `slot_usage` changes one +//! rule's answer is judged on its own merits for that rule. Rule 5 is +//! class-level and has no slot to attribute, so it is emitted once per +//! (hierarchy, shared URI) instead. +//! +//! ## Instance rules +//! +//! 6. **Repeated identity.** Two elements of one list carry the same identity +//! label: a delta addressing it cannot say which is meant. +//! 7. **Positional despite a declared identity.** Some element of an inlined +//! list yields no label — the element class declares an identity, and the +//! data leaves the slot it names empty — so the list is not keyed-shaped and +//! is addressed positionally after all. Nothing here is invalid (a `key` +//! that is not `required` may legitimately be absent), so no schema rule and +//! no validation can see it; only the data can. +//! +//! Rule 6 does not consult `diff.linkml.io/opaque` or `ignore`: a repeated +//! identity contradicts the element class's own constraint, which is +//! class-level truth that diff vocabulary on the slot cannot suppress. Rule 7 +//! does honour both, and skips reference lists, because it claims only that the +//! list is addressed positionally — a claim those slots make false rather than +//! excused. +//! +//! # Sharp edge, deliberately not its own rule +//! +//! A subclass may switch the designator off with +//! `slot_usage: { theSlot: { designates_type: false } }` while the slot stays +//! (or becomes) the class's `key`. The override is respected, so the designator +//! machinery stops filling the slot — and nothing else fills it either. The +//! class then declares a key that no element ever carries a value for, which is +//! not a schema defect the schema can be read for: the declaration is +//! well-formed, and only the data shows that the key is always empty. It +//! collapses into instance rule 7, which is where it is reported. use crate::diff::{ element_identity_label, identity_key_slot, slot_is_ignored, slot_is_opaque, OPAQUE_ANNOTATION, @@ -50,7 +112,23 @@ use crate::{LinkMLInstance, ValidationProblemType, ValidationResult, ValidationR use linkml_schemaview::identifier::Identifier; use linkml_schemaview::schemaview::{ClassView, SchemaView}; use linkml_schemaview::slotview::SlotView; -use std::collections::{BTreeMap, HashMap, HashSet}; +use std::collections::{BTreeMap, BTreeSet, HashMap, HashSet}; + +/// The shape every schema rule below is asked of: an inlined list whose +/// per-element delta paths are diff's to address, and which therefore has to +/// answer "where does element identity come from?". +/// +/// A dict is addressed by its keys, a reference list by the referents' own +/// identifiers, an `opaque` slot by nothing (the value is replaced whole) and +/// an `ignore`d slot not at all. None of them can be mis-addressed by a +/// mis-declared element identity, so none of them is any rule's business. +fn slot_addresses_elements_by_position_or_label(slot: &SlotView) -> bool { + use linkml_schemaview::slotview::{SlotContainerMode, SlotInlineMode}; + slot.determine_slot_container_mode() == SlotContainerMode::List + && slot.determine_slot_inline_mode() != SlotInlineMode::Reference + && !slot_is_opaque(slot) + && !slot_is_ignored(slot) +} /// Whether this is a multivalued inlined slot whose element identity comes from /// nowhere — the engine's answer, from which the reporting loop subtracts the @@ -60,19 +138,9 @@ use std::collections::{BTreeMap, HashMap, HashSet}; /// of an inherited slot on its parent class, to decide which class introduced /// the problem. fn slot_lacks_element_identity(slot: &SlotView) -> bool { - use linkml_schemaview::slotview::{SlotContainerMode, SlotInlineMode}; - if slot.determine_slot_container_mode() != SlotContainerMode::List { + if !slot_addresses_elements_by_position_or_label(slot) { return false; } - if slot.determine_slot_inline_mode() == SlotInlineMode::Reference { - return false; // elements are references, not inlined - } - if slot_is_opaque(slot) { - return false; // identity declared: nowhere, replace the value as a whole - } - if slot_is_ignored(slot) { - return false; // outside diff's scope entirely: no deltas, no identity - } if let Some(rc) = slot.get_range_class() { // The engine's notion of a key, not the metamodel's: a key that is the // class's type designator identifies the class, never the element, and @@ -99,41 +167,130 @@ fn identity_unique_key_names(rc: &ClassView) -> Vec { .collect() } -/// Whether this slot's element identity is *ambiguous* rather than absent: its -/// range class offers several `unique_keys` entries to derive it from, and only -/// the name-sorted first is load-bearing. +/// The classes whose `unique_keys` can label an element of a list ranged on +/// `rc`: `rc` itself, and every class descending from it. /// -/// Returns the range class name and the candidate entry names (sorted, so the -/// load-bearing one is first). This flags slots the identity-less rule passes: -/// the identity exists, but which of the declarations provides it was decided +/// A list ranged on a class holds elements of every class descending from it, +/// and each element is labelled by its **own** merged `unique_keys` — so both +/// `unique_keys` rules below are questions about the whole family, not about +/// the one declaration the slot's range happens to name (spike D4d). +/// +/// Mixin users are excluded (`include_mixins: false`): applying a mixin does +/// not make a class an instance of it, so it cannot put that class into a list +/// ranged on the mixin. +/// +/// A failure to resolve the descendants degrades to "no descendants" rather +/// than to a panic or a swallowed rule: the family always contains `rc`, so the +/// rules keep the pre-D4d answer instead of disappearing. +fn identity_class_family(rc: &ClassView) -> Vec { + let mut family = vec![rc.clone()]; + if let Ok(descendants) = rc.get_descendants(true, false) { + family.extend(descendants); + } + family +} + +/// The `unique_keys` entry this class's elements are actually labelled by: +/// the name-sorted first entry that names any slots, mirroring +/// `crate::diff::element_unique_key_label`. +/// +/// `None` when the class's identity does not come from `unique_keys` at all — +/// either it has a (non-designator) key/identifier, which outranks every entry, +/// or it declares no usable entry and its elements go unlabelled. +fn load_bearing_unique_key(rc: &ClassView) -> Option { + if identity_key_slot(rc).is_some() { + return None; // the key outranks unique_keys entirely + } + identity_unique_key_names(rc).into_iter().next() +} + +/// Whether this slot's element identity is *ambiguous* rather than absent: +/// the range class and its descendants offer several `unique_keys` entries to +/// derive it from, and only the name-sorted first is load-bearing. +/// +/// Returns the range class name, the name-sorted candidate entries, and the one +/// the range class *itself* resolves to — which is the first candidate unless a +/// descendant declares an earlier-sorting entry, the split case the divergence +/// rule is the voice for. This flags slots the identity-less rule passes: the +/// identity exists, but which of the declarations provides it was decided /// alphabetically rather than by the author. /// -/// A class with a `key`/`identifier` slot is not ambiguous however many -/// `unique_keys` it declares: the key outranks them all, so none of them is -/// load-bearing and adding one changes nothing. A key that is the class's type -/// designator outranks nothing — the engine looks past it — so such a class is -/// judged on its `unique_keys` like any other; the designator rule speaks for -/// it first regardless, since that is the defect worth reporting. -fn slot_has_ambiguous_unique_keys(slot: &SlotView) -> Option<(String, Vec)> { - use linkml_schemaview::slotview::{SlotContainerMode, SlotInlineMode}; - if slot.determine_slot_container_mode() != SlotContainerMode::List { +/// The candidates are unioned over [`identity_class_family`], because an entry +/// a descendant adds is an entry some element of the list is really labelled by +/// (spike D4d). Family members whose identity is a key are skipped: their +/// entries are never load-bearing, so adding one to them re-addresses nothing. +/// +/// A range class with a `key`/`identifier` slot is not ambiguous however many +/// `unique_keys` it declares, for the same reason. A key that is the class's +/// type designator outranks nothing — the engine looks past it — so such a +/// class is judged on its `unique_keys` like any other; the designator rule +/// speaks for it first regardless, since that is the defect worth reporting. +fn slot_has_ambiguous_unique_keys(slot: &SlotView) -> Option<(String, Vec, String)> { + if !slot_addresses_elements_by_position_or_label(slot) { return None; } - if slot.determine_slot_inline_mode() == SlotInlineMode::Reference { - return None; // elements are references, not inlined + let rc = slot.get_range_class()?; + // The range class's own answer. `None` means its elements carry no label at + // all, which is the identity-less rule's business, not this one's. + let own = load_bearing_unique_key(&rc)?; + let mut names: BTreeSet = BTreeSet::new(); + for cv in identity_class_family(&rc) { + if identity_key_slot(&cv).is_some() { + continue; + } + names.extend(identity_unique_key_names(&cv)); } - if slot_is_opaque(slot) || slot_is_ignored(slot) { - return None; // no per-element delta paths to re-address + if names.len() < 2 { + return None; + } + Some((rc.name().to_string(), names.into_iter().collect(), own)) +} + +/// Per distinct load-bearing `unique_keys` entry, the classes of one range +/// class's family that resolve to it. Entry-sorted, each class list name-sorted. +type LabelSpaceGroups = Vec<(String, Vec)>; + +/// Whether the classes a list ranged on this slot can hold resolve **different** +/// load-bearing `unique_keys` entries: one list, two label spaces (spike D4b/d). +/// +/// Returns the range class name and, per distinct entry, the name-sorted +/// classes resolving to it. +/// +/// This is a sharper defect than the several-entries one and is reported *in +/// addition* to it. `Gadget{gadget_identity:[code]}` with +/// `Widget is_a Gadget{aaa_widget_identity:[serial]}` labels its `Gadget` +/// elements by `code` and its `Widget` elements by `serial`, in one list: +/// navigating by a base-class label never finds a `Widget`, and a `Widget` +/// serial colliding with a `Gadget` code produces two elements with one label +/// while violating neither class's constraint. Namespacing labels by the entry +/// that produced them is the deep fix and is out of scope for this branch, so +/// the author is told instead. +/// +/// Family members whose identity is a key are excluded, as everywhere: their +/// entries are not load-bearing and so cannot split anything. +fn slot_has_split_identity_label_space(slot: &SlotView) -> Option<(String, LabelSpaceGroups)> { + if !slot_addresses_elements_by_position_or_label(slot) { + return None; } let rc = slot.get_range_class()?; - if identity_key_slot(&rc).is_some() { - return None; // the key outranks unique_keys entirely + let mut by_entry: BTreeMap> = BTreeMap::new(); + for cv in identity_class_family(&rc) { + if let Some(entry) = load_bearing_unique_key(&cv) { + by_entry + .entry(entry) + .or_default() + .push(cv.name().to_string()); + } } - let names = identity_unique_key_names(&rc); - if names.len() < 2 { + if by_entry.len() < 2 { return None; } - Some((rc.name().to_string(), names)) + let mut groups: LabelSpaceGroups = by_entry.into_iter().collect(); + for (_, classes) in groups.iter_mut() { + classes.sort(); + classes.dedup(); + } + Some((rc.name().to_string(), groups)) } /// Whether this slot's element identity, though declared, cannot discriminate @@ -152,15 +309,10 @@ fn slot_has_ambiguous_unique_keys(slot: &SlotView) -> Option<(String, Vec Option<(String, String)> { - use linkml_schemaview::slotview::{SlotContainerMode, SlotInlineMode}; - if slot.determine_slot_container_mode() != SlotContainerMode::List { - return None; // the dict form keyed by the designator is legitimate - } - if slot.determine_slot_inline_mode() == SlotInlineMode::Reference { - return None; // elements are references, not inlined - } - if slot_is_opaque(slot) || slot_is_ignored(slot) { - return None; // no per-element identity is being claimed + // Includes "not the dict form": a mapping keyed by the designator is + // legitimate, and says at-most-one-element-per-subtype. + if !slot_addresses_elements_by_position_or_label(slot) { + return None; } let rc = slot.get_range_class()?; // Asked of the metamodel's key, not the engine's: this rule exists to @@ -235,37 +387,217 @@ where !flagged(&parent_slot) } -/// The warning text for a range class offering several `unique_keys`. +/// The warning text for a range class family offering several `unique_keys`. /// -/// `names` is name-sorted, so `names[0]` is the load-bearing entry. +/// `names` is the name-sorted union over the family; `own` is the entry the +/// range class itself resolves to, which is `names[0]` unless a descendant +/// declares an earlier-sorting entry — the split case, which the divergence +/// warning is the voice for. fn ambiguous_unique_keys_detail( class_name: &str, slot_name: &str, range_class: &str, names: &[String], + own: &str, ) -> String { let quoted: Vec = names.iter().map(|n| format!("'{n}'")).collect(); format!( "elements of '{}.{}' take their identity from the unique_keys of \ - element class '{}', which declares {}: {}. Only {} is load-bearing — \ - the metamodel does not preserve declaration order, so the name-sorted \ - first entry is used, and every delta path for this slot is addressed \ - by it. Adding an earlier-sorting entry silently re-addresses them all. \ - Keep one entry, or rename deliberately.", + element class '{}' and of the classes descending from it, which \ + between them declare {} candidate entries: {}. Only one of them can be \ + load-bearing for a given element — the metamodel does not preserve \ + declaration order, so the name-sorted first entry that element's own \ + class offers is used, which for '{}' itself is '{}'. Every delta path \ + for this slot is addressed by it, and adding an earlier-sorting entry \ + anywhere in the family silently re-addresses them. Keep one entry, or \ + rename deliberately.", class_name, slot_name, range_class, names.len(), quoted.join(", "), - quoted.first().map(String::as_str).unwrap_or("none"), + range_class, + own, ) } -/// Schema-level lint: warn for every multivalued inlined slot whose element -/// identity comes from nowhere; for every list whose identity is the range -/// class's type designator, which cannot tell the elements of a homogeneous -/// list apart; and for every one whose identity is derived from a class -/// offering more than one `unique_keys` entry to derive it from. +/// The warning text for a range class family whose members resolve different +/// load-bearing `unique_keys` entries. +/// +/// `groups` is entry-sorted, and each group's classes are name-sorted, so the +/// text is stable across runs. +fn split_label_space_detail( + class_name: &str, + slot_name: &str, + range_class: &str, + groups: &[(String, Vec)], +) -> String { + let described: Vec = groups + .iter() + .map(|(entry, classes)| { + // Bounded: a wide hierarchy must not turn one warning into a wall. + // Three names name the split; the rest are counted. + let shown: Vec = classes.iter().take(3).map(|c| format!("'{c}'")).collect(); + let rest = classes.len().saturating_sub(shown.len()); + let more = if rest > 0 { + format!(" and {rest} more") + } else { + String::new() + }; + format!("'{entry}' ({}{})", shown.join(", "), more) + }) + .collect(); + format!( + "elements of '{}.{}' do not share one identity label space: a list \ + ranged on '{}' holds elements of every class descending from it, and \ + they resolve {} different load-bearing unique_keys entries — {}. Each \ + element is labelled by the entry its own class resolves to, so a delta \ + path written against one of them cannot address an element of another, \ + and two elements resolving different entries can produce the same \ + label without either class's uniqueness constraint being violated. \ + Declare the identity once, on '{}', so every element of the list is \ + labelled the same way.", + class_name, + slot_name, + range_class, + groups.len(), + described.join("; "), + range_class, + ) +} + +/// How deep the linter walks an `is_a` chain looking for a hierarchy root. +/// +/// A valid schema's `is_a` graph is a tree, so the bound is never reached; it +/// exists so that a cyclic one fails schema validation rather than hanging the +/// linter that was asked to explain it. +const MAX_IS_A_DEPTH: usize = 100; + +/// The topmost `is_a` ancestor of `class` — the class that names its hierarchy. +fn hierarchy_root(class: &ClassView) -> ClassView { + let mut current = class.clone(); + for _ in 0..MAX_IS_A_DEPTH { + match current.parent_class() { + Ok(Some(parent)) => current = parent, + _ => break, + } + } + current +} + +/// The warning text for two classes of one hierarchy sharing a `class_uri`. +fn shared_class_uri_detail(root: &str, uri: &str, classes: &[String], designator: &str) -> String { + let quoted: Vec = classes.iter().map(|c| format!("'{c}'")).collect(); + format!( + "{} declare the same class_uri '{}' and belong to one is_a hierarchy, \ + rooted at '{}', which designates its type through '{}' \ + (designates_type). A designator value is a class URI, so '{}' names \ + every one of them at once: the loader resolves it to a single class, \ + stably but by an ordering the schema does not state and nothing \ + promises to keep. Instances meaning any of the others load as the \ + winner without a diagnostic, and diff then pairs elements of different \ + classes as if they were one. Give each class a distinct class_uri. The \ + loader's choice is deliberately unchanged — this is a warning about \ + the declarations, not a behavioural fix.", + quoted.join(" and "), + uri, + root, + designator, + uri, + ) +} + +/// The warning text for a list addressed positionally although its element +/// class declares an identity: some element leaves the identity slot empty. +fn missing_labels_detail( + missing: usize, + total: usize, + range_class: &str, + identity: &str, +) -> String { + format!( + "{missing} of {total} elements of this list carry no identity label, \ + although their element class '{range_class}' declares one ({identity}). \ + A list is addressed by identity only when every element yields a label, \ + so this one is addressed positionally — ambiguous under multi-sourced \ + operation, and silently so, since an optional identity slot left empty \ + is valid data. Fill the identity slot on every element, make it \ + required, or declare the slot {OPAQUE_ANNOTATION} if the value is meant \ + to be replaced as a whole." + ) +} + +/// How the range class declares element identity, for the warning above. +fn declared_identity_description(rc: &ClassView) -> Option { + if let Some(key) = identity_key_slot(rc) { + return Some(format!("its key/identifier '{}'", key.name)); + } + let names = identity_unique_key_names(rc); + let first = names.first()?; + Some(format!("its unique_keys entry '{first}'")) +} + +/// Schema-level, class-level rule: two classes of one `is_a` hierarchy declare +/// the same `class_uri`, and the hierarchy carries a type designator (spike D8). +/// +/// Unlike the slot rules, this one has no "introducing class" to report at: +/// there is no slot, and the defect belongs to the pair of declarations rather +/// than to either of them. Emitting once per (hierarchy, shared URI) buys the +/// same thing [`introduces_flagged_slot`] buys the others — one warning per +/// thing the author would edit — so no gate predicate is needed. +/// +/// The URI compared is the class's canonical one, which is the `class_uri` when +/// declared and the schema-derived default otherwise. Two defaults can never +/// collide (they are derived from the class name), so a collision always means +/// at least one explicit declaration. +fn lint_shared_class_uris(classes: &[ClassView], sink: &mut ValidationResultSink) { + let mut hierarchies: BTreeMap<(String, String), Vec<&ClassView>> = BTreeMap::new(); + for class in classes { + let root = hierarchy_root(class); + hierarchies + .entry((root.schema_id().to_string(), root.name().to_string())) + .or_default() + .push(class); + } + for ((_, root_name), mut members) in hierarchies { + members.sort_by_key(|c| c.name().to_string()); + // The designator is declared once and inherited, so the first member + // carrying one names it for the whole hierarchy. + let Some(designator) = members + .iter() + .find_map(|c| c.get_type_designator_slot()) + .map(|d| d.name.clone()) + else { + continue; // nothing dispatches on a class URI here: not this defect + }; + let mut by_uri: BTreeMap> = BTreeMap::new(); + for member in &members { + by_uri + .entry(member.canonical_uri().to_string()) + .or_default() + .push(member.name().to_string()); + } + for (uri, mut names) in by_uri { + names.dedup(); + if names.len() < 2 { + continue; + } + sink.push_warning( + ValidationProblemType::AmbiguousElementIdentity, + names.clone(), + shared_class_uri_detail(&root_name, &uri, &names, &designator), + ); + } + } +} + +/// Schema-level lint: the module's rules 1–5 — an element identity that comes +/// from nowhere, one that is the range class's type designator and so cannot +/// tell the elements of a homogeneous list apart, one derived from a family of +/// classes offering more than one `unique_keys` entry, one where that family +/// resolves *different* entries, and two classes of a designator-carrying +/// hierarchy answering to the same `class_uri`. +/// /// Warnings only — the schema stays usable. pub fn lint_element_identity(sv: &SchemaView) -> Vec { let mut sink = ValidationResultSink::default(); @@ -273,6 +605,9 @@ pub fn lint_element_identity(sv: &SchemaView) -> Vec { let mut class_ids = sv.get_class_ids(); class_ids.sort(); let mut seen: HashSet<(String, String)> = HashSet::new(); + // Kept for the class-level rule below, which is not about any one slot and + // so cannot be answered inside the slot loop. + let mut visited: Vec = Vec::new(); for class_id in class_ids { let Ok(Some(class)) = sv.get_class(&Identifier::new(&class_id), &conv) else { continue; @@ -316,7 +651,7 @@ pub fn lint_element_identity(sv: &SchemaView) -> Vec { continue; } if !slot_is_identity_less_only(slot) { - if let Some((rc_name, names)) = slot_has_ambiguous_unique_keys(slot) { + if let Some((rc_name, names, own)) = slot_has_ambiguous_unique_keys(slot) { if introduces_flagged_slot(&class, &slot.name, |s| { slot_has_ambiguous_unique_keys(s).is_some() }) { @@ -328,10 +663,28 @@ pub fn lint_element_identity(sv: &SchemaView) -> Vec { &slot.name, &rc_name, &names, + &own, ), ); } } + // A split label space is a sharper defect than an ambiguous + // candidate set, and an additional one: both warnings fire, and + // each gets its own introduces-gate predicate, because a + // subclass can narrow the range to a non-diverging class while + // leaving the candidate set as wide as it was (and the other + // way round). + if let Some((rc_name, groups)) = slot_has_split_identity_label_space(slot) { + if introduces_flagged_slot(&class, &slot.name, |s| { + slot_has_split_identity_label_space(s).is_some() + }) { + sink.push_warning( + ValidationProblemType::AmbiguousElementIdentity, + vec![class.name().to_string(), slot.name.clone()], + split_label_space_detail(class.name(), &slot.name, &rc_name, &groups), + ); + } + } continue; } if !introduces_flagged_slot(&class, &slot.name, slot_is_identity_less_only) { @@ -370,7 +723,9 @@ pub fn lint_element_identity(sv: &SchemaView) -> Vec { detail, ); } + visited.push(class); } + lint_shared_class_uris(&visited, &mut sink); let mut warnings = sink.into_vec(); // The classes are visited in sorted id order, but a class's own slots come // from `ClassView::slots()`, which is HashMap-backed, so the warnings for a @@ -380,11 +735,18 @@ pub fn lint_element_identity(sv: &SchemaView) -> Vec { warnings } -/// Data-level lint: warn for every list container whose elements repeat a -/// declared identity (key/identifier or unique_keys value). +/// Data-level lint: the module's rules 6 and 7 — a list whose elements repeat a +/// declared identity (key/identifier or `unique_keys` value), and one addressed +/// positionally although its element class declares an identity, because some +/// element leaves the slot that identity names empty. +/// +/// Both are things only the data can show: repeated and absent values are +/// alike valid against a schema that declares a non-`required` identity. /// -/// Deliberately does NOT consult `diff.linkml.io/opaque`: a schema constraint -/// is class-level truth, and diff vocabulary never suppresses it. +/// The duplicate rule deliberately does NOT consult `diff.linkml.io/opaque`: +/// a schema constraint is class-level truth, and diff vocabulary never +/// suppresses it. The positional rule does honour it, for the reason spelled +/// out at [`check_missing_labels`]. pub fn lint_instance_identity(value: &LinkMLInstance) -> Vec { let mut sink = ValidationResultSink::default(); let mut path = Vec::new(); @@ -394,8 +756,9 @@ pub fn lint_instance_identity(value: &LinkMLInstance) -> Vec { fn walk(v: &LinkMLInstance, path: &mut Vec, sink: &mut ValidationResultSink) { match v { - LinkMLInstance::List { values, .. } => { + LinkMLInstance::List { values, slot, .. } => { check_duplicates(values, path, sink); + check_missing_labels(values, slot, path, sink); for (i, child) in values.iter().enumerate() { path.push(i.to_string()); walk(child, path, sink); @@ -416,6 +779,69 @@ fn walk(v: &LinkMLInstance, path: &mut Vec, sink: &mut ValidationResultS } } +/// Warns when this list is addressed positionally *although* its element class +/// declares an identity: some element leaves the identity slot empty and yields +/// no label, so the keyed shape fails (spike D7). +/// +/// The duplicate rule above is the other half of the same question. Together +/// they cover both ways a declared identity fails to address a list — repeated +/// labels and missing ones — and neither is visible in the schema: a `key` that +/// is not `required` may legitimately be absent, so only the data can show it. +/// The half-labelled list is the same defect at a smaller scale and is counted +/// the same way. +/// +/// Silent when nothing is declared: a class with no identity is *meant* to be +/// positional, the schema lint has already said so, and repeating it once per +/// container in the data would bury the rule that matters. Silent for a range +/// that is not a class, for the same reason. A key that is the class's type +/// designator does not count as a declaration either — the engine looks past it +/// (spec addendum rule 1), so the class declares no element identity at all and +/// the schema-level designator rule is its voice. +/// +/// Unlike the duplicate rule, this one **does** honour `opaque` and `ignore`, +/// and skips reference lists — it asks +/// [`slot_addresses_elements_by_position_or_label`] exactly as the schema rules +/// do. The two rules differ because they claim different things. A repeated +/// `unique_keys` value contradicts the class's own constraint whatever the slot +/// is annotated with, so diff vocabulary cannot silence it. This rule claims +/// only that the list *is addressed positionally*, and for a slot answering +/// "replaced as a whole" or "outside diff's scope" that claim is simply false. +/// A reference list is the sharper case: its elements are identifier strings +/// and can never carry an inlined element's identity label, so the rule would +/// fire on every such slot of every document — the shipped downstream corpus +/// has exactly one list matching it, and it is a reference list. +fn check_missing_labels( + values: &[LinkMLInstance], + slot: &SlotView, + path: &[String], + sink: &mut ValidationResultSink, +) { + if values.is_empty() { + return; // nothing to label, nothing to address + } + if !slot_addresses_elements_by_position_or_label(slot) { + return; // references, dicts, opaque and ignored slots: see above + } + let Some(rc) = slot.get_range_class() else { + return; // scalars and enums have no class to declare an identity on + }; + let Some(identity) = declared_identity_description(&rc) else { + return; // no identity declared: positional is what was asked for + }; + let missing = values + .iter() + .filter(|v| element_identity_label(v).is_none()) + .count(); + if missing == 0 { + return; + } + sink.push_warning( + ValidationProblemType::AmbiguousElementIdentity, + path.to_vec(), + missing_labels_detail(missing, values.len(), rc.name(), &identity), + ); +} + fn check_duplicates(values: &[LinkMLInstance], path: &[String], sink: &mut ValidationResultSink) { let mut seen: HashMap = HashMap::new(); for v in values { diff --git a/src/runtime/src/lib.rs b/src/runtime/src/lib.rs index 49171cc..cd284bb 100644 --- a/src/runtime/src/lib.rs +++ b/src/runtime/src/lib.rs @@ -158,8 +158,11 @@ pub enum ValidationProblemType { SlotRangeViolation, MaxCountViolation, ParsingError, - /// A multivalued inlined slot whose elements have no declared identity - /// (opt-in, [`crate::lint_element_identity`]). + /// Element identity that cannot address a list unambiguously: none is + /// declared, several declarations compete, two classes answer to one URI + /// (opt-in, [`crate::lint_element_identity`]) — or one is declared but the + /// data leaves it empty, so the list is positional after all (opt-in, + /// [`crate::lint_instance_identity`]). AmbiguousElementIdentity, /// Elements of one list repeat a declared identity (opt-in, /// [`crate::lint_instance_identity`]). diff --git a/src/runtime/tests/data/identity_descendant_unique_keys.yaml b/src/runtime/tests/data/identity_descendant_unique_keys.yaml new file mode 100644 index 0000000..6546be2 --- /dev/null +++ b/src/runtime/tests/data/identity_descendant_unique_keys.yaml @@ -0,0 +1,115 @@ +id: https://w3id.org/linkml/examples/identity_descendant_unique_keys +name: identity_descendant_unique_keys +description: |- + `unique_keys` ambiguity across an inheritance hierarchy (spike D4d). + + A list ranged on a class holds elements of that class *and of every class + descending from it*, and each element's label comes from the name-sorted + first entry of its OWN merged `unique_keys`. Inspecting only the range class + therefore answers the wrong question twice over: + + - `boxes` (range `Box`) — `Box` declares one entry, so the range class alone + looks unambiguous; `BigBox` adds a later-sorting one. Two entries are now + reachable through this slot, and the load-bearing one is still `by_id` for + every class: the candidate set is ambiguous, the label space is not. + - `gadgets` (range `Gadget`) — `Gadget` declares one entry, `Widget` adds an + earlier-sorting one. `Gadget` elements are labelled by `code` and `Widget` + elements by `serial`: one list, two label spaces, and a `Widget` serial can + collide with a `Gadget` code without either class's constraint being + violated. That is the split the second warning names. + + Guard rails: `crates` (a descendant that declares no entries of its own) and + `keyed` (a descendant whose `key` outranks `unique_keys`, so its entries are + never load-bearing and cannot split anything) must both stay silent. +license: https://creativecommons.org/publicdomain/zero/1.0/ +imports: + - linkml:types +prefixes: + identity: https://w3id.org/linkml/examples/identity_descendant_unique_keys/ + linkml: https://w3id.org/linkml/ +default_prefix: identity +default_range: string + +classes: + Depot: + attributes: + # descendant adds a LATER-sorting entry: ambiguous candidate set, one + # label space -> the several-entries warning only + boxes: + range: Box + multivalued: true + inlined_as_list: true + # descendant adds an EARLIER-sorting entry: split label space -> the + # several-entries warning AND the divergence warning + gadgets: + range: Gadget + multivalued: true + inlined_as_list: true + # descendant declares nothing of its own: one entry, one label space + crates: + range: Crate + multivalued: true + inlined_as_list: true + # the descendant's key outranks its unique_keys entirely + keyed: + range: Plate + multivalued: true + inlined_as_list: true + + Box: + unique_keys: + by_id: + unique_key_slots: [boxId] + attributes: + boxId: {range: string, required: true} + + BigBox: + is_a: Box + unique_keys: + zz_by_volume: + unique_key_slots: [volume] + attributes: + volume: {range: string, required: true} + + Gadget: + unique_keys: + gadget_identity: + unique_key_slots: [code] + attributes: + code: {range: string, required: true} + + Widget: + is_a: Gadget + unique_keys: + aaa_widget_identity: + unique_key_slots: [serial] + attributes: + serial: {range: string, required: true} + + Crate: + unique_keys: + crate_identity: + unique_key_slots: [crateId] + attributes: + crateId: {range: string, required: true} + + BigCrate: + is_a: Crate + attributes: + capacity: {range: string} + + Plate: + unique_keys: + plate_identity: + unique_key_slots: [code] + attributes: + code: {range: string, required: true} + + StampedPlate: + is_a: Plate + unique_keys: + aaa_stamp_identity: + unique_key_slots: [stamp] + attributes: + stampId: {range: string, key: true} + stamp: {range: string, required: true} diff --git a/src/runtime/tests/data/identity_missing_labels.yaml b/src/runtime/tests/data/identity_missing_labels.yaml new file mode 100644 index 0000000..d706104 --- /dev/null +++ b/src/runtime/tests/data/identity_missing_labels.yaml @@ -0,0 +1,121 @@ +id: https://w3id.org/linkml/examples/identity_missing_labels +name: identity_missing_labels +description: |- + Data whose lists are addressed *positionally despite a declared identity*: + the element class declares a key or `unique_keys`, but the data leaves the + slot it names empty, so the element yields no identity label and the whole + list falls back to positional addressing. + + The declaration is not wrong and the data is not invalid — a `key` slot that + is not `required` may legitimately be absent — so neither the schema lint nor + validation has anything to say. Only the instance lint can see it, and only + by looking at the data. + + `UnfillableKey` is the spike's D9 shape: a base class declares the type + designator, a subclass promotes it to `key` and then turns the designator + off with `slot_usage`. Nothing fills the key afterwards — the designator + machinery no longer recognises the slot — so every element of such a list is + unlabelled. It is deliberately not detected as its own schema-lint rule; it + collapses into this one, which is what the linter rustdoc says. +license: https://creativecommons.org/publicdomain/zero/1.0/ +imports: + - linkml:types +prefixes: + identity: https://w3id.org/linkml/examples/identity_missing_labels/ + linkml: https://w3id.org/linkml/ +default_prefix: identity +default_range: string + +classes: + Registry: + attributes: + name: {range: string} + # optional key: data may omit it, and then the list is positional + entries: + range: OptKey + multivalued: true + inlined_as_list: true + # the same shape declared with unique_keys instead of a key + records: + range: OptUnique + multivalued: true + inlined_as_list: true + # D9: a key that the designator override left nothing to fill + unfillable: + range: UnfillableKey + multivalued: true + inlined_as_list: true + # no identity is declared at all: the identity-less schema rule already + # speaks for this slot, and a positional list is exactly what was asked + # for. The instance lint must stay silent. + vertices: + range: Vertex + multivalued: true + inlined_as_list: true + # scalars have no element class to declare identity on + tags: + range: string + multivalued: true + # `opaque` answers the addressing question already ("replaced as a + # whole"), so "this list is addressed positionally" is not true of it. + # Unlike the duplicate rule, which reports a violated class constraint + # whatever the slot says, this one stays quiet here. + archivedEntries: + range: OptKey + multivalued: true + inlined_as_list: true + annotations: + diff.linkml.io/opaque: true + # outside diff's scope entirely + draftEntries: + range: OptKey + multivalued: true + inlined_as_list: true + annotations: + diff.linkml.io/ignore: true + # a REFERENCE list: the elements are identifier strings, which can never + # carry an inlined element's identity label. asset360's `NetElement.ports` + # is exactly this shape, and it is the only list in the whole downstream + # corpus the rule would otherwise have fired on — every one of those + # warnings would be noise. + references: + range: Referenced + multivalued: true + inlined: false + + OptKey: + attributes: + code: {range: string, key: true} + note: {range: string} + + OptUnique: + unique_keys: + by_serial: + unique_key_slots: [serial] + attributes: + serial: {range: string} + note: {range: string} + + # base declares the designator; the subclass promotes it to the key and + # switches the designator off, leaving a key nothing fills + DesignatorBase: + attributes: + typeURI: {range: uriorcurie, designates_type: true} + note: {range: string} + + UnfillableKey: + is_a: DesignatorBase + slot_usage: + typeURI: + key: true + designates_type: false + + Referenced: + attributes: + id: {range: string, identifier: true} + note: {range: string} + + Vertex: + attributes: + x: {range: float} + y: {range: float} diff --git a/src/runtime/tests/data/identity_shared_uri_designator.yaml b/src/runtime/tests/data/identity_shared_uri_designator.yaml new file mode 100644 index 0000000..34abadc --- /dev/null +++ b/src/runtime/tests/data/identity_shared_uri_designator.yaml @@ -0,0 +1,77 @@ +id: https://w3id.org/linkml/examples/identity_shared_uri_designator +name: identity_shared_uri_designator +description: |- + Two classes of one `is_a` hierarchy declaring the SAME `class_uri`, while the + hierarchy carries a type designator (spike D8). + + The designator's value is what the loader dispatches on, and it is looked up + by class URI. When two classes of one hierarchy answer to the same URI, the + loader picks one of them — stably, but by an ordering nothing in the schema + states and nothing promises to keep. Every instance carrying that designator + value loads as the winner, whatever the author meant; a diff then pairs two + elements that "are" different classes but compare equal by URI. + + The loader behaviour is deliberately left alone here: this is a warning, so + the author learns that the two declarations are not distinguishable at load + time and can rename one. + + The controls are the two halves of the condition, each removed in turn: + `PlainA`/`PlainB` share a URI in a hierarchy with no designator (nothing + dispatches on the URI, so nothing is arbitrary), and `LoneA`/`LoneB` carry + designators and share a URI but belong to two unrelated hierarchies (no + single designated slot ever has to choose between them). +license: https://creativecommons.org/publicdomain/zero/1.0/ +imports: + - linkml:types +prefixes: + shared: https://w3id.org/linkml/examples/identity_shared_uri_designator/ + linkml: https://w3id.org/linkml/ +default_prefix: shared +default_range: string + +classes: + Container: + attributes: + things: + range: TypedThing + multivalued: true + inlined_as_list: true + + TypedThing: + attributes: + typeURI: {range: uriorcurie, designates_type: true} + thingId: {range: string, key: true} + + Alpha: + is_a: TypedThing + class_uri: shared:Same + + Beta: + is_a: TypedThing + class_uri: shared:Same + + # control: same-URI siblings, no designator anywhere in the hierarchy + Plain: + attributes: + plainId: {range: string, key: true} + + PlainA: + is_a: Plain + class_uri: shared:PlainSame + + PlainB: + is_a: Plain + class_uri: shared:PlainSame + + # control: designators and a shared URI, but two unrelated hierarchies + LoneA: + class_uri: shared:LoneSame + attributes: + typeURI: {range: uriorcurie, designates_type: true} + loneId: {range: string, key: true} + + LoneB: + class_uri: shared:LoneSame + attributes: + typeURI: {range: uriorcurie, designates_type: true} + loneId: {range: string, key: true} diff --git a/src/runtime/tests/identity_lint.rs b/src/runtime/tests/identity_lint.rs index 2e0a758..8e05395 100644 --- a/src/runtime/tests/identity_lint.rs +++ b/src/runtime/tests/identity_lint.rs @@ -26,6 +26,28 @@ fn schema_view(file: &str) -> SchemaView { sv } +/// Loads `data` as an instance of `class` against a schema from `tests/data`. +/// +/// The shared [`Fixture`] is pinned to `identity.yaml`'s `Service`; the rules +/// below each get their own schema, so they need the same service generically. +fn load_into(file: &str, class: &str, data: JsonValue) -> LinkMLInstance { + let mut p = PathBuf::from(env!("CARGO_MANIFEST_DIR")); + p.push("tests/data"); + p.push(file); + let schema = from_yaml(&p).unwrap(); + let mut sv = SchemaView::new(); + sv.add_schema(schema.clone()).unwrap(); + let conv = converter_from_schema(&schema); + let cv = sv + .get_class(&Identifier::new(class), &conv) + .unwrap() + .expect("class not found"); + load_json_str(&data.to_string(), &sv, &cv, &conv) + .unwrap() + .into_instance() + .unwrap() +} + fn fixture() -> Fixture { let mut p = PathBuf::from(env!("CARGO_MANIFEST_DIR")); p.push("tests/data/identity.yaml"); @@ -207,9 +229,12 @@ fn schema_lint_warning_order_is_deterministic() { #[test] fn schema_lint_flags_a_list_whose_identity_is_the_type_designator() { // A key that is also the type designator is constant across a homogeneous - // list: every vertex of a ring carries the same value, so keyed matching - // would collapse an N-vertex ring to one element. The class "has a key", so - // the identity-less rule passes it — this rule is what sees it. + // list — every vertex of a ring carries the same value — and one value per + // subtype across a polymorphic one. Neither is element identity, so the + // engine looks past such a key entirely (spec addendum rule 1) and the list + // falls back to the class's unique_keys, or to positional addressing. This + // rule is the author-facing voice for exactly that shape, and the only one: + // it is asked first and the other unique_keys rules never see the slot. let sv = schema_view("identity_type_designator_key.yaml"); let warnings = lint_element_identity(&sv); let subjects: Vec> = warnings.iter().map(|w| w.subject.clone()).collect(); @@ -220,8 +245,10 @@ fn schema_lint_flags_a_list_whose_identity_is_the_type_designator() { // the subclass promotes it to the key with `slot_usage`, so the // rule sees it only through `SlotView::definition()`'s chain merge vec!["Ring".to_string(), "inheritedVertices".to_string()], - // the designator key outranks `unique_keys`, so `markers` is flagged - // by this rule, once, and not by the several-unique_keys rule + // `Marker` declares two `unique_keys` the designator key used to + // shadow and no longer does, so the several-entries rule matches it + // too; the designator rule is asked first and is its sole voice, so + // `markers` is flagged once, by this rule vec!["Ring".to_string(), "markers".to_string()], vec!["Ring".to_string(), "vertices".to_string()], ], @@ -447,3 +474,244 @@ fn schema_lint_leaves_single_entry_and_keyed_classes_alone() { ); } } + +#[test] +fn data_lint_flags_a_list_addressed_positionally_despite_a_declared_identity() { + // The element class declares an identity, but the data leaves the slot it + // names empty: the element yields no label, the list stops being + // keyed-shaped, and every delta addressing it goes back to being positional + // — silently, because nothing here is invalid. A `key` that is not + // `required` may legitimately be absent, so only the data can show it. + let inst = load_into( + "identity_missing_labels.yaml", + "Registry", + json!({ + "name": "reg", + // all elements unlabelled: the absent-optional-key case + "entries": [{"note": "a"}, {"note": "b"}], + // half-labelled: one element carries the unique_keys slot, one does not + "records": [{"serial": "S1"}, {"note": "b"}], + // D9: the designator override left the key with nothing to fill it + "unfillable": [{"note": "a"}, {"note": "b"}], + }), + ); + let warnings = lint_instance_identity(&inst); + let subjects: Vec> = warnings.iter().map(|w| w.subject.clone()).collect(); + assert_eq!( + subjects, + vec![ + vec!["entries".to_string()], + vec!["records".to_string()], + vec!["unfillable".to_string()], + ], + "one warning per container, at the container's path: {warnings:#?}" + ); + for w in &warnings { + assert_eq!( + w.problem_type, + ValidationProblemType::AmbiguousElementIdentity + ); + assert!(!w.severity.is_error(), "the linter warns, never errors"); + assert!( + w.detail.contains("positional"), + "the warning must say what the engine falls back to: {}", + w.detail + ); + } + let detail = |slot: &str| { + warnings + .iter() + .find(|w| w.subject[0] == slot) + .map(|w| w.detail.clone()) + .unwrap_or_default() + }; + assert!( + detail("entries").contains("2 of 2") && detail("entries").contains("'OptKey'"), + "the warning must count the unlabelled elements and name the class \ + whose identity is unfilled: {}", + detail("entries") + ); + assert!( + detail("records").contains("1 of 2"), + "a half-labelled list is the same defect, and its count says so: {}", + detail("records") + ); +} + +#[test] +fn data_lint_positional_despite_identity_only_speaks_for_inlined_addressable_lists() { + // Guard rails, each for a different reason. + // + // A class that declares no identity is *meant* to be positional: the schema + // lint already speaks for it, and repeating that once per container in the + // data would drown the rule that matters. Scalars have no element class at + // all. `opaque` and `ignore` answer the addressing question themselves, so + // "this list is addressed positionally" is not true of them — the duplicate + // rule ignores both annotations because a repeated identity contradicts the + // class's own constraint, which is a different claim. + // + // The reference list is the one that matters in practice: its elements are + // identifier strings, which can never carry an inlined element's identity + // label, so without this guard the rule fires on every reference list ranged + // on a keyed class. asset360's `NetElement.ports` is that shape, and it is + // the ONLY list in the downstream corpus the unguarded rule fired on. + let inst = load_into( + "identity_missing_labels.yaml", + "Registry", + json!({ + "name": "reg", + "vertices": [{"x": 1.0, "y": 2.0}, {"x": 1.0, "y": 2.0}], + "tags": ["a", "b"], + "archivedEntries": [{"note": "a"}], + "draftEntries": [{"note": "a"}], + "references": ["urn:a", "urn:b"], + }), + ); + let warnings = lint_instance_identity(&inst); + assert!(warnings.is_empty(), "{warnings:#?}"); +} + +#[test] +fn data_lint_positional_despite_identity_is_silent_on_fully_labelled_lists() { + // The rule must not fire on the lists it was written to leave alone: every + // element labelled is exactly the keyed shape. + let inst = load_into( + "identity_missing_labels.yaml", + "Registry", + json!({ + "name": "reg", + "entries": [{"code": "a"}, {"code": "b"}], + "records": [{"serial": "S1"}, {"serial": "S2"}], + // an empty list has no unlabelled element to complain about + "unfillable": [], + }), + ); + let warnings = lint_instance_identity(&inst); + assert!(warnings.is_empty(), "{warnings:#?}"); +} + +#[test] +fn schema_lint_counts_unique_keys_entries_across_the_range_class_descendants() { + // A list ranged on a class holds elements of every class descending from it, + // and each element is labelled by its OWN merged `unique_keys`. Inspecting + // only the range class misses a descendant's entry entirely: `Box` alone + // declares one entry and looks unambiguous, while `BigBox` adds a second + // that any element of the list may be labelled by. + let sv = schema_view("identity_descendant_unique_keys.yaml"); + let warnings = lint_element_identity(&sv); + let boxes: Vec<_> = warnings + .iter() + .filter(|w| w.subject == vec!["Depot".to_string(), "boxes".to_string()]) + .collect(); + assert_eq!( + boxes.len(), + 1, + "a later-sorting descendant entry widens the candidate set without \ + splitting the label space: one warning: {warnings:#?}" + ); + assert!( + boxes[0].detail.contains("'by_id'") && boxes[0].detail.contains("'zz_by_volume'"), + "the warning must name every candidate, wherever it is declared: {}", + boxes[0].detail + ); + // guard rails: a descendant declaring nothing, and one whose key outranks + // its entries, leave the range class as unambiguous as it was + let flagged: Vec = warnings.iter().map(|w| w.subject[1].clone()).collect(); + for silent in ["crates", "keyed"] { + assert!( + !flagged.contains(&silent.to_string()), + "{silent} must stay silent, got {warnings:#?}" + ); + } +} + +#[test] +fn schema_lint_flags_descendants_that_resolve_different_load_bearing_entries() { + // The split label space: `Gadget` elements are labelled by `gadget_identity` + // and `Widget` elements by the earlier-sorting `aaa_widget_identity` it + // adds. One list, two label spaces — navigating by a base label misses a + // Widget, and a Widget serial colliding with a Gadget code violates neither + // class's constraint. This is an ADDITIONAL warning: the several-entries + // rule still fires, because the candidate set is ambiguous too. + let sv = schema_view("identity_descendant_unique_keys.yaml"); + let warnings = lint_element_identity(&sv); + let gadgets: Vec<_> = warnings + .iter() + .filter(|w| w.subject == vec!["Depot".to_string(), "gadgets".to_string()]) + .collect(); + assert_eq!( + gadgets.len(), + 2, + "the split gets its own warning on top of the several-entries one: \ + {warnings:#?}" + ); + let split = gadgets + .iter() + .find(|w| w.detail.contains("Widget")) + .expect("one of the two must name the diverging descendant"); + assert_eq!( + split.problem_type, + ValidationProblemType::AmbiguousElementIdentity + ); + assert!(!split.severity.is_error(), "the linter warns, never errors"); + for needle in [ + "'Gadget'", + "'gadget_identity'", + "'Widget'", + "'aaa_widget_identity'", + ] { + assert!( + split.detail.contains(needle), + "the warning must name each diverging class and the entry it \ + resolves to; missing {needle}: {}", + split.detail + ); + } + // `boxes` agrees on `by_id` throughout, so it must NOT get this warning + assert_eq!( + warnings + .iter() + .filter(|w| w.subject[1] == "boxes") + .filter(|w| w.detail.contains("zz_by_volume") && w.detail.contains("'BigBox'")) + .count(), + 0, + "a widened candidate set is not a split label space: {warnings:#?}" + ); +} + +#[test] +fn schema_lint_flags_a_shared_class_uri_within_a_designator_hierarchy() { + // Two classes of one hierarchy answering to one `class_uri`, in a hierarchy + // whose designator is dispatched by exactly that URI: the loader picks one + // of them, stably but by an ordering the schema never states. + let sv = schema_view("identity_shared_uri_designator.yaml"); + let warnings = lint_element_identity(&sv); + let subjects: Vec> = warnings.iter().map(|w| w.subject.clone()).collect(); + assert_eq!( + subjects, + vec![vec!["Alpha".to_string(), "Beta".to_string()]], + "the sharing classes are the subject, and the controls stay silent: \ + {warnings:#?}" + ); + let w = &warnings[0]; + assert_eq!( + w.problem_type, + ValidationProblemType::AmbiguousElementIdentity + ); + assert!(!w.severity.is_error(), "the linter warns, never errors"); + assert!( + w.detail.contains("Alpha") && w.detail.contains("Beta"), + "the warning must name both classes: {}", + w.detail + ); + assert!( + w.detail.contains("designates_type") && w.detail.contains("class_uri"), + "the warning must name both halves of the condition: {}", + w.detail + ); + assert!( + w.detail.contains("typeURI"), + "the warning must name the designator the URI is dispatched by: {}", + w.detail + ); +} From fc3ce5310befae881faa2e22c3a1e52c60cd3f94 Mon Sep 17 00:00:00 2001 From: ejsyx Date: Wed, 19 Aug 2026 14:35:54 +0200 Subject: [PATCH 48/58] fix(runtime): identity lint gate subtraction, key-labelled split groups MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Four review items on the rule-6 lint extensions. Gate subtraction (real lost warning). `introduces_flagged_slot` asks "is the parent's slot flagged for the same reason?", and rule 1's predicate subtracts the designator case from that question. Rules 3 and 4 were gated on the raw predicates, but since the designator key stopped shadowing its class's `unique_keys` (spec addendum rule 1) a designator-keyed range class satisfies both raw shapes — while the designator rule is asked first and is the slot's only voice, so the parent emitted neither warning. A subclass that slot_usage-retargets the slot onto a keyless multi-entry class was therefore suppressed by a parent warning that does not exist, and the ambiguity was reported nowhere. Both gates now subtract the designator case, symmetric with rule 1. Key-labelled classes are their own split group. The divergence rule skipped family members with a non-designator key, claiming their entries "cannot split anything". Their *entries* are indeed not load-bearing — which is why the several-entries rule still skips them — but the key itself labels the elements and occupies its own label space: `Plate` elements addressed by a `plate_identity` value and `StampedPlate` elements by a `stampId` collide across the two spaces without breaking either class's constraint. Grouping is now on `identity_labelling`, which renders the declaration `element_identity_label` actually reads, so a key and an entry are different groups and never collide as group names. `declared_identity_description` collapses onto the same function — it asks the identical question — which also makes the instance rule's message distinguish `key` from `identifier`. Docs: the module rustdoc summary miscounted the rules (four + two = six; it is five + two = seven), and the Python binding docstrings plus the matching .pyi stubs predated rules 5-7. Sanctioned exact-set change: `Depot.keyed` now warns (divergence only, not several-entries), so `schema_lint_counts_...` was narrowed to the candidate-set claim and a new test asserts the divergence warning. Harness: fixture-block movement only; asset360 27 and rinf 19 warnings unchanged, self-diffs empty — no downstream schema has a key-vs-entry family. Co-Authored-By: Claude Fable 5 --- .../python/linkml_runtime_rust/_native.pyi | 31 +++- src/python/src/lib.rs | 29 +++- src/runtime/src/identity_lint.rs | 160 +++++++++++++----- .../data/identity_descendant_unique_keys.yaml | 82 ++++++++- src/runtime/tests/identity_lint.rs | 120 ++++++++++++- 5 files changed, 355 insertions(+), 67 deletions(-) diff --git a/src/python/python/linkml_runtime_rust/_native.pyi b/src/python/python/linkml_runtime_rust/_native.pyi index 68b7869..a5f19e8 100644 --- a/src/python/python/linkml_runtime_rust/_native.pyi +++ b/src/python/python/linkml_runtime_rust/_native.pyi @@ -5485,18 +5485,37 @@ def import_turtle(reader:typing.Any, schema_view:SchemaView, root_classes:typing def lint_element_identity(schema_view:SchemaView) -> builtins.list[ValidationResult]: r""" - Schema-level lint: warn for every multivalued inlined slot whose element - identity comes from nowhere. - + Schema-level lint: warn where a multivalued inlined slot's element identity + is absent, ambiguous, or cannot address the list. + + Five rules: (1) no identity declared at all; (2) the declared identity is + the element class's type designator, whose value describes the class rather + than the element; (3) several ``unique_keys`` entries to choose from across + the range class and its descendants, of which only the name-sorted first is + load-bearing; (4) those classes labelled in different ways, so one list + carries two label spaces; (5) two classes of one ``is_a`` hierarchy + declaring the same ``class_uri`` while the hierarchy designates its type. + Warnings only — the schema stays usable. Results are deterministic: sorted by subject, deduplicated across class URIs, and an inherited slot is - reported once, at the class that introduces the problem. + reported once, at the class that introduces the problem. Rules 1-4 are + per-slot and their ``subject`` is ``[class_name, slot_name]``; rule 5 is + class-level and its ``subject`` is the list of classes sharing the URI. """ def lint_instance_identity(instance:LinkMLInstance) -> builtins.list[ValidationResult]: r""" - Data-level lint: warn for every list whose elements repeat a declared - identity (key/identifier or ``unique_keys`` value). + Data-level lint: warn where loaded data defeats a declared element identity. + + Two rules: a list whose elements repeat a declared identity — key/identifier + or ``unique_keys`` value — reported as ``duplicate_element_identity``; and a + list addressed positionally *despite* a declared identity, because some + element leaves the slot that identity names empty, reported as + ``ambiguous_element_identity``. Neither is visible in the schema: an + identity slot that is not ``required`` may be absent, and repeated or + missing values are alike valid data. + + Warnings only. ``subject`` is the container's instance path. """ def load_json(source:typing.Any, sv:SchemaView, class_view:ClassView) -> tuple[typing.Optional[LinkMLInstance], builtins.list[ValidationResult]]: ... diff --git a/src/python/src/lib.rs b/src/python/src/lib.rs index 6168c73..cb00180 100644 --- a/src/python/src/lib.rs +++ b/src/python/src/lib.rs @@ -1567,12 +1567,22 @@ fn py_patch( // ── Identity lints ────────────────────────────────────────────────────────── -/// Schema-level lint: warn for every multivalued inlined slot whose element -/// identity comes from nowhere. +/// Schema-level lint: warn where a multivalued inlined slot's element identity +/// is absent, ambiguous, or cannot address the list. +/// +/// Five rules: (1) no identity declared at all; (2) the declared identity is +/// the element class's type designator, whose value describes the class rather +/// than the element; (3) several ``unique_keys`` entries to choose from across +/// the range class and its descendants, of which only the name-sorted first is +/// load-bearing; (4) those classes labelled in different ways, so one list +/// carries two label spaces; (5) two classes of one ``is_a`` hierarchy +/// declaring the same ``class_uri`` while the hierarchy designates its type. /// /// Warnings only — the schema stays usable. Results are deterministic: sorted /// by subject, deduplicated across class URIs, and an inherited slot is -/// reported once, at the class that introduces the problem. +/// reported once, at the class that introduces the problem. Rules 1-4 are +/// per-slot and their ``subject`` is ``[class_name, slot_name]``; rule 5 is +/// class-level and its ``subject`` is the list of classes sharing the URI. #[cfg_attr(feature = "stubgen", gen_stub_pyfunction)] #[pyfunction(name = "lint_element_identity")] fn py_lint_element_identity( @@ -1582,8 +1592,17 @@ fn py_lint_element_identity( validation_results_to_py(py, lint_element_identity(schema_view.as_rust())) } -/// Data-level lint: warn for every list whose elements repeat a declared -/// identity (key/identifier or ``unique_keys`` value). +/// Data-level lint: warn where loaded data defeats a declared element identity. +/// +/// Two rules: a list whose elements repeat a declared identity — key/identifier +/// or ``unique_keys`` value — reported as ``duplicate_element_identity``; and a +/// list addressed positionally *despite* a declared identity, because some +/// element leaves the slot that identity names empty, reported as +/// ``ambiguous_element_identity``. Neither is visible in the schema: an +/// identity slot that is not ``required`` may be absent, and repeated or +/// missing values are alike valid data. +/// +/// Warnings only. ``subject`` is the container's instance path. #[cfg_attr(feature = "stubgen", gen_stub_pyfunction)] #[pyfunction(name = "lint_instance_identity")] fn py_lint_instance_identity( diff --git a/src/runtime/src/identity_lint.rs b/src/runtime/src/identity_lint.rs index d1d3f1e..824a5d2 100644 --- a/src/runtime/src/identity_lint.rs +++ b/src/runtime/src/identity_lint.rs @@ -25,8 +25,8 @@ //! //! # The rules //! -//! [`lint_element_identity`] asks four questions of a schema, and -//! [`lint_instance_identity`] two of a loaded instance. All six warn; none +//! [`lint_element_identity`] asks five questions of a schema, and +//! [`lint_instance_identity`] two of a loaded instance. All seven warn; none //! errors, and none changes what the engine does. //! //! ## Schema rules @@ -54,13 +54,17 @@ //! candidates are counted across the range class **and every class //! descending from it**, since a list ranged on a class holds elements of //! all of them. -//! 4. **A split label space.** Those same classes resolve *different* -//! load-bearing entries: `Gadget` elements labelled by one entry and -//! `Widget is_a Gadget` elements by another, in one list. A path written -//! against one label space cannot address an element of the other, and two -//! elements resolving different entries can carry the same label without -//! violating either class's uniqueness constraint. Reported in addition to -//! rule 3, which such a family always also matches. +//! 4. **A split label space.** Those same classes are labelled *different +//! ways*: `Gadget` elements by one `unique_keys` entry and +//! `Widget is_a Gadget` elements by another, in one list — or one of them by +//! a `key` and the other by an entry, which splits the space just as +//! thoroughly. A path written against one label space cannot address an +//! element of the other, and two elements labelled different ways can carry +//! the same label without violating either class's uniqueness constraint. +//! Rule 3 and this one ask different questions of the same family: rule 3 is +//! about which *entry* is chosen, so a key-labelled class is outside it, +//! while this one is about which *declaration* labels each element, so a +//! key-labelled class is one of the groups. //! 5. **A shared `class_uri` under a designator.** Two classes of one `is_a` //! hierarchy declare the same `class_uri` while the hierarchy designates its //! type. A designator value is a class URI, so it names both classes at @@ -72,8 +76,11 @@ //! flagged slot inherited unchanged by a descendant is not repeated there, //! since the declaration the author would edit lives on the ancestor. Each rule //! gates on its own predicate, so a subclass whose `slot_usage` changes one -//! rule's answer is judged on its own merits for that rule. Rule 5 is -//! class-level and has no slot to attribute, so it is emitted once per +//! rule's answer is judged on its own merits for that rule — and each of rules +//! 1, 3 and 4 subtracts rule 2's cases from its gate, because a designator-keyed +//! range class matches all three raw shapes while rule 2 is the slot's only +//! voice: a parent that emitted no warning must not suppress a subclass's. +//! Rule 5 is class-level and has no slot to attribute, so it is emitted once per //! (hierarchy, shared URI) instead. //! //! ## Instance rules @@ -246,14 +253,38 @@ fn slot_has_ambiguous_unique_keys(slot: &SlotView) -> Option<(String, Vec Option { + if let Some(key) = identity_key_slot(rc) { + let kind = if key.definition().identifier == Some(true) { + "identifier" + } else { + "key" + }; + return Some(format!("{kind} '{}'", key.name)); + } + let entry = identity_unique_key_names(rc).into_iter().next()?; + Some(format!("unique_keys entry '{entry}'")) +} + +/// Per distinct identity labelling, the classes of one range class's family +/// that resolve to it. Labelling-sorted, each class list name-sorted. type LabelSpaceGroups = Vec<(String, Vec)>; /// Whether the classes a list ranged on this slot can hold resolve **different** -/// load-bearing `unique_keys` entries: one list, two label spaces (spike D4b/d). +/// identity labellings: one list, two label spaces (spike D4b/d). /// -/// Returns the range class name and, per distinct entry, the name-sorted +/// Returns the range class name and, per distinct labelling, the name-sorted /// classes resolving to it. /// /// This is a sharper defect than the several-entries one and is reported *in @@ -266,26 +297,32 @@ type LabelSpaceGroups = Vec<(String, Vec)>; /// that produced them is the deep fix and is out of scope for this branch, so /// the author is told instead. /// -/// Family members whose identity is a key are excluded, as everywhere: their -/// entries are not load-bearing and so cannot split anything. +/// A family member whose identity is a `key` is counted as its own group, not +/// skipped. Its `unique_keys` entries are indeed never load-bearing — which is +/// why the several-entries rule ignores them — but the key itself labels its +/// elements, and it splits the list's label space exactly as a second entry +/// would: `Plate` elements addressed by a `plate_identity` value and +/// `StampedPlate` elements by a `stampId` collide across the two spaces without +/// breaking either class's constraint. The two rules ask different questions of +/// the same family, and only one of them is about entry *choice*. fn slot_has_split_identity_label_space(slot: &SlotView) -> Option<(String, LabelSpaceGroups)> { if !slot_addresses_elements_by_position_or_label(slot) { return None; } let rc = slot.get_range_class()?; - let mut by_entry: BTreeMap> = BTreeMap::new(); + let mut by_labelling: BTreeMap> = BTreeMap::new(); for cv in identity_class_family(&rc) { - if let Some(entry) = load_bearing_unique_key(&cv) { - by_entry - .entry(entry) + if let Some(labelling) = identity_labelling(&cv) { + by_labelling + .entry(labelling) .or_default() .push(cv.name().to_string()); } } - if by_entry.len() < 2 { + if by_labelling.len() < 2 { return None; } - let mut groups: LabelSpaceGroups = by_entry.into_iter().collect(); + let mut groups: LabelSpaceGroups = by_labelling.into_iter().collect(); for (_, classes) in groups.iter_mut() { classes.sort(); classes.dedup(); @@ -336,6 +373,32 @@ fn slot_is_identity_less_only(slot: &SlotView) -> bool { slot_lacks_element_identity(slot) && slot_identity_is_type_designator(slot).is_none() } +/// The several-entries rule as the reporting loop applies it. +/// +/// Same subtraction, same reason as [`slot_is_identity_less_only`], and needed +/// for the same reason it is: the designator key stopped shadowing its class's +/// `unique_keys` (spec addendum rule 1), so a designator-keyed range class with +/// several entries satisfies the raw shape while the designator rule remains +/// the slot's only voice. +/// +/// Used only as [`introduces_flagged_slot`]'s predicate, where the omission +/// loses a real warning: a parent whose slot fires the designator rule would +/// otherwise suppress a subclass that `slot_usage`-retargets the slot onto a +/// keyless multi-entry class, and the ambiguity would be reported nowhere. The +/// reporting loop's own call site does not need it — a designator-keyed slot +/// has already `continue`d by then. +fn slot_has_ambiguous_unique_keys_only(slot: &SlotView) -> bool { + slot_has_ambiguous_unique_keys(slot).is_some() + && slot_identity_is_type_designator(slot).is_none() +} + +/// The divergence rule as the reporting loop applies it, subtracted exactly as +/// [`slot_has_ambiguous_unique_keys_only`] is and for the same reason. +fn slot_has_split_identity_label_space_only(slot: &SlotView) -> bool { + slot_has_split_identity_label_space(slot).is_some() + && slot_identity_is_type_designator(slot).is_none() +} + /// The warning text for a list whose identity is its element class's type /// designator. fn type_designator_identity_detail( @@ -422,10 +485,10 @@ fn ambiguous_unique_keys_detail( } /// The warning text for a range class family whose members resolve different -/// load-bearing `unique_keys` entries. +/// identity labellings. /// -/// `groups` is entry-sorted, and each group's classes are name-sorted, so the -/// text is stable across runs. +/// `groups` is labelling-sorted, and each group's classes are name-sorted, so +/// the text is stable across runs. fn split_label_space_detail( class_name: &str, slot_name: &str, @@ -434,7 +497,7 @@ fn split_label_space_detail( ) -> String { let described: Vec = groups .iter() - .map(|(entry, classes)| { + .map(|(labelling, classes)| { // Bounded: a wide hierarchy must not turn one warning into a wall. // Three names name the split; the rest are counted. let shown: Vec = classes.iter().take(3).map(|c| format!("'{c}'")).collect(); @@ -444,19 +507,19 @@ fn split_label_space_detail( } else { String::new() }; - format!("'{entry}' ({}{})", shown.join(", "), more) + format!("{labelling} ({}{})", shown.join(", "), more) }) .collect(); format!( "elements of '{}.{}' do not share one identity label space: a list \ ranged on '{}' holds elements of every class descending from it, and \ - they resolve {} different load-bearing unique_keys entries — {}. Each \ - element is labelled by the entry its own class resolves to, so a delta \ - path written against one of them cannot address an element of another, \ - and two elements resolving different entries can produce the same \ - label without either class's uniqueness constraint being violated. \ - Declare the identity once, on '{}', so every element of the list is \ - labelled the same way.", + they are labelled {} different ways — {}. Each element is labelled by \ + the declaration its own class resolves to, so a delta path written \ + against one of them cannot address an element of another, and two \ + elements labelled different ways can produce the same label without \ + either class's uniqueness constraint being violated. Declare the \ + identity once, on '{}', so every element of the list is labelled the \ + same way.", class_name, slot_name, range_class, @@ -528,13 +591,12 @@ fn missing_labels_detail( } /// How the range class declares element identity, for the warning above. +/// +/// The same question [`identity_labelling`] answers for the divergence rule — +/// "which declaration does `element_identity_label` read?" — so it is the same +/// function, and `None` means the same thing in both: no identity is declared. fn declared_identity_description(rc: &ClassView) -> Option { - if let Some(key) = identity_key_slot(rc) { - return Some(format!("its key/identifier '{}'", key.name)); - } - let names = identity_unique_key_names(rc); - let first = names.first()?; - Some(format!("its unique_keys entry '{first}'")) + Some(format!("its {}", identity_labelling(rc)?)) } /// Schema-level, class-level rule: two classes of one `is_a` hierarchy declare @@ -652,9 +714,11 @@ pub fn lint_element_identity(sv: &SchemaView) -> Vec { } if !slot_is_identity_less_only(slot) { if let Some((rc_name, names, own)) = slot_has_ambiguous_unique_keys(slot) { - if introduces_flagged_slot(&class, &slot.name, |s| { - slot_has_ambiguous_unique_keys(s).is_some() - }) { + if introduces_flagged_slot( + &class, + &slot.name, + slot_has_ambiguous_unique_keys_only, + ) { sink.push_warning( ValidationProblemType::AmbiguousElementIdentity, vec![class.name().to_string(), slot.name.clone()], @@ -675,9 +739,11 @@ pub fn lint_element_identity(sv: &SchemaView) -> Vec { // leaving the candidate set as wide as it was (and the other // way round). if let Some((rc_name, groups)) = slot_has_split_identity_label_space(slot) { - if introduces_flagged_slot(&class, &slot.name, |s| { - slot_has_split_identity_label_space(s).is_some() - }) { + if introduces_flagged_slot( + &class, + &slot.name, + slot_has_split_identity_label_space_only, + ) { sink.push_warning( ValidationProblemType::AmbiguousElementIdentity, vec![class.name().to_string(), slot.name.clone()], diff --git a/src/runtime/tests/data/identity_descendant_unique_keys.yaml b/src/runtime/tests/data/identity_descendant_unique_keys.yaml index 6546be2..a72f91c 100644 --- a/src/runtime/tests/data/identity_descendant_unique_keys.yaml +++ b/src/runtime/tests/data/identity_descendant_unique_keys.yaml @@ -18,9 +18,25 @@ description: |- collide with a `Gadget` code without either class's constraint being violated. That is the split the second warning names. - Guard rails: `crates` (a descendant that declares no entries of its own) and - `keyed` (a descendant whose `key` outranks `unique_keys`, so its entries are - never load-bearing and cannot split anything) must both stay silent. + - `keyed` (range `Plate`) — `StampedPlate` declares a `key`, which outranks + its `unique_keys` entirely. Its entries are therefore never load-bearing + and do not widen the candidate set — but the key itself is a label space: + `Plate` elements are labelled by `plate_identity` and `StampedPlate` + elements by `stampId`, and the two can collide exactly as two entries can. + The divergence rule counts a key-labelled class as its own group, so this + slot gets the divergence warning and *not* the several-entries one. + + Guard rail: `crates` (a descendant that declares no entries of its own) must + stay silent. + + `Base.stamps` is the gate-subtraction case. Its range class is + designator-keyed, so the designator rule is its only voice — but the raw + several-entries and divergence shapes also match it (the designator key no + longer shadows `DesignatorMarker`'s entries, and `AaaSubMarker` adds an + earlier-sorting one). A subclass that `slot_usage`-retargets the slot to a + class those rules really do speak for must therefore not be suppressed by a + parent that never emitted their warning: `Retargeted` and `SplitRetargeted` + are the two halves of that. license: https://creativecommons.org/publicdomain/zero/1.0/ imports: - linkml:types @@ -50,12 +66,39 @@ classes: range: Crate multivalued: true inlined_as_list: true - # the descendant's key outranks its unique_keys entirely + # the descendant's key outranks its own unique_keys, so the candidate set + # is unchanged — but the key is a second label space -> divergence only keyed: range: Plate multivalued: true inlined_as_list: true + # the gate-subtraction cases: this slot's range class is designator-keyed, so + # the designator rule is its sole voice, even though the raw several-entries + # and divergence shapes both match it + Base: + attributes: + stamps: + range: DesignatorMarker + multivalued: true + inlined_as_list: true + + # retargets the inherited slot onto a keyless two-entry class: the + # several-entries rule speaks for it HERE, and the parent — which only ever + # emitted the designator warning — cannot stand in for that + Retargeted: + is_a: Base + slot_usage: + stamps: + range: TwoEntry + + # the same, for the divergence rule + SplitRetargeted: + is_a: Base + slot_usage: + stamps: + range: Gadget + Box: unique_keys: by_id: @@ -113,3 +156,34 @@ classes: attributes: stampId: {range: string, key: true} stamp: {range: string, required: true} + + DesignatorMarker: + unique_keys: + by_code: + unique_key_slots: [code] + zz_by_label: + unique_key_slots: [label] + attributes: + typeURI: {range: uriorcurie, key: true, designates_type: true} + code: {range: string, required: true} + label: {range: string, required: true} + + # an earlier-sorting entry, so `DesignatorMarker`'s family also matches the + # divergence shape raw — the second half of the gate-subtraction case + AaaSubMarker: + is_a: DesignatorMarker + unique_keys: + aaa_sub_identity: + unique_key_slots: [sub] + attributes: + sub: {range: string, required: true} + + TwoEntry: + unique_keys: + by_alpha: + unique_key_slots: [alpha] + zz_by_beta: + unique_key_slots: [beta] + attributes: + alpha: {range: string, required: true} + beta: {range: string, required: true} diff --git a/src/runtime/tests/identity_lint.rs b/src/runtime/tests/identity_lint.rs index 8e05395..028d010 100644 --- a/src/runtime/tests/identity_lint.rs +++ b/src/runtime/tests/identity_lint.rs @@ -614,15 +614,125 @@ fn schema_lint_counts_unique_keys_entries_across_the_range_class_descendants() { "the warning must name every candidate, wherever it is declared: {}", boxes[0].detail ); - // guard rails: a descendant declaring nothing, and one whose key outranks - // its entries, leave the range class as unambiguous as it was + // guard rail: a descendant declaring nothing of its own leaves the range + // class as unambiguous as it was let flagged: Vec = warnings.iter().map(|w| w.subject[1].clone()).collect(); - for silent in ["crates", "keyed"] { + assert!( + !flagged.contains(&"crates".to_string()), + "crates must stay silent, got {warnings:#?}" + ); + // `StampedPlate`'s key outranks its own `unique_keys`, so its entries never + // widen the candidate set: this rule has nothing to say about `keyed`. (The + // divergence rule does — the key is its own label space.) + let keyed_candidates: Vec<_> = warnings + .iter() + .filter(|w| w.subject[1] == "keyed") + .filter(|w| w.detail.contains("candidate entries")) + .collect(); + assert!( + keyed_candidates.is_empty(), + "a key-labelled descendant does not widen the candidate set: \ + {keyed_candidates:#?}" + ); +} + +#[test] +fn schema_lint_splits_a_label_space_between_a_key_and_a_unique_keys_entry() { + // A descendant that declares a `key` does not widen the candidate set — its + // own `unique_keys` stop being load-bearing — but it very much occupies its + // own label space: `Plate` elements are labelled by `plate_identity` and + // `StampedPlate` elements by `stampId`, in one list. A path written against + // one cannot address an element of the other, and a `stampId` colliding + // with a `plate_identity` value breaks neither class's constraint. Counting + // a key-labelled class as its own group is what sees that. + let sv = schema_view("identity_descendant_unique_keys.yaml"); + let warnings = lint_element_identity(&sv); + let keyed: Vec<_> = warnings + .iter() + .filter(|w| w.subject == vec!["Depot".to_string(), "keyed".to_string()]) + .collect(); + assert_eq!( + keyed.len(), + 1, + "the divergence warning, and only it, speaks for `keyed`: {warnings:#?}" + ); + for needle in ["'Plate'", "'plate_identity'", "'StampedPlate'", "'stampId'"] { assert!( - !flagged.contains(&silent.to_string()), - "{silent} must stay silent, got {warnings:#?}" + keyed[0].detail.contains(needle), + "the warning must name both labellings and the class each belongs \ + to; missing {needle}: {}", + keyed[0].detail ); } + assert!( + keyed[0].detail.contains("key 'stampId'"), + "the warning must say that one of the two labellings is a key, not a \ + unique_keys entry: {}", + keyed[0].detail + ); +} + +#[test] +fn schema_lint_reports_a_retargeted_slot_the_parent_never_warned_about() { + // The introduces-gate asks "is the parent's slot flagged for the same + // reason?". For the two unique_keys rules that question has to subtract the + // designator case, exactly as the identity-less rule's gate does: a + // designator-keyed range class satisfies both raw shapes (the key no longer + // shadows the entries), but the designator rule is asked first and is the + // slot's ONLY voice, so the parent never emitted either warning. Without + // the subtraction a subclass that `slot_usage`-retargets the slot onto a + // class those rules really do speak for is suppressed by a parent warning + // that does not exist, and the ambiguity is reported nowhere. + let sv = schema_view("identity_descendant_unique_keys.yaml"); + let warnings = lint_element_identity(&sv); + let subject = |c: &str| vec![c.to_string(), "stamps".to_string()]; + + let base: Vec<_> = warnings + .iter() + .filter(|w| w.subject == subject("Base")) + .collect(); + assert_eq!( + base.len(), + 1, + "the designator rule speaks alone: {warnings:#?}" + ); + assert!( + base[0].detail.contains("designates_type"), + "and it is the designator rule: {}", + base[0].detail + ); + + let retargeted: Vec<_> = warnings + .iter() + .filter(|w| w.subject == subject("Retargeted")) + .collect(); + assert_eq!( + retargeted.len(), + 1, + "a retarget onto a keyless two-entry class must be reported here: \ + {warnings:#?}" + ); + assert!( + retargeted[0].detail.contains("'by_alpha'") + && retargeted[0].detail.contains("'zz_by_beta'"), + "and by the several-entries rule: {}", + retargeted[0].detail + ); + + let split: Vec<_> = warnings + .iter() + .filter(|w| w.subject == subject("SplitRetargeted")) + .collect(); + assert_eq!( + split.len(), + 2, + "a retarget onto a split family must get both warnings here: \ + {warnings:#?}" + ); + assert!( + split.iter().any(|w| w.detail.contains("label space")), + "one of them being the divergence rule: {split:#?}" + ); } #[test] From 3aaa676d9ad11bbfb19e5c5f1e088583f9b478ae Mon Sep 17 00:00:00 2001 From: ejsyx Date: Wed, 19 Aug 2026 14:55:50 +0200 Subject: [PATCH 49/58] feat(tools): --lint-identity flag on linkml-validate MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `lint_instance_identity` was reachable from no CLI at all, so the two instance rules — a list repeating a declared identity, and one addressed positionally because some element leaves the identity slot empty (spec addendum rule 6, D7) — were invisible to the downstream differential harness however the corpus was exercised. Only the schema lint had a CLI. Modelled on `linkml-schema-validate --lint-identity`: opt-in, warnings never change the exit code, and deliberately skipped when the data does not validate (the lint asks how a list's elements are addressed, and a tree whose loading already went wrong answers about the damage rather than about the data). Text output uses the same `warning[]: ` form; JSON gains `identity_warnings` / `identity_lint_skipped` / `identity_lint_skipped_reason`, using `ValidationProblemType::label()` as that binary already does. The existing `issues` shape is untouched. Without the flag the output is byte-identical to before — verified by running the pre-change and post-change binaries over valid and invalid data, in text and JSON, and diffing including exit codes. The JSON lint keys are absent, not null, when the flag is off. `identity_missing_labels_data.json` is the harness canary: data that trips both instance rules while its guard-rail slots (undeclared range, scalar range, reference list, `opaque`) stay silent, so a zero warning count can never mean "the flag stopped being passed". Co-Authored-By: Claude Fable 5 --- .../data/identity_missing_labels_data.json | 10 ++ src/tools/src/bin/linkml_validate.rs | 95 +++++++++++++++++-- 2 files changed, 98 insertions(+), 7 deletions(-) create mode 100644 src/runtime/tests/data/identity_missing_labels_data.json diff --git a/src/runtime/tests/data/identity_missing_labels_data.json b/src/runtime/tests/data/identity_missing_labels_data.json new file mode 100644 index 0000000..1a74c6f --- /dev/null +++ b/src/runtime/tests/data/identity_missing_labels_data.json @@ -0,0 +1,10 @@ +{ + "name": "reg", + "entries": [{"note": "a"}, {"note": "b"}], + "records": [{"serial": "S1"}, {"note": "b"}], + "unfillable": [{"note": "a"}, {"note": "b"}], + "vertices": [{"x": 1.0, "y": 2.0}, {"x": 1.0, "y": 2.0}], + "tags": ["a", "a"], + "references": ["urn:a", "urn:b"], + "archivedEntries": [{"code": "dup"}, {"code": "dup"}] +} diff --git a/src/tools/src/bin/linkml_validate.rs b/src/tools/src/bin/linkml_validate.rs index 5b9ae4e..5d125ba 100644 --- a/src/tools/src/bin/linkml_validate.rs +++ b/src/tools/src/bin/linkml_validate.rs @@ -1,6 +1,7 @@ use clap::Parser; use linkml_runtime::{ - load_json_file, load_yaml_file, ValidationResult, ValidationSeverity, ValidationValue, + lint_instance_identity, load_json_file, load_yaml_file, ValidationResult, ValidationSeverity, + ValidationValue, }; use linkml_schemaview::identifier::Identifier; use linkml_schemaview::io::from_yaml; @@ -21,6 +22,11 @@ struct Args { /// Emit machine-readable JSON instead of human-readable text #[arg(long)] json: bool, + /// Opt-in: warn where this data defeats a declared element identity — a + /// list repeating one, or one addressed positionally because some element + /// leaves the identity slot empty. Warnings never change the exit code. + #[arg(long, default_value_t = false)] + lint_identity: bool, } fn main() -> Result<(), Box> { @@ -50,10 +56,28 @@ fn main() -> Result<(), Box> { } else { load_yaml_file(data_path, &sv, &class_view, &conv)? }; + let instance = load_result.instance; let validation_issues = load_result.validation_issues; let is_valid = validation_issues.is_empty(); + // Opt-in instance-identity lint, deliberately skipped when the data does + // not validate — the same stance `linkml-schema-validate` takes towards a + // schema with errors. The lint asks how a list's elements are addressed, + // and answering that over a tree whose loading already went wrong produces + // answers about the damage rather than about the data. Reporting the + // validation issues first is also the only useful output in that case. + // + // Warnings never change the exit code: it stays whatever validation made it. + let identity_warnings = match (args.lint_identity, is_valid, &instance) { + (true, true, Some(value)) => lint_instance_identity(value), + _ => Vec::new(), + }; if args.json { - emit_json(is_valid, &validation_issues)?; + emit_json( + is_valid, + &validation_issues, + args.lint_identity, + &identity_warnings, + )?; if is_valid { Ok(()) } else { @@ -61,6 +85,9 @@ fn main() -> Result<(), Box> { } } else if is_valid { println!("valid"); + for w in &identity_warnings { + println!("warning[{}]: {}", w.subject.join("."), w.detail); + } Ok(()) } else { for issue in &validation_issues { @@ -71,11 +98,44 @@ fn main() -> Result<(), Box> { }; println!("{:?} at {}: {}", issue.problem_type, location, issue.detail); } + if args.lint_identity { + println!("note: --lint-identity skipped: fix the validation errors above first"); + } std::process::exit(1); } } -fn emit_json(valid: bool, issues: &[ValidationResult]) -> Result<(), serde_json::Error> { +/// The identity warnings, in the shape `linkml-schema-validate` already emits +/// them, so the two CLIs report one lint the same way. +/// +/// Uses `ValidationProblemType::label()` rather than this binary's older +/// `Debug` spelling for validation issues: the label is the shared +/// machine-readable name (the Python binding reports the same one), and a +/// variant rename cannot change it behind the CLI's back. The existing +/// `issues` shape is left exactly as it was — its `Debug` spelling is a +/// published contract of this binary. +fn identity_warnings_json(warnings: &[ValidationResult]) -> serde_json::Value { + serde_json::Value::Array( + warnings + .iter() + .map(|w| { + json!({ + "type": w.problem_type.label(), + "severity": severity_label(&w.severity), + "subject": w.subject, + "detail": w.detail, + }) + }) + .collect(), + ) +} + +fn emit_json( + valid: bool, + issues: &[ValidationResult], + lint_identity: bool, + identity_warnings: &[ValidationResult], +) -> Result<(), serde_json::Error> { let issues_json: Vec<_> = issues .iter() .map(|issue| { @@ -96,10 +156,31 @@ fn emit_json(valid: bool, issues: &[ValidationResult]) -> Result<(), serde_json: }) }) .collect(); - let output = json!({ - "valid": valid, - "issues": issues_json, - }); + // Without the flag the document is byte-identical to what it always was: + // the lint keys are absent, not null. A consumer that never opts in cannot + // tell this version from the previous one. + let output = if !lint_identity { + json!({ + "valid": valid, + "issues": issues_json, + }) + } else if valid { + json!({ + "valid": valid, + "issues": issues_json, + "identity_warnings": identity_warnings_json(identity_warnings), + "identity_lint_skipped": false, + "identity_lint_skipped_reason": serde_json::Value::Null, + }) + } else { + json!({ + "valid": valid, + "issues": issues_json, + "identity_warnings": serde_json::Value::Null, + "identity_lint_skipped": true, + "identity_lint_skipped_reason": "data has validation issues; fix them and re-run", + }) + }; println!("{}", serde_json::to_string_pretty(&output)?); Ok(()) } From 6162bbcc7534864569954523a5d64371eed0b616 Mon Sep 17 00:00:00 2001 From: ejsyx Date: Wed, 19 Aug 2026 15:06:54 +0200 Subject: [PATCH 50/58] docs: hardening close-out cross-check Two sentences in the pre-addendum spec text no longer read true after the 2026-08-19 addendum landed: - The Non-goal section defined the identity label as "key/identifier first, else the class's unique_keys" and said path segments are the labels. Addendum rule 1 excludes a key that is the element class's type designator, and rule 2 IRI-expands label components whose slot ranges on uri/uriorcurie. Both are now named inline. - Option 1 claimed a truthful key always means "nothing needs to change". That is false for the one slot rule 1 excludes, in list form. The dict form the option actually exhibits (SpotLocation_coordinates) is unaffected, and the added paragraph says which is which. Docs only; no code, test or fixture touched. Co-Authored-By: Claude Fable 5 --- .../2026-08-17-inlined-multivalued-element-identity-design.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/superpowers/specs/2026-08-17-inlined-multivalued-element-identity-design.md b/docs/superpowers/specs/2026-08-17-inlined-multivalued-element-identity-design.md index c8bafb8..49e8172 100644 --- a/docs/superpowers/specs/2026-08-17-inlined-multivalued-element-identity-design.md +++ b/docs/superpowers/specs/2026-08-17-inlined-multivalued-element-identity-design.md @@ -21,7 +21,7 @@ The offered options are: Positional index deltas remain the inferred semantics for slots that declare nothing — they are still valuable for projects not dealing with multiple sources producing deltas for the same object at the same time, and the opt-in linter is the only place that complains about them. -One deliberate compatibility break ships with this design (a spike finding): **keyed matching becomes uniform**. A list is matched by element identity only when every element on both sides yields an identity label (key/identifier first, else the class's `unique_keys`) and the labels are unique within each side; path segments are then the labels. In every other case matching is positional and path segments are plain numeric indices. This removes two behaviours of the old fallback: key values were opportunistically mixed into positional paths, and duplicate key values within a "keyed" list were silently collapsed by the matcher. Consumers see cleaner, uniform delta paths; `patch` keeps accepting both numeric and label segments, and reports a delta as failed instead of guessing when a label matches more than one element. +One deliberate compatibility break ships with this design (a spike finding): **keyed matching becomes uniform**. A list is matched by element identity only when every element on both sides yields an identity label (key/identifier first — unless that key is the element class's type designator, which the addendum's rule 1 never accepts as element identity — else the class's `unique_keys`) and the labels are unique within each side; path segments are then the labels, IRI-expanded when the slot they came from ranges on `uri`/`uriorcurie` (addendum rule 2). In every other case matching is positional and path segments are plain numeric indices. This removes two behaviours of the old fallback: key values were opportunistically mixed into positional paths, and duplicate key values within a "keyed" list were silently collapsed by the matcher. Consumers see cleaner, uniform delta paths; `patch` keeps accepting both numeric and label segments, and reports a delta as failed instead of guessing when a label matches more than one element. ## Proposed solution @@ -197,6 +197,8 @@ The linter's question is always the same — *where does element identity come f When the element class declares a `key` or `identifier` slot that is truthful for all of its data, nothing needs to change — the diff already matches elements by it. `SpotLocation_coordinates` is the example: a dict keyed by `typeURI`, at most one coordinate per positioning system, exactly what the data means. +One slot cannot answer this way: the class's own type designator. Its value is a function of the element's class, not of the element, so the addendum's rule 1 makes the engine look past it in **list** form and fall through to `unique_keys` (else position). The dict form above is unaffected — a mapping keyed by the designator is exactly the at-most-one-per-subtype statement it looks like — and the linter says so on the slot that needs the other answer. + ### Option 2 — a composed key (content): declare `unique_keys` — the phone number solution `hasNumberFunction` is already the de-facto identity — the SHACL shape says so. Declare exactly that rule with the existing LinkML meta `unique_keys`: From 8b2ea94920cc35d415a424bfe182c2d212d1ecb7 Mon Sep 17 00:00:00 2001 From: ejsyx Date: Wed, 19 Aug 2026 15:27:29 +0200 Subject: [PATCH 51/58] refactor(tools): hoist the CLIs' shared rendering helpers MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `severity_label` and `identity_warnings_json` were duplicated verbatim in `linkml_validate` and `linkml_schema_validate`, and `format_path` existed privately in the tools lib beside a copy of `severity_label`. All three are rendering decisions both CLIs must make identically — two binaries printing one lint two ways is a difference a reader has to explain to themselves — so they belong in the one place the binaries already share. No output changes: the moved bodies are the ones that were there. Co-Authored-By: Claude Fable 5 --- src/tools/src/bin/linkml_schema_validate.rs | 30 +--------- src/tools/src/lib.rs | 65 +++++++++++++++------ 2 files changed, 49 insertions(+), 46 deletions(-) diff --git a/src/tools/src/bin/linkml_schema_validate.rs b/src/tools/src/bin/linkml_schema_validate.rs index b24cecf..558adae 100644 --- a/src/tools/src/bin/linkml_schema_validate.rs +++ b/src/tools/src/bin/linkml_schema_validate.rs @@ -1,8 +1,8 @@ use clap::{Parser, ValueEnum}; -use linkml_runtime::{ValidationResult, ValidationSeverity}; #[cfg(feature = "resolve")] use linkml_schemaview::resolve::resolve_schemas_from; use linkml_schemaview::{identifier::Identifier, io::from_yaml, schemaview::SchemaView, Converter}; +use linkml_tools::validation_utils::identity_warnings_json; use std::path::PathBuf; #[derive(Parser)] @@ -108,34 +108,6 @@ fn enum_exists( } } -fn severity_label(severity: &ValidationSeverity) -> &'static str { - match severity { - ValidationSeverity::Fatal => "fatal", - ValidationSeverity::Error => "error", - ValidationSeverity::Warning => "warning", - ValidationSeverity::Info => "info", - } -} - -fn identity_warnings_json(warnings: &[ValidationResult]) -> serde_json::Value { - serde_json::Value::Array( - warnings - .iter() - .map(|w| { - serde_json::json!({ - // The shared machine label, never `Debug` formatting: it - // is the same spelling the Python binding reports, and a - // variant rename cannot change it behind the CLI's back. - "type": w.problem_type.label(), - "severity": severity_label(&w.severity), - "subject": w.subject, - "detail": w.detail, - }) - }) - .collect(), - ) -} - fn main() -> Result<(), Box> { let args = Args::parse(); let schema = from_yaml(&args.schema)?; diff --git a/src/tools/src/lib.rs b/src/tools/src/lib.rs index 3784200..a1cd79e 100644 --- a/src/tools/src/lib.rs +++ b/src/tools/src/lib.rs @@ -1,9 +1,12 @@ -use linkml_runtime::{InstancePath, ValidationSeverity}; - /// Helpers shared by CLI binaries. +/// +/// Everything here is a *rendering* decision that more than one binary makes, +/// and that the binaries must make identically: two CLIs printing one lint two +/// ways is a difference a reader has to explain to themselves. The rule for +/// what belongs here is that — shared vocabulary — not "utility". pub mod validation_utils { - use super::{format_path, severity_label}; - use linkml_runtime::ValidationResult; + use linkml_runtime::{InstancePath, ValidationResult, ValidationSeverity}; + use serde_json::json; use std::path::Path; /// Print validation diagnostics to stderr but keep execution going. @@ -26,21 +29,49 @@ pub mod validation_utils { ); } } -} -fn severity_label(severity: &ValidationSeverity) -> &'static str { - match severity { - ValidationSeverity::Fatal => "fatal", - ValidationSeverity::Error => "error", - ValidationSeverity::Warning => "warning", - ValidationSeverity::Info => "info", + /// The machine-readable name of a severity, as every CLI spells it. + pub fn severity_label(severity: &ValidationSeverity) -> &'static str { + match severity { + ValidationSeverity::Fatal => "fatal", + ValidationSeverity::Error => "error", + ValidationSeverity::Warning => "warning", + ValidationSeverity::Info => "info", + } + } + + /// An instance path as the CLIs display it; the empty path is the root. + pub fn format_path(path: &InstancePath) -> String { + if path.is_empty() { + "".to_string() + } else { + path.join(".") + } } -} -fn format_path(path: &InstancePath) -> String { - if path.is_empty() { - "".to_string() - } else { - path.join(".") + /// The identity-lint warnings, in the one shape both `linkml-validate` and + /// `linkml-schema-validate` emit them: the two CLIs report one lint the + /// same way, so a consumer reading either document reads the same keys. + /// + /// Uses [`linkml_runtime::ValidationProblemType::label`] rather than the + /// binaries' older `Debug` spelling for validation issues: the label is the + /// shared machine-readable name (the Python binding reports the same one), + /// and a variant rename cannot change it behind the CLIs' back. The + /// separate `issues` shape of `linkml-validate` keeps its `Debug` spelling + /// — that one is a published contract of that binary. + pub fn identity_warnings_json(warnings: &[ValidationResult]) -> serde_json::Value { + serde_json::Value::Array( + warnings + .iter() + .map(|w| { + json!({ + "type": w.problem_type.label(), + "severity": severity_label(&w.severity), + "subject": w.subject, + "detail": w.detail, + }) + }) + .collect(), + ) } } From 930a1cbadbb668c1fdeba3552adedb734b21dc1f Mon Sep 17 00:00:00 2001 From: ejsyx Date: Wed, 19 Aug 2026 15:27:40 +0200 Subject: [PATCH 52/58] fix(tools): a warning is not an error in linkml-validate MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `is_valid = validation_issues.is_empty()` predates the spec addendum. Until the addendum every diagnostic the loader could produce was an error, so "no diagnostics" and "no errors" were one predicate; rules 2 and 5 added the first three warning-severity issues (designator canonicalisation, dict-key divergence, unaccepted designator key) and the two parted company. A document whose only finding was a warning was therefore reported as invalid: exit 1, `valid: false`, the warning printed in the error list, and `--lint-identity` suppressed with "fix the validation errors above first" — an error message about a document that has none. Validity is now decided by errors alone (`LoadResult::has_errors`), which also un-gates the lint: a warning says the document is unusual, not that it failed to load, and the lint's answers over it are exactly as sound as over a silent one. Non-error issues are printed in the valid branch too, marked with their severity so they cannot be read as errors; the error branch keeps its published unmarked shape, so no document that already failed moves a line. In JSON, `valid` reflects errors and the issues array is untouched (it always carried `severity`); the skip reason now names errors, since errors are what gates it. Byte-identical for every warning-free document, which is every document that predates the addendum — pinned by a test, and by the differential harness. Fixtures live under src/tools/tests/data rather than the runtime fixture directory: the differential harness treats the latter as corpus. Co-Authored-By: Claude Fable 5 --- src/tools/src/bin/linkml_validate.rs | 86 +++++----- src/tools/tests/data/validate_clean.json | 9 + src/tools/tests/data/validate_errors.json | 9 + .../tests/data/validate_warning_only.json | 9 + .../tests/data/validate_warning_only.yaml | 53 ++++++ src/tools/tests/validate_cli.rs | 162 ++++++++++++++++++ 6 files changed, 283 insertions(+), 45 deletions(-) create mode 100644 src/tools/tests/data/validate_clean.json create mode 100644 src/tools/tests/data/validate_errors.json create mode 100644 src/tools/tests/data/validate_warning_only.json create mode 100644 src/tools/tests/data/validate_warning_only.yaml create mode 100644 src/tools/tests/validate_cli.rs diff --git a/src/tools/src/bin/linkml_validate.rs b/src/tools/src/bin/linkml_validate.rs index 5d125ba..ae25b9a 100644 --- a/src/tools/src/bin/linkml_validate.rs +++ b/src/tools/src/bin/linkml_validate.rs @@ -1,13 +1,13 @@ use clap::Parser; use linkml_runtime::{ - lint_instance_identity, load_json_file, load_yaml_file, ValidationResult, ValidationSeverity, - ValidationValue, + lint_instance_identity, load_json_file, load_yaml_file, ValidationResult, ValidationValue, }; use linkml_schemaview::identifier::Identifier; use linkml_schemaview::io::from_yaml; #[cfg(feature = "resolve")] use linkml_schemaview::resolve::resolve_schemas_from; use linkml_schemaview::schemaview::SchemaView; +use linkml_tools::validation_utils::{format_path, identity_warnings_json, severity_label}; use serde_json::json; use std::path::PathBuf; @@ -56,15 +56,25 @@ fn main() -> Result<(), Box> { } else { load_yaml_file(data_path, &sv, &class_view, &conv)? }; + // Validity is decided by ERRORS, not by diagnostics. Until the spec + // addendum the loader could only produce errors, so `issues.is_empty()` + // and "no errors" were one predicate; rules 2 and 5 made the loader emit + // warnings, and the two parted company. A document whose only finding is a + // warning is valid: it exits 0, and the warning is reported rather than + // being dressed up as the reason for a failure. + let is_valid = !load_result.has_errors(); let instance = load_result.instance; let validation_issues = load_result.validation_issues; - let is_valid = validation_issues.is_empty(); // Opt-in instance-identity lint, deliberately skipped when the data does // not validate — the same stance `linkml-schema-validate` takes towards a // schema with errors. The lint asks how a list's elements are addressed, // and answering that over a tree whose loading already went wrong produces // answers about the damage rather than about the data. Reporting the - // validation issues first is also the only useful output in that case. + // validation errors first is also the only useful output in that case. + // + // The gate is errors alone: a warning says the document is unusual, not + // that it failed to load, and the lint's answers over it are exactly as + // sound as over a silent one. // // Warnings never change the exit code: it stays whatever validation made it. let identity_warnings = match (args.lint_identity, is_valid, &instance) { @@ -85,18 +95,38 @@ fn main() -> Result<(), Box> { } } else if is_valid { println!("valid"); + // Non-error diagnostics, which a valid document may now carry. Marked + // with their severity, in front of the shape the error list below uses: + // the same line without the marker is what this binary prints for an + // error, and a warning must not be mistakable for one. + for issue in &validation_issues { + println!( + "{}: {:?} at {}: {}", + severity_label(&issue.severity), + issue.problem_type, + format_path(&issue.subject), + issue.detail + ); + } for w in &identity_warnings { println!("warning[{}]: {}", w.subject.join("."), w.detail); } Ok(()) } else { + // Deliberately unmarked, and deliberately including the non-error + // issues: this list is a published output shape of the binary, and + // re-spelling every line of it would move the output of every document + // that has ever failed to validate, to say something about the few that + // also carry a warning. The marker above is where the distinction is + // needed — there, an unmarked line would read as an error on a document + // that has none. for issue in &validation_issues { - let location = if issue.subject.is_empty() { - "".to_string() - } else { - issue.subject.join(".") - }; - println!("{:?} at {}: {}", issue.problem_type, location, issue.detail); + println!( + "{:?} at {}: {}", + issue.problem_type, + format_path(&issue.subject), + issue.detail + ); } if args.lint_identity { println!("note: --lint-identity skipped: fix the validation errors above first"); @@ -105,31 +135,6 @@ fn main() -> Result<(), Box> { } } -/// The identity warnings, in the shape `linkml-schema-validate` already emits -/// them, so the two CLIs report one lint the same way. -/// -/// Uses `ValidationProblemType::label()` rather than this binary's older -/// `Debug` spelling for validation issues: the label is the shared -/// machine-readable name (the Python binding reports the same one), and a -/// variant rename cannot change it behind the CLI's back. The existing -/// `issues` shape is left exactly as it was — its `Debug` spelling is a -/// published contract of this binary. -fn identity_warnings_json(warnings: &[ValidationResult]) -> serde_json::Value { - serde_json::Value::Array( - warnings - .iter() - .map(|w| { - json!({ - "type": w.problem_type.label(), - "severity": severity_label(&w.severity), - "subject": w.subject, - "detail": w.detail, - }) - }) - .collect(), - ) -} - fn emit_json( valid: bool, issues: &[ValidationResult], @@ -178,18 +183,9 @@ fn emit_json( "issues": issues_json, "identity_warnings": serde_json::Value::Null, "identity_lint_skipped": true, - "identity_lint_skipped_reason": "data has validation issues; fix them and re-run", + "identity_lint_skipped_reason": "data has validation errors; fix them and re-run", }) }; println!("{}", serde_json::to_string_pretty(&output)?); Ok(()) } - -fn severity_label(severity: &ValidationSeverity) -> &'static str { - match severity { - ValidationSeverity::Fatal => "fatal", - ValidationSeverity::Error => "error", - ValidationSeverity::Warning => "warning", - ValidationSeverity::Info => "info", - } -} diff --git a/src/tools/tests/data/validate_clean.json b/src/tools/tests/data/validate_clean.json new file mode 100644 index 0000000..8b3ce71 --- /dev/null +++ b/src/tools/tests/data/validate_clean.json @@ -0,0 +1,9 @@ +{ + "people": { + "p1": {"pid": "p1", "name": "Ann"} + }, + "readings": [ + {"code": "c1", "value": "1"}, + {"code": "c2", "value": "2"} + ] +} diff --git a/src/tools/tests/data/validate_errors.json b/src/tools/tests/data/validate_errors.json new file mode 100644 index 0000000..b47bb8c --- /dev/null +++ b/src/tools/tests/data/validate_errors.json @@ -0,0 +1,9 @@ +{ + "people": { + "p1": {"pid": "p1", "name": "Ann", "not_a_slot": "x"} + }, + "readings": [ + {"code": "c1", "value": "1"}, + {"code": "c1", "value": "2"} + ] +} diff --git a/src/tools/tests/data/validate_warning_only.json b/src/tools/tests/data/validate_warning_only.json new file mode 100644 index 0000000..bbc57de --- /dev/null +++ b/src/tools/tests/data/validate_warning_only.json @@ -0,0 +1,9 @@ +{ + "people": { + "p1": {"pid": "p2", "name": "Ann"} + }, + "readings": [ + {"code": "c1", "value": "1"}, + {"code": "c1", "value": "2"} + ] +} diff --git a/src/tools/tests/data/validate_warning_only.yaml b/src/tools/tests/data/validate_warning_only.yaml new file mode 100644 index 0000000..63495ac --- /dev/null +++ b/src/tools/tests/data/validate_warning_only.yaml @@ -0,0 +1,53 @@ +id: https://w3id.org/linkml/examples/validate_warning_only +name: validate_warning_only +description: |- + Fixture for the `linkml-validate` warning/error contract (spec addendum + rules 5 and 6, and the exit-code fix that followed them). + + Deliberately lives under `src/tools/tests/data` rather than + `src/runtime/tests/data`: the differential harness treats every file of the + runtime fixture directory as corpus, so a fixture added there to pin a CLI + contract would move harness output that has nothing to do with the change. + + Two shapes, so one document can carry a load-time warning *and* a finding + from the opt-in instance lint at the same time: + + * `Container.people` — an ordinary keyed class in inlined-dict form. A + payload key that contradicts the dict key is a load-time **warning** + (rule 5); the document is still valid. + * `Container.readings` — a list whose element class declares its identity + through `unique_keys`. Repeating a value there is what + `--lint-identity` reports. +license: https://creativecommons.org/publicdomain/zero/1.0/ +imports: + - linkml:types +prefixes: + vwo: https://w3id.org/linkml/examples/validate_warning_only/ + linkml: https://w3id.org/linkml/ +default_prefix: vwo +default_range: string + +classes: + Container: + attributes: + people: + range: Person + multivalued: true + inlined: true + readings: + range: Reading + multivalued: true + inlined: true + + Person: + attributes: + pid: {range: string, key: true, required: true} + name: {range: string} + + Reading: + unique_keys: + main: + unique_key_slots: [code] + attributes: + code: {range: string} + value: {range: string} diff --git a/src/tools/tests/validate_cli.rs b/src/tools/tests/validate_cli.rs new file mode 100644 index 0000000..64c015a --- /dev/null +++ b/src/tools/tests/validate_cli.rs @@ -0,0 +1,162 @@ +//! `linkml-validate`'s validity contract: **errors decide validity, warnings +//! never do**. +//! +//! The loader gained warning-severity diagnostics with the spec addendum +//! (designator canonicalisation, rule 5's dict-key reconciliation) — before it, +//! every diagnostic it could produce was an error, so "no diagnostics" and "no +//! errors" were the same predicate and the CLI could use either. They are not +//! the same predicate any more: a document whose only finding is a warning was +//! reported as invalid, exited 1, printed the warning in the error list, and +//! had `--lint-identity` suppressed with "fix the validation errors above +//! first" — an error message about a document that has none. +//! +//! Pinned here, because none of it is visible from a library test: +//! +//! * a warning-only document is **valid**: exit 0, `valid: true`, and the +//! warning is still reported (text mode marks its severity, JSON mode carries +//! it in `issues` with `severity: "warning"` as it always did); +//! * `--lint-identity` **runs** on such a document — the lint is gated on +//! errors, not on diagnostics; +//! * an **error** still skips the lint, with the reason stated in both modes; +//! * a warning-free document invoked without the flag produces byte-for-byte +//! the output it always did. + +use assert_cmd::Command; + +const CLASS: &str = "Container"; + +fn data_path(name: &str) -> std::path::PathBuf { + let mut p = std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR")); + p.push("tests/data"); + p.push(name); + p +} + +/// Run `linkml-validate` against the fixture schema, returning (exit code, stdout). +fn validate(data: &str, extra: &[&str]) -> (i32, String) { + let mut cmd = Command::cargo_bin("linkml-validate").unwrap(); + cmd.arg(data_path("validate_warning_only.yaml")) + .arg(CLASS) + .arg(data_path(data)) + .args(extra); + let out = cmd.output().unwrap(); + ( + out.status.code().unwrap_or(-1), + String::from_utf8(out.stdout).unwrap(), + ) +} + +fn validate_json(data: &str, extra: &[&str]) -> (i32, serde_json::Value) { + let mut args = vec!["--json"]; + args.extend_from_slice(extra); + let (code, out) = validate(data, &args); + (code, serde_json::from_str(&out).expect("JSON on stdout")) +} + +// --------------------------------------------------------------------------- +// A warning is not an error +// --------------------------------------------------------------------------- + +/// Text mode. The document's only load diagnostic is rule 5's dict-key +/// divergence warning, so the document is valid — and the warning is still +/// printed, marked with its severity so it cannot be read as an error. +#[test] +fn warning_only_document_is_valid_in_text_mode() { + let (code, out) = validate("validate_warning_only.json", &[]); + assert_eq!(code, 0, "a warning must not fail the run: {out}"); + assert!(out.starts_with("valid\n"), "{out}"); + assert!( + out.contains("warning: SlotRangeViolation at people.p1.pid:"), + "the warning is reported, with its severity: {out}" + ); + assert!( + !out.contains("fix the validation errors above first"), + "there are no errors to fix: {out}" + ); +} + +/// JSON mode. `valid` reflects errors only; the warning stays in `issues`, +/// where it always carried `severity: "warning"`. +#[test] +fn warning_only_document_is_valid_in_json_mode() { + let (code, doc) = validate_json("validate_warning_only.json", &[]); + assert_eq!(code, 0); + assert_eq!(doc["valid"], serde_json::json!(true), "{doc:#}"); + let issues = doc["issues"].as_array().expect("issues array"); + assert_eq!(issues.len(), 1, "{doc:#}"); + assert_eq!(issues[0]["severity"], serde_json::json!("warning")); +} + +/// The lint is gated on *errors*. This document has a warning and a genuine +/// duplicate identity, and the flag must report the duplicate. +#[test] +fn lint_runs_on_a_warning_only_document() { + let (code, out) = validate("validate_warning_only.json", &["--lint-identity"]); + assert_eq!(code, 0, "{out}"); + assert!( + out.contains("warning[readings]:") && out.contains("share the declared identity 'c1'"), + "the instance lint ran and reported: {out}" + ); + + let (code, doc) = validate_json("validate_warning_only.json", &["--lint-identity"]); + assert_eq!(code, 0); + assert_eq!(doc["identity_lint_skipped"], serde_json::json!(false)); + assert_eq!(doc["identity_lint_skipped_reason"], serde_json::Value::Null); + let warnings = doc["identity_warnings"].as_array().expect("array"); + assert_eq!(warnings.len(), 1, "{doc:#}"); + assert_eq!(warnings[0]["subject"], serde_json::json!(["readings"])); +} + +// --------------------------------------------------------------------------- +// An error still is one +// --------------------------------------------------------------------------- + +/// Same duplicate identity, plus an error. The lint is skipped and says so — +/// its answers over a tree whose loading went wrong would be about the damage. +#[test] +fn errors_skip_the_lint_and_say_so() { + let (code, out) = validate("validate_errors.json", &["--lint-identity"]); + assert_eq!(code, 1, "{out}"); + assert!( + out.contains("note: --lint-identity skipped: fix the validation errors above first"), + "{out}" + ); + assert!( + !out.contains("share the declared identity"), + "no lint output when the lint was skipped: {out}" + ); + + let (code, doc) = validate_json("validate_errors.json", &["--lint-identity"]); + assert_eq!(code, 1); + assert_eq!(doc["valid"], serde_json::json!(false), "{doc:#}"); + assert_eq!(doc["identity_lint_skipped"], serde_json::json!(true)); + assert!( + doc["identity_lint_skipped_reason"] + .as_str() + .unwrap_or_default() + .contains("error"), + "the reason names errors, since errors are what gates the lint: {doc:#}" + ); + assert_eq!(doc["identity_warnings"], serde_json::Value::Null); +} + +// --------------------------------------------------------------------------- +// Nothing moved for a document with no diagnostics +// --------------------------------------------------------------------------- + +/// Byte-identity: a warning-free document, no flag. Every document that +/// predates the addendum's warnings is this case, which is what makes the +/// change above a fix rather than an output break. +#[test] +fn warning_free_document_without_the_flag_is_unchanged() { + let (code, out) = validate("validate_clean.json", &[]); + assert_eq!(code, 0); + assert_eq!(out, "valid\n"); + + let (code, out) = validate("validate_clean.json", &["--json"]); + assert_eq!(code, 0); + assert_eq!( + out, "{\n \"issues\": [],\n \"valid\": true\n}\n", + "the no-flag JSON document keeps exactly its two keys" + ); +} From f2aefc4636bdb9c4681346ab8d8dd27e3c0e420e Mon Sep 17 00:00:00 2001 From: ejsyx Date: Wed, 19 Aug 2026 15:30:06 +0200 Subject: [PATCH 53/58] fix(runtime): diff's changed-key check compares canonical identity MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `treat_changed_identifier_as_new_object` compared the two sides' key scalars as raw JSON values — rule 2's last unconverted resolve site. A `uri`-ranged non-designator key respelled curie↔uri yields ONE identity label, so the list matched the two elements as one element; this branch then compared the same values as strings, found them different, and emitted a whole-element `Update` at a path that addresses the element by the identity it had just declared changed. The delta contradicted itself, and a re-spelling that should have diffed as one field became a replacement of the whole element. Both sides now go through `scalar_slot_string`, the function every identity label is built from. The branch keeps reading the metamodel's key slot rather than `identity_key_slot` — the two questions differ in *which* slot to read, never in what makes two values of it the same value. A key that is a genuinely different IRI is still a replacement. `unique_keys`-derived identity never reached this branch, which is why the case survived rule 2's first pass; the fixture gains `KeyedSystem` to express the same component as a `key`. Co-Authored-By: Claude Fable 5 --- src/runtime/src/diff.rs | 20 ++++-- .../tests/data/identity_canonical.yaml | 14 ++++ .../tests/identity_canonicalization.rs | 67 +++++++++++++++++++ 3 files changed, 95 insertions(+), 6 deletions(-) diff --git a/src/runtime/src/diff.rs b/src/runtime/src/diff.rs index d8dabbe..5f2beb9 100644 --- a/src/runtime/src/diff.rs +++ b/src/runtime/src/diff.rs @@ -470,17 +470,25 @@ pub fn diff(source: &LinkMLInstance, target: &LinkMLInstance, opts: DiffOptions) // Update has already been emitted. What is left for this // branch — and why it stays — is a changed key value within // ONE class: same class, different element. + // + // The comparison is the canonical one (rule 2), through + // [`scalar_slot_string`] — the very function identity + // labels are built from. Raw strings were the rule's last + // unconverted site: a `uri`-ranged key respelled + // curie↔uri makes ONE identity label, so the list above + // matched the two elements as one element, and this branch + // then called that element replaced — a whole-element + // `Update` at a path that addresses it by the identity the + // branch had just declared changed. What the two questions + // disagree about is *which slot* to read (see above), never + // what makes two values of it the same value. let key_slot_name = sc .key_or_identifier_slot() .or_else(|| tc.key_or_identifier_slot()) .map(|s| s.name.clone()); if let Some(ks) = key_slot_name { - let sid = sm.get(&ks); - let tid = tm.get(&ks); - if let ( - Some(LinkMLInstance::Scalar { value: s_id, .. }), - Some(LinkMLInstance::Scalar { value: t_id, .. }), - ) = (sid, tid) + if let (Some(s_id), Some(t_id)) = + (scalar_slot_string(sm, &ks), scalar_slot_string(tm, &ks)) { if s_id != t_id { out.push(Delta { diff --git a/src/runtime/tests/data/identity_canonical.yaml b/src/runtime/tests/data/identity_canonical.yaml index de0f29b..139c68c 100644 --- a/src/runtime/tests/data/identity_canonical.yaml +++ b/src/runtime/tests/data/identity_canonical.yaml @@ -44,6 +44,12 @@ classes: range: System multivalued: true inlined_as_list: true + # the same, where that component is a `key` — the one input diff's + # changed-key check reads, which `unique_keys` never reaches + keyed_systems: + range: KeyedSystem + multivalued: true + inlined_as_list: true Shape: unique_keys: @@ -113,3 +119,11 @@ classes: attributes: systemType: {range: uri, required: true} value: {range: string} + + # `System`'s identity comes from `unique_keys`, which diff's changed-key + # check never consults; this one declares the same component as a `key`, so + # the check does read it, and its comparison has to be the canonical one too. + KeyedSystem: + attributes: + systemType: {range: uri, key: true, required: true} + value: {range: string} diff --git a/src/runtime/tests/identity_canonicalization.rs b/src/runtime/tests/identity_canonicalization.rs index 9a89697..b5addf5 100644 --- a/src/runtime/tests/identity_canonicalization.rs +++ b/src/runtime/tests/identity_canonicalization.rs @@ -84,6 +84,11 @@ fn systems(items: Vec) -> JsonValue { json!({ "systems": items }) } +/// The same list shape over `KeyedSystem`, whose identity component is a `key`. +fn keyed_systems(items: Vec) -> JsonValue { + json!({ "keyed_systems": items }) +} + // --------------------------------------------------------------------------- // D1 — designator values canonicalised at load // --------------------------------------------------------------------------- @@ -361,6 +366,68 @@ fn respelled_element_is_matched_never_replaced() { ); } +/// The same re-spelling, on a class whose identity component is a `key`. +/// +/// Rule 2's last raw comparison lived here: the list matched the two elements +/// as one identity (labels are canonicalised), diff then recursed into the +/// pair, and `treat_changed_identifier_as_new_object` — which reads the +/// metamodel's key slot, not the identity label — compared `ex:WGS84` with its +/// own expansion as two strings and called the element replaced. The delta was +/// self-contradictory: a whole-element `Update` at a path that addresses the +/// element by the very identity the branch had just declared changed. +/// +/// `unique_keys`-derived identity (the test above) never reached this branch at +/// all, which is why the case survived rule 2's first pass. +#[test] +fn respelled_key_is_matched_never_replaced() { + let f = fixture(); + let deltas = diff2( + &f, + keyed_systems(vec![json!({"systemType": WGS84_CURIE, "value": "one"})]), + keyed_systems(vec![json!({"systemType": WGS84_URI, "value": "two"})]), + ); + let mut paths: Vec> = deltas.iter().map(|d| d.path.clone()).collect(); + paths.sort(); + assert_eq!( + paths, + vec![ + vec![ + "keyed_systems".to_string(), + WGS84_URI.to_string(), + "systemType".to_string() + ], + vec![ + "keyed_systems".to_string(), + WGS84_URI.to_string(), + "value".to_string() + ], + ], + "field-level deltas under one element, never a replacement: {deltas:#?}" + ); +} + +/// The complement, and the reason the branch stays: a key value that is a +/// *different* IRI is a different element, and the whole-element replacement is +/// exactly right. Only spelling was ever meant to be forgiven. +#[test] +fn a_genuinely_changed_key_is_still_a_replacement() { + let f = fixture(); + let deltas = diff2( + &f, + keyed_systems(vec![json!({"systemType": WGS84_URI, "value": "one"})]), + keyed_systems(vec![json!({"systemType": ETRS89_URI, "value": "one"})]), + ); + assert!( + deltas + .iter() + .any(|d| d.op == DeltaOp::Remove && d.path == vec!["keyed_systems", WGS84_URI]) + && deltas + .iter() + .any(|d| d.op == DeltaOp::Add && d.path == vec!["keyed_systems", ETRS89_URI]), + "two different IRIs are two different elements: {deltas:#?}" + ); +} + /// The invariant the two halves share: whatever segment diff emits, /// `resolve_list_segment` must find — even when the document patch is applied /// to spells the identity the other way round. From 716334263a92041ba1e22456ecdd6b27428d775e Mon Sep 17 00:00:00 2001 From: ejsyx Date: Wed, 19 Aug 2026 15:31:55 +0200 Subject: [PATCH 54/58] fix(runtime): rule 5's subject says which rule spoke MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The shared-`class_uri` warning's subject was the bare list of sharing classes. Every other rule in the module has a `[class_name, slot_name]` subject, and both CLIs render a subject by joining its segments with `.`, so `[Alpha, Beta]` printed as `warning[Alpha.Beta]` — indistinguishable from a slot warning about `Beta` on class `Alpha`, and just as indistinguishable to anything grouping findings by subject. The subject now leads with a `shared_class_uri` marker. It cannot collide with a class name, since it is not one, and it costs the rest of the rule nothing: the detail, the problem type and the emission gate are unchanged. Co-Authored-By: Claude Fable 5 --- .../python/linkml_runtime_rust/_native.pyi | 4 +++- src/python/src/lib.rs | 4 +++- src/runtime/src/identity_lint.rs | 24 +++++++++++++++++-- src/runtime/tests/identity_lint.rs | 18 +++++++++++--- 4 files changed, 43 insertions(+), 7 deletions(-) diff --git a/src/python/python/linkml_runtime_rust/_native.pyi b/src/python/python/linkml_runtime_rust/_native.pyi index a5f19e8..22f67af 100644 --- a/src/python/python/linkml_runtime_rust/_native.pyi +++ b/src/python/python/linkml_runtime_rust/_native.pyi @@ -5500,7 +5500,9 @@ def lint_element_identity(schema_view:SchemaView) -> builtins.list[ValidationRes by subject, deduplicated across class URIs, and an inherited slot is reported once, at the class that introduces the problem. Rules 1-4 are per-slot and their ``subject`` is ``[class_name, slot_name]``; rule 5 is - class-level and its ``subject`` is the list of classes sharing the URI. + class-level and its ``subject`` is ``"shared_class_uri"`` followed by the + classes sharing the URI — the marker distinguishes it from a per-slot + subject, which a rendering that joins the segments could not otherwise do. """ def lint_instance_identity(instance:LinkMLInstance) -> builtins.list[ValidationResult]: diff --git a/src/python/src/lib.rs b/src/python/src/lib.rs index cb00180..d977cf6 100644 --- a/src/python/src/lib.rs +++ b/src/python/src/lib.rs @@ -1582,7 +1582,9 @@ fn py_patch( /// by subject, deduplicated across class URIs, and an inherited slot is /// reported once, at the class that introduces the problem. Rules 1-4 are /// per-slot and their ``subject`` is ``[class_name, slot_name]``; rule 5 is -/// class-level and its ``subject`` is the list of classes sharing the URI. +/// class-level and its ``subject`` is ``"shared_class_uri"`` followed by the +/// classes sharing the URI — the marker distinguishes it from a per-slot +/// subject, which a rendering that joins the segments could not otherwise do. #[cfg_attr(feature = "stubgen", gen_stub_pyfunction)] #[pyfunction(name = "lint_element_identity")] fn py_lint_element_identity( diff --git a/src/runtime/src/identity_lint.rs b/src/runtime/src/identity_lint.rs index 824a5d2..f18d934 100644 --- a/src/runtime/src/identity_lint.rs +++ b/src/runtime/src/identity_lint.rs @@ -81,7 +81,10 @@ //! range class matches all three raw shapes while rule 2 is the slot's only //! voice: a parent that emitted no warning must not suppress a subclass's. //! Rule 5 is class-level and has no slot to attribute, so it is emitted once per -//! (hierarchy, shared URI) instead. +//! (hierarchy, shared URI) instead, and its subject is the sharing classes +//! behind a `shared_class_uri` marker — every other rule's subject is +//! `[class, slot]`, and a consumer joining the segments could not otherwise +//! tell the two apart. //! //! ## Instance rules //! @@ -548,6 +551,18 @@ fn hierarchy_root(class: &ClassView) -> ClassView { current } +/// First subject segment of every rule-5 warning, marking what the segments +/// after it are. +/// +/// Every other rule in this module has a subject of the form +/// `[class_name, slot_name]`, and both CLIs render a subject by joining it with +/// `.`. Rule 5's subject is a *list of classes*, so two of them printed as +/// `Alpha.Beta` — a slot warning about `Beta` on class `Alpha`, as far as any +/// reader or any consumer grouping findings by subject could tell. The marker +/// makes the subject say which rule produced it; it cannot collide with a class +/// name, since it is not one. +const SHARED_CLASS_URI_SUBJECT: &str = "shared_class_uri"; + /// The warning text for two classes of one hierarchy sharing a `class_uri`. fn shared_class_uri_detail(root: &str, uri: &str, classes: &[String], designator: &str) -> String { let quoted: Vec = classes.iter().map(|c| format!("'{c}'")).collect(); @@ -612,6 +627,9 @@ fn declared_identity_description(rc: &ClassView) -> Option { /// declared and the schema-derived default otherwise. Two defaults can never /// collide (they are derived from the class name), so a collision always means /// at least one explicit declaration. +/// +/// The subject is [`SHARED_CLASS_URI_SUBJECT`] followed by the sharing classes: +/// see that constant for why the marker is there. fn lint_shared_class_uris(classes: &[ClassView], sink: &mut ValidationResultSink) { let mut hierarchies: BTreeMap<(String, String), Vec<&ClassView>> = BTreeMap::new(); for class in classes { @@ -644,9 +662,11 @@ fn lint_shared_class_uris(classes: &[ClassView], sink: &mut ValidationResultSink if names.len() < 2 { continue; } + let mut subject = vec![SHARED_CLASS_URI_SUBJECT.to_string()]; + subject.extend(names.iter().cloned()); sink.push_warning( ValidationProblemType::AmbiguousElementIdentity, - names.clone(), + subject, shared_class_uri_detail(&root_name, &uri, &names, &designator), ); } diff --git a/src/runtime/tests/identity_lint.rs b/src/runtime/tests/identity_lint.rs index 028d010..7d11ffa 100644 --- a/src/runtime/tests/identity_lint.rs +++ b/src/runtime/tests/identity_lint.rs @@ -797,11 +797,23 @@ fn schema_lint_flags_a_shared_class_uri_within_a_designator_hierarchy() { let sv = schema_view("identity_shared_uri_designator.yaml"); let warnings = lint_element_identity(&sv); let subjects: Vec> = warnings.iter().map(|w| w.subject.clone()).collect(); + // The classes that share the URI, behind a discriminator. Rules 1-4 are + // per-slot and subject `[class, slot]`; both CLIs render a subject by + // joining it with `.`, so a bare `[Alpha, Beta]` printed as `Alpha.Beta` — + // indistinguishable from a slot warning about `Beta` on class `Alpha`, and + // just as indistinguishable to anything grouping findings by subject. The + // leading segment says which rule spoke, and no `[class, slot]` subject can + // collide with it: it is not a class name in any schema being linted, since + // it is not a class name at all. assert_eq!( subjects, - vec![vec!["Alpha".to_string(), "Beta".to_string()]], - "the sharing classes are the subject, and the controls stay silent: \ - {warnings:#?}" + vec![vec![ + "shared_class_uri".to_string(), + "Alpha".to_string(), + "Beta".to_string() + ]], + "the sharing classes are the subject, behind the rule's discriminator, \ + and the controls stay silent: {warnings:#?}" ); let w = &warnings[0]; assert_eq!( From d8fb2d6134a87984230dc7ccd6437508f2392aa1 Mon Sep 17 00:00:00 2001 From: ejsyx Date: Wed, 19 Aug 2026 15:33:37 +0200 Subject: [PATCH 55/58] fix(runtime): the keyed segment resolver refuses ambiguity too MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `resolve_list_segment`'s keyed branch took the FIRST normalising match while its positional branch refused two — one function answering "which element does this segment mean?" two ways. Both branches now share one helper, so neither can drift into resolving an ambiguity the other refuses. The second hit is barely reachable in the keyed branch (labels are unique there, so two of them can only normalise alike under different converters — a heterogeneous list drawn from schemas that disagree about a prefix), which is why it is a refusal and not a `debug_assert`: the case exists, and asserting it away would turn "I cannot tell which element you mean" into a crash. Left untested for the same reason it is hard to reach; the shared helper is what keeps the rule true. Also records, in `reconcile_dict_key_with_payload`'s rustdoc, why its two checks compare by different notions of equality: set membership is the raw match the canonicaliser performs, while "do these two spellings name one element" is `canonical_identity_component`'s question everywhere else. Co-Authored-By: Claude Fable 5 --- src/runtime/src/diff.rs | 37 ++++++++++++++++++++++++++----------- src/runtime/src/lib.rs | 12 ++++++++++++ 2 files changed, 38 insertions(+), 11 deletions(-) diff --git a/src/runtime/src/diff.rs b/src/runtime/src/diff.rs index 5f2beb9..f470317 100644 --- a/src/runtime/src/diff.rs +++ b/src/runtime/src/diff.rs @@ -955,9 +955,33 @@ pub(crate) fn resolve_list_segment(values: &[LinkMLInstance], key: &str) -> Opti .map(|i| i + from) }; let matches = |i: usize| segment_matches_label(&values[i], labels[i].as_deref(), key); + // The one normalising hit, or nothing: two elements the segment could mean + // is a question, not an answer, and this function reports rather than + // guesses. Both branches below share it, so neither can drift into + // resolving an ambiguity the other refuses. + // + // In the keyed branch the second hit is barely reachable: labels are unique + // there, and two of them can only normalise to one string if they were + // normalised by *different converters* — a heterogeneous list whose element + // classes come from schemas that disagree about a prefix. It is left as a + // refusal rather than an assertion precisely because it is reachable at all: + // a `debug_assert` would turn "I cannot tell which element you mean" into a + // crash. + let unique_normalised = || { + let mut hit: Option = None; + for i in 0..values.len() { + if matches(i) { + if hit.is_some() { + return None; + } + hit = Some(i); + } + } + hit + }; if list_is_keyed_shaped_from_labels(&labels) { // Labels are unique here, so an exact hit is the only exact hit. - return exact(0).or_else(|| (0..values.len()).find(|i| matches(*i))); + return exact(0).or_else(unique_normalised); } // Positional list: numeric index first (the segments diff produces for // these lists), then a single unambiguous label hit for drift tolerance. @@ -974,16 +998,7 @@ pub(crate) fn resolve_list_segment(values: &[LinkMLInstance], key: &str) -> Opti None => Some(first), }; } - let mut hit: Option = None; - for i in 0..values.len() { - if matches(i) { - if hit.is_some() { - return None; - } - hit = Some(i); - } - } - hit + unique_normalised() } fn try_update_scalar_in_place( diff --git a/src/runtime/src/lib.rs b/src/runtime/src/lib.rs index cd284bb..1122f02 100644 --- a/src/runtime/src/lib.rs +++ b/src/runtime/src/lib.rs @@ -998,6 +998,18 @@ impl LinkMLInstance { /// the key, and `populate_type_designator` filled the slot from the range /// class, so the entry looked complete. /// + /// The two checks compare **by different notions of equality**, and that is + /// deliberate rather than an oversight. Check 2 asks whether a value is in + /// the accepted designator set, and membership of that set is the raw string + /// match `canonicalize_type_designator` performs verbatim; deciding it here + /// on an IRI-equal-but-differently-spelled basis would wave a value through + /// that the canonicaliser then rewrites and reports itself — two voices for + /// one fact. Check 1 asks whether two spellings name the same element, which + /// is exactly what [`crate::diff::canonical_identity_component`] answers + /// everywhere else (spec addendum rule 2), so a CURIE key and an expanded + /// stored value are one identity here as they are in diff and patch. The + /// questions differ, so the equalities do. + /// /// The key slot itself is [`ClassView::key_or_identifier_slot`] on the /// slot's **range** class, not on the class finally selected: the key has to /// be read (and injected) before `select_class` can run, so a key declared From a4b20878e236d624d877f10731c711372d017c20 Mon Sep 17 00:00:00 2001 From: ejsyx Date: Wed, 19 Aug 2026 15:37:40 +0200 Subject: [PATCH 56/58] fix(tools): validate marks a warning as one wherever it prints it MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The previous commit marked severity only in the valid branch, leaving a warning listed beside errors in the invalid branch spelled exactly like them — so the Critical's own complaint ("the CLI prints a warning as if an error") survived for documents that carry both, which is the shape asset360's committed signal-obj-with-track.json has. Both lists now go through one printer: an error keeps the bare `Type at path: detail` line, so no document whose diagnostics are all errors moves a byte, and anything that is not an error says what it is. Co-Authored-By: Claude Fable 5 --- src/tools/src/bin/linkml_validate.rs | 51 +++++++++++++----------- src/tools/tests/data/validate_mixed.json | 8 ++++ src/tools/tests/validate_cli.rs | 23 ++++++++++- 3 files changed, 57 insertions(+), 25 deletions(-) create mode 100644 src/tools/tests/data/validate_mixed.json diff --git a/src/tools/src/bin/linkml_validate.rs b/src/tools/src/bin/linkml_validate.rs index ae25b9a..6ce0b12 100644 --- a/src/tools/src/bin/linkml_validate.rs +++ b/src/tools/src/bin/linkml_validate.rs @@ -95,38 +95,17 @@ fn main() -> Result<(), Box> { } } else if is_valid { println!("valid"); - // Non-error diagnostics, which a valid document may now carry. Marked - // with their severity, in front of the shape the error list below uses: - // the same line without the marker is what this binary prints for an - // error, and a warning must not be mistakable for one. + // A valid document may now carry diagnostics, all of them non-error. for issue in &validation_issues { - println!( - "{}: {:?} at {}: {}", - severity_label(&issue.severity), - issue.problem_type, - format_path(&issue.subject), - issue.detail - ); + print_issue(issue); } for w in &identity_warnings { println!("warning[{}]: {}", w.subject.join("."), w.detail); } Ok(()) } else { - // Deliberately unmarked, and deliberately including the non-error - // issues: this list is a published output shape of the binary, and - // re-spelling every line of it would move the output of every document - // that has ever failed to validate, to say something about the few that - // also carry a warning. The marker above is where the distinction is - // needed — there, an unmarked line would read as an error on a document - // that has none. for issue in &validation_issues { - println!( - "{:?} at {}: {}", - issue.problem_type, - format_path(&issue.subject), - issue.detail - ); + print_issue(issue); } if args.lint_identity { println!("note: --lint-identity skipped: fix the validation errors above first"); @@ -135,6 +114,30 @@ fn main() -> Result<(), Box> { } } +/// One validation issue, in text mode, wherever it appears. +/// +/// An error keeps the bare `Type at path: detail` line this binary has always +/// printed — so a document whose diagnostics are all errors prints exactly what +/// it always printed, which is every document that predates the spec addendum's +/// warnings. Anything that is *not* an error is named, because the alternative +/// is a warning that looks like the error it is listed beside: in the valid +/// branch it would be an error on a document that has none, and in the error +/// branch it would be one more thing the reader is told to fix before re-running. +fn print_issue(issue: &ValidationResult) { + let location = format_path(&issue.subject); + if issue.severity.is_error() { + println!("{:?} at {}: {}", issue.problem_type, location, issue.detail); + } else { + println!( + "{}: {:?} at {}: {}", + severity_label(&issue.severity), + issue.problem_type, + location, + issue.detail + ); + } +} + fn emit_json( valid: bool, issues: &[ValidationResult], diff --git a/src/tools/tests/data/validate_mixed.json b/src/tools/tests/data/validate_mixed.json new file mode 100644 index 0000000..fff1635 --- /dev/null +++ b/src/tools/tests/data/validate_mixed.json @@ -0,0 +1,8 @@ +{ + "people": { + "p1": {"pid": "p2", "name": "Ann"} + }, + "readings": [ + {"code": "c1", "not_a_slot": "x"} + ] +} diff --git a/src/tools/tests/validate_cli.rs b/src/tools/tests/validate_cli.rs index 64c015a..8466910 100644 --- a/src/tools/tests/validate_cli.rs +++ b/src/tools/tests/validate_cli.rs @@ -17,7 +17,8 @@ //! it in `issues` with `severity: "warning"` as it always did); //! * `--lint-identity` **runs** on such a document — the lint is gated on //! errors, not on diagnostics; -//! * an **error** still skips the lint, with the reason stated in both modes; +//! * an **error** still skips the lint, with the reason stated in both modes, +//! and a warning listed beside errors is still marked as a warning; //! * a warning-free document invoked without the flag produces byte-for-byte //! the output it always did. @@ -140,6 +141,26 @@ fn errors_skip_the_lint_and_say_so() { assert_eq!(doc["identity_warnings"], serde_json::Value::Null); } +/// A document with both. The error decides validity, and the two lines are +/// told apart: the error keeps the bare shape this binary has always printed +/// for one, the warning says what it is. Anything else asks the reader to fix +/// "errors" that include a warning — the same misreading the exit code used to +/// force, one line further down. +#[test] +fn a_warning_beside_an_error_is_still_marked_as_a_warning() { + let (code, out) = validate("validate_mixed.json", &[]); + assert_eq!(code, 1, "{out}"); + assert!( + out.contains("warning: SlotRangeViolation at people.p1.pid:"), + "the warning names its severity: {out}" + ); + assert!( + out.contains("UndeclaredSlot at readings.0.not_a_slot:") + && !out.contains("error: UndeclaredSlot"), + "the error keeps the published unmarked shape: {out}" + ); +} + // --------------------------------------------------------------------------- // Nothing moved for a document with no diagnostics // --------------------------------------------------------------------------- From a793ffb1ffbe2945df0c9c8a53bc4f35b2f43e69 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ezechi=C3=ABl=20Syx?= Date: Fri, 21 Aug 2026 12:55:02 +0200 Subject: [PATCH 57/58] fix(runtime): unresolved list Update re-adds when the payload names its address MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ae8e8d0 made every `Update` whose list address resolves to nothing report, which breaks the multi-source merge it matters most for: one source drops an element, another that still holds it describes it as an `Update` (its delta was computed against an older golden), and the element silently disappears from the merged record. The payload decides instead of the op. An element carries its own identity — key/identifier, else a `unique_keys`-derived label — so it can be checked against the address it was filed under: - identity names the address -> append, the re-add the merge intends - identity names a different element -> report; appending would duplicate a label and knock the list out of keyed matching entirely - no identity at all -> append only in a positional or emptied list, never in an identity-addressed one This keeps what ae8e8d0 was really protecting: a stale positional `Update` into a keyed list still reports rather than overwriting whichever element moved into that index. `resolve_list_index` is what refuses that address, not the leaf guard, so restoring the append does not restore the clobber. A build failure on this path now reports rather than erroring the whole patch — the value was never going to be applied. Co-Authored-By: Claude Opus 5 (1M context) --- src/runtime/src/diff.rs | 40 +++++++++++++++++---- src/runtime/tests/diff_unique_keys.rs | 51 +++++++++++++++++++++++---- 2 files changed, 78 insertions(+), 13 deletions(-) diff --git a/src/runtime/src/diff.rs b/src/runtime/src/diff.rs index f470317..e05297d 100644 --- a/src/runtime/src/diff.rs +++ b/src/runtime/src/diff.rs @@ -1107,6 +1107,7 @@ where fn apply_list_leaf_delta( values: &mut Vec, idx_opt: Option, + key: &str, owner_id: NodeId, trace: &mut PatchTrace, opts: PatchOptions, @@ -1118,13 +1119,38 @@ where { match op { DeltaOp::Add | DeltaOp::Update => { - // An `Update` addressing an element that is not there is a stale - // address, not an invitation to append: report, never guess. Only - // `Add` treats "no such index" as "put it at the end". Checked - // before `build_child` so a failed address reports rather than - // surfacing a build error for a value that is never applied. + // An `Update` whose address resolves to no element is either a + // source re-asserting an element some other source dropped — the + // multi-source case, where appending is the intended merge — or a + // stale address, where appending would invent an element. The + // payload's own identity decides which: it must name the element + // the path addresses. `Add` always appends. if idx_opt.is_none() && matches!(op, DeltaOp::Update) { - return Ok(false); + // Build failure means the value could never be applied + // anyway: report the path instead of erroring the patch. + let Some(new_child) = build_or_fail(build_child) else { + return Ok(false); + }; + let allowed = match element_identity_label(&new_child) { + // Identity present: only the address that names it may + // append. Blocks stale positional Updates into a keyed + // list, which used to overwrite the wrong element. The + // comparison is the resolver's, so an address that WOULD + // have resolved to this element had it still been there + // is the address that may re-add it — a segment spelled + // as a CURIE against an IRI-expanded label included. + Some(label) => segment_matches_label(&new_child, Some(&label), key), + // No identity to check (scalar element, unkeyed range): + // only a positional or emptied list may grow this way. + None => !list_is_keyed_shaped(values), + }; + if !allowed { + return Ok(false); + } + mark_added_subtree(&new_child, trace); + values.push(new_child); + trace.updated.push(owner_id); + return Ok(true); } let Some(new_child) = build_or_fail(build_child) else { return Ok(false); @@ -1429,7 +1455,7 @@ fn apply_delta_list( let value = newv.cloned().unwrap_or(JsonValue::Null); let slot_clone = slot.clone(); let class_clone = class.clone(); - return apply_list_leaf_delta(values, idx_opt, owner_id, trace, opts, op, || { + return apply_list_leaf_delta(values, idx_opt, key, owner_id, trace, opts, op, || { with_converter(schema_view, value, move |val, sv, conv| { let mut diags = ValidationResultSink::default(); let value = LinkMLInstance::build_list_item_for_slot( diff --git a/src/runtime/tests/diff_unique_keys.rs b/src/runtime/tests/diff_unique_keys.rs index 4acdc6e..f2493b3 100644 --- a/src/runtime/tests/diff_unique_keys.rs +++ b/src/runtime/tests/diff_unique_keys.rs @@ -348,8 +348,9 @@ fn patch_refuses_positional_update_into_identity_addressed_list() { let f = fixture(); let golden = f.load(phones(vec![e(), n()])); // A stale positional Update against a keyed-shaped list resolves to no - // element. It must report, not append: an unresolved address is never an - // invitation to grow the list. + // element, and its payload's identity ("Emergency_Number") does not name + // the address ("0"). It must report: appending would invent an element, + // and on `main` this address overwrote whichever element sat at index 0. let mut e2 = e(); e2["phoneNumber"] = json!("09/999.99.99"); let delta = Delta { @@ -369,12 +370,14 @@ fn patch_refuses_positional_update_into_identity_addressed_list() { } #[test] -fn patch_refuses_update_whose_label_matches_nothing() { +fn update_whose_payload_names_its_address_is_re_added() { let f = fixture(); let golden = f.load(phones(vec![e(), n()])); - // An Update addressing an element that is not there: the producer meant to - // edit an existing Operator entry, and the golden has none. Reporting is - // the only honest answer — appending would invent an edit as a creation. + // Multi-source: one source dropped the Operator entry, another still has + // it and describes it as an Update (its delta was computed against an + // older golden). The payload's identity names exactly the element the path + // addresses, so the Update re-adds it — the field comes back rather than + // being reported as a failure and silently lost. let delta = Delta { path: vec!["hasPhoneNumber".to_string(), "Operator".to_string()], op: DeltaOp::Update, @@ -387,6 +390,42 @@ fn patch_refuses_update_whose_label_matches_nothing() { PatchOptions::default(), ) .unwrap(); + assert!(trace.failed.is_empty(), "failed: {:?}", trace.failed); + assert!( + patched.equals(&f.load(phones(vec![e(), n(), o()])), true), + "{}", + patched.to_json() + ); + // And the round trip is clean: re-diffing the result against the intent + // yields nothing. + assert!( + diff2(&f, patched.to_json(), phones(vec![e(), n(), o()])).is_empty(), + "re-added element must be indistinguishable from an Add" + ); +} + +#[test] +fn patch_refuses_update_whose_payload_contradicts_its_address() { + let f = fixture(); + let golden = f.load(phones(vec![e(), n()])); + // A label address that resolves to nothing AND a payload naming a + // different element: the address is stale, not a re-add. Appending would + // duplicate the Non_Urgent_Communication entry the list already carries + // and knock the whole list off keyed matching. + let mut n2 = n(); + n2["phoneNumber"] = json!("09/999.99.99"); + let delta = Delta { + path: vec!["hasPhoneNumber".to_string(), "Operator".to_string()], + op: DeltaOp::Update, + old: Some(o()), + new: Some(n2), + }; + let (patched, trace) = patch( + &golden, + std::slice::from_ref(&delta), + PatchOptions::default(), + ) + .unwrap(); assert_eq!(trace.failed, vec![delta.path.clone()]); assert!(patched.equals(&golden, true), "nothing may be appended"); } From 3021b68d19a322a941776e296aa667b40cf80171 Mon Sep 17 00:00:00 2001 From: ejsyx Date: Fri, 21 Aug 2026 13:28:30 +0200 Subject: [PATCH 58/58] fix(runtime): allow clippy's argument-count lint on apply_list_leaf_delta Clippy (-D warnings) rejects the helper's 8 arguments; its sibling delta-application helpers in this file already carry the same allow. Co-Authored-By: Claude Fable 5 --- src/runtime/src/diff.rs | 1 + 1 file changed, 1 insertion(+) diff --git a/src/runtime/src/diff.rs b/src/runtime/src/diff.rs index e05297d..bf7948d 100644 --- a/src/runtime/src/diff.rs +++ b/src/runtime/src/diff.rs @@ -1104,6 +1104,7 @@ where } } +#[allow(clippy::too_many_arguments)] fn apply_list_leaf_delta( values: &mut Vec, idx_opt: Option,