diff --git a/docs/superpowers/plans/2026-08-18-inlined-multivalued-element-identity.md b/docs/superpowers/plans/2026-08-18-inlined-multivalued-element-identity.md new file mode 100644 index 0000000..b66da00 --- /dev/null +++ b/docs/superpowers/plans/2026-08-18-inlined-multivalued-element-identity.md @@ -0,0 +1,1675 @@ +# Inlined Multivalued Element Identity Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Make element identity for inlined multivalued slots declarable — key/identifier, `unique_keys` composed key, or `diff.linkml.io/opaque` (nowhere) — and add an opt-in linter that flags slots where identity comes from nowhere. + +**Architecture:** Three additive features in the existing crates. (1) A `diff.linkml.io/opaque` slot annotation makes `diff` stop all recursion and emit one whole-value `Update`, and makes `patch` refuse to descend below the slot. (2) `diff`'s keyed-list matching learns a fallback identity derived from the range class's (inheritance-merged) `unique_keys`; `patch` learns to resolve those labels. (3) A new `identity_lint` module offers two opt-in entry points (schema-level ambiguity lint, data-level duplicate-identity lint) that are **not** wired into default validation. No changes to `DiffOptions`/`PatchOptions` shapes; behaviour only changes for schemas that declare the annotation or `unique_keys` (no current fixture or consumer does). + +**Tech Stack:** Rust workspace (`linkml_runtime`, `schemaview`, `linkml_tools`, `linkml_runtime_python` via PyO3), cargo integration tests. + +**Spec:** `docs/superpowers/specs/2026-08-17-inlined-multivalued-element-identity-design.md` + +## Global Constraints + +- **Non-goal + deliberate break (spec, Non-goal section):** slots that declare nothing keep positional semantics, but keyed matching becomes uniform: a list matches by identity iff every element on both sides yields an identity label (key/identifier first, else `unique_keys`) AND the labels are unique within each side; every other case is positional with plain numeric path segments. This deliberately removes (a) opportunistic key labels mixed into positional paths and (b) the silent collapse of duplicate key values under keyed matching. Existing tests that assert those two removed behaviours are updated — the implementer's report must list each updated test with its old assertion and why it changed. All other existing tests pass unchanged. +- **"Report, never guess" (spec):** a patch that cannot locate its target unambiguously returns `Ok(false)` so the path lands in `PatchTrace::failed`. No fuzzy fallbacks. +- **Layering (spec):** validation/lint code never reads `diff.linkml.io/*` to *suppress* a schema-constraint finding. The schema-level linter may skip opaque slots (identity is declared: nowhere); the data-level duplicate check never consults the annotation. +- `PatchTrace::failed` stays `Vec>` — do not import the abandoned branch's `PatchFailure` struct. +- Every task ends green: `cargo test --workspace` (or the named `-p` subset), `cargo fmt --all`, `cargo clippy --workspace --all-targets -- -D warnings` at the final task. +- Commit after every task; message style from `git log`: `feat(runtime): ...`, `test(runtime): ...`, `feat(schemaview): ...`. + +--- + +### Task 1: `ClassView::unique_keys()` — inheritance-merged accessor + +The metamodel already parses `unique_keys` (`src/metamodel/src/lib.rs:11520`, `ClassDefinition.unique_keys: Option>>`) but `ClassView::def()` returns the raw definition: keys declared on an `is_a` parent or mixin are invisible (ClassView merges slots, not unique_keys). Everything downstream (diff matching, linter) needs a merged, deterministic view. + +**Files:** +- Modify: `src/schemaview/src/classview.rs` (new method near `key_or_identifier_slot()`, ~line 606) +- Create: `src/schemaview/tests/unique_keys.rs` +- Create: `src/schemaview/tests/data/unique_keys.yaml` + +**Interfaces:** +- Consumes: `ClassView::def()`, `ClassView::parent_class()` (classview.rs:590), the mixin-resolution pattern of `collect_ancestors_map` (classview.rs:624), `linkml_meta::UniqueKey { unique_key_name, unique_key_slots: Vec, consider_nulls_inequal, .. }`. +- Produces: `pub fn unique_keys(&self) -> Vec<(String, UniqueKey)>` on `ClassView` — merged across `is_a` and mixins (nearest definition wins per name), **sorted by name** (HashMap order is nondeterministic and diff paths must be stable). Tasks 4, 5, 6 rely on exactly this signature. + +- [ ] **Step 1: Write the fixture schema** + +`src/schemaview/tests/data/unique_keys.yaml` (copy the header — `id`, `name`, `prefixes`, `default_range`, types/imports — from an existing fixture in `src/schemaview/tests/data/` so types resolve the same way): + +```yaml +classes: + Base: + unique_keys: + by_code: + unique_key_slots: [code] + shared_name: + unique_key_slots: [base_field] + attributes: + code: {range: string} + base_field: {range: string} + MixinCls: + mixin: true + unique_keys: + by_tag: + unique_key_slots: [tag] + attributes: + tag: {range: string} + Child: + is_a: Base + mixins: [MixinCls] + unique_keys: + shared_name: # overrides Base's entry of the same name + unique_key_slots: [child_field] + attributes: + child_field: {range: string} + Plain: + attributes: + whatever: {range: string} +``` + +- [ ] **Step 2: Write the failing test** + +`src/schemaview/tests/unique_keys.rs` (mirror the SchemaView setup of a neighbouring test, e.g. `class_lookup.rs`): + +```rust +use linkml_schemaview::identifier::{converter_from_schema, Identifier}; +use linkml_schemaview::io::from_yaml; +use linkml_schemaview::schemaview::SchemaView; +use std::path::PathBuf; + +fn fixture() -> SchemaView { + let mut p = PathBuf::from(env!("CARGO_MANIFEST_DIR")); + p.push("tests/data/unique_keys.yaml"); + let schema = from_yaml(&p).unwrap(); + let mut sv = SchemaView::new(); + sv.add_schema(schema).unwrap(); + sv +} + +#[test] +fn unique_keys_merge_across_is_a_and_mixins_nearest_wins() { + let sv = fixture(); + let conv = sv.converter(); + let child = sv + .get_class(&Identifier::new("Child"), &conv) + .unwrap() + .expect("class not found"); + let uks = child.unique_keys(); + let names: Vec<&str> = uks.iter().map(|(n, _)| n.as_str()).collect(); + // name-sorted, merged from Base (by_code), MixinCls (by_tag), Child (shared_name override) + assert_eq!(names, vec!["by_code", "by_tag", "shared_name"]); + let shared = &uks.iter().find(|(n, _)| n == "shared_name").unwrap().1; + assert_eq!( + shared.unique_key_slots, + vec!["child_field".to_string()], + "the nearest declaration must win" + ); +} + +#[test] +fn class_without_unique_keys_yields_empty() { + let sv = fixture(); + let conv = sv.converter(); + let plain = sv + .get_class(&Identifier::new("Plain"), &conv) + .unwrap() + .expect("class not found"); + assert!(plain.unique_keys().is_empty()); +} +``` + +- [ ] **Step 3: Run tests to verify they fail** + +Run: `cargo test -p schemaview --test unique_keys` +Expected: FAIL to compile — `unique_keys` method not found on `ClassView`. + +- [ ] **Step 4: Implement the accessor** + +In `src/schemaview/src/classview.rs`, near `key_or_identifier_slot()`. BFS from `self` through `is_a` parents and mixins so nearer declarations claim a name first (`or_insert`). The sketch below uses `parent_class()` for the `is_a` chain; resolve mixin names to `ClassView`s the same way `collect_ancestors_map` (classview.rs:624) does — adjust to that private API rather than inventing a new resolution path. + +```rust + /// The class's `unique_keys`, merged across the inheritance chain + /// (`is_a` parents and mixins), nearest declaration winning per name. + /// Name-sorted: declaration order is lost in the underlying map and + /// consumers (diff path segments) need a deterministic order. + pub fn unique_keys(&self) -> Vec<(String, UniqueKey)> { + use std::collections::{HashMap, VecDeque}; + let mut merged: HashMap = HashMap::new(); + let mut queue: VecDeque = VecDeque::from([self.clone()]); + let mut seen: std::collections::HashSet = Default::default(); + while let Some(cv) = queue.pop_front() { + if !seen.insert(cv.name().to_string()) { + continue; + } + if let Some(uks) = cv.def().unique_keys.as_ref() { + for (name, uk) in uks { + merged + .entry(name.clone()) + .or_insert_with(|| (**uk).clone()); + } + } + if let Ok(Some(parent)) = cv.parent_class() { + queue.push_back(parent); + } + // + push each resolved mixin ClassView (resolution as in + // collect_ancestors_map, classview.rs:624) + } + let mut out: Vec<(String, UniqueKey)> = merged.into_iter().collect(); + out.sort_by(|a, b| a.0.cmp(&b.0)); + out + } +``` + +Import `linkml_meta::UniqueKey` at the top of the file (check the existing `linkml_meta` imports there for the path). + +- [ ] **Step 5: Run tests to verify they pass** + +Run: `cargo test -p schemaview --test unique_keys` +Expected: PASS (both tests). + +- [ ] **Step 6: Run the schemaview suite for regressions** + +Run: `cargo test -p schemaview` +Expected: PASS, no other test touched. + +- [ ] **Step 7: Commit** + +```bash +git add src/schemaview/src/classview.rs src/schemaview/tests/unique_keys.rs src/schemaview/tests/data/unique_keys.yaml +git commit -m "feat(schemaview): ClassView::unique_keys merged across is_a and mixins" +``` + +--- + +### Task 2: shared fixture + `diff.linkml.io/opaque` on the diff side + +**Files:** +- Create: `src/runtime/tests/data/identity.yaml` +- Create: `src/runtime/tests/diff_opaque.rs` +- Modify: `src/runtime/src/diff.rs` (annotation const + check + early-return in `inner`, near lines 10–21 and 121–125) +- Modify: `src/runtime/src/lib.rs` (re-export `OPAQUE_ANNOTATION` in the `pub use diff::{...}` list, line 49) + +**Interfaces:** +- Consumes: `slot_is_ignored` pattern (`diff.rs:12-21`), `LinkMLInstance::equals(other, treat_missing_as_null)`. +- Produces: `pub const OPAQUE_ANNOTATION: &str = "diff.linkml.io/opaque"` and `pub(crate) fn slot_is_opaque(slot: &SlotView) -> bool` in `diff.rs` (Tasks 3 and 6 use both); the fixture `identity.yaml` (all later runtime test files load it). + +- [ ] **Step 1: Write the fixture schema** + +`src/runtime/tests/data/identity.yaml`. Copy the header (`id`, `name`, `prefixes`, `default_range`, and the types/imports mechanism) from `src/runtime/tests/data/personinfo.yaml` so `string`/`float`/`uri` resolve identically. Classes (this is the spec's asset360 material, condensed): + +```yaml +classes: + Service: + attributes: + name: {range: string} + # spec Example 1 with its correct resolution: identity = the function + hasPhoneNumber: + range: ServicePhoneNumber + multivalued: true + inlined_as_list: true + # same shape, nothing declared: must keep positional behaviour; linter flags it + plainPhoneNumber: + range: PlainPhoneNumber + multivalued: true + inlined_as_list: true + # inherited unique_keys (via is_a) must also drive matching + escalation: + range: EmergencyPhoneNumber + multivalued: true + inlined_as_list: true + # composite two-slot unique key + contacts: + range: Contact + multivalued: true + inlined_as_list: true + # opaque + a keyed element class: data lint must still check unique_keys + archivedContacts: + range: Contact + multivalued: true + inlined_as_list: true + annotations: + diff.linkml.io/opaque: true + # scalar list, nothing declared: positional; linter flags it + tags: + range: string + multivalued: true + # scalar list declared opaque + opaqueTags: + range: string + multivalued: true + annotations: + diff.linkml.io/opaque: true + # spec Example 2's ring: opaque list of bare vertices + outline: + range: Vertex + multivalued: true + inlined_as_list: true + annotations: + diff.linkml.io/opaque: true + # single-valued opaque object: "stop all recursion" is not list-specific + profile: + range: Profile + inlined: true + annotations: + diff.linkml.io/opaque: true + # keyed class inlined as dict: already unambiguous, linter stays silent + labels: + range: Label + multivalued: true + inlined: true + # keyed class inlined as list: the uniform guard applies to key labels too + labelList: + range: Label + multivalued: true + inlined_as_list: true + # reference list (not inlined): out of the linter's scope + operators: + range: Operator + multivalued: true + inlined: false + area: + range: AreaLocation + inlined: true + + ServicePhoneNumber: + unique_keys: + one_number_per_function: + unique_key_slots: [hasNumberFunction] + attributes: + phoneNumber: {range: string} + hasNumberFunction: {range: NumberFunction, required: true} + + PlainPhoneNumber: + attributes: + phoneNumber: {range: string} + hasNumberFunction: {range: NumberFunction, required: true} + + EmergencyPhoneNumber: + is_a: ServicePhoneNumber + attributes: + note: {range: string} + + Contact: + unique_keys: + contact_identity: + unique_key_slots: [kind, phone] + attributes: + kind: {range: string, required: true} + phone: {range: string, required: true} + note: {range: string} + + Label: + attributes: + lang: {range: string, key: true, required: true} + text: {range: string} + + Profile: + attributes: + bio: {range: string} + motto: {range: string} + + Operator: + attributes: + opId: {range: string, identifier: true} + opName: {range: string} + + AreaLocation: + attributes: + polygons: + range: Polygon + multivalued: true + inlined_as_list: true + + Polygon: + unique_keys: + one_polygon_per_positioning_system: + unique_key_slots: [positioningSystemType] + attributes: + positioningSystemType: {range: uri, required: true} + coordinates: + range: Vertex + multivalued: true + inlined_as_list: true + annotations: + diff.linkml.io/opaque: true + + Vertex: + attributes: + x: {range: float} + y: {range: float} + +enums: + NumberFunction: + permissible_values: + Emergency_Number: + Non_Urgent_Communication: + Operator: +``` + +- [ ] **Step 2: Write the failing tests** + +`src/runtime/tests/diff_opaque.rs`. The move/insert/drop/reverse loop is recycled near-verbatim from the abandoned branch's `array_slot_emits_one_whole_slot_update` (its `feat/container-shapes-and-verify-old` branch, `src/runtime/tests/diff_shapes.rs:158-197`) — same assertions, annotation renamed. + +```rust +use linkml_runtime::{diff, load_json_str, patch, DiffOptions, LinkMLInstance, PatchOptions}; +use linkml_schemaview::identifier::{converter_from_schema, Identifier}; +use linkml_schemaview::io::from_yaml; +use linkml_schemaview::schemaview::{ClassView, SchemaView}; +use linkml_schemaview::Converter; +use linkml_runtime::{Delta, DeltaOp}; +use serde_json::{json, Value as JsonValue}; +use std::path::PathBuf; + +struct Fixture { + sv: SchemaView, + conv: Converter, + service: ClassView, +} + +fn fixture() -> Fixture { + let mut p = PathBuf::from(env!("CARGO_MANIFEST_DIR")); + p.push("tests/data/identity.yaml"); + let schema = from_yaml(&p).unwrap(); + let mut sv = SchemaView::new(); + sv.add_schema(schema.clone()).unwrap(); + let conv = converter_from_schema(&schema); + let service = sv + .get_class(&Identifier::new("Service"), &conv) + .unwrap() + .expect("class not found"); + Fixture { sv, conv, service } +} + +impl Fixture { + fn load(&self, v: JsonValue) -> LinkMLInstance { + load_json_str(&v.to_string(), &self.sv, &self.service, &self.conv) + .unwrap() + .into_instance() + .unwrap() + } +} + +fn diff2(f: &Fixture, before: JsonValue, after: JsonValue) -> Vec { + diff(&f.load(before), &f.load(after), DiffOptions::new(true)) +} + +fn only(deltas: &[Delta]) -> &Delta { + assert_eq!(deltas.len(), 1, "expected exactly one delta: {deltas:#?}"); + &deltas[0] +} + +fn square() -> Vec { + vec![ + json!({"x": 4.35, "y": 50.85}), + json!({"x": 4.36, "y": 50.85}), + json!({"x": 4.36, "y": 50.86}), + json!({"x": 4.35, "y": 50.86}), + ] +} + +fn outline(items: Vec) -> JsonValue { + json!({"name": "svc", "outline": items}) +} + +#[test] +fn opaque_ring_edit_is_one_whole_slot_update() { + let f = fixture(); + + let mut moved = square(); + moved[1] = json!({"x": 4.37, "y": 50.85}); + let mut inserted = square(); + inserted.insert(2, json!({"x": 4.365, "y": 50.855})); + let dropped_first = square()[1..].to_vec(); + let mut reversed = square(); + reversed.reverse(); + + for (label, after) in [ + ("move one vertex", moved), + ("insert a vertex mid-ring", inserted), + ("drop the first vertex", dropped_first), + ("reverse the ring", reversed), + ] { + let deltas = diff2(&f, outline(square()), outline(after)); + let delta = only(&deltas); + assert_eq!(delta.path, vec!["outline".to_string()], "{label}"); + assert_eq!(delta.op, DeltaOp::Update, "{label}"); + assert_eq!( + delta.old.as_ref().and_then(|v| v.as_array()).map(|a| a.len()), + Some(4), + "{label}: old must be the whole slot" + ); + assert!(delta.new.as_ref().is_some_and(|v| v.is_array()), "{label}"); + } +} + +#[test] +fn opaque_slot_unchanged_emits_nothing() { + let f = fixture(); + let deltas = diff2(&f, outline(square()), outline(square())); + assert!(deltas.is_empty(), "{deltas:#?}"); +} + +#[test] +fn opaque_scalar_list_is_one_whole_slot_update() { + let f = fixture(); + let before = json!({"name": "svc", "opaqueTags": ["a", "b"]}); + let after = json!({"name": "svc", "opaqueTags": ["b", "c", "d"]}); + let deltas = diff2(&f, before, after); + let delta = only(&deltas); + assert_eq!(delta.path, vec!["opaqueTags".to_string()]); + assert_eq!(delta.op, DeltaOp::Update); +} + +#[test] +fn opaque_single_valued_object_is_one_whole_value_update() { + let f = fixture(); + let before = json!({"name": "svc", "profile": {"bio": "b", "motto": "old"}}); + let after = json!({"name": "svc", "profile": {"bio": "b", "motto": "new"}}); + let deltas = diff2(&f, before, after); + let delta = only(&deltas); + assert_eq!(delta.path, vec!["profile".to_string()]); + assert_eq!(delta.op, DeltaOp::Update); + assert_eq!(delta.old, Some(json!({"bio": "b", "motto": "old"}))); + assert_eq!(delta.new, Some(json!({"bio": "b", "motto": "new"}))); +} + +#[test] +fn opaque_whole_slot_update_round_trips_through_patch() { + let f = fixture(); + let before = outline(square()); + let mut moved = square(); + moved[1] = json!({"x": 4.37, "y": 50.85}); + let after = outline(moved); + let deltas = diff2(&f, before.clone(), after.clone()); + let (patched, trace) = patch(&f.load(before), &deltas, PatchOptions::default()).unwrap(); + assert!(trace.failed.is_empty(), "{:?}", trace.failed); + assert!( + patched.equals(&f.load(after), true), + "round-trip mismatch: {}", + patched.to_json() + ); +} +``` + +(If `Delta`/`DeltaOp`/`PatchOptions` import paths differ, copy the exact imports from `src/runtime/tests/diff.rs`.) + +- [ ] **Step 3: Run tests to verify they fail** + +Run: `cargo test -p linkml_runtime --test diff_opaque` +Expected: FAIL — `opaque_ring_edit_is_one_whole_slot_update` sees many positional deltas instead of one; the single-valued and scalar cases see field-level/element-level deltas. (The round-trip test may pass already; that's fine.) + +- [ ] **Step 4: Implement the diff side** + +In `src/runtime/src/diff.rs`, next to `IGNORE_ANNOTATION` (line 10): + +```rust +/// Slot annotation declaring that element identity comes from nowhere: stop +/// all recursion, the slot's value is one atomic unit. Any change below the +/// slot is described as a single whole-value `Update` at the slot path, and +/// `patch` refuses paths that descend below it. +pub const OPAQUE_ANNOTATION: &str = "diff.linkml.io/opaque"; + +pub(crate) fn slot_is_opaque(slot: &SlotView) -> bool { + if slot.definitions().is_empty() { + return false; + } + slot.definition() + .annotations + .as_ref() + .map(|a| a.contains_key(OPAQUE_ANNOTATION)) + .unwrap_or(false) +} +``` + +(Presence-based, exactly like `slot_is_ignored` — the annotation's *presence* is the declaration, mirroring `diff.linkml.io/ignore`.) + +In `diff`'s `inner`, extend the existing slot guard (lines 121–125): + +```rust + if let Some(sl) = slot { + if slot_is_ignored(sl) { + return; + } + if slot_is_opaque(sl) { + if !s.equals(t, opts.treat_missing_as_null) { + out.push(Delta { + path: path.clone(), + op: DeltaOp::Update, + old: Some(s.to_json()), + new: Some(t.to_json()), + }); + } + return; + } + } +``` + +Note: `inner` receives `slot: Some(..)` only for direct slot values (the object arm); list/mapping elements recurse with `None`, so the check fires exactly at slot level. A slot appearing/disappearing entirely is handled by the object arm's missing-slot branches, which already emit whole-value deltas without recursing. + +In `src/runtime/src/lib.rs:49`, add `OPAQUE_ANNOTATION` to the diff re-export list. + +- [ ] **Step 5: Run tests to verify they pass** + +Run: `cargo test -p linkml_runtime --test diff_opaque` +Expected: PASS (all 5). + +- [ ] **Step 6: Run the full runtime suite for regressions** + +Run: `cargo test -p linkml_runtime` +Expected: PASS — no existing fixture carries the annotation, so nothing else may change. + +- [ ] **Step 7: Commit** + +```bash +git add src/runtime/tests/data/identity.yaml src/runtime/tests/diff_opaque.rs src/runtime/src/diff.rs src/runtime/src/lib.rs +git commit -m "feat(runtime): diff.linkml.io/opaque stops recursion, replaces whole value" +``` + +--- + +### Task 3: opaque on the patch side — refuse to descend + +A patch produced outside this diff lib may address structure below an opaque slot (e.g. positional vertex paths). Locating anything below the slot would be a guess; the spec says such deltas are reported as failed. + +**Files:** +- Modify: `src/runtime/src/diff.rs` (`apply_delta_object` descent at ~line 873, `apply_delta_mapping` at ~880, `apply_delta_list` at ~928) +- Test: `src/runtime/tests/diff_opaque.rs` (extend) + +**Interfaces:** +- Consumes: `slot_is_opaque` from Task 2. +- Produces: patch behaviour only — no new names. + +- [ ] **Step 1: Write the failing tests** + +Append to `src/runtime/tests/diff_opaque.rs`: + +```rust +#[test] +fn patch_below_opaque_slot_is_reported_failed_not_guessed() { + let f = fixture(); + let golden = f.load(outline(square())); + let delta = Delta { + path: vec!["outline".to_string(), "1".to_string(), "x".to_string()], + op: DeltaOp::Update, + old: Some(json!(4.36)), + new: Some(json!(9.99)), + }; + let (patched, trace) = patch(&golden, &[delta.clone()], PatchOptions::default()).unwrap(); + assert_eq!(trace.failed, vec![delta.path.clone()]); + assert!( + patched.equals(&golden, true), + "nothing may change: {}", + patched.to_json() + ); +} + +#[test] +fn patch_at_opaque_slot_path_still_applies_whole_value() { + let f = fixture(); + let golden = f.load(outline(square())); + let mut moved = square(); + moved[1] = json!({"x": 4.37, "y": 50.85}); + let delta = Delta { + path: vec!["outline".to_string()], + op: DeltaOp::Update, + old: Some(json!(square())), + new: Some(json!(moved.clone())), + }; + let (patched, trace) = patch(&golden, &[delta], PatchOptions::default()).unwrap(); + assert!(trace.failed.is_empty(), "{:?}", trace.failed); + assert!(patched.equals(&f.load(outline(moved)), true)); +} +``` + +- [ ] **Step 2: Run tests to verify the new one fails** + +Run: `cargo test -p linkml_runtime --test diff_opaque` +Expected: `patch_below_opaque_slot_is_reported_failed_not_guessed` FAILS (the positional path currently applies); the whole-value test passes. + +- [ ] **Step 3: Implement the refusal** + +In `apply_delta_object` (diff.rs, the descent after the `path.len() == 1` early return, ~line 873): + +```rust + if let Some(child) = values.get_mut(key) { + let slot = class.slots().iter().find(|s| s.name == *key); + if slot.is_some_and(slot_is_opaque) { + // The path descends below an opaque slot: it addresses structure + // the slot does not expose. Report, never guess. + return Ok(false); + } + return apply_delta_linkml_inner(child, &path[1..], op, newv, trace, opts); + } +``` + +And defensively at the top of `apply_delta_mapping` and `apply_delta_list` (reached with a non-empty path only when the delta addresses *inside* the container — which is below the container's slot). This covers patches whose root instance is itself a list/mapping: + +```rust + if slot_is_opaque(slot) { + return Ok(false); + } +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `cargo test -p linkml_runtime --test diff_opaque` +Expected: PASS (all 7). + +- [ ] **Step 5: Full runtime suite** + +Run: `cargo test -p linkml_runtime` +Expected: PASS. + +- [ ] **Step 6: Commit** + +```bash +git add src/runtime/src/diff.rs src/runtime/tests/diff_opaque.rs +git commit -m "feat(runtime): patch refuses to descend below an opaque slot" +``` + +--- + +### Task 4: `unique_keys`-derived identity in diff list matching + +**Files:** +- Modify: `src/runtime/src/diff.rs` (extract label helpers, extend the `(List, List)` arm at lines 211–296) +- Create: `src/runtime/tests/diff_unique_keys.rs` + +**Interfaces:** +- Consumes: `ClassView::unique_keys()` (Task 1), the fixture (Task 2). +- Produces (all in `diff.rs`, used by Tasks 5 and 6): + - `pub(crate) fn element_key_label(v: &LinkMLInstance) -> Option` — the existing key/identifier label, extracted from the inline closure. + - `pub(crate) fn element_unique_key_label(v: &LinkMLInstance) -> Option` — label from the first (name-sorted) non-empty `unique_keys` entry; single-slot keys yield the bare scalar string, composite keys yield the JSON-array encoding of the values in `unique_key_slots` order (e.g. `["Emergency","02/111.11.11"]`). + - `pub(crate) fn element_identity_label(v: &LinkMLInstance) -> Option` — `element_key_label` first, `element_unique_key_label` as fallback. + - `fn scalar_slot_string(values: &HashMap, slot_name: &str) -> Option`. + +**Precedence and guard rules (from the spec's Non-goal section — the uniform rule):** +1. opaque > key/identifier > unique_keys > positional. +2. A list is matched by identity iff every element on both sides yields an identity label (`element_identity_label`) AND the labels are unique within each side. The guard is uniform — it applies to key/identifier labels exactly as to `unique_keys` labels (removing today's silent collapse of duplicate keys). +3. The positional branch uses plain numeric segments (`i.to_string()`) only — the old opportunistic mixing of key values into positional paths is removed. + +- [ ] **Step 1: Write the failing tests** + +`src/runtime/tests/diff_unique_keys.rs` — reuse the exact `Fixture`/`diff2`/`only` harness from `diff_opaque.rs` (copy it; integration tests don't share modules). Data builders: + +```rust +fn e() -> JsonValue { + json!({"phoneNumber": "09/241.25.00", "hasNumberFunction": "Emergency_Number"}) +} +fn n() -> JsonValue { + json!({"phoneNumber": "09/241.25.03", "hasNumberFunction": "Non_Urgent_Communication"}) +} +fn o() -> JsonValue { + json!({"phoneNumber": "09/241.25.10", "hasNumberFunction": "Operator"}) +} +fn phones(items: Vec) -> JsonValue { + json!({"name": "svc", "hasPhoneNumber": items}) +} +``` + +Tests: + +```rust +#[test] +fn unique_key_matching_targets_field_edits_by_key() { + let f = fixture(); + let mut n2 = n(); + n2["phoneNumber"] = json!("09/241.25.99"); + let deltas = diff2(&f, phones(vec![e(), n()]), phones(vec![e(), n2])); + let delta = only(&deltas); + assert_eq!( + delta.path, + vec![ + "hasPhoneNumber".to_string(), + "Non_Urgent_Communication".to_string(), + "phoneNumber".to_string() + ] + ); + assert_eq!(delta.op, DeltaOp::Update); +} + +#[test] +fn unique_key_matching_ignores_reorder() { + let f = fixture(); + let deltas = diff2(&f, phones(vec![e(), n(), o()]), phones(vec![o(), n(), e()])); + assert!(deltas.is_empty(), "reorder must be invisible: {deltas:#?}"); +} + +#[test] +fn unique_key_remove_and_add_are_key_addressed() { + let f = fixture(); + let deltas = diff2(&f, phones(vec![e(), n()]), phones(vec![n()])); + let delta = only(&deltas); + assert_eq!(delta.op, DeltaOp::Remove); + assert_eq!( + delta.path, + vec!["hasPhoneNumber".to_string(), "Emergency_Number".to_string()] + ); + assert_eq!(delta.old, Some(e())); + + let deltas = diff2(&f, phones(vec![e(), n()]), phones(vec![e(), n(), o()])); + let delta = only(&deltas); + assert_eq!(delta.op, DeltaOp::Add); + assert_eq!( + delta.path, + vec!["hasPhoneNumber".to_string(), "Operator".to_string()] + ); + assert_eq!(delta.new, Some(o())); +} + +#[test] +fn changing_the_key_slot_is_remove_plus_add() { + let f = fixture(); + let mut moved = e(); + moved["hasNumberFunction"] = json!("Operator"); + let deltas = diff2(&f, phones(vec![e(), n()]), phones(vec![moved.clone(), n()])); + assert_eq!(deltas.len(), 2, "{deltas:#?}"); + assert!(deltas + .iter() + .any(|d| d.op == DeltaOp::Remove && d.old.as_ref() == Some(&e()))); + assert!(deltas + .iter() + .any(|d| d.op == DeltaOp::Add && d.new.as_ref() == Some(&moved))); +} + +#[test] +fn duplicate_unique_key_data_falls_back_to_positional() { + let f = fixture(); + // two Emergency numbers: violates the class claim; data must keep + // today's positional behaviour, with numeric path segments + let e2 = json!({"phoneNumber": "09/000.00.00", "hasNumberFunction": "Emergency_Number"}); + let mut e2_edit = e2.clone(); + e2_edit["phoneNumber"] = json!("09/111.11.11"); + let deltas = diff2( + &f, + phones(vec![e(), e2]), + phones(vec![e(), e2_edit]), + ); + let delta = only(&deltas); + assert_eq!( + delta.path, + vec![ + "hasPhoneNumber".to_string(), + "1".to_string(), + "phoneNumber".to_string() + ], + "positional fallback must use numeric segments, never the duplicate label" + ); +} + +#[test] +fn duplicate_key_data_falls_back_to_positional_not_collapse() { + let f = fixture(); + // Label declares `lang` as key; a list that repeats the key must not be + // silently collapsed by keyed matching — uniform guard, positional fallback. + let before = json!({"name": "svc", "labelList": [ + {"lang": "nl", "text": "a"}, {"lang": "nl", "text": "b"}]}); + let after = json!({"name": "svc", "labelList": [ + {"lang": "nl", "text": "a"}, {"lang": "nl", "text": "B"}]}); + let deltas = diff2(&f, before, after); + let delta = only(&deltas); + assert_eq!( + delta.path, + vec!["labelList".to_string(), "1".to_string(), "text".to_string()], + "duplicate key labels must fall back to plain numeric segments" + ); +} + +#[test] +fn undeclared_class_keeps_positional_cascade() { + let f = fixture(); + // PlainPhoneNumber has no unique_keys: removal still cascades as today + let before = json!({"name": "svc", "plainPhoneNumber": [e(), n(), o()]}); + let after = json!({"name": "svc", "plainPhoneNumber": [n(), o()]}); + let deltas = diff2(&f, before, after); + assert_eq!(deltas.len(), 3, "{deltas:#?}"); +} + +#[test] +fn composite_unique_key_uses_json_array_segment() { + let f = fixture(); + let a = json!({"kind": "Emergency", "phone": "02/111.11.11", "note": "old"}); + let mut a2 = a.clone(); + a2["note"] = json!("new"); + let b = json!({"kind": "Operator", "phone": "02/333.33.33"}); + let before = json!({"name": "svc", "contacts": [a, b.clone()]}); + let after = json!({"name": "svc", "contacts": [a2, b]}); + let deltas = diff2(&f, before, after); + let delta = only(&deltas); + assert_eq!( + delta.path, + vec![ + "contacts".to_string(), + r#"["Emergency","02/111.11.11"]"#.to_string(), + "note".to_string() + ] + ); +} + +#[test] +fn inherited_unique_keys_drive_matching() { + let f = fixture(); + // EmergencyPhoneNumber inherits one_number_per_function via is_a + let x = json!({"phoneNumber": "1", "hasNumberFunction": "Emergency_Number", "note": "a"}); + let mut x2 = x.clone(); + x2["note"] = json!("b"); + let y = json!({"phoneNumber": "2", "hasNumberFunction": "Operator", "note": "c"}); + let before = json!({"name": "svc", "escalation": [x, y.clone()]}); + let after = json!({"name": "svc", "escalation": [x2, y]}); + let deltas = diff2(&f, before, after); + let delta = only(&deltas); + assert_eq!( + delta.path, + vec![ + "escalation".to_string(), + "Emergency_Number".to_string(), + "note".to_string() + ] + ); +} +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `cargo test -p linkml_runtime --test diff_unique_keys` +Expected: the matching tests FAIL (positional deltas today); `duplicate_key_data_falls_back_to_positional_not_collapse` also FAILS today (current keyed matching silently collapses the duplicates — the defect the uniform guard removes). `duplicate_unique_key_data_falls_back_to_positional` and `undeclared_class_keeps_positional_cascade` PASS already and must stay green throughout. + +- [ ] **Step 3: Implement** + +In `src/runtime/src/diff.rs`, replace the inline `label` closure of the `(List, List)` arm (lines 212–226) with module-level helpers, then rework the arm: + +```rust +pub(crate) fn scalar_slot_string( + values: &std::collections::HashMap, + slot_name: &str, +) -> Option { + if let Some(LinkMLInstance::Scalar { value, .. }) = values.get(slot_name) { + return match value { + JsonValue::String(s) => Some(s.clone()), + other => Some(other.to_string()), + }; + } + None +} + +/// The key/identifier value identifying `v` among its list siblings, if any. +pub(crate) fn element_key_label(v: &LinkMLInstance) -> Option { + if let LinkMLInstance::Object { values, class, .. } = v { + let id_slot = class.key_or_identifier_slot()?; + return scalar_slot_string(values, &id_slot.name); + } + None +} + +/// A matching label derived from the range class's merged `unique_keys`. +/// +/// The name-sorted first entry with a non-empty slot list is the matching +/// identity (declaration order is not preserved by the metamodel, and diff +/// paths must be stable). Single-slot keys use the bare scalar value as the +/// label and path segment; composite keys use the JSON array encoding of the +/// values in `unique_key_slots` order (`["Emergency","02/111.11.11"]`) — +/// unambiguous, parseable, and displayable. +pub(crate) fn element_unique_key_label(v: &LinkMLInstance) -> Option { + if let LinkMLInstance::Object { values, class, .. } = v { + let uks = class.unique_keys(); + let (_, uk) = uks.iter().find(|(_, uk)| !uk.unique_key_slots.is_empty())?; + let parts: Option> = uk + .unique_key_slots + .iter() + .map(|s| scalar_slot_string(values, s)) + .collect(); + let mut parts = parts?; + return Some(if parts.len() == 1 { + parts.remove(0) + } else { + serde_json::to_string(&parts).expect("Vec serializes") + }); + } + None +} + +/// Identity for keyed list matching: a key/identifier slot outranks a +/// `unique_keys` claim. +pub(crate) fn element_identity_label(v: &LinkMLInstance) -> Option { + element_key_label(v).or_else(|| element_unique_key_label(v)) +} + +fn labels_are_unique(elements: &[LinkMLInstance], label: F) -> bool +where + F: Fn(&LinkMLInstance) -> Option, +{ + let mut seen = std::collections::HashSet::new(); + elements.iter().filter_map(label).all(|l| seen.insert(l)) +} +``` + +The `(List, List)` arm becomes: + +```rust +(LinkMLInstance::List { values: sl, .. }, LinkMLInstance::List { values: tl, .. }) => { + let identity = |v: &LinkMLInstance| -> Option { element_identity_label(v) }; + // Uniform rule (spec, Non-goal section): keyed matching iff every element + // on both sides carries an identity label and the labels are unique + // within each side. Duplicate labels (a list repeating a key, or data + // violating a unique_keys claim) fall back to positional — matching + // duplicates by label would silently collapse elements. + let keyed = sl.iter().all(|v| identity(v).is_some()) + && tl.iter().all(|v| identity(v).is_some()) + && labels_are_unique(sl, &identity) + && labels_are_unique(tl, &identity); + if keyed { + // ... identical to the current keyed block (lines 233-266), + // with every `label(..)` call replaced by `identity(..)` ... + } else { + // ... the current positional block (lines 267-296), with the + // opportunistic label chain REPLACED by plain numeric segments: + // every path segment is `i.to_string()`. + } +} +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `cargo test -p linkml_runtime --test diff_unique_keys` +Expected: PASS (all 9). + +- [ ] **Step 5: Full runtime suite — the compatibility gate** + +Run: `cargo test -p linkml_runtime && cargo test -p schemaview` +Expected: mostly PASS. Existing tests that assert the two deliberately removed behaviours — opportunistic key labels inside positional paths, or keyed matching of lists with duplicate labels — may fail; update each such test to the uniform rule and list every updated test in your report with its old assertion and why it changed. Any other failure means the implementation drifted — fix the implementation, not the test. + +- [ ] **Step 6: Commit** + +```bash +git add src/runtime/src/diff.rs src/runtime/tests/diff_unique_keys.rs +git commit -m "feat(runtime): match inlined list elements by unique_keys-derived identity" +``` + +--- + +### Task 5: `unique_keys` labels on the patch side + +**Files:** +- Modify: `src/runtime/src/diff.rs` (`resolve_list_index`, lines 530–558) +- Test: `src/runtime/tests/diff_unique_keys.rs` (extend) + +**Interfaces:** +- Consumes: `element_identity_label` (Task 4). +- Produces: patch behaviour only. Note the deliberate break (Global Constraints): duplicate labels in the current list now refuse (`Ok(false)` → `trace.failed`) instead of first-match; this applies to key/identifier labels exactly as to `unique_keys` labels. + +- [ ] **Step 1: Write the failing tests** + +Append to `diff_unique_keys.rs`: + +```rust +#[test] +fn patch_locates_element_by_unique_key_under_drift() { + let f = fixture(); + // producer saw [E, N]; golden drifted to [N, E, O] + let golden = f.load(phones(vec![n(), e(), o()])); + let delta = Delta { + path: vec![ + "hasPhoneNumber".to_string(), + "Emergency_Number".to_string(), + "phoneNumber".to_string(), + ], + op: DeltaOp::Update, + old: Some(json!("09/241.25.00")), + new: Some(json!("09/999.99.99")), + }; + let (patched, trace) = patch(&golden, &[delta], PatchOptions::default()).unwrap(); + assert!(trace.failed.is_empty(), "{:?}", trace.failed); + let mut e2 = e(); + e2["phoneNumber"] = json!("09/999.99.99"); + assert!( + patched.equals(&f.load(phones(vec![n(), e2, o()])), true), + "the edit must land on E wherever it sits: {}", + patched.to_json() + ); +} + +#[test] +fn patch_reports_ambiguous_unique_key_instead_of_guessing() { + let f = fixture(); + // golden drifted into two Emergency elements: locating "the" one is a guess + let e2 = json!({"phoneNumber": "09/000.00.00", "hasNumberFunction": "Emergency_Number"}); + let golden = f.load(phones(vec![e(), e2])); + let delta = Delta { + path: vec![ + "hasPhoneNumber".to_string(), + "Emergency_Number".to_string(), + "phoneNumber".to_string(), + ], + op: DeltaOp::Update, + old: Some(json!("09/241.25.00")), + new: Some(json!("09/999.99.99")), + }; + let (patched, trace) = patch(&golden, &[delta.clone()], PatchOptions::default()).unwrap(); + assert_eq!(trace.failed, vec![delta.path.clone()]); + assert!(patched.equals(&golden, true), "nothing may change"); +} + +#[test] +fn patch_refuses_positional_segment_into_identity_addressed_list() { + let f = fixture(); + let golden = f.load(phones(vec![e(), n()])); + // A stale positional patch aimed at a list whose elements all carry + // unique identity labels: applying "index 0" would be a guess, and for + // numeric-valued identity labels it would silently hit the wrong element. + let delta = Delta { + path: vec!["hasPhoneNumber".to_string(), "0".to_string()], + op: DeltaOp::Remove, + old: Some(e()), + new: None, + }; + let (patched, trace) = + patch(&golden, std::slice::from_ref(&delta), PatchOptions::default()).unwrap(); + assert_eq!(trace.failed, vec![delta.path.clone()]); + assert!(patched.equals(&golden, true), "nothing may be removed"); +} + +#[test] +fn patch_refuses_ambiguous_duplicate_key_labels() { + let f = fixture(); + // Duplicate key/identifier labels refuse exactly like duplicate + // unique_keys labels — the uniform rule on the patch side. + let golden = f.load(json!({"name": "svc", "labelList": [ + {"lang": "nl", "text": "a"}, {"lang": "nl", "text": "b"}]})); + let delta = Delta { + path: vec!["labelList".to_string(), "nl".to_string(), "text".to_string()], + op: DeltaOp::Update, + old: Some(json!("a")), + new: Some(json!("z")), + }; + let (patched, trace) = patch(&golden, &[delta.clone()], PatchOptions::default()).unwrap(); + assert_eq!(trace.failed, vec![delta.path.clone()]); + assert!(patched.equals(&golden, true), "nothing may change"); +} + +#[test] +fn unique_key_deltas_round_trip_through_patch() { + let f = fixture(); + let mut n2 = n(); + n2["phoneNumber"] = json!("09/241.25.99"); + for (before, after) in [ + (phones(vec![e(), n()]), phones(vec![e(), n2])), // field edit + (phones(vec![e(), n()]), phones(vec![n()])), // remove + (phones(vec![e(), n()]), phones(vec![e(), n(), o()])), // add + ] { + let deltas = diff2(&f, before.clone(), after.clone()); + let (patched, trace) = patch(&f.load(before), &deltas, PatchOptions::default()).unwrap(); + assert!(trace.failed.is_empty(), "{:?}", trace.failed); + assert!(patched.equals(&f.load(after), true), "{}", patched.to_json()); + } +} +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `cargo test -p linkml_runtime --test diff_unique_keys` +Expected: the two locate tests FAIL (segment `Emergency_Number` resolves to no index today); the remove/add legs of the round-trip may also fail. + +- [ ] **Step 3: Implement** + +Rewrite `resolve_list_index` (diff.rs:530) as one unified resolver that mirrors how diff produces segments. A list whose elements all carry unique identity labels ("keyed-shaped") is addressed by label ONLY — diff emits label segments for exactly these lists, and resolving a numeric segment positionally against one would silently write the wrong element whenever an identity label is itself numeric (integer-ranged key slots). Positional lists keep numeric-first, with a single unambiguous label hit as drift tolerance. The old key/identifier `find_map` block is replaced entirely: + +```rust +fn resolve_list_index(values: &[LinkMLInstance], key: &str) -> Option { + // A list whose elements all carry unique identity labels is addressed by + // label ONLY: diff emits label segments for exactly these lists, and a + // numeric segment aimed at one (a stale positional patch) would be a + // guess — report, never guess. This also keeps integer-valued identity + // labels (e.g. a year as unique key) unambiguous: they resolve as + // labels, never as positions. + let labels: Vec> = values.iter().map(element_identity_label).collect(); + let keyed_shaped = !values.is_empty() && labels.iter().all(|l| l.is_some()) && { + let mut seen = std::collections::HashSet::new(); + labels.iter().flatten().all(|l| seen.insert(l.clone())) + }; + if keyed_shaped { + return labels.iter().position(|l| l.as_deref() == Some(key)); + } + // Positional list: numeric index first (the segments diff produces for + // these lists), then a single unambiguous label hit for drift tolerance. + if let Ok(idx) = key.parse::() { + if idx < values.len() { + return Some(idx); + } + } + let mut hit: Option = None; + for (i, l) in labels.iter().enumerate() { + if l.as_deref() == Some(key) { + if hit.is_some() { + return None; + } + hit = Some(i); + } + } + hit +} +``` + +(An `Add` whose label segment matches nothing returns `None`, which takes the existing append path in `apply_list_leaf_delta` — correct for both list shapes.) + +Note on `Add` deltas: an `Add` whose unique-key segment resolves to no element takes the existing `idx_opt = None` append path in `apply_list_leaf_delta` (line 687) — that is correct and needs no change. + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `cargo test -p linkml_runtime --test diff_unique_keys` +Expected: PASS (all 13). + +- [ ] **Step 5: Full runtime suite — the compatibility gate** + +Run: `cargo test -p linkml_runtime` +Expected: mostly PASS. Existing tests that assert the removed first-match-on-duplicate-labels patch behaviour may fail; update each such test to the uniform refuse-ambiguity rule and list every updated test in your report with its old assertion and why it changed. Any other failure means the implementation drifted — fix the implementation, not the test. + +- [ ] **Step 6: Commit** + +```bash +git add src/runtime/src/diff.rs src/runtime/tests/diff_unique_keys.rs +git commit -m "feat(runtime): patch resolves identity-label path segments, refuses ambiguity" +``` + +--- + +### Task 6: the opt-in identity linter + +Two opt-in entry points in a new module. Neither is called from `validate_issues` or any load path — that is the spec's non-goal. + +**Files:** +- Create: `src/runtime/src/identity_lint.rs` +- Modify: `src/runtime/src/lib.rs` (add `pub mod identity_lint;` next to `pub mod diff;` at line 28; add two variants to `ValidationProblemType` at lines 150–157; re-export `identity_lint::{lint_element_identity, lint_instance_identity}`) +- Create: `src/runtime/tests/identity_lint.rs` + +**Interfaces:** +- Consumes: `ClassView::unique_keys()` (Task 1), `slot_is_opaque`/`OPAQUE_ANNOTATION` (Task 2), `element_identity_label` (Task 4), `SlotView::determine_slot_container_mode()` / `determine_slot_inline_mode()` / `get_range_class()` / `is_range_scalar()` (slotview.rs:500–564), `ValidationResultSink` (`push_warning`, `into_vec`), `SchemaView::{get_class_ids, get_class, converter}`. +- Produces: + - `ValidationProblemType::AmbiguousElementIdentity` and `ValidationProblemType::DuplicateElementIdentity` (new enum variants — the compiler will flag every `match` that needs extending, including the Python binding's problem-type stringification; extend them all). + - `pub fn lint_element_identity(sv: &SchemaView) -> Vec` — schema-level. + - `pub fn lint_instance_identity(value: &LinkMLInstance) -> Vec` — data-level. + +**Lint rules (schema-level), per class × slot:** +- Only `SlotContainerMode::List` slots are candidates (`Mapping` is keyed by construction, `SingleValue` has no elements). +- Skip `SlotInlineMode::Reference` (elements are references, not inlined — out of the spec's scope). +- Skip slots carrying `diff.linkml.io/ignore` or `diff.linkml.io/opaque` (identity declared: nowhere). +- Object range: OK if the range class has `key_or_identifier_slot()` or a non-empty `unique_keys()`. Otherwise warn. +- Scalar range (`is_range_scalar`): always warn (positional identity; the only declaration available is opaque). +- Severity: **Warning**, never error. `subject` = `vec![class_name, slot_name]`. + +**Lint rules (data-level), per `List` node in the instance tree:** +- Elements whose class declares a key/identifier or `unique_keys` and whose `element_identity_label` repeats within the container → one `DuplicateElementIdentity` warning per repeated label, at the container's path. +- Deliberately does **not** consult the opaque annotation (layering: `diff.linkml.io/*` never suppresses a schema-constraint finding). The remodel in the spec is what makes rings silent — their `Vertex` class declares nothing. +- Elements with no declared identity (no key, no unique_keys) are never flagged — repeated content is data, not an error. + +- [ ] **Step 1: Write the failing tests** + +`src/runtime/tests/identity_lint.rs` (same `fixture()` harness copied from `diff_opaque.rs`, plus these): + +```rust +use linkml_runtime::{lint_element_identity, lint_instance_identity, ValidationProblemType}; + +#[test] +fn schema_lint_flags_exactly_the_undeclared_positional_slots() { + let f = fixture(); + let warnings = lint_element_identity(&f.sv); + let mut flagged: Vec<(String, String)> = warnings + .iter() + .map(|w| (w.subject[0].clone(), w.subject[1].clone())) + .collect(); + flagged.sort(); + assert_eq!( + flagged, + vec![ + ("Service".to_string(), "plainPhoneNumber".to_string()), + ("Service".to_string(), "tags".to_string()), + ], + "everything else declares its identity source: {warnings:#?}" + ); + for w in &warnings { + assert_eq!(w.problem_type, ValidationProblemType::AmbiguousElementIdentity); + assert!(!w.severity.is_error(), "the linter warns, never errors"); + assert!( + w.detail.contains("unique_keys") && w.detail.contains("diff.linkml.io/opaque"), + "the warning must name the author's options: {}", + w.detail + ); + } +} + +#[test] +fn data_lint_flags_duplicate_declared_identities() { + let f = fixture(); + let dup = json!({"phoneNumber": "09/000.00.00", "hasNumberFunction": "Emergency_Number"}); + let inst = f.load(phones(vec![e(), dup])); + let warnings = lint_instance_identity(&inst); + assert_eq!(warnings.len(), 1, "{warnings:#?}"); + assert_eq!(warnings[0].problem_type, ValidationProblemType::DuplicateElementIdentity); + assert_eq!(warnings[0].subject, vec!["hasPhoneNumber".to_string()]); + assert!(!warnings[0].severity.is_error()); +} + +#[test] +fn data_lint_is_silent_on_clean_and_undeclared_data() { + let f = fixture(); + // unique phone functions, repeated scalar tags, repeated identity-less vertices + let inst = f.load(json!({ + "name": "svc", + "hasPhoneNumber": [e(), n()], + "tags": ["a", "a"], + "outline": [{"x": 1.0, "y": 2.0}, {"x": 1.0, "y": 2.0}] + })); + let warnings = lint_instance_identity(&inst); + assert!(warnings.is_empty(), "{warnings:#?}"); +} + +#[test] +fn data_lint_does_not_let_opaque_suppress_a_schema_constraint() { + let f = fixture(); + // archivedContacts is opaque, but Contact declares unique_keys: duplicates + // still violate the class's claim. diff vocabulary never silences schema truth. + let c = json!({"kind": "Emergency", "phone": "02/111.11.11"}); + let inst = f.load(json!({"name": "svc", "archivedContacts": [c.clone(), c]})); + let warnings = lint_instance_identity(&inst); + assert_eq!(warnings.len(), 1, "{warnings:#?}"); + assert_eq!(warnings[0].subject, vec!["archivedContacts".to_string()]); +} +``` + +(Reuse `e()`, `n()`, `phones()` builders from `diff_unique_keys.rs` — copy them in.) + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `cargo test -p linkml_runtime --test identity_lint` +Expected: FAIL to compile — module and variants don't exist. + +- [ ] **Step 3: Implement the module** + +`src/runtime/src/identity_lint.rs`: + +```rust +//! Opt-in element-identity linter. +//! +//! Answers, per multivalued inlined slot: **where does element identity come +//! from?** A key (or identifier) on the element class, a composed key +//! declared with `unique_keys`, or the `diff.linkml.io/opaque` annotation +//! (nowhere — the slot's value is replaced as a whole). A slot with none of +//! these produces positional deltas, which are ambiguous when several sources +//! produce deltas for the same object concurrently. +//! +//! Neither entry point is wired into loading or [`crate::validate_issues`]: +//! projects that do not opt in keep today's inferred semantics. +//! +//! When a slot is flagged, the data-model author has four options (worked +//! examples in `docs/superpowers/specs/2026-08-17-inlined-multivalued-element-identity-design.md`): +//! 1. declare a `key`/`identifier` slot on the element class; +//! 2. declare the identity as `unique_keys` (composed keys supported); +//! 3. annotate the slot `diff.linkml.io/opaque` — replace the value as a whole; +//! 4. remodel, when one class would need two identity answers. + +use crate::diff::{element_identity_label, slot_is_opaque, OPAQUE_ANNOTATION}; +use crate::{ + LinkMLInstance, ValidationProblemType, ValidationResult, ValidationResultSink, +}; +use linkml_schemaview::identifier::Identifier; +use linkml_schemaview::schemaview::SchemaView; +use std::collections::HashMap; + +/// Schema-level lint: warn for every multivalued inlined slot whose element +/// identity comes from nowhere. Warnings only — the schema stays usable. +pub fn lint_element_identity(sv: &SchemaView) -> Vec { + use linkml_schemaview::slotview::{SlotContainerMode, SlotInlineMode}; + let mut sink = ValidationResultSink::default(); + let conv = sv.converter(); + let mut class_ids = sv.get_class_ids(); + class_ids.sort(); + for class_id in class_ids { + let Ok(Some(class)) = sv.get_class(&Identifier::new(&class_id), &conv) else { + continue; + }; + for slot in class.slots() { + if slot.determine_slot_container_mode() != SlotContainerMode::List { + continue; + } + if slot.determine_slot_inline_mode() == SlotInlineMode::Reference { + continue; // elements are references, not inlined + } + if slot_is_opaque(slot) { + continue; // identity declared: nowhere, replace as a whole + } + if let Some(rc) = slot.get_range_class() { + if rc.key_or_identifier_slot().is_some() || !rc.unique_keys().is_empty() { + continue; + } + } + sink.push_warning( + ValidationProblemType::AmbiguousElementIdentity, + vec![class.name().to_string(), slot.name.clone()], + format!( + "elements of '{}.{}' have no declared identity: deltas are \ + positional and ambiguous under multi-sourced operation. \ + Declare a key/identifier or unique_keys on the element \ + class, annotate the slot with {} to replace the value as \ + a whole, or remodel.", + class.name(), + slot.name, + OPAQUE_ANNOTATION + ), + ); + } + } + sink.into_vec() +} + +/// Data-level lint: warn for every list container whose elements repeat a +/// declared identity (key/identifier or unique_keys value). +/// +/// Deliberately does NOT consult `diff.linkml.io/opaque`: a schema constraint +/// is class-level truth, and diff vocabulary never suppresses it. +pub fn lint_instance_identity(value: &LinkMLInstance) -> Vec { + let mut sink = ValidationResultSink::default(); + let mut path = Vec::new(); + walk(value, &mut path, &mut sink); + sink.into_vec() +} + +fn walk(v: &LinkMLInstance, path: &mut Vec, sink: &mut ValidationResultSink) { + match v { + LinkMLInstance::List { values, .. } => { + check_duplicates(values, path, sink); + for (i, child) in values.iter().enumerate() { + path.push(i.to_string()); + walk(child, path, sink); + path.pop(); + } + } + LinkMLInstance::Object { values, .. } | LinkMLInstance::Mapping { values, .. } => { + for (k, child) in values { + path.push(k.clone()); + walk(child, path, sink); + path.pop(); + } + } + LinkMLInstance::Scalar { .. } | LinkMLInstance::Null { .. } => {} + } +} + +fn check_duplicates( + values: &[LinkMLInstance], + path: &[String], + sink: &mut ValidationResultSink, +) { + let mut seen: HashMap = HashMap::new(); + for v in values { + if let Some(label) = element_identity_label(v) { + *seen.entry(label).or_insert(0) += 1; + } + } + let mut dups: Vec<(String, usize)> = + seen.into_iter().filter(|(_, n)| *n > 1).collect(); + dups.sort(); + for (label, n) in dups { + sink.push_warning( + ValidationProblemType::DuplicateElementIdentity, + path.to_vec(), + format!( + "{n} elements share the declared identity '{label}'; deltas \ + addressing it are ambiguous" + ), + ); + } +} +``` + +In `src/runtime/src/lib.rs`: +- add the two `ValidationProblemType` variants, +- `pub mod identity_lint;`, +- `pub use identity_lint::{lint_element_identity, lint_instance_identity};`. + +Then chase compile errors: every exhaustive `match` on `ValidationProblemType` (runtime, python crate) needs the two new arms — stringify them as `"ambiguous_element_identity"` / `"duplicate_element_identity"` following the existing naming style found at those match sites. + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `cargo test -p linkml_runtime --test identity_lint` +Expected: PASS (all 4). + +- [ ] **Step 5: Workspace build + runtime suite** + +Run: `cargo build --workspace && cargo test -p linkml_runtime` +Expected: PASS (workspace build catches the python-crate match arms). + +- [ ] **Step 6: Commit** + +```bash +git add src/runtime/src/identity_lint.rs src/runtime/src/lib.rs src/runtime/tests/identity_lint.rs src/python/src/lib.rs +git commit -m "feat(runtime): opt-in element-identity linter (schema + instance)" +``` + +--- + +### Task 7: Python bindings + +The library's real consumer (consolidator-server) is Python. Expose the two lint functions; `diff`/`patch` signatures are unchanged, so no other binding moves. + +**Files:** +- Modify: `src/python/src/lib.rs` (two new `#[pyfunction]`s + registration in the `_native` module at lines 739–756) +- Modify: `src/python/python/linkml_runtime_rust/_native.pyi` (two stubs) + +**Interfaces:** +- Consumes: `linkml_runtime::{lint_element_identity, lint_instance_identity}` (Task 6), the existing `validation_results_to_py` helper (python lib.rs:1070) and `PyValidationResult`. +- Produces: Python functions `lint_element_identity(schema_view) -> list[ValidationResult]` and `lint_instance_identity(instance) -> list[ValidationResult]`. + +- [ ] **Step 1: Implement the bindings** + +Follow `py_diff` (registered at python lib.rs:745) for how the wrapper types expose their inner `SchemaView`/`LinkMLInstance` — use the same accessor pattern, then: + +```rust +#[cfg_attr(feature = "stubgen", gen_stub_pyfunction)] +#[pyfunction] +#[pyo3(name = "lint_element_identity")] +fn py_lint_element_identity( + py: Python<'_>, + schema_view: &PySchemaView, +) -> PyResult>> { + validation_results_to_py( + py, + linkml_runtime::lint_element_identity(schema_view_inner(schema_view)), + ) +} + +#[cfg_attr(feature = "stubgen", gen_stub_pyfunction)] +#[pyfunction] +#[pyo3(name = "lint_instance_identity")] +fn py_lint_instance_identity( + py: Python<'_>, + instance: &PyLinkMLInstance, +) -> PyResult>> { + validation_results_to_py( + py, + linkml_runtime::lint_instance_identity(instance_inner(instance)), + ) +} +``` + +(`schema_view_inner`/`instance_inner` stand for however `py_diff`/`py_patch` reach the wrapped Rust values — reuse that exact mechanism, whatever it is named; also mirror the `gen_stub_pyfunction` usage of neighbouring functions, including whether they gate it behind the `stubgen` feature.) Register both with `m.add_function(wrap_pyfunction!(...))?;` next to `py_diff`. + +- [ ] **Step 2: Add the stubs** + +In `_native.pyi`, next to the `diff`/`patch` stubs: + +```python +def lint_element_identity(schema_view: SchemaView) -> list[ValidationResult]: ... +def lint_instance_identity(instance: LinkMLInstance) -> list[ValidationResult]: ... +``` + +(Match the actual parameter/class names used by the neighbouring stubs in that file.) + +- [ ] **Step 3: Verify it compiles** + +Run: `cargo check -p linkml_runtime_python` +Expected: clean. If the repo has a Python test suite under `src/python/`, also run it the way its README/CI does; otherwise compilation plus stub review is the gate here. + +- [ ] **Step 4: Commit** + +```bash +git add src/python/src/lib.rs src/python/python/linkml_runtime_rust/_native.pyi +git commit -m "feat(python): expose element-identity lint functions" +``` + +--- + +### Task 8: CLI flag on `linkml-schema-validate` + +**Files:** +- Modify: `src/tools/src/bin/linkml_schema_validate.rs` (new `--lint-identity` flag) +- Modify: `src/tools/Cargo.toml` only if the tools crate does not already depend on `linkml_runtime` (the diff/patch bins suggest it does). + +**Interfaces:** +- Consumes: `linkml_runtime::lint_element_identity` (Task 6). +- Produces: `linkml-schema-validate --lint-identity` prints one line per warning and (like warnings elsewhere in the tool) does not change the exit code. + +- [ ] **Step 1: Implement** + +Add to the `Args` struct: + +```rust + /// Opt-in: warn for multivalued inlined slots whose element identity + /// comes from nowhere (positional, ambiguous deltas in multi-sourced use). + #[arg(long, default_value_t = false)] + lint_identity: bool, +``` + +After the existing validation logic completes (both output formats), run and print: + +```rust + if args.lint_identity { + for w in linkml_runtime::lint_element_identity(&sv) { + println!("warning[{}]: {}", w.subject.join("."), w.detail); + } + } +``` + +(Adapt the printing to the tool's existing `OutputFormat` handling: in `Json` mode, emit the warnings as a JSON array the same way existing results are serialized there — follow the surrounding code.) + +- [ ] **Step 2: Verify manually against the fixture** + +Run: `cargo run -p linkml_tools --bin linkml-schema-validate -- src/runtime/tests/data/identity.yaml --lint-identity` +Expected: exactly two warning lines — `Service.plainPhoneNumber` and `Service.tags` — plus whatever the normal validation prints. Then run without the flag and confirm no identity warnings appear (opt-in). + +- [ ] **Step 3: Build check** + +Run: `cargo build -p linkml_tools` +Expected: clean. + +- [ ] **Step 4: Commit** + +```bash +git add src/tools/src/bin/linkml_schema_validate.rs src/tools/Cargo.toml +git commit -m "feat(tools): --lint-identity flag on linkml-schema-validate" +``` + +--- + +### Task 9: docs polish + final verification + +**Files:** +- Modify: `src/runtime/src/diff.rs` (rustdoc only: extend the `Delta` doc comment's path-segment sentence at lines 35–50 to mention unique_keys-derived segments and the composite JSON-array encoding; extend the `diff` doc comment at lines 103–111 with one paragraph on opaque slots) +- Modify: `src/runtime/src/identity_lint.rs` (confirm the module doc lists the four author options and links the spec — it is the authoring-guide surface the spec promises) + +**Interfaces:** none — documentation and the final gate. + +- [ ] **Step 1: Write the rustdoc additions** + +On `Delta` (after the existing segment sentence, line 38–39): + +``` +/// For inlined lists whose element class declares `unique_keys`, the segment is +/// the unique-key value (single-slot keys), or the JSON array encoding of the +/// values in `unique_key_slots` order (composite keys), e.g. +/// `["Emergency","02/111.11.11"]`. +``` + +On `diff` (after the null/missing semantics list): + +``` +/// Slots annotated `diff.linkml.io/opaque` stop all recursion: any change at or +/// below the slot is described as a single whole-value `Update` at the slot +/// path. See [`OPAQUE_ANNOTATION`]. +``` + +- [ ] **Step 2: Full workspace gate** + +Run: + +```bash +cargo fmt --all +cargo clippy --workspace --all-targets -- -D warnings +cargo test --workspace +``` + +Expected: all green. Fix anything clippy raises in the new code only. + +- [ ] **Step 3: Spot-check the spec's headline scenarios end to end** + +Run: `cargo test -p linkml_runtime --test diff_unique_keys --test diff_opaque --test identity_lint` +Expected: PASS — this is the spec's Example 1 (phones via unique_keys), Example 2 (opaque ring + bare Vertex + polygon unique key), and the linter contract. + +- [ ] **Step 4: Commit** + +```bash +git add src/runtime/src/diff.rs src/runtime/src/identity_lint.rs +git commit -m "docs(runtime): document opaque annotation and unique_keys path segments" +``` + +--- + +# Addendum tasks (2026-08-19): designator and canonicalization hardening + +> Argued from the spec's "Addendum (2026-08-19)" section — its numbered rules are the binding requirements. Detailed empirical findings: `.superpowers/sdd/2026-08-18-inlined-multivalued-element-identity/spike-findings.md` (D1–D9). These tasks are specified rule-and-test-first rather than code-verbatim: each names its binding spec rule, its exact observable behaviour changes, and its test cases; implementers derive the code. **Every task's verification includes the differential harness gate (Task 10): re-run, and attribute every changed output line to a spec-addendum rule; unattributable changes fail the task.** + +### Task 10: downstream differential harness + baseline + +**Files:** Create scripts + baselines under `.superpowers/sdd/2026-08-18-inlined-multivalued-element-identity/harness/` (gitignored workspace — nothing committed to the repo). + +Build a script (bash or python, your choice) that, given a repo checkout, produces a deterministic text report over a fixed corpus, using the repo's CLIs (`linkml-validate`, `linkml-diff`, `linkml-patch`, `linkml-schema-validate --lint-identity`, `linkml-convert` where useful): +- Corpus A (real downstream): every committed instance JSON under /home/ejsyx/projects/consolidator-server/components/py (skip .worktrees), each validated against the asset360 schema root and its class where derivable (signal/goldenrecords/deltas test files; record per-file: validate output, load→convert normal form, self-diff = must be empty). +- Corpus B (in-repo): the runtime test fixtures' data files + meta.yaml; the asset360 schema lint output (already known: 27 warnings). +- Pairwise diffs: for the consolidator delta fixtures directory, diff meaningful before/after pairs if identifiable; otherwise diff each file against a jq-mutated variant (deterministic mutation) to exercise list matching. +- Output: one sorted, stable report file per corpus. Capture BASELINE at current HEAD into `harness/baseline/`. The gate for later tasks: `diff -u baseline/ current/` — reviewed line by line. + +**Verification:** run twice at HEAD — byte-identical reports (determinism). Report corpus size (file counts) and any files that fail to load at baseline (they are baseline facts, not defects). + +### Task 11: D3 — a type designator is never an element identity (spec rule 1) + +**Files:** Modify `src/runtime/src/diff.rs` (`element_key_label`); `src/runtime/src/identity_lint.rs` only if gating needs alignment; tests `src/runtime/tests/diff_unique_keys.rs` + `identity_lint.rs` + fixture(s). + +`element_key_label` returns `None` when the key/identifier slot's merged definition has `designates_type == Some(true)`. Behaviour deltas to pin with tests: (1) a class with a designator key AND `unique_keys` now matches by the unique_keys label (the D3 shadow case: reorder of `{x,y}`-keyed elements emits zero deltas); (2) a designator-keyed class WITHOUT unique_keys is positional (and: no double lint fire — the designator-key rule remains the only voice for it; check `slot_lacks_element_identity`'s key gate and keep exactly one warning per such slot); (3) polymorphic designator-keyed list (one element per subtype) is now positional — pin the new behaviour. Harness gate: expected attributable changes only in designator-keyed polymorphic/shadowed cases. + +### Task 12: D1 + D6 — identity compares meaning, not spelling (spec rule 2) + +**Files:** Modify `src/runtime/src/lib.rs` (boxing chokepoints: `parse_object_fixed_class`, `parse_object_value`, `build_mapping_entry_for_slot` — designator canonicalization after class selection, mirroring `coerce_scalar_to_range`), `src/runtime/src/diff.rs` (`scalar_slot_string` IRI expansion for uri/uriorcurie-descended ranges — hits all four resolve sites through the existing shared helpers; thread a Converter via the class's schemaview handle), tests + fixtures. + +Tests to pin: designator spelled as curie / full URI / implicit all load to ONE canonical stored value and `to_json` emits it; junk designator value (matches no accepted value) becomes a load-time validation warning + canonical fill (decide error-vs-warning by the loader-tolerance precedent: warning); curie-vs-expanded uri unique_keys values are ONE identity across diff, patch, navigate, instance lint (one test each); RDF-vs-JSON agreement (load the same content both ways where the harness corpus allows, or a focused turtle_import test). Harness gate: attributable changes = canonical designator spellings in convert output for files that spelled them differently; self-diffs stay empty. + +### Task 13: D2 — class change is whole-element replacement; patch never hard-errors (spec rules 3+4) + +**Files:** Modify `src/runtime/src/diff.rs` (object arm class comparison; apply paths' builder-error handling), tests. + +Tests: Bolt-vs-Nut same-key diff = ONE whole-element Update (both directions); patch of a hand-built bad delta (unbuildable value at a resolved location) → `Ok`, path in `trace.failed`, tree untouched, and OTHER deltas in the same batch still apply; existing suites' Err expectations updated only where they asserted the old contract (list each). Check `src/runtime/src/blame/` compiles and its semantics commentary still holds. Harness gate: no expected corpus changes (defensive: attribute any). + +### Task 14: D5 — the inlined-dict key is real data (spec rule 5) + +**Files:** Modify `src/runtime/src/lib.rs` (`build_mapping_entry_for_slot` + validation sink usage), tests. + +Tests: plain keyed class dict entry without payload key → key slot filled from dict key, NO MissingSlotValue error; payload key present and equal → clean; payload key disagreeing with dict key → validation WARNING naming both; designator-keyed dict whose key is not an accepted designator value → validation warning; designator-keyed dict key that IS accepted → designator canonicalized per Task 12 and consistent. Harness gate: expected attributable changes = disappearing MissingSlotValue errors and new divergence warnings on real dict data (the known asset360 LinearCoordinate divergence should surface as a warning — cite it in the report). + +### Task 15: lint extensions + docs (spec rule 6) + +**Files:** Modify `src/runtime/src/identity_lint.rs` (+ tests/fixtures), rustdoc. + +(1) Instance lint: "positional despite declared identity" warning when keyed-shaped fails due to MISSING labels while the range class declares key/unique_keys (D7; also gives half-labelled lists a voice). (2) Schema lint: multi-unique_keys check unions entries across `rc.get_descendants(true,false)`; additional warning when descendants resolve different load-bearing entries (D4d — the split-label-space case). (3) Schema lint: warn when two classes in one is_a hierarchy share a `class_uri` and the hierarchy carries a designator (D8, warn-only). (4) Rustdoc: D9 note (`slot_usage: designates_type: false` leaves an unfillable key) + module doc lists all rules. Each rule: RED/GREEN with its own fixture case; existing exact-set tests stay green or are extended deliberately (list changes). Harness gate: asset360 lint delta reported and attributed (new warnings expected; count them). + +### Task 16: hardening close-out + +Full gates (fmt, scoped clippy, `cargo test --workspace --exclude linkml_runtime_python`, `cargo check -p linkml_runtime_python`), final harness run with the complete attribution table (baseline → final), asset360 lint before/after summary, and one rustdoc/spec cross-check that every addendum rule is implemented or explicitly listed as out-of-scope. Docs-only fixes allowed; anything behavioural found here is reported, not fixed. diff --git a/docs/superpowers/specs/2026-08-17-inlined-multivalued-element-identity-design.md b/docs/superpowers/specs/2026-08-17-inlined-multivalued-element-identity-design.md new file mode 100644 index 0000000..49e8172 --- /dev/null +++ b/docs/superpowers/specs/2026-08-17-inlined-multivalued-element-identity-design.md @@ -0,0 +1,328 @@ +# Element Identity for Inlined Multivalued Slots — Design + +**Status:** Draft, awaiting user review before plan-writing. +**Date:** 2026-08-17 +**Branch:** `feat/inlined-multivalued-element-identity` +**Supersedes:** the abandoned shapes work on branch `feat/container-shapes-and-verify-old` (see "Recyclable material" at the end). + +## Goal + +Answer the question, for each inlined multivalued shape: **where does element identity come from?** + +The offered options are: + +- a **key** (or identifier) — a slot on the element class, +- a **composed key** (content) — declared with the existing LinkML meta `unique_keys`, +- **opaque** — nowhere; the value is replaced as a whole. + +"Where does identity for an element come from" should be easy to answer for a data model author. + +## Non-goal + +Positional index deltas remain the inferred semantics for slots that declare nothing — they are still valuable for projects not dealing with multiple sources producing deltas for the same object at the same time, and the opt-in linter is the only place that complains about them. + +One deliberate compatibility break ships with this design (a spike finding): **keyed matching becomes uniform**. A list is matched by element identity only when every element on both sides yields an identity label (key/identifier first — unless that key is the element class's type designator, which the addendum's rule 1 never accepts as element identity — else the class's `unique_keys`) and the labels are unique within each side; path segments are then the labels, IRI-expanded when the slot they came from ranges on `uri`/`uriorcurie` (addendum rule 2). In every other case matching is positional and path segments are plain numeric indices. This removes two behaviours of the old fallback: key values were opportunistically mixed into positional paths, and duplicate key values within a "keyed" list were silently collapsed by the matcher. Consumers see cleaner, uniform delta paths; `patch` keeps accepting both numeric and label segments, and reports a delta as failed instead of guessing when a label matches more than one element. + +## Proposed solution + +Inlined multivalued slots cause ambiguous deltas downstream. + +1. We add a **`diff.linkml.io/opaque` annotation**, with the meaning: **stop all recursion, replace the whole value.** +2. Together with **new support for the existing LinkML meta `unique_keys`**, allowing to declare composed keys. +3. An **opt-in linter** warns when your data model still allows for ambiguous deltas in multi-sourced operation. +4. We provide **examples explaining the nuanced options** a data model author has when the linter flags ambiguity (worked out below on the constituting examples). + +Consequence of this strict mode is that some LinkML schemas need to be reworked to be compliant, and more cases of patches (produced outside this diff lib) cannot be applied fully. + +## Problem statement + +When the element class of an inlined multivalued slot carries no key or identifier, the diff falls back to positional index paths (`src/runtime/src/diff.rs:267-296` — `for i in 0..max_len`, numeric path segments). A positional path is only meaningful against the exact list the producer saw. In multi-sourced operation the golden record has drifted by the time a patch arrives, and the index silently selects a different element — no error, wrong data. + +Both constituting examples are real, from the asset360 model at +`consolidator-server/components/py/asset360-model/asset360_model/schemas/asset360/repository/v1.0.0/`. + +### Example 1 — phone numbers: data loss and duplication under current inferred semantics + +`Service.hasPhoneNumber` is an inlined list of anonymous value objects (`tunnels.yaml:1954-1999`, locale annotations elided): + +```yaml + Service: + attributes: + hasPhoneNumber: + description: Phone number of the resource. + range: ServicePhoneNumber + multivalued: true + inlined_as_list: true + + ServicePhoneNumber: + description: >- + A phone number associated with a Service, classified by its function + (emergency, non-urgent, operator, etc.). Inlined into Service. + attributes: + phoneNumber: + range: string + hasNumberFunction: + range: NumberFunction + required: true +``` + +`ServicePhoneNumber` has no `identifier`, no `key`, and the schema declares no `unique_keys` anywhere. Elements have no stable identity at all — yet the model *does* have an identity rule in mind. It lives in hand-written SHACL (`constraints.shacl.ttl:546-612`), invisible to LinkML: + +```turtle +asset360:Service_OnePhoneNumberPerFunctionShape + a sh:NodeShape ; + sh:targetClass asset360:Service ; + sh:message "A service cannot have two phone numbers with the same function."@en ; + sh:property [ + sh:path asset360:hasPhoneNumber ; + sh:qualifiedValueShape [ sh:path asset360:hasNumberFunction ; sh:hasValue "Emergency_Number" ] ; + sh:qualifiedMaxCount 1 + ] ; + # ... one qualifiedMaxCount block per NumberFunction value ... +``` + +So `hasNumberFunction` *behaves* as a key — but LinkML sees an unkeyed list, and the diff sees positions. (The ingestion layer even carries an external identity per phone number — `source_id_key: new_uri` in `changeset-generator/.../ce_kwoa_a1552/service_phone_number.yaml` — which is dropped at the schema boundary.) + +**Data loss.** Two sources both derived their deltas from the same snapshot: + +```json +"hasPhoneNumber": [ + {"phoneNumber": "09/241.25.00", "hasNumberFunction": "Emergency_Number"}, + {"phoneNumber": "09/241.25.03", "hasNumberFunction": "Non_Urgent_Communication"} +] +``` + +- Source A corrects the non-urgent number: `Update hasPhoneNumber/1/phoneNumber` → `"09/241.25.99"`. +- Source B removes the emergency number. Positionally that is a cascade: `Update hasPhoneNumber/0/*` (the non-urgent content shifts into index 0, with the *stale* phone number `09/241.25.03`) plus `Remove hasPhoneNumber/1`. + +Apply A then B: B's cascade overwrites index 0 with the stale snapshot value — A's correction is silently reverted. Apply B then A: A's path `hasPhoneNumber/1` now points at nothing (or at whatever drifted in) — the edit is lost or lands on the wrong element. Either order corrupts; neither reports an error. + +**Duplication.** Two ingest sources independently discover the same new operator number and both emit `Add hasPhoneNumber/2 = {"phoneNumber": "09/241.25.10", "hasNumberFunction": "Operator"}`. Both apply; the golden record now holds the same phone number twice. Under key-based identity the second add would have been recognised as the same element. + +The correct resolution for this slot is Option 2 below: declare the SHACL rule as `unique_keys`. + +### Example 2 — `PositioningSystemCoordinate`: one class, two identity shapes + +`PositioningSystemCoordinate` declares a key — `typeURI`, the type designator, which plays the role of the coordinate-system discriminator (`rsm.yaml:610-625`, `asset360.yaml:541-550`, elided): + +```yaml + PositioningSystemCoordinate: + is_a: ObservableProperty + description: A tuple of coordinates in a given positioning system. + slots: + - typeURI # range: uri, designates_type: true + attributes: + PositioningSystemCoordinate_positioningSystem: + range: PositioningSystem + slot_usage: + typeURI: + key: true +``` + +**Shape B — keyed dict (the key is real).** `SpotLocation` holds at most one coordinate per coordinate-system type (`rsm.yaml:678-696`): + +```yaml + SpotLocation: + is_a: BaseLocation + attributes: + SpotLocation_coordinates: + multivalued: true + inlined: true + inlined_as_list: false # JSON object keyed by typeURI + range: PositioningSystemCoordinate +``` + +Real committed data (`asset360-model/tests/data/signal-obj-with-track.json:38-96`, trimmed) — one `LinearCoordinate`, one `GeographicCoordinate`, keyed by class URI: + +```json +"SpotLocation_coordinates": { + "https://data.infrabel.be/asset360-rsm-subset/LinearCoordinate": { + "measure": {"Quantity_unit": ".../unit/Kilometer", "NumericQuantity_value": 526.0}, + "typeURI": "http://rsm.uic.org/RSM12#EAID_CB107995_3610_4622_824B_708281B24CEA" + }, + "https://data.infrabel.be/asset360-rsm-subset/GeographicCoordinate": { + "typeURI": "https://data.infrabel.be/asset360-rsm-subset/GeographicCoordinate", + "latitude": 50.820240734349845, + "longitude": 4.316083008990688 + } +} +``` + +~25k records rely on this keyed behaviour. (The same pattern recurs one level up: `locations` is a dict of `BaseLocation` keyed by the `locationrole` enum slot, `rsm.yaml:734-742`.) + +**Shape A — vertex ring (the key is constant).** `Polyline` and `Polygon` hold the *same* class as an ordered ring (`rsm.yaml:567-608`): + +```yaml + Polyline: + is_a: NamedResource + attributes: + PolyLine_coordinates: + range: PositioningSystemCoordinate + multivalued: true + inlined_as_list: true + + Polygon: + is_a: NamedResource + attributes: + Polygon_coordinates: + range: PositioningSystemCoordinate + multivalued: true + inlined_as_list: true +``` + +Every vertex of a ring is the same coordinate subclass, so the declared key (`typeURI`) is **constant across elements**. Real committed data shows it plainly (`consolidator_api/goldenrecords/tests/data/a_trail.json`, trimmed to two of four vertices — all four repeat the identical `typeURI` and the identical positioning system): + +```json +"polylines": [{ + "PolyLine_coordinates": [ + {"x": 214888.6, "y": 97029.71, + "typeURI": "https://data.infrabel.be/asset360/MicroCoordinate", + "PositioningSystemCoordinate_positioningSystem": {"typeURI": "http://rsm.uic.org/RSM12#EAID_4160AA98_..."}}, + {"x": 214819.42, "y": 97029.71, + "typeURI": "https://data.infrabel.be/asset360/MicroCoordinate", + "PositioningSystemCoordinate_positioningSystem": {"typeURI": "http://rsm.uic.org/RSM12#EAID_4160AA98_..."}} + ] +}] +``` + +Position is the only identity, ring data legitimately repeats the key value (a closed ring may even repeat a whole vertex), and vertex order is meaningful. The per-vertex `typeURI` and positioning system are pure redundancy: the system is a property of the ring, not of the vertex. + +One class, two identity shapes. That ring data violates any `unique_keys` the class would declare — so the class can't declare one, and letting the opaque annotation suppress a schema constraint would be a layering inversion we do not accept. The resolution is to rework the model: move the positioning-system identity up a layer and give rings a bare vertex class that never inherits the key. One valid remodeling is worked out in Option 4 below; `SpotLocation_coordinates` and the coordinate hierarchy stay untouched. + +## The options when the linter flags a slot + +The linter's question is always the same — *where does element identity come from?* — and the author has exactly three answers, plus a rework escape hatch. Each constituting example gets its correct resolution below. + +### Option 1 — a key (or identifier): identity a single slot already provides + +When the element class declares a `key` or `identifier` slot that is truthful for all of its data, nothing needs to change — the diff already matches elements by it. `SpotLocation_coordinates` is the example: a dict keyed by `typeURI`, at most one coordinate per positioning system, exactly what the data means. + +One slot cannot answer this way: the class's own type designator. Its value is a function of the element's class, not of the element, so the addendum's rule 1 makes the engine look past it in **list** form and fall through to `unique_keys` (else position). The dict form above is unaffected — a mapping keyed by the designator is exactly the at-most-one-per-subtype statement it looks like — and the linter says so on the slot that needs the other answer. + +### Option 2 — a composed key (content): declare `unique_keys` — the phone number solution + +`hasNumberFunction` is already the de-facto identity — the SHACL shape says so. Declare exactly that rule with the existing LinkML meta `unique_keys`: + +```yaml + ServicePhoneNumber: + unique_keys: + one_number_per_function: + unique_key_slots: + - hasNumberFunction + attributes: + phoneNumber: + range: string + hasNumberFunction: + range: NumberFunction + required: true +``` + +This is the SHACL constraint expressed verbatim in the schema, and it is purely additive: `unique_keys` changes neither serialization nor loading of the deployed list data (unlike promoting the slot to `key: true`, which changes the container's serialization contract). Element identity is the function: deltas are addressed by it (`hasPhoneNumber/Emergency_Number/phoneNumber`), immune to drift and reorder, and the duplicate-add from Example 1 collides into the same element instead of duplicating it. + +`unique_key_slots` composes: had the model allowed several numbers per function, `[hasNumberFunction, phoneNumber]` would make the full content the identity — at the price that every correction becomes a remove-plus-add of the whole element. Here that composition would be wrong: it would permit two numbers for the same function, contradicting the SHACL rule. + +### Option 3 — opaque: identity comes from nowhere, say so + +For the vertex ring, elements genuinely have no identity — a moved vertex, an inserted vertex, a reversed ring are all edits *of the geometry*, not of a vertex: + +```yaml + Polygon_coordinates: + range: Vertex + multivalued: true + inlined_as_list: true + annotations: + diff.linkml.io/opaque: true +``` + +All recursion stops at the slot: any change below it is exactly one `Update` at the slot path carrying the whole old and new value. Two sources editing the same ring conflict visibly at the ring level — whole value against whole value — instead of interleaving vertex indices into a corrupt geometry. + +### Option 4 — remodel: when one class needs two answers — the coordinate solution + +`PositioningSystemCoordinate` needs Option 1 in `SpotLocation` and Option 3 in `Polyline`/`Polygon`, and its ring data violates the very declaration the keyed usage needs. No slot-level override can fix that — identity declarations stay class-level truths; slots choose only *whether* to recurse, never *what identity means*. Rework the model instead. One valid remodeling (shown for `Polygon`; `Polyline` is symmetric): + +```yaml + AreaLocation: + attributes: + polygons: + range: Polygon + multivalued: true + inlined_as_list: true + + Polygon: + is_a: NamedResource + unique_keys: + one_polygon_per_positioning_system: + unique_key_slots: + - positioningSystemType + attributes: + positioningSystemType: # the identity the vertices used to repeat, + range: uri # lifted up to the ring layer + required: true + Polygon_positioningSystem: + range: PositioningSystem + inlined: true + Polygon_coordinates: + range: Vertex + multivalued: true + inlined_as_list: true + annotations: + diff.linkml.io/opaque: true + + Vertex: + attributes: + x: {range: float} + y: {range: float} + z: {range: float} +``` + +The move is to introduce a layer so the key slot no longer sits on the elements holding the geometry data: + +- `locations` is already keyed by `locationrole` — that layer exists. +- A polygon's identity within `polygons` is its positioning system, declared as a `unique_keys` composed key on `Polygon`. The schema's own description already says the list holds the same shape once per positioning system; the redundant per-vertex `typeURI` / positioning system move up to the ring layer, which is what they always described. +- The ring becomes an opaque list of a bare `Vertex` class that never inherits `key: typeURI`, so no declaration is violated by repeated vertices. +- `SpotLocation_coordinates` and the `PositioningSystemCoordinate` hierarchy are untouched; the ~25k keyed lookup records keep their behaviour. + +## Consequence of strict mode + +- Some LinkML schemas need to be reworked to be compliant — in asset360 concretely: `ServicePhoneNumber` gains a `unique_keys` declaration, and the polygon/polyline rings move their positioning-system identity up a layer and become opaque lists of a bare `Vertex` class. +- More cases of patches produced outside this diff lib cannot be applied fully: a patch that addresses elements positionally has no meaning against an opaque slot (only whole-value updates apply) and no reliable meaning against a keyed/composed-key container. Such deltas are reported as failed rather than guessed at. + +## Recyclable material + +The abandoned branch `feat/container-shapes-and-verify-old` (local, single commit `312e6d8`) and the follow-up spike `spike/unique-keys-vs-opaque` contain fixtures and tests that carry over; the `DiffShape::Set`/`ShapeConfig` machinery itself is superseded by this design. + +Recycled (near-verbatim, renamed to `opaque` where the branch says `array`): + +- `diff_shapes.rs:158-219` — `array_slot_emits_one_whole_slot_update` (move / insert / drop / reverse a vertex ring, each exactly one whole-slot `Update`), the scalar-list variant, and `array_slot_unchanged_emits_nothing`. +- `diff_shapes.rs:419-463` — `keyed_slot_keeps_minimal_field_level_deltas`, as the regression guard that keyed containers keep minimal field-level deltas. +- `diff_shapes.rs:467-492` — `undeclared_slots_keep_positional_behaviour`, the non-goal's compatibility guard. +- `load_duplicate_keys.rs:113-181` — missing `key` is an error / missing `identifier` warns / type-designator carve-out, as linter severity fixtures (the designator carve-out exists precisely because of coordinate classes like Example 2). +- Multiplicity guards (`set_diff_respects_multiplicity`, the `["a","a"]` load assertion): a repeated element is data, never deduped. +- From the spike: `opaque_*` tests, `coordinates_match_by_unique_keys_derived_key`, and the `unique_keys` ambiguity-warning pair. + +Not recycled: the `Set` content-matching diff/patch (`diff_set`, `apply_set_leaf_delta`, the drift-location trio) and the single-slot `shape_key` override — both replaced by `unique_keys`-declared identity. + +## Addendum (2026-08-19): designator and canonicalization hardening + +An empirical spike (typeURI × polymorphism × induced URIs, findings D1–D9) showed the shipped identity machinery honours *spellings* and *declarations* where it must honour *meaning*. Since this branch already carries a sanctioned compatibility break, the following rules are added to the design. Every behavioural change below must be attributable, output-line by output-line, in the downstream differential harness (consolidator-server corpus) before it merges. + +### Rules + +1. **A type designator is never an element identity (D3).** `element_key_label` skips a key/identifier slot whose merged definition has `designates_type: true`; identity falls through to `unique_keys`, else the list is positional. Rationale: a designator's value is a function of the element's class — constant across any homogeneous list by construction. The schema-level designator-key lint rule remains as the author-facing voice; this rule makes the engine agree with it. (Blast radius: homogeneous rings already fell back to positional via the uniqueness guard; the change moves polymorphic designator-keyed lists and classes whose real `unique_keys` was shadowed.) +2. **Identity compares meaning, not spelling (D1 + D6).** + - Designator values are canonicalized at load (the boxing chokepoint, mirroring int/float coercion): once the element's class is selected, the stored designator value is the class's canonical designator value. JSON and RDF loaders then agree; `to_json` emits the canonical form. + - Identity-label components whose slot range descends from `uri`/`uriorcurie` are IRI-expanded before comparison, in **all** resolve sites at once: diff emission, `resolve_list_segment` (patch + navigate), and the instance lint. A curie and its expansion are one identity. +3. **A class change is a whole-element replacement (D2a).** When diff pairs two objects whose classes differ, it emits a single whole-element `Update` — never field-level recursion across classes. +4. **`patch` never hard-errors on an unappliable delta (D2b).** A delta whose value cannot be built or applied at its resolved location records its path in `trace.failed` and leaves the tree untouched; `Err` is reserved for infrastructure failures. One bad delta must not void a batch. +5. **The inlined-dict key is real data (D5).** `build_mapping_entry_for_slot` injects the dict key into the element's key/identifier slot when the payload omits it (the LinkML `inlined` contract); a payload value that disagrees with the dict key is a load-time validation warning. For designator-keyed dicts, a dict key that is not an accepted designator value is a load-time validation warning. +6. **Lint extensions (D7, D4d, D8, D9).** + - Instance lint: a list that is addressed positionally *despite* a declared identity (some element yields no label) is warned — missing labels get a voice, not only duplicates. + - Schema lint: the multi-`unique_keys` ambiguity check unions entries across the range class's descendants and warns when descendants resolve different load-bearing entries (split label space). + - Schema lint: warn when two classes within one `is_a` hierarchy share a `class_uri` and the hierarchy carries a designator (stable-but-arbitrary class selection); the loader behaviour itself is unchanged (deliberately out of scope — too hot for this branch). + - `slot_usage: designates_type: false` leaving an unfillable `key` is documented in the linter rustdoc, not specially detected. + +### Explicitly out of scope (recorded, not fixed here) + +- Loader preference of native-URI matches over shared `class_uri` matches (D8's fix half). +- Namespacing identity labels by the `unique_keys` entry that produced them (D4's deep fix). +- `key: true` implying `required: true` at validation time. diff --git a/src/python/python/linkml_runtime_rust/_native.pyi b/src/python/python/linkml_runtime_rust/_native.pyi index 6897745..22f67af 100644 --- a/src/python/python/linkml_runtime_rust/_native.pyi +++ b/src/python/python/linkml_runtime_rust/_native.pyi @@ -2789,7 +2789,12 @@ class LinkMLInstance: def __getitem__(self, key:typing.Any) -> LinkMLInstance: ... def navigate(self, path:typing.Any) -> typing.Optional[LinkMLInstance]: r""" - Navigate by a path of strings (map keys or list indices). + Navigate by a path of strings: slot names, mapping keys, and — for + lists — element identity labels (identifier/key or `unique_keys` value) + when the list carries unique labels, numeric indices otherwise. This is + the same addressing `diff` emits and `patch` applies, so delta paths are + navigable; a numeric segment aimed at a label-addressed list resolves to + nothing rather than to that position. Returns a new LinkMLInstance if found, otherwise None. """ def keys(self) -> builtins.list[builtins.str]: ... @@ -5478,6 +5483,43 @@ def import_turtle(reader:typing.Any, schema_view:SchemaView, root_classes:typing Import RDF/Turtle into a streaming iterator of LinkML instances. """ +def lint_element_identity(schema_view:SchemaView) -> builtins.list[ValidationResult]: + r""" + Schema-level lint: warn where a multivalued inlined slot's element identity + is absent, ambiguous, or cannot address the list. + + Five rules: (1) no identity declared at all; (2) the declared identity is + the element class's type designator, whose value describes the class rather + than the element; (3) several ``unique_keys`` entries to choose from across + the range class and its descendants, of which only the name-sorted first is + load-bearing; (4) those classes labelled in different ways, so one list + carries two label spaces; (5) two classes of one ``is_a`` hierarchy + declaring the same ``class_uri`` while the hierarchy designates its type. + + Warnings only — the schema stays usable. Results are deterministic: sorted + by subject, deduplicated across class URIs, and an inherited slot is + reported once, at the class that introduces the problem. Rules 1-4 are + per-slot and their ``subject`` is ``[class_name, slot_name]``; rule 5 is + class-level and its ``subject`` is ``"shared_class_uri"`` followed by the + classes sharing the URI — the marker distinguishes it from a per-slot + subject, which a rendering that joins the segments could not otherwise do. + """ + +def lint_instance_identity(instance:LinkMLInstance) -> builtins.list[ValidationResult]: + r""" + Data-level lint: warn where loaded data defeats a declared element identity. + + Two rules: a list whose elements repeat a declared identity — key/identifier + or ``unique_keys`` value — reported as ``duplicate_element_identity``; and a + list addressed positionally *despite* a declared identity, because some + element leaves the slot that identity names empty, reported as + ``ambiguous_element_identity``. Neither is visible in the schema: an + identity slot that is not ``required`` may be absent, and repeated or + missing values are alike valid data. + + Warnings only. ``subject`` is the container's instance path. + """ + def load_json(source:typing.Any, sv:SchemaView, class_view:ClassView) -> tuple[typing.Optional[LinkMLInstance], builtins.list[ValidationResult]]: ... def load_yaml(source:typing.Any, sv:SchemaView, class_view:ClassView) -> tuple[typing.Optional[LinkMLInstance], builtins.list[ValidationResult]]: ... diff --git a/src/python/src/lib.rs b/src/python/src/lib.rs index aa2f14e..d977cf6 100644 --- a/src/python/src/lib.rs +++ b/src/python/src/lib.rs @@ -4,8 +4,8 @@ use linkml_runtime::diff::{ }; use linkml_runtime::turtle::{turtle_to_string, TurtleOptions}; use linkml_runtime::{ - load_json_str, load_yaml_str, validate_issues, LinkMLInstance, LoadResult, NodeId, - ValidationProblemType, ValidationResult, ValidationSeverity, ValidationValue, + lint_element_identity, lint_instance_identity, load_json_str, load_yaml_str, validate_issues, + LinkMLInstance, LoadResult, NodeId, ValidationResult, ValidationSeverity, ValidationValue, }; use linkml_schemaview::identifier::Identifier; use linkml_schemaview::io; @@ -744,6 +744,8 @@ pub fn runtime_module(m: &Bound<'_, PyModule>) -> PyResult<()> { m.add_function(wrap_pyfunction!(load_json, m)?)?; m.add_function(wrap_pyfunction!(py_diff, m)?)?; m.add_function(wrap_pyfunction!(py_patch, m)?)?; + m.add_function(wrap_pyfunction!(py_lint_element_identity, m)?)?; + m.add_function(wrap_pyfunction!(py_lint_instance_identity, m)?)?; m.add_function(wrap_pyfunction!(py_import_turtle, m)?)?; m.add_function(wrap_pyfunction!(py_import_ntriples, m)?)?; m.add_function(wrap_pyfunction!(py_export_turtle, m)?)?; @@ -874,7 +876,7 @@ impl From for PyValidationResult { impl PyValidationResult { #[getter] fn r#type(&self) -> String { - validation_problem_type_label(&self.inner.problem_type).to_string() + self.inner.problem_type.label().to_string() } #[getter] @@ -915,7 +917,7 @@ impl PyValidationResult { fn __repr__(&self) -> PyResult { Ok(format!( "ValidationResult(type='{}', severity='{}', subject={:?}, detail={})", - validation_problem_type_label(&self.inner.problem_type), + self.inner.problem_type.label(), severity_label(&self.inner.severity), self.inner.subject, self.inner.detail @@ -1045,17 +1047,6 @@ fn validation_value_to_py(py: Python<'_>, value: &ValidationValue) -> PyResult

&'static str { - match problem_type { - ValidationProblemType::UndeclaredSlot => "undeclared_slot", - ValidationProblemType::InapplicableSlot => "inapplicable_slot", - ValidationProblemType::MissingSlotValue => "missing_slot_value", - ValidationProblemType::SlotRangeViolation => "slot_range_violation", - ValidationProblemType::MaxCountViolation => "max_count_violation", - ValidationProblemType::ParsingError => "parsing_error", - } -} - fn severity_label(severity: &ValidationSeverity) -> &'static str { match severity { ValidationSeverity::Fatal => "fatal", @@ -1223,7 +1214,12 @@ impl PyLinkMLInstance { } } - /// Navigate by a path of strings (map keys or list indices). + /// Navigate by a path of strings: slot names, mapping keys, and — for + /// lists — element identity labels (identifier/key or `unique_keys` value) + /// when the list carries unique labels, numeric indices otherwise. This is + /// the same addressing `diff` emits and `patch` applies, so delta paths are + /// navigable; a numeric segment aimed at a label-addressed list resolves to + /// nothing rather than to that position. /// Returns a new LinkMLInstance if found, otherwise None. #[pyo3(name = "navigate")] fn py_navigate<'py>( @@ -1569,6 +1565,55 @@ fn py_patch( Py::new(py, result) } +// ── Identity lints ────────────────────────────────────────────────────────── + +/// Schema-level lint: warn where a multivalued inlined slot's element identity +/// is absent, ambiguous, or cannot address the list. +/// +/// Five rules: (1) no identity declared at all; (2) the declared identity is +/// the element class's type designator, whose value describes the class rather +/// than the element; (3) several ``unique_keys`` entries to choose from across +/// the range class and its descendants, of which only the name-sorted first is +/// load-bearing; (4) those classes labelled in different ways, so one list +/// carries two label spaces; (5) two classes of one ``is_a`` hierarchy +/// declaring the same ``class_uri`` while the hierarchy designates its type. +/// +/// Warnings only — the schema stays usable. Results are deterministic: sorted +/// by subject, deduplicated across class URIs, and an inherited slot is +/// reported once, at the class that introduces the problem. Rules 1-4 are +/// per-slot and their ``subject`` is ``[class_name, slot_name]``; rule 5 is +/// class-level and its ``subject`` is ``"shared_class_uri"`` followed by the +/// classes sharing the URI — the marker distinguishes it from a per-slot +/// subject, which a rendering that joins the segments could not otherwise do. +#[cfg_attr(feature = "stubgen", gen_stub_pyfunction)] +#[pyfunction(name = "lint_element_identity")] +fn py_lint_element_identity( + py: Python<'_>, + schema_view: &PySchemaView, +) -> PyResult>> { + validation_results_to_py(py, lint_element_identity(schema_view.as_rust())) +} + +/// Data-level lint: warn where loaded data defeats a declared element identity. +/// +/// Two rules: a list whose elements repeat a declared identity — key/identifier +/// or ``unique_keys`` value — reported as ``duplicate_element_identity``; and a +/// list addressed positionally *despite* a declared identity, because some +/// element leaves the slot that identity names empty, reported as +/// ``ambiguous_element_identity``. Neither is visible in the schema: an +/// identity slot that is not ``required`` may be absent, and repeated or +/// missing values are alike valid data. +/// +/// Warnings only. ``subject`` is the container's instance path. +#[cfg_attr(feature = "stubgen", gen_stub_pyfunction)] +#[pyfunction(name = "lint_instance_identity")] +fn py_lint_instance_identity( + py: Python<'_>, + instance: &PyLinkMLInstance, +) -> PyResult>> { + validation_results_to_py(py, lint_instance_identity(&instance.value)) +} + // ── RDF import/export ─────────────────────────────────────────────────────── /// Streaming iterator over harvested LinkML instances. diff --git a/src/python/tests/python_navigate.rs b/src/python/tests/python_navigate.rs index d25fb9e..788e3bc 100644 --- a/src/python/tests/python_navigate.rs +++ b/src/python/tests/python_navigate.rs @@ -62,13 +62,20 @@ value = load_no_errors(lr, lr.load_yaml, data_path, sv, cls) assert 'objects' in value.keys() assert value.navigate(['objects']) is not None -# Navigate list index then nested keys to scalar -name = value.navigate(['objects','2','has_medical_history','0','diagnosis','name']) +# Navigate a list by element identity label, then nested keys to a scalar. +# `objects` ranges on NamedThing, which declares an `id` identifier, so the +# list is addressed by that label, never by position. `has_medical_history` +# declares no identity and stays numeric. +name = value.navigate(['objects','P:002','has_medical_history','0','diagnosis','name']) assert name is not None assert name.as_python() == 'headache' # Non-existent path -> None -assert value.navigate(['objects','1000']) is None +assert value.navigate(['objects','P:404']) is None + +# A numeric segment aimed at a label-addressed list resolves to nothing +# rather than to that position: it never guesses an element. +assert value.navigate(['objects','2']) is None "# ); }); diff --git a/src/runtime/src/blame/mod.rs b/src/runtime/src/blame/mod.rs index d8d832f..14634a6 100644 --- a/src/runtime/src/blame/mod.rs +++ b/src/runtime/src/blame/mod.rs @@ -22,6 +22,12 @@ use std::fmt; /// strategy: every node touched by the patch (added or updated according to the /// [`PatchTrace`]) will have the supplied metadata cloned into the provided /// `blame` map. +/// +/// Deltas that did not apply are in [`PatchTrace::failed`] and touch nothing, +/// so they leave no blame entry — and, since [`patch`] never hard-errors on +/// one (spec addendum rule 4), an `Ok` here does not mean the whole batch +/// landed. Callers that treat a patch as atomic must check `trace.failed` +/// themselves; the blame map records what actually happened either way. pub fn patch_with_blame( value: &LinkMLInstance, deltas: &[Delta], diff --git a/src/runtime/src/diff.rs b/src/runtime/src/diff.rs index 9c186ab..bf7948d 100644 --- a/src/runtime/src/diff.rs +++ b/src/runtime/src/diff.rs @@ -1,6 +1,7 @@ use crate::{LResult, LinkMLInstance, NodeId, ValidationResultSink}; use linkml_schemaview::{ converter::Converter, + identifier::Identifier, schemaview::{ClassView, SchemaView, SlotView}, }; use serde::{Deserialize, Serialize}; @@ -9,7 +10,7 @@ use std::collections::hash_map::Entry; const IGNORE_ANNOTATION: &str = "diff.linkml.io/ignore"; -fn slot_is_ignored(slot: &SlotView) -> bool { +pub(crate) fn slot_is_ignored(slot: &SlotView) -> bool { if slot.definitions().is_empty() { return false; } @@ -20,6 +21,243 @@ fn slot_is_ignored(slot: &SlotView) -> bool { .unwrap_or(false) } +/// Slot annotation declaring that element identity comes from nowhere: stop +/// all recursion, the slot's value is one atomic unit. Any change below the +/// slot is described as a single whole-value `Update` at the slot path, and +/// `patch` refuses paths that descend below it. +pub const OPAQUE_ANNOTATION: &str = "diff.linkml.io/opaque"; + +pub(crate) fn slot_is_opaque(slot: &SlotView) -> bool { + if slot.definitions().is_empty() { + return false; + } + slot.definition() + .annotations + .as_ref() + .map(|a| a.contains_key(OPAQUE_ANNOTATION)) + .unwrap_or(false) +} + +/// One component of an element's identity label, canonicalised so that +/// identity compares meaning and not spelling (spec addendum rule 2, D6). +/// +/// A slot whose range descends from `uri`/`uriorcurie` holds an IRI, and `ex:WGS84` +/// and `https://example.org/canon/WGS84` are the same IRI. Compared as raw +/// strings they are two identities: the same element re-spelled diffs as a +/// Remove + Add, `navigate_path` by the expansion misses the CURIE-spelled +/// element, and the instance lint calls a genuine duplicate unique. +/// +/// The expansion lives here, in the one function every identity label is built +/// from, so all four resolve sites move together: diff emission (via +/// [`element_key_label`] / [`element_unique_key_label`]), [`resolve_list_segment`] +/// for `patch` and `navigate_path`, and the instance lint. That is what keeps +/// "the segments diff emits are exactly what the resolver computes" true by +/// construction rather than by three coincidences. +/// +/// A bare name (no `:` at all) and a CURIE with an unregistered prefix are left +/// verbatim: `Identifier::to_uri` refuses them, and inventing an expansion +/// against the default prefix would rename identities the schema never claimed +/// were IRIs. +pub(crate) fn canonical_identity_component(raw: &str, slot: &SlotView) -> String { + if !slot.is_range_iri() { + return raw.to_string(); + } + let Some(conv) = slot.sv.converter_for_schema(slot.schema_id()) else { + return raw.to_string(); + }; + match Identifier::new(raw).to_uri(&conv) { + Ok(uri) => uri.0, + Err(_) => raw.to_string(), + } +} + +pub(crate) fn scalar_slot_string( + values: &std::collections::HashMap, + slot_name: &str, +) -> Option { + if let Some(LinkMLInstance::Scalar { value, slot, .. }) = values.get(slot_name) { + return match value { + JsonValue::String(s) => Some(canonical_identity_component(s, slot)), + other => Some(other.to_string()), + }; + } + None +} + +/// Are these two `ClassView`s the same class? +/// +/// Schema-qualified name, not pointer equality: a `ClassView` is a derived, +/// freshly built view, so two views of one class are routinely distinct values, +/// and a name alone is only unique within its schema. This is the question +/// "did diff pair two objects of different classes?" (spec addendum rule 3), +/// which is about the *declaration* the element instantiates — deliberately +/// finer than [`crate::LinkMLInstance::equals`]'s canonical-URI comparison, +/// which two classes sharing a `class_uri` (spike D8) also satisfy. +fn class_identity_equal(a: &ClassView, b: &ClassView) -> bool { + a.name() == b.name() && a.schema_id() == b.schema_id() +} + +/// The class's key/identifier slot, when it can identify an *element*. +/// +/// A key (or identifier) that is also the class's type designator is skipped: a +/// designator's value is a function of the element's *class*, not of the +/// element, so it is constant across any homogeneous list by construction and +/// says "one element per subtype" across a polymorphic one. Neither is element +/// identity. Identity then falls through to `unique_keys` — which such a class +/// may well declare, and which the designator key used to shadow — else the +/// list is positional. [`crate::lint_element_identity`] is the author-facing +/// voice for the same shape; this is the engine agreeing with it. +/// +/// This governs *labelling* — how an element is addressed among its siblings — +/// and so is used by every resolve site (diff emission, `resolve_list_segment` +/// for patch and navigate, the instance lint). It is deliberately **not** used +/// by `diff`'s changed-key check, which asks a different question: see the +/// comment at `treat_changed_identifier_as_new_object`. +pub(crate) fn identity_key_slot(class: &ClassView) -> Option<&SlotView> { + let slot = class.key_or_identifier_slot()?; + if slot.definition().designates_type == Some(true) { + return None; + } + Some(slot) +} + +/// The key/identifier value identifying `v` among its list siblings, if any. +pub(crate) fn element_key_label(v: &LinkMLInstance) -> Option { + if let LinkMLInstance::Object { values, class, .. } = v { + let id_slot = identity_key_slot(class)?; + return scalar_slot_string(values, &id_slot.name); + } + None +} + +/// A matching label derived from the range class's merged `unique_keys`. +/// +/// The name-sorted first entry with a non-empty slot list is the matching +/// identity (declaration order is not preserved by the metamodel, and diff +/// paths must be stable). Single-slot keys use the bare scalar value as the +/// label and path segment; composite keys use the JSON array encoding of the +/// values in `unique_key_slots` order (`["Emergency","02/111.11.11"]`) — +/// unambiguous, parseable, and displayable. +pub(crate) fn element_unique_key_label(v: &LinkMLInstance) -> Option { + if let LinkMLInstance::Object { values, class, .. } = v { + let uks = class.unique_keys(); + let (_, uk) = uks.iter().find(|(_, uk)| !uk.unique_key_slots.is_empty())?; + let parts: Option> = uk + .unique_key_slots + .iter() + .map(|s| scalar_slot_string(values, s)) + .collect(); + let mut parts = parts?; + return Some(if parts.len() == 1 { + parts.remove(0) + } else { + // Infallible JSON array encoding (the crate denies `expect`). + JsonValue::Array(parts.into_iter().map(JsonValue::String).collect()).to_string() + }); + } + None +} + +/// Identity for keyed list matching: a key/identifier slot outranks a +/// `unique_keys` claim — unless that key is the class's type designator, which +/// is never element identity (see [`identity_key_slot`]). +pub(crate) fn element_identity_label(v: &LinkMLInstance) -> Option { + element_key_label(v).or_else(|| element_unique_key_label(v)) +} + +/// The single slot `v`'s identity label was read from, when the label *is* one +/// scalar: a key/identifier, or a one-slot `unique_keys` entry. A composite +/// `unique_keys` entry encodes a JSON array and has no single source slot. +/// +/// Mirrors [`element_identity_label`]'s precedence exactly, including its +/// fall-through when the key slot carries no value — the point is to name the +/// slot that produced the label, so that a path segment can be normalised the +/// same way the label was. +fn identity_label_slot(v: &LinkMLInstance) -> Option<&SlotView> { + let LinkMLInstance::Object { values, class, .. } = v else { + return None; + }; + if let Some(slot) = identity_key_slot(class) { + if scalar_slot_string(values, &slot.name).is_some() { + return Some(slot); + } + } + let uks = class.unique_keys(); + let (_, uk) = uks.iter().find(|(_, uk)| !uk.unique_key_slots.is_empty())?; + let [only] = uk.unique_key_slots.as_slice() else { + return None; + }; + class.slots().iter().find(|s| s.name == *only) +} + +/// Does the path segment `key` address the element `v`, whose identity label is +/// `label`? +/// +/// The other half of spec addendum rule 2's "a curie and its expansion are one +/// identity". Labels are already IRI-expanded on the way out +/// ([`canonical_identity_component`]); a segment arriving from outside — a +/// stored delta, a hand-written patch, a caller's `navigate_path` — has been +/// through no such thing. Normalising it through the *same* slot the label came +/// from makes the comparison symmetric, so `ex:WGS84` addresses an element +/// whose label expanded to `https://example.org/canon/WGS84` and vice versa. +/// +/// Segments diff itself emits already equal the label outright and never reach +/// the expansion. +fn segment_matches_label(v: &LinkMLInstance, label: Option<&str>, key: &str) -> bool { + let Some(label) = label else { + return false; + }; + if label == key { + return true; + } + match identity_label_slot(v) { + Some(slot) => canonical_identity_component(key, slot) == label, + None => false, + } +} + +fn labels_are_unique(elements: &[LinkMLInstance], label: F) -> bool +where + F: Fn(&LinkMLInstance) -> Option, +{ + let mut seen = std::collections::HashSet::new(); + elements.iter().filter_map(label).all(|l| seen.insert(l)) +} + +/// Whether this one list is addressed by identity label: it is non-empty, +/// every element carries an identity label, and the labels are unique. +/// +/// This is the predicate that decides how a list is *addressed*, asked of a +/// single list. `diff` needs the same answer of both sides at once (a keyed +/// match needs identity on both), but every consumer that has only one list in +/// front of it — `patch`'s segment resolver, `navigate_path`, and diff's +/// keyed-source fallback — must agree, or a path one of them emits is a path +/// another cannot resolve. +/// +/// Derives labels lazily and stops at the first element that has none — the +/// answer for an unlabelled list is settled by its first bare element, however +/// long the list is. [`list_is_keyed_shaped_from_labels`] is the variant for a +/// caller that has already paid for the labels. +pub(crate) fn list_is_keyed_shaped(values: &[LinkMLInstance]) -> bool { + !values.is_empty() + && values.iter().all(|v| element_identity_label(v).is_some()) + && labels_are_unique(values, element_identity_label) +} + +/// [`list_is_keyed_shaped`] for a caller that already has the labels. +/// +/// The predicate itself, over labels rather than elements. Deriving a label is +/// not free — it walks the class's merged `unique_keys` and IRI-expands the +/// components — and [`resolve_list_segment`] needs both the labels and this +/// answer, so it must not pay for them twice. +fn list_is_keyed_shaped_from_labels(labels: &[Option]) -> bool { + let mut seen = std::collections::HashSet::new(); + !labels.is_empty() + && labels + .iter() + .all(|l| matches!(l, Some(l) if seen.insert(l.as_str()))) +} + /// Operation applied by a [`Delta`]. #[derive(Clone, Debug, PartialEq, Serialize, Deserialize)] #[serde(rename_all = "lowercase")] @@ -35,8 +273,22 @@ pub enum DeltaOp { /// Semantic delta emitted by [`diff`] and consumed by [`patch`]. /// /// The `path` identifies the location within the instance tree. Each segment is a -/// slot name, mapping key, list index, or (when available) the identifier/key slot -/// value for inlined objects in lists. +/// slot name, mapping key, list index, or — for inlined objects in lists matched +/// by identity — the element's identity label: its identifier/key slot value, or +/// failing that a value derived from the range class's `unique_keys`. Lists whose +/// elements do not all carry a *unique* identity label are addressed by numeric +/// index instead. A key/identifier that is the class's type designator does not +/// count: it labels the class, not the element, so identity falls through to +/// `unique_keys` or to the index (see [`identity_key_slot`]). +/// +/// For a `unique_keys`-derived segment, a single-slot key contributes the bare +/// value of that slot, while a composite key contributes the JSON array encoding +/// of the values in `unique_key_slots` order, e.g. `["Emergency","02/111.11.11"]`. +/// When the range class declares several `unique_keys`, the identity comes from +/// the name-sorted first entry with a non-empty slot list — the metamodel does +/// not preserve declaration order, and paths have to be stable. The hazard is +/// schema evolution: adding an earlier-sorting entry re-addresses every path for +/// that slot, so [`crate::lint_element_identity`] warns about any such class. /// /// Operations are expressed jointly via [`Delta::op`], `old`, and `new`: /// @@ -109,6 +361,36 @@ impl DiffOptions { /// - X → missing (object slot): ignored by default; `Update` to null when `treat_missing_as_null`. /// - X → missing (mapping key): ignored by default; `Remove` when `treat_missing_as_null`. /// - X → missing (list element): always `Remove` (lists are positional/complete). +/// +/// Slots annotated `diff.linkml.io/opaque` stop all recursion: any change at or +/// below the slot is described as a single whole-value `Update` at the slot +/// path. See [`OPAQUE_ANNOTATION`]. +/// +/// Two paired objects of *different classes* are likewise one whole-element +/// `Update`, never a field-by-field recursion across the two class definitions: +/// the element did not change, it was replaced. "Different class" is the +/// schema-qualified name, but the `Update` is still suppressed when the two +/// objects compare equal, so two classes sharing a `class_uri` with identical +/// content emit nothing. +/// +/// The qualification is by schema *id*, not by `SchemaView` instance: two views +/// built separately over the same schema qualify their classes identically and +/// diff as finely as ever. Where it bites is a genuinely cross-schema pairing — +/// one tree typed by `…/schema/v1`, the other by `…/schema/v2`, or by two +/// schemas that merely declare the same class name. Then no paired object is +/// ever "the same class" and every one of them coarsens to a whole-element +/// `Update`: correct, and patchable where the two schemas still agree about the +/// element's shape. Where they genuinely disagree, the `Update`'s payload will +/// not build against the source schema and lands in [`PatchTrace::failed`] — +/// still an improvement on the field-level recursion this replaced, which +/// produced deltas that could not apply *and* had no single path to report. +/// +/// Lists are matched by element identity when both sides carry unique identity +/// labels, and positionally otherwise — with one exception: when the *source* +/// list alone is label-addressed (the target repeats or lacks a label), the +/// change is described as a single whole-value `Update` at the list's path. +/// `patch` addresses a label-addressed list by label only, so positional +/// segments aimed at one could never be applied. pub fn diff(source: &LinkMLInstance, target: &LinkMLInstance, opts: DiffOptions) -> Vec { fn inner( path: &mut Vec, @@ -122,6 +404,17 @@ pub fn diff(source: &LinkMLInstance, target: &LinkMLInstance, opts: DiffOptions) if slot_is_ignored(sl) { return; } + if slot_is_opaque(sl) { + if !s.equals(t, opts.treat_missing_as_null) { + out.push(Delta { + path: path.clone(), + op: DeltaOp::Update, + old: Some(s.to_json()), + new: Some(t.to_json()), + }); + } + return; + } } match (s, t) { ( @@ -136,20 +429,66 @@ pub fn diff(source: &LinkMLInstance, target: &LinkMLInstance, opts: DiffOptions) .. }, ) => { + // Spec addendum rule 3 (D2a): the two sides are different + // *kinds* of thing, so the change is one whole-element + // replacement — never field recursion across two classes. + // Recursing produced deltas describing one class's slots on + // the other's element (`thread` on a `Nut`), which no builder + // can apply: the diff was unpatchable by construction. + // + // Guarded by `equals` so the invariant "objects the crate + // considers equal produce no delta" survives: `equals` compares + // class identity by canonical URI, so two *differently named* + // classes sharing a `class_uri` (spike D8) with matching + // assignments stay silent, exactly as before. + // + // Not governed by `treat_changed_identifier_as_new_object`: + // that flag chooses how to describe a changed *key* within one + // class, while a cross-class recursion is unpatchable whatever + // the caller prefers. + if !class_identity_equal(sc, tc) { + if !s.equals(t, opts.treat_missing_as_null) { + out.push(Delta { + path: path.clone(), + op: DeltaOp::Update, + old: Some(s.to_json()), + new: Some(t.to_json()), + }); + } + return; + } // If objects have an identifier or key slot and it changed, treat as whole-object replacement // This applies for single-valued and list-valued inlined objects. if opts.treat_changed_identifier_as_new_object { + // Deliberately the metamodel's key, not [`identity_key_slot`]: + // this asks "is this still the same thing?", not "how is it + // labelled among its siblings". + // + // Rule 3 above now owns the *designator* case: a designator + // value is canonicalised at load (rule 2), so a changed + // designator means a changed class and the whole-element + // Update has already been emitted. What is left for this + // branch — and why it stays — is a changed key value within + // ONE class: same class, different element. + // + // The comparison is the canonical one (rule 2), through + // [`scalar_slot_string`] — the very function identity + // labels are built from. Raw strings were the rule's last + // unconverted site: a `uri`-ranged key respelled + // curie↔uri makes ONE identity label, so the list above + // matched the two elements as one element, and this branch + // then called that element replaced — a whole-element + // `Update` at a path that addresses it by the identity the + // branch had just declared changed. What the two questions + // disagree about is *which slot* to read (see above), never + // what makes two values of it the same value. let key_slot_name = sc .key_or_identifier_slot() .or_else(|| tc.key_or_identifier_slot()) .map(|s| s.name.clone()); if let Some(ks) = key_slot_name { - let sid = sm.get(&ks); - let tid = tm.get(&ks); - if let ( - Some(LinkMLInstance::Scalar { value: s_id, .. }), - Some(LinkMLInstance::Scalar { value: t_id, .. }), - ) = (sid, tid) + if let (Some(s_id), Some(t_id)) = + (scalar_slot_string(sm, &ks), scalar_slot_string(tm, &ks)) { if s_id != t_id { out.push(Delta { @@ -209,36 +548,28 @@ pub fn diff(source: &LinkMLInstance, target: &LinkMLInstance, opts: DiffOptions) } } (LinkMLInstance::List { values: sl, .. }, LinkMLInstance::List { values: tl, .. }) => { - let label = |v: &LinkMLInstance| -> Option { - if let LinkMLInstance::Object { values, class, .. } = v { - if let Some(id_slot) = class.key_or_identifier_slot() { - if let Some(LinkMLInstance::Scalar { value, .. }) = - values.get(&id_slot.name) - { - return match value { - JsonValue::String(s) => Some(s.clone()), - other => Some(other.to_string()), - }; - } - } - } - None - }; - // If every item in both lists carries an identifier, match by id - // rather than by position. Positional diff corrupts mid-list + let identity = |v: &LinkMLInstance| -> Option { element_identity_label(v) }; + // Uniform rule (spec, Non-goal section): keyed matching iff every + // element on both sides carries an identity label and the labels + // are unique within each side. Positional diff corrupts mid-list // removes/inserts into shifted Updates, and on patch the label - // resolver can remove the wrong duplicate. - let keyed = - sl.iter().all(|v| label(v).is_some()) && tl.iter().all(|v| label(v).is_some()); + // resolver can remove the wrong duplicate. Duplicate labels (a + // list repeating a key, or data violating a unique_keys claim) + // fall back to positional — matching duplicates by label would + // silently collapse elements. + let keyed = sl.iter().all(|v| identity(v).is_some()) + && tl.iter().all(|v| identity(v).is_some()) + && labels_are_unique(sl, identity) + && labels_are_unique(tl, identity); if keyed { use std::collections::HashSet; - let src_ids: HashSet = sl.iter().filter_map(&label).collect(); + let src_ids: HashSet = sl.iter().filter_map(&identity).collect(); let tgt_by_id: std::collections::HashMap = tl .iter() - .filter_map(|v| label(v).map(|id| (id, v))) + .filter_map(|v| identity(v).map(|id| (id, v))) .collect(); for sv in sl { - let Some(id) = label(sv) else { continue }; + let Some(id) = identity(sv) else { continue }; path.push(id.clone()); match tgt_by_id.get(&id) { Some(tv) => inner(path, None, sv, tv, opts, out), @@ -252,7 +583,7 @@ pub fn diff(source: &LinkMLInstance, target: &LinkMLInstance, opts: DiffOptions) path.pop(); } for tv in tl { - let Some(id) = label(tv) else { continue }; + let Some(id) = identity(tv) else { continue }; if !src_ids.contains(&id) { path.push(id); out.push(Delta { @@ -264,17 +595,28 @@ pub fn diff(source: &LinkMLInstance, target: &LinkMLInstance, opts: DiffOptions) path.pop(); } } + } else if list_is_keyed_shaped(sl) { + // The source alone is keyed-shaped: `patch` resolves such a + // list by label ONLY, so positional segments aimed at it are + // unappliable by design and `patch(a, diff(a, b))` would + // refuse the very deltas we just emitted. What actually + // happened is honestly a whole-value change — this list + // stopped having coherent element identity — so say that, + // once, at the slot. + if !s.equals(t, opts.treat_missing_as_null) { + out.push(Delta { + path: path.clone(), + op: DeltaOp::Update, + old: Some(s.to_json()), + new: Some(t.to_json()), + }); + } } else { let max_len = std::cmp::max(sl.len(), tl.len()); for i in 0..max_len { - let step = if let Some(sv) = sl.get(i) { - label(sv) - .or_else(|| tl.get(i).and_then(&label)) - .unwrap_or_else(|| i.to_string()) - } else { - tl.get(i).and_then(&label).unwrap_or_else(|| i.to_string()) - }; - path.push(step); + // Plain numeric segments only: a label that failed the + // keyed guard cannot address an element unambiguously. + path.push(i.to_string()); match (sl.get(i), tl.get(i)) { (Some(sv), Some(tv)) => inner(path, None, sv, tv, opts, out), (Some(sv), None) => out.push(Delta { @@ -373,7 +715,11 @@ pub struct PatchTrace { pub deleted: Vec, /// Node IDs of nodes that were directly updated (e.g., parent containers, scalars). pub updated: Vec, - /// Paths of deltas that could not be applied (missing targets, etc.). + /// Paths of deltas that could not be applied: an address that resolves to + /// nothing or resolves ambiguously, a path descending below an opaque slot, + /// or a payload that cannot be built at the location it addresses (spec + /// addendum rule 4). Each such delta leaves the tree untouched and the rest + /// of the batch still applies. pub failed: Vec>, } @@ -392,6 +738,37 @@ impl Default for PatchOptions { } } +/// Apply `deltas` to a clone of `source`, returning the result and a +/// [`PatchTrace`]. A delta whose path cannot be resolved is reported in +/// [`PatchTrace::failed`] rather than guessed at. +/// +/// One bad delta never voids the batch (spec addendum rule 4): a delta whose +/// payload cannot be built at the location it addresses — a scalar where the +/// range is a class, a slot the resolved element's class does not declare — is +/// reported the same way, with the tree untouched, and the remaining deltas +/// still apply. Callers wanting "all or nothing" check `trace.failed` and +/// discard the result themselves. +/// +/// With that, `patch` no longer fails: every way a delta can go wrong is a +/// `trace.failed` entry, and `LinkMLError` carries validation problems, not +/// infrastructure ones. The `LResult` return is retained for API stability — +/// treat an `Err` as unreachable rather than as the place to look for a +/// rejected delta. +/// +/// **List segments are resolved against the list's CURRENT state**, as the +/// deltas are applied in order — not against a snapshot of the list the deltas +/// were produced from. On a list whose elements carry *duplicate* identity +/// labels this is observable: such a list is addressed numerically, but a delta +/// that removes the duplication flips it to identity-addressed mid-sequence, +/// and every numeric segment still queued in the same patch then resolves to +/// nothing and is reported in `failed`. The patch stops short; it never lands +/// an edit on a guessed element. +/// +/// This is confined to lists whose element identity is already degenerate — +/// exactly what [`crate::lint_element_identity`] (the schema declares no +/// identity) and [`crate::lint_instance_identity`] (the data repeats one) exist +/// to flag. Declaring an identity, or `diff.linkml.io/opaque`, removes the +/// situation rather than working around it. pub fn patch( source: &LinkMLInstance, deltas: &[Delta], @@ -482,6 +859,27 @@ where builder(value, schema_view, &conv) } +/// Build a delta's replacement value, turning a build failure into "this delta +/// did not apply" (spec addendum rule 4, D2b). +/// +/// A delta carries a JSON payload that may be nonsense at the location it +/// addresses: a scalar where the slot's range is a class, a slot the resolved +/// element's class does not declare, an enum value outside the permissible set. +/// Propagating the builder's `Err` voided the entire batch — one stale or +/// hand-written delta and every other delta in the same patch was lost, with no +/// record of which one was at fault. The delta's path goes to +/// [`PatchTrace::failed`] instead and the tree is left untouched; `Err` from +/// `patch` is reserved for infrastructure failure. +/// +/// Called before any mutation at every apply site, so "failed" always means +/// "nothing happened", never "half happened". +fn build_or_fail(build: F) -> Option +where + F: FnOnce() -> LResult, +{ + build().ok() +} + fn current_class_and_slot(current: &LinkMLInstance) -> (Option, Option) { match current { LinkMLInstance::Object { class, .. } => (Some(class.clone()), None), @@ -527,34 +925,80 @@ fn replace_child_subtree( } } -fn resolve_list_index(values: &[LinkMLInstance], key: &str) -> Option { +/// Resolve one path segment against a list, to the index of the element it +/// addresses. +/// +/// The single rule every consumer of a delta path shares — `patch` when it +/// applies one, [`crate::LinkMLInstance::navigate_path`] when it follows one. +/// Keeping it in one function is what makes "diff emits it, patch applies it, +/// navigate finds it" true by construction rather than by three coincidences. +pub(crate) fn resolve_list_segment(values: &[LinkMLInstance], key: &str) -> Option { + // A list whose elements all carry unique identity labels is addressed by + // label ONLY: diff emits label segments for exactly these lists, and a + // numeric segment aimed at one (a stale positional patch) would be a + // guess — report, never guess. This also keeps integer-valued identity + // labels (e.g. a year as unique key) unambiguous: they resolve as + // labels, never as positions. + let labels: Vec> = values.iter().map(element_identity_label).collect(); + // Exact label equality first, everywhere. Two reasons, one line: + // *correctness* — a segment that IS an element's label must address that + // element, never a sibling whose differently-spelled label happens to + // normalise to the same string (only reachable in a heterogeneous list, + // where siblings draw their labels from different slots); and *cost* — the + // normalising comparison re-derives the label slot per element, walking the + // class's merged `unique_keys`, so the common case (segments diff emitted, + // which equal the labels outright) should never pay for it. + let exact = |from: usize| { + labels[from..] + .iter() + .position(|l| l.as_deref() == Some(key)) + .map(|i| i + from) + }; + let matches = |i: usize| segment_matches_label(&values[i], labels[i].as_deref(), key); + // The one normalising hit, or nothing: two elements the segment could mean + // is a question, not an answer, and this function reports rather than + // guesses. Both branches below share it, so neither can drift into + // resolving an ambiguity the other refuses. + // + // In the keyed branch the second hit is barely reachable: labels are unique + // there, and two of them can only normalise to one string if they were + // normalised by *different converters* — a heterogeneous list whose element + // classes come from schemas that disagree about a prefix. It is left as a + // refusal rather than an assertion precisely because it is reachable at all: + // a `debug_assert` would turn "I cannot tell which element you mean" into a + // crash. + let unique_normalised = || { + let mut hit: Option = None; + for i in 0..values.len() { + if matches(i) { + if hit.is_some() { + return None; + } + hit = Some(i); + } + } + hit + }; + if list_is_keyed_shaped_from_labels(&labels) { + // Labels are unique here, so an exact hit is the only exact hit. + return exact(0).or_else(unique_normalised); + } + // Positional list: numeric index first (the segments diff produces for + // these lists), then a single unambiguous label hit for drift tolerance. if let Ok(idx) = key.parse::() { if idx < values.len() { return Some(idx); } } - values.iter().enumerate().find_map(|(i, v)| { - if let LinkMLInstance::Object { - values: mv, class, .. - } = v - { - class - .key_or_identifier_slot() - .and_then(|id_slot| mv.get(&id_slot.name)) - .and_then(|child| match child { - LinkMLInstance::Scalar { value, .. } => match value { - JsonValue::String(s) => (s == key).then_some(i), - other => { - let key_json = JsonValue::String(key.to_string()); - (other == &key_json).then_some(i) - } - }, - _ => None, - }) - } else { - None - } - }) + if let Some(first) = exact(0) { + // Ambiguity is refused, not guessed at — but only exact hits compete + // with an exact hit. + return match exact(first + 1) { + Some(_) => None, + None => Some(first), + }; + } + unique_normalised() } fn try_update_scalar_in_place( @@ -615,7 +1059,9 @@ where { match op { DeltaOp::Add | DeltaOp::Update => { - let new_child = build_child()?; + let Some(new_child) = build_or_fail(build_child) else { + return Ok(false); + }; match values.entry(key.to_string()) { Entry::Occupied(mut entry) => { let existing = entry.get_mut(); @@ -658,9 +1104,11 @@ where } } +#[allow(clippy::too_many_arguments)] fn apply_list_leaf_delta( values: &mut Vec, idx_opt: Option, + key: &str, owner_id: NodeId, trace: &mut PatchTrace, opts: PatchOptions, @@ -672,7 +1120,42 @@ where { match op { DeltaOp::Add | DeltaOp::Update => { - let new_child = build_child()?; + // An `Update` whose address resolves to no element is either a + // source re-asserting an element some other source dropped — the + // multi-source case, where appending is the intended merge — or a + // stale address, where appending would invent an element. The + // payload's own identity decides which: it must name the element + // the path addresses. `Add` always appends. + if idx_opt.is_none() && matches!(op, DeltaOp::Update) { + // Build failure means the value could never be applied + // anyway: report the path instead of erroring the patch. + let Some(new_child) = build_or_fail(build_child) else { + return Ok(false); + }; + let allowed = match element_identity_label(&new_child) { + // Identity present: only the address that names it may + // append. Blocks stale positional Updates into a keyed + // list, which used to overwrite the wrong element. The + // comparison is the resolver's, so an address that WOULD + // have resolved to this element had it still been there + // is the address that may re-add it — a segment spelled + // as a CURIE against an IRI-expanded label included. + Some(label) => segment_matches_label(&new_child, Some(&label), key), + // No identity to check (scalar element, unkeyed range): + // only a positional or emptied list may grow this way. + None => !list_is_keyed_shaped(values), + }; + if !allowed { + return Ok(false); + } + mark_added_subtree(&new_child, trace); + values.push(new_child); + trace.updated.push(owner_id); + return Ok(true); + } + let Some(new_child) = build_or_fail(build_child) else { + return Ok(false); + }; if let Some(idx) = idx_opt { let existing = &mut values[idx]; if should_skip_update(existing, &new_child, opts) { @@ -801,10 +1284,14 @@ fn apply_delta_root( let (class_opt, slot_opt) = current_class_and_slot(current); if let Some(cls) = class_opt { let slot_clone = slot_opt.clone(); - let new_node = with_converter(schema_view, v, move |value, sv, conv| { - LinkMLInstance::from_json(value, cls, slot_clone, sv, conv, false) - .into_instance_tolerate_errors() - })?; + let Some(new_node) = build_or_fail(|| { + with_converter(schema_view, v, move |value, sv, conv| { + LinkMLInstance::from_json(value, cls, slot_clone, sv, conv, false) + .into_instance_tolerate_errors() + }) + }) else { + return Ok(false); + }; mark_added_subtree(&new_node, trace); *current = new_node; Ok(true) @@ -818,10 +1305,14 @@ fn apply_delta_root( let (class_opt, slot_opt) = current_class_and_slot(current); if let Some(cls) = class_opt { let slot_clone = slot_opt.clone(); - let new_node = with_converter(schema_view, v, move |value, sv, conv| { - LinkMLInstance::from_json(value, cls, slot_clone, sv, conv, false) - .into_instance_tolerate_errors() - })?; + let Some(new_node) = build_or_fail(|| { + with_converter(schema_view, v, move |value, sv, conv| { + LinkMLInstance::from_json(value, cls, slot_clone, sv, conv, false) + .into_instance_tolerate_errors() + }) + }) else { + return Ok(false); + }; if should_skip_update(current, &new_node, opts) { return Ok(true); } @@ -871,6 +1362,12 @@ fn apply_delta_object( ); } if let Some(child) = values.get_mut(key) { + let slot = class.slots().iter().find(|s| s.name == *key); + if slot.is_some_and(slot_is_opaque) { + // The path descends below an opaque slot: it addresses structure + // the slot does not expose. Report, never guess. + return Ok(false); + } return apply_delta_linkml_inner(child, &path[1..], op, newv, trace, opts); } Ok(false) @@ -888,10 +1385,20 @@ fn apply_delta_mapping( trace: &mut PatchTrace, opts: PatchOptions, ) -> LResult { + // A non-empty path here addresses *inside* the mapping, which is below the + // mapping's own slot. Reached when the patched root is itself a mapping. + if slot_is_opaque(slot) { + return Ok(false); + } let key = &path[0]; if path.len() == 1 { let value = newv.cloned().unwrap_or(JsonValue::Null); let slot_clone = slot.clone(); + // The delta's own path segment is the entry's dict key, and rule 5 + // makes that key the element's key-slot value: a delta that adds an + // entry must build the same object the loader would have built for the + // same key. + let key_clone = key.clone(); return apply_hashmap_leaf_delta( values, key, @@ -904,6 +1411,7 @@ fn apply_delta_mapping( let mut diags = ValidationResultSink::default(); let value = LinkMLInstance::build_mapping_entry_for_slot( &slot_clone, + &key_clone, val, sv, conv, @@ -937,13 +1445,18 @@ fn apply_delta_list( trace: &mut PatchTrace, opts: PatchOptions, ) -> LResult { + // A non-empty path here addresses *inside* the list, which is below the + // list's own slot. Reached when the patched root is itself a list. + if slot_is_opaque(slot) { + return Ok(false); + } let key = &path[0]; - let idx_opt = resolve_list_index(values, key); + let idx_opt = resolve_list_segment(values, key); if path.len() == 1 { let value = newv.cloned().unwrap_or(JsonValue::Null); let slot_clone = slot.clone(); let class_clone = class.clone(); - return apply_list_leaf_delta(values, idx_opt, owner_id, trace, opts, op, || { + return apply_list_leaf_delta(values, idx_opt, key, owner_id, trace, opts, op, || { with_converter(schema_view, value, move |val, sv, conv| { let mut diags = ValidationResultSink::default(); let value = LinkMLInstance::build_list_item_for_slot( diff --git a/src/runtime/src/identity_lint.rs b/src/runtime/src/identity_lint.rs new file mode 100644 index 0000000..f18d934 --- /dev/null +++ b/src/runtime/src/identity_lint.rs @@ -0,0 +1,950 @@ +//! Opt-in element-identity linter. +//! +//! Answers, per multivalued inlined slot: **where does element identity come +//! from?** A key (or identifier) on the element class, a composed key +//! declared with `unique_keys`, or the `diff.linkml.io/opaque` annotation +//! (nowhere — the slot's value is replaced as a whole). A slot with none of +//! these produces positional deltas, which are ambiguous when several sources +//! produce deltas for the same object concurrently. +//! +//! Neither entry point is wired into loading or [`crate::validate_issues`]: +//! projects that do not opt in keep today's inferred semantics. +//! +//! When a slot is flagged, the data-model author has four options (worked +//! examples in `docs/superpowers/specs/2026-08-17-inlined-multivalued-element-identity-design.md`): +//! 1. declare a `key`/`identifier` slot on the element class; +//! 2. declare the identity as `unique_keys` (composed keys supported); +//! 3. annotate the slot `diff.linkml.io/opaque` — replace the value as a whole; +//! 4. remodel, when one class would need two identity answers. +//! +//! A slot annotated `diff.linkml.io/ignore` is never flagged either, for a +//! different reason: diff skips such a slot entirely, so it produces no deltas +//! and has no element identity to declare. `ignore` silences the lint by +//! removing the slot from diff's scope; `opaque` silences it by answering the +//! question with "nowhere — replace the value as a whole". +//! +//! # The rules +//! +//! [`lint_element_identity`] asks five questions of a schema, and +//! [`lint_instance_identity`] two of a loaded instance. All seven warn; none +//! errors, and none changes what the engine does. +//! +//! ## Schema rules +//! +//! 1. **No declared identity.** A multivalued inlined list whose element class +//! declares no key/identifier and no `unique_keys` (or whose range is not a +//! class at all). Its deltas are positional, and positional deltas are +//! ambiguous when several sources produce deltas for one object +//! concurrently. This is the rule the four options above answer. +//! 2. **The identity is the type designator.** The element class's key (or +//! identifier) is its own `designates_type` slot, whose value describes the +//! class and not the element: constant across a homogeneous list, one value +//! per subtype across a polymorphic one. The engine ignores such a key +//! outright (spec addendum rule 1), so the slot also matches rule 1's shape +//! (nothing else is declared) or rule 3's (the `unique_keys` the key used to +//! shadow). This is the sharpest diagnosis and is asked first, so a +//! designator-keyed slot yields exactly one warning — this one. The dict +//! form of the same class is left alone: a mapping keyed by the designator +//! legitimately says at-most-one-element-per-subtype. +//! 3. **Several `unique_keys` to choose from.** Which of them provides the +//! identity? Only the name-sorted first does (declaration order is not +//! preserved by the metamodel), so a class offering several has an +//! alphabetically-decided identity, and adding an earlier-sorting entry +//! silently re-addresses every delta path for every slot ranged on it. The +//! candidates are counted across the range class **and every class +//! descending from it**, since a list ranged on a class holds elements of +//! all of them. +//! 4. **A split label space.** Those same classes are labelled *different +//! ways*: `Gadget` elements by one `unique_keys` entry and +//! `Widget is_a Gadget` elements by another, in one list — or one of them by +//! a `key` and the other by an entry, which splits the space just as +//! thoroughly. A path written against one label space cannot address an +//! element of the other, and two elements labelled different ways can carry +//! the same label without violating either class's uniqueness constraint. +//! Rule 3 and this one ask different questions of the same family: rule 3 is +//! about which *entry* is chosen, so a key-labelled class is outside it, +//! while this one is about which *declaration* labels each element, so a +//! key-labelled class is one of the groups. +//! 5. **A shared `class_uri` under a designator.** Two classes of one `is_a` +//! hierarchy declare the same `class_uri` while the hierarchy designates its +//! type. A designator value is a class URI, so it names both classes at +//! once and the loader resolves it to one of them — stably, but by an +//! ordering the schema does not state. Warning only: the loader's choice is +//! deliberately unchanged. +//! +//! Slot warnings are reported at the class that **introduces** the slot: a +//! flagged slot inherited unchanged by a descendant is not repeated there, +//! since the declaration the author would edit lives on the ancestor. Each rule +//! gates on its own predicate, so a subclass whose `slot_usage` changes one +//! rule's answer is judged on its own merits for that rule — and each of rules +//! 1, 3 and 4 subtracts rule 2's cases from its gate, because a designator-keyed +//! range class matches all three raw shapes while rule 2 is the slot's only +//! voice: a parent that emitted no warning must not suppress a subclass's. +//! Rule 5 is class-level and has no slot to attribute, so it is emitted once per +//! (hierarchy, shared URI) instead, and its subject is the sharing classes +//! behind a `shared_class_uri` marker — every other rule's subject is +//! `[class, slot]`, and a consumer joining the segments could not otherwise +//! tell the two apart. +//! +//! ## Instance rules +//! +//! 6. **Repeated identity.** Two elements of one list carry the same identity +//! label: a delta addressing it cannot say which is meant. +//! 7. **Positional despite a declared identity.** Some element of an inlined +//! list yields no label — the element class declares an identity, and the +//! data leaves the slot it names empty — so the list is not keyed-shaped and +//! is addressed positionally after all. Nothing here is invalid (a `key` +//! that is not `required` may legitimately be absent), so no schema rule and +//! no validation can see it; only the data can. +//! +//! Rule 6 does not consult `diff.linkml.io/opaque` or `ignore`: a repeated +//! identity contradicts the element class's own constraint, which is +//! class-level truth that diff vocabulary on the slot cannot suppress. Rule 7 +//! does honour both, and skips reference lists, because it claims only that the +//! list is addressed positionally — a claim those slots make false rather than +//! excused. +//! +//! # Sharp edge, deliberately not its own rule +//! +//! A subclass may switch the designator off with +//! `slot_usage: { theSlot: { designates_type: false } }` while the slot stays +//! (or becomes) the class's `key`. The override is respected, so the designator +//! machinery stops filling the slot — and nothing else fills it either. The +//! class then declares a key that no element ever carries a value for, which is +//! not a schema defect the schema can be read for: the declaration is +//! well-formed, and only the data shows that the key is always empty. It +//! collapses into instance rule 7, which is where it is reported. + +use crate::diff::{ + element_identity_label, identity_key_slot, slot_is_ignored, slot_is_opaque, OPAQUE_ANNOTATION, +}; +use crate::{LinkMLInstance, ValidationProblemType, ValidationResult, ValidationResultSink}; +use linkml_schemaview::identifier::Identifier; +use linkml_schemaview::schemaview::{ClassView, SchemaView}; +use linkml_schemaview::slotview::SlotView; +use std::collections::{BTreeMap, BTreeSet, HashMap, HashSet}; + +/// The shape every schema rule below is asked of: an inlined list whose +/// per-element delta paths are diff's to address, and which therefore has to +/// answer "where does element identity come from?". +/// +/// A dict is addressed by its keys, a reference list by the referents' own +/// identifiers, an `opaque` slot by nothing (the value is replaced whole) and +/// an `ignore`d slot not at all. None of them can be mis-addressed by a +/// mis-declared element identity, so none of them is any rule's business. +fn slot_addresses_elements_by_position_or_label(slot: &SlotView) -> bool { + use linkml_schemaview::slotview::{SlotContainerMode, SlotInlineMode}; + slot.determine_slot_container_mode() == SlotContainerMode::List + && slot.determine_slot_inline_mode() != SlotInlineMode::Reference + && !slot_is_opaque(slot) + && !slot_is_ignored(slot) +} + +/// Whether this is a multivalued inlined slot whose element identity comes from +/// nowhere — the engine's answer, from which the reporting loop subtracts the +/// designator case (see [`slot_is_identity_less_only`]). +/// +/// Split out from the reporting loop because the same question has to be asked +/// of an inherited slot on its parent class, to decide which class introduced +/// the problem. +fn slot_lacks_element_identity(slot: &SlotView) -> bool { + if !slot_addresses_elements_by_position_or_label(slot) { + return false; + } + if let Some(rc) = slot.get_range_class() { + // The engine's notion of a key, not the metamodel's: a key that is the + // class's type designator identifies the class, never the element, and + // `element_identity_label` looks straight past it (spec addendum rule + // 1). Such a class does lack element identity — but the designator rule + // below diagnoses it more precisely and is its only voice, so the + // reporting loop asks that question first. + if identity_key_slot(&rc).is_some() || !rc.unique_keys().is_empty() { + return false; + } + } + true +} + +/// The `unique_keys` entries a class offers as element identity, name-sorted. +/// +/// An entry with no slots names nothing and can never be load-bearing, so it is +/// not a candidate and does not make an otherwise-single-entry class ambiguous. +fn identity_unique_key_names(rc: &ClassView) -> Vec { + rc.unique_keys() + .into_iter() + .filter(|(_, uk)| !uk.unique_key_slots.is_empty()) + .map(|(name, _)| name) + .collect() +} + +/// The classes whose `unique_keys` can label an element of a list ranged on +/// `rc`: `rc` itself, and every class descending from it. +/// +/// A list ranged on a class holds elements of every class descending from it, +/// and each element is labelled by its **own** merged `unique_keys` — so both +/// `unique_keys` rules below are questions about the whole family, not about +/// the one declaration the slot's range happens to name (spike D4d). +/// +/// Mixin users are excluded (`include_mixins: false`): applying a mixin does +/// not make a class an instance of it, so it cannot put that class into a list +/// ranged on the mixin. +/// +/// A failure to resolve the descendants degrades to "no descendants" rather +/// than to a panic or a swallowed rule: the family always contains `rc`, so the +/// rules keep the pre-D4d answer instead of disappearing. +fn identity_class_family(rc: &ClassView) -> Vec { + let mut family = vec![rc.clone()]; + if let Ok(descendants) = rc.get_descendants(true, false) { + family.extend(descendants); + } + family +} + +/// The `unique_keys` entry this class's elements are actually labelled by: +/// the name-sorted first entry that names any slots, mirroring +/// `crate::diff::element_unique_key_label`. +/// +/// `None` when the class's identity does not come from `unique_keys` at all — +/// either it has a (non-designator) key/identifier, which outranks every entry, +/// or it declares no usable entry and its elements go unlabelled. +fn load_bearing_unique_key(rc: &ClassView) -> Option { + if identity_key_slot(rc).is_some() { + return None; // the key outranks unique_keys entirely + } + identity_unique_key_names(rc).into_iter().next() +} + +/// Whether this slot's element identity is *ambiguous* rather than absent: +/// the range class and its descendants offer several `unique_keys` entries to +/// derive it from, and only the name-sorted first is load-bearing. +/// +/// Returns the range class name, the name-sorted candidate entries, and the one +/// the range class *itself* resolves to — which is the first candidate unless a +/// descendant declares an earlier-sorting entry, the split case the divergence +/// rule is the voice for. This flags slots the identity-less rule passes: the +/// identity exists, but which of the declarations provides it was decided +/// alphabetically rather than by the author. +/// +/// The candidates are unioned over [`identity_class_family`], because an entry +/// a descendant adds is an entry some element of the list is really labelled by +/// (spike D4d). Family members whose identity is a key are skipped: their +/// entries are never load-bearing, so adding one to them re-addresses nothing. +/// +/// A range class with a `key`/`identifier` slot is not ambiguous however many +/// `unique_keys` it declares, for the same reason. A key that is the class's +/// type designator outranks nothing — the engine looks past it — so such a +/// class is judged on its `unique_keys` like any other; the designator rule +/// speaks for it first regardless, since that is the defect worth reporting. +fn slot_has_ambiguous_unique_keys(slot: &SlotView) -> Option<(String, Vec, String)> { + if !slot_addresses_elements_by_position_or_label(slot) { + return None; + } + let rc = slot.get_range_class()?; + // The range class's own answer. `None` means its elements carry no label at + // all, which is the identity-less rule's business, not this one's. + let own = load_bearing_unique_key(&rc)?; + let mut names: BTreeSet = BTreeSet::new(); + for cv in identity_class_family(&rc) { + if identity_key_slot(&cv).is_some() { + continue; + } + names.extend(identity_unique_key_names(&cv)); + } + if names.len() < 2 { + return None; + } + Some((rc.name().to_string(), names.into_iter().collect(), own)) +} + +/// How one class labels its elements, as the divergence rule compares it: the +/// rendered description of the declaration `element_identity_label` reads. +/// +/// `None` for a class whose elements carry no label at all — that is the +/// identity-less rule's business, and an unlabelled class does not occupy a +/// label space to be split from anyone. +/// +/// A key and a `unique_keys` entry are *different* labellings even when they +/// read the same slot, so they are rendered differently and never collide as +/// group names. This mirrors `element_identity_label`'s precedence exactly: the +/// key first, the name-sorted first entry otherwise. +fn identity_labelling(rc: &ClassView) -> Option { + if let Some(key) = identity_key_slot(rc) { + let kind = if key.definition().identifier == Some(true) { + "identifier" + } else { + "key" + }; + return Some(format!("{kind} '{}'", key.name)); + } + let entry = identity_unique_key_names(rc).into_iter().next()?; + Some(format!("unique_keys entry '{entry}'")) +} + +/// Per distinct identity labelling, the classes of one range class's family +/// that resolve to it. Labelling-sorted, each class list name-sorted. +type LabelSpaceGroups = Vec<(String, Vec)>; + +/// Whether the classes a list ranged on this slot can hold resolve **different** +/// identity labellings: one list, two label spaces (spike D4b/d). +/// +/// Returns the range class name and, per distinct labelling, the name-sorted +/// classes resolving to it. +/// +/// This is a sharper defect than the several-entries one and is reported *in +/// addition* to it. `Gadget{gadget_identity:[code]}` with +/// `Widget is_a Gadget{aaa_widget_identity:[serial]}` labels its `Gadget` +/// elements by `code` and its `Widget` elements by `serial`, in one list: +/// navigating by a base-class label never finds a `Widget`, and a `Widget` +/// serial colliding with a `Gadget` code produces two elements with one label +/// while violating neither class's constraint. Namespacing labels by the entry +/// that produced them is the deep fix and is out of scope for this branch, so +/// the author is told instead. +/// +/// A family member whose identity is a `key` is counted as its own group, not +/// skipped. Its `unique_keys` entries are indeed never load-bearing — which is +/// why the several-entries rule ignores them — but the key itself labels its +/// elements, and it splits the list's label space exactly as a second entry +/// would: `Plate` elements addressed by a `plate_identity` value and +/// `StampedPlate` elements by a `stampId` collide across the two spaces without +/// breaking either class's constraint. The two rules ask different questions of +/// the same family, and only one of them is about entry *choice*. +fn slot_has_split_identity_label_space(slot: &SlotView) -> Option<(String, LabelSpaceGroups)> { + if !slot_addresses_elements_by_position_or_label(slot) { + return None; + } + let rc = slot.get_range_class()?; + let mut by_labelling: BTreeMap> = BTreeMap::new(); + for cv in identity_class_family(&rc) { + if let Some(labelling) = identity_labelling(&cv) { + by_labelling + .entry(labelling) + .or_default() + .push(cv.name().to_string()); + } + } + if by_labelling.len() < 2 { + return None; + } + let mut groups: LabelSpaceGroups = by_labelling.into_iter().collect(); + for (_, classes) in groups.iter_mut() { + classes.sort(); + classes.dedup(); + } + Some((rc.name().to_string(), groups)) +} + +/// Whether this slot's element identity, though declared, cannot discriminate +/// between the elements of a list: the range class's key (or identifier) is its +/// type designator, whose value is fixed per class. +/// +/// Returns the range class name and the designator slot's name. +/// +/// Only asked of the list form. The dict form of the same class is a different, +/// legitimate model — a mapping keyed by the designator says at-most-one +/// element per subtype — so it is deliberately left alone. +/// +/// This is the sharpest of the three rules and speaks first: the engine ignores +/// a designator key outright, so the same slot also matches the identity-less +/// shape (when the class declares no `unique_keys`) or the several-`unique_keys` +/// shape (the entries the key used to shadow), and only this rule names the +/// declaration the author would actually edit. +fn slot_identity_is_type_designator(slot: &SlotView) -> Option<(String, String)> { + // Includes "not the dict form": a mapping keyed by the designator is + // legitimate, and says at-most-one-element-per-subtype. + if !slot_addresses_elements_by_position_or_label(slot) { + return None; + } + let rc = slot.get_range_class()?; + // Asked of the metamodel's key, not the engine's: this rule exists to + // report the key `element_identity_label` deliberately looks past. + let key = rc.key_or_identifier_slot()?; + if key.definition().designates_type != Some(true) { + return None; + } + Some((rc.name().to_string(), key.name.clone())) +} + +/// The identity-less rule as the reporting loop applies it. +/// +/// A designator-keyed class declaring no `unique_keys` does lack element +/// identity, but the designator rule diagnoses it precisely and is its only +/// voice. Subtracting that case here keeps the "flagged for the same reason" +/// contract of [`introduces_flagged_slot`] honest: a subclass whose `slot_usage` +/// swaps a designator-keyed range for a bare one is still judged on its own +/// merits. +fn slot_is_identity_less_only(slot: &SlotView) -> bool { + slot_lacks_element_identity(slot) && slot_identity_is_type_designator(slot).is_none() +} + +/// The several-entries rule as the reporting loop applies it. +/// +/// Same subtraction, same reason as [`slot_is_identity_less_only`], and needed +/// for the same reason it is: the designator key stopped shadowing its class's +/// `unique_keys` (spec addendum rule 1), so a designator-keyed range class with +/// several entries satisfies the raw shape while the designator rule remains +/// the slot's only voice. +/// +/// Used only as [`introduces_flagged_slot`]'s predicate, where the omission +/// loses a real warning: a parent whose slot fires the designator rule would +/// otherwise suppress a subclass that `slot_usage`-retargets the slot onto a +/// keyless multi-entry class, and the ambiguity would be reported nowhere. The +/// reporting loop's own call site does not need it — a designator-keyed slot +/// has already `continue`d by then. +fn slot_has_ambiguous_unique_keys_only(slot: &SlotView) -> bool { + slot_has_ambiguous_unique_keys(slot).is_some() + && slot_identity_is_type_designator(slot).is_none() +} + +/// The divergence rule as the reporting loop applies it, subtracted exactly as +/// [`slot_has_ambiguous_unique_keys_only`] is and for the same reason. +fn slot_has_split_identity_label_space_only(slot: &SlotView) -> bool { + slot_has_split_identity_label_space(slot).is_some() + && slot_identity_is_type_designator(slot).is_none() +} + +/// The warning text for a list whose identity is its element class's type +/// designator. +fn type_designator_identity_detail( + class_name: &str, + slot_name: &str, + range_class: &str, + designator: &str, +) -> String { + format!( + "elements of '{class_name}.{slot_name}' declare their identity as \ + '{range_class}.{designator}', which is the type designator \ + (designates_type). Its value is a function of the element's class, not \ + of the element: constant across a homogeneous list, and one value per \ + subtype across a polymorphic one. It is therefore never element \ + identity, and the diff engine ignores the key outright — the list is \ + addressed by the element class's unique_keys if it declares any and \ + those labels are unique within the list, positionally otherwise. \ + Declare an identity that varies per element: a \ + discriminating key/identifier or unique_keys on '{range_class}', or \ + range the list on a bare element class that does not declare the \ + designator as its key — or use the dict form instead, if \ + at-most-one-element-per-subtype is what the key really means." + ) +} + +/// Whether `class` is where a flagged slot should be reported, rather than an +/// ancestor it merely inherits the problem from. +/// +/// Answers no only when the direct `is_a` parent carries a slot of the same +/// name that is flagged *for the same reason*. Applied at every level this +/// leaves exactly the topmost flagged declarer, and it keeps a subclass whose +/// `slot_usage` changes the answer — narrowing the range to a keyed class, or +/// widening it away from one — judged on its own merits in both directions. +/// +/// Only the `is_a` chain is walked. A slot arriving from a mixin is reported on +/// the class using the mixin as well as on the mixin itself: a mixin can be +/// applied to unrelated classes, so there is no single owning declaration to +/// point at, and chasing one is not worth the complexity. +fn introduces_flagged_slot(class: &ClassView, slot_name: &str, flagged: F) -> bool +where + F: Fn(&SlotView) -> bool, +{ + let Ok(Some(parent)) = class.parent_class() else { + return true; // no is_a parent: this class is the declarer + }; + let Some(parent_slot) = parent.slot(&Identifier::new(slot_name)) else { + return true; // the parent does not have it: introduced here + }; + !flagged(&parent_slot) +} + +/// The warning text for a range class family offering several `unique_keys`. +/// +/// `names` is the name-sorted union over the family; `own` is the entry the +/// range class itself resolves to, which is `names[0]` unless a descendant +/// declares an earlier-sorting entry — the split case, which the divergence +/// warning is the voice for. +fn ambiguous_unique_keys_detail( + class_name: &str, + slot_name: &str, + range_class: &str, + names: &[String], + own: &str, +) -> String { + let quoted: Vec = names.iter().map(|n| format!("'{n}'")).collect(); + format!( + "elements of '{}.{}' take their identity from the unique_keys of \ + element class '{}' and of the classes descending from it, which \ + between them declare {} candidate entries: {}. Only one of them can be \ + load-bearing for a given element — the metamodel does not preserve \ + declaration order, so the name-sorted first entry that element's own \ + class offers is used, which for '{}' itself is '{}'. Every delta path \ + for this slot is addressed by it, and adding an earlier-sorting entry \ + anywhere in the family silently re-addresses them. Keep one entry, or \ + rename deliberately.", + class_name, + slot_name, + range_class, + names.len(), + quoted.join(", "), + range_class, + own, + ) +} + +/// The warning text for a range class family whose members resolve different +/// identity labellings. +/// +/// `groups` is labelling-sorted, and each group's classes are name-sorted, so +/// the text is stable across runs. +fn split_label_space_detail( + class_name: &str, + slot_name: &str, + range_class: &str, + groups: &[(String, Vec)], +) -> String { + let described: Vec = groups + .iter() + .map(|(labelling, classes)| { + // Bounded: a wide hierarchy must not turn one warning into a wall. + // Three names name the split; the rest are counted. + let shown: Vec = classes.iter().take(3).map(|c| format!("'{c}'")).collect(); + let rest = classes.len().saturating_sub(shown.len()); + let more = if rest > 0 { + format!(" and {rest} more") + } else { + String::new() + }; + format!("{labelling} ({}{})", shown.join(", "), more) + }) + .collect(); + format!( + "elements of '{}.{}' do not share one identity label space: a list \ + ranged on '{}' holds elements of every class descending from it, and \ + they are labelled {} different ways — {}. Each element is labelled by \ + the declaration its own class resolves to, so a delta path written \ + against one of them cannot address an element of another, and two \ + elements labelled different ways can produce the same label without \ + either class's uniqueness constraint being violated. Declare the \ + identity once, on '{}', so every element of the list is labelled the \ + same way.", + class_name, + slot_name, + range_class, + groups.len(), + described.join("; "), + range_class, + ) +} + +/// How deep the linter walks an `is_a` chain looking for a hierarchy root. +/// +/// A valid schema's `is_a` graph is a tree, so the bound is never reached; it +/// exists so that a cyclic one fails schema validation rather than hanging the +/// linter that was asked to explain it. +const MAX_IS_A_DEPTH: usize = 100; + +/// The topmost `is_a` ancestor of `class` — the class that names its hierarchy. +fn hierarchy_root(class: &ClassView) -> ClassView { + let mut current = class.clone(); + for _ in 0..MAX_IS_A_DEPTH { + match current.parent_class() { + Ok(Some(parent)) => current = parent, + _ => break, + } + } + current +} + +/// First subject segment of every rule-5 warning, marking what the segments +/// after it are. +/// +/// Every other rule in this module has a subject of the form +/// `[class_name, slot_name]`, and both CLIs render a subject by joining it with +/// `.`. Rule 5's subject is a *list of classes*, so two of them printed as +/// `Alpha.Beta` — a slot warning about `Beta` on class `Alpha`, as far as any +/// reader or any consumer grouping findings by subject could tell. The marker +/// makes the subject say which rule produced it; it cannot collide with a class +/// name, since it is not one. +const SHARED_CLASS_URI_SUBJECT: &str = "shared_class_uri"; + +/// The warning text for two classes of one hierarchy sharing a `class_uri`. +fn shared_class_uri_detail(root: &str, uri: &str, classes: &[String], designator: &str) -> String { + let quoted: Vec = classes.iter().map(|c| format!("'{c}'")).collect(); + format!( + "{} declare the same class_uri '{}' and belong to one is_a hierarchy, \ + rooted at '{}', which designates its type through '{}' \ + (designates_type). A designator value is a class URI, so '{}' names \ + every one of them at once: the loader resolves it to a single class, \ + stably but by an ordering the schema does not state and nothing \ + promises to keep. Instances meaning any of the others load as the \ + winner without a diagnostic, and diff then pairs elements of different \ + classes as if they were one. Give each class a distinct class_uri. The \ + loader's choice is deliberately unchanged — this is a warning about \ + the declarations, not a behavioural fix.", + quoted.join(" and "), + uri, + root, + designator, + uri, + ) +} + +/// The warning text for a list addressed positionally although its element +/// class declares an identity: some element leaves the identity slot empty. +fn missing_labels_detail( + missing: usize, + total: usize, + range_class: &str, + identity: &str, +) -> String { + format!( + "{missing} of {total} elements of this list carry no identity label, \ + although their element class '{range_class}' declares one ({identity}). \ + A list is addressed by identity only when every element yields a label, \ + so this one is addressed positionally — ambiguous under multi-sourced \ + operation, and silently so, since an optional identity slot left empty \ + is valid data. Fill the identity slot on every element, make it \ + required, or declare the slot {OPAQUE_ANNOTATION} if the value is meant \ + to be replaced as a whole." + ) +} + +/// How the range class declares element identity, for the warning above. +/// +/// The same question [`identity_labelling`] answers for the divergence rule — +/// "which declaration does `element_identity_label` read?" — so it is the same +/// function, and `None` means the same thing in both: no identity is declared. +fn declared_identity_description(rc: &ClassView) -> Option { + Some(format!("its {}", identity_labelling(rc)?)) +} + +/// Schema-level, class-level rule: two classes of one `is_a` hierarchy declare +/// the same `class_uri`, and the hierarchy carries a type designator (spike D8). +/// +/// Unlike the slot rules, this one has no "introducing class" to report at: +/// there is no slot, and the defect belongs to the pair of declarations rather +/// than to either of them. Emitting once per (hierarchy, shared URI) buys the +/// same thing [`introduces_flagged_slot`] buys the others — one warning per +/// thing the author would edit — so no gate predicate is needed. +/// +/// The URI compared is the class's canonical one, which is the `class_uri` when +/// declared and the schema-derived default otherwise. Two defaults can never +/// collide (they are derived from the class name), so a collision always means +/// at least one explicit declaration. +/// +/// The subject is [`SHARED_CLASS_URI_SUBJECT`] followed by the sharing classes: +/// see that constant for why the marker is there. +fn lint_shared_class_uris(classes: &[ClassView], sink: &mut ValidationResultSink) { + let mut hierarchies: BTreeMap<(String, String), Vec<&ClassView>> = BTreeMap::new(); + for class in classes { + let root = hierarchy_root(class); + hierarchies + .entry((root.schema_id().to_string(), root.name().to_string())) + .or_default() + .push(class); + } + for ((_, root_name), mut members) in hierarchies { + members.sort_by_key(|c| c.name().to_string()); + // The designator is declared once and inherited, so the first member + // carrying one names it for the whole hierarchy. + let Some(designator) = members + .iter() + .find_map(|c| c.get_type_designator_slot()) + .map(|d| d.name.clone()) + else { + continue; // nothing dispatches on a class URI here: not this defect + }; + let mut by_uri: BTreeMap> = BTreeMap::new(); + for member in &members { + by_uri + .entry(member.canonical_uri().to_string()) + .or_default() + .push(member.name().to_string()); + } + for (uri, mut names) in by_uri { + names.dedup(); + if names.len() < 2 { + continue; + } + let mut subject = vec![SHARED_CLASS_URI_SUBJECT.to_string()]; + subject.extend(names.iter().cloned()); + sink.push_warning( + ValidationProblemType::AmbiguousElementIdentity, + subject, + shared_class_uri_detail(&root_name, &uri, &names, &designator), + ); + } + } +} + +/// Schema-level lint: the module's rules 1–5 — an element identity that comes +/// from nowhere, one that is the range class's type designator and so cannot +/// tell the elements of a homogeneous list apart, one derived from a family of +/// classes offering more than one `unique_keys` entry, one where that family +/// resolves *different* entries, and two classes of a designator-carrying +/// hierarchy answering to the same `class_uri`. +/// +/// Warnings only — the schema stays usable. +pub fn lint_element_identity(sv: &SchemaView) -> Vec { + let mut sink = ValidationResultSink::default(); + let conv = sv.converter(); + let mut class_ids = sv.get_class_ids(); + class_ids.sort(); + let mut seen: HashSet<(String, String)> = HashSet::new(); + // Kept for the class-level rule below, which is not about any one slot and + // so cannot be answered inside the slot loop. + let mut visited: Vec = Vec::new(); + for class_id in class_ids { + let Ok(Some(class)) = sv.get_class(&Identifier::new(&class_id), &conv) else { + continue; + }; + // `get_class_ids` yields one id per class *URI*: a class declaring an + // explicit `class_uri` is indexed under both that and its default URI, + // so walking the ids naively reports each of its slots twice. + // + // Key the seen-set on (schema, name), which is unique per class by + // construction. Keying on the class URI would be wrong in the other + // direction: LinkML lets distinct classes declare the same `class_uri` + // (meta.yaml's `Anything` and extensions.yaml's `AnyValue` both declare + // `linkml:Any`), and that would silently drop the second class's + // warnings — a false negative, worse in a lint than a duplicate. + if !seen.insert((class.schema_id().to_string(), class.name().to_string())) { + continue; + } + for slot in class.slots() { + // Report at the class that introduces the slot: repeating an + // inherited warning on every descendant buries the one declaration + // the author would actually edit. Applies to all three rules below. + // The designator rule speaks first, and alone. Since the engine + // stopped accepting a designator as element identity, such a slot + // also matches the identity-less shape or the several-unique_keys + // shape, and one warning per slot means the sharpest one wins. + if let Some((rc_name, designator)) = slot_identity_is_type_designator(slot) { + if introduces_flagged_slot(&class, &slot.name, |s| { + slot_identity_is_type_designator(s).is_some() + }) { + sink.push_warning( + ValidationProblemType::AmbiguousElementIdentity, + vec![class.name().to_string(), slot.name.clone()], + type_designator_identity_detail( + class.name(), + &slot.name, + &rc_name, + &designator, + ), + ); + } + continue; + } + if !slot_is_identity_less_only(slot) { + if let Some((rc_name, names, own)) = slot_has_ambiguous_unique_keys(slot) { + if introduces_flagged_slot( + &class, + &slot.name, + slot_has_ambiguous_unique_keys_only, + ) { + sink.push_warning( + ValidationProblemType::AmbiguousElementIdentity, + vec![class.name().to_string(), slot.name.clone()], + ambiguous_unique_keys_detail( + class.name(), + &slot.name, + &rc_name, + &names, + &own, + ), + ); + } + } + // A split label space is a sharper defect than an ambiguous + // candidate set, and an additional one: both warnings fire, and + // each gets its own introduces-gate predicate, because a + // subclass can narrow the range to a non-diverging class while + // leaving the candidate set as wide as it was (and the other + // way round). + if let Some((rc_name, groups)) = slot_has_split_identity_label_space(slot) { + if introduces_flagged_slot( + &class, + &slot.name, + slot_has_split_identity_label_space_only, + ) { + sink.push_warning( + ValidationProblemType::AmbiguousElementIdentity, + vec![class.name().to_string(), slot.name.clone()], + split_label_space_detail(class.name(), &slot.name, &rc_name, &groups), + ); + } + } + continue; + } + if !introduces_flagged_slot(&class, &slot.name, slot_is_identity_less_only) { + continue; + } + let range_class = slot.get_range_class(); + // The advice has to fit the range: a scalar- or enum-ranged slot has + // no element class on which a key could be declared. + let detail = match &range_class { + Some(rc) => format!( + "elements of '{}.{}' have no declared identity: deltas are \ + positional and ambiguous under multi-sourced operation. \ + Declare a key/identifier or unique_keys on the element \ + class '{}', annotate the slot with {} to replace the value \ + as a whole, or remodel.", + class.name(), + slot.name, + rc.name(), + OPAQUE_ANNOTATION + ), + None => format!( + "elements of '{}.{}' have no declared identity: the range is \ + not a class, so deltas can only be positional, and they are \ + ambiguous under multi-sourced operation. Annotate the slot \ + with {} to replace the value as a whole, or remodel the \ + range into a class that declares a key/identifier or \ + unique_keys.", + class.name(), + slot.name, + OPAQUE_ANNOTATION + ), + }; + sink.push_warning( + ValidationProblemType::AmbiguousElementIdentity, + vec![class.name().to_string(), slot.name.clone()], + detail, + ); + } + visited.push(class); + } + lint_shared_class_uris(&visited, &mut sink); + let mut warnings = sink.into_vec(); + // The classes are visited in sorted id order, but a class's own slots come + // from `ClassView::slots()`, which is HashMap-backed, so the warnings for a + // single class arrive in an order that varies between runs. Sort here, once, + // so every consumer inherits a stable, diffable order. + warnings.sort_by(|a, b| a.subject.cmp(&b.subject)); + warnings +} + +/// Data-level lint: the module's rules 6 and 7 — a list whose elements repeat a +/// declared identity (key/identifier or `unique_keys` value), and one addressed +/// positionally although its element class declares an identity, because some +/// element leaves the slot that identity names empty. +/// +/// Both are things only the data can show: repeated and absent values are +/// alike valid against a schema that declares a non-`required` identity. +/// +/// The duplicate rule deliberately does NOT consult `diff.linkml.io/opaque`: +/// a schema constraint is class-level truth, and diff vocabulary never +/// suppresses it. The positional rule does honour it, for the reason spelled +/// out at [`check_missing_labels`]. +pub fn lint_instance_identity(value: &LinkMLInstance) -> Vec { + let mut sink = ValidationResultSink::default(); + let mut path = Vec::new(); + walk(value, &mut path, &mut sink); + sink.into_vec() +} + +fn walk(v: &LinkMLInstance, path: &mut Vec, sink: &mut ValidationResultSink) { + match v { + LinkMLInstance::List { values, slot, .. } => { + check_duplicates(values, path, sink); + check_missing_labels(values, slot, path, sink); + for (i, child) in values.iter().enumerate() { + path.push(i.to_string()); + walk(child, path, sink); + path.pop(); + } + } + LinkMLInstance::Object { values, .. } | LinkMLInstance::Mapping { values, .. } => { + // Name-sorted, so the warning order is stable across runs: the + // values are a `HashMap`, whose iteration order is not. + let ordered: BTreeMap<&String, &LinkMLInstance> = values.iter().collect(); + for (k, child) in ordered { + path.push(k.clone()); + walk(child, path, sink); + path.pop(); + } + } + LinkMLInstance::Scalar { .. } | LinkMLInstance::Null { .. } => {} + } +} + +/// Warns when this list is addressed positionally *although* its element class +/// declares an identity: some element leaves the identity slot empty and yields +/// no label, so the keyed shape fails (spike D7). +/// +/// The duplicate rule above is the other half of the same question. Together +/// they cover both ways a declared identity fails to address a list — repeated +/// labels and missing ones — and neither is visible in the schema: a `key` that +/// is not `required` may legitimately be absent, so only the data can show it. +/// The half-labelled list is the same defect at a smaller scale and is counted +/// the same way. +/// +/// Silent when nothing is declared: a class with no identity is *meant* to be +/// positional, the schema lint has already said so, and repeating it once per +/// container in the data would bury the rule that matters. Silent for a range +/// that is not a class, for the same reason. A key that is the class's type +/// designator does not count as a declaration either — the engine looks past it +/// (spec addendum rule 1), so the class declares no element identity at all and +/// the schema-level designator rule is its voice. +/// +/// Unlike the duplicate rule, this one **does** honour `opaque` and `ignore`, +/// and skips reference lists — it asks +/// [`slot_addresses_elements_by_position_or_label`] exactly as the schema rules +/// do. The two rules differ because they claim different things. A repeated +/// `unique_keys` value contradicts the class's own constraint whatever the slot +/// is annotated with, so diff vocabulary cannot silence it. This rule claims +/// only that the list *is addressed positionally*, and for a slot answering +/// "replaced as a whole" or "outside diff's scope" that claim is simply false. +/// A reference list is the sharper case: its elements are identifier strings +/// and can never carry an inlined element's identity label, so the rule would +/// fire on every such slot of every document — the shipped downstream corpus +/// has exactly one list matching it, and it is a reference list. +fn check_missing_labels( + values: &[LinkMLInstance], + slot: &SlotView, + path: &[String], + sink: &mut ValidationResultSink, +) { + if values.is_empty() { + return; // nothing to label, nothing to address + } + if !slot_addresses_elements_by_position_or_label(slot) { + return; // references, dicts, opaque and ignored slots: see above + } + let Some(rc) = slot.get_range_class() else { + return; // scalars and enums have no class to declare an identity on + }; + let Some(identity) = declared_identity_description(&rc) else { + return; // no identity declared: positional is what was asked for + }; + let missing = values + .iter() + .filter(|v| element_identity_label(v).is_none()) + .count(); + if missing == 0 { + return; + } + sink.push_warning( + ValidationProblemType::AmbiguousElementIdentity, + path.to_vec(), + missing_labels_detail(missing, values.len(), rc.name(), &identity), + ); +} + +fn check_duplicates(values: &[LinkMLInstance], path: &[String], sink: &mut ValidationResultSink) { + let mut seen: HashMap = HashMap::new(); + for v in values { + if let Some(label) = element_identity_label(v) { + *seen.entry(label).or_insert(0) += 1; + } + } + let mut dups: Vec<(String, usize)> = seen.into_iter().filter(|(_, n)| *n > 1).collect(); + dups.sort(); + for (label, n) in dups { + sink.push_warning( + ValidationProblemType::DuplicateElementIdentity, + path.to_vec(), + format!( + "{n} elements share the declared identity '{label}'; deltas \ + addressing it are ambiguous" + ), + ); + } +} diff --git a/src/runtime/src/lib.rs b/src/runtime/src/lib.rs index 65dea99..1122f02 100644 --- a/src/runtime/src/lib.rs +++ b/src/runtime/src/lib.rs @@ -26,6 +26,7 @@ use constraints::{run_object_constraints, run_slot_constraints}; pub mod blame; pub mod diff; +pub mod identity_lint; #[cfg(feature = "ttl")] pub mod rdf_export; #[cfg(feature = "ttl")] @@ -46,7 +47,10 @@ pub use blame::{ blame_map_to_paths, format_blame_map, format_blame_map_with, get_blame_info, patch_with_blame, record_blame_from_trace, }; -pub use diff::{diff, patch, Delta, DeltaOp, DiffOptions, PatchOptions, PatchTrace}; +pub use diff::{ + diff, patch, Delta, DeltaOp, DiffOptions, PatchOptions, PatchTrace, OPAQUE_ANNOTATION, +}; +pub use identity_lint::{lint_element_identity, lint_instance_identity}; #[derive(Debug)] pub struct LinkMLError { validation_issues: Vec, @@ -154,6 +158,37 @@ pub enum ValidationProblemType { SlotRangeViolation, MaxCountViolation, ParsingError, + /// Element identity that cannot address a list unambiguously: none is + /// declared, several declarations compete, two classes answer to one URI + /// (opt-in, [`crate::lint_element_identity`]) — or one is declared but the + /// data leaves it empty, so the list is positional after all (opt-in, + /// [`crate::lint_instance_identity`]). + AmbiguousElementIdentity, + /// Elements of one list repeat a declared identity (opt-in, + /// [`crate::lint_instance_identity`]). + DuplicateElementIdentity, +} + +impl ValidationProblemType { + /// The stable machine-readable name of this variant. + /// + /// Every machine surface — the Python binding's `problem_type`, the CLI's + /// JSON `type` — reports the value through this one function, so the two + /// cannot spell the same variant differently and a variant rename cannot + /// silently change either contract. `Debug` formatting is a Rust + /// implementation detail and is deliberately not it. + pub fn label(&self) -> &'static str { + match self { + Self::UndeclaredSlot => "undeclared_slot", + Self::InapplicableSlot => "inapplicable_slot", + Self::MissingSlotValue => "missing_slot_value", + Self::SlotRangeViolation => "slot_range_violation", + Self::MaxCountViolation => "max_count_violation", + Self::ParsingError => "parsing_error", + Self::AmbiguousElementIdentity => "ambiguous_element_identity", + Self::DuplicateElementIdentity => "duplicate_element_identity", + } + } } pub type InstancePath = Vec; @@ -534,8 +569,20 @@ impl LinkMLInstance { } } - /// Navigate the value by a path of strings, where each element is either - /// a dictionary key (for maps) or a list index (for lists). + /// Navigate the value by a path of strings, where each segment is a slot + /// name, a mapping key, or — for lists — whatever addresses an element: + /// its identity label (identifier/key slot value, or a value derived from + /// the range class's `unique_keys`) for a list whose elements all carry + /// unique labels, and a numeric index otherwise. + /// + /// List segments resolve through the same rule [`diff()`](crate::diff()) + /// emits and [`patch()`](crate::patch()) applies, so a delta path is + /// navigable by construction. + /// In particular a numeric segment aimed at a label-addressed list resolves + /// to nothing rather than to that position: when a label happens to be + /// `"0"`, position and label name different elements, and guessing which + /// one was meant is exactly the wrong-element hazard. + /// /// Returns `Some(&LinkMLInstance)` if the full path can be resolved, otherwise `None`. pub fn navigate_path(&self, path: I) -> Option<&LinkMLInstance> where @@ -550,42 +597,7 @@ impl LinkMLInstance { current = values.get(key)?; } LinkMLInstance::List { values, .. } => { - // Support either numeric index or identifier/key-based selection - if let Ok(idx) = key.parse::() { - current = values.get(idx)?; - } else { - // Attempt identifier-based lookup for object elements - let mut found: Option<&LinkMLInstance> = None; - for v in values.iter() { - if let LinkMLInstance::Object { - values: mv, class, .. - } = v - { - if let Some(id_slot) = class.key_or_identifier_slot() { - if let Some(LinkMLInstance::Scalar { value, .. }) = - mv.get(&id_slot.name) - { - match value { - JsonValue::String(sv) => { - if sv == key { - found = Some(v); - break; - } - } - other => { - let sv = other.to_string(); - if sv == key { - found = Some(v); - break; - } - } - } - } - } - } - } - current = found?; - } + current = values.get(diff::resolve_list_segment(values, key)?)?; } LinkMLInstance::Mapping { values, .. } => { current = values.get(key)?; @@ -856,10 +868,240 @@ impl LinkMLInstance { .unwrap_or_else(|| base.clone()) } + /// Rewrite an already-present type designator value to the canonical value + /// of the class that was actually selected — identity compares meaning, not + /// spelling (spec addendum rule 2, finding D1). + /// + /// A designator says "this element is a `Circle`", and a `uriorcurie`-ranged + /// one can say it as `canon:Circle`, as the expanded IRI, or not at all. + /// Stored verbatim, those are three different strings: the same element + /// authored by two producers diffs as a whole-element Remove + Add, and the + /// RDF loader — which never harvests the designator predicate and always + /// fills it from the class ([`Self::populate_type_designator`], called from + /// `turtle_import`) — disagrees with the JSON loader on every document. + /// Canonicalising at the boxing chokepoint, exactly as + /// [`Self::coerce_scalar_to_range`] does for the int/float ambiguity, makes + /// the representation canonical before any diff/equals/patch/`to_json` sees + /// it, and makes the two loaders agree by construction. + /// + /// It is also what makes `diff`'s changed-key check ("a changed designator + /// means a different class") unconditionally true rather than true only for + /// consistently-spelled data. + /// + /// A supplied value that is *no* accepted designator value of the selected + /// class (a bare class name against a `uri` range, a typo, a stale class + /// name) is data the loader cannot honour. Following the loader-tolerance + /// precedent it is a **warning**, not an error: the instance still loads and + /// the slot carries the canonical value of the class `select_class` picked. + /// + /// Only a `Scalar` is rewritten. An explicit `null` and a structurally wrong + /// value (list, object) are left for the range checks that already ran to + /// describe; replacing them would hide the shape of the offending data. + fn canonicalize_type_designator( + values: &mut HashMap, + class: &ClassView, + conv: &Converter, + path: &[String], + validation_issues: &mut ValidationResultSink, + ) { + let Some(type_slot_def) = class.get_type_designator_slot() else { + return; + }; + // Cheapest checks first: the two commonest cases — the designator is + // absent (the RDF-shaped and the omit-it-and-let-the-loader-fill-it + // documents), or it is already canonical (every document a previous + // load emitted) — must not pay for a URI computation. Neither + // `get_type_designator_value` nor `get_accepted_type_designator_values` + // is cached: each walks `type_ancestors` and expands URIs through the + // converter, and this runs once per object built. + let Some(LinkMLInstance::Scalar { value, .. }) = values.get_mut(&type_slot_def.name) else { + return; + }; + let supplied = match &*value { + JsonValue::String(s) => s.clone(), + other => other.to_string(), + }; + let Ok(canonical) = class.get_type_designator_value(type_slot_def, conv) else { + return; + }; + let canonical = canonical.to_string(); + if supplied == canonical { + return; + } + // Only a value that differs from the canonical one is worth the second + // walk. An `Err` here means the accepted set is unknown, not empty: + // warning-and-rewriting on it would punish the data for a schema the + // view cannot resolve, so leave the value alone — the same posture the + // canonical-value line above takes. + let Ok(accepted) = class.get_accepted_type_designator_values(type_slot_def, conv) else { + return; + }; + if !accepted.iter().any(|v| v.to_string() == supplied) { + let mut p = path.to_vec(); + p.push(type_slot_def.name.clone()); + validation_issues.push_warning( + ValidationProblemType::SlotRangeViolation, + p, + format!( + "type designator value `{supplied}` is not an accepted designator \ + value for class `{}`; stored as `{canonical}`", + class.name() + ), + ); + } + *value = JsonValue::String(canonical); + } + + /// Hear what an inlined-dict entry's key says when the payload also states + /// it (spec addendum rule 5, finding D5). + /// + /// Called only when the payload *did* supply the key slot — the omitted + /// case is filled from the dict key by the caller, and any complaint about + /// that injected value is then [`Self::canonicalize_type_designator`]'s to + /// make, in its own words, on the value that was actually stored. + /// + /// Two things are checked, and both are warnings, never errors: the + /// document loads, because the LinkML `inlined` contract makes both halves + /// legal on their own and only their *combination* is contradictory. + /// + /// 1. **Divergence.** The dict key and the element's key-slot value say + /// different things about the same slot. The payload is the element's + /// data and is what gets stored; the dict key is the address and is what + /// the mapping stays keyed by (`parse_mapping_slot` inserts under the raw + /// key). Neither can be dropped without losing information the document + /// contains, so both are named and the author decides. + /// + /// The comparison is against the value **as stored**, which is why this + /// runs after [`Self::canonicalize_type_designator`] and + /// [`Self::populate_type_designator`] rather than against the raw + /// payload. A designator payload is rewritten to its class's canonical + /// spelling at load, so comparing the raw payload would call + /// `{"": {"typeURI": ""}}` a divergence while the + /// two ends of the message — key and "the stored value" — had in fact + /// already been made to agree, and reloading the emitted document would + /// be silent. The rule is therefore: warn when the entry, *as it will be + /// written back out*, contradicts the key it is written under. + /// + /// Both sides go through [`crate::diff::canonical_identity_component`], + /// the same function every identity label is built from, so a CURIE key + /// and an expanded stored value are one identity (rule 2) rather than a + /// divergence. What survives is the genuinely contradictory case: + /// `{"": {"typeURI": ""}}`, where both spellings + /// name one class but only the payload is canonicalised, so the loaded + /// entry really is addressed by one IRI and carries another. That is + /// asset360's committed `SpotLocation_coordinates` shape. + /// + /// 2. **An unaccepted designator key.** When the key slot *is* the class's + /// type designator the dict key names the entry's class, and a key that + /// is no accepted designator value of the class actually selected names + /// nothing. Before this rule the case was silent twice over: nothing read + /// the key, and `populate_type_designator` filled the slot from the range + /// class, so the entry looked complete. + /// + /// The two checks compare **by different notions of equality**, and that is + /// deliberate rather than an oversight. Check 2 asks whether a value is in + /// the accepted designator set, and membership of that set is the raw string + /// match `canonicalize_type_designator` performs verbatim; deciding it here + /// on an IRI-equal-but-differently-spelled basis would wave a value through + /// that the canonicaliser then rewrites and reports itself — two voices for + /// one fact. Check 1 asks whether two spellings name the same element, which + /// is exactly what [`crate::diff::canonical_identity_component`] answers + /// everywhere else (spec addendum rule 2), so a CURIE key and an expanded + /// stored value are one identity here as they are in diff and patch. The + /// questions differ, so the equalities do. + /// + /// The key slot itself is [`ClassView::key_or_identifier_slot`] on the + /// slot's **range** class, not on the class finally selected: the key has to + /// be read (and injected) before `select_class` can run, so a key declared + /// only by a subclass — via `slot_usage` on a descendant — is chicken-and-egg + /// and gets neither the injection nor these checks. + fn reconcile_dict_key_with_payload( + entry_key: &str, + key_slot: &SlotView, + values: &HashMap, + selected: &ClassView, + conv: &Converter, + path: &[String], + validation_issues: &mut ValidationResultSink, + ) { + let mut p = path.to_vec(); + p.push(key_slot.name.clone()); + + if key_slot.definition().designates_type == Some(true) { + // Deliberately a raw string compare, unlike the divergence half + // below: the accepted set is what `canonicalize_type_designator` + // (Task 12, rule 2) matches against, verbatim, and a key that this + // check waved through on an IRI-equal-but-differently-spelled basis + // would then be rewritten by that function and reported by *it* + // instead — two voices for one fact. + // + // `Err` means the accepted set is unknown, not empty — the same + // posture `canonicalize_type_designator` takes: do not punish the + // data for a schema the view cannot resolve. + if let Some(td) = selected.get_type_designator_slot() { + if let Ok(accepted) = selected.get_accepted_type_designator_values(td, conv) { + if !accepted.iter().any(|v| v.to_string() == entry_key) { + validation_issues.push_warning( + ValidationProblemType::SlotRangeViolation, + p.clone(), + format!( + "inlined-dict key `{entry_key}` is not an accepted designator \ + value for class `{}`", + selected.name() + ), + ); + } + } + } + } + + // A `Null` and a structurally wrong value (list, object) are left for + // the range checks that already ran to describe; calling them a + // divergence would say the wrong thing about the wrong problem. + let Some(LinkMLInstance::Scalar { value, .. }) = values.get(&key_slot.name) else { + return; + }; + let stored = match value { + JsonValue::String(s) => s.clone(), + other => other.to_string(), + }; + if crate::diff::canonical_identity_component(entry_key, key_slot) + == crate::diff::canonical_identity_component(&stored, key_slot) + { + return; + } + validation_issues.push_warning( + ValidationProblemType::SlotRangeViolation, + p, + format!( + "inlined-dict key `{entry_key}` disagrees with `{stored}`, the value stored in \ + key slot `{}`; the payload is the element's data and the dict key is its \ + address, so the entry keeps both and stays addressed by its dict key", + key_slot.name + ), + ); + } + + /// The dict key of an inlined mapping entry, as the value its key slot + /// takes when the payload omitted it (spec addendum rule 5). + /// + /// A JSON object key is a string and is injected as one. A numerically + /// ranged key slot therefore receives the string spelling of its key: the + /// loader runs no scalar type check (see [`crate::constraints`] — enum, + /// regex and min/max only), and inventing a second, untested numeric parse + /// here would be a coercion path of its own rather than this rule. + fn dict_key_value(entry_key: &str) -> JsonValue { + JsonValue::String(entry_key.to_string()) + } + /// If the class has a `designates_type` slot and the values map does not /// already contain it, insert a Scalar value with the class's type /// designator value. This ensures round-trip fidelity for formats like /// JSON that lack an intrinsic typing mechanism (unlike RDF's rdf:type). + /// + /// The complement of [`Self::canonicalize_type_designator`], which settles + /// the case where the data *did* supply a value; JSON boxing runs both, RDF + /// harvesting only ever needs this one (it skips the designator predicate). fn populate_type_designator( values: &mut HashMap, class: &ClassView, @@ -934,6 +1176,7 @@ impl LinkMLInstance { path.clone(), validation_issues, ); + Self::canonicalize_type_designator(&mut values, class, conv, &path, validation_issues); Self::populate_type_designator(&mut values, class, sv, conv); Ok(LinkMLInstance::Object { node_id: new_node_id(), @@ -1024,6 +1267,7 @@ impl LinkMLInstance { for (k, v) in map.into_iter() { let child = Self::build_mapping_entry_for_slot( sl, + &k, v, sv, conv, @@ -1171,6 +1415,7 @@ impl LinkMLInstance { path.clone(), validation_issues, ); + Self::canonicalize_type_designator(&mut values, &chosen, conv, &path, validation_issues); Self::populate_type_designator(&mut values, &chosen, sv, conv); Ok(LinkMLInstance::Object { node_id: new_node_id(), @@ -1395,8 +1640,27 @@ impl LinkMLInstance { ) } + /// Build one entry of an inlined mapping. + /// + /// `entry_key` is the entry's dict key, which the LinkML `inlined` contract + /// makes the element's key/identifier *value* — real data, not just an + /// address (spec addendum rule 5, finding D5). It is injected into the key + /// slot when the payload omits it, which happens before class selection (so + /// a designator-keyed dict selects the class its key names), before the + /// object constraints run (so a `required` key slot is satisfied by the key + /// the document did supply, instead of erroring on legal data), and before + /// [`Self::canonicalize_type_designator`] (so an injected designator value + /// is canonicalised exactly as a payload-supplied one is). When the payload + /// does supply it, [`Self::reconcile_dict_key_with_payload`] speaks. + /// + /// The key slot is read from the slot's **range** class only. It has to be: + /// the key is injected *before* `select_class` runs, so consulting the + /// selected class would be circular. A key declared solely by a descendant + /// — `slot_usage: {k: {key: true}}` on a subclass, the range class having no + /// key of its own — therefore gets neither the injection nor the checks. pub(crate) fn build_mapping_entry_for_slot( map_slot: &SlotView, + entry_key: &str, value: JsonValue, sv: &SchemaView, conv: &Converter, @@ -1419,7 +1683,20 @@ impl LinkMLInstance { ) })?; match value { - JsonValue::Object(m) => { + JsonValue::Object(mut m) => { + let key_slot = range_cv.key_or_identifier_slot().cloned(); + // Alias-aware, because the payload may spell the slot by any + // name `slot_matches_key` accepts — the same lookup the child + // loop below uses, so "did the payload supply it?" and "which + // entry is it?" cannot disagree. + let payload_states_key = key_slot + .as_ref() + .is_some_and(|ks| m.keys().any(|ck| slot_matches_key(ks, ck))); + if let Some(ks) = &key_slot { + if !payload_states_key { + m.insert(ks.name.clone(), Self::dict_key_value(entry_key)); + } + } let selected = Self::select_class(&m, &range_cv, sv, conv); let mut child_values = HashMap::new(); for (ck, cv) in m.into_iter() { @@ -1455,7 +1732,30 @@ impl LinkMLInstance { path.clone(), validation_issues, ); + Self::canonicalize_type_designator( + &mut child_values, + &selected, + conv, + &path, + validation_issues, + ); Self::populate_type_designator(&mut child_values, &selected, sv, conv); + // After both designator passes, so the comparison is against + // the value the entry will actually carry out. Skipped when the + // key was injected: the entry then agrees with its key by + // construction, and a rejected *injected* designator value is + // `canonicalize_type_designator`'s to report. + if let (Some(ks), true) = (key_slot.as_ref(), payload_states_key) { + Self::reconcile_dict_key_with_payload( + entry_key, + ks, + &child_values, + &selected, + conv, + &path, + validation_issues, + ); + } Ok(LinkMLInstance::Object { node_id: new_node_id(), values: child_values, @@ -1465,10 +1765,8 @@ impl LinkMLInstance { }) } other => { - let key_slot_name = range_cv - .key_or_identifier_slot() - .map(|s| s.name.as_str()) - .unwrap_or(""); + let key_slot = range_cv.key_or_identifier_slot().cloned(); + let key_slot_name = key_slot.as_ref().map(|s| s.name.as_str()).unwrap_or(""); let scalar_slot = Self::find_scalar_slot_for_inlined_map(&range_cv, key_slot_name) .ok_or_else(|| { LinkMLError::single( @@ -1480,6 +1778,34 @@ impl LinkMLInstance { ), ) })?; + // The compact form states exactly two things: the dict key and + // one scalar. Both are named here, because either can be the + // class's *designator* and so name the entry's class. + // + // `find_scalar_slot_for_inlined_map` picks the first non-key + // scalar slot, which can be the designator: then + // `{"w1": "canon:FancyWidget"}` says "widget w1 is a + // FancyWidget", exactly as the object form's `{"typeURI": ...}` + // does. The dict key is the designator whenever the key slot + // and the designator slot are one — the asset360 + // `PositioningSystemCoordinate` shape. This arm otherwise + // hardwires the slot's range class, so without selecting here + // canonicalisation would rewrite that designator to the *range* + // class's value and warn about data that was right — the + // misclassification spec rule 2 exists to prevent. Selection + // stays scoped to that one question: a compact entry that names + // no designator at all names no class and keeps the range class. + let mut named = serde_json::Map::new(); + named.insert(scalar_slot.name.clone(), other.clone()); + if let Some(ks) = &key_slot { + named.insert(ks.name.clone(), Self::dict_key_value(entry_key)); + } + let entry_class = match range_cv.get_type_designator_slot() { + Some(td) if named.contains_key(&td.name) => { + Self::select_class(&named, &range_cv, sv, conv) + } + _ => range_cv.clone(), + }; let mut child_values = HashMap::new(); child_values.insert( scalar_slot.name.clone(), @@ -1487,22 +1813,44 @@ impl LinkMLInstance { node_id: new_node_id(), value: Self::coerce_scalar_to_range(other, Some(&scalar_slot)), slot: scalar_slot.clone(), - class: Some(range_cv.clone()), + class: Some(entry_class.clone()), sv: sv.clone(), }, ); + // Rule 5: the compact form has no payload for the key slot at + // all, so the dict key is the only thing that can fill it — + // there is nothing here to diverge from. + if let Some(ks) = &key_slot { + child_values.insert( + ks.name.clone(), + LinkMLInstance::Scalar { + node_id: new_node_id(), + value: Self::dict_key_value(entry_key), + slot: ks.clone(), + class: Some(entry_class.clone()), + sv: sv.clone(), + }, + ); + } run_object_constraints( - &range_cv, + &entry_class, &child_values, &HashMap::new(), path.clone(), validation_issues, ); - Self::populate_type_designator(&mut child_values, &range_cv, sv, conv); + Self::canonicalize_type_designator( + &mut child_values, + &entry_class, + conv, + &path, + validation_issues, + ); + Self::populate_type_designator(&mut child_values, &entry_class, sv, conv); Ok(LinkMLInstance::Object { node_id: new_node_id(), values: child_values, - class: range_cv, + class: entry_class, sv: sv.clone(), unknown_fields: HashMap::new(), }) diff --git a/src/runtime/tests/data/example_personinfo_data.yaml b/src/runtime/tests/data/example_personinfo_data.yaml index 7c847a1..cff2701 100644 --- a/src/runtime/tests/data/example_personinfo_data.yaml +++ b/src/runtime/tests/data/example_personinfo_data.yaml @@ -1,14 +1,14 @@ objects: - id: T - objecttype: https://w3id.org/linkml/examples/personinfo/Organization + objecttype: http://schema.org/Organization name: club - id: P:001 name: fred bloggs - objecttype: https://w3id.org/linkml/examples/personinfo/Person + objecttype: http://schema.org/Person primary_email: fred.bloggs@example.com age_in_years: 33 - id: P:002 - objecttype: https://w3id.org/linkml/examples/personinfo/Person + objecttype: http://schema.org/Person name: joe schmoe primary_email: joe.schmoe@example.com has_employment_history: diff --git a/src/runtime/tests/data/example_personinfo_data_2.yaml b/src/runtime/tests/data/example_personinfo_data_2.yaml index c2451c0..2500902 100644 --- a/src/runtime/tests/data/example_personinfo_data_2.yaml +++ b/src/runtime/tests/data/example_personinfo_data_2.yaml @@ -1,25 +1,32 @@ objects: - id: T - objecttype: https://w3id.org/linkml/examples/personinfo/Organization + objecttype: http://schema.org/Organization name: club - id: P:001 name: fred bloggs - objecttype: https://w3id.org/linkml/examples/personinfo/Person + objecttype: http://schema.org/Person primary_email: fred.bloggs@example.com age_in_years: 33 - id: P:002 - objecttype: https://w3id.org/linkml/examples/personinfo/Person + objecttype: http://schema.org/Person name: joe schmoe primary_email: joe.schmoe@example.com has_employment_history: - employed_at: ROR:1 started_at_time: 2019-01-01 is_current: true + # `role` is the key slot of this inlined dict. Spec addendum rule 5 makes + # the dict key real data, so the loader fills `role` from it — this file is + # the *expected* side of the `linkml-patch` CLI round-trip and therefore + # states its own normal form. `example_personinfo_data.yaml`, the source + # side, still omits it, so the omitted case stays exercised end to end. has_familial_relationships: brother: + role: brother related_to: P:001 type: SIBLING_OF mother: + role: mother related_to: P:003 type: PARENT_OF has_medical_history: @@ -34,7 +41,7 @@ objects: name: trepanation objecttype: https://w3id.org/linkml/examples/personinfo/ProcedureConcept - id: P:003 - objecttype: https://w3id.org/linkml/examples/personinfo/Person + objecttype: http://schema.org/Person name: alice smith primary_email: alice.smith@example.com age_in_years: 34 diff --git a/src/runtime/tests/data/identity.yaml b/src/runtime/tests/data/identity.yaml new file mode 100644 index 0000000..ed0bc87 --- /dev/null +++ b/src/runtime/tests/data/identity.yaml @@ -0,0 +1,163 @@ +id: https://w3id.org/linkml/examples/identity +name: identity +description: |- + Shared fixture for element-identity features: unique_keys, the + `diff.linkml.io/opaque` annotation, and the identity linter. +license: https://creativecommons.org/publicdomain/zero/1.0/ +imports: + - linkml:types +prefixes: + identity: https://w3id.org/linkml/examples/identity/ + linkml: https://w3id.org/linkml/ +default_prefix: identity +default_range: string + +classes: + Service: + # An explicit class_uri indexes the class under BOTH this URI and its + # default one, so a linter walking `get_class_ids()` sees it twice. + class_uri: identity:ServiceEndpoint + attributes: + name: {range: string} + # spec Example 1 with its correct resolution: identity = the function + hasPhoneNumber: + range: ServicePhoneNumber + multivalued: true + inlined_as_list: true + # same shape, nothing declared: must keep positional behaviour; linter flags it + plainPhoneNumber: + range: PlainPhoneNumber + multivalued: true + inlined_as_list: true + # inherited unique_keys (via is_a) must also drive matching + escalation: + range: EmergencyPhoneNumber + multivalued: true + inlined_as_list: true + # composite two-slot unique key + contacts: + range: Contact + multivalued: true + inlined_as_list: true + # opaque + a keyed element class: data lint must still check unique_keys + archivedContacts: + range: Contact + multivalued: true + inlined_as_list: true + annotations: + diff.linkml.io/opaque: true + # scalar list, nothing declared: positional; linter flags it + tags: + range: string + multivalued: true + # scalar list declared opaque + opaqueTags: + range: string + multivalued: true + annotations: + diff.linkml.io/opaque: true + # spec Example 2's ring: opaque list of bare vertices + outline: + range: Vertex + multivalued: true + inlined_as_list: true + annotations: + diff.linkml.io/opaque: true + # single-valued opaque object: "stop all recursion" is not list-specific + profile: + range: Profile + inlined: true + annotations: + diff.linkml.io/opaque: true + # keyed class inlined as dict: already unambiguous, linter stays silent + labels: + range: Label + multivalued: true + inlined: true + # keyed class inlined as list: the uniform guard applies to key labels too + labelList: + range: Label + multivalued: true + inlined_as_list: true + # reference list (not inlined): out of the linter's scope + operators: + range: Operator + multivalued: true + inlined: false + area: + range: AreaLocation + inlined: true + + ServicePhoneNumber: + unique_keys: + one_number_per_function: + unique_key_slots: [hasNumberFunction] + attributes: + phoneNumber: {range: string} + hasNumberFunction: {range: NumberFunction, required: true} + + PlainPhoneNumber: + attributes: + phoneNumber: {range: string} + hasNumberFunction: {range: NumberFunction, required: true} + + EmergencyPhoneNumber: + is_a: ServicePhoneNumber + attributes: + note: {range: string} + + Contact: + unique_keys: + contact_identity: + unique_key_slots: [kind, phone] + attributes: + kind: {range: string, required: true} + phone: {range: string, required: true} + note: {range: string} + + Label: + attributes: + lang: {range: string, key: true, required: true} + text: {range: string} + + Profile: + attributes: + bio: {range: string} + motto: {range: string} + + Operator: + attributes: + opId: {range: string, identifier: true} + opName: {range: string} + + AreaLocation: + attributes: + polygons: + range: Polygon + multivalued: true + inlined_as_list: true + + Polygon: + unique_keys: + one_polygon_per_positioning_system: + unique_key_slots: [positioningSystemType] + attributes: + positioningSystemType: {range: uri, required: true} + coordinates: + range: Vertex + multivalued: true + inlined_as_list: true + annotations: + diff.linkml.io/opaque: true + + Vertex: + attributes: + x: {range: float} + y: {range: float} + +enums: + NumberFunction: + permissible_values: + Emergency_Number: + Non_Urgent_Communication: + Operator: diff --git a/src/runtime/tests/data/identity_canonical.yaml b/src/runtime/tests/data/identity_canonical.yaml new file mode 100644 index 0000000..139c68c --- /dev/null +++ b/src/runtime/tests/data/identity_canonical.yaml @@ -0,0 +1,129 @@ +id: https://w3id.org/linkml/examples/identity_canonical +name: identity_canonical +description: |- + Fixture for spec addendum rule 2 — identity compares meaning, not spelling. + + Two shapes are exercised: + + * a type designator (`Shape.typeURI`, `Node.kind`, `Widget.typeURI`) whose + value data may spell as a CURIE, as a full URI, or omit entirely — all + three must box to the one canonical value the class declares, and a value + that is no accepted designator value at all must load with a warning. + * an identity component whose range descends from `uri`/`uriorcurie` + (`System.systemType`), where a CURIE and its expansion are one identity. +license: https://creativecommons.org/publicdomain/zero/1.0/ +imports: + - linkml:types +prefixes: + canon: https://w3id.org/linkml/examples/identity_canonical/ + ex: https://example.org/canon/ + linkml: https://w3id.org/linkml/ +default_prefix: canon +default_range: string + +classes: + Container: + attributes: + # polymorphic list; identity is unique_keys[label], the designator is not + shapes: + range: Shape + multivalued: true + inlined_as_list: true + # designator on a `uri`-ranged slot: canonical form is the expanded URI + nodes: + range: Node + multivalued: true + inlined_as_list: true + # dict form, so `build_mapping_entry_for_slot` is exercised too + widgets: + range: Widget + multivalued: true + inlined: true + # D6: identity component whose range descends from `uri` + systems: + range: System + multivalued: true + inlined_as_list: true + # the same, where that component is a `key` — the one input diff's + # changed-key check reads, which `unique_keys` never reaches + keyed_systems: + range: KeyedSystem + multivalued: true + inlined_as_list: true + + Shape: + unique_keys: + by_label: + unique_key_slots: [label] + attributes: + typeURI: + range: uriorcurie + designates_type: true + label: {range: string, required: true} + + Circle: + is_a: Shape + attributes: + radius: {range: float} + + Node: + unique_keys: + by_name: + unique_key_slots: [name] + attributes: + kind: + range: uri + designates_type: true + name: {range: string, required: true} + + # declares a `class_uri`, so its canonical designator value and its + # schema-native URI differ — data may legitimately spell either + LeafNode: + is_a: Node + class_uri: ex:Leaf + attributes: + depth: {range: integer} + + # the guard for `ClassView::get_uri(native, expand)`: a `class_uri` and no + # distinguishing slot at all, so the ONLY thing that can select this class is + # its designator value. Spelled with its schema-native URI it is reachable + # only if that spelling is in the accepted set; if `get_uri(true, true)` ever + # returns the `class_uri` again, this class silently becomes `Node` and the + # designator gets rewritten to `Node`'s value. + BareLeaf: + is_a: Node + class_uri: ex:BareLeaf + + # `find_scalar_slot_for_inlined_map` picks the first non-key scalar slot — + # here the designator — so `{"w1": "canon:FancyWidget"}` is a legal compact + # dict entry that names its own class. + # `wid` is `required` on purpose: the dict-key injection of spec addendum + # rule 5 is what satisfies it for both the object and the compact form, so a + # loader that stopped injecting fails here rather than quietly. + Widget: + attributes: + wid: {range: string, key: true, required: true} + typeURI: + range: uriorcurie + designates_type: true + + FancyWidget: + is_a: Widget + attributes: + sparkle: {range: string} + + System: + unique_keys: + by_system_type: + unique_key_slots: [systemType] + attributes: + systemType: {range: uri, required: true} + value: {range: string} + + # `System`'s identity comes from `unique_keys`, which diff's changed-key + # check never consults; this one declares the same component as a `key`, so + # the check does read it, and its comparison has to be the canonical one too. + KeyedSystem: + attributes: + systemType: {range: uri, key: true, required: true} + value: {range: string} diff --git a/src/runtime/tests/data/identity_class_change.yaml b/src/runtime/tests/data/identity_class_change.yaml new file mode 100644 index 0000000..bbc8b09 --- /dev/null +++ b/src/runtime/tests/data/identity_class_change.yaml @@ -0,0 +1,63 @@ +id: https://w3id.org/linkml/examples/identity_class_change +name: identity_class_change +description: |- + Fixture for spec addendum rule 3 — a class change is a whole-element + replacement — and rule 4 — `patch` never hard-errors on an unappliable delta. + + `Part` declares the element identity (`unique_keys: [code]`) and the type + designator; `Bolt` and `Nut` are siblings carrying *disjoint* extra slots. + Two elements can therefore share one identity label while being different + things: diff must say so once, at the element, instead of recursing + field-by-field across two classes and emitting deltas (`thread` on a `Nut`) + that no builder can apply. +license: https://creativecommons.org/publicdomain/zero/1.0/ +imports: + - linkml:types +prefixes: + part: https://w3id.org/linkml/examples/identity_class_change/ + linkml: https://w3id.org/linkml/ +default_prefix: part +default_range: string + +classes: + Inventory: + attributes: + parts: + range: Part + multivalued: true + inlined_as_list: true + # single-valued inlined object: rule 3 is not list-specific + featured: + range: Part + inlined: true + # a class with a real key slot: the case diff's changed-key branch still + # owns after rule 3 (same class, changed key value) + owner: + range: Owner + inlined: true + + Part: + unique_keys: + by_code: + unique_key_slots: [code] + attributes: + typeURI: + range: uriorcurie + designates_type: true + code: {range: string, required: true} + note: {range: string} + + Bolt: + is_a: Part + attributes: + thread: {range: string} + + Nut: + is_a: Part + attributes: + pitch: {range: string} + + Owner: + attributes: + oid: {range: string, key: true} + oname: {range: string} diff --git a/src/runtime/tests/data/identity_descendant_unique_keys.yaml b/src/runtime/tests/data/identity_descendant_unique_keys.yaml new file mode 100644 index 0000000..a72f91c --- /dev/null +++ b/src/runtime/tests/data/identity_descendant_unique_keys.yaml @@ -0,0 +1,189 @@ +id: https://w3id.org/linkml/examples/identity_descendant_unique_keys +name: identity_descendant_unique_keys +description: |- + `unique_keys` ambiguity across an inheritance hierarchy (spike D4d). + + A list ranged on a class holds elements of that class *and of every class + descending from it*, and each element's label comes from the name-sorted + first entry of its OWN merged `unique_keys`. Inspecting only the range class + therefore answers the wrong question twice over: + + - `boxes` (range `Box`) — `Box` declares one entry, so the range class alone + looks unambiguous; `BigBox` adds a later-sorting one. Two entries are now + reachable through this slot, and the load-bearing one is still `by_id` for + every class: the candidate set is ambiguous, the label space is not. + - `gadgets` (range `Gadget`) — `Gadget` declares one entry, `Widget` adds an + earlier-sorting one. `Gadget` elements are labelled by `code` and `Widget` + elements by `serial`: one list, two label spaces, and a `Widget` serial can + collide with a `Gadget` code without either class's constraint being + violated. That is the split the second warning names. + + - `keyed` (range `Plate`) — `StampedPlate` declares a `key`, which outranks + its `unique_keys` entirely. Its entries are therefore never load-bearing + and do not widen the candidate set — but the key itself is a label space: + `Plate` elements are labelled by `plate_identity` and `StampedPlate` + elements by `stampId`, and the two can collide exactly as two entries can. + The divergence rule counts a key-labelled class as its own group, so this + slot gets the divergence warning and *not* the several-entries one. + + Guard rail: `crates` (a descendant that declares no entries of its own) must + stay silent. + + `Base.stamps` is the gate-subtraction case. Its range class is + designator-keyed, so the designator rule is its only voice — but the raw + several-entries and divergence shapes also match it (the designator key no + longer shadows `DesignatorMarker`'s entries, and `AaaSubMarker` adds an + earlier-sorting one). A subclass that `slot_usage`-retargets the slot to a + class those rules really do speak for must therefore not be suppressed by a + parent that never emitted their warning: `Retargeted` and `SplitRetargeted` + are the two halves of that. +license: https://creativecommons.org/publicdomain/zero/1.0/ +imports: + - linkml:types +prefixes: + identity: https://w3id.org/linkml/examples/identity_descendant_unique_keys/ + linkml: https://w3id.org/linkml/ +default_prefix: identity +default_range: string + +classes: + Depot: + attributes: + # descendant adds a LATER-sorting entry: ambiguous candidate set, one + # label space -> the several-entries warning only + boxes: + range: Box + multivalued: true + inlined_as_list: true + # descendant adds an EARLIER-sorting entry: split label space -> the + # several-entries warning AND the divergence warning + gadgets: + range: Gadget + multivalued: true + inlined_as_list: true + # descendant declares nothing of its own: one entry, one label space + crates: + range: Crate + multivalued: true + inlined_as_list: true + # the descendant's key outranks its own unique_keys, so the candidate set + # is unchanged — but the key is a second label space -> divergence only + keyed: + range: Plate + multivalued: true + inlined_as_list: true + + # the gate-subtraction cases: this slot's range class is designator-keyed, so + # the designator rule is its sole voice, even though the raw several-entries + # and divergence shapes both match it + Base: + attributes: + stamps: + range: DesignatorMarker + multivalued: true + inlined_as_list: true + + # retargets the inherited slot onto a keyless two-entry class: the + # several-entries rule speaks for it HERE, and the parent — which only ever + # emitted the designator warning — cannot stand in for that + Retargeted: + is_a: Base + slot_usage: + stamps: + range: TwoEntry + + # the same, for the divergence rule + SplitRetargeted: + is_a: Base + slot_usage: + stamps: + range: Gadget + + Box: + unique_keys: + by_id: + unique_key_slots: [boxId] + attributes: + boxId: {range: string, required: true} + + BigBox: + is_a: Box + unique_keys: + zz_by_volume: + unique_key_slots: [volume] + attributes: + volume: {range: string, required: true} + + Gadget: + unique_keys: + gadget_identity: + unique_key_slots: [code] + attributes: + code: {range: string, required: true} + + Widget: + is_a: Gadget + unique_keys: + aaa_widget_identity: + unique_key_slots: [serial] + attributes: + serial: {range: string, required: true} + + Crate: + unique_keys: + crate_identity: + unique_key_slots: [crateId] + attributes: + crateId: {range: string, required: true} + + BigCrate: + is_a: Crate + attributes: + capacity: {range: string} + + Plate: + unique_keys: + plate_identity: + unique_key_slots: [code] + attributes: + code: {range: string, required: true} + + StampedPlate: + is_a: Plate + unique_keys: + aaa_stamp_identity: + unique_key_slots: [stamp] + attributes: + stampId: {range: string, key: true} + stamp: {range: string, required: true} + + DesignatorMarker: + unique_keys: + by_code: + unique_key_slots: [code] + zz_by_label: + unique_key_slots: [label] + attributes: + typeURI: {range: uriorcurie, key: true, designates_type: true} + code: {range: string, required: true} + label: {range: string, required: true} + + # an earlier-sorting entry, so `DesignatorMarker`'s family also matches the + # divergence shape raw — the second half of the gate-subtraction case + AaaSubMarker: + is_a: DesignatorMarker + unique_keys: + aaa_sub_identity: + unique_key_slots: [sub] + attributes: + sub: {range: string, required: true} + + TwoEntry: + unique_keys: + by_alpha: + unique_key_slots: [alpha] + zz_by_beta: + unique_key_slots: [beta] + attributes: + alpha: {range: string, required: true} + beta: {range: string, required: true} diff --git a/src/runtime/tests/data/identity_inheritance.yaml b/src/runtime/tests/data/identity_inheritance.yaml new file mode 100644 index 0000000..9626d05 --- /dev/null +++ b/src/runtime/tests/data/identity_inheritance.yaml @@ -0,0 +1,79 @@ +id: https://w3id.org/linkml/examples/identity-inheritance +name: identity_inheritance +description: |- + Regression fixture for "report a flagged slot once, at the class that + introduces it". + + A flagged slot is inherited by every descendant, so reporting it per class + buries the one place the author can fix it under a pile of copies. The lint + reports only the topmost class whose direct `is_a` parent does not already + carry the same slot in the same flagged state. + + The classes below cover the cases that pull against each other: + Base introduces the flagged `outline` -> reported + Middle inherits it unchanged -> silent + Leaf inherits it via Middle -> silent (recursion) + Fixed narrows `outline` to a keyed range -> silent, on its own merits + Broken widens a clean slot to an ambiguous + range, so IT introduces the problem -> reported + Tagged gets `tags` from a mixin, not is_a -> reported (see rustdoc) +license: https://creativecommons.org/publicdomain/zero/1.0/ +imports: + - linkml:types +prefixes: + inherit: https://w3id.org/linkml/examples/identity-inheritance/ + linkml: https://w3id.org/linkml/ +default_prefix: inherit +default_range: string + +classes: + Base: + attributes: + # Vertex declares no key, identifier or unique_keys: flagged here. + outline: + range: Vertex + multivalued: true + inlined_as_list: true + # Keyed declares a key: not flagged. + keyed: + range: Keyed + multivalued: true + inlined_as_list: true + + Middle: + is_a: Base + + Leaf: + is_a: Middle + + Fixed: + is_a: Base + slot_usage: + outline: + range: Keyed + + Broken: + is_a: Base + slot_usage: + keyed: + range: Vertex + + HasTagsMixin: + mixin: true + attributes: + tags: + range: string + multivalued: true + + Tagged: + mixins: + - HasTagsMixin + + Keyed: + attributes: + k: {range: string, key: true} + + Vertex: + attributes: + x: {range: float} + y: {range: float} diff --git a/src/runtime/tests/data/identity_missing_labels.yaml b/src/runtime/tests/data/identity_missing_labels.yaml new file mode 100644 index 0000000..d706104 --- /dev/null +++ b/src/runtime/tests/data/identity_missing_labels.yaml @@ -0,0 +1,121 @@ +id: https://w3id.org/linkml/examples/identity_missing_labels +name: identity_missing_labels +description: |- + Data whose lists are addressed *positionally despite a declared identity*: + the element class declares a key or `unique_keys`, but the data leaves the + slot it names empty, so the element yields no identity label and the whole + list falls back to positional addressing. + + The declaration is not wrong and the data is not invalid — a `key` slot that + is not `required` may legitimately be absent — so neither the schema lint nor + validation has anything to say. Only the instance lint can see it, and only + by looking at the data. + + `UnfillableKey` is the spike's D9 shape: a base class declares the type + designator, a subclass promotes it to `key` and then turns the designator + off with `slot_usage`. Nothing fills the key afterwards — the designator + machinery no longer recognises the slot — so every element of such a list is + unlabelled. It is deliberately not detected as its own schema-lint rule; it + collapses into this one, which is what the linter rustdoc says. +license: https://creativecommons.org/publicdomain/zero/1.0/ +imports: + - linkml:types +prefixes: + identity: https://w3id.org/linkml/examples/identity_missing_labels/ + linkml: https://w3id.org/linkml/ +default_prefix: identity +default_range: string + +classes: + Registry: + attributes: + name: {range: string} + # optional key: data may omit it, and then the list is positional + entries: + range: OptKey + multivalued: true + inlined_as_list: true + # the same shape declared with unique_keys instead of a key + records: + range: OptUnique + multivalued: true + inlined_as_list: true + # D9: a key that the designator override left nothing to fill + unfillable: + range: UnfillableKey + multivalued: true + inlined_as_list: true + # no identity is declared at all: the identity-less schema rule already + # speaks for this slot, and a positional list is exactly what was asked + # for. The instance lint must stay silent. + vertices: + range: Vertex + multivalued: true + inlined_as_list: true + # scalars have no element class to declare identity on + tags: + range: string + multivalued: true + # `opaque` answers the addressing question already ("replaced as a + # whole"), so "this list is addressed positionally" is not true of it. + # Unlike the duplicate rule, which reports a violated class constraint + # whatever the slot says, this one stays quiet here. + archivedEntries: + range: OptKey + multivalued: true + inlined_as_list: true + annotations: + diff.linkml.io/opaque: true + # outside diff's scope entirely + draftEntries: + range: OptKey + multivalued: true + inlined_as_list: true + annotations: + diff.linkml.io/ignore: true + # a REFERENCE list: the elements are identifier strings, which can never + # carry an inlined element's identity label. asset360's `NetElement.ports` + # is exactly this shape, and it is the only list in the whole downstream + # corpus the rule would otherwise have fired on — every one of those + # warnings would be noise. + references: + range: Referenced + multivalued: true + inlined: false + + OptKey: + attributes: + code: {range: string, key: true} + note: {range: string} + + OptUnique: + unique_keys: + by_serial: + unique_key_slots: [serial] + attributes: + serial: {range: string} + note: {range: string} + + # base declares the designator; the subclass promotes it to the key and + # switches the designator off, leaving a key nothing fills + DesignatorBase: + attributes: + typeURI: {range: uriorcurie, designates_type: true} + note: {range: string} + + UnfillableKey: + is_a: DesignatorBase + slot_usage: + typeURI: + key: true + designates_type: false + + Referenced: + attributes: + id: {range: string, identifier: true} + note: {range: string} + + Vertex: + attributes: + x: {range: float} + y: {range: float} diff --git a/src/runtime/tests/data/identity_missing_labels_data.json b/src/runtime/tests/data/identity_missing_labels_data.json new file mode 100644 index 0000000..1a74c6f --- /dev/null +++ b/src/runtime/tests/data/identity_missing_labels_data.json @@ -0,0 +1,10 @@ +{ + "name": "reg", + "entries": [{"note": "a"}, {"note": "b"}], + "records": [{"serial": "S1"}, {"note": "b"}], + "unfillable": [{"note": "a"}, {"note": "b"}], + "vertices": [{"x": 1.0, "y": 2.0}, {"x": 1.0, "y": 2.0}], + "tags": ["a", "a"], + "references": ["urn:a", "urn:b"], + "archivedEntries": [{"code": "dup"}, {"code": "dup"}] +} diff --git a/src/runtime/tests/data/identity_multiple_unique_keys.yaml b/src/runtime/tests/data/identity_multiple_unique_keys.yaml new file mode 100644 index 0000000..a7d01ac --- /dev/null +++ b/src/runtime/tests/data/identity_multiple_unique_keys.yaml @@ -0,0 +1,79 @@ +id: https://w3id.org/linkml/examples/identity_multiple_unique_keys +name: identity_multiple_unique_keys +description: |- + Element identity derived from `unique_keys` when a class declares more than + one entry: only the name-sorted first is load-bearing, so the schema author + has to be told which one every delta path is addressed by. +license: https://creativecommons.org/publicdomain/zero/1.0/ +imports: + - linkml:types +prefixes: + identity: https://w3id.org/linkml/examples/identity_multiple_unique_keys/ + linkml: https://w3id.org/linkml/ +default_prefix: identity +default_range: string + +classes: + Catalog: + attributes: + # two candidate identities: the lint must name the load-bearing one + badges: + range: Badge + multivalued: true + inlined_as_list: true + # exactly one entry: unambiguous, stays silent + tickets: + range: Ticket + multivalued: true + inlined_as_list: true + # a key slot outranks unique_keys entirely: nothing to disambiguate + seats: + range: Seat + multivalued: true + inlined_as_list: true + # identity declared as "nowhere": no per-element paths to re-address + archivedBadges: + range: Badge + multivalued: true + inlined_as_list: true + annotations: + diff.linkml.io/opaque: true + # outside diff's scope entirely + draftBadges: + range: Badge + multivalued: true + inlined_as_list: true + annotations: + diff.linkml.io/ignore: true + + # inherits `badges` unchanged: reported at Catalog, not here + SubCatalog: + is_a: Catalog + + Badge: + unique_keys: + by_code: + unique_key_slots: [code] + zz_by_label: + unique_key_slots: [label] + attributes: + code: {range: string, required: true} + label: {range: string, required: true} + + Ticket: + unique_keys: + by_serial: + unique_key_slots: [serial] + attributes: + serial: {range: string, required: true} + + Seat: + unique_keys: + by_number: + unique_key_slots: [number] + by_row: + unique_key_slots: [row] + attributes: + seatId: {range: string, key: true} + row: {range: string, required: true} + number: {range: string, required: true} diff --git a/src/runtime/tests/data/identity_shared_class_uri.yaml b/src/runtime/tests/data/identity_shared_class_uri.yaml new file mode 100644 index 0000000..718f605 --- /dev/null +++ b/src/runtime/tests/data/identity_shared_class_uri.yaml @@ -0,0 +1,56 @@ +id: https://w3id.org/linkml/examples/identity-shared-class-uri +name: identity_shared_class_uri +description: |- + Regression fixture for the identity linter's class-visit dedupe. + + LinkML permits several distinct classes to declare the SAME `class_uri` + (in this repo, `meta.yaml`'s `Anything` and `extensions.yaml`'s `AnyValue` + both declare `linkml:Any`). A linter that dedupes its class visits by class + URI treats the second such class as already-seen and silently drops all of + its warnings — a false negative, which is worse than the duplicate + reporting the dedupe exists to prevent. + + `SharedUriA` and `SharedUriB` declare one shared `class_uri` and each holds + one multivalued inlined slot with no declared element identity, so both must + be reported. `TwoUris` additionally declares an explicit `class_uri`, which + indexes it under both that and its default URI: it must be reported once, + not twice. The two requirements pull in opposite directions, which is the + point of keeping them in one fixture. +license: https://creativecommons.org/publicdomain/zero/1.0/ +imports: + - linkml:types +prefixes: + shared: https://w3id.org/linkml/examples/identity-shared-class-uri/ + linkml: https://w3id.org/linkml/ +default_prefix: shared +default_range: string + +classes: + SharedUriA: + class_uri: shared:SharedThing + attributes: + itemsA: + range: Vertex + multivalued: true + inlined_as_list: true + + SharedUriB: + class_uri: shared:SharedThing + attributes: + itemsB: + range: Vertex + multivalued: true + inlined_as_list: true + + TwoUris: + class_uri: shared:SomethingElse + attributes: + itemsC: + range: Vertex + multivalued: true + inlined_as_list: true + + Vertex: + attributes: + x: {range: float} + y: {range: float} diff --git a/src/runtime/tests/data/identity_shared_uri_designator.yaml b/src/runtime/tests/data/identity_shared_uri_designator.yaml new file mode 100644 index 0000000..34abadc --- /dev/null +++ b/src/runtime/tests/data/identity_shared_uri_designator.yaml @@ -0,0 +1,77 @@ +id: https://w3id.org/linkml/examples/identity_shared_uri_designator +name: identity_shared_uri_designator +description: |- + Two classes of one `is_a` hierarchy declaring the SAME `class_uri`, while the + hierarchy carries a type designator (spike D8). + + The designator's value is what the loader dispatches on, and it is looked up + by class URI. When two classes of one hierarchy answer to the same URI, the + loader picks one of them — stably, but by an ordering nothing in the schema + states and nothing promises to keep. Every instance carrying that designator + value loads as the winner, whatever the author meant; a diff then pairs two + elements that "are" different classes but compare equal by URI. + + The loader behaviour is deliberately left alone here: this is a warning, so + the author learns that the two declarations are not distinguishable at load + time and can rename one. + + The controls are the two halves of the condition, each removed in turn: + `PlainA`/`PlainB` share a URI in a hierarchy with no designator (nothing + dispatches on the URI, so nothing is arbitrary), and `LoneA`/`LoneB` carry + designators and share a URI but belong to two unrelated hierarchies (no + single designated slot ever has to choose between them). +license: https://creativecommons.org/publicdomain/zero/1.0/ +imports: + - linkml:types +prefixes: + shared: https://w3id.org/linkml/examples/identity_shared_uri_designator/ + linkml: https://w3id.org/linkml/ +default_prefix: shared +default_range: string + +classes: + Container: + attributes: + things: + range: TypedThing + multivalued: true + inlined_as_list: true + + TypedThing: + attributes: + typeURI: {range: uriorcurie, designates_type: true} + thingId: {range: string, key: true} + + Alpha: + is_a: TypedThing + class_uri: shared:Same + + Beta: + is_a: TypedThing + class_uri: shared:Same + + # control: same-URI siblings, no designator anywhere in the hierarchy + Plain: + attributes: + plainId: {range: string, key: true} + + PlainA: + is_a: Plain + class_uri: shared:PlainSame + + PlainB: + is_a: Plain + class_uri: shared:PlainSame + + # control: designators and a shared URI, but two unrelated hierarchies + LoneA: + class_uri: shared:LoneSame + attributes: + typeURI: {range: uriorcurie, designates_type: true} + loneId: {range: string, key: true} + + LoneB: + class_uri: shared:LoneSame + attributes: + typeURI: {range: uriorcurie, designates_type: true} + loneId: {range: string, key: true} diff --git a/src/runtime/tests/data/identity_type_designator_key.yaml b/src/runtime/tests/data/identity_type_designator_key.yaml new file mode 100644 index 0000000..f18a910 --- /dev/null +++ b/src/runtime/tests/data/identity_type_designator_key.yaml @@ -0,0 +1,127 @@ +id: https://w3id.org/linkml/examples/identity_type_designator_key +name: identity_type_designator_key +description: |- + Element identity declared as a key that is also the class's type designator. + A designator's value describes the element's class, never the element, so it + is never element identity: the diff engine ignores such a key outright and + addresses the list by the element class's unique_keys when it declares any + (and their labels are unique within the list), positionally otherwise. The + dict (mapping) form of the same class is legitimate: it means + at-most-one-element-per-subtype. +license: https://creativecommons.org/publicdomain/zero/1.0/ +imports: + - linkml:types +prefixes: + identity: https://w3id.org/linkml/examples/identity_type_designator_key/ + linkml: https://w3id.org/linkml/ +default_prefix: identity +default_range: string + +classes: + Ring: + attributes: + # a homogeneous vertex ring whose element class declares nothing but the + # designator key: the engine ignores that key, no identity is left, and + # the ring is addressed positionally (even at length one) + vertices: + range: Coordinate + multivalued: true + inlined_as_list: true + # the shadow case: the designator key used to outrank unique_keys and no + # longer does, so `by_code` (name-sorted first) is the real identity the + # engine matches on. Still flagged by the designator rule only — the + # sharpest diagnosis, and the sole voice for a designator-keyed slot + markers: + range: Marker + multivalued: true + inlined_as_list: true + # dict form: keyed by the designator means at-most-one-per-subtype, which + # is a legitimate model + byType: + range: Coordinate + multivalued: true + inlined: true + # identity declared as "nowhere": the value is replaced as a whole + archivedVertices: + range: Coordinate + multivalued: true + inlined_as_list: true + annotations: + diff.linkml.io/opaque: true + # outside diff's scope entirely + draftVertices: + range: Coordinate + multivalued: true + inlined_as_list: true + annotations: + diff.linkml.io/ignore: true + # the wild shape (asset360's PositioningSystemCoordinate): the designator + # is declared on a base class and a subclass promotes it to the key with + # `slot_usage`. The rule only sees this because `SlotView::definition()` + # merges the is_a chain; without that it is a silent false negative. + # Also the polymorphic case: with `SpecialKeyedTypedThing` in the list the + # designator values differ per element, which is a fact about the classes + # present, not an element identity — positional too. + inheritedVertices: + range: KeyedTypedThing + multivalued: true + inlined_as_list: true + # an ordinary key, discriminating per element: nothing to warn about + points: + range: Point + multivalued: true + inlined_as_list: true + + # inherits `vertices` unchanged: reported at Ring, not here + SubRing: + is_a: Ring + + Coordinate: + attributes: + typeURI: + range: uriorcurie + key: true + designates_type: true + x: {range: float, required: true} + y: {range: float, required: true} + + Marker: + unique_keys: + by_code: + unique_key_slots: [code] + zz_by_label: + unique_key_slots: [label] + attributes: + typeURI: + range: uriorcurie + key: true + designates_type: true + code: {range: string, required: true} + label: {range: string, required: true} + + # declares the designator but no key: nothing ranges a list on it directly + TypedThing: + attributes: + typeURI: + range: uri + designates_type: true + label: {range: string} + + # promotes the inherited designator to the key, exactly as + # PositioningSystemCoordinate does + KeyedTypedThing: + is_a: TypedThing + slot_usage: + typeURI: + key: true + + # a second concrete subtype, so a list ranged on `KeyedTypedThing` can be + # polymorphic: one element per subtype, each carrying its own designator value + SpecialKeyedTypedThing: + is_a: KeyedTypedThing + + Point: + attributes: + pointId: {range: string, key: true} + x: {range: float, required: true} + y: {range: float, required: true} diff --git a/src/runtime/tests/data/inlined_dict_key.yaml b/src/runtime/tests/data/inlined_dict_key.yaml new file mode 100644 index 0000000..1890120 --- /dev/null +++ b/src/runtime/tests/data/inlined_dict_key.yaml @@ -0,0 +1,72 @@ +id: https://w3id.org/linkml/examples/inlined_dict_key +name: inlined_dict_key +description: |- + Fixture for spec addendum rule 5 — the inlined-dict key is real data (D5). + + Three shapes: + + * `Container.people` — an ordinary keyed class in dict form. The dict key is + the element's `pid`; a payload that omits it must be filled from the key, + and a payload that contradicts it must be heard. + * `Container.coords` — a dict whose range class's key slot *is* its type + designator, so the dict key names the entry's class. The asset360 shape. + * `Container.systems` — a key slot whose range descends from `uri`, where a + CURIE dict key and its expansion in the payload are one identity. +license: https://creativecommons.org/publicdomain/zero/1.0/ +imports: + - linkml:types +prefixes: + dk: https://w3id.org/linkml/examples/inlined_dict_key/ + ex: https://example.org/dk/ + linkml: https://w3id.org/linkml/ +default_prefix: dk +default_range: string + +classes: + Container: + attributes: + people: + range: Person + multivalued: true + inlined: true + coords: + range: Coordinate + multivalued: true + inlined: true + systems: + range: System + multivalued: true + inlined: true + + # `pid` is `required` on purpose: the injection is what satisfies it, and a + # loader that stopped injecting would fail loudly here rather than quietly. + Person: + attributes: + pid: {range: string, key: true, required: true} + name: {range: string} + + # key and designator on one slot — the asset360 `PositioningSystemCoordinate` + # shape (`slot_usage: typeURI: {key: true}` over a `designates_type` slot). + Coordinate: + attributes: + typeURI: + range: uriorcurie + designates_type: true + key: true + value: {range: string} + + # declares a `class_uri`, so its canonical designator value and its + # schema-native URI are two different accepted spellings — exactly the + # asset360 `LinearCoordinate` divergence. + LinearCoordinate: + is_a: Coordinate + class_uri: ex:Linear + attributes: + measure: {range: float} + + # a key slot whose range descends from `uri`: a CURIE dict key and the + # expanded payload are one identity (rule 2), so they must not diverge. + System: + attributes: + systemType: {range: uri, key: true, required: true} + label: {range: string} diff --git a/src/runtime/tests/diff_class_change.rs b/src/runtime/tests/diff_class_change.rs new file mode 100644 index 0000000..397705f --- /dev/null +++ b/src/runtime/tests/diff_class_change.rs @@ -0,0 +1,215 @@ +//! Spec addendum rules 3 and 4 (spike findings D2). +//! +//! Rule 3: when `diff` pairs two objects whose classes differ, the change is a +//! whole-element `Update` — never field-level recursion across two classes. +//! Rule 4: a delta whose value cannot be built or applied at its resolved +//! location records its path in `PatchTrace::failed`; it never voids the batch. + +use linkml_runtime::{ + diff, load_json_str, patch, Delta, DeltaOp, DiffOptions, LinkMLInstance, PatchOptions, +}; +use linkml_schemaview::identifier::{converter_from_schema, Identifier}; +use linkml_schemaview::io::from_yaml; +use linkml_schemaview::schemaview::{ClassView, SchemaView}; +use linkml_schemaview::Converter; +use serde_json::{json, Value as JsonValue}; +use std::path::PathBuf; + +struct Fixture { + sv: SchemaView, + conv: Converter, + inventory: ClassView, +} + +fn fixture() -> Fixture { + let mut p = PathBuf::from(env!("CARGO_MANIFEST_DIR")); + p.push("tests/data/identity_class_change.yaml"); + let schema = from_yaml(&p).unwrap(); + let mut sv = SchemaView::new(); + sv.add_schema(schema.clone()).unwrap(); + let conv = converter_from_schema(&schema); + let inventory = sv + .get_class(&Identifier::new("Inventory"), &conv) + .unwrap() + .expect("class not found"); + Fixture { + sv, + conv, + inventory, + } +} + +impl Fixture { + fn load(&self, v: JsonValue) -> LinkMLInstance { + load_json_str(&v.to_string(), &self.sv, &self.inventory, &self.conv) + .unwrap() + .into_instance() + .unwrap() + } +} + +fn bolt() -> JsonValue { + json!({"typeURI": "part:Bolt", "code": "B1", "thread": "M8"}) +} +fn nut() -> JsonValue { + json!({"typeURI": "part:Nut", "code": "B1", "pitch": "1.25"}) +} + +fn class_name_at(v: &LinkMLInstance, path: &[&str]) -> String { + let segs: Vec = path.iter().map(|s| s.to_string()).collect(); + match v.navigate_path(&segs).expect("path must resolve") { + LinkMLInstance::Object { class, .. } => class.name().to_string(), + other => panic!("expected an object, got {}", other.to_json()), + } +} + +/// Rule 3, both directions: same identity label, different class. +#[test] +fn class_change_is_one_whole_element_update() { + let f = fixture(); + for (before, after, want_class) in [(bolt(), nut(), "Nut"), (nut(), bolt(), "Bolt")] { + let src = f.load(json!({"parts": [before.clone()]})); + let tgt = f.load(json!({"parts": [after.clone()]})); + let deltas = diff(&src, &tgt, DiffOptions::new(true)); + assert_eq!( + deltas.len(), + 1, + "a class change is ONE delta, not field recursion: {deltas:#?}" + ); + let d = &deltas[0]; + assert_eq!(d.path, vec!["parts".to_string(), "B1".to_string()]); + assert_eq!(d.op, DeltaOp::Update); + assert_eq!( + d.new.as_ref().and_then(|v| v.get("code")), + Some(&json!("B1")) + ); + + // Rule 3 round trip: patch rebuilds the element as the NEW class, + // polymorphically, through the slot. + let (patched, trace) = patch(&src, &deltas, PatchOptions::default()).unwrap(); + assert!(trace.failed.is_empty(), "{:?}", trace.failed); + assert!(patched.equals(&tgt, true), "patched: {}", patched.to_json()); + assert_eq!(class_name_at(&patched, &["parts", "B1"]), want_class); + } +} + +/// Rule 3 is not list-specific: a single-valued inlined object changing class +/// is the same whole-element replacement. +#[test] +fn class_change_of_single_valued_object_is_one_update() { + let f = fixture(); + let src = f.load(json!({"featured": bolt()})); + let tgt = f.load(json!({"featured": nut()})); + let deltas = diff(&src, &tgt, DiffOptions::new(true)); + assert_eq!(deltas.len(), 1, "{deltas:#?}"); + assert_eq!(deltas[0].path, vec!["featured".to_string()]); + assert_eq!(deltas[0].op, DeltaOp::Update); + let (patched, trace) = patch(&src, &deltas, PatchOptions::default()).unwrap(); + assert!(trace.failed.is_empty(), "{:?}", trace.failed); + assert!(patched.equals(&tgt, true), "{}", patched.to_json()); + assert_eq!(class_name_at(&patched, &["featured"]), "Nut"); +} + +/// Same class, changed key value: still a whole-element replacement, and the +/// key-value branch that produces it is untouched by rule 3. +#[test] +fn same_class_key_change_still_replaces_whole_element() { + let f = fixture(); + let src = f.load(json!({"owner": {"oid": "o1", "oname": "Ada"}})); + let tgt = f.load(json!({"owner": {"oid": "o2", "oname": "Ada"}})); + let deltas = diff( + &src, + &tgt, + DiffOptions { + treat_changed_identifier_as_new_object: true, + ..DiffOptions::new(true) + }, + ); + assert_eq!(deltas.len(), 1, "{deltas:#?}"); + assert_eq!(deltas[0].path, vec!["owner".to_string()]); + assert_eq!(deltas[0].op, DeltaOp::Update); +} + +/// Rule 3 qualifies class identity by schema *id*, not by `SchemaView` +/// instance: two views built separately over the same schema must still diff +/// field-by-field. (Cross-*schema* pairings do coarsen to whole-element +/// Updates — that is the documented, intended consequence.) +#[test] +fn separate_schema_views_over_one_schema_still_diff_finely() { + let a = fixture(); + let b = fixture(); + let src = a.load(json!({"parts": [bolt()]})); + let mut edited = bolt(); + edited["thread"] = json!("M10"); + let tgt = b.load(json!({"parts": [edited]})); + let deltas = diff(&src, &tgt, DiffOptions::new(true)); + assert_eq!(deltas.len(), 1, "{deltas:#?}"); + assert_eq!( + deltas[0].path, + vec!["parts".to_string(), "B1".to_string(), "thread".to_string()], + "a second SchemaView over the same schema is not a class change" + ); +} + +/// Rule 4: a hand-built delta whose value cannot be built at its resolved +/// location (`thread` is a `Bolt` slot; the element is a `Nut`) fails soft — +/// `Ok`, path in `trace.failed`, tree untouched — and the OTHER deltas in the +/// same batch still apply. This is the spike's exact probe. +#[test] +fn unbuildable_delta_fails_soft_without_voiding_the_batch() { + let f = fixture(); + let golden = f.load(json!({"parts": [nut()]})); + let bad = Delta { + path: vec!["parts".to_string(), "B1".to_string(), "thread".to_string()], + op: DeltaOp::Update, + old: Some(json!("M8")), + new: Some(json!("M10")), + }; + let good = Delta { + path: vec!["parts".to_string(), "B1".to_string(), "note".to_string()], + op: DeltaOp::Add, + old: None, + new: Some(json!("stocked")), + }; + let (patched, trace) = patch( + &golden, + &[bad.clone(), good.clone()], + PatchOptions::default(), + ) + .unwrap(); + assert_eq!(trace.failed, vec![bad.path.clone()]); + assert!( + patched.equals( + &f.load(json!({"parts": [{"typeURI": "part:Nut", "code": "B1", + "pitch": "1.25", "note": "stocked"}]})), + true + ), + "the good delta must still land, the bad one must change nothing: {}", + patched.to_json() + ); + assert!( + patched + .navigate_path(&["parts".to_string(), "B1".to_string(), "thread".to_string()]) + .is_none(), + "the unbuildable delta must leave no trace in the tree" + ); +} + +/// Rule 4 at the whole-element level: an `Update` carrying a payload no class +/// in the slot's range can accept fails soft too. +#[test] +fn unbuildable_whole_element_update_fails_soft() { + let f = fixture(); + let golden = f.load(json!({"parts": [nut()]})); + let bad = Delta { + path: vec!["parts".to_string(), "B1".to_string()], + op: DeltaOp::Update, + old: Some(nut()), + // a scalar where the slot's range is a class: unbuildable + new: Some(json!("not-an-object")), + }; + let (patched, trace) = patch(&golden, std::slice::from_ref(&bad), PatchOptions::default()) + .expect("a bad delta must not void the batch"); + assert_eq!(trace.failed, vec![bad.path.clone()]); + assert!(patched.equals(&golden, true), "nothing may change"); +} diff --git a/src/runtime/tests/diff_opaque.rs b/src/runtime/tests/diff_opaque.rs new file mode 100644 index 0000000..7938dfd --- /dev/null +++ b/src/runtime/tests/diff_opaque.rs @@ -0,0 +1,184 @@ +use linkml_runtime::{ + diff, load_json_str, patch, Delta, DeltaOp, DiffOptions, LinkMLInstance, PatchOptions, +}; +use linkml_schemaview::identifier::{converter_from_schema, Identifier}; +use linkml_schemaview::io::from_yaml; +use linkml_schemaview::schemaview::{ClassView, SchemaView}; +use linkml_schemaview::Converter; +use serde_json::{json, Value as JsonValue}; +use std::path::PathBuf; + +struct Fixture { + sv: SchemaView, + conv: Converter, + service: ClassView, +} + +fn fixture() -> Fixture { + let mut p = PathBuf::from(env!("CARGO_MANIFEST_DIR")); + p.push("tests/data/identity.yaml"); + let schema = from_yaml(&p).unwrap(); + let mut sv = SchemaView::new(); + sv.add_schema(schema.clone()).unwrap(); + let conv = converter_from_schema(&schema); + let service = sv + .get_class(&Identifier::new("Service"), &conv) + .unwrap() + .expect("class not found"); + Fixture { sv, conv, service } +} + +impl Fixture { + fn load(&self, v: JsonValue) -> LinkMLInstance { + load_json_str(&v.to_string(), &self.sv, &self.service, &self.conv) + .unwrap() + .into_instance() + .unwrap() + } +} + +fn diff2(f: &Fixture, before: JsonValue, after: JsonValue) -> Vec { + diff(&f.load(before), &f.load(after), DiffOptions::new(true)) +} + +fn only(deltas: &[Delta]) -> &Delta { + assert_eq!(deltas.len(), 1, "expected exactly one delta: {deltas:#?}"); + &deltas[0] +} + +fn square() -> Vec { + vec![ + json!({"x": 4.35, "y": 50.85}), + json!({"x": 4.36, "y": 50.85}), + json!({"x": 4.36, "y": 50.86}), + json!({"x": 4.35, "y": 50.86}), + ] +} + +fn outline(items: Vec) -> JsonValue { + json!({"name": "svc", "outline": items}) +} + +#[test] +fn opaque_ring_edit_is_one_whole_slot_update() { + let f = fixture(); + + let mut moved = square(); + moved[1] = json!({"x": 4.37, "y": 50.85}); + let mut inserted = square(); + inserted.insert(2, json!({"x": 4.365, "y": 50.855})); + let dropped_first = square()[1..].to_vec(); + let mut reversed = square(); + reversed.reverse(); + + for (label, after) in [ + ("move one vertex", moved), + ("insert a vertex mid-ring", inserted), + ("drop the first vertex", dropped_first), + ("reverse the ring", reversed), + ] { + let deltas = diff2(&f, outline(square()), outline(after)); + let delta = only(&deltas); + assert_eq!(delta.path, vec!["outline".to_string()], "{label}"); + assert_eq!(delta.op, DeltaOp::Update, "{label}"); + assert_eq!( + delta + .old + .as_ref() + .and_then(|v| v.as_array()) + .map(|a| a.len()), + Some(4), + "{label}: old must be the whole slot" + ); + assert!(delta.new.as_ref().is_some_and(|v| v.is_array()), "{label}"); + } +} + +#[test] +fn opaque_slot_unchanged_emits_nothing() { + let f = fixture(); + let deltas = diff2(&f, outline(square()), outline(square())); + assert!(deltas.is_empty(), "{deltas:#?}"); +} + +#[test] +fn opaque_scalar_list_is_one_whole_slot_update() { + let f = fixture(); + let before = json!({"name": "svc", "opaqueTags": ["a", "b"]}); + let after = json!({"name": "svc", "opaqueTags": ["b", "c", "d"]}); + let deltas = diff2(&f, before, after); + let delta = only(&deltas); + assert_eq!(delta.path, vec!["opaqueTags".to_string()]); + assert_eq!(delta.op, DeltaOp::Update); +} + +#[test] +fn opaque_single_valued_object_is_one_whole_value_update() { + let f = fixture(); + let before = json!({"name": "svc", "profile": {"bio": "b", "motto": "old"}}); + let after = json!({"name": "svc", "profile": {"bio": "b", "motto": "new"}}); + let deltas = diff2(&f, before, after); + let delta = only(&deltas); + assert_eq!(delta.path, vec!["profile".to_string()]); + assert_eq!(delta.op, DeltaOp::Update); + assert_eq!(delta.old, Some(json!({"bio": "b", "motto": "old"}))); + assert_eq!(delta.new, Some(json!({"bio": "b", "motto": "new"}))); +} + +#[test] +fn opaque_whole_slot_update_round_trips_through_patch() { + let f = fixture(); + let before = outline(square()); + let mut moved = square(); + moved[1] = json!({"x": 4.37, "y": 50.85}); + let after = outline(moved); + let deltas = diff2(&f, before.clone(), after.clone()); + let (patched, trace) = patch(&f.load(before), &deltas, PatchOptions::default()).unwrap(); + assert!(trace.failed.is_empty(), "{:?}", trace.failed); + assert!( + patched.equals(&f.load(after), true), + "round-trip mismatch: {}", + patched.to_json() + ); +} + +#[test] +fn patch_below_opaque_slot_is_reported_failed_not_guessed() { + let f = fixture(); + let golden = f.load(outline(square())); + let delta = Delta { + path: vec!["outline".to_string(), "1".to_string(), "x".to_string()], + op: DeltaOp::Update, + old: Some(json!(4.36)), + new: Some(json!(9.99)), + }; + let (patched, trace) = patch( + &golden, + std::slice::from_ref(&delta), + PatchOptions::default(), + ) + .unwrap(); + assert_eq!(trace.failed, vec![delta.path.clone()]); + assert!( + patched.equals(&golden, true), + "nothing may change: {}", + patched.to_json() + ); +} + +#[test] +fn patch_at_opaque_slot_path_still_applies_whole_value() { + let f = fixture(); + let golden = f.load(outline(square())); + let mut moved = square(); + moved[1] = json!({"x": 4.37, "y": 50.85}); + let delta = Delta { + path: vec!["outline".to_string()], + op: DeltaOp::Update, + old: Some(json!(square())), + new: Some(json!(moved.clone())), + }; + let (patched, trace) = patch(&golden, &[delta], PatchOptions::default()).unwrap(); + assert!(trace.failed.is_empty(), "{:?}", trace.failed); + assert!(patched.equals(&f.load(outline(moved)), true)); +} diff --git a/src/runtime/tests/diff_unique_keys.rs b/src/runtime/tests/diff_unique_keys.rs new file mode 100644 index 0000000..f2493b3 --- /dev/null +++ b/src/runtime/tests/diff_unique_keys.rs @@ -0,0 +1,657 @@ +use linkml_runtime::{ + diff, load_json_str, patch, Delta, DeltaOp, DiffOptions, LinkMLInstance, PatchOptions, +}; +use linkml_schemaview::identifier::{converter_from_schema, Identifier}; +use linkml_schemaview::io::from_yaml; +use linkml_schemaview::schemaview::{ClassView, SchemaView}; +use linkml_schemaview::Converter; +use serde_json::{json, Value as JsonValue}; +use std::path::PathBuf; + +struct Fixture { + sv: SchemaView, + conv: Converter, + service: ClassView, +} + +fn fixture() -> Fixture { + let mut p = PathBuf::from(env!("CARGO_MANIFEST_DIR")); + p.push("tests/data/identity.yaml"); + let schema = from_yaml(&p).unwrap(); + let mut sv = SchemaView::new(); + sv.add_schema(schema.clone()).unwrap(); + let conv = converter_from_schema(&schema); + let service = sv + .get_class(&Identifier::new("Service"), &conv) + .unwrap() + .expect("class not found"); + Fixture { sv, conv, service } +} + +impl Fixture { + fn load(&self, v: JsonValue) -> LinkMLInstance { + load_json_str(&v.to_string(), &self.sv, &self.service, &self.conv) + .unwrap() + .into_instance() + .unwrap() + } +} + +fn diff2(f: &Fixture, before: JsonValue, after: JsonValue) -> Vec { + diff(&f.load(before), &f.load(after), DiffOptions::new(true)) +} + +fn only(deltas: &[Delta]) -> &Delta { + assert_eq!(deltas.len(), 1, "expected exactly one delta: {deltas:#?}"); + &deltas[0] +} + +fn e() -> JsonValue { + json!({"phoneNumber": "09/241.25.00", "hasNumberFunction": "Emergency_Number"}) +} +fn n() -> JsonValue { + json!({"phoneNumber": "09/241.25.03", "hasNumberFunction": "Non_Urgent_Communication"}) +} +fn o() -> JsonValue { + json!({"phoneNumber": "09/241.25.10", "hasNumberFunction": "Operator"}) +} +fn phones(items: Vec) -> JsonValue { + json!({"name": "svc", "hasPhoneNumber": items}) +} + +#[test] +fn unique_key_matching_targets_field_edits_by_key() { + let f = fixture(); + let mut n2 = n(); + n2["phoneNumber"] = json!("09/241.25.99"); + let deltas = diff2(&f, phones(vec![e(), n()]), phones(vec![e(), n2])); + let delta = only(&deltas); + assert_eq!( + delta.path, + vec![ + "hasPhoneNumber".to_string(), + "Non_Urgent_Communication".to_string(), + "phoneNumber".to_string() + ] + ); + assert_eq!(delta.op, DeltaOp::Update); +} + +#[test] +fn unique_key_matching_ignores_reorder() { + let f = fixture(); + let deltas = diff2(&f, phones(vec![e(), n(), o()]), phones(vec![o(), n(), e()])); + assert!(deltas.is_empty(), "reorder must be invisible: {deltas:#?}"); +} + +#[test] +fn unique_key_remove_and_add_are_key_addressed() { + let f = fixture(); + let deltas = diff2(&f, phones(vec![e(), n()]), phones(vec![n()])); + let delta = only(&deltas); + assert_eq!(delta.op, DeltaOp::Remove); + assert_eq!( + delta.path, + vec!["hasPhoneNumber".to_string(), "Emergency_Number".to_string()] + ); + assert_eq!(delta.old, Some(e())); + + let deltas = diff2(&f, phones(vec![e(), n()]), phones(vec![e(), n(), o()])); + let delta = only(&deltas); + assert_eq!(delta.op, DeltaOp::Add); + assert_eq!( + delta.path, + vec!["hasPhoneNumber".to_string(), "Operator".to_string()] + ); + assert_eq!(delta.new, Some(o())); +} + +#[test] +fn changing_the_key_slot_is_remove_plus_add() { + let f = fixture(); + let mut moved = e(); + moved["hasNumberFunction"] = json!("Operator"); + let deltas = diff2(&f, phones(vec![e(), n()]), phones(vec![moved.clone(), n()])); + assert_eq!(deltas.len(), 2, "{deltas:#?}"); + assert!(deltas + .iter() + .any(|d| d.op == DeltaOp::Remove && d.old.as_ref() == Some(&e()))); + assert!(deltas + .iter() + .any(|d| d.op == DeltaOp::Add && d.new.as_ref() == Some(&moved))); +} + +#[test] +fn duplicate_unique_key_data_falls_back_to_positional() { + let f = fixture(); + // two Emergency numbers: violates the class claim; data must keep + // today's positional behaviour, with numeric path segments + let e2 = json!({"phoneNumber": "09/000.00.00", "hasNumberFunction": "Emergency_Number"}); + let mut e2_edit = e2.clone(); + e2_edit["phoneNumber"] = json!("09/111.11.11"); + let deltas = diff2(&f, phones(vec![e(), e2]), phones(vec![e(), e2_edit])); + let delta = only(&deltas); + assert_eq!( + delta.path, + vec![ + "hasPhoneNumber".to_string(), + "1".to_string(), + "phoneNumber".to_string() + ], + "positional fallback must use numeric segments, never the duplicate label" + ); +} + +#[test] +fn duplicate_key_data_falls_back_to_positional_not_collapse() { + let f = fixture(); + // Label declares `lang` as key; a list that repeats the key must not be + // silently collapsed by keyed matching — uniform guard, positional fallback. + let before = json!({"name": "svc", "labelList": [ + {"lang": "nl", "text": "a"}, {"lang": "nl", "text": "b"}]}); + let after = json!({"name": "svc", "labelList": [ + {"lang": "nl", "text": "a"}, {"lang": "nl", "text": "B"}]}); + let deltas = diff2(&f, before, after); + let delta = only(&deltas); + assert_eq!( + delta.path, + vec!["labelList".to_string(), "1".to_string(), "text".to_string()], + "duplicate key labels must fall back to plain numeric segments" + ); +} + +#[test] +fn undeclared_class_keeps_positional_cascade() { + let f = fixture(); + // PlainPhoneNumber has no unique_keys: removal still cascades as today. + // Both surviving elements shift up, and each differs from the element that + // used to sit at its index in both slots: 2 x 2 shifted-slot Updates plus + // the trailing Remove. + let before = json!({"name": "svc", "plainPhoneNumber": [e(), n(), o()]}); + let after = json!({"name": "svc", "plainPhoneNumber": [n(), o()]}); + let deltas = diff2(&f, before, after); + assert_eq!(deltas.len(), 5, "{deltas:#?}"); + assert!( + deltas.iter().all(|d| d.path[1].parse::().is_ok()), + "positional cascade must use numeric segments: {deltas:#?}" + ); +} + +#[test] +fn composite_unique_key_uses_json_array_segment() { + let f = fixture(); + let a = json!({"kind": "Emergency", "phone": "02/111.11.11", "note": "old"}); + let mut a2 = a.clone(); + a2["note"] = json!("new"); + let b = json!({"kind": "Operator", "phone": "02/333.33.33"}); + let before = json!({"name": "svc", "contacts": [a, b.clone()]}); + let after = json!({"name": "svc", "contacts": [a2, b]}); + let deltas = diff2(&f, before, after); + let delta = only(&deltas); + assert_eq!( + delta.path, + vec![ + "contacts".to_string(), + r#"["Emergency","02/111.11.11"]"#.to_string(), + "note".to_string() + ] + ); +} + +#[test] +fn inherited_unique_keys_drive_matching() { + let f = fixture(); + // EmergencyPhoneNumber inherits one_number_per_function via is_a + let x = json!({"phoneNumber": "1", "hasNumberFunction": "Emergency_Number", "note": "a"}); + let mut x2 = x.clone(); + x2["note"] = json!("b"); + let y = json!({"phoneNumber": "2", "hasNumberFunction": "Operator", "note": "c"}); + let before = json!({"name": "svc", "escalation": [x, y.clone()]}); + let after = json!({"name": "svc", "escalation": [x2, y]}); + let deltas = diff2(&f, before, after); + let delta = only(&deltas); + assert_eq!( + delta.path, + vec![ + "escalation".to_string(), + "Emergency_Number".to_string(), + "note".to_string() + ] + ); +} + +#[test] +fn patch_locates_element_by_unique_key_under_drift() { + let f = fixture(); + // producer saw [E, N]; golden drifted to [N, E, O] + let golden = f.load(phones(vec![n(), e(), o()])); + let delta = Delta { + path: vec![ + "hasPhoneNumber".to_string(), + "Emergency_Number".to_string(), + "phoneNumber".to_string(), + ], + op: DeltaOp::Update, + old: Some(json!("09/241.25.00")), + new: Some(json!("09/999.99.99")), + }; + let (patched, trace) = patch(&golden, &[delta], PatchOptions::default()).unwrap(); + assert!(trace.failed.is_empty(), "{:?}", trace.failed); + let mut e2 = e(); + e2["phoneNumber"] = json!("09/999.99.99"); + assert!( + patched.equals(&f.load(phones(vec![n(), e2, o()])), true), + "the edit must land on E wherever it sits: {}", + patched.to_json() + ); +} + +#[test] +fn patch_reports_ambiguous_unique_key_instead_of_guessing() { + let f = fixture(); + // golden drifted into two Emergency elements: locating "the" one is a guess + let e2 = json!({"phoneNumber": "09/000.00.00", "hasNumberFunction": "Emergency_Number"}); + let golden = f.load(phones(vec![e(), e2])); + let delta = Delta { + path: vec![ + "hasPhoneNumber".to_string(), + "Emergency_Number".to_string(), + "phoneNumber".to_string(), + ], + op: DeltaOp::Update, + old: Some(json!("09/241.25.00")), + new: Some(json!("09/999.99.99")), + }; + let (patched, trace) = patch( + &golden, + std::slice::from_ref(&delta), + PatchOptions::default(), + ) + .unwrap(); + assert_eq!(trace.failed, vec![delta.path.clone()]); + assert!(patched.equals(&golden, true), "nothing may change"); +} + +#[test] +fn patch_refuses_positional_segment_into_identity_addressed_list() { + let f = fixture(); + let golden = f.load(phones(vec![e(), n()])); + // A stale positional patch aimed at a list whose elements all carry + // unique identity labels: applying "index 0" would be a guess, and for + // numeric-valued identity labels it would silently hit the wrong element. + let delta = Delta { + path: vec!["hasPhoneNumber".to_string(), "0".to_string()], + op: DeltaOp::Remove, + old: Some(e()), + new: None, + }; + let (patched, trace) = patch( + &golden, + std::slice::from_ref(&delta), + PatchOptions::default(), + ) + .unwrap(); + assert_eq!(trace.failed, vec![delta.path.clone()]); + assert!(patched.equals(&golden, true), "nothing may be removed"); +} + +#[test] +fn patch_refuses_ambiguous_duplicate_key_labels() { + let f = fixture(); + // Duplicate key/identifier labels refuse exactly like duplicate + // unique_keys labels — the uniform rule on the patch side. + let golden = f.load(json!({"name": "svc", "labelList": [ + {"lang": "nl", "text": "a"}, {"lang": "nl", "text": "b"}]})); + let delta = Delta { + path: vec![ + "labelList".to_string(), + "nl".to_string(), + "text".to_string(), + ], + op: DeltaOp::Update, + old: Some(json!("a")), + new: Some(json!("z")), + }; + let (patched, trace) = patch( + &golden, + std::slice::from_ref(&delta), + PatchOptions::default(), + ) + .unwrap(); + assert_eq!(trace.failed, vec![delta.path.clone()]); + assert!(patched.equals(&golden, true), "nothing may change"); +} + +#[test] +fn unique_key_deltas_round_trip_through_patch() { + let f = fixture(); + let mut n2 = n(); + n2["phoneNumber"] = json!("09/241.25.99"); + for (before, after) in [ + (phones(vec![e(), n()]), phones(vec![e(), n2])), // field edit + (phones(vec![e(), n()]), phones(vec![n()])), // remove + (phones(vec![e(), n()]), phones(vec![e(), n(), o()])), // add + ] { + let deltas = diff2(&f, before.clone(), after.clone()); + let (patched, trace) = patch(&f.load(before), &deltas, PatchOptions::default()).unwrap(); + assert!(trace.failed.is_empty(), "{:?}", trace.failed); + assert!( + patched.equals(&f.load(after), true), + "{}", + patched.to_json() + ); + } +} + +#[test] +fn patch_refuses_positional_update_into_identity_addressed_list() { + let f = fixture(); + let golden = f.load(phones(vec![e(), n()])); + // A stale positional Update against a keyed-shaped list resolves to no + // element, and its payload's identity ("Emergency_Number") does not name + // the address ("0"). It must report: appending would invent an element, + // and on `main` this address overwrote whichever element sat at index 0. + let mut e2 = e(); + e2["phoneNumber"] = json!("09/999.99.99"); + let delta = Delta { + path: vec!["hasPhoneNumber".to_string(), "0".to_string()], + op: DeltaOp::Update, + old: Some(e()), + new: Some(e2), + }; + let (patched, trace) = patch( + &golden, + std::slice::from_ref(&delta), + PatchOptions::default(), + ) + .unwrap(); + assert_eq!(trace.failed, vec![delta.path.clone()]); + assert!(patched.equals(&golden, true), "nothing may be appended"); +} + +#[test] +fn update_whose_payload_names_its_address_is_re_added() { + let f = fixture(); + let golden = f.load(phones(vec![e(), n()])); + // Multi-source: one source dropped the Operator entry, another still has + // it and describes it as an Update (its delta was computed against an + // older golden). The payload's identity names exactly the element the path + // addresses, so the Update re-adds it — the field comes back rather than + // being reported as a failure and silently lost. + let delta = Delta { + path: vec!["hasPhoneNumber".to_string(), "Operator".to_string()], + op: DeltaOp::Update, + old: Some(o()), + new: Some(o()), + }; + let (patched, trace) = patch( + &golden, + std::slice::from_ref(&delta), + PatchOptions::default(), + ) + .unwrap(); + assert!(trace.failed.is_empty(), "failed: {:?}", trace.failed); + assert!( + patched.equals(&f.load(phones(vec![e(), n(), o()])), true), + "{}", + patched.to_json() + ); + // And the round trip is clean: re-diffing the result against the intent + // yields nothing. + assert!( + diff2(&f, patched.to_json(), phones(vec![e(), n(), o()])).is_empty(), + "re-added element must be indistinguishable from an Add" + ); +} + +#[test] +fn patch_refuses_update_whose_payload_contradicts_its_address() { + let f = fixture(); + let golden = f.load(phones(vec![e(), n()])); + // A label address that resolves to nothing AND a payload naming a + // different element: the address is stale, not a re-add. Appending would + // duplicate the Non_Urgent_Communication entry the list already carries + // and knock the whole list off keyed matching. + let mut n2 = n(); + n2["phoneNumber"] = json!("09/999.99.99"); + let delta = Delta { + path: vec!["hasPhoneNumber".to_string(), "Operator".to_string()], + op: DeltaOp::Update, + old: Some(o()), + new: Some(n2), + }; + let (patched, trace) = patch( + &golden, + std::slice::from_ref(&delta), + PatchOptions::default(), + ) + .unwrap(); + assert_eq!(trace.failed, vec![delta.path.clone()]); + assert!(patched.equals(&golden, true), "nothing may be appended"); +} + +/// A second element carrying the same identity label as [`e`]. +fn e2() -> JsonValue { + json!({"phoneNumber": "09/000.00.00", "hasNumberFunction": "Emergency_Number"}) +} + +#[test] +fn keyed_source_to_duplicated_target_is_one_whole_slot_update() { + let f = fixture(); + // The source list is keyed-shaped, the target repeats a label. Positional + // segments against a keyed-shaped source are unappliable by design (patch + // resolves such a list by label only), so the honest description is: this + // list stopped having coherent element identity — one whole-slot Update. + let before = phones(vec![e(), n()]); + let after = phones(vec![e(), e2()]); + let deltas = diff2(&f, before.clone(), after.clone()); + let delta = only(&deltas); + assert_eq!(delta.path, vec!["hasPhoneNumber".to_string()]); + assert_eq!(delta.op, DeltaOp::Update); + assert_eq!(delta.old, Some(json!([e(), n()]))); + assert_eq!(delta.new, Some(json!([e(), e2()]))); + + let (patched, trace) = patch(&f.load(before), &deltas, PatchOptions::default()).unwrap(); + assert!(trace.failed.is_empty(), "{:?}", trace.failed); + assert!( + patched.equals(&f.load(after), true), + "patch(a, diff(a,b)) must equal b: {}", + patched.to_json() + ); +} + +#[test] +fn duplicated_source_to_keyed_target_stays_positional_and_round_trips() { + let f = fixture(); + // The mirror image: the source is NOT keyed-shaped, so numeric segments + // are exactly what patch resolves against it. Keep positional deltas. + // + // The second element differs from its target only in the key-bearing slot, + // so the whole edit is one delta. A multi-delta variant is order-dependent + // for a reason unrelated to the source-keyed fallback, documented on + // `patch`: once the key edit lands the list becomes keyed-shaped, and any + // numeric segment still queued is reported failed rather than guessed. + let dup = json!({"phoneNumber": "09/241.25.03", "hasNumberFunction": "Emergency_Number"}); + let before = phones(vec![e(), dup]); + let after = phones(vec![e(), n()]); + let deltas = diff2(&f, before.clone(), after.clone()); + assert!(!deltas.is_empty(), "expected positional deltas"); + for d in &deltas { + assert_eq!( + d.path[0], "hasPhoneNumber", + "unexpected delta path: {:?}", + d.path + ); + assert!( + d.path.len() > 1 && d.path[1].parse::().is_ok(), + "a non-keyed-shaped source keeps numeric segments: {:?}", + d.path + ); + } + let (patched, trace) = patch(&f.load(before), &deltas, PatchOptions::default()).unwrap(); + assert!(trace.failed.is_empty(), "{:?}", trace.failed); + assert!( + patched.equals(&f.load(after), true), + "patch(a, diff(a,b)) must equal b: {}", + patched.to_json() + ); +} + +// --------------------------------------------------------------------------- +// D3 — a type designator is never an element identity (spec addendum rule 1). +// +// A designator's value is a function of the element's class, so it cannot tell +// two elements of one class apart. The engine skips a designator key entirely: +// identity falls through to `unique_keys`, else the list is positional. + +struct DesignatorFixture { + sv: SchemaView, + conv: Converter, + ring: ClassView, +} + +fn designator_fixture() -> DesignatorFixture { + let mut p = PathBuf::from(env!("CARGO_MANIFEST_DIR")); + p.push("tests/data/identity_type_designator_key.yaml"); + let schema = from_yaml(&p).unwrap(); + let mut sv = SchemaView::new(); + sv.add_schema(schema.clone()).unwrap(); + let conv = converter_from_schema(&schema); + let ring = sv + .get_class(&Identifier::new("Ring"), &conv) + .unwrap() + .expect("class not found"); + DesignatorFixture { sv, conv, ring } +} + +impl DesignatorFixture { + fn load(&self, v: JsonValue) -> LinkMLInstance { + load_json_str(&v.to_string(), &self.sv, &self.ring, &self.conv) + .unwrap() + .into_instance() + .unwrap() + } + + fn diff2(&self, before: JsonValue, after: JsonValue) -> Vec { + diff( + &self.load(before), + &self.load(after), + DiffOptions::new(true), + ) + } +} + +fn m1() -> JsonValue { + json!({"code": "M1", "label": "one"}) +} +fn m2() -> JsonValue { + json!({"code": "M2", "label": "two"}) +} + +#[test] +fn designator_key_does_not_shadow_unique_keys() { + // `Marker` declares BOTH a designator key (`typeURI`) and `unique_keys`. + // The designator labels every marker alike; the real identity is `by_code` + // (the name-sorted first entry). Reordering the list must be a no-op. + let f = designator_fixture(); + let deltas = f.diff2( + json!({"markers": [m1(), m2()]}), + json!({"markers": [m2(), m1()]}), + ); + assert!( + deltas.is_empty(), + "a reorder of unique_keys-identified markers is not a change: {deltas:#?}" + ); +} + +#[test] +fn designator_key_shadow_field_edit_is_addressed_by_the_unique_key_label() { + let f = designator_fixture(); + let mut edited = m2(); + edited["label"] = json!("TWO"); + let before = json!({"markers": [m1(), m2()]}); + let after = json!({"markers": [m1(), edited]}); + let deltas = f.diff2(before.clone(), after.clone()); + let delta = only(&deltas); + assert_eq!( + delta.path, + vec!["markers".to_string(), "M2".to_string(), "label".to_string()], + "the unique_keys label addresses the edit, not the constant designator" + ); + assert_eq!(delta.op, DeltaOp::Update); + + // diff ↔ patch ↔ navigate stay symmetric: the emitted label resolves + // through `resolve_list_segment` for both consumers. + let loaded = f.load(before); + assert!( + loaded + .navigate_path(["markers", "M2", "label"]) + .is_some_and(|v| v.to_json() == json!("two")), + "navigate must resolve the label diff emitted" + ); + let (patched, trace) = patch(&loaded, &deltas, PatchOptions::default()).unwrap(); + assert!(trace.failed.is_empty(), "{:?}", trace.failed); + assert!( + patched.equals(&f.load(after), true), + "patch(a, diff(a,b)) must equal b: {}", + patched.to_json() + ); +} + +#[test] +fn designator_keyed_class_without_unique_keys_is_positional() { + // `Coordinate` is keyed by its designator and declares no `unique_keys`, so + // it has no element identity at all: even a one-element list — where the + // constant designator would pass the uniqueness guard — is positional. + let f = designator_fixture(); + let before = json!({"vertices": [{"x": 1.0, "y": 2.0}]}); + let after = json!({"vertices": [{"x": 1.0, "y": 9.0}]}); + let deltas = f.diff2(before.clone(), after.clone()); + let delta = only(&deltas); + assert_eq!( + delta.path, + vec!["vertices".to_string(), "0".to_string(), "y".to_string()], + "a designator key yields no label, so the list is addressed by index" + ); + + let (patched, trace) = patch(&f.load(before), &deltas, PatchOptions::default()).unwrap(); + assert!(trace.failed.is_empty(), "{:?}", trace.failed); + assert!( + patched.equals(&f.load(after), true), + "{}", + patched.to_json() + ); +} + +#[test] +fn polymorphic_designator_keyed_list_is_positional() { + // One element per subtype: the designator values differ, so they used to + // pass as a per-element identity ("at most one element per subtype"). + // A designator still describes the class, never the element — positional. + let f = designator_fixture(); + let base = json!({"typeURI": "identity:KeyedTypedThing", "label": "base"}); + let special = json!({"typeURI": "identity:SpecialKeyedTypedThing", "label": "special"}); + let mut edited = base.clone(); + edited["label"] = json!("BASE"); + let before = json!({"inheritedVertices": [base, special.clone()]}); + let after = json!({"inheritedVertices": [edited, special]}); + let deltas = f.diff2(before.clone(), after.clone()); + let delta = only(&deltas); + assert_eq!( + delta.path, + vec![ + "inheritedVertices".to_string(), + "0".to_string(), + "label".to_string() + ], + "a polymorphic designator-keyed list is addressed by index too" + ); + + let (patched, trace) = patch(&f.load(before), &deltas, PatchOptions::default()).unwrap(); + assert!(trace.failed.is_empty(), "{:?}", trace.failed); + assert!( + patched.equals(&f.load(after), true), + "{}", + patched.to_json() + ); +} diff --git a/src/runtime/tests/equality.rs b/src/runtime/tests/equality.rs index 3418a63..6aa6d2e 100644 --- a/src/runtime/tests/equality.rs +++ b/src/runtime/tests/equality.rs @@ -66,8 +66,10 @@ objects: "#; let v1 = load_yaml_doc(doc_with_null, &sv, &container, &conv); let v2 = load_yaml_doc(doc_without_slot, &sv, &container, &conv); - let p1 = v1.navigate_path(["objects", "0"]).unwrap(); - let p2 = v2.navigate_path(["objects", "0"]).unwrap(); + // `objects` is a list of NamedThing, which declares an `id` identifier: + // it is addressed by that label, never by position. + let p1 = v1.navigate_path(["objects", "P:1"]).unwrap(); + let p2 = v2.navigate_path(["objects", "P:1"]).unwrap(); assert!( p1.equals(p2, true), "Person with null assignment should equal omission" @@ -112,8 +114,10 @@ objects: "#; let v1 = load_yaml_doc(doc_a, &sv, &container, &conv); let v2 = load_yaml_doc(doc_b, &sv, &container, &conv); - let p1 = v1.navigate_path(["objects", "0"]).unwrap(); - let p2 = v2.navigate_path(["objects", "0"]).unwrap(); + // `objects` is a list of NamedThing, which declares an `id` identifier: + // it is addressed by that label, never by position. + let p1 = v1.navigate_path(["objects", "P:1"]).unwrap(); + let p2 = v2.navigate_path(["objects", "P:1"]).unwrap(); assert!(matches!(p1, LinkMLInstance::Object { .. })); assert!(matches!(p2, LinkMLInstance::Object { .. })); assert!(!p1.equals(p2, true), "List order must affect equality"); @@ -196,9 +200,9 @@ objects: let v1 = load_yaml_doc(doc1, &sv, &container, &conv); let v2 = load_yaml_doc(doc2, &sv, &container, &conv); let v3 = load_yaml_doc(doc3, &sv, &container, &conv); - let g1 = v1.navigate_path(["objects", "0", "gender"]).unwrap(); - let g2 = v2.navigate_path(["objects", "0", "gender"]).unwrap(); - let g3 = v3.navigate_path(["objects", "0", "gender"]).unwrap(); + let g1 = v1.navigate_path(["objects", "P:1", "gender"]).unwrap(); + let g2 = v2.navigate_path(["objects", "P:2", "gender"]).unwrap(); + let g3 = v3.navigate_path(["objects", "P:3", "gender"]).unwrap(); assert!(g1.equals(g2, true)); assert!(!g1.equals(g3, true)); } diff --git a/src/runtime/tests/identity_canonicalization.rs b/src/runtime/tests/identity_canonicalization.rs new file mode 100644 index 0000000..b5addf5 --- /dev/null +++ b/src/runtime/tests/identity_canonicalization.rs @@ -0,0 +1,646 @@ +//! Spec addendum rule 2 — identity compares meaning, not spelling (D1 + D6). +//! +//! Two halves, pinned together because they only hold together: +//! +//! * designator values are canonicalised at the boxing chokepoint, so every +//! spelling of "this element is a `Circle`" becomes the one value the class +//! declares, and `to_json` emits it; +//! * identity-label components whose range descends from `uri`/`uriorcurie` +//! are IRI-expanded before comparison at *every* resolve site, so a CURIE and +//! its expansion are one identity in diff, patch, navigate and the lint. + +use linkml_runtime::{ + diff, lint_instance_identity, load_json_str, patch, Delta, DeltaOp, DiffOptions, + LinkMLInstance, PatchOptions, ValidationProblemType, ValidationResult, +}; +use linkml_schemaview::identifier::{converter_from_schema, Identifier}; +use linkml_schemaview::io::from_yaml; +use linkml_schemaview::schemaview::{ClassView, SchemaView}; +use linkml_schemaview::Converter; +use serde_json::{json, Value as JsonValue}; +use std::path::PathBuf; + +const CIRCLE_CURIE: &str = "canon:Circle"; +/// `LeafNode`'s canonical designator value: its `class_uri`, expanded (the slot +/// range is `uri`). +const LEAF_URI: &str = "https://example.org/canon/Leaf"; +/// The same class's *schema-native* URI — a different spelling of the same +/// meaning, and one the accepted-value set must recognise. +const LEAF_NATIVE_URI: &str = "https://w3id.org/linkml/examples/identity_canonical/LeafNode"; +const FANCY_CURIE: &str = "canon:FancyWidget"; +const WGS84_CURIE: &str = "ex:WGS84"; +const WGS84_URI: &str = "https://example.org/canon/WGS84"; +const ETRS89_CURIE: &str = "ex:ETRS89"; +const ETRS89_URI: &str = "https://example.org/canon/ETRS89"; + +struct Fixture { + sv: SchemaView, + conv: Converter, + container: ClassView, +} + +fn fixture() -> Fixture { + let mut p = PathBuf::from(env!("CARGO_MANIFEST_DIR")); + p.push("tests/data/identity_canonical.yaml"); + let schema = from_yaml(&p).unwrap(); + let mut sv = SchemaView::new(); + sv.add_schema(schema.clone()).unwrap(); + let conv = converter_from_schema(&schema); + let container = sv + .get_class(&Identifier::new("Container"), &conv) + .unwrap() + .expect("class not found"); + Fixture { + sv, + conv, + container, + } +} + +impl Fixture { + fn load_result(&self, v: JsonValue) -> (LinkMLInstance, Vec) { + let r = load_json_str(&v.to_string(), &self.sv, &self.container, &self.conv).unwrap(); + let issues = r.validation_issues.clone(); + (r.into_instance().expect("instance must load"), issues) + } + fn load(&self, v: JsonValue) -> LinkMLInstance { + self.load_result(v).0 + } + fn json(&self, v: JsonValue) -> JsonValue { + self.load(v).to_json() + } +} + +fn diff2(f: &Fixture, before: JsonValue, after: JsonValue) -> Vec { + diff(&f.load(before), &f.load(after), DiffOptions::new(true)) +} + +fn only(deltas: &[Delta]) -> &Delta { + assert_eq!(deltas.len(), 1, "expected exactly one delta: {deltas:#?}"); + &deltas[0] +} + +fn systems(items: Vec) -> JsonValue { + json!({ "systems": items }) +} + +/// The same list shape over `KeyedSystem`, whose identity component is a `key`. +fn keyed_systems(items: Vec) -> JsonValue { + json!({ "keyed_systems": items }) +} + +// --------------------------------------------------------------------------- +// D1 — designator values canonicalised at load +// --------------------------------------------------------------------------- + +/// CURIE, full URI and omitted-entirely are three spellings of one fact. All +/// three must box to the class's canonical designator value, and `to_json` +/// must emit that value. +#[test] +fn every_designator_spelling_boxes_to_the_one_canonical_value() { + let f = fixture(); + let out = f.json(json!({ + "shapes": [ + {"typeURI": CIRCLE_CURIE, "label": "a", "radius": 1.0}, + {"typeURI": "https://w3id.org/linkml/examples/identity_canonical/Circle", + "label": "b", "radius": 2.0}, + {"label": "c", "radius": 3.0} + ] + })); + let shapes = out.get("shapes").and_then(JsonValue::as_array).unwrap(); + let spellings: Vec<&JsonValue> = shapes.iter().map(|s| &s["typeURI"]).collect(); + assert_eq!( + spellings, + vec![ + &json!(CIRCLE_CURIE), + &json!(CIRCLE_CURIE), + &json!(CIRCLE_CURIE) + ], + "one canonical designator value, whatever the data spelled: {out:#?}" + ); +} + +/// The canonical form follows the designator slot's *range*: a `uri`-ranged +/// designator canonicalises to the expanded URI, not to a CURIE. +/// +/// `LeafNode` declares a `class_uri`, so it has two legitimate URI spellings and +/// two CURIE spellings. All four — and the omitted case — mean the same class +/// and must land on the one canonical value, silently. +#[test] +fn uri_ranged_designator_canonicalises_to_the_expanded_class_uri() { + let f = fixture(); + let doc = json!({ + "nodes": [ + {"kind": "canon:LeafNode", "name": "n1", "depth": 1}, + {"kind": LEAF_NATIVE_URI, "name": "n2", "depth": 2}, + {"kind": "ex:Leaf", "name": "n3", "depth": 3}, + {"kind": LEAF_URI, "name": "n4", "depth": 4}, + {"name": "n5", "depth": 5} + ] + }); + let (inst, issues) = f.load_result(doc); + assert!( + issues.is_empty(), + "every spelling above is an accepted designator value: {issues:#?}" + ); + let out = inst.to_json(); + let nodes = out.get("nodes").and_then(JsonValue::as_array).unwrap(); + assert_eq!(nodes.len(), 5); + for n in nodes { + assert_eq!(n["kind"], json!(LEAF_URI), "in {out:#?}"); + } +} + +/// The dict (mapping) boxing chokepoint canonicalises too — `build_mapping_ +/// entry_for_slot` builds objects the list path never sees. +#[test] +fn dict_form_designator_is_canonicalised() { + let f = fixture(); + let out = f.json(json!({ + "widgets": { + "w1": {"wid": "w1", "typeURI": + "https://w3id.org/linkml/examples/identity_canonical/FancyWidget", + "sparkle": "yes"}, + "w2": {"wid": "w2", "typeURI": FANCY_CURIE, "sparkle": "no"} + } + })); + let widgets = out.get("widgets").unwrap(); + assert_eq!(widgets["w1"]["typeURI"], json!(FANCY_CURIE), "in {out:#?}"); + assert_eq!(widgets["w2"]["typeURI"], json!(FANCY_CURIE), "in {out:#?}"); +} + +/// The *compact* dict entry — a bare scalar rather than an object — is built by +/// a second arm of `build_mapping_entry_for_slot` that hardwires the slot's +/// range class. When the scalar slot it fills is the class's designator, the +/// entry names its own class, and canonicalising against the range class would +/// rewrite `FancyWidget` to `Widget` and warn about data that was right. +#[test] +fn compact_dict_entry_naming_a_subclass_is_not_rewritten_to_the_range_class() { + let f = fixture(); + let (inst, issues) = f.load_result(json!({ + "widgets": { + "w1": "https://w3id.org/linkml/examples/identity_canonical/FancyWidget", + "w2": FANCY_CURIE + } + })); + assert!( + issues.is_empty(), + "the entry names an accepted designator value of a real subclass: {issues:#?}" + ); + let out = inst.to_json(); + let widgets = out.get("widgets").unwrap(); + assert_eq!( + widgets["w1"]["typeURI"], + json!(FANCY_CURIE), + "canonicalised, and to the *subclass* the entry named: {out:#?}" + ); + assert_eq!(widgets["w2"]["typeURI"], json!(FANCY_CURIE), "in {out:#?}"); + // `Widget.wid` is `required`, and the compact form supplies no payload for + // it at all — the dict-key injection of rule 5 is the only thing that can + // satisfy it. The `issues.is_empty()` assertion above is the guard. + assert_eq!(widgets["w1"]["wid"], json!("w1"), "in {out:#?}"); + assert_eq!(widgets["w2"]["wid"], json!("w2"), "in {out:#?}"); +} + +/// The guard for `ClassView::get_uri(native, expand)`: `BareLeaf` declares a +/// `class_uri` and no distinguishing slot, so its schema-native URI is the only +/// thing that can name it. If that spelling ever drops out of the accepted set +/// the element silently becomes a `Node` and its designator is rewritten to +/// `Node`'s value — this test is what makes that loud. +#[test] +fn a_subclass_is_selected_by_its_native_uri_alone() { + let f = fixture(); + let (inst, issues) = f.load_result(json!({ + "nodes": [{ + "kind": "https://w3id.org/linkml/examples/identity_canonical/BareLeaf", + "name": "b1" + }] + })); + assert!(issues.is_empty(), "an accepted spelling: {issues:#?}"); + let out = inst.to_json(); + assert_eq!( + out["nodes"][0]["kind"], + json!("https://example.org/canon/BareLeaf"), + "selected as BareLeaf and canonicalised to its class_uri: {out:#?}" + ); +} + +/// A designator value matching *no* accepted designator value is data the +/// loader cannot honour. Following the loader-tolerance precedent it is a +/// warning, not an error: the instance still loads, and the slot is filled +/// with the canonical value of the class that was actually selected. +#[test] +fn junk_designator_value_warns_and_is_replaced_by_the_canonical_value() { + let f = fixture(); + let (inst, issues) = f.load_result(json!({ + "shapes": [{"typeURI": "Circle", "label": "a", "radius": 1.0}] + })); + let out = inst.to_json(); + assert_eq!( + out["shapes"][0]["typeURI"], + json!(CIRCLE_CURIE), + "junk designator must be replaced by the canonical value: {out:#?}" + ); + assert!( + !issues.iter().any(|i| i.severity.is_error()), + "junk designator must not be an error: {issues:#?}" + ); + let warned: Vec<&ValidationResult> = issues + .iter() + .filter(|i| { + i.problem_type == ValidationProblemType::SlotRangeViolation + && i.subject.last().map(String::as_str) == Some("typeURI") + }) + .collect(); + assert_eq!( + warned.len(), + 1, + "exactly one designator warning expected: {issues:#?}" + ); + assert!( + warned[0].detail.contains("Circle"), + "the warning must name the rejected value: {:?}", + warned[0].detail + ); +} + +/// The canonical spelling is what makes a *matching* spelling silent: nothing +/// warns when the data already spelled an accepted value. +#[test] +fn accepted_designator_spellings_do_not_warn() { + let f = fixture(); + let (_, issues) = f.load_result(json!({ + "shapes": [ + {"typeURI": CIRCLE_CURIE, "label": "a", "radius": 1.0}, + {"typeURI": "https://w3id.org/linkml/examples/identity_canonical/Circle", + "label": "b", "radius": 2.0}, + {"label": "c", "radius": 3.0} + ] + })); + assert!(issues.is_empty(), "no diagnostics expected: {issues:#?}"); +} + +// --------------------------------------------------------------------------- +// D6 — uri/uriorcurie identity components compared as IRIs +// --------------------------------------------------------------------------- + +/// A reorder is invisible even when the list is addressed by IRI labels and the +/// document mixes spellings — the labels are stable, the order is not identity. +#[test] +fn mixed_spelling_reorder_is_a_zero_delta_diff() { + let f = fixture(); + let deltas = diff2( + &f, + systems(vec![ + json!({"systemType": WGS84_CURIE, "value": "one"}), + json!({"systemType": ETRS89_URI, "value": "two"}), + ]), + systems(vec![ + json!({"systemType": ETRS89_URI, "value": "two"}), + json!({"systemType": WGS84_CURIE, "value": "one"}), + ]), + ); + assert!(deltas.is_empty(), "reorder must be invisible: {deltas:#?}"); +} + +/// The label a keyed list is addressed by is the *expanded* IRI, even when the +/// document spells the CURIE. This is what every other site then agrees with. +#[test] +fn identity_path_segments_are_expanded_iris() { + let f = fixture(); + let deltas = diff2( + &f, + systems(vec![json!({"systemType": WGS84_CURIE, "value": "one"})]), + systems(vec![json!({"systemType": WGS84_CURIE, "value": "two"})]), + ); + let d = only(&deltas); + assert_eq!(d.op, DeltaOp::Update); + assert_eq!( + d.path, + vec![ + "systems".to_string(), + WGS84_URI.to_string(), + "value".to_string() + ], + "path segments are IRIs: {deltas:#?}" + ); +} + +/// The two sides spell one element's identity differently. Rule 2 makes them +/// one identity, so the element is *matched* and the change is described +/// field-by-field under its IRI — never a Remove + Add of the whole element. +/// +/// The re-spelling itself stays a delta: only the identity *comparison* is +/// IRI-normalised. The stored value of an ordinary `uri`-ranged slot is the +/// author's data, unlike a type designator, whose value is a function of the +/// class and is therefore rewritten at load. +#[test] +fn respelled_element_is_matched_never_replaced() { + let f = fixture(); + let deltas = diff2( + &f, + systems(vec![json!({"systemType": WGS84_CURIE, "value": "one"})]), + systems(vec![json!({"systemType": WGS84_URI, "value": "two"})]), + ); + assert!( + deltas.iter().all(|d| d.op == DeltaOp::Update), + "no element churn: {deltas:#?}" + ); + let mut paths: Vec> = deltas.iter().map(|d| d.path.clone()).collect(); + paths.sort(); + assert_eq!( + paths, + vec![ + vec![ + "systems".to_string(), + WGS84_URI.to_string(), + "systemType".to_string() + ], + vec![ + "systems".to_string(), + WGS84_URI.to_string(), + "value".to_string() + ], + ], + "one element, addressed by its IRI: {deltas:#?}" + ); +} + +/// The same re-spelling, on a class whose identity component is a `key`. +/// +/// Rule 2's last raw comparison lived here: the list matched the two elements +/// as one identity (labels are canonicalised), diff then recursed into the +/// pair, and `treat_changed_identifier_as_new_object` — which reads the +/// metamodel's key slot, not the identity label — compared `ex:WGS84` with its +/// own expansion as two strings and called the element replaced. The delta was +/// self-contradictory: a whole-element `Update` at a path that addresses the +/// element by the very identity the branch had just declared changed. +/// +/// `unique_keys`-derived identity (the test above) never reached this branch at +/// all, which is why the case survived rule 2's first pass. +#[test] +fn respelled_key_is_matched_never_replaced() { + let f = fixture(); + let deltas = diff2( + &f, + keyed_systems(vec![json!({"systemType": WGS84_CURIE, "value": "one"})]), + keyed_systems(vec![json!({"systemType": WGS84_URI, "value": "two"})]), + ); + let mut paths: Vec> = deltas.iter().map(|d| d.path.clone()).collect(); + paths.sort(); + assert_eq!( + paths, + vec![ + vec![ + "keyed_systems".to_string(), + WGS84_URI.to_string(), + "systemType".to_string() + ], + vec![ + "keyed_systems".to_string(), + WGS84_URI.to_string(), + "value".to_string() + ], + ], + "field-level deltas under one element, never a replacement: {deltas:#?}" + ); +} + +/// The complement, and the reason the branch stays: a key value that is a +/// *different* IRI is a different element, and the whole-element replacement is +/// exactly right. Only spelling was ever meant to be forgiven. +#[test] +fn a_genuinely_changed_key_is_still_a_replacement() { + let f = fixture(); + let deltas = diff2( + &f, + keyed_systems(vec![json!({"systemType": WGS84_URI, "value": "one"})]), + keyed_systems(vec![json!({"systemType": ETRS89_URI, "value": "one"})]), + ); + assert!( + deltas + .iter() + .any(|d| d.op == DeltaOp::Remove && d.path == vec!["keyed_systems", WGS84_URI]) + && deltas + .iter() + .any(|d| d.op == DeltaOp::Add && d.path == vec!["keyed_systems", ETRS89_URI]), + "two different IRIs are two different elements: {deltas:#?}" + ); +} + +/// The invariant the two halves share: whatever segment diff emits, +/// `resolve_list_segment` must find — even when the document patch is applied +/// to spells the identity the other way round. +#[test] +fn patch_locates_the_element_under_spelling_drift() { + let f = fixture(); + let deltas = diff2( + &f, + systems(vec![json!({"systemType": WGS84_URI, "value": "one"})]), + systems(vec![json!({"systemType": WGS84_URI, "value": "two"})]), + ); + // The document being patched spells the same system as a CURIE. + let drifted = f.load(systems(vec![ + json!({"systemType": WGS84_CURIE, "value": "one"}), + ])); + let (patched, trace) = patch(&drifted, &deltas, PatchOptions::default()).unwrap(); + assert!(trace.failed.is_empty(), "delta must locate: {trace:#?}"); + assert_eq!(patched.to_json()["systems"][0]["value"], json!("two")); +} + +/// The symmetric half: a segment that arrives already spelled as a CURIE — a +/// stored delta, a hand-written patch — must address the element whose label +/// expanded to the IRI. Labels are normalised on the way out; the incoming +/// segment is normalised through the same slot on the way in. +#[test] +fn a_curie_spelled_segment_applies_against_expanded_labels() { + let f = fixture(); + let doc = systems(vec![ + json!({"systemType": WGS84_URI, "value": "one"}), + json!({"systemType": ETRS89_URI, "value": "two"}), + ]); + let hand_written = vec![Delta { + path: vec![ + "systems".to_string(), + WGS84_CURIE.to_string(), + "value".to_string(), + ], + op: DeltaOp::Update, + old: Some(json!("one")), + new: Some(json!("two")), + }]; + let src = f.load(doc); + let (patched, trace) = patch(&src, &hand_written, PatchOptions::default()).unwrap(); + assert!( + trace.failed.is_empty(), + "a curie segment and an expanded label are one identity: {trace:#?}" + ); + assert_eq!(patched.to_json()["systems"][0]["value"], json!("two")); + // …and the same segment navigates. + assert_eq!( + src.navigate_path(["systems", WGS84_CURIE, "value"]) + .map(LinkMLInstance::to_json), + Some(json!("one")) + ); +} + +/// `navigate_path` shares the resolver, so either spelling addresses the +/// element regardless of how the document spells it. +#[test] +fn navigate_finds_the_element_by_the_expanded_iri() { + let f = fixture(); + let inst = f.load(systems(vec![ + json!({"systemType": WGS84_CURIE, "value": "one"}), + json!({"systemType": ETRS89_CURIE, "value": "two"}), + ])); + let hit = inst + .navigate_path(["systems", WGS84_URI, "value"]) + .expect("expanded IRI must address the CURIE-spelled element"); + assert_eq!(hit.to_json(), json!("one")); +} + +/// Two spellings that expand to the same IRI are a duplicate identity, and the +/// instance lint is the voice that says so. +#[test] +fn instance_lint_sees_a_duplicate_across_spellings() { + let f = fixture(); + let inst = f.load(systems(vec![ + json!({"systemType": WGS84_CURIE, "value": "one"}), + json!({"systemType": WGS84_URI, "value": "two"}), + ])); + let warnings = lint_instance_identity(&inst); + let dups: Vec<&ValidationResult> = warnings + .iter() + .filter(|w| w.problem_type == ValidationProblemType::DuplicateElementIdentity) + .collect(); + assert_eq!( + dups.len(), + 1, + "a curie and its expansion collide: {warnings:#?}" + ); + assert!( + dups[0].detail.contains(WGS84_URI), + "the duplicate is reported by its expanded IRI: {:?}", + dups[0].detail + ); +} + +/// Explicit round-trip pin for the shared-rule invariant: the segments diff +/// emits over a mixed-spelling pair are exactly what the resolver computes, so +/// every one of them addresses a real node and re-applying the diff reproduces +/// the target (list order is not identity, so the comparison is order-free). +#[test] +fn mixed_spelling_diff_segments_round_trip_through_the_resolver() { + let f = fixture(); + let before = systems(vec![ + json!({"systemType": WGS84_CURIE, "value": "one"}), + json!({"systemType": ETRS89_URI, "value": "two"}), + ]); + let after = systems(vec![ + json!({"systemType": ETRS89_CURIE, "value": "TWO"}), + json!({"systemType": WGS84_URI, "value": "ONE"}), + json!({"systemType": "ex:LAMBERT", "value": "three"}), + ]); + let deltas = diff2(&f, before.clone(), after.clone()); + assert!(!deltas.is_empty(), "there is real change here"); + let src = f.load(before); + for d in &deltas { + // Every emitted path but that of an Add must address an existing node. + if d.op != DeltaOp::Add { + assert!( + src.navigate_path(d.path.iter()).is_some(), + "diff emitted a segment its own resolver cannot follow: {:?}", + d.path + ); + } + } + let (patched, trace) = patch(&src, &deltas, PatchOptions::default()).unwrap(); + assert!(trace.failed.is_empty(), "{trace:#?}"); + assert_eq!( + sorted_systems(&patched.to_json()), + sorted_systems(&f.json(after)) + ); +} + +/// `systems` as a set: keyed patching edits in place, so the surviving element +/// order is the source's, and order is not what a keyed list is compared by. +fn sorted_systems(v: &JsonValue) -> Vec { + let mut out: Vec = v["systems"] + .as_array() + .unwrap() + .iter() + .map(JsonValue::to_string) + .collect(); + out.sort(); + out +} + +// --------------------------------------------------------------------------- +// JSON / RDF agreement +// --------------------------------------------------------------------------- + +/// The RDF loader has always written the canonical designator (it never +/// harvests the designator predicate, it fills it from the class). Once JSON +/// canonicalises too, the two loaders agree on a document that spelled the +/// designator non-canonically. +#[cfg(feature = "ttl")] +#[test] +fn json_and_rdf_loaders_agree_on_the_designator_spelling() { + use linkml_runtime::rdf_import::{import_turtle, ImportOptions}; + use linkml_runtime::turtle::{turtle_to_string, TurtleOptions}; + + let mut p = PathBuf::from(env!("CARGO_MANIFEST_DIR")); + p.push("tests/data/identity_canonical.yaml"); + let schema = from_yaml(&p).unwrap(); + let mut sv = SchemaView::new(); + sv.add_schema(schema.clone()).unwrap(); + let conv = converter_from_schema(&schema); + let container = sv + .get_class(&Identifier::new("Container"), &conv) + .unwrap() + .expect("class not found"); + + let doc = json!({ + "shapes": [{ + "typeURI": "https://w3id.org/linkml/examples/identity_canonical/Circle", + "label": "a", + "radius": 1.0 + }] + }); + let from_json = load_json_str(&doc.to_string(), &sv, &container, &conv) + .unwrap() + .into_instance() + .unwrap(); + + let ttl = turtle_to_string( + &from_json, + &sv, + &schema, + &conv, + TurtleOptions { skolem: false }, + ) + .unwrap(); + let stream = import_turtle( + std::io::Cursor::new(ttl.as_bytes()), + sv.clone(), + conv.clone(), + &["Container"], + ImportOptions::default(), + ) + .unwrap(); + let from_rdf = stream + .filter_map(|r| r.ok()) + .find_map(|(c, i)| if c == "Container" { Some(i) } else { None }) + .expect("one Container"); + + assert_eq!( + from_rdf.to_json()["shapes"][0]["typeURI"], + json!(CIRCLE_CURIE), + "the RDF loader's canonical designator" + ); + assert_eq!( + from_json.to_json()["shapes"][0]["typeURI"], + from_rdf.to_json()["shapes"][0]["typeURI"], + "JSON and RDF loaders must agree" + ); +} diff --git a/src/runtime/tests/identity_lint.rs b/src/runtime/tests/identity_lint.rs new file mode 100644 index 0000000..7d11ffa --- /dev/null +++ b/src/runtime/tests/identity_lint.rs @@ -0,0 +1,839 @@ +use linkml_runtime::{ + lint_element_identity, lint_instance_identity, load_json_str, LinkMLInstance, + ValidationProblemType, +}; +use linkml_schemaview::identifier::{converter_from_schema, Identifier}; +use linkml_schemaview::io::from_yaml; +use linkml_schemaview::schemaview::{ClassView, SchemaView}; +use linkml_schemaview::Converter; +use serde_json::{json, Value as JsonValue}; +use std::path::PathBuf; + +struct Fixture { + sv: SchemaView, + conv: Converter, + service: ClassView, +} + +/// Loads a schema from `tests/data` into its own `SchemaView`. +fn schema_view(file: &str) -> SchemaView { + let mut p = PathBuf::from(env!("CARGO_MANIFEST_DIR")); + p.push("tests/data"); + p.push(file); + let schema = from_yaml(&p).unwrap(); + let mut sv = SchemaView::new(); + sv.add_schema(schema).unwrap(); + sv +} + +/// Loads `data` as an instance of `class` against a schema from `tests/data`. +/// +/// The shared [`Fixture`] is pinned to `identity.yaml`'s `Service`; the rules +/// below each get their own schema, so they need the same service generically. +fn load_into(file: &str, class: &str, data: JsonValue) -> LinkMLInstance { + let mut p = PathBuf::from(env!("CARGO_MANIFEST_DIR")); + p.push("tests/data"); + p.push(file); + let schema = from_yaml(&p).unwrap(); + let mut sv = SchemaView::new(); + sv.add_schema(schema.clone()).unwrap(); + let conv = converter_from_schema(&schema); + let cv = sv + .get_class(&Identifier::new(class), &conv) + .unwrap() + .expect("class not found"); + load_json_str(&data.to_string(), &sv, &cv, &conv) + .unwrap() + .into_instance() + .unwrap() +} + +fn fixture() -> Fixture { + let mut p = PathBuf::from(env!("CARGO_MANIFEST_DIR")); + p.push("tests/data/identity.yaml"); + let schema = from_yaml(&p).unwrap(); + let mut sv = SchemaView::new(); + sv.add_schema(schema.clone()).unwrap(); + let conv = converter_from_schema(&schema); + let service = sv + .get_class(&Identifier::new("Service"), &conv) + .unwrap() + .expect("class not found"); + Fixture { sv, conv, service } +} + +impl Fixture { + fn load(&self, v: JsonValue) -> LinkMLInstance { + load_json_str(&v.to_string(), &self.sv, &self.service, &self.conv) + .unwrap() + .into_instance() + .unwrap() + } +} + +fn e() -> JsonValue { + json!({"phoneNumber": "09/241.25.00", "hasNumberFunction": "Emergency_Number"}) +} +fn n() -> JsonValue { + json!({"phoneNumber": "09/241.25.03", "hasNumberFunction": "Non_Urgent_Communication"}) +} +fn phones(items: Vec) -> JsonValue { + json!({"name": "svc", "hasPhoneNumber": items}) +} + +#[test] +fn schema_lint_flags_exactly_the_undeclared_positional_slots() { + let f = fixture(); + let warnings = lint_element_identity(&f.sv); + let mut flagged: Vec<(String, String)> = warnings + .iter() + .map(|w| (w.subject[0].clone(), w.subject[1].clone())) + .collect(); + flagged.sort(); + assert_eq!( + flagged, + vec![ + ("Service".to_string(), "plainPhoneNumber".to_string()), + ("Service".to_string(), "tags".to_string()), + ], + "everything else declares its identity source: {warnings:#?}" + ); + for w in &warnings { + assert_eq!( + w.problem_type, + ValidationProblemType::AmbiguousElementIdentity + ); + assert!(!w.severity.is_error(), "the linter warns, never errors"); + assert!( + w.detail.contains("unique_keys") && w.detail.contains("diff.linkml.io/opaque"), + "the warning must name the author's options: {}", + w.detail + ); + } + + let detail = |slot: &str| { + warnings + .iter() + .find(|w| w.subject[1] == slot) + .map(|w| w.detail.clone()) + .unwrap_or_default() + }; + assert!( + detail("plainPhoneNumber").contains("element class 'PlainPhoneNumber'"), + "an object range must be told which class to declare the identity on: {}", + detail("plainPhoneNumber") + ); + assert!( + !detail("tags").contains("element class"), + "a scalar range has no element class to declare keys on: {}", + detail("tags") + ); +} + +#[test] +fn schema_lint_visits_a_class_with_an_explicit_class_uri_exactly_once() { + // `Service` declares `class_uri: identity:ServiceEndpoint`, so schemaview + // indexes it under both that URI and its default one and `get_class_ids()` + // yields it twice. Walking those ids naively reports every one of the + // class's slots twice. + let f = fixture(); + let warnings = lint_element_identity(&f.sv); + let mut subjects: Vec> = warnings.iter().map(|w| w.subject.clone()).collect(); + subjects.sort(); + let mut unique = subjects.clone(); + unique.dedup(); + assert_eq!( + subjects, unique, + "each slot must be reported once, not once per class URI: {warnings:#?}" + ); +} + +#[test] +fn schema_lint_reports_both_classes_that_share_one_class_uri() { + // LinkML lets distinct classes declare the same `class_uri` (meta.yaml's + // `Anything` and extensions.yaml's `AnyValue` both use `linkml:Any`). + // Deduping visits by class_uri makes the second class look already-seen and + // silently drops every one of its warnings — a false negative, which is + // worse in a lint than the duplicate reporting it was meant to fix. + // + // The fixture pulls in both directions at once: `SharedUriA`/`SharedUriB` + // share one `class_uri` and must BOTH be reported, while `TwoUris` is + // indexed under two URIs of its own and must be reported only once. + let sv = schema_view("identity_shared_class_uri.yaml"); + let subjects: Vec> = lint_element_identity(&sv) + .iter() + .map(|w| w.subject.clone()) + .collect(); + assert_eq!( + subjects, + vec![ + vec!["SharedUriA".to_string(), "itemsA".to_string()], + vec!["SharedUriB".to_string(), "itemsB".to_string()], + vec!["TwoUris".to_string(), "itemsC".to_string()], + ], + "classes sharing a class_uri must each be linted, and a class indexed \ + under two URIs must be linted once" + ); +} + +#[test] +fn schema_lint_reports_an_inherited_slot_only_where_it_is_introduced() { + // A flagged slot is inherited by every descendant, so reporting it per + // class buries the one place the author can fix it. Only the topmost + // flagged declarer is reported — but a subclass whose `slot_usage` changes + // the identity answer is judged on its own merits, in both directions. + let sv = schema_view("identity_inheritance.yaml"); + let subjects: Vec> = lint_element_identity(&sv) + .iter() + .map(|w| w.subject.clone()) + .collect(); + assert_eq!( + subjects, + vec![ + // introduces `outline`; Middle, Leaf and Broken inherit it unchanged + vec!["Base".to_string(), "outline".to_string()], + // widens an inherited clean slot into an ambiguous one: introduced here + vec!["Broken".to_string(), "keyed".to_string()], + // mixin-provided, reported both on the mixin and on its user + vec!["HasTagsMixin".to_string(), "tags".to_string()], + vec!["Tagged".to_string(), "tags".to_string()], + ], + "an inherited flagged slot must be reported once, at its introducing class" + ); + // Spelled out, because these are the cases a naive fix gets wrong: + for silent in ["Middle", "Leaf", "Fixed"] { + assert!( + !subjects.iter().any(|s| s[0] == silent), + "{silent} must stay silent, got {subjects:?}" + ); + } +} + +#[test] +fn schema_lint_warning_order_is_deterministic() { + // A class's slots come from `ClassView::slots()`, which is HashMap-backed, + // so the per-class warning order is process-dependent. Each fresh + // `SchemaView` gets its own hash seed, so an unsorted walk eventually + // disagrees with itself. + let expected = vec![ + vec!["Service".to_string(), "plainPhoneNumber".to_string()], + vec!["Service".to_string(), "tags".to_string()], + ]; + for _ in 0..20 { + let warnings = lint_element_identity(&fixture().sv); + let subjects: Vec> = warnings.iter().map(|w| w.subject.clone()).collect(); + assert_eq!(subjects, expected, "{warnings:#?}"); + } +} + +#[test] +fn schema_lint_flags_a_list_whose_identity_is_the_type_designator() { + // A key that is also the type designator is constant across a homogeneous + // list — every vertex of a ring carries the same value — and one value per + // subtype across a polymorphic one. Neither is element identity, so the + // engine looks past such a key entirely (spec addendum rule 1) and the list + // falls back to the class's unique_keys, or to positional addressing. This + // rule is the author-facing voice for exactly that shape, and the only one: + // it is asked first and the other unique_keys rules never see the slot. + let sv = schema_view("identity_type_designator_key.yaml"); + let warnings = lint_element_identity(&sv); + let subjects: Vec> = warnings.iter().map(|w| w.subject.clone()).collect(); + assert_eq!( + subjects, + vec![ + // the wild shape: the designator is declared on a base class and + // the subclass promotes it to the key with `slot_usage`, so the + // rule sees it only through `SlotView::definition()`'s chain merge + vec!["Ring".to_string(), "inheritedVertices".to_string()], + // `Marker` declares two `unique_keys` the designator key used to + // shadow and no longer does, so the several-entries rule matches it + // too; the designator rule is asked first and is its sole voice, so + // `markers` is flagged once, by this rule + vec!["Ring".to_string(), "markers".to_string()], + vec!["Ring".to_string(), "vertices".to_string()], + ], + "only the designator-keyed list slots are flagged, once each, at their \ + introducing class: {warnings:#?}" + ); + let w = warnings + .iter() + .find(|w| w.subject[1] == "vertices") + .expect("the ring slot must be flagged"); + assert_eq!( + w.problem_type, + ValidationProblemType::AmbiguousElementIdentity + ); + assert!(!w.severity.is_error(), "the linter warns, never errors"); + assert!( + w.detail.contains("typeURI") && w.detail.contains("Coordinate"), + "the warning must name the designator slot and its class: {}", + w.detail + ); + assert!( + w.detail.contains("designates_type"), + "the warning must say why the key is not discriminating: {}", + w.detail + ); + assert!( + w.detail.contains("positional"), + "the warning must say what the diff engine actually does: {}", + w.detail + ); + let inherited = warnings + .iter() + .find(|w| w.subject[1] == "inheritedVertices") + .expect("a designator promoted to key by slot_usage must be flagged too"); + assert!( + inherited.detail.contains("'KeyedTypedThing.typeURI'"), + "the warning must name the subclass that declares the key, and the \ + inherited slot it declares it on: {}", + inherited.detail + ); +} + +#[test] +fn schema_lint_fires_once_per_designator_keyed_slot_and_names_the_designator() { + // The engine no longer accepts a designator as element identity (spec + // addendum rule 1), so a designator-keyed class declaring no `unique_keys` + // is *also* the shape the identity-less rule looks for, and one whose real + // `unique_keys` were shadowed is the shape the several-unique_keys rule + // looks for. The designator rule is the sharper diagnosis of both and + // stays their only voice: exactly one warning per slot, naming the + // designator that cannot discriminate. + let sv = schema_view("identity_type_designator_key.yaml"); + let warnings = lint_element_identity(&sv); + for (slot, why) in [ + ("vertices", "a designator-keyed class with no unique_keys"), + ( + "markers", + "a designator-keyed class whose unique_keys it shadowed", + ), + ] { + let fired: Vec<_> = warnings + .iter() + .filter(|w| w.subject == vec!["Ring".to_string(), slot.to_string()]) + .collect(); + assert_eq!( + fired.len(), + 1, + "{why} must produce exactly one warning for '{slot}': {warnings:#?}" + ); + assert!( + fired[0].detail.contains("designates_type") && fired[0].detail.contains("typeURI"), + "the designator rule must be the voice for '{slot}': {}", + fired[0].detail + ); + } +} + +#[test] +fn schema_lint_leaves_designator_dicts_and_ordinary_keys_alone() { + // Guard rails for the designator rule: the dict form keyed by the designator + // means at-most-one-element-per-subtype and is legitimate; an ordinary key + // discriminates per element; `opaque` / `ignore` answer the question already; + // and an unchanged inherited slot belongs to its introducing class. + let sv = schema_view("identity_type_designator_key.yaml"); + let warnings = lint_element_identity(&sv); + let flagged: Vec = warnings.iter().map(|w| w.subject[1].clone()).collect(); + for silent in ["byType", "points", "archivedVertices", "draftVertices"] { + assert!( + !flagged.contains(&silent.to_string()), + "{silent} must stay silent, got {warnings:#?}" + ); + } + assert!( + !warnings.iter().any(|w| w.subject[0] == "SubRing"), + "an unchanged inherited slot is reported at Ring only: {warnings:#?}" + ); +} + +#[test] +fn data_lint_flags_duplicate_declared_identities() { + let f = fixture(); + let dup = json!({"phoneNumber": "09/000.00.00", "hasNumberFunction": "Emergency_Number"}); + let inst = f.load(phones(vec![e(), dup])); + let warnings = lint_instance_identity(&inst); + assert_eq!(warnings.len(), 1, "{warnings:#?}"); + assert_eq!( + warnings[0].problem_type, + ValidationProblemType::DuplicateElementIdentity + ); + assert_eq!(warnings[0].subject, vec!["hasPhoneNumber".to_string()]); + assert!(!warnings[0].severity.is_error()); +} + +#[test] +fn data_lint_warning_order_is_deterministic_across_sibling_containers() { + let f = fixture(); + // Three sibling containers, each with a duplicated declared identity. The + // instance's slots live in a HashMap, so the warnings must be emitted in a + // stable (name-sorted) order rather than in process-dependent hash order. + let c = json!({"kind": "Emergency", "phone": "02/111.11.11"}); + let dup = json!({"phoneNumber": "09/000.00.00", "hasNumberFunction": "Emergency_Number"}); + let data = json!({ + "name": "svc", + "archivedContacts": [c.clone(), c.clone()], + "contacts": [c.clone(), c], + "hasPhoneNumber": [e(), dup], + }); + let expected = vec![ + vec!["archivedContacts".to_string()], + vec!["contacts".to_string()], + vec!["hasPhoneNumber".to_string()], + ]; + // Repeated on freshly built instances: each `HashMap` gets its own hash + // seed, so an unordered walk would eventually disagree with itself. + for _ in 0..10 { + let warnings = lint_instance_identity(&f.load(data.clone())); + let subjects: Vec> = warnings.iter().map(|w| w.subject.clone()).collect(); + assert_eq!(subjects, expected, "{warnings:#?}"); + } +} + +#[test] +fn data_lint_is_silent_on_clean_and_undeclared_data() { + let f = fixture(); + // unique phone functions, repeated scalar tags, repeated identity-less vertices + let inst = f.load(json!({ + "name": "svc", + "hasPhoneNumber": [e(), n()], + "tags": ["a", "a"], + "outline": [{"x": 1.0, "y": 2.0}, {"x": 1.0, "y": 2.0}] + })); + let warnings = lint_instance_identity(&inst); + assert!(warnings.is_empty(), "{warnings:#?}"); +} + +#[test] +fn data_lint_does_not_let_opaque_suppress_a_schema_constraint() { + let f = fixture(); + // archivedContacts is opaque, but Contact declares unique_keys: duplicates + // still violate the class's claim. diff vocabulary never silences schema truth. + let c = json!({"kind": "Emergency", "phone": "02/111.11.11"}); + let inst = f.load(json!({"name": "svc", "archivedContacts": [c.clone(), c]})); + let warnings = lint_instance_identity(&inst); + assert_eq!(warnings.len(), 1, "{warnings:#?}"); + assert_eq!(warnings[0].subject, vec!["archivedContacts".to_string()]); +} + +#[test] +fn schema_lint_names_the_load_bearing_unique_key_when_a_class_declares_several() { + // Declaration order is not preserved by the metamodel, so element identity + // is derived from the name-sorted first `unique_keys` entry. A class with + // two entries therefore has a silent, alphabetically-decided identity: + // adding an earlier-sorting entry re-addresses every delta path for every + // slot ranged on it, with nothing to notice it by. + let sv = schema_view("identity_multiple_unique_keys.yaml"); + let warnings = lint_element_identity(&sv); + let subjects: Vec> = warnings.iter().map(|w| w.subject.clone()).collect(); + assert_eq!( + subjects, + vec![vec!["Catalog".to_string(), "badges".to_string()]], + "only the ambiguous slot is flagged, once, at its introducing class: \ + {warnings:#?}" + ); + let w = &warnings[0]; + assert_eq!( + w.problem_type, + ValidationProblemType::AmbiguousElementIdentity + ); + assert!(!w.severity.is_error(), "the linter warns, never errors"); + assert!( + w.detail.contains("'by_code'") && w.detail.contains("'zz_by_label'"), + "the warning must name every candidate: {}", + w.detail + ); + assert!( + w.detail.contains("Badge"), + "the warning must name the element class the entries live on: {}", + w.detail + ); + let winner = w.detail.find("'by_code'").unwrap_or(usize::MAX); + let other = w.detail.find("'zz_by_label'").unwrap_or(0); + assert!( + winner < other, + "the load-bearing entry must be named first, and identified as such: {}", + w.detail + ); +} + +#[test] +fn schema_lint_leaves_single_entry_and_keyed_classes_alone() { + // Guard rails for the ambiguity rule: one entry is unambiguous, a key slot + // outranks `unique_keys` so the entries are not load-bearing at all, and + // `opaque` / `ignore` mean there are no per-element paths to re-address. + let sv = schema_view("identity_multiple_unique_keys.yaml"); + let flagged: Vec = lint_element_identity(&sv) + .iter() + .map(|w| w.subject[1].clone()) + .collect(); + for silent in ["tickets", "seats", "archivedBadges", "draftBadges"] { + assert!( + !flagged.contains(&silent.to_string()), + "{silent} must stay silent, got {flagged:?}" + ); + } +} + +#[test] +fn data_lint_flags_a_list_addressed_positionally_despite_a_declared_identity() { + // The element class declares an identity, but the data leaves the slot it + // names empty: the element yields no label, the list stops being + // keyed-shaped, and every delta addressing it goes back to being positional + // — silently, because nothing here is invalid. A `key` that is not + // `required` may legitimately be absent, so only the data can show it. + let inst = load_into( + "identity_missing_labels.yaml", + "Registry", + json!({ + "name": "reg", + // all elements unlabelled: the absent-optional-key case + "entries": [{"note": "a"}, {"note": "b"}], + // half-labelled: one element carries the unique_keys slot, one does not + "records": [{"serial": "S1"}, {"note": "b"}], + // D9: the designator override left the key with nothing to fill it + "unfillable": [{"note": "a"}, {"note": "b"}], + }), + ); + let warnings = lint_instance_identity(&inst); + let subjects: Vec> = warnings.iter().map(|w| w.subject.clone()).collect(); + assert_eq!( + subjects, + vec![ + vec!["entries".to_string()], + vec!["records".to_string()], + vec!["unfillable".to_string()], + ], + "one warning per container, at the container's path: {warnings:#?}" + ); + for w in &warnings { + assert_eq!( + w.problem_type, + ValidationProblemType::AmbiguousElementIdentity + ); + assert!(!w.severity.is_error(), "the linter warns, never errors"); + assert!( + w.detail.contains("positional"), + "the warning must say what the engine falls back to: {}", + w.detail + ); + } + let detail = |slot: &str| { + warnings + .iter() + .find(|w| w.subject[0] == slot) + .map(|w| w.detail.clone()) + .unwrap_or_default() + }; + assert!( + detail("entries").contains("2 of 2") && detail("entries").contains("'OptKey'"), + "the warning must count the unlabelled elements and name the class \ + whose identity is unfilled: {}", + detail("entries") + ); + assert!( + detail("records").contains("1 of 2"), + "a half-labelled list is the same defect, and its count says so: {}", + detail("records") + ); +} + +#[test] +fn data_lint_positional_despite_identity_only_speaks_for_inlined_addressable_lists() { + // Guard rails, each for a different reason. + // + // A class that declares no identity is *meant* to be positional: the schema + // lint already speaks for it, and repeating that once per container in the + // data would drown the rule that matters. Scalars have no element class at + // all. `opaque` and `ignore` answer the addressing question themselves, so + // "this list is addressed positionally" is not true of them — the duplicate + // rule ignores both annotations because a repeated identity contradicts the + // class's own constraint, which is a different claim. + // + // The reference list is the one that matters in practice: its elements are + // identifier strings, which can never carry an inlined element's identity + // label, so without this guard the rule fires on every reference list ranged + // on a keyed class. asset360's `NetElement.ports` is that shape, and it is + // the ONLY list in the downstream corpus the unguarded rule fired on. + let inst = load_into( + "identity_missing_labels.yaml", + "Registry", + json!({ + "name": "reg", + "vertices": [{"x": 1.0, "y": 2.0}, {"x": 1.0, "y": 2.0}], + "tags": ["a", "b"], + "archivedEntries": [{"note": "a"}], + "draftEntries": [{"note": "a"}], + "references": ["urn:a", "urn:b"], + }), + ); + let warnings = lint_instance_identity(&inst); + assert!(warnings.is_empty(), "{warnings:#?}"); +} + +#[test] +fn data_lint_positional_despite_identity_is_silent_on_fully_labelled_lists() { + // The rule must not fire on the lists it was written to leave alone: every + // element labelled is exactly the keyed shape. + let inst = load_into( + "identity_missing_labels.yaml", + "Registry", + json!({ + "name": "reg", + "entries": [{"code": "a"}, {"code": "b"}], + "records": [{"serial": "S1"}, {"serial": "S2"}], + // an empty list has no unlabelled element to complain about + "unfillable": [], + }), + ); + let warnings = lint_instance_identity(&inst); + assert!(warnings.is_empty(), "{warnings:#?}"); +} + +#[test] +fn schema_lint_counts_unique_keys_entries_across_the_range_class_descendants() { + // A list ranged on a class holds elements of every class descending from it, + // and each element is labelled by its OWN merged `unique_keys`. Inspecting + // only the range class misses a descendant's entry entirely: `Box` alone + // declares one entry and looks unambiguous, while `BigBox` adds a second + // that any element of the list may be labelled by. + let sv = schema_view("identity_descendant_unique_keys.yaml"); + let warnings = lint_element_identity(&sv); + let boxes: Vec<_> = warnings + .iter() + .filter(|w| w.subject == vec!["Depot".to_string(), "boxes".to_string()]) + .collect(); + assert_eq!( + boxes.len(), + 1, + "a later-sorting descendant entry widens the candidate set without \ + splitting the label space: one warning: {warnings:#?}" + ); + assert!( + boxes[0].detail.contains("'by_id'") && boxes[0].detail.contains("'zz_by_volume'"), + "the warning must name every candidate, wherever it is declared: {}", + boxes[0].detail + ); + // guard rail: a descendant declaring nothing of its own leaves the range + // class as unambiguous as it was + let flagged: Vec = warnings.iter().map(|w| w.subject[1].clone()).collect(); + assert!( + !flagged.contains(&"crates".to_string()), + "crates must stay silent, got {warnings:#?}" + ); + // `StampedPlate`'s key outranks its own `unique_keys`, so its entries never + // widen the candidate set: this rule has nothing to say about `keyed`. (The + // divergence rule does — the key is its own label space.) + let keyed_candidates: Vec<_> = warnings + .iter() + .filter(|w| w.subject[1] == "keyed") + .filter(|w| w.detail.contains("candidate entries")) + .collect(); + assert!( + keyed_candidates.is_empty(), + "a key-labelled descendant does not widen the candidate set: \ + {keyed_candidates:#?}" + ); +} + +#[test] +fn schema_lint_splits_a_label_space_between_a_key_and_a_unique_keys_entry() { + // A descendant that declares a `key` does not widen the candidate set — its + // own `unique_keys` stop being load-bearing — but it very much occupies its + // own label space: `Plate` elements are labelled by `plate_identity` and + // `StampedPlate` elements by `stampId`, in one list. A path written against + // one cannot address an element of the other, and a `stampId` colliding + // with a `plate_identity` value breaks neither class's constraint. Counting + // a key-labelled class as its own group is what sees that. + let sv = schema_view("identity_descendant_unique_keys.yaml"); + let warnings = lint_element_identity(&sv); + let keyed: Vec<_> = warnings + .iter() + .filter(|w| w.subject == vec!["Depot".to_string(), "keyed".to_string()]) + .collect(); + assert_eq!( + keyed.len(), + 1, + "the divergence warning, and only it, speaks for `keyed`: {warnings:#?}" + ); + for needle in ["'Plate'", "'plate_identity'", "'StampedPlate'", "'stampId'"] { + assert!( + keyed[0].detail.contains(needle), + "the warning must name both labellings and the class each belongs \ + to; missing {needle}: {}", + keyed[0].detail + ); + } + assert!( + keyed[0].detail.contains("key 'stampId'"), + "the warning must say that one of the two labellings is a key, not a \ + unique_keys entry: {}", + keyed[0].detail + ); +} + +#[test] +fn schema_lint_reports_a_retargeted_slot_the_parent_never_warned_about() { + // The introduces-gate asks "is the parent's slot flagged for the same + // reason?". For the two unique_keys rules that question has to subtract the + // designator case, exactly as the identity-less rule's gate does: a + // designator-keyed range class satisfies both raw shapes (the key no longer + // shadows the entries), but the designator rule is asked first and is the + // slot's ONLY voice, so the parent never emitted either warning. Without + // the subtraction a subclass that `slot_usage`-retargets the slot onto a + // class those rules really do speak for is suppressed by a parent warning + // that does not exist, and the ambiguity is reported nowhere. + let sv = schema_view("identity_descendant_unique_keys.yaml"); + let warnings = lint_element_identity(&sv); + let subject = |c: &str| vec![c.to_string(), "stamps".to_string()]; + + let base: Vec<_> = warnings + .iter() + .filter(|w| w.subject == subject("Base")) + .collect(); + assert_eq!( + base.len(), + 1, + "the designator rule speaks alone: {warnings:#?}" + ); + assert!( + base[0].detail.contains("designates_type"), + "and it is the designator rule: {}", + base[0].detail + ); + + let retargeted: Vec<_> = warnings + .iter() + .filter(|w| w.subject == subject("Retargeted")) + .collect(); + assert_eq!( + retargeted.len(), + 1, + "a retarget onto a keyless two-entry class must be reported here: \ + {warnings:#?}" + ); + assert!( + retargeted[0].detail.contains("'by_alpha'") + && retargeted[0].detail.contains("'zz_by_beta'"), + "and by the several-entries rule: {}", + retargeted[0].detail + ); + + let split: Vec<_> = warnings + .iter() + .filter(|w| w.subject == subject("SplitRetargeted")) + .collect(); + assert_eq!( + split.len(), + 2, + "a retarget onto a split family must get both warnings here: \ + {warnings:#?}" + ); + assert!( + split.iter().any(|w| w.detail.contains("label space")), + "one of them being the divergence rule: {split:#?}" + ); +} + +#[test] +fn schema_lint_flags_descendants_that_resolve_different_load_bearing_entries() { + // The split label space: `Gadget` elements are labelled by `gadget_identity` + // and `Widget` elements by the earlier-sorting `aaa_widget_identity` it + // adds. One list, two label spaces — navigating by a base label misses a + // Widget, and a Widget serial colliding with a Gadget code violates neither + // class's constraint. This is an ADDITIONAL warning: the several-entries + // rule still fires, because the candidate set is ambiguous too. + let sv = schema_view("identity_descendant_unique_keys.yaml"); + let warnings = lint_element_identity(&sv); + let gadgets: Vec<_> = warnings + .iter() + .filter(|w| w.subject == vec!["Depot".to_string(), "gadgets".to_string()]) + .collect(); + assert_eq!( + gadgets.len(), + 2, + "the split gets its own warning on top of the several-entries one: \ + {warnings:#?}" + ); + let split = gadgets + .iter() + .find(|w| w.detail.contains("Widget")) + .expect("one of the two must name the diverging descendant"); + assert_eq!( + split.problem_type, + ValidationProblemType::AmbiguousElementIdentity + ); + assert!(!split.severity.is_error(), "the linter warns, never errors"); + for needle in [ + "'Gadget'", + "'gadget_identity'", + "'Widget'", + "'aaa_widget_identity'", + ] { + assert!( + split.detail.contains(needle), + "the warning must name each diverging class and the entry it \ + resolves to; missing {needle}: {}", + split.detail + ); + } + // `boxes` agrees on `by_id` throughout, so it must NOT get this warning + assert_eq!( + warnings + .iter() + .filter(|w| w.subject[1] == "boxes") + .filter(|w| w.detail.contains("zz_by_volume") && w.detail.contains("'BigBox'")) + .count(), + 0, + "a widened candidate set is not a split label space: {warnings:#?}" + ); +} + +#[test] +fn schema_lint_flags_a_shared_class_uri_within_a_designator_hierarchy() { + // Two classes of one hierarchy answering to one `class_uri`, in a hierarchy + // whose designator is dispatched by exactly that URI: the loader picks one + // of them, stably but by an ordering the schema never states. + let sv = schema_view("identity_shared_uri_designator.yaml"); + let warnings = lint_element_identity(&sv); + let subjects: Vec> = warnings.iter().map(|w| w.subject.clone()).collect(); + // The classes that share the URI, behind a discriminator. Rules 1-4 are + // per-slot and subject `[class, slot]`; both CLIs render a subject by + // joining it with `.`, so a bare `[Alpha, Beta]` printed as `Alpha.Beta` — + // indistinguishable from a slot warning about `Beta` on class `Alpha`, and + // just as indistinguishable to anything grouping findings by subject. The + // leading segment says which rule spoke, and no `[class, slot]` subject can + // collide with it: it is not a class name in any schema being linted, since + // it is not a class name at all. + assert_eq!( + subjects, + vec![vec![ + "shared_class_uri".to_string(), + "Alpha".to_string(), + "Beta".to_string() + ]], + "the sharing classes are the subject, behind the rule's discriminator, \ + and the controls stay silent: {warnings:#?}" + ); + let w = &warnings[0]; + assert_eq!( + w.problem_type, + ValidationProblemType::AmbiguousElementIdentity + ); + assert!(!w.severity.is_error(), "the linter warns, never errors"); + assert!( + w.detail.contains("Alpha") && w.detail.contains("Beta"), + "the warning must name both classes: {}", + w.detail + ); + assert!( + w.detail.contains("designates_type") && w.detail.contains("class_uri"), + "the warning must name both halves of the condition: {}", + w.detail + ); + assert!( + w.detail.contains("typeURI"), + "the warning must name the designator the URI is dispatched by: {}", + w.detail + ); +} diff --git a/src/runtime/tests/inlined_dict_key.rs b/src/runtime/tests/inlined_dict_key.rs new file mode 100644 index 0000000..94b2dfa --- /dev/null +++ b/src/runtime/tests/inlined_dict_key.rs @@ -0,0 +1,351 @@ +//! Spec addendum rule 5 — the inlined-dict key is real data (D5). +//! +//! LinkML's `inlined` dict form says the mapping key *is* the element's +//! key/identifier value; the payload is allowed to leave it out. The loader +//! used never to write it back, so a `required` key slot produced a +//! `MissingSlotValue` error on perfectly legal data, and the object that came +//! out of the load did not carry its own key. +//! +//! Three facts are pinned here: +//! +//! * the dict key is **injected** into the key slot when the payload omits it, +//! before the required/cardinality constraints run and before the type +//! designator is filled, so it flows through the same canonicalisation every +//! other designator value does (rule 2); +//! * a payload value that **disagrees** with the dict key is a warning naming +//! both. The payload value is data and is what gets *stored*; the dict key is +//! the address and is what the mapping stays keyed by; +//! * for a dict whose key slot **is** the type designator, a dict key that is +//! no accepted designator value is a warning. + +use linkml_runtime::{ + load_json_str, LinkMLInstance, ValidationProblemType, ValidationResult, ValidationSeverity, +}; +use linkml_schemaview::identifier::{converter_from_schema, Identifier}; +use linkml_schemaview::io::from_yaml; +use linkml_schemaview::schemaview::{ClassView, SchemaView}; +use linkml_schemaview::Converter; +use serde_json::{json, Value as JsonValue}; +use std::path::PathBuf; + +/// `LinearCoordinate`'s canonical designator value: its `class_uri` (the slot +/// range is `uriorcurie`, so the canonical spelling is the CURIE). +const LINEAR_CANONICAL: &str = "ex:Linear"; +/// The same class's *schema-native* URI — a second accepted spelling, and the +/// one asset360's committed data uses as the dict key while spelling the +/// payload with the `class_uri`. +const LINEAR_NATIVE: &str = "https://w3id.org/linkml/examples/inlined_dict_key/LinearCoordinate"; +const WGS84_CURIE: &str = "ex:WGS84"; +const WGS84_URI: &str = "https://example.org/dk/WGS84"; + +struct Fixture { + sv: SchemaView, + conv: Converter, + container: ClassView, +} + +fn fixture() -> Fixture { + let mut p = PathBuf::from(env!("CARGO_MANIFEST_DIR")); + p.push("tests/data/inlined_dict_key.yaml"); + let schema = from_yaml(&p).unwrap(); + let mut sv = SchemaView::new(); + sv.add_schema(schema.clone()).unwrap(); + let conv = converter_from_schema(&schema); + let container = sv + .get_class(&Identifier::new("Container"), &conv) + .unwrap() + .expect("class not found"); + Fixture { + sv, + conv, + container, + } +} + +impl Fixture { + fn load_result(&self, v: JsonValue) -> (LinkMLInstance, Vec) { + let r = load_json_str(&v.to_string(), &self.sv, &self.container, &self.conv).unwrap(); + let issues = r.validation_issues.clone(); + (r.into_instance().expect("instance must load"), issues) + } +} + +fn errors(issues: &[ValidationResult]) -> Vec<&ValidationResult> { + issues.iter().filter(|i| i.severity.is_error()).collect() +} + +// --------------------------------------------------------------------------- +// Injection +// --------------------------------------------------------------------------- + +/// The payload omits the key slot, as the `inlined` contract allows. The dict +/// key fills it: no `MissingSlotValue` for the `required` key, and the object +/// carries its own key out through `to_json`. +#[test] +fn dict_key_fills_the_key_slot_the_payload_omitted() { + let f = fixture(); + let (inst, issues) = f.load_result(json!({ + "people": {"p1": {"name": "Ann"}, "p2": {"name": "Bo"}} + })); + assert!( + errors(&issues).is_empty(), + "the dict key satisfies the required key slot: {issues:#?}" + ); + assert!( + !issues + .iter() + .any(|i| i.problem_type == ValidationProblemType::MissingSlotValue), + "no missing-value diagnostic of any severity: {issues:#?}" + ); + let out = inst.to_json(); + assert_eq!(out["people"]["p1"]["pid"], json!("p1"), "in {out:#?}"); + assert_eq!(out["people"]["p2"]["pid"], json!("p2"), "in {out:#?}"); +} + +/// The payload states the same key the dict does. Nothing to inject, nothing +/// to say. +#[test] +fn payload_key_equal_to_the_dict_key_is_silent() { + let f = fixture(); + let (inst, issues) = f.load_result(json!({ + "people": {"p1": {"pid": "p1", "name": "Ann"}} + })); + assert!(issues.is_empty(), "no diagnostics expected: {issues:#?}"); + assert_eq!(inst.to_json()["people"]["p1"]["pid"], json!("p1")); +} + +/// The payload contradicts the dict key. Rule 5 makes this a **warning**, not +/// an error, and the ruling on which value survives is recorded here: the +/// payload is the element's data and is stored; the dict key is the address and +/// the mapping stays keyed by it. Both values are named in the message so the +/// author can see which one to change. +#[test] +fn payload_key_disagreeing_with_the_dict_key_warns_and_the_payload_is_stored() { + let f = fixture(); + let (inst, issues) = f.load_result(json!({ + "people": {"p1": {"pid": "p2", "name": "Ann"}} + })); + assert!( + errors(&issues).is_empty(), + "a divergence is a warning, never an error: {issues:#?}" + ); + let warned: Vec<&ValidationResult> = issues + .iter() + .filter(|i| i.severity == ValidationSeverity::Warning) + .collect(); + assert_eq!(warned.len(), 1, "exactly one warning: {issues:#?}"); + let detail = &warned[0].detail; + assert!( + detail.contains("p1") && detail.contains("p2"), + "the warning must name both values: {detail:?}" + ); + assert_eq!( + warned[0].subject, + vec!["people".to_string(), "p1".to_string(), "pid".to_string()], + "reported at the key slot of the entry the dict key addresses" + ); + let out = inst.to_json(); + assert_eq!( + out["people"]["p1"]["pid"], + json!("p2"), + "the payload value is the stored data: {out:#?}" + ); + assert!( + out["people"].get("p2").is_none(), + "the mapping stays addressed by its dict key: {out:#?}" + ); +} + +/// A key slot whose range descends from `uri` compares as an IRI, not as a +/// string (rule 2): a CURIE dict key and its expansion in the payload are one +/// identity and must not be reported as a divergence. +#[test] +fn a_curie_dict_key_and_its_expanded_payload_are_one_identity() { + let f = fixture(); + let (_, issues) = f.load_result(json!({ + "systems": {WGS84_CURIE: {"systemType": WGS84_URI, "label": "one"}} + })); + assert!( + issues.is_empty(), + "a curie and its expansion are one identity: {issues:#?}" + ); +} + +// --------------------------------------------------------------------------- +// Designator-keyed dicts +// --------------------------------------------------------------------------- + +/// The dict key names the entry's class. An accepted spelling selects that +/// class, and the injected value is canonicalised on the way in exactly as a +/// payload-supplied designator is — the stored value stays canonical. +#[test] +fn an_accepted_designator_dict_key_selects_the_class_and_is_canonicalised() { + let f = fixture(); + let (inst, issues) = f.load_result(json!({ + "coords": {LINEAR_NATIVE: {"measure": 1.5}} + })); + assert!( + issues.is_empty(), + "the native URI is an accepted designator value: {issues:#?}" + ); + let out = inst.to_json(); + let entry = &out["coords"][LINEAR_NATIVE]; + assert_eq!( + entry["typeURI"], + json!(LINEAR_CANONICAL), + "injected from the dict key and canonicalised: {out:#?}" + ); + assert_eq!( + entry["measure"], + json!(1.5), + "the subclass's own slot survives, so the subclass was selected: {out:#?}" + ); +} + +/// A dict key that is no accepted designator value at all is a warning — the +/// case D5 found silently masked, because `populate_type_designator` filled the +/// slot from the range class and nothing ever looked at the key. +#[test] +fn a_designator_dict_key_that_is_not_accepted_warns() { + let f = fixture(); + let (inst, issues) = f.load_result(json!({ + "coords": {"Whatever": {"value": "v"}} + })); + assert!( + errors(&issues).is_empty(), + "an unaccepted key is a warning, never an error: {issues:#?}" + ); + assert_eq!(issues.len(), 1, "exactly one warning: {issues:#?}"); + assert!( + issues[0].detail.contains("Whatever"), + "the warning must name the rejected key: {:?}", + issues[0].detail + ); + let out = inst.to_json(); + assert_eq!( + out["coords"]["Whatever"]["typeURI"], + json!("dk:Coordinate"), + "the range class's canonical value is what is stored: {out:#?}" + ); +} + +/// The payload is fine on its own terms but the dict key is junk. The key's own +/// rejection has to be heard even when nothing else is wrong with the entry. +#[test] +fn an_unaccepted_designator_key_is_heard_even_when_the_payload_is_accepted() { + let f = fixture(); + let (_, issues) = f.load_result(json!({ + "coords": {"Whatever": {"typeURI": LINEAR_CANONICAL, "measure": 2.0}} + })); + assert!(errors(&issues).is_empty(), "warnings only: {issues:#?}"); + assert!( + issues.iter().any(|i| i.detail.contains("Whatever") + && i.detail.contains("not an accepted designator value")), + "the unaccepted dict key must be named: {issues:#?}" + ); +} + +/// The compact form of a designator-*keyed* dict: the dict key is the only +/// thing that names the class, and the bare scalar fills the first ordinary +/// slot. Selection, injection and canonicalisation all have to come off the key +/// alone. +#[test] +fn a_compact_entry_takes_its_class_from_a_designator_dict_key() { + let f = fixture(); + let (inst, issues) = f.load_result(json!({ + "coords": {LINEAR_NATIVE: "somevalue"} + })); + assert!(issues.is_empty(), "no diagnostics expected: {issues:#?}"); + let out = inst.to_json(); + let entry = &out["coords"][LINEAR_NATIVE]; + assert_eq!( + entry["typeURI"], + json!(LINEAR_CANONICAL), + "the class the key named, canonicalised: {out:#?}" + ); + assert_eq!( + entry["value"], + json!("somevalue"), + "the compact scalar lands in the first ordinary slot: {out:#?}" + ); +} + +/// The asset360 shape, reduced: the dict key spells the class with its +/// schema-native URI and the payload spells it with the `class_uri`. Both are +/// accepted designator values of the same class, but the payload is +/// canonicalised at load and the key is not, so the *loaded* entry really is +/// addressed by one IRI and carries another. That contradiction is the warning. +#[test] +fn native_uri_key_against_class_uri_payload_is_a_divergence() { + let f = fixture(); + let (inst, issues) = f.load_result(json!({ + "coords": {LINEAR_NATIVE: {"typeURI": LINEAR_CANONICAL, "measure": 3.0}} + })); + assert!(errors(&issues).is_empty(), "warnings only: {issues:#?}"); + let divergences: Vec<&ValidationResult> = issues + .iter() + .filter(|i| i.detail.contains("disagrees")) + .collect(); + assert_eq!(divergences.len(), 1, "one divergence: {issues:#?}"); + assert!( + divergences[0].detail.contains(LINEAR_NATIVE) + && divergences[0].detail.contains(LINEAR_CANONICAL), + "both spellings named: {:?}", + divergences[0].detail + ); + let out = inst.to_json(); + assert_eq!( + out["coords"][LINEAR_NATIVE]["typeURI"], + json!(LINEAR_CANONICAL), + "the payload value, canonicalised, is the stored data: {out:#?}" + ); +} + +/// The mirror image of the test above, and the reason the divergence check +/// compares the value **as stored** rather than the raw payload: here the key is +/// the canonical spelling and the payload is the native URI, so rule 2 rewrites +/// the payload to the key's own spelling. The entry that comes out agrees with +/// the key it is written under, and reloading that output is silent — so the +/// load must be silent too. Comparing raw payloads would warn here, and the +/// message would name a "stored" value that is not what was stored. +#[test] +fn a_payload_canonicalised_onto_the_dict_key_is_not_a_divergence() { + let f = fixture(); + let (inst, issues) = f.load_result(json!({ + "coords": {LINEAR_CANONICAL: {"typeURI": LINEAR_NATIVE, "measure": 4.0}} + })); + assert!( + issues.is_empty(), + "canonicalisation made the two agree before anything was stored: {issues:#?}" + ); + let out = inst.to_json(); + assert_eq!( + out["coords"][LINEAR_CANONICAL]["typeURI"], + json!(LINEAR_CANONICAL), + "in {out:#?}" + ); + // The invariant that makes the silence correct: the emitted document reloads + // without a word. + let (_, reloaded) = f.load_result(out.clone()); + assert!(reloaded.is_empty(), "round-trip must stay silent: {out:#?}"); +} + +/// The same invariant from the other side: the asset360 divergence is real +/// precisely because it *survives* the round-trip — the emitted document is +/// still addressed by one IRI and still carries another. +#[test] +fn a_real_divergence_survives_the_round_trip() { + let f = fixture(); + let (inst, _) = f.load_result(json!({ + "coords": {LINEAR_NATIVE: {"typeURI": LINEAR_CANONICAL, "measure": 3.0}} + })); + let (_, reloaded) = f.load_result(inst.to_json()); + assert_eq!( + reloaded + .iter() + .filter(|i| i.detail.contains("disagrees")) + .count(), + 1, + "the contradiction is in the data, not in the load: {reloaded:#?}" + ); +} diff --git a/src/runtime/tests/navigate.rs b/src/runtime/tests/navigate.rs index 5c04155..9900022 100644 --- a/src/runtime/tests/navigate.rs +++ b/src/runtime/tests/navigate.rs @@ -35,16 +35,114 @@ fn navigate_basic() { match &v { linkml_runtime::LinkMLInstance::Object { values, .. } => { assert!(values.contains_key("objects")); + // `objects` is inlined as a list of NamedThing, which declares an + // `id` identifier: it is addressed by that label, not by position. + // `has_medical_history` declares no identity, so it stays numeric. + // Mirrors src/python/tests/python_navigate.rs one-to-one: + // `PyLinkMLInstance::navigate` delegates straight to this method, + // and that test's assertions live inside a `py_run!` string that + // cannot be executed where the python crate does not link. let inner = v.navigate_path([ "objects", - "2", + "P:002", "has_medical_history", "0", "diagnosis", "name", ]); - assert!(inner.is_some()); + assert_eq!( + inner.map(|n| n.to_json()), + Some(serde_json::json!("headache")) + ); + assert!( + v.navigate_path(["objects", "2"]).is_none(), + "a numeric segment must not address a label-addressed list" + ); + assert!( + v.navigate_path(["objects", "P:404"]).is_none(), + "a label matching no element resolves to nothing" + ); } _ => panic!("expected map at root"), } } + +/// The identity fixture, loaded as a `Service` instance. +fn service(data: serde_json::Value) -> linkml_runtime::LinkMLInstance { + let schema = from_yaml(Path::new(&info_path("identity.yaml"))).unwrap(); + let mut sv = SchemaView::new(); + sv.add_schema(schema.clone()).unwrap(); + let conv = converter_from_schema(&schema); + let class = sv + .get_class(&Identifier::new("Service"), &conv) + .unwrap() + .expect("class not found"); + linkml_runtime::load_json_str(&data.to_string(), &sv, &class, &conv) + .unwrap() + .into_instance() + .unwrap() +} + +fn scalar(v: Option<&linkml_runtime::LinkMLInstance>) -> Option { + v.map(|v| v.to_json()) +} + +#[test] +fn navigate_resolves_a_unique_keys_segment() { + // `diff` addresses this list by its `unique_keys`-derived label, so a + // delta path that names one has to be navigable. + let v = service(serde_json::json!({"name": "svc", "hasPhoneNumber": [ + {"phoneNumber": "09/241.25.00", "hasNumberFunction": "Emergency_Number"}, + {"phoneNumber": "09/241.25.03", "hasNumberFunction": "Non_Urgent_Communication"}]})); + assert_eq!( + scalar(v.navigate_path(["hasPhoneNumber", "Non_Urgent_Communication", "phoneNumber"])), + Some(serde_json::json!("09/241.25.03")) + ); +} + +#[test] +fn navigate_resolves_a_composite_unique_key_segment() { + let v = service(serde_json::json!({"name": "svc", "contacts": [ + {"kind": "Emergency", "phone": "02/111.11.11", "note": "first"}, + {"kind": "Operator", "phone": "02/333.33.33", "note": "second"}]})); + assert_eq!( + scalar(v.navigate_path(["contacts", r#"["Emergency","02/111.11.11"]"#, "note"])), + Some(serde_json::json!("first")) + ); +} + +#[test] +fn navigate_refuses_a_numeric_segment_into_a_label_addressed_list() { + // `patch` refuses this ("report, never guess"); navigating must not + // silently hand back a different element than the one the path names. + let v = service(serde_json::json!({"name": "svc", "hasPhoneNumber": [ + {"phoneNumber": "09/241.25.00", "hasNumberFunction": "Emergency_Number"}, + {"phoneNumber": "09/241.25.03", "hasNumberFunction": "Non_Urgent_Communication"}]})); + assert!(v.navigate_path(["hasPhoneNumber", "0"]).is_none()); +} + +#[test] +fn navigate_prefers_the_label_over_the_index_when_a_label_is_numeric() { + // `lang` is a key slot, and one element's key IS "0". Index-first + // navigation hands back the element at position 0 — the wrong one. + let v = service(serde_json::json!({"name": "svc", "labelList": [ + {"lang": "nl", "text": "dutch"}, + {"lang": "0", "text": "zero"}]})); + assert_eq!( + scalar(v.navigate_path(["labelList", "0", "text"])), + Some(serde_json::json!("zero")) + ); +} + +#[test] +fn navigate_still_indexes_a_positional_list_numerically() { + // PlainPhoneNumber declares no identity: diff emits numeric segments for + // this list, and they must keep resolving. + let v = service(serde_json::json!({"name": "svc", "plainPhoneNumber": [ + {"phoneNumber": "09/241.25.00", "hasNumberFunction": "Emergency_Number"}, + {"phoneNumber": "09/241.25.03", "hasNumberFunction": "Non_Urgent_Communication"}]})); + assert_eq!( + scalar(v.navigate_path(["plainPhoneNumber", "1", "phoneNumber"])), + Some(serde_json::json!("09/241.25.03")) + ); +} diff --git a/src/runtime/tests/validation.rs b/src/runtime/tests/validation.rs index 399798e..cab56d7 100644 --- a/src/runtime/tests/validation.rs +++ b/src/runtime/tests/validation.rs @@ -155,9 +155,13 @@ fn validation_issue_paths_include_list_indices_once() { "1".to_string(), "primary_email".to_string(), ]; + // The fixture's second object declares `objecttype` as Person, so the bad + // email is a pattern violation on a declared slot rather than an unknown + // slot. This test is about the *path* carrying the list index exactly once, + // but it still pins a problem type so the filter stays a real assertion. let diag_paths: Vec<_> = diags .iter() - .filter(|d| matches!(d.problem_type, ValidationProblemType::UndeclaredSlot)) + .filter(|d| matches!(d.problem_type, ValidationProblemType::SlotRangeViolation)) .map(|d| d.subject.clone()) .collect(); assert!( diff --git a/src/schemaview/src/classview.rs b/src/schemaview/src/classview.rs index ca23d95..2aa6b0d 100644 --- a/src/schemaview/src/classview.rs +++ b/src/schemaview/src/classview.rs @@ -1,8 +1,8 @@ -use std::collections::{HashMap, HashSet}; +use std::collections::{HashMap, HashSet, VecDeque}; use std::sync::{Arc, OnceLock}; use crate::converter::Converter; -use linkml_meta::{ClassDefinition, SchemaDefinition, SlotDefinition}; +use linkml_meta::{ClassDefinition, SchemaDefinition, SlotDefinition, UniqueKey}; use crate::identifier::{Identifier, IdentifierError}; use crate::schemaview::{CanonicalIds, SchemaView, SchemaViewError}; @@ -254,6 +254,14 @@ impl ClassView { /// - `native=true` uses the schema's default prefix; `native=false` prefers /// an explicit `class_uri` when set. /// - `expand=true` returns a full URI; `expand=false` returns a CURIE. + /// + /// All four combinations are distinct for a class that declares a + /// `class_uri`. In particular `(native: true, expand: true)` is the + /// *schema-native* URI (``) and not the + /// `class_uri` — that is what `(native: false, expand: true)` returns. + /// [`Self::get_accepted_type_designator_values`] depends on the difference: + /// data that spells a designator with the schema-native URI of a class that + /// also declares a `class_uri` still *means* that class. pub fn get_uri( &self, conv: &Converter, @@ -279,7 +287,7 @@ impl ClassView { expand: bool, ) -> Result { match (native, expand) { - (true, true) => Ok(ids.canonical_uri()), + (true, true) => Ok(ids.native_uri()), (true, false) => { if let Some(curie) = ids.native_curie() { Ok(curie) @@ -306,9 +314,6 @@ impl ClassView { native: bool, expand: bool, ) -> Result { - if native && expand { - return Ok(self.canonical_uri()); - } let schema = self .data .sv @@ -621,6 +626,47 @@ impl ClassView { .find(|s| s.definition().identifier.unwrap_or(false)) } + /// Returns the class's `unique_keys`, merged across the inheritance chain + /// (`is_a` parents and mixins), with the nearest declaration winning per name. + /// + /// The result is sorted by unique key name: declaration order is lost in the + /// underlying map, and consumers (such as diff path segments) need a + /// deterministic order. + pub fn unique_keys(&self) -> Vec<(String, UniqueKey)> { + let mut merged: HashMap = HashMap::new(); + let mut queue: VecDeque = VecDeque::from([self.clone()]); + let mut seen: HashSet = HashSet::new(); + + while let Some(cv) = queue.pop_front() { + if !seen.insert(cv.canonical_uri().to_string()) { + continue; + } + if let Some(uks) = cv.def().unique_keys.as_ref() { + for (name, uk) in uks { + merged.entry(name.clone()).or_insert_with(|| (**uk).clone()); + } + } + if let Ok(Some(parent)) = cv.parent_class() { + queue.push_back(parent); + } + if let Some(mixins) = &cv.data.class.mixins { + if let Some(conv) = cv.data.sv.converter_for_schema(&cv.data.schema_uri) { + for mixin in mixins { + if let Ok(Some(mixin_view)) = + cv.data.sv.get_class(&Identifier::new(mixin), &conv) + { + queue.push_back(mixin_view); + } + } + } + } + } + + let mut out: Vec<(String, UniqueKey)> = merged.into_iter().collect(); + out.sort_by(|a, b| a.0.cmp(&b.0)); + out + } + fn collect_ancestors_map( class_view: &ClassView, include_mixins: bool, diff --git a/src/schemaview/src/resolve.rs b/src/schemaview/src/resolve.rs index b920e09..c29bb52 100644 --- a/src/schemaview/src/resolve.rs +++ b/src/schemaview/src/resolve.rs @@ -2,7 +2,8 @@ use crate::{ io::{from_uri, from_yaml}, schemaview::SchemaView, }; -use std::path::Path; +use std::collections::HashMap; +use std::path::{Path, PathBuf}; fn get_uri_for_id(id: &str) -> Option<&'static str> { match id { @@ -15,68 +16,168 @@ fn get_uri_for_id(id: &str) -> Option<&'static str> { } } -pub fn resolve_schemas(sv: &mut SchemaView) -> Result<(), String> { - let unresolved = sv.get_unresolved_schemas(); - if unresolved.is_empty() { +/// Directory a schema's relative imports should be resolved against, given the +/// path the schema itself was loaded from. Accepts either the schema file or a +/// directory, so callers can pass whichever they have. +fn source_dir_of(path: &Path) -> Option { + let canonical = std::fs::canonicalize(path).ok()?; + if canonical.is_dir() { + Some(canonical) + } else { + canonical.parent().map(|p| p.to_path_buf()) + } +} + +/// Returns `path` if it names an existing file, retrying with a `.yaml`/`.yml` +/// extension — LinkML imports habitually omit it (`imports: [./types]`). +fn existing_schema_file(path: PathBuf) -> Option { + if path.is_file() { + return Some(path); + } + for ext in ["yaml", "yml"] { + let candidate = path.with_extension(ext); + if candidate.is_file() { + return Some(candidate); + } + } + None +} + +/// Locates the file behind a relative import of `schema_id`. +/// +/// Bases are tried in order of how much we actually know: +/// 1. the directory of the file `schema_id` was itself loaded from, +/// 2. the directory of the import URI that pulled `schema_id` in, +/// 3. the process working directory — the historical behaviour, kept last so +/// that schemas importing CWD-relative paths keep resolving. +fn locate_import( + sv: &SchemaView, + schema_id: &str, + uri: &str, + source_dirs: &HashMap, +) -> Option { + let raw = Path::new(uri); + if raw.is_absolute() { + return existing_schema_file(raw.to_path_buf()); + } + let mut bases: Vec = Vec::new(); + if let Some(dir) = source_dirs.get(schema_id) { + bases.push(dir.clone()); + } + if let Some(resolution_uri) = sv.get_resolution_uri_of_schema(schema_id) { + if let Some(parent) = Path::new(&resolution_uri).parent() { + bases.push(parent.to_path_buf()); + } + } + bases.push(PathBuf::new()); + for base in bases { + if let Some(found) = existing_schema_file(base.join(raw)) { + return Some(found); + } + } + None +} + +/// Loads the schema behind one unresolved import and records where it came +/// from, so that schema's own relative imports can be resolved in a later round. +fn resolve_one( + sv: &mut SchemaView, + schema_id: &str, + uri: &str, + source_dirs: &mut HashMap, +) -> Result<(), String> { + let import_ref = Some((schema_id.to_string(), uri.to_string())); + + if let Some(resolved_uri) = get_uri_for_id(uri) { + let schema = from_uri(resolved_uri) + .map_err(|e| format!("Failed to load schema from {}: {}", resolved_uri, e))?; + sv.add_schema_with_import_ref(schema, import_ref)?; return Ok(()); } - for (schema_id, uri) in unresolved { - let schema_source_uri = sv.get_resolution_uri_of_schema(&schema_id); - if let Some(resolved_uri) = get_uri_for_id(&uri) { - // Load the schema from the resolved URI - let schema = match from_uri(resolved_uri) { - Ok(s) => s, - Err(e) => { - return Err(format!( - "Failed to load schema from {}: {}", - resolved_uri, e - )) - } - }; - sv.add_schema_with_import_ref(schema, Some((schema_id, uri)))?; - } else { - // Attempt to treat the unresolved entry as a local file path - let mut path = Path::new(&uri).to_owned(); - if !path.is_absolute() { - // imported_from_dir = parent from schema_source_uri - let imported_from_dir = - Path::new(schema_source_uri.as_deref().unwrap_or("unknown")) - .parent() - .map(|p| p.to_path_buf()); - if let Some(dir) = imported_from_dir { - path = dir.join(path); - } - if !path.exists() && path.with_extension("yaml").exists() { - path.set_extension("yaml"); - } - if !path.exists() && path.with_extension("yml").exists() { - path.set_extension("yml"); - } + let Some(path) = locate_import(sv, schema_id, uri, source_dirs) else { + return Err(format!( + "No resolution found for URI: {} imported from {}", + uri, + source_dirs + .get(schema_id) + .map(|d| d.display().to_string()) + .or_else(|| sv.get_resolution_uri_of_schema(schema_id)) + .unwrap_or_else(|| schema_id.to_owned()) + )); + }; + + let schema = from_yaml(&path) + .map_err(|e| format!("Failed to load schema from {}: {}", path.display(), e))?; + let loaded_id = schema.id.clone(); + sv.add_schema_with_import_ref(schema, import_ref)?; + if let Some(dir) = source_dir_of(&path) { + source_dirs.insert(loaded_id, dir); + } + Ok(()) +} + +/// Resolves imports repeatedly until nothing is left or a round achieves +/// nothing, so that imports-of-imports are reached. +/// +/// A failure no longer abandons the rest of the pass: every import is attempted +/// each round, and only when a round resolves nothing do the accumulated +/// messages become the error. An import that failed while other work was still +/// progressing gets retried, because that later work may be exactly what makes +/// it resolvable. +fn resolve_to_fixpoint( + sv: &mut SchemaView, + mut source_dirs: HashMap, +) -> Result<(), String> { + loop { + let unresolved = sv.get_unresolved_schemas(); + if unresolved.is_empty() { + return Ok(()); + } + let mut progressed = false; + let mut failures: Vec = Vec::new(); + for (schema_id, uri) in unresolved { + match resolve_one(sv, &schema_id, &uri, &mut source_dirs) { + Ok(()) => progressed = true, + Err(e) => failures.push(e), } - if path.exists() { - let schema = match from_yaml(&path) { - Ok(s) => s, - Err(e) => { - return Err(format!( - "Failed to load schema from {}: {}", - path.display(), - e - )) - } - }; - sv.add_schema_with_import_ref( - schema.clone(), - Some((schema_id.clone(), uri.clone())), - )?; + } + if !progressed { + return Err(if failures.is_empty() { + "import resolution stalled with imports still unresolved".to_string() } else { - return Err(format!( - "No resolution found for URI: {} imported from {}", - uri, - schema_source_uri.unwrap_or("unknown".to_owned()) - )); - } + failures.join("\n") + }); } } - Ok(()) +} + +/// Resolves every import reachable from the schemas already in `sv`. +/// +/// Relative imports of those schemas are resolved against the process working +/// directory, because nothing records where they were loaded from. Prefer +/// [`resolve_schemas_from`], which removes that guess. +pub fn resolve_schemas(sv: &mut SchemaView) -> Result<(), String> { + resolve_to_fixpoint(sv, HashMap::new()) +} + +/// Resolves every import reachable from the schemas already in `sv`, treating +/// `root_source` — the path they were loaded from — as the base for their +/// relative imports. Schemas pulled in during resolution get the same treatment +/// from their own file, so `a.yaml` importing `./b` works at any depth and from +/// any working directory. +/// +/// `root_source` may be the schema file or its directory. +pub fn resolve_schemas_from(sv: &mut SchemaView, root_source: &Path) -> Result<(), String> { + let mut source_dirs = HashMap::new(); + if let Some(dir) = source_dir_of(root_source) { + // Everything currently in the view came from `root_source`: the callers + // load exactly one schema before resolving. + sv.with_schema_definitions(|schemas| { + for id in schemas.keys() { + source_dirs.insert(id.clone(), dir.clone()); + } + }); + } + resolve_to_fixpoint(sv, source_dirs) } diff --git a/src/schemaview/src/slotview.rs b/src/schemaview/src/slotview.rs index 7beadc3..0100bd5 100644 --- a/src/schemaview/src/slotview.rs +++ b/src/schemaview/src/slotview.rs @@ -538,6 +538,18 @@ impl SlotView { .is_some_and(|ri| ri.is_integer()) } + /// Returns `true` when the primary range's type hierarchy contains `uri` or + /// `uriorcurie` — the value denotes an IRI, whatever it is spelled as. + /// + /// RDF serialization uses this to emit a named node rather than a literal; + /// the runtime's identity machinery uses it to know that a CURIE and its + /// expansion are the same value and must compare equal. + pub fn is_range_iri(&self) -> bool { + self.get_range_info() + .first() + .is_some_and(|ri| ri.is_range_iri) + } + /// Returns the resolved container shape for this slot. /// /// This resolves the interacting `multivalued`, `inlined`, and diff --git a/src/schemaview/tests/class_uri.rs b/src/schemaview/tests/class_uri.rs index c615504..47de95c 100644 --- a/src/schemaview/tests/class_uri.rs +++ b/src/schemaview/tests/class_uri.rs @@ -29,6 +29,18 @@ fn class_get_uri() { cv.get_uri(&conv, true, false).unwrap().to_string(), "personinfo:Person" ); + assert_eq!( + cv.get_uri(&conv, false, true).unwrap().to_string(), + "https://w3id.org/linkml/Person" + ); + // `native` must survive expansion: this is the schema-native URI, not the + // `class_uri`. `get_accepted_type_designator_values` relies on the two being + // distinct — data spelling a designator with the native URI of a class that + // also declares a `class_uri` still means that class. + assert_eq!( + cv.get_uri(&conv, true, true).unwrap().to_string(), + "https://w3id.org/linkml/examples/personinfo/Person" + ); assert_eq!( cv.canonical_uri().to_string(), "https://w3id.org/linkml/Person" diff --git a/src/schemaview/tests/data/missing_imports.yaml b/src/schemaview/tests/data/missing_imports.yaml new file mode 100644 index 0000000..834b564 --- /dev/null +++ b/src/schemaview/tests/data/missing_imports.yaml @@ -0,0 +1,15 @@ +id: http://example.com/missing_imports +name: missing_imports +prefixes: + missing_imports: http://example.com/missing_imports/ +default_prefix: missing_imports +default_range: string +# Neither file exists: resolution must report BOTH, not stop at the first. +imports: + - ./definitely_not_here + - ./also_not_here +classes: + Stub: + attributes: + id: + identifier: true diff --git a/src/schemaview/tests/data/nested_a.yaml b/src/schemaview/tests/data/nested_a.yaml new file mode 100644 index 0000000..bb46b4d --- /dev/null +++ b/src/schemaview/tests/data/nested_a.yaml @@ -0,0 +1,17 @@ +id: http://example.com/nested_a +name: nested_a +prefixes: + nested_a: http://example.com/nested_a/ + nested_b: http://example.com/nested_b/ +default_prefix: nested_a +default_range: string +# An import of an import: only reachable if resolution loops to a fixpoint. +imports: + - ./nested_b +classes: + MiddleThing: + attributes: + id: + identifier: true + leaf: + range: nested_b:LeafThing diff --git a/src/schemaview/tests/data/nested_b.yaml b/src/schemaview/tests/data/nested_b.yaml new file mode 100644 index 0000000..df12948 --- /dev/null +++ b/src/schemaview/tests/data/nested_b.yaml @@ -0,0 +1,12 @@ +id: http://example.com/nested_b +name: nested_b +prefixes: + nested_b: http://example.com/nested_b/ +default_prefix: nested_b +default_range: string +classes: + LeafThing: + attributes: + id: + identifier: true + label: {} diff --git a/src/schemaview/tests/data/nested_root.yaml b/src/schemaview/tests/data/nested_root.yaml new file mode 100644 index 0000000..aa9d474 --- /dev/null +++ b/src/schemaview/tests/data/nested_root.yaml @@ -0,0 +1,17 @@ +id: http://example.com/nested_root +name: nested_root +prefixes: + nested_root: http://example.com/nested_root/ + nested_a: http://example.com/nested_a/ +default_prefix: nested_root +default_range: string +# Relative to THIS file's directory, not the process CWD. +imports: + - ./nested_a +classes: + RootThing: + attributes: + id: + identifier: true + middle: + range: nested_a:MiddleThing diff --git a/src/schemaview/tests/data/unique_keys.yaml b/src/schemaview/tests/data/unique_keys.yaml new file mode 100644 index 0000000..37cdcf1 --- /dev/null +++ b/src/schemaview/tests/data/unique_keys.yaml @@ -0,0 +1,35 @@ +id: http://example.org/unique_keys +name: unique_keys +prefixes: + ex: http://example.org/unique_keys/ +default_prefix: ex +default_range: string + +classes: + Base: + unique_keys: + by_code: + unique_key_slots: [code] + shared_name: + unique_key_slots: [base_field] + attributes: + code: {range: string} + base_field: {range: string} + MixinCls: + mixin: true + unique_keys: + by_tag: + unique_key_slots: [tag] + attributes: + tag: {range: string} + Child: + is_a: Base + mixins: [MixinCls] + unique_keys: + shared_name: + unique_key_slots: [child_field] + attributes: + child_field: {range: string} + Plain: + attributes: + whatever: {range: string} diff --git a/src/schemaview/tests/nested_local_import.rs b/src/schemaview/tests/nested_local_import.rs new file mode 100644 index 0000000..a36f2eb --- /dev/null +++ b/src/schemaview/tests/nested_local_import.rs @@ -0,0 +1,91 @@ +#![cfg(feature = "resolve")] +//! Relative imports must resolve against the importing schema's own file, and +//! resolution must keep going until nothing is left to resolve. +//! +//! `nested_root.yaml` imports `./nested_a`, which in turn imports `./nested_b`. +//! Two independent things have to work for `nested_b` to arrive: +//! +//! 1. **Fixpoint.** `nested_a`'s import only becomes visible after `nested_a` +//! itself is loaded, so a single pass over the initially-unresolved list can +//! never reach `nested_b`. +//! 2. **Source-relative bases.** `./nested_a` is meaningless relative to the +//! process CWD (these tests run from the crate root, not `tests/data`), so +//! the base directory has to come from the file each schema was loaded from. + +use linkml_schemaview::io::from_yaml; +use linkml_schemaview::resolve::{resolve_schemas, resolve_schemas_from}; +use linkml_schemaview::schemaview::SchemaView; +use std::path::PathBuf; + +fn data_path(name: &str) -> PathBuf { + let mut p = PathBuf::from(env!("CARGO_MANIFEST_DIR")); + p.push("tests"); + p.push("data"); + p.push(name); + p +} + +fn view_of(name: &str) -> (SchemaView, PathBuf) { + let path = data_path(name); + let schema = from_yaml(&path).unwrap(); + let mut sv = SchemaView::new(); + sv.add_schema(schema).unwrap(); + (sv, path) +} + +#[test] +fn resolves_an_import_of_an_import_relative_to_each_source_file() { + let (mut sv, path) = view_of("nested_root.yaml"); + // The process CWD is the crate root, so `./nested_a` does not exist + // relative to it: only the seeded source directory can resolve this. + assert!( + !PathBuf::from("./nested_a.yaml").exists(), + "precondition: the fixture must not be reachable from the CWD, \ + otherwise this test would pass for the wrong reason" + ); + + resolve_schemas_from(&mut sv, &path).unwrap(); + + assert!( + sv.get_unresolved_schemas().is_empty(), + "left unresolved: {:?}", + sv.get_unresolved_schemas() + ); + assert!( + sv.get_schema("http://example.com/nested_a").is_some(), + "the root's own import must resolve against the root's directory" + ); + assert!( + sv.get_schema("http://example.com/nested_b").is_some(), + "the import of an import must resolve against ITS importer's directory, \ + which requires both the fixpoint loop and per-schema source tracking" + ); +} + +#[test] +fn unseeded_resolution_still_honours_cwd_relative_imports() { + // `local_main.yaml` imports `tests/data/local_target.yaml`, a CWD-relative + // path. Seeding a source directory must not break that older style: the + // source directory is the first base tried, not the only one. + let (mut sv, path) = view_of("local_main.yaml"); + resolve_schemas_from(&mut sv, &path).unwrap(); + assert!(sv.get_unresolved_schemas().is_empty()); + assert!(sv.get_schema("http://example.com/local_target").is_some()); + + // ...and the unseeded entry point behaves exactly as before. + let (mut sv, _) = view_of("local_main.yaml"); + resolve_schemas(&mut sv).unwrap(); + assert!(sv.get_schema("http://example.com/local_target").is_some()); +} + +#[test] +fn reports_every_unresolvable_import_rather_than_only_the_first() { + // Two bad imports in one schema: the error must mention both, since a pass + // that returns on the first failure hides the rest of the work. + let (mut sv, path) = view_of("missing_imports.yaml"); + let err = resolve_schemas_from(&mut sv, &path).unwrap_err(); + assert!( + err.contains("definitely_not_here") && err.contains("also_not_here"), + "both failures must be reported, got: {err}" + ); +} diff --git a/src/schemaview/tests/unique_keys.rs b/src/schemaview/tests/unique_keys.rs new file mode 100644 index 0000000..94b102d --- /dev/null +++ b/src/schemaview/tests/unique_keys.rs @@ -0,0 +1,44 @@ +use linkml_schemaview::identifier::Identifier; +use linkml_schemaview::io::from_yaml; +use linkml_schemaview::schemaview::SchemaView; +use std::path::PathBuf; + +fn fixture() -> SchemaView { + let mut p = PathBuf::from(env!("CARGO_MANIFEST_DIR")); + p.push("tests/data/unique_keys.yaml"); + let schema = from_yaml(&p).unwrap(); + let mut sv = SchemaView::new(); + sv.add_schema(schema).unwrap(); + sv +} + +#[test] +fn unique_keys_merge_across_is_a_and_mixins_nearest_wins() { + let sv = fixture(); + let conv = sv.converter(); + let child = sv + .get_class(&Identifier::new("Child"), &conv) + .unwrap() + .expect("class not found"); + let uks = child.unique_keys(); + let names: Vec<&str> = uks.iter().map(|(n, _)| n.as_str()).collect(); + // name-sorted, merged from Base (by_code), MixinCls (by_tag), Child (shared_name override) + assert_eq!(names, vec!["by_code", "by_tag", "shared_name"]); + let shared = &uks.iter().find(|(n, _)| n == "shared_name").unwrap().1; + assert_eq!( + shared.unique_key_slots, + vec!["child_field".to_string()], + "the nearest declaration must win" + ); +} + +#[test] +fn class_without_unique_keys_yields_empty() { + let sv = fixture(); + let conv = sv.converter(); + let plain = sv + .get_class(&Identifier::new("Plain"), &conv) + .unwrap() + .expect("class not found"); + assert!(plain.unique_keys().is_empty()); +} diff --git a/src/tools/src/bin/linkml_convert.rs b/src/tools/src/bin/linkml_convert.rs index 4cd1b33..da33697 100644 --- a/src/tools/src/bin/linkml_convert.rs +++ b/src/tools/src/bin/linkml_convert.rs @@ -10,7 +10,7 @@ use linkml_runtime::{ #[cfg(feature = "ttl")] use linkml_schemaview::io::from_yaml; #[cfg(all(feature = "ttl", feature = "resolve"))] -use linkml_schemaview::resolve::resolve_schemas; +use linkml_schemaview::resolve::resolve_schemas_from; #[cfg(feature = "ttl")] use linkml_schemaview::schemaview::SchemaView; #[cfg(feature = "ttl")] @@ -99,7 +99,7 @@ fn main() -> Result<(), Box> { #[cfg(feature = "resolve")] { eprintln!("Resolving schemas..."); - resolve_schemas(&mut sv).map_err(|e| e.to_string())?; + resolve_schemas_from(&mut sv, &args.schema).map_err(|e| e.to_string())?; eprintln!("Schemas resolved"); } let conv = sv.converter(); diff --git a/src/tools/src/bin/linkml_diff.rs b/src/tools/src/bin/linkml_diff.rs index ec4a85b..6cdc3ab 100644 --- a/src/tools/src/bin/linkml_diff.rs +++ b/src/tools/src/bin/linkml_diff.rs @@ -2,7 +2,7 @@ use clap::Parser; use linkml_runtime::{diff, load_json_file, load_yaml_file, DiffOptions}; use linkml_schemaview::io::from_yaml; #[cfg(feature = "resolve")] -use linkml_schemaview::resolve::resolve_schemas; +use linkml_schemaview::resolve::resolve_schemas_from; use linkml_schemaview::schemaview::{ClassView, SchemaView}; use linkml_schemaview::Converter; use std::fs::File; @@ -61,7 +61,7 @@ fn main() -> Result<(), Box> { let mut sv = SchemaView::new(); sv.add_schema(schema.clone()).map_err(|e| e.to_string())?; #[cfg(feature = "resolve")] - resolve_schemas(&mut sv).map_err(|e| e.to_string())?; + resolve_schemas_from(&mut sv, &args.schema).map_err(|e| e.to_string())?; let conv = sv.converter(); let class_view = sv.get_tree_root_or(args.class.as_deref()).ok_or_else(|| { format!( diff --git a/src/tools/src/bin/linkml_patch.rs b/src/tools/src/bin/linkml_patch.rs index 3405c41..7a748ae 100644 --- a/src/tools/src/bin/linkml_patch.rs +++ b/src/tools/src/bin/linkml_patch.rs @@ -2,7 +2,7 @@ use clap::Parser; use linkml_runtime::{load_json_file, load_yaml_file, patch, Delta}; use linkml_schemaview::io::from_yaml; #[cfg(feature = "resolve")] -use linkml_schemaview::resolve::resolve_schemas; +use linkml_schemaview::resolve::resolve_schemas_from; use linkml_schemaview::schemaview::{ClassView, SchemaView}; use linkml_schemaview::Converter; use std::fs::File; @@ -11,8 +11,34 @@ use std::path::{Path, PathBuf}; use linkml_tools::validation_utils::report_validation_issues; +/// Exit code for a patch that applied some, but not all, of its deltas. +/// +/// Partial application is designed behaviour, not an error: a delta whose +/// target has drifted away is recorded and skipped so the rest of the batch +/// still lands. It is also not something a script should have to parse stderr +/// to notice, and the old builder-error `Err` at least gave it a non-zero +/// status. A code of its own restores the machine signal without claiming the +/// run failed. +/// +/// **3, not 2**: clap exits `2` on a usage error (an unknown flag, a missing +/// argument), which this tool does not get to choose. Claiming 2 would make a +/// typo'd flag indistinguishable from a partially applied patch — the one +/// reading a script could least afford to confuse. +const EXIT_PARTIAL: i32 = 3; + #[derive(Parser)] -#[command(name = "linkml-patch")] +#[command( + name = "linkml-patch", + about = "Apply a delta file to a LinkML instance document", + long_about = "Apply a delta file to a LinkML instance document. + +Exit codes: + 0 every delta applied + 1 hard failure: unreadable files, schema, parse or write errors + 2 argument or usage error (the command-line parser's convention) + 3 partial application: some deltas could not be applied, and their paths are + listed on stderr; the patched document is still written" +)] struct Args { /// LinkML schema YAML file schema: PathBuf, @@ -75,16 +101,48 @@ fn write_value( serde_yaml::to_writer(&mut writer, &json)?; } writer.write_all(b"\n")?; + // Explicit: the partial-application path leaves via `process::exit`, which + // runs no destructors. + writer.flush()?; Ok(()) } +/// Report the deltas the patch could not apply. +/// +/// `patch` never hard-errors on an unappliable delta: it records the delta's +/// path and applies the rest of the batch. Dropping the trace made that +/// invisible here — a patch that skipped half its deltas wrote a file that +/// looked clean. The lines go to stderr, so the patched document on stdout is +/// byte-identical to before for a fully applied patch; [`EXIT_PARTIAL`] carries +/// the same news to a caller that does not read prose. +fn report_failed_deltas(path: &Path, failed: &[Vec]) { + if failed.is_empty() { + return; + } + eprintln!( + "{} of the deltas in '{}' could not be applied; the rest were applied.", + failed.len(), + path.display() + ); + for delta_path in failed { + eprintln!( + " - {}", + if delta_path.is_empty() { + "".to_string() + } else { + delta_path.join(".") + } + ); + } +} + fn main() -> Result<(), Box> { let args = Args::parse(); let schema = from_yaml(&args.schema)?; let mut sv = SchemaView::new(); sv.add_schema(schema.clone()).map_err(|e| e.to_string())?; #[cfg(feature = "resolve")] - resolve_schemas(&mut sv).map_err(|e| e.to_string())?; + resolve_schemas_from(&mut sv, &args.schema).map_err(|e| e.to_string())?; let conv = sv.converter(); let class_view = sv.get_tree_root_or(args.class.as_deref()).ok_or_else(|| { format!( @@ -105,7 +163,7 @@ fn main() -> Result<(), Box> { } else { serde_yaml::from_str(&delta_text)? }; - let (patched, _trace) = patch( + let (patched, trace) = patch( &src, &deltas, linkml_runtime::diff::PatchOptions { @@ -113,6 +171,12 @@ fn main() -> Result<(), Box> { treat_missing_as_null: args.treat_missing_as_null, }, )?; + report_failed_deltas(&args.delta, &trace.failed); write_value(args.output.as_deref(), &patched)?; + if !trace.failed.is_empty() { + // The document is written first: a partial patch is a result, not a + // discarded run. + std::process::exit(EXIT_PARTIAL); + } Ok(()) } diff --git a/src/tools/src/bin/linkml_schema_validate.rs b/src/tools/src/bin/linkml_schema_validate.rs index 53aa457..558adae 100644 --- a/src/tools/src/bin/linkml_schema_validate.rs +++ b/src/tools/src/bin/linkml_schema_validate.rs @@ -1,7 +1,8 @@ use clap::{Parser, ValueEnum}; #[cfg(feature = "resolve")] -use linkml_schemaview::resolve::resolve_schemas; +use linkml_schemaview::resolve::resolve_schemas_from; use linkml_schemaview::{identifier::Identifier, io::from_yaml, schemaview::SchemaView, Converter}; +use linkml_tools::validation_utils::identity_warnings_json; use std::path::PathBuf; #[derive(Parser)] @@ -12,6 +13,11 @@ struct Args { /// Output format #[arg(long, value_enum, default_value_t = OutputFormat::Text)] output: OutputFormat, + /// Opt-in: warn for multivalued inlined slots whose element identity comes + /// from nowhere (positional, ambiguous deltas in multi-sourced use). + /// Warnings never change the exit code. + #[arg(long, default_value_t = false)] + lint_identity: bool, } #[derive(ValueEnum, Clone)] @@ -108,7 +114,7 @@ fn main() -> Result<(), Box> { let mut sv = SchemaView::new(); sv.add_schema(schema.clone()).map_err(|e| e.to_string())?; #[cfg(feature = "resolve")] - if let Err(e) = resolve_schemas(&mut sv) { + if let Err(e) = resolve_schemas_from(&mut sv, &args.schema) { eprintln!("{e}"); } let conv = sv.converter(); @@ -177,17 +183,64 @@ fn main() -> Result<(), Box> { })?; if errors.is_empty() { + // Opt-in identity lint. It runs against the same SchemaView the + // validation above used, so classes pulled in by `resolve_schemas` + // from `imports:` are linted too. Warnings only — the exit code is + // whatever the validation produced. + let identity_warnings = if args.lint_identity { + linkml_runtime::lint_element_identity(&sv) + } else { + Vec::new() + }; match args.output { - OutputFormat::Text => println!("schema valid"), + OutputFormat::Text => { + println!("schema valid"); + for w in &identity_warnings { + println!("warning[{}]: {}", w.subject.join("."), w.detail); + } + } + OutputFormat::Json if args.lint_identity => { + println!( + "{}", + serde_json::to_string_pretty(&serde_json::json!({ + "status": "valid", + "errors": errors, + "identity_warnings": identity_warnings_json(&identity_warnings), + "identity_lint_skipped": false, + "identity_lint_skipped_reason": serde_json::Value::Null, + }))? + ); + } OutputFormat::Json => println!("{}", serde_json::json!({"status":"valid"})), } Ok(()) } else { + // The lint is deliberately skipped when the schema does not validate: + // it asks "where does this slot's element identity come from?" of a + // schema graph that is known to be incomplete, so its answers would be + // wrong (an unresolved import turns a class range into "not a class"). + // Reporting the errors first is also the only useful output here. match args.output { OutputFormat::Text => { for e in &errors { println!("{e}"); } + if args.lint_identity { + println!("note: --lint-identity skipped: fix the schema errors above first"); + } + } + OutputFormat::Json if args.lint_identity => { + println!( + "{}", + serde_json::to_string_pretty(&serde_json::json!({ + "status": "invalid", + "errors": errors, + "identity_warnings": serde_json::Value::Null, + "identity_lint_skipped": true, + "identity_lint_skipped_reason": + "schema has errors; fix them and re-run", + }))? + ); } OutputFormat::Json => { println!("{}", serde_json::to_string_pretty(&errors)?); diff --git a/src/tools/src/bin/linkml_validate.rs b/src/tools/src/bin/linkml_validate.rs index bc3da77..6ce0b12 100644 --- a/src/tools/src/bin/linkml_validate.rs +++ b/src/tools/src/bin/linkml_validate.rs @@ -1,12 +1,13 @@ use clap::Parser; use linkml_runtime::{ - load_json_file, load_yaml_file, ValidationResult, ValidationSeverity, ValidationValue, + lint_instance_identity, load_json_file, load_yaml_file, ValidationResult, ValidationValue, }; use linkml_schemaview::identifier::Identifier; use linkml_schemaview::io::from_yaml; #[cfg(feature = "resolve")] -use linkml_schemaview::resolve::resolve_schemas; +use linkml_schemaview::resolve::resolve_schemas_from; use linkml_schemaview::schemaview::SchemaView; +use linkml_tools::validation_utils::{format_path, identity_warnings_json, severity_label}; use serde_json::json; use std::path::PathBuf; @@ -21,6 +22,11 @@ struct Args { /// Emit machine-readable JSON instead of human-readable text #[arg(long)] json: bool, + /// Opt-in: warn where this data defeats a declared element identity — a + /// list repeating one, or one addressed positionally because some element + /// leaves the identity slot empty. Warnings never change the exit code. + #[arg(long, default_value_t = false)] + lint_identity: bool, } fn main() -> Result<(), Box> { @@ -34,7 +40,7 @@ fn main() -> Result<(), Box> { ) .map_err(|e| e.to_string())?; #[cfg(feature = "resolve")] - resolve_schemas(&mut sv).map_err(|e| e.to_string())?; + resolve_schemas_from(&mut sv, &args.schema).map_err(|e| e.to_string())?; let conv = sv.converter(); let class_view = sv .get_class(&Identifier::new(&args.class), &conv) @@ -50,10 +56,38 @@ fn main() -> Result<(), Box> { } else { load_yaml_file(data_path, &sv, &class_view, &conv)? }; + // Validity is decided by ERRORS, not by diagnostics. Until the spec + // addendum the loader could only produce errors, so `issues.is_empty()` + // and "no errors" were one predicate; rules 2 and 5 made the loader emit + // warnings, and the two parted company. A document whose only finding is a + // warning is valid: it exits 0, and the warning is reported rather than + // being dressed up as the reason for a failure. + let is_valid = !load_result.has_errors(); + let instance = load_result.instance; let validation_issues = load_result.validation_issues; - let is_valid = validation_issues.is_empty(); + // Opt-in instance-identity lint, deliberately skipped when the data does + // not validate — the same stance `linkml-schema-validate` takes towards a + // schema with errors. The lint asks how a list's elements are addressed, + // and answering that over a tree whose loading already went wrong produces + // answers about the damage rather than about the data. Reporting the + // validation errors first is also the only useful output in that case. + // + // The gate is errors alone: a warning says the document is unusual, not + // that it failed to load, and the lint's answers over it are exactly as + // sound as over a silent one. + // + // Warnings never change the exit code: it stays whatever validation made it. + let identity_warnings = match (args.lint_identity, is_valid, &instance) { + (true, true, Some(value)) => lint_instance_identity(value), + _ => Vec::new(), + }; if args.json { - emit_json(is_valid, &validation_issues)?; + emit_json( + is_valid, + &validation_issues, + args.lint_identity, + &identity_warnings, + )?; if is_valid { Ok(()) } else { @@ -61,21 +95,55 @@ fn main() -> Result<(), Box> { } } else if is_valid { println!("valid"); + // A valid document may now carry diagnostics, all of them non-error. + for issue in &validation_issues { + print_issue(issue); + } + for w in &identity_warnings { + println!("warning[{}]: {}", w.subject.join("."), w.detail); + } Ok(()) } else { for issue in &validation_issues { - let location = if issue.subject.is_empty() { - "".to_string() - } else { - issue.subject.join(".") - }; - println!("{:?} at {}: {}", issue.problem_type, location, issue.detail); + print_issue(issue); + } + if args.lint_identity { + println!("note: --lint-identity skipped: fix the validation errors above first"); } std::process::exit(1); } } -fn emit_json(valid: bool, issues: &[ValidationResult]) -> Result<(), serde_json::Error> { +/// One validation issue, in text mode, wherever it appears. +/// +/// An error keeps the bare `Type at path: detail` line this binary has always +/// printed — so a document whose diagnostics are all errors prints exactly what +/// it always printed, which is every document that predates the spec addendum's +/// warnings. Anything that is *not* an error is named, because the alternative +/// is a warning that looks like the error it is listed beside: in the valid +/// branch it would be an error on a document that has none, and in the error +/// branch it would be one more thing the reader is told to fix before re-running. +fn print_issue(issue: &ValidationResult) { + let location = format_path(&issue.subject); + if issue.severity.is_error() { + println!("{:?} at {}: {}", issue.problem_type, location, issue.detail); + } else { + println!( + "{}: {:?} at {}: {}", + severity_label(&issue.severity), + issue.problem_type, + location, + issue.detail + ); + } +} + +fn emit_json( + valid: bool, + issues: &[ValidationResult], + lint_identity: bool, + identity_warnings: &[ValidationResult], +) -> Result<(), serde_json::Error> { let issues_json: Vec<_> = issues .iter() .map(|issue| { @@ -96,19 +164,31 @@ fn emit_json(valid: bool, issues: &[ValidationResult]) -> Result<(), serde_json: }) }) .collect(); - let output = json!({ - "valid": valid, - "issues": issues_json, - }); + // Without the flag the document is byte-identical to what it always was: + // the lint keys are absent, not null. A consumer that never opts in cannot + // tell this version from the previous one. + let output = if !lint_identity { + json!({ + "valid": valid, + "issues": issues_json, + }) + } else if valid { + json!({ + "valid": valid, + "issues": issues_json, + "identity_warnings": identity_warnings_json(identity_warnings), + "identity_lint_skipped": false, + "identity_lint_skipped_reason": serde_json::Value::Null, + }) + } else { + json!({ + "valid": valid, + "issues": issues_json, + "identity_warnings": serde_json::Value::Null, + "identity_lint_skipped": true, + "identity_lint_skipped_reason": "data has validation errors; fix them and re-run", + }) + }; println!("{}", serde_json::to_string_pretty(&output)?); Ok(()) } - -fn severity_label(severity: &ValidationSeverity) -> &'static str { - match severity { - ValidationSeverity::Fatal => "fatal", - ValidationSeverity::Error => "error", - ValidationSeverity::Warning => "warning", - ValidationSeverity::Info => "info", - } -} diff --git a/src/tools/src/lib.rs b/src/tools/src/lib.rs index 3784200..a1cd79e 100644 --- a/src/tools/src/lib.rs +++ b/src/tools/src/lib.rs @@ -1,9 +1,12 @@ -use linkml_runtime::{InstancePath, ValidationSeverity}; - /// Helpers shared by CLI binaries. +/// +/// Everything here is a *rendering* decision that more than one binary makes, +/// and that the binaries must make identically: two CLIs printing one lint two +/// ways is a difference a reader has to explain to themselves. The rule for +/// what belongs here is that — shared vocabulary — not "utility". pub mod validation_utils { - use super::{format_path, severity_label}; - use linkml_runtime::ValidationResult; + use linkml_runtime::{InstancePath, ValidationResult, ValidationSeverity}; + use serde_json::json; use std::path::Path; /// Print validation diagnostics to stderr but keep execution going. @@ -26,21 +29,49 @@ pub mod validation_utils { ); } } -} -fn severity_label(severity: &ValidationSeverity) -> &'static str { - match severity { - ValidationSeverity::Fatal => "fatal", - ValidationSeverity::Error => "error", - ValidationSeverity::Warning => "warning", - ValidationSeverity::Info => "info", + /// The machine-readable name of a severity, as every CLI spells it. + pub fn severity_label(severity: &ValidationSeverity) -> &'static str { + match severity { + ValidationSeverity::Fatal => "fatal", + ValidationSeverity::Error => "error", + ValidationSeverity::Warning => "warning", + ValidationSeverity::Info => "info", + } + } + + /// An instance path as the CLIs display it; the empty path is the root. + pub fn format_path(path: &InstancePath) -> String { + if path.is_empty() { + "".to_string() + } else { + path.join(".") + } } -} -fn format_path(path: &InstancePath) -> String { - if path.is_empty() { - "".to_string() - } else { - path.join(".") + /// The identity-lint warnings, in the one shape both `linkml-validate` and + /// `linkml-schema-validate` emit them: the two CLIs report one lint the + /// same way, so a consumer reading either document reads the same keys. + /// + /// Uses [`linkml_runtime::ValidationProblemType::label`] rather than the + /// binaries' older `Debug` spelling for validation issues: the label is the + /// shared machine-readable name (the Python binding reports the same one), + /// and a variant rename cannot change it behind the CLIs' back. The + /// separate `issues` shape of `linkml-validate` keeps its `Debug` spelling + /// — that one is a published contract of that binary. + pub fn identity_warnings_json(warnings: &[ValidationResult]) -> serde_json::Value { + serde_json::Value::Array( + warnings + .iter() + .map(|w| { + json!({ + "type": w.problem_type.label(), + "severity": severity_label(&w.severity), + "subject": w.subject, + "detail": w.detail, + }) + }) + .collect(), + ) } } diff --git a/src/tools/tests/data/validate_clean.json b/src/tools/tests/data/validate_clean.json new file mode 100644 index 0000000..8b3ce71 --- /dev/null +++ b/src/tools/tests/data/validate_clean.json @@ -0,0 +1,9 @@ +{ + "people": { + "p1": {"pid": "p1", "name": "Ann"} + }, + "readings": [ + {"code": "c1", "value": "1"}, + {"code": "c2", "value": "2"} + ] +} diff --git a/src/tools/tests/data/validate_errors.json b/src/tools/tests/data/validate_errors.json new file mode 100644 index 0000000..b47bb8c --- /dev/null +++ b/src/tools/tests/data/validate_errors.json @@ -0,0 +1,9 @@ +{ + "people": { + "p1": {"pid": "p1", "name": "Ann", "not_a_slot": "x"} + }, + "readings": [ + {"code": "c1", "value": "1"}, + {"code": "c1", "value": "2"} + ] +} diff --git a/src/tools/tests/data/validate_mixed.json b/src/tools/tests/data/validate_mixed.json new file mode 100644 index 0000000..fff1635 --- /dev/null +++ b/src/tools/tests/data/validate_mixed.json @@ -0,0 +1,8 @@ +{ + "people": { + "p1": {"pid": "p2", "name": "Ann"} + }, + "readings": [ + {"code": "c1", "not_a_slot": "x"} + ] +} diff --git a/src/tools/tests/data/validate_warning_only.json b/src/tools/tests/data/validate_warning_only.json new file mode 100644 index 0000000..bbc57de --- /dev/null +++ b/src/tools/tests/data/validate_warning_only.json @@ -0,0 +1,9 @@ +{ + "people": { + "p1": {"pid": "p2", "name": "Ann"} + }, + "readings": [ + {"code": "c1", "value": "1"}, + {"code": "c1", "value": "2"} + ] +} diff --git a/src/tools/tests/data/validate_warning_only.yaml b/src/tools/tests/data/validate_warning_only.yaml new file mode 100644 index 0000000..63495ac --- /dev/null +++ b/src/tools/tests/data/validate_warning_only.yaml @@ -0,0 +1,53 @@ +id: https://w3id.org/linkml/examples/validate_warning_only +name: validate_warning_only +description: |- + Fixture for the `linkml-validate` warning/error contract (spec addendum + rules 5 and 6, and the exit-code fix that followed them). + + Deliberately lives under `src/tools/tests/data` rather than + `src/runtime/tests/data`: the differential harness treats every file of the + runtime fixture directory as corpus, so a fixture added there to pin a CLI + contract would move harness output that has nothing to do with the change. + + Two shapes, so one document can carry a load-time warning *and* a finding + from the opt-in instance lint at the same time: + + * `Container.people` — an ordinary keyed class in inlined-dict form. A + payload key that contradicts the dict key is a load-time **warning** + (rule 5); the document is still valid. + * `Container.readings` — a list whose element class declares its identity + through `unique_keys`. Repeating a value there is what + `--lint-identity` reports. +license: https://creativecommons.org/publicdomain/zero/1.0/ +imports: + - linkml:types +prefixes: + vwo: https://w3id.org/linkml/examples/validate_warning_only/ + linkml: https://w3id.org/linkml/ +default_prefix: vwo +default_range: string + +classes: + Container: + attributes: + people: + range: Person + multivalued: true + inlined: true + readings: + range: Reading + multivalued: true + inlined: true + + Person: + attributes: + pid: {range: string, key: true, required: true} + name: {range: string} + + Reading: + unique_keys: + main: + unique_key_slots: [code] + attributes: + code: {range: string} + value: {range: string} diff --git a/src/tools/tests/diff_cli.rs b/src/tools/tests/diff_cli.rs index 765ae8a..a3c139b 100644 --- a/src/tools/tests/diff_cli.rs +++ b/src/tools/tests/diff_cli.rs @@ -45,3 +45,108 @@ fn cli_diff_and_patch_personinfo() { serde_yaml::from_str(&std::fs::read_to_string(&tgt).unwrap()).unwrap(); assert_eq!(out_data, tgt_data); } + +/// `patch` records an unappliable delta instead of voiding the batch, so the +/// CLI has to say which delta was dropped — otherwise a half-applied patch +/// writes a file that looks clean — and has to signal it in the exit status, +/// which is `3`: partial application, not a failed run, and not `2`, which the +/// argument parser has already claimed for usage errors. +/// +/// Both ways a delta can be unappliable are exercised: an address that resolves +/// to nothing (soft before rule 4 as well) and a payload that cannot be BUILT +/// at the location it addresses — `no_such_slot` is declared by no class, so +/// the builder has no slot to box the scalar against. That second one used to +/// propagate as an `Err` and void the whole batch; this test is what pins rule +/// 4's soft path all the way out to the CLI surface. +#[test] +fn cli_patch_reports_failed_delta_paths_and_exits_3() { + let schema = info_path("personinfo.yaml"); + let src = info_path("example_personinfo_data.yaml"); + let tmp = tempfile::tempdir().unwrap(); + let delta = tmp.path().join("delta.json"); + let out = tmp.path().join("out.yaml"); + std::fs::write( + &delta, + r#"[{"path": ["objects", "P:404", "name"], "op": "update", + "old": "nobody", "new": "somebody"}, + {"path": ["objects", "P:001", "no_such_slot"], "op": "update", + "old": "x", "new": "y"}, + {"path": ["objects", "P:001", "name"], "op": "update", + "old": "fred bloggs", "new": "fred b."}]"#, + ) + .unwrap(); + + let mut cmd = Command::cargo_bin("linkml-patch").unwrap(); + cmd.arg(&schema) + .arg("-c") + .arg("Container") + .arg(&src) + .arg(&delta) + .arg("-o") + .arg(&out); + let assert = cmd.assert().code(3); + let stderr = String::from_utf8(assert.get_output().stderr.clone()).unwrap(); + assert!( + stderr.contains("could not be applied") + && stderr.contains("objects.P:404.name") + && stderr.contains("objects.P:001.no_such_slot"), + "both dropped deltas must be named: {stderr}" + ); + assert!( + !stderr.contains("objects.P:001.name\n"), + "the applied delta must not be reported: {stderr}" + ); + let patched = std::fs::read_to_string(&out).unwrap(); + assert!( + patched.contains("fred b."), + "the good delta must still land: {patched}" + ); + assert!( + !patched.contains("no_such_slot"), + "the unbuildable delta must leave no trace: {patched}" + ); +} + +/// The other half of the exit contract: a patch that applies every delta exits +/// `0`, and says nothing on stderr. +#[test] +fn cli_patch_exits_0_when_every_delta_applies() { + let schema = info_path("personinfo.yaml"); + let src = info_path("example_personinfo_data.yaml"); + let tmp = tempfile::tempdir().unwrap(); + let delta = tmp.path().join("delta.json"); + let out = tmp.path().join("out.yaml"); + std::fs::write( + &delta, + r#"[{"path": ["objects", "P:001", "name"], "op": "update", + "old": "fred bloggs", "new": "fred b."}]"#, + ) + .unwrap(); + + let mut cmd = Command::cargo_bin("linkml-patch").unwrap(); + cmd.arg(&schema) + .arg("-c") + .arg("Container") + .arg(&src) + .arg(&delta) + .arg("-o") + .arg(&out); + let assert = cmd.assert().code(0); + let stderr = String::from_utf8(assert.get_output().stderr.clone()).unwrap(); + assert!( + !stderr.contains("could not be applied"), + "a clean patch reports nothing: {stderr}" + ); +} + +/// A usage error is `2`, the argument parser's own code — which is why partial +/// application is `3`. A script that cannot tell a typo'd flag from a +/// half-applied patch has no contract at all. +#[test] +fn cli_patch_usage_error_exits_2_not_the_partial_code() { + Command::cargo_bin("linkml-patch") + .unwrap() + .arg("--no-such-flag") + .assert() + .code(2); +} diff --git a/src/tools/tests/schema_validate.rs b/src/tools/tests/schema_validate.rs index 1d2dee2..3c34b57 100644 --- a/src/tools/tests/schema_validate.rs +++ b/src/tools/tests/schema_validate.rs @@ -27,3 +27,78 @@ fn person_schema_missing_slot() { cmd.arg(&schema); cmd.assert().success(); } + +/// The JSON `--lint-identity` payload, and the process's success. +fn lint_identity_json(schema: &str) -> (bool, serde_json::Value) { + let mut cmd = Command::cargo_bin("linkml-schema-validate").unwrap(); + cmd.arg(data_path(schema)) + .arg("--lint-identity") + .arg("--output") + .arg("json"); + let out = cmd.output().unwrap(); + let stdout = String::from_utf8(out.stdout).unwrap(); + let parsed = + serde_json::from_str(&stdout).unwrap_or_else(|e| panic!("not JSON ({e}): {stdout}")); + (out.status.success(), parsed) +} + +/// The five keys the `--lint-identity --output json` contract promises, on +/// every outcome. Consumers branch on `identity_lint_skipped`, so a key that +/// appears only on one path is a key they cannot rely on. +const LINT_JSON_KEYS: [&str; 5] = [ + "errors", + "identity_lint_skipped", + "identity_lint_skipped_reason", + "identity_warnings", + "status", +]; + +fn keys(v: &serde_json::Value) -> Vec { + v.as_object() + .expect("a JSON object") + .keys() + .cloned() + .collect() +} + +#[test] +fn lint_identity_json_shape_when_the_schema_is_valid() { + let (ok, v) = lint_identity_json("identity.yaml"); + assert!(ok, "the identity fixture must validate: {v}"); + assert_eq!(keys(&v), LINT_JSON_KEYS); + assert_eq!(v["status"], "valid"); + assert_eq!(v["errors"], serde_json::json!([])); + assert_eq!(v["identity_lint_skipped"], false); + assert!(v["identity_lint_skipped_reason"].is_null()); + + let warnings = v["identity_warnings"].as_array().expect("an array"); + assert!(!warnings.is_empty(), "the fixture has flagged slots: {v}"); + for w in warnings { + assert_eq!(keys(w), ["detail", "severity", "subject", "type"]); + // The machine-readable spelling, shared with the Python binding's + // `problem_type`. `Debug` formatting would make a variant rename a + // silent change of this contract, and would spell it differently from + // the other binding for the same value. + assert_eq!( + w["type"], "ambiguous_element_identity", + "the JSON type must be the shared snake_case label: {w}" + ); + assert_eq!(w["severity"], "warning"); + assert!(w["subject"].is_array()); + assert!(w["detail"].as_str().is_some_and(|d| !d.is_empty())); + } +} + +#[test] +fn lint_identity_json_shape_when_the_lint_is_skipped() { + let (ok, v) = lint_identity_json("invalid_schema.yaml"); + assert!(!ok, "an invalid schema must still exit non-zero: {v}"); + assert_eq!(keys(&v), LINT_JSON_KEYS); + assert_eq!(v["status"], "invalid"); + assert!(!v["errors"].as_array().expect("an array").is_empty()); + assert!(v["identity_warnings"].is_null()); + assert_eq!(v["identity_lint_skipped"], true); + assert!(v["identity_lint_skipped_reason"] + .as_str() + .is_some_and(|r| !r.is_empty())); +} diff --git a/src/tools/tests/validate_cli.rs b/src/tools/tests/validate_cli.rs new file mode 100644 index 0000000..8466910 --- /dev/null +++ b/src/tools/tests/validate_cli.rs @@ -0,0 +1,183 @@ +//! `linkml-validate`'s validity contract: **errors decide validity, warnings +//! never do**. +//! +//! The loader gained warning-severity diagnostics with the spec addendum +//! (designator canonicalisation, rule 5's dict-key reconciliation) — before it, +//! every diagnostic it could produce was an error, so "no diagnostics" and "no +//! errors" were the same predicate and the CLI could use either. They are not +//! the same predicate any more: a document whose only finding is a warning was +//! reported as invalid, exited 1, printed the warning in the error list, and +//! had `--lint-identity` suppressed with "fix the validation errors above +//! first" — an error message about a document that has none. +//! +//! Pinned here, because none of it is visible from a library test: +//! +//! * a warning-only document is **valid**: exit 0, `valid: true`, and the +//! warning is still reported (text mode marks its severity, JSON mode carries +//! it in `issues` with `severity: "warning"` as it always did); +//! * `--lint-identity` **runs** on such a document — the lint is gated on +//! errors, not on diagnostics; +//! * an **error** still skips the lint, with the reason stated in both modes, +//! and a warning listed beside errors is still marked as a warning; +//! * a warning-free document invoked without the flag produces byte-for-byte +//! the output it always did. + +use assert_cmd::Command; + +const CLASS: &str = "Container"; + +fn data_path(name: &str) -> std::path::PathBuf { + let mut p = std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR")); + p.push("tests/data"); + p.push(name); + p +} + +/// Run `linkml-validate` against the fixture schema, returning (exit code, stdout). +fn validate(data: &str, extra: &[&str]) -> (i32, String) { + let mut cmd = Command::cargo_bin("linkml-validate").unwrap(); + cmd.arg(data_path("validate_warning_only.yaml")) + .arg(CLASS) + .arg(data_path(data)) + .args(extra); + let out = cmd.output().unwrap(); + ( + out.status.code().unwrap_or(-1), + String::from_utf8(out.stdout).unwrap(), + ) +} + +fn validate_json(data: &str, extra: &[&str]) -> (i32, serde_json::Value) { + let mut args = vec!["--json"]; + args.extend_from_slice(extra); + let (code, out) = validate(data, &args); + (code, serde_json::from_str(&out).expect("JSON on stdout")) +} + +// --------------------------------------------------------------------------- +// A warning is not an error +// --------------------------------------------------------------------------- + +/// Text mode. The document's only load diagnostic is rule 5's dict-key +/// divergence warning, so the document is valid — and the warning is still +/// printed, marked with its severity so it cannot be read as an error. +#[test] +fn warning_only_document_is_valid_in_text_mode() { + let (code, out) = validate("validate_warning_only.json", &[]); + assert_eq!(code, 0, "a warning must not fail the run: {out}"); + assert!(out.starts_with("valid\n"), "{out}"); + assert!( + out.contains("warning: SlotRangeViolation at people.p1.pid:"), + "the warning is reported, with its severity: {out}" + ); + assert!( + !out.contains("fix the validation errors above first"), + "there are no errors to fix: {out}" + ); +} + +/// JSON mode. `valid` reflects errors only; the warning stays in `issues`, +/// where it always carried `severity: "warning"`. +#[test] +fn warning_only_document_is_valid_in_json_mode() { + let (code, doc) = validate_json("validate_warning_only.json", &[]); + assert_eq!(code, 0); + assert_eq!(doc["valid"], serde_json::json!(true), "{doc:#}"); + let issues = doc["issues"].as_array().expect("issues array"); + assert_eq!(issues.len(), 1, "{doc:#}"); + assert_eq!(issues[0]["severity"], serde_json::json!("warning")); +} + +/// The lint is gated on *errors*. This document has a warning and a genuine +/// duplicate identity, and the flag must report the duplicate. +#[test] +fn lint_runs_on_a_warning_only_document() { + let (code, out) = validate("validate_warning_only.json", &["--lint-identity"]); + assert_eq!(code, 0, "{out}"); + assert!( + out.contains("warning[readings]:") && out.contains("share the declared identity 'c1'"), + "the instance lint ran and reported: {out}" + ); + + let (code, doc) = validate_json("validate_warning_only.json", &["--lint-identity"]); + assert_eq!(code, 0); + assert_eq!(doc["identity_lint_skipped"], serde_json::json!(false)); + assert_eq!(doc["identity_lint_skipped_reason"], serde_json::Value::Null); + let warnings = doc["identity_warnings"].as_array().expect("array"); + assert_eq!(warnings.len(), 1, "{doc:#}"); + assert_eq!(warnings[0]["subject"], serde_json::json!(["readings"])); +} + +// --------------------------------------------------------------------------- +// An error still is one +// --------------------------------------------------------------------------- + +/// Same duplicate identity, plus an error. The lint is skipped and says so — +/// its answers over a tree whose loading went wrong would be about the damage. +#[test] +fn errors_skip_the_lint_and_say_so() { + let (code, out) = validate("validate_errors.json", &["--lint-identity"]); + assert_eq!(code, 1, "{out}"); + assert!( + out.contains("note: --lint-identity skipped: fix the validation errors above first"), + "{out}" + ); + assert!( + !out.contains("share the declared identity"), + "no lint output when the lint was skipped: {out}" + ); + + let (code, doc) = validate_json("validate_errors.json", &["--lint-identity"]); + assert_eq!(code, 1); + assert_eq!(doc["valid"], serde_json::json!(false), "{doc:#}"); + assert_eq!(doc["identity_lint_skipped"], serde_json::json!(true)); + assert!( + doc["identity_lint_skipped_reason"] + .as_str() + .unwrap_or_default() + .contains("error"), + "the reason names errors, since errors are what gates the lint: {doc:#}" + ); + assert_eq!(doc["identity_warnings"], serde_json::Value::Null); +} + +/// A document with both. The error decides validity, and the two lines are +/// told apart: the error keeps the bare shape this binary has always printed +/// for one, the warning says what it is. Anything else asks the reader to fix +/// "errors" that include a warning — the same misreading the exit code used to +/// force, one line further down. +#[test] +fn a_warning_beside_an_error_is_still_marked_as_a_warning() { + let (code, out) = validate("validate_mixed.json", &[]); + assert_eq!(code, 1, "{out}"); + assert!( + out.contains("warning: SlotRangeViolation at people.p1.pid:"), + "the warning names its severity: {out}" + ); + assert!( + out.contains("UndeclaredSlot at readings.0.not_a_slot:") + && !out.contains("error: UndeclaredSlot"), + "the error keeps the published unmarked shape: {out}" + ); +} + +// --------------------------------------------------------------------------- +// Nothing moved for a document with no diagnostics +// --------------------------------------------------------------------------- + +/// Byte-identity: a warning-free document, no flag. Every document that +/// predates the addendum's warnings is this case, which is what makes the +/// change above a fix rather than an output break. +#[test] +fn warning_free_document_without_the_flag_is_unchanged() { + let (code, out) = validate("validate_clean.json", &[]); + assert_eq!(code, 0); + assert_eq!(out, "valid\n"); + + let (code, out) = validate("validate_clean.json", &["--json"]); + assert_eq!(code, 0); + assert_eq!( + out, "{\n \"issues\": [],\n \"valid\": true\n}\n", + "the no-flag JSON document keeps exactly its two keys" + ); +}