Skip to content

fix(schemaview): resolve default_curi_maps and builtin linkml: import prefixes - #106

Open
ABorakati wants to merge 1 commit into
Kapernikov:mainfrom
ABorakati:fix/default-curi-maps-and-builtin-import-prefixes
Open

fix(schemaview): resolve default_curi_maps and builtin linkml: import prefixes#106
ABorakati wants to merge 1 commit into
Kapernikov:mainfrom
ABorakati:fix/default-curi-maps-and-builtin-import-prefixes

Conversation

@ABorakati

@ABorakati ABorakati commented Jul 2, 2026

Copy link
Copy Markdown

EDIT: FYI full disclosure - this is vibe coded with Claude Sonnet 5- I didn't mean to push yet*, but its fine
The code looks good, I have tried to stay faithful to the upstream python (which I do know, but I am in the process of learning rust, so can't guarantee..)
Feel free to reject/ suggest revisions / whatever

Fixes #105.

What

converter_from_schemas (identifier.rs) previously only registered prefixes declared inline under a schema's prefixes:, plus a hardcoded 3-entry fallback (rdfs/rdf/dcterms). It never read default_curi_maps, and prefixes from builtin linkml: imports (linkml:types, linkml:mappings, linkml:extensions, linkml:annotations, linkml:units) were only reachable via the network-only resolve feature — which never runs automatically during add_schema's own indexing step, the exact point where CurieError gets raised. A schema relying on either mechanism to resolve a prefix like schema:/owl: without declaring it inline (a common pattern — LinkML's own metamodel does this) failed add_schema outright.

linkml_runtime.SchemaView/SchemaLoader resolve both, with "explicit always wins" precedence (Namespaces.add_prefixmap's k not in self rule).

The fix

Added builtin_prefix_contributions, merging in — synchronously, no network access, no dependency on imports already being loaded as separate schemas:

  • default_curi_maps entries resolved against a small bundled registry (semweb_context today; trivially extensible via one match arm — an unrecognised name is left unresolved exactly as before, so this is purely additive)
  • prefixes of the 5 builtin linkml: schemas (matching both the CURIE and fully-expanded-URI import forms), transcribed verbatim from linkml_model's actual schema files. Uses the same identity list resolve::get_uri_for_id already has for full-content network resolution — this PR only ever contributes prefixes, so it doesn't duplicate or conflict with that mechanism, which still owns resolving imported classes/slots.

Merge precedence matches Namespaces.add_prefixmap exactly: explicit prefixes: always win.

A bug this surfaced along the way

Building the merged table required grouping same-URI prefixes as Converter::Record synonyms (the way the pre-existing explicit-prefix path already does), rather than adding each builtin prefix independently via add_prefix. semweb_context has both dc and dcterms mapping to the same URI (http://purl.org/dc/terms/), and Converter::add_record rejects a second record for an already-claimed URI — so a flat per-prefix merge silently drops the second prefix. This was caught by the existing io::tests::test_resolve_schemas unit test (loads the real LinkML metamodel schema), which started failing with CurieError(NotFound("dcterms")) during development until the merge was restructured to group by URI first.

Tests

Added src/schemaview/tests/default_curi_maps.rs:

  • a schema resolving schema: via imports: [linkml:types] alone (the reported bug's exact shape)
  • a schema resolving owl: via default_curi_maps: [semweb_context] alone
  • an explicit-prefix-wins precedence test
  • an expanded-URI-form import test (https://w3id.org/linkml/types vs. linkml:types)
  • a control test proving a plain schema using neither mechanism is unaffected (no behaviour change for the common case)

Verification

  • cargo test --workspace: all passing, 0 failures
  • cargo clippy -p schemaview --all-targets: clean (no new warnings; pre-existing warnings are all in the generated linkml_meta crate, untouched by this change)
  • Confirmed against a real linkml-runtime==1.11.1 / linkml==1.11.1 install that the bundled prefix tables match linkml_model's actual schema files and prefixcommons' semweb_context.jsonld byte-for-byte

… prefixes

converter_from_schemas (identifier.rs) only ever registered prefixes
declared inline under a schema's `prefixes:`, plus a hardcoded 3-entry
fallback (rdfs/rdf/dcterms). It never read `default_curi_maps` (parsed
correctly onto SchemaDefinition, but otherwise unused), and prefixes from
builtin `linkml:` imports (linkml:types, linkml:mappings, linkml:extensions,
linkml:annotations, linkml:units — the schemas bundled with the LinkML
language itself) were only reachable via the network-only `resolve`
feature, which never runs automatically during add_schema's own indexing
step. A schema relying on either mechanism to resolve a prefix like
`schema:`/`owl:` without declaring it inline (a common pattern — LinkML's
own metamodel does this) failed `add_schema` outright with
CurieError(NotFound(..)) the moment it indexed a class/slot using that
prefix.

linkml_runtime.SchemaView/SchemaLoader resolve both: default_curi_maps via
Namespaces.add_prefixmap (looked up against the prefixcommons/prefixmaps
registries), and imported schemas' prefixes via recursively loading each
import and merging its `prefixes:` in — both with "explicit always wins"
precedence (`k not in self`).

Added builtin_prefix_contributions, merging in:
  - default_curi_maps entries resolved against a small bundled registry
    (semweb_context today; trivially extensible — an unrecognised name is
    left unresolved exactly as before)
  - prefixes of the 5 builtin linkml: schemas (matching both the CURIE and
    fully-expanded URI import forms), transcribed verbatim from
    linkml_model's actual schema files (same identity list resolve.rs
    already uses for full-content network resolution — this only ever
    contributes prefixes, never duplicating or conflicting with that
    mechanism)
without overriding anything already registered, mirroring
Namespaces.add_prefixmap's precedence exactly. Fixed synchronously, no
network access, no dependency on imports having already been loaded as
separate schemas.

Building the merged prefix table required grouping same-URI prefixes as
Converter::Record synonyms (as the existing explicit-prefix path already
did) rather than adding them independently: semweb_context has both `dc`
and `dcterms` mapping to the same URI, and Converter::add_record rejects a
second record for an already-claimed URI, silently dropping the second
prefix if added as an independent record (caught by the existing
`io::tests::test_resolve_schemas` unit test, which loads the real LinkML
metamodel schema and started failing with
CurieError(NotFound("dcterms")) until this was fixed).

Added src/schemaview/tests/default_curi_maps.rs: a schema resolving
`schema:` via `imports: [linkml:types]` alone (the reported bug's exact
shape), a schema resolving `owl:` via `default_curi_maps: [semweb_context]`
alone, an explicit-prefix-wins precedence test, an expanded-URI-form import
test, and a control test proving a plain schema with neither mechanism is
unaffected.

Verified: cargo test --workspace, 0 failures.
@kervel

kervel commented Aug 13, 2026

Copy link
Copy Markdown

hey sorry for late reply, i was offline. looking at it now

@kervel

kervel commented Aug 13, 2026

Copy link
Copy Markdown

Thanks for this — the direction is right, and the research behind the bundled tables is careful. I checked all six transcribed tables against prefixcommons/registry/semweb_context.jsonld and the linkml_model schema files locally and they're byte-accurate, including the surprising dc -> http://purl.org/dc/terms/. cargo test -p schemaview, cargo test -p linkml_runtime and a --no-default-features build all pass.

The tables are fine; it's the merge logic that needs another look. Three things I'd want fixed before this goes in.

1. Two prefixes point at the same URI, and only one of them survives as the name we write

dc and dcterms both map to http://purl.org/dc/terms/. The merge keeps the first one it sees as the canonical name and demotes the rest to aliases — and dc is listed first. It also means the existing add_missing_prefix("dcterms", …) at identifier.rs:405 goes quiet, since dcterms is now a known alias.

Example, on meta.yaml:

main:    compress("http://purl.org/dc/terms/title") -> dcterms:title
pr-106:  compress("http://purl.org/dc/terms/title") -> dc:title

The Turtle writer builds its @prefix block from those, so every semweb_context schema we serialize switches to dc:meta.yaml, mappings.yaml, personinfo.yaml. Nothing in the suite catches it. Dropping dc from the table, or putting dcterms first, fixes it.

2. Two prefixes point at the same URI again, this time from two different tables — so which one wins is random

semweb_context calls http://www.geneontology.org/formats/oboInOwl#oboInOwl, and the linkml:mappings table calls it OIO. First one visited wins, and SchemaView::converter() iterates a HashMap, which Rust reseeds per process.

Example, same two schemas, only the visit order differs:

order [a, b] -> oboInOwl:foo
order [b, a] -> OIO:foo

So identical input can serialize differently on consecutive runs. Sorting the contributions before merging, and only adding a builtin prefix as an alias when the URI already has an owner, makes it deterministic.

3. The builtin schemas import each other, and we're only reading one level deep

The five builtins aren't independent — extensions pulls in types, annotations pulls in types and extensions, mappings pulls in types and declares semweb_context itself, and units pulls in all four. Python walks that chain and merges everything it finds.

Here we only take each entry's own prefixes: block. So a schema with imports: [linkml:units] that uses xsd: (or schema:, skos:, OIO:, IAO:) still fails at add_schema with CurieError(NotFound("xsd")) — the same failure #105 is about. In practice only linkml:types is actually fixed; the other four still fall over.

The new tests pass because meta.yaml lists all five imports explicitly and never has to follow the chain. A test importing just linkml:units would show it. Fix is to fold each table's transitive imports into it, plus semweb_context for mappings.

Smaller things

  • identifier.rs:251 — the comment says an unrecognised curie map "is simply left unresolved", but per your own test-file docstring it's a hard CurieError(NotFound) on add_schema. So default_curi_maps: [obo_context] (or monarch_context, idot_context, go_context) still fails to load. Worth correcting so the next reader doesn't assume a soft fallback.
  • identifier.rs:226 — "no behaviour change for schemas that don't use these builtins" doesn't hold inside a shared SchemaView: one converter spans all loaded schemas, so prefixes contributed by schema A change schema B's compress output. plain_schema_without_builtins_is_unaffected only builds a single-schema converter, so it doesn't cover that.
  • identifier.rs:288 — the literal import matching diverges from resolve::get_uri_for_id, which sees imports after curie2uri expansion. A schema declaring lml: https://w3id.org/linkml/ with imports: [lml:types] gets full content resolution but no prefix contribution — the pre-PR failure. Expanding the import through curie2uri before matching would make the two mechanisms agree.

@kervel

kervel commented Aug 13, 2026

Copy link
Copy Markdown

AI assisted comment above. i think the main things is duplicate prefixes (do we need them ?) and lack of recursion in resolving

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

converter_from_schemas ignores default_curi_maps; builtin linkml: import prefixes only reachable via network-only resolve feature

2 participants