fix(turtle): make skolem IRIs collision-free and deterministic - #114
Merged
Conversation
Skolem IRIs were built from the child's key or its position alone, with the
slot it hangs off left out of the path. Two keyless objects under different
slots of the same parent therefore got the same IRI and were merged into a
single node:
sk:r1 sk:homes <.../r1/0> .
<.../r1/0> sk:street "home street" .
sk:r1 sk:works <.../r1/0> . # same subject
<.../r1/0> sk:street "work street" .
Objects with neither a key nor a position fell back to a `gen{N}` global
counter walked in `HashMap` iteration order, so the same input produced
different IRIs on different runs. Inlined mappings were worse still: their
members were numbered by enumeration order over a `HashMap`, discarding the
mapping key that already identified them.
Put the slot in the path and drop the counter. A skolem IRI is now a pure
function of the path walked to reach the node:
<parent>/<slot> single-valued slot
<parent>/<slot>/<key> keyed member (mapping key or key slot)
<parent>/<slot>/<index> keyless list member
Path segments keep the RFC 3986 unreserved set intact rather than escaping
every non-alphanumeric, so snake_case slot names and keys stay readable;
characters that cannot appear in a segment are still percent-encoded.
Not addressed: write_turtle's top-level `Mapping` arm numbers members the
same way, but `LinkMLInstance::Mapping` carries a mandatory slot and no
loader produces one at the document root, so the path is unreachable and
left untested rather than changed blind.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug
Skolem IRIs are built from the child's key or list position, but not from the slot the child hangs off. Anything keyless at the same position under a different slot collides.
Input —
src/runtime/tests/data/skolem_data.yaml, an instance ofPerson(shown as JSON):{ "id": "sktest:p1", "home_addresses": [ { "street": "home street" } ], "work_addresses": [ { "street": "work street" } ], "primary_contact": { "phone": "555-0001" }, "backup_contact": { "phone": "555-0002" }, "accounts": { "savings_2024": { "balance": "100" }, "odd label/with slash": { "balance": "200" } } }Six distinct child objects.
--skolemonmainemits four subjects, because three of them land onp1/0:<.../p1/0>is simultaneously the savings account, the home address and the work address — one node typed as bothAccountandAddress, carryingbalance "100",street "home street"andstreet "work street". Silent data corruption.Three separate defects are visible above:
home_addresses[0],work_addresses[0]and the first account all reduce top1/0.gen{N}is a global counter walked inHashMapiteration order, used for objects with neither key nor position (primary_contact,backup_contact). The same input file gives different IRIs on different runs — six runs of one fixture alternated betweenbackup_contact → gen1andbackup_contact → gen2.accountsis keyed bysavings_2024/odd label/with slash, but members were numbered0/1by enumeration order over aHashMap— discarding the key that already identified them, and picking a different one each run.The fix
Put the slot in the path, drop the counter, use the mapping key where there is one. A skolem IRI becomes a pure function of the path walked to reach the node:
<parent>/<slot>key: trueslot)<parent>/<slot>/<key><parent>/<slot>/<index>Same input, after:
Six inputs, six subjects, every IRI readable back to the path that produced it. Triple order still varies run to run (the writer walks a
HashMap), but the graph no longer does.Path segments now keep the RFC 3986 unreserved set (
-._~) intact instead of escaping every non-alphanumeric. LinkML slot names are overwhelmingly snake_case, so the old encoder would have renderedhome_addressesashome%5Faddressesonce slot names entered the path. Characters that genuinely cannot appear in a path segment are still percent-encoded — seeodd label/with slashabove.On
.well-known/genid/Worth recording, since it prompted the review: RDF 1.1 Concepts §3.5 recommends
<authority>/.well-known/genid/<opaque>for skolem IRIs — but only for systems that "want Skolem IRIs to be recognizable outside of the system boundaries", i.e. marked as artifacts a consumer may map back to blank nodes. These path-derived IRIs are meant to be stable, meaningful addresses, which is the opposite claim. Deliberately not adoptinggenid.Tests
New
src/runtime/tests/turtle_skolem.rs, over the fixture above:skolem_iris_do_not_collide_across_slots— the keyless children get distinct IRIs; home and work streets land on different subjectsskolem_iris_encode_the_slot_they_hang_off— exact expected IRI setskolem_iris_are_keyed_by_key_not_index— mapping keys name their members;_survives,/and space are escapedskolem_iris_are_stable_across_serializations— eight independent load+serialize rounds produce an identical triple setAll four fail on
mainand pass here. Full workspace suite: 229 passed, 0 failed.cargo fmt --checkclean; no new clippy warnings.src/tools/tests/cli.rsupdated — it asserted on the oldroot/gen1name, nowroot/obj.Not addressed
write_turtle's top-levelMappingarm numbers members the same positional way.LinkMLInstance::Mappingcarries a mandatoryslotand no loader produces one at the document root, so I could not reach it from a test and left it rather than changing untested code. Flagging it as latent.Compatibility
This changes the IRI of every keyless nested object in
--skolemoutput. Anything that has stored those IRIs will need to re-derive them.🤖 Generated with Claude Code