Skip to content

chore(metamodel): make regeneration reproducible, and pin down the dropped-typeof bug - #112

Merged
kervel merged 2 commits into
mainfrom
chore/reproducible-metamodel-regen
Aug 25, 2026
Merged

chore(metamodel): make regeneration reproducible, and pin down the dropped-typeof bug#112
kervel merged 2 commits into
mainfrom
chore/reproducible-metamodel-regen

Conversation

@kervel

@kervel kervel commented Aug 25, 2026

Copy link
Copy Markdown

src/metamodel (linkml_meta) is generated by gen-rust from a linkml checkout, but nothing
recorded which generator revision produced the committed crate. That makes a regen
unreproducible and, worse, silently destructive: a regen from the wrong revision reverts whatever
the committed crate has that the generator no longer emits.

Not hypothetical — this was found while trying to regenerate. The per-key map merge strategy
(option_map_overwrite, which keeps a slot_usage override of annotations from dropping the
inherited ones, added in f79d7fe) was absent from the generator branch that was checked out; a
regen would have replaced it with whole-map overwrite_except_none and no test would have failed.

What this changes

regen.sh:

  • resolves the repo root from the script location, and the sibling linkml checkout from the main
    working tree, so it works from a git worktree instead of silently looking for ../linkml next to
    the worktree;
  • takes LINKML_DIR as an override;
  • stamps src/metamodel/GENERATED_FROM with the generator revision used and with every
    generator commit not yet in linkml/main — while that list is non-empty, the crate cannot be
    reproduced from upstream linkml alone;
  • grows --check, which regenerates into a temp dir, formats it the same way, diffs against the
    committed crate, and writes nothing.

README.md documents both, plus the rule they enforce: a generator fix goes upstream as a linkml PR
before the regenerated crate is committed here.

The bug this was gating, recorded as tests

LinkML spells the keys typeof and abstract. Both are Rust keywords, so the generated fields are
typeof_/abstract_, and nothing maps between the two spellings — serde drops the value without
error and a derived type inherits nothing:

type_ancestors(mainTrackLength)  ->  ["mainTrackLength"]
                          expected  ["mainTrackLength", "trackLength", "integer"]

determine_rdf_type_info is not at fault; its ancestor walk is correct and is being handed a
truncated list. Consequences: rdf_datatype_iri is not inherited, is_range_iri is not inherited
(so a type declared typeof: uri serializes as a literal instead of a named node),
is_integer/is_floating_point are false, and abstract: is dropped on four structs.

The fix is a generator change (linkml/linkml#3919) plus a regen, neither of which belongs in this
PR. So the behaviour is committed as seven tests that fail today and are #[ignore]d with a pointer
to #108 — five in schemaview/tests/typeof_inheritance.rs and two in
runtime/tests/turtle_custom_types.rs. Confirmed failing for the right reason:

geotest:mirror "https://mirror.brussels.be" ;   # should be <https://mirror.brussels.be>
geotest:track_length "1200" ;                   # should carry ^^xsd:integer

Un-ignore them in the commit that lands the regenerated metamodel.

Current --check output

Drift against the local generator stack, all of it the output of two open linkml PRs — the five
serde renames (#3919) and the *OrSubtype base-class variants (#3662). Everything else in the crate
reproduces byte-for-byte after formatting, Cargo.toml included.

Tracking, one issue per upstream PR so they close as they merge: #108 (#3919), #110 (#3662), #111
(#3661, no effect on this crate's output), with #109 for the regeneration itself.

Notes

  • --check needs a local linkml checkout, so it cannot run in CI as-is.
  • src/metamodel/Cargo.toml~ is a committed editor backup; --check excludes it rather than
    deleting it here.

cargo fmt --all --check and cargo test --workspace (72 suites) are clean.

Frank Dekervel and others added 2 commits August 25, 2026 10:37
…d-typeof bug

The metamodel is generated by gen-rust from a linkml checkout, but nothing
recorded which generator revision produced the committed crate. A regen from a
different revision silently reverts whatever the crate has that the generator no
longer emits -- as happened to the per-key map merge strategy, with no test
failing.

regen.sh now:

* resolves the repo root from the script location and the sibling linkml
  checkout from the main working tree, so it works from a git worktree;
* takes LINKML_DIR as an override;
* stamps src/metamodel/GENERATED_FROM with the generator revision used and with
  every generator commit not yet in linkml/main -- while that list is non-empty
  the crate cannot be reproduced from upstream linkml alone;
* grows a --check mode that regenerates into a temp dir, formats it the same
  way, and diffs against the committed crate without writing anything.

--check reports drift today, and all of it is the output of two open linkml PRs
(linkml/linkml#3919, linkml/linkml#3662). Tracked in #109.

Also record the bug that motivated this, as failing tests rather than prose:
LinkML spells the keys `typeof` and `abstract`, both Rust keywords, so the
generated fields are `typeof_`/`abstract_` with nothing mapping between the two
spellings -- serde drops the value without error and a derived type inherits
nothing. type_ancestors truncates to the type itself, so rdf_datatype_iri and
is_range_iri are not inherited and a type declared `typeof: uri` serializes as a
literal instead of a named node.

The fix is a generator change (linkml/linkml#3919) plus a regen, so the tests are
#[ignore]d with a pointer to #108 rather than deleted; they are expected to pass
unchanged once the regenerated metamodel lands. Un-ignore them in that commit.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
One tracking issue per upstream generator PR (#108, #110, #111) so each closes
when its PR merges; #109 is the regeneration task they gate.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@kervel
kervel merged commit a165525 into main Aug 25, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant