feat(dynamic): rulesets embed instance-specific numeric ids — not portable across environments #76
Description
Activity
Handoff note (2026-07-10): deliberately not started — this is design work needing its own plan (the issue body carries the full live evidence and direction: extend the typed query DSL from #14 so ruleset-embedded group/role ids become logical keys resolved per env, with a numeric escape hatch mirroring #49's).
Current live impact, for whoever picks this up: eqrm-dev's 7 dynamic groups now carry prod-numeric query ids (harmless — unknown ids match nothing — but dev auto-memberships won't compute meaningfully until this lands or the dev rulesets are hand-localized).Implementation plan (the "own plan" this issue asks for)
Grounding: the portability mechanism already exists.
build.tsrunsresolver.resolveValue(d.fields, …)over the folded fields bag, andcollectRefs/deepMapRefs(src/resolve/refs.ts) walk intodynamic.ruleset, so a logicalrefanywhere in a ruleset query resolves to the per-host id before the diff — and the resolved form normalizes byte-faithfully (number↔number). This is now pinned + documented by PR #82:- regression tests over the
{ ref: "./file.json" }snapshot-file path: a{__ctRef}marker resolves to each host's id in the PUT (dev 42 vs prod 7), and a matching instance is a byte-faithful no-op; docs/dynamic-groups.md→ "Portable snapshot files across environments" documents the marker form, the numeric escape hatch, and the interim cross-env caveat.
So the remaining work is purely ergonomics: today an adopted snapshot comes out of CT as raw ids and the author must hand-edit markers in. Proposed staged implementation:
Stage 1 —
var → ref-kindcatalog (pure, unit-tested)A small table mapping ChurchQuery
varnames to aRefKind, e.g.ctgroup.id/ctgroup.parentId→group,ctgroup.campusId→campus,ctgroup.groupTypeId→group-type,role.id→role-def. Live-verify the realvarnames against captured prod rulesets before committing the table (do not guess). Unknownvars are left untouched (escape hatch). Lives next tosrc/config/query.ts.Stage 2 — reverse-rewrite helper (pure over an id→key map)
portablizeRuleset(ruleset, { idToKeyByKind })walks the query; for each id in a knownvarposition, if the id maps to a managed logical key (fromReverseResolver/ state, per kind), replace the number withref.<kind>(key); otherwise leave the number and collect a warning{ var, id }. Deterministic, no network — the caller supplies the id→key maps. Unit-test with a fixture prod ruleset.Stage 3 — wire into
ct adopt group --with-dynamiccaptureDynamic(src/commands/adopt-group.ts) currently writes the normalized raw-id ruleset. Add a--portable-rulesetsopt (default off first, flip later) that runs Stage 2 usingReverseResolver(already instantiated in that command) to build the per-kind id→key maps from managed state, then writes markers instead of raw ids. Emit the Stage-2 warnings aswarn(...)lines: "left N unmanaged id(s) numeric in .json — operational groups, not portable (escape hatch)". This is the issue's "numeric passthrough + warning, mirroring #49".Stage 4 — live verification (dev)
Adopt a real dynamic group with
--portable-rulesets, confirm: (a)ct plan --env devis a no-op (markers resolve to the same ids on the capture host), (b) hand-map the same file's keys onto a second env's state and confirm it resolves to that env's ids. Gated live-write test intests/dynamic.integration.test.ts.Open design decisions (need a call before Stage 1)
role.idkind:role-def(global role catalog) vsgroup-role(per-group pairing)? Depends on the realvarsemantics — resolve during Stage-1 live verification.- Default on/off for
--portable-rulesets(safety: auto-rewriting an operational id you thought was managed would silently change semantics — start opt-in). - Sidecar vs inline: inline
{__ctRef}markers (chosen — already works, one file) vs a.map.jsonsidecar (keeps the snapshot byte-identical to CT). Inline is simpler and already supported; recommending inline.
Estimated: Stages 1–2 are small pure/testable units; Stage 3 is the real integration; Stage 4 needs a dev write. No change to the resolve/diff engine (already handles markers).
- regression tests over the
Stages 1–3 landed in PR #86 (merged). Grounded the
var → RefKindcatalog in the real captured prod rulesets (ct-structure/rulesets/*.json) rather than guessing — the full set of entity-bearing vars is exactlyctgroup.id→group,ctgroup.campusId/person.campusId→campus,ctgroup.groupTypeId→group-type,role.id→role-def (single id from the global/group/rolescatalog — notgroup-role, which is a compound pair).ctgroup.groupStatusId(no REST catalog, #67),person.isArchived/person.dateOfDeath(literals) are the escape hatch — left numeric.- Stage 1
VAR_REF_KINDScatalog + Stage 2 pureportablizeRuleset()(byte-faithful round-trip test: portablize →deepMapRefsback → equals normalized original) insrc/config/query-refs.ts. - Stage 3
ct adopt group --with-dynamic --portable-rulesets(default OFF) — builds per-kind id→key maps from theReverseResolvercatalogs + managed state (forgroup), rewrites markers, warns on unmanaged ids left numeric. - No resolve/diff engine change (it already resolves
{ __ctRef }markers, pinned by test+docs(dynamic): pin & document portable ruleset snapshots across environments (#76) #82). 549 tests pass.
Remaining: Stage 4 — live-dev verification (needs eqrm-dev credentials): adopt a real dynamic group with
--portable-rulesets, confirmct plan --env devis a no-op (markers resolve to the same ids on the capture host) + a second-env hand-map check. Leaving this issue open for Stage 4.- Stage 1
- added a commit that references this issue
on Jul 11, 2026 Shipped + applied live (2026-07-11). The role-def approach from #86 was wrong —
role.idin these rulesets is a groupTypeRoleId (per-group-type role), and role names aren't globally unique (prod: 3 "Leiter", 6 "Organisator", 6 "Mitglied"), so role-def refs threw "ambiguous". Fixed by a context-awaregroup-type-roleref (PR #87, v1.3.1) that resolves by (group-type, role-name) — unique (0 collisions across 46 prod roles); it also rewrites the out-of-queryprocess.*.handleMembership.groupTypeRoleId.Live localization (eqrm/ct-structure PR #9): re-captured the dynamic rulesets with
ct adopt group --with-dynamic --portable-rulesets --env prod. Verified:- prod = full no-op — every marker resolves back to the exact prod id (byte-faithful).
- dev resolves to dev ids — e.g. skidscheckinopsmz:
ctgroup.id [112,8]→[57,981](bereich_kids/team_kidsdienst),role.id [84,85,17,16]→[174,177,111,108](Local Lead/Team Leiter+Organisator),process 66→144(Struktur/Mitglied). Applied to dev; dev+prod drift check both clean.
Two accepted limits (dev-data, documented):
- Operational groups (Team Connect, Bereich Integration, etc.) are outside the scaffold → their ids stay numeric (escape hatch).
sintegrationmeeting+merkmal_alle_2_5_mzare all-operational, so they localize little/nothing. - eqrm-dev has duplicate role names within a group type (CT-added
isDefaultroles alongside adopted ones — e.g. Community "leader" fix(dynamic): key portable ruleset roles by (group-type, role-name), not role-def (#76) #87 & #207), making (group-type, role-name) ambiguous on dev.sflow2_5viewer(references Community) is left on raw prod ids until the dev duplicates are cleaned up.
The typed-query DSL portability mechanism (#82) + the adopt-time rewrite (#86 groups/types, #87 roles) are all in place. Remaining is purely dev-data cleanup (dedupe Community/Connectgruppe roles), not tooling.
Closing — the tooling side of this issue is complete and live-verified.
- test+docs(dynamic): pin & document portable ruleset snapshots across environments (#76) #82 pinned + documented that
{ __ctRef }markers resolve insidedynamic.rulesetbefore the diff, and that the resolved form normalizes byte-faithfully. - feat(dynamic): auto-portablize captured ruleset ids on adopt (#76) #86 added the
var → RefKindcatalog (grounded in real captured prod rulesets, not guessed), the pureportablizeRuleset()rewrite, andct adopt group --with-dynamic --portable-rulesets. - fix(dynamic): key portable ruleset roles by (group-type, role-name), not role-def (#76) #87 (v1.3.1) fixed the role kind:
role.idis a groupTypeRoleId, so refs are keyed by (group-type, role-name) — 0 collisions across 46 prod roles. - Live (2026-07-11, ct-structure PR feat: automated releases — install ct from the GitHub Releases page #9): re-captured rulesets are a full no-op on prod and resolve to dev ids on dev (e.g.
ctgroup.id [112,8] → [57,981],role.id [84,85,17,16] → [174,177,111,108],process 66 → 144). Applied to dev; drift clean on both.
Two documented, accepted limits remain, and neither is a tooling gap:
- Groups outside the managed scaffold (operational groups) keep numeric ids — the deliberate escape hatch, warned about at adopt time.
- eqrm-dev has duplicate role names within a group type (CT-added
isDefaultroles), which makes (group-type, role-name) ambiguous there — dev-data cleanup, tracked as instance work, not tool work.
The north star ("zero numeric ids for managed entities; same config drives dev and prod") now holds at the ruleset boundary.
- test+docs(dynamic): pin & document portable ruleset snapshots across environments (#76) #82 pinned + documented that
Live dev rehearsal (#23): the verbatim
rulesets/*.jsonfiles (adopted from prod, referenced viadynamic: true) embed prod-specific numeric ids inside the ChurchQuery filters — e.g. sintegrationmeeting's query filters onctgroup.id [148,1228,32,…]androle.id [16,84,85,…](prod ids). Applying to eqrm-dev would write a ruleset whose ids point at different/nonexistent dev entities, so the auto-group computes wrong (likely empty) memberships on dev.This breaks the north star ("zero numeric ids in the config… same config drives dev and prod", #22/#19) at the ruleset boundary: everything else is logical-ref portable; rulesets are byte-faithful snapshots by design (#14, deliberate for diff fidelity).
Direction: the typed query DSL (#14 Task 7) already models ChurchQuery — extend it (or a ruleset-template form) so group/role references inside rulesets can be expressed as logical keys resolved per environment at plan time, with the resolved form still normalizing/diffing byte-faithfully against CT. Non-managed groups referenced in queries (e.g. operational groups outside the scaffold) need an escape hatch (numeric passthrough + warning, mirroring #49's scope escape hatch).
Interim semantics (accepted for the current rehearsal): applying prod-captured rulesets to dev is mechanically fine (CT accepts; unknown ids → empty matches) but semantically wrong for dev memberships — documented caveat, not silent.
Part of the #19 epic's portability story; substantial design work — needs its own plan before implementation.