From efe93691a299679644fb90f78476e542824c818d Mon Sep 17 00:00:00 2001 From: Felix Kotschenreuther Date: Fri, 10 Jul 2026 14:23:31 +0200 Subject: [PATCH] test+docs(dynamic): pin & document portable ruleset snapshots across environments (#76) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The resolver already rewrites logical refs anywhere inside a dynamic ruleset to the per-host id before diffing (via collectRefs/deepMapRefs over the folded fields bag), and the resolved form normalizes byte-faithfully against CT. That made INLINE churchQuery rulesets portable, but the property was only tested for the inline form — the { ref: "./file.json" } snapshot path (what adopt captures) was uncovered, and nothing documented how to make a captured snapshot portable. - tests: two regression tests over the FILE path — a {__ctRef} marker embedded in a ruleset JSON file resolves to each host's campus id in the PUT (dev 42 vs prod 7), and once resolved diffs byte-faithfully (a matching instance is a no-op, no re-PUT). Locks the resolution/normalization behaviour #76's tooling will build on so a future refactor can't silently regress it. - docs: new 'Portable snapshot files across environments' section in docs/dynamic-groups.md — why captured snapshots embed instance-specific ids, the two ways to make one portable (re-author with q/churchQuery/ref, or drop a {__ctRef} marker into the JSON), the numeric-passthrough escape hatch for operational groups, and the interim 'documented, not silent' cross-env caveat. No engine change: this pins and documents existing behaviour. The adopt-time auto-rewrite of captured ids remains open on #76 (design posted there). --- docs/dynamic-groups.md | 81 ++++++++++++++---- tests/portable-refs.test.ts | 160 ++++++++++++++++++++++++++++++++++-- 2 files changed, 215 insertions(+), 26 deletions(-) diff --git a/docs/dynamic-groups.md b/docs/dynamic-groups.md index 6742dec..cc742a2 100644 --- a/docs/dynamic-groups.md +++ b/docs/dynamic-groups.md @@ -6,7 +6,7 @@ group's dynamic-group configuration — the ruleset and its status — as an opt-in extra on `ct.group(...)`, diffed and applied exactly like any other managed field. -> People are still never touched. `ct-cli` writes the *ruleset that computes* +> People are still never touched. `ct-cli` writes the _ruleset that computes_ > membership, never a person↔group relationship directly. ## The `dynamic` block @@ -18,9 +18,7 @@ ct.group({ groupTypeId: 1, dynamic: { status: "manual", // "active" | "manual" | "inactive" | "none" - ruleset: { - /* RuleSet object | { ref: "./x.json" } | churchQuery(...) build */ - }, + ruleset: {/* RuleSet object | { ref: "./x.json" } | churchQuery(...) build */}, }, }); ``` @@ -41,7 +39,7 @@ group hierarchy. Validation lives in `src/config/context.ts` (`toDesired`). the `dynamic` block (the DSL still requires a `ruleset` object — `{}` is fine, since its content is irrelevant on demote) and set `status: "none"`. `ct apply` responds with `DELETE - /dynamicgroups/{id}/ruleset` followed by `PUT /dynamicgroups/{id}/status` +/dynamicgroups/{id}/ruleset` followed by `PUT /dynamicgroups/{id}/status` with `{ dynamicGroupStatus: "none" }`. Statuses are validated against exactly `["active", "inactive", "manual", @@ -65,7 +63,10 @@ created/updated — so `ct apply` always writes against the group's real relative to the process CWD at fold time by `resolveRulesetRef` (`src/engine/dynamic.ts`); the referenced file's JSON contents are used verbatim as the ruleset. Handy for pasting an exported ruleset without - inlining a huge JSONLogic tree into the `.ts` config. + inlining a huge JSONLogic tree into the `.ts` config. A captured file + embeds that instance's numeric ids — see [Portable snapshot files across + environments](#portable-snapshot-files-across-environments-76) to make it + drive dev and prod from one file. 3. **Typed query builder** — build the JSONLogic filter with `q` and wrap it with `churchQuery` (`src/config/query.ts`, re-exported from `src/config/context.ts`). @@ -73,6 +74,52 @@ created/updated — so `ct apply` always writes against the group's real All three ultimately produce a plain JS object with the same shape; which one you pick is purely a matter of how you want to author/version it. +### Portable snapshot files across environments (#76) + +A ruleset **captured from a live instance** — `ct adopt group --with-dynamic`, or +a hand-exported `{ ref: "./rulesets/.json" }` file — is byte-faithful by +design: the query filters embed _that instance's_ numeric ids (e.g. +`ctgroup.id ∈ [148, 1228, 32]`, `role.id ∈ [16, 84]`). Those ids are +**instance-specific**. Point the same file at another environment and CT accepts +it, but the ids resolve to different or nonexistent entities there, so the +auto-group computes the wrong (usually empty) membership. This is the one place a +snapshot is _not_ portable — everything else in the config is logical-ref +portable (#20/#22). + +To make a snapshot portable, replace an **entity id in a query `var` position** +with a logical reference. The resolver rewrites references anywhere inside the +ruleset to the per-host id at plan time, and the resolved form still +normalizes/diffs **byte-faithfully** against CT (so a matching instance stays a +no-op — it does not re-`PUT` on every apply). Two equivalent ways to author it: + +- **Re-author with the typed builder** (preferred for readability): + `churchQuery(q.oneof("ctgroup.id", [ref.group("jugend-mainz"), ref.group("jugend-berlin")]))`. +- **Edit the JSON file in place** — a reference serialises to a plain, + hand-editable JSON leaf, so in a captured `rulesets/.json` you can swap a + raw id for a marker directly: + + ```jsonc + // before (prod-specific, not portable): + { "==": [{ "var": "ctgroup.campusId" }, 148] } + // after (portable — resolves to each host's Mainz campus id at plan time): + { "==": [{ "var": "ctgroup.campusId" }, { "__ctRef": true, "kind": "campus", "key": "mainz" }] } + ``` + + Marker `kind`s: `campus`, `group`, `group-type`, `role-def` (`key` is the + logical key / slug). See `ref` in `src/resolve/refs.ts`. + +**Escape hatch — raw numeric ids pass through untouched.** A query that +references an _operational_ group outside the managed scaffold (no logical key to +resolve against) can keep the plain number; you then own its per-environment +correctness. This mirrors the permission scope escape hatch (#49): prefer a +reference, fall back to a number where no managed key exists. + +> **Interim caveat (until adopt auto-rewrites ids — #76):** applying a +> raw-id prod snapshot to a _different_ environment is mechanically fine (CT +> accepts it; unknown ids → empty matches) but **semantically wrong** for that +> environment's memberships. Treat it as a known, documented gap — not a silent +> one — when rehearsing prod configs against dev. + ### The `RuleSet` shape Real ChurchTools rulesets (captured from `GET /dynamicgroups/{id}/ruleset` — @@ -84,7 +131,7 @@ see `tests/fixtures/dynamic/README.md`) look like: "shorty": "Autom. Mitgliedschaft Alle Mainz", "personIdFieldName": "person.id", "importance": 0, - "query": { "method": "ChurchQuery", "params": { "..." : "..." } }, + "query": { "method": "ChurchQuery", "params": { "...": "..." } }, "process": {}, } ``` @@ -96,15 +143,15 @@ sibling of `query`, not an argument to `churchQuery(...)`. `q` (`src/config/query.ts`) emits a JSONLogic tree: -| helper | emits | -| ------------------------------ | ----------------------------------------- | -| `q.and(...nodes)` | `{ and: nodes }` | -| `q.or(...nodes)` | `{ or: nodes }` | -| `q.not(node)` | `{ "!": [node] }` | -| `q.var(name)` | `{ var: name }` | -| `q.eq(varName, value)` | `{ "==": [{ var: varName }, value] }` | -| `q.oneof(varName, values)` | `{ oneof: [{ var: varName }, values] }` | -| `q.isnull(varName)` | `{ isnull: [{ var: varName }] }` | +| helper | emits | +| -------------------------- | --------------------------------------- | +| `q.and(...nodes)` | `{ and: nodes }` | +| `q.or(...nodes)` | `{ or: nodes }` | +| `q.not(node)` | `{ "!": [node] }` | +| `q.var(name)` | `{ var: name }` | +| `q.eq(varName, value)` | `{ "==": [{ var: varName }, value] }` | +| `q.oneof(varName, values)` | `{ oneof: [{ var: varName }, values] }` | +| `q.isnull(varName)` | `{ isnull: [{ var: varName }] }` | `var` values may be **logical references** or **raw ids**. Prefer a reference so the ruleset is portable across hosts (#20): `q.eq("ctgroup.campusId", @@ -186,7 +233,7 @@ dev instance. Dynamic-group state is only ever fetched for a group that is **both**: (a) already under state management, and (b) opted in via a `dynamic` block in -the *current* config (`dynamicField.fold` in `src/engine/synthetic.ts`). A +the _current_ config (`dynamicField.fold` in `src/engine/synthetic.ts`). A managed group without a `dynamic` block is never queried for its ruleset/status — any auto-group configuration it happens to have in ChurchTools stays completely invisible to `ct plan` / `ct apply`, exactly diff --git a/tests/portable-refs.test.ts b/tests/portable-refs.test.ts index b2459b9..1040b01 100644 --- a/tests/portable-refs.test.ts +++ b/tests/portable-refs.test.ts @@ -4,6 +4,9 @@ * acceptance test — one config yielding valid plans against two different hosts (states + catalogs). */ import { describe, it, expect } from "vitest"; +import { mkdtempSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; import { evaluateConfig, q, churchQuery, ref } from "../src/config/context.js"; import { buildPlan } from "../src/engine/build.js"; import { executePlan } from "../src/engine/execute.js"; @@ -104,7 +107,12 @@ describe("apply-time pending re-resolution (same-run campus + group)", () => { }); const state = emptyState("h"); state.resources.all_mainz = { - type: "group", id: 100, key: "all_mainz", fields: { name: "All", groupTypeId: 1 }, adoptedAt: "t", updatedAt: "t", + type: "group", + id: 100, + key: "all_mainz", + fields: { name: "All", groupTypeId: 1 }, + adoptedAt: "t", + updatedAt: "t", }; // /groups/100 fetches clean; its ruleset 404s (not yet a dynamic group) → the manual ruleset is a change. const host = fakeHost({ "/groups/100": { name: "All", groupTypeId: 1 } }, { "POST /campuses": 42 }); @@ -113,7 +121,9 @@ describe("apply-time pending re-resolution (same-run campus + group)", () => { const rulesetPut = host.calls.find((c) => c.path === "/dynamicgroups/100/ruleset" && c.method === "PUT")!; // PUT envelope: { dynamicGroupRuleSet: [ruleset] } (live-decoded, #77). - const body = rulesetPut.body as { dynamicGroupRuleSet: [{ query: { params: { filter: { "==": unknown[] } } } }] }; + const body = rulesetPut.body as { + dynamicGroupRuleSet: [{ query: { params: { filter: { "==": unknown[] } } } }]; + }; // The campus ref, pending at plan time, is the freshly-created id (42) in the PUT — not a sentinel. expect(body.dynamicGroupRuleSet[0].query.params.filter["=="][1]).toBe(42); }); @@ -136,7 +146,12 @@ describe("apply-time pending re-resolution (same-run campus + group)", () => { }); const state = emptyState("h"); state.resources.all_kids = { - type: "group", id: 100, key: "all_kids", fields: { name: "All Kids", groupTypeId: 1 }, adoptedAt: "t", updatedAt: "t", + type: "group", + id: 100, + key: "all_kids", + fields: { name: "All Kids", groupTypeId: 1 }, + adoptedAt: "t", + updatedAt: "t", }; const host = fakeHost({ "/groups/100": { name: "All Kids", groupTypeId: 1 } }, { "POST /groups": 55 }); const { plan } = await buildPlan(host, state, resources); @@ -146,7 +161,9 @@ describe("apply-time pending re-resolution (same-run campus + group)", () => { const putIdx = host.calls.findIndex((c) => c.method === "PUT" && c.path === "/dynamicgroups/100/ruleset"); expect(createIdx).toBeGreaterThanOrEqual(0); expect(putIdx).toBeGreaterThan(createIdx); // target created before the referencing ruleset writes - const body = host.calls[putIdx]!.body as { dynamicGroupRuleSet: [{ query: { params: { filter: { "==": unknown[] } } } }] }; + const body = host.calls[putIdx]!.body as { + dynamicGroupRuleSet: [{ query: { params: { filter: { "==": unknown[] } } } }]; + }; expect(body.dynamicGroupRuleSet[0].query.params.filter["=="][1]).toBe(55); // the fresh id, not a sentinel }); }); @@ -189,9 +206,12 @@ describe("permission domainId resolution", () => { const { permissions } = await evaluateConfig((ct) => { ct.groupRole({ key: "p", group: "kids", role: "Leiter", grants: ["churchgroup:administer groups"] }); }); - const state: State = { ...emptyState("h"), resources: { - kids: { type: "group", id: 42, key: "kids", fields: {}, adoptedAt: "t", updatedAt: "t" }, - } }; + const state: State = { + ...emptyState("h"), + resources: { + kids: { type: "group", id: 42, key: "kids", fields: {}, adoptedAt: "t", updatedAt: "t" }, + }, + }; const client = { get: async (path: string): Promise => { if (path === "/groups/42/roles") return [{ id: 7001, name: "Leiter" }] as T; @@ -250,8 +270,18 @@ describe("acceptance: one config, two hosts", () => { const b = await planFor(77, emptyState("https://b.church.tools")); const groupOf = (p: typeof a.plan) => p.items.find((i) => i.key === "kids")!; - expect(groupOf(a.plan).changes).toContainEqual({ field: "groupTypeId", from: undefined, to: 2, source: "config" }); - expect(groupOf(b.plan).changes).toContainEqual({ field: "groupTypeId", from: undefined, to: 77, source: "config" }); + expect(groupOf(a.plan).changes).toContainEqual({ + field: "groupTypeId", + from: undefined, + to: 2, + source: "config", + }); + expect(groupOf(b.plan).changes).toContainEqual({ + field: "groupTypeId", + from: undefined, + to: 77, + source: "config", + }); // Permission domainId is resolved per host from the same logical ref. expect(a.items[0]?.domainId).toBe(2); @@ -262,3 +292,115 @@ describe("acceptance: one config, two hosts", () => { expect(b.plan.items.filter((i) => i.action === "create")).toHaveLength(2); }); }); + +describe("portable ruleset snapshot files (#76)", () => { + // The existing coverage above proves a logical ref resolves per-host when the ruleset is authored + // INLINE (a `churchQuery(...)` build in the config module). #76 is about the OTHER supply form: a + // captured `{ ref: "./rulesets/.json" }` snapshot FILE. Adopted snapshots come out of CT as + // byte-faithful JSON with that instance's raw numeric ids baked into the query — not portable. + // + // The portable form replaces an entity id in a query `var` position with a logical `{__ctRef}` + // marker (exactly what `ref.campus("mainz")` serialises to as JSON). These tests pin two properties + // the eventual #76 tooling depends on, so a future refactor of the resolution/normalization pass + // can't silently regress them: + // 1. a marker embedded in a ruleset FILE resolves to each host's id at plan time, and + // 2. once resolved, the ruleset diffs BYTE-FAITHFULLY against CT — a matching instance is a no-op. + const dir = mkdtempSync(join(tmpdir(), "ct-portable-rs-")); + const rulesetFile = "portable.json"; + // A campus id expressed as a logical marker instead of a raw instance-specific number. JSON.stringify + // of `ref.campus("mainz")` is `{"__ctRef":true,"kind":"campus","key":"mainz"}` — a plain, hand-editable + // JSON leaf that `isRef` recognises when the file is parsed back in. + const authoredRuleset = { + description: "Auto members in Mainz", + importance: 0, + personIdFieldName: "person.id", + query: churchQuery(q.eq("ctgroup.campusId", ref.campus("mainz"))), + process: {}, + }; + writeFileSync(join(dir, rulesetFile), JSON.stringify(authoredRuleset)); + + /** State with only the (already-managed) dynamic group — the campus resolves from the live `/campuses` catalog. */ + function stateWithGroup(host: string): State { + const s = emptyState(host); + s.resources.all_mainz = { + type: "group", + id: 100, + key: "all_mainz", + fields: { name: "All", groupTypeId: 1 }, + adoptedAt: "t", + updatedAt: "t", + }; + return s; + } + + async function config() { + return ( + await evaluateConfig((ct) => { + ct.group({ + key: "all_mainz", + name: "All", + groupTypeId: 1, + dynamic: { status: "manual", ruleset: { ref: `./${rulesetFile}` } }, + }); + }) + ).resources; + } + + it("resolves the file's marker to each host's campus id in the ruleset PUT (dev vs prod)", async () => { + // Same config + same snapshot file, two instances whose "Mainz" campus has a different id. + for (const [host, campusId] of [ + ["https://dev.church.tools", 42], + ["https://prod.church.tools", 7], + ] as const) { + const resources = await config(); + const state = stateWithGroup(host); + // `/campuses` catalog gives this host's Mainz id; the group's ruleset 404s (not yet a dynamic + // group) so the manual ruleset is a create → a PUT on apply, whose body we can inspect. + const client = fakeHost({ + "/groups/100": { name: "All", groupTypeId: 1 }, + "/campuses": [{ id: campusId, name: "Mainz" }], + }); + const { plan } = await buildPlan(client, state, resources, { configDir: dir }); + await executePlan(plan, { client, state, statePath: "s.json", save: noSave, now: () => "t" }); + + const put = client.calls.find((c) => c.method === "PUT" && c.path === "/dynamicgroups/100/ruleset")!; + const body = put.body as { + dynamicGroupRuleSet: [{ query: { params: { filter: { "==": unknown[] } } } }]; + }; + // The marker became THIS host's campus id — not a leaked `{__ctRef}` sentinel, not the other host's id. + expect(body.dynamicGroupRuleSet[0].query.params.filter["=="][1]).toBe(campusId); + } + }); + + it("is a byte-faithful no-op when CT already holds the ruleset with the resolved id", async () => { + const campusId = 42; + const resources = await config(); + const state = stateWithGroup("https://dev.church.tools"); + // CT's live ruleset already filters on the resolved campus id (42) — what a prior apply wrote. + // GET returns the single-element `[RuleSet]` array with read-only timestamps CT adds. + const liveRuleset = { + description: "Auto members in Mainz", + importance: 0, + personIdFieldName: "person.id", + query: churchQuery(q.eq("ctgroup.campusId", campusId)), + process: {}, + dynamicGroupUpdateStarted: "2026-07-10T00:00:00Z", + dynamicGroupUpdateFinished: "2026-07-10T00:01:00Z", + }; + const client = fakeHost({ + "/groups/100": { name: "All", groupTypeId: 1 }, + "/campuses": [{ id: campusId, name: "Mainz" }], + "/dynamicgroups/100/ruleset": [liveRuleset], + "/dynamicgroups/100/status": { dynamicGroupStatus: "manual" }, + }); + const { plan, fetchErrors } = await buildPlan(client, state, resources, { configDir: dir }); + + expect(fetchErrors).toEqual([]); + const item = plan.items.find((i) => i.key === "all_mainz")!; + // The desired ruleset (marker → 42, normalized) equals CT's stored ruleset (timestamps stripped), + // so there is no `dynamic` change and the whole group item is a no-op — the resolved logical form + // diffs byte-faithfully, exactly what a portable snapshot needs to not re-PUT on every apply. + expect(item.action).toBe("no-op"); + expect(item.changes.some((c) => c.field === "dynamic")).toBe(false); + }); +});