Skip to content

test+docs(dynamic): pin & document portable ruleset snapshots across environments (#76) #82

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
2000game merged 1 commit into from
Jul 10, 2026
Merged

test+docs(dynamic): pin & document portable ruleset snapshots across environments (#76) #82

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
81 changes: 64 additions & 17 deletions docs/dynamic-groups.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ group's dynamic-group configuration — the ruleset and its status — as an
opt-in extra on `ct.group(...)`, diffed and applied exactly like any other
managed field.

> People are still never touched. `ct-cli` writes the *ruleset that computes*
> People are still never touched. `ct-cli` writes the _ruleset that computes_
> membership, never a person↔group relationship directly.

## The `dynamic` block
Expand All @@ -18,9 +18,7 @@ ct.group({
groupTypeId: 1,
dynamic: {
status: "manual", // "active" | "manual" | "inactive" | "none"
ruleset: {
/* RuleSet object | { ref: "./x.json" } | churchQuery(...) build */
},
ruleset: {/* RuleSet object | { ref: "./x.json" } | churchQuery(...) build */},
},
});
```
Expand All @@ -41,7 +39,7 @@ group hierarchy. Validation lives in `src/config/context.ts` (`toDesired`).
the `dynamic` block (the DSL still requires a `ruleset` object — `{}` is
fine, since its content is irrelevant on demote) and set
`status: "none"`. `ct apply` responds with `DELETE
/dynamicgroups/{id}/ruleset` followed by `PUT /dynamicgroups/{id}/status`
/dynamicgroups/{id}/ruleset` followed by `PUT /dynamicgroups/{id}/status`
with `{ dynamicGroupStatus: "none" }`.

Statuses are validated against exactly `["active", "inactive", "manual",
Expand All @@ -65,14 +63,63 @@ created/updated — so `ct apply` always writes against the group's real
relative to the process CWD at fold time by `resolveRulesetRef`
(`src/engine/dynamic.ts`); the referenced file's JSON contents are used
verbatim as the ruleset. Handy for pasting an exported ruleset without
inlining a huge JSONLogic tree into the `.ts` config.
inlining a huge JSONLogic tree into the `.ts` config. A captured file
embeds that instance's numeric ids — see [Portable snapshot files across
environments](#portable-snapshot-files-across-environments-76) to make it
drive dev and prod from one file.
3. **Typed query builder** — build the JSONLogic filter with `q` and wrap it
with `churchQuery` (`src/config/query.ts`, re-exported from
`src/config/context.ts`).

All three ultimately produce a plain JS object with the same shape; which one
you pick is purely a matter of how you want to author/version it.

### Portable snapshot files across environments (#76)

A ruleset **captured from a live instance** — `ct adopt group --with-dynamic`, or
a hand-exported `{ ref: "./rulesets/<key>.json" }` file — is byte-faithful by
design: the query filters embed _that instance's_ numeric ids (e.g.
`ctgroup.id ∈ [148, 1228, 32]`, `role.id ∈ [16, 84]`). Those ids are
**instance-specific**. Point the same file at another environment and CT accepts
it, but the ids resolve to different or nonexistent entities there, so the
auto-group computes the wrong (usually empty) membership. This is the one place a
snapshot is _not_ portable — everything else in the config is logical-ref
portable (#20/#22).

To make a snapshot portable, replace an **entity id in a query `var` position**
with a logical reference. The resolver rewrites references anywhere inside the
ruleset to the per-host id at plan time, and the resolved form still
normalizes/diffs **byte-faithfully** against CT (so a matching instance stays a
no-op — it does not re-`PUT` on every apply). Two equivalent ways to author it:

- **Re-author with the typed builder** (preferred for readability):
`churchQuery(q.oneof("ctgroup.id", [ref.group("jugend-mainz"), ref.group("jugend-berlin")]))`.
- **Edit the JSON file in place** — a reference serialises to a plain,
hand-editable JSON leaf, so in a captured `rulesets/<key>.json` you can swap a
raw id for a marker directly:

```jsonc
// before (prod-specific, not portable):
{ "==": [{ "var": "ctgroup.campusId" }, 148] }
// after (portable — resolves to each host's Mainz campus id at plan time):
{ "==": [{ "var": "ctgroup.campusId" }, { "__ctRef": true, "kind": "campus", "key": "mainz" }] }
```

Marker `kind`s: `campus`, `group`, `group-type`, `role-def` (`key` is the
logical key / slug). See `ref` in `src/resolve/refs.ts`.

**Escape hatch — raw numeric ids pass through untouched.** A query that
references an _operational_ group outside the managed scaffold (no logical key to
resolve against) can keep the plain number; you then own its per-environment
correctness. This mirrors the permission scope escape hatch (#49): prefer a
reference, fall back to a number where no managed key exists.

> **Interim caveat (until adopt auto-rewrites ids — #76):** applying a
> raw-id prod snapshot to a _different_ environment is mechanically fine (CT
> accepts it; unknown ids → empty matches) but **semantically wrong** for that
> environment's memberships. Treat it as a known, documented gap — not a silent
> one — when rehearsing prod configs against dev.

### The `RuleSet` shape

Real ChurchTools rulesets (captured from `GET /dynamicgroups/{id}/ruleset` —
Expand All @@ -84,7 +131,7 @@ see `tests/fixtures/dynamic/README.md`) look like:
"shorty": "Autom. Mitgliedschaft Alle Mainz",
"personIdFieldName": "person.id",
"importance": 0,
"query": { "method": "ChurchQuery", "params": { "..." : "..." } },
"query": { "method": "ChurchQuery", "params": { "...": "..." } },
"process": {},
}
```
Expand All @@ -96,15 +143,15 @@ sibling of `query`, not an argument to `churchQuery(...)`.

`q` (`src/config/query.ts`) emits a JSONLogic tree:

| helper | emits |
| ------------------------------ | ----------------------------------------- |
| `q.and(...nodes)` | `{ and: nodes }` |
| `q.or(...nodes)` | `{ or: nodes }` |
| `q.not(node)` | `{ "!": [node] }` |
| `q.var(name)` | `{ var: name }` |
| `q.eq(varName, value)` | `{ "==": [{ var: varName }, value] }` |
| `q.oneof(varName, values)` | `{ oneof: [{ var: varName }, values] }` |
| `q.isnull(varName)` | `{ isnull: [{ var: varName }] }` |
| helper | emits |
| -------------------------- | --------------------------------------- |
| `q.and(...nodes)` | `{ and: nodes }` |
| `q.or(...nodes)` | `{ or: nodes }` |
| `q.not(node)` | `{ "!": [node] }` |
| `q.var(name)` | `{ var: name }` |
| `q.eq(varName, value)` | `{ "==": [{ var: varName }, value] }` |
| `q.oneof(varName, values)` | `{ oneof: [{ var: varName }, values] }` |
| `q.isnull(varName)` | `{ isnull: [{ var: varName }] }` |

`var` values may be **logical references** or **raw ids**. Prefer a reference so
the ruleset is portable across hosts (#20): `q.eq("ctgroup.campusId",
Expand Down Expand Up @@ -186,7 +233,7 @@ dev instance.

Dynamic-group state is only ever fetched for a group that is **both**: (a)
already under state management, and (b) opted in via a `dynamic` block in
the *current* config (`dynamicField.fold` in `src/engine/synthetic.ts`). A
the _current_ config (`dynamicField.fold` in `src/engine/synthetic.ts`). A
managed group without a `dynamic` block is never queried for its
ruleset/status — any auto-group configuration it happens to have in
ChurchTools stays completely invisible to `ct plan` / `ct apply`, exactly
Expand Down
160 changes: 151 additions & 9 deletions tests/portable-refs.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@
* acceptance test — one config yielding valid plans against two different hosts (states + catalogs).
*/
import { describe, it, expect } from "vitest";
import { mkdtempSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { evaluateConfig, q, churchQuery, ref } from "../src/config/context.js";
import { buildPlan } from "../src/engine/build.js";
import { executePlan } from "../src/engine/execute.js";
Expand Down Expand Up @@ -104,7 +107,12 @@ describe("apply-time pending re-resolution (same-run campus + group)", () => {
});
const state = emptyState("h");
state.resources.all_mainz = {
type: "group", id: 100, key: "all_mainz", fields: { name: "All", groupTypeId: 1 }, adoptedAt: "t", updatedAt: "t",
type: "group",
id: 100,
key: "all_mainz",
fields: { name: "All", groupTypeId: 1 },
adoptedAt: "t",
updatedAt: "t",
};
// /groups/100 fetches clean; its ruleset 404s (not yet a dynamic group) → the manual ruleset is a change.
const host = fakeHost({ "/groups/100": { name: "All", groupTypeId: 1 } }, { "POST /campuses": 42 });
Expand All @@ -113,7 +121,9 @@ describe("apply-time pending re-resolution (same-run campus + group)", () => {

const rulesetPut = host.calls.find((c) => c.path === "/dynamicgroups/100/ruleset" && c.method === "PUT")!;
// PUT envelope: { dynamicGroupRuleSet: [ruleset] } (live-decoded, #77).
const body = rulesetPut.body as { dynamicGroupRuleSet: [{ query: { params: { filter: { "==": unknown[] } } } }] };
const body = rulesetPut.body as {
dynamicGroupRuleSet: [{ query: { params: { filter: { "==": unknown[] } } } }];
};
// The campus ref, pending at plan time, is the freshly-created id (42) in the PUT — not a sentinel.
expect(body.dynamicGroupRuleSet[0].query.params.filter["=="][1]).toBe(42);
});
Expand All @@ -136,7 +146,12 @@ describe("apply-time pending re-resolution (same-run campus + group)", () => {
});
const state = emptyState("h");
state.resources.all_kids = {
type: "group", id: 100, key: "all_kids", fields: { name: "All Kids", groupTypeId: 1 }, adoptedAt: "t", updatedAt: "t",
type: "group",
id: 100,
key: "all_kids",
fields: { name: "All Kids", groupTypeId: 1 },
adoptedAt: "t",
updatedAt: "t",
};
const host = fakeHost({ "/groups/100": { name: "All Kids", groupTypeId: 1 } }, { "POST /groups": 55 });
const { plan } = await buildPlan(host, state, resources);
Expand All @@ -146,7 +161,9 @@ describe("apply-time pending re-resolution (same-run campus + group)", () => {
const putIdx = host.calls.findIndex((c) => c.method === "PUT" && c.path === "/dynamicgroups/100/ruleset");
expect(createIdx).toBeGreaterThanOrEqual(0);
expect(putIdx).toBeGreaterThan(createIdx); // target created before the referencing ruleset writes
const body = host.calls[putIdx]!.body as { dynamicGroupRuleSet: [{ query: { params: { filter: { "==": unknown[] } } } }] };
const body = host.calls[putIdx]!.body as {
dynamicGroupRuleSet: [{ query: { params: { filter: { "==": unknown[] } } } }];
};
expect(body.dynamicGroupRuleSet[0].query.params.filter["=="][1]).toBe(55); // the fresh id, not a sentinel
});
});
Expand Down Expand Up @@ -189,9 +206,12 @@ describe("permission domainId resolution", () => {
const { permissions } = await evaluateConfig((ct) => {
ct.groupRole({ key: "p", group: "kids", role: "Leiter", grants: ["churchgroup:administer groups"] });
});
const state: State = { ...emptyState("h"), resources: {
kids: { type: "group", id: 42, key: "kids", fields: {}, adoptedAt: "t", updatedAt: "t" },
} };
const state: State = {
...emptyState("h"),
resources: {
kids: { type: "group", id: 42, key: "kids", fields: {}, adoptedAt: "t", updatedAt: "t" },
},
};
const client = {
get: async <T>(path: string): Promise<T> => {
if (path === "/groups/42/roles") return [{ id: 7001, name: "Leiter" }] as T;
Expand Down Expand Up @@ -250,8 +270,18 @@ describe("acceptance: one config, two hosts", () => {
const b = await planFor(77, emptyState("https://b.church.tools"));

const groupOf = (p: typeof a.plan) => p.items.find((i) => i.key === "kids")!;
expect(groupOf(a.plan).changes).toContainEqual({ field: "groupTypeId", from: undefined, to: 2, source: "config" });
expect(groupOf(b.plan).changes).toContainEqual({ field: "groupTypeId", from: undefined, to: 77, source: "config" });
expect(groupOf(a.plan).changes).toContainEqual({
field: "groupTypeId",
from: undefined,
to: 2,
source: "config",
});
expect(groupOf(b.plan).changes).toContainEqual({
field: "groupTypeId",
from: undefined,
to: 77,
source: "config",
});

// Permission domainId is resolved per host from the same logical ref.
expect(a.items[0]?.domainId).toBe(2);
Expand All @@ -262,3 +292,115 @@ describe("acceptance: one config, two hosts", () => {
expect(b.plan.items.filter((i) => i.action === "create")).toHaveLength(2);
});
});

describe("portable ruleset snapshot files (#76)", () => {
// The existing coverage above proves a logical ref resolves per-host when the ruleset is authored
// INLINE (a `churchQuery(...)` build in the config module). #76 is about the OTHER supply form: a
// captured `{ ref: "./rulesets/<key>.json" }` snapshot FILE. Adopted snapshots come out of CT as
// byte-faithful JSON with that instance's raw numeric ids baked into the query — not portable.
//
// The portable form replaces an entity id in a query `var` position with a logical `{__ctRef}`
// marker (exactly what `ref.campus("mainz")` serialises to as JSON). These tests pin two properties
// the eventual #76 tooling depends on, so a future refactor of the resolution/normalization pass
// can't silently regress them:
// 1. a marker embedded in a ruleset FILE resolves to each host's id at plan time, and
// 2. once resolved, the ruleset diffs BYTE-FAITHFULLY against CT — a matching instance is a no-op.
const dir = mkdtempSync(join(tmpdir(), "ct-portable-rs-"));
const rulesetFile = "portable.json";
// A campus id expressed as a logical marker instead of a raw instance-specific number. JSON.stringify
// of `ref.campus("mainz")` is `{"__ctRef":true,"kind":"campus","key":"mainz"}` — a plain, hand-editable
// JSON leaf that `isRef` recognises when the file is parsed back in.
const authoredRuleset = {
description: "Auto members in Mainz",
importance: 0,
personIdFieldName: "person.id",
query: churchQuery(q.eq("ctgroup.campusId", ref.campus("mainz"))),
process: {},
};
writeFileSync(join(dir, rulesetFile), JSON.stringify(authoredRuleset));

/** State with only the (already-managed) dynamic group — the campus resolves from the live `/campuses` catalog. */
function stateWithGroup(host: string): State {
const s = emptyState(host);
s.resources.all_mainz = {
type: "group",
id: 100,
key: "all_mainz",
fields: { name: "All", groupTypeId: 1 },
adoptedAt: "t",
updatedAt: "t",
};
return s;
}

async function config() {
return (
await evaluateConfig((ct) => {
ct.group({
key: "all_mainz",
name: "All",
groupTypeId: 1,
dynamic: { status: "manual", ruleset: { ref: `./${rulesetFile}` } },
});
})
).resources;
}

it("resolves the file's marker to each host's campus id in the ruleset PUT (dev vs prod)", async () => {
// Same config + same snapshot file, two instances whose "Mainz" campus has a different id.
for (const [host, campusId] of [
["https://dev.church.tools", 42],
["https://prod.church.tools", 7],
] as const) {
const resources = await config();
const state = stateWithGroup(host);
// `/campuses` catalog gives this host's Mainz id; the group's ruleset 404s (not yet a dynamic
// group) so the manual ruleset is a create → a PUT on apply, whose body we can inspect.
const client = fakeHost({
"/groups/100": { name: "All", groupTypeId: 1 },
"/campuses": [{ id: campusId, name: "Mainz" }],
});
const { plan } = await buildPlan(client, state, resources, { configDir: dir });
await executePlan(plan, { client, state, statePath: "s.json", save: noSave, now: () => "t" });

const put = client.calls.find((c) => c.method === "PUT" && c.path === "/dynamicgroups/100/ruleset")!;
const body = put.body as {
dynamicGroupRuleSet: [{ query: { params: { filter: { "==": unknown[] } } } }];
};
// The marker became THIS host's campus id — not a leaked `{__ctRef}` sentinel, not the other host's id.
expect(body.dynamicGroupRuleSet[0].query.params.filter["=="][1]).toBe(campusId);
}
});

it("is a byte-faithful no-op when CT already holds the ruleset with the resolved id", async () => {
const campusId = 42;
const resources = await config();
const state = stateWithGroup("https://dev.church.tools");
// CT's live ruleset already filters on the resolved campus id (42) — what a prior apply wrote.
// GET returns the single-element `[RuleSet]` array with read-only timestamps CT adds.
const liveRuleset = {
description: "Auto members in Mainz",
importance: 0,
personIdFieldName: "person.id",
query: churchQuery(q.eq("ctgroup.campusId", campusId)),
process: {},
dynamicGroupUpdateStarted: "2026-07-10T00:00:00Z",
dynamicGroupUpdateFinished: "2026-07-10T00:01:00Z",
};
const client = fakeHost({
"/groups/100": { name: "All", groupTypeId: 1 },
"/campuses": [{ id: campusId, name: "Mainz" }],
"/dynamicgroups/100/ruleset": [liveRuleset],
"/dynamicgroups/100/status": { dynamicGroupStatus: "manual" },
});
const { plan, fetchErrors } = await buildPlan(client, state, resources, { configDir: dir });

expect(fetchErrors).toEqual([]);
const item = plan.items.find((i) => i.key === "all_mainz")!;
// The desired ruleset (marker → 42, normalized) equals CT's stored ruleset (timestamps stripped),
// so there is no `dynamic` change and the whole group item is a no-op — the resolved logical form
// diffs byte-faithfully, exactly what a portable snapshot needs to not re-PUT on every apply.
expect(item.action).toBe("no-op");
expect(item.changes.some((c) => c.field === "dynamic")).toBe(false);
});
});
Loading