diff --git a/.changeset/adr-0078-status-calibration.md b/.changeset/adr-0078-status-calibration.md new file mode 100644 index 0000000000..e6c1019c6c --- /dev/null +++ b/.changeset/adr-0078-status-calibration.md @@ -0,0 +1,10 @@ +--- +--- + +docs(adr-0078): the completeness-gate ADR now says what shipped (#4787). Its status line still read `Proposed … core mechanism unbuilt (2026-07-16 audit)` while the mechanism it called unbuilt had been merged four times over — `packages/spec/src/kernel/functional-completeness.ts` (the shared predicate), `packages/lint/src/validate-functional-completeness.ts` (registered `gating` in `authoring-rules.ts`, so `os build` / `os validate` / `os lint` fail on an error-severity finding), and the registration-time twin in `packages/objectql/src/registry.ts`. An ADR that says "not built" about a built gate is a live misinformation source: the next reader re-opens the project or routes around a gate that is already load-bearing. + +Calibrated to `Accepted — framework implemented; the cloud half of §2/§3 pending (cloud)`, the house form other landed ADRs use, and **not** marked done wholesale — a phased ADR flagged "all finished" is worse than one flagged "not started", because it stops anyone checking. Per phase: Phase 1 ✅ (#4547), Phase 2 🟡 framework half only (#4577 — the blueprint `expression` slot and the strict/lenient key-parity pin; the `objectBody`/`editBuildFieldDef` allow-list rewrite is cloud-owned), Phase 3 ✅ as far as verification supports (#4565 — `webhook/without-triggers`; two Tier-B candidates deliberately unshipped as unverified, three reclassified as runtime/driver gaps that are not authoring-lint items at all, and the nav-target candidate re-homed to ADR-0072 in #4574), Phase 4 ✅ decided (#4599). §2's graph-lint re-homing stays open and is called out as the one item that can silently rot. + +Two disciplines the implementation proved it could not proceed without are now written into the ADR as decision §6 rather than living only in changesets: **a rule ships only with the runtime line that makes it true**, and **a runtime comment blessing an empty case must be corroborated as still reachable** (the webhook skip site blessed a "manual-only" mode that #3196 had removed). Their corollary — the deliberate NON-rules, `multiselect` without `options` above all — is pinned in `scripts/adr-anchors.json` too, so the file realizing the decision points back at it. Phase 4's generative rule sweep is recorded as **rejected, not deferred**: generation cannot produce the skip-site citation §6 requires, so waiting for more data never fixes the route. + +Documentation only; releases nothing. diff --git a/docs/adr/0078-no-silently-inert-metadata.md b/docs/adr/0078-no-silently-inert-metadata.md index 6dacce271a..ff1c41b8d4 100644 --- a/docs/adr/0078-no-silently-inert-metadata.md +++ b/docs/adr/0078-no-silently-inert-metadata.md @@ -1,10 +1,10 @@ # ADR-0078: A Zod-valid metadata instance that is functionally incomplete must fail loudly at author time, uniformly across surfaces (the completeness gate) -**Status**: Proposed (2026-06-28) — philosophy adopted piecemeal, core mechanism unbuilt (2026-07-16 audit): the shared per-type completeness predicate in `@objectstack/spec` and the `validate-functional-completeness` lint do NOT exist; only narrow per-shape applications landed (page `source` refine, form-layout/semantic-roles/widget-bindings lints). The principle is however actively cited and enforced case-by-case (e.g. `bootstrapDeclaredPermissions` closed the `stack.permissions` inert-metadata violation, ADR-0086 D5). +**Status**: Accepted — framework implemented; the cloud half of §2/§3 pending (cloud) (proposed 2026-06-28 · calibrated 2026-08-03). The core mechanism this ADR was written to build **exists and runs**: the shared per-type predicate is `packages/spec/src/kernel/functional-completeness.ts`, the author-time gate is `packages/lint/src/validate-functional-completeness.ts` (registered `gating` in `authoring-rules.ts`, so `os build` / `os validate` / `os lint` enforce it), and the registration-time twin is in `packages/objectql/src/registry.ts`. Landed as Phase 1 (#4547), Phase 3 (#4565), Phase 4 (#4599) and the framework half of Phase 2 (#4577); tracked by #4544, calibrated by #4787. **Not** everything is done — §2's graph-lint re-homing and §3's cloud materialization fix are cloud-owned and unverified from this repo, and two Tier-B candidates are deliberately unshipped. Per-phase detail in [Phasing](#phasing). This supersedes the 2026-07-16 audit reading, which predates every PR above. **Deciders**: ObjectStack Protocol Architects **Builds on**: [ADR-0049](./0049-no-unenforced-security-properties.md) (enforce-or-remove gate — *property* level), [ADR-0054](./0054-runtime-proof-for-authorable-surface.md) (prove-it-runs gate — *integration* level), [ADR-0038](./0038-build-verification-loop.md) (build → verify → self-correct), [ADR-0032](./0032-unified-expression-layer.md) (validate-by-default, no silent failure), [ADR-0072](./0072-reference-scope-and-resolvability.md) (reference resolvability — the reference-side sibling of completeness) **Sibling**: [ADR-0077](./0077-authoring-surface-boundary-hook-flow-validation.md) — 0077 makes *surface-selection* traps loud (a before-flow can't veto); this ADR makes *instance-completeness* traps loud (a bare summary computes nothing). Same "loud-not-silent, AI-is-the-author" family; orthogonal axis. -**Consumers**: `@objectstack/spec` (per-type completeness predicates, sibling of `aggregation-policy.ts`'s `isIncoherentAggregate`; the liveness ledger), `@objectstack/lint` (the new `validate-functional-completeness` validator run by `os build`/`os validate`/`os lint`), `@objectstack/cloud` `service-ai-studio` (graph-lint reuses the shared predicate; the authoring-path config-drop fix), `@objectstack/objectql` (optional registration-time diagnostic), the `objectstack-data`/`objectstack-ui`/`objectstack-automation` skills. +**Consumers**: `@objectstack/spec` (per-type completeness predicates, sibling of `aggregation-policy.ts`'s `isIncoherentAggregate`; the liveness ledger), `@objectstack/lint` (the new `validate-functional-completeness` validator run by `os build`/`os validate`/`os lint`), `@objectstack/cloud` `service-ai-studio` (graph-lint reuses the shared predicate; the authoring-path config-drop fix), `@objectstack/objectql` + `@objectstack/plugin-webhooks` (registration-time diagnostic — shipped, no longer optional), the `objectstack-data`/`objectstack-ui`/`objectstack-automation` skills. **Surfaced by**: [cloud#687](https://github.com/objectstack-ai/cloud/pull/687) (an AI-built `summary` field shipped as a dead `{type:'summary'}` shell; the dependent "occupancy rate" was forever 0 while the agent reported it done) and the follow-on functional-completeness audit (`docs/audits/2026-06-metadata-functional-completeness.md`). --- @@ -54,8 +54,8 @@ For a metadata type, a config whose **omission makes an otherwise-valid instance The check is authored **once** as a pure per-type predicate in `@objectstack/spec`, a sibling of `data/aggregation-policy.ts`'s `isIncoherentAggregate` (the ADR-0019 shared-predicate pattern that already lets `os validate` and cloud graph-lint agree on aggregate coherence): -- **`@objectstack/lint`** gains `validate-functional-completeness.ts` (sibling of `validate-widget-bindings.ts`) consuming the predicate → `os build` / `os validate` / `os lint` / MCP / hand-authoring are covered. -- **Cloud `service-ai-studio` graph-lint** imports the **same** predicate (as it already imports `isIncoherentAggregate`) instead of keeping a divergent copy → the AI-build path stays covered, in lockstep with the framework. +- **`@objectstack/lint`** gains `validate-functional-completeness.ts` (sibling of `validate-widget-bindings.ts`) consuming the predicate → `os build` / `os validate` / `os lint` / MCP / hand-authoring are covered. — ✅ **Shipped** (#4547). The rule is registered `tier: 'gating'`, `input: 'normalized'`, `commands: ALL` in `packages/lint/src/authoring-rules.ts`, so an error-severity finding fails all three commands, and it reads the pre-parse tier so its findings survive an unrelated schema error elsewhere in the stack. +- **Cloud `service-ai-studio` graph-lint** imports the **same** predicate (as it already imports `isIncoherentAggregate`) instead of keeping a divergent copy → the AI-build path stays covered, in lockstep with the framework. — ⬜ **Open, cloud-owned.** The framework exports the predicate for exactly this (`@objectstack/spec/kernel`), but nothing in this repo can prove cloud consumes it; until it does, the AI-build path keeps a divergent copy and the half of §2 that kills the *drift* (not just the asymmetry) is unrealized. This is the one open item that can silently rot: a rule added here does not reach the cloud path on its own. Graph-only checks that need the cross-artifact graph (a summary's child-FK resolvability, a formula's dependency on a broken sibling) **stay** in cloud graph-lint — only the pure per-instance core is shared. The ledger (`packages/spec/liveness/`) may annotate which properties participate in a completeness contract. @@ -69,30 +69,52 @@ The bare summary was not (only) a model mistake: `objectBody`/`editBuildFieldDef This is cloud-side and back-compat-free; it is the single highest-leverage fix (one change revives a swath of field types at once). +**Landing note (2026-08-03).** The **framework half** shipped in #4577: `BlueprintFieldSchema` now has an `expression` slot, and a test pins that the lenient schema and the OpenAI-strict mirror carry **exactly the same keys** — the drift that dropped `formula` when `summaryOperations` was added can no longer happen silently. The **cloud half** (spread-with-denylist in `objectBody`/`editBuildFieldDef`, the remaining blueprint slots, real `FieldType` enums on the granular tools) is cloud-owned; it is *not* verifiable from this repo, so nothing here should be read as asserting it landed. + ### 4. Loud-not-silent at the runtime seam (optional, ratcheted) Where the runtime *skips* an inert instance (`buildSummaryIndex`'s `continue`, a trigger that fails to bind), it emits a **dev-mode diagnostic** at registration rather than swallowing it silently. This is the only layer that is **authoring-tool-agnostic** — it catches an inert instance regardless of which surface (including ones we haven't built) produced it, and complements ADR-0054's prove-it-runs. **No hard Zod `.refine()`** that rejects existing metadata at registration (back-compat); strictness lives in lint/diagnostic, which is observable and non-breaking. +**Landing note (2026-08-03).** ✅ **Built** in Phase 4 (#4599) — this stopped being "optional". The diagnostic sits on `SchemaRegistry.registerObject`, the choke point *every* metadata door converges on (declared stacks, plugin objects, `extend` contributions, `saveMetaItem`, raw `registerObject` calls), because the author-time gate only protects metadata that actually passes through `os build`/`validate`/`lint` — and #3896 (Setup inserting `sys_sharing_rule` rows directly) plus cloud's `rowColor.mapping` (`as never` past tsc) prove the other doors are real, not hypothetical. It runs the **same** `checkFieldCompleteness` and emits the **same rule ids** the lint reports, one aggregated line per object, deduped per object. It **warns and never throws**: §1's `error` severity means *this instance is dead*, not *the system is dead*, and an inert field must not kill a boot thousands of healthy objects share. `plugin-webhooks`' `auto-enqueuer.ts` skip warns with `webhook/without-triggers` for the same reason. `view/layout-without-binding` stays author-time-only — views do not register through this choke point. + ### 5. Ratchet, not retrofit -v1 = the shared predicate + the audit's **verified Tier-A** shapes (authoring config-drop, action-locations, relationship-reference, date-view binding) + cloud#687's already-shipped summary/formula rules re-homed onto the predicate. Tier-B shapes land **each behind a verification pass** (the sharing-rule lesson). The long tail and a generative completeness pass are deferred, **gated on proven need** — never a one-shot demand to make all ~60 candidates lints. +v1 = the shared predicate + the audit's **verified Tier-A** shapes (authoring config-drop, action-locations, relationship-reference, date-view binding) + cloud#687's already-shipped summary/formula rules re-homed onto the predicate. Tier-B shapes land **each behind a verification pass** (the sharing-rule lesson). The long tail is deferred, **gated on proven need** — never a one-shot demand to make all ~60 candidates lints. A **generative** completeness pass was part of this deferral when the ADR was written; Phase 4 **rejected** it outright rather than deferring it further — see §6 and Non-goals. + +### 6. A rule ships only with the runtime line that makes it true + +*(Added at calibration, 2026-08-03. Not a new decision so much as the one the implementation proved it could not proceed without — recorded here because it is the rule that governs every future addition to the predicate.)* + +Each entry in the shared predicate names the **exact runtime site that silently skips the instance**, in the module doc and in the finding's own message — `engine.ts`'s `if (!d.summaryOperations) continue`, `$expand`'s `if (!referenceObject) continue`, `record-validator.ts`'s empty-allowed-list branch, `auto-enqueuer.ts`'s `if (triggers.size === 0) … return null`. A candidate without that citation is not a rule; it is a guess with an error message attached. + +This is not caution for its own sake. The audit's scariest candidate — a "sharing rule fails open and shares every record" — **collapsed on a three-file read**, and the sibling unknown-key campaign shipped four confidently wrong prescriptions before adopting the same discipline. A false prescription is worse than a missing rule: it tells an AI author to "fix" working metadata, and it burns the gate's credibility, which is the only thing making the gate obeyed. + +Two corollaries, both pinned by tests so they cannot be quietly relaxed: + +- **A deliberate NON-rule is recorded with the evidence that exempts it.** `multiselect` without `options` is *not* flagged because `record-validator.ts` says, verbatim, `// free-form (tags without options)` — the runtime blesses it as a mode, which is §1 case (3), genuinely optional. `user` relationships (implicit `sys_user` target) and `timeline`/`tree` views (no renderer verification pass yet) are exempt for their own stated reasons. Adding a rule for any of these is a *regression*, and the test suite is where that attempt fails first. +- **A runtime blessing must be corroborated as still reachable.** The webhook skip site's own comment blesses the empty case as "a manual-only webhook" — structurally identical to the `multiselect` exemption, and on that evidence alone the candidate stays unenforced. But `webhook.zod.ts` (#3196) records that the `api` trigger was **removed** because no manual fire path exists, so the blessed mode is unreachable and the rule is `error` after all. A comment states what its author believed; beliefs go stale when a sibling feature is deleted. ## Phasing -- **Phase 1.** Land the shared per-type completeness predicate in `@objectstack/spec`; re-home cloud#687's summary/formula rules onto it; add `validate-functional-completeness` to `@objectstack/lint` so `os build`/`os validate` enforce the field-level core (closing the path asymmetry for the shapes already shipped in cloud). -- **Phase 2.** The cloud authoring-path config-drop fix (§3) — spread-not-allow-list, blueprint field-shape slots, real `FieldType` enums — with the matrix of dropped keys from the audit. -- **Phase 3.** Verify-then-enforce the Tier-A remainder (action-locations, relationship-reference, date-view binding) and the Tier-B set, one verified shape at a time, across both `@objectstack/lint` and graph-lint. -- **Phase 4 (deferred, evidence-gated).** Registration-time diagnostics (§4) and a generative completeness pass (compose with ADR-0054's `@objectstack/verify` `deriveCrudCases`). Pursued only once the static rules prove out. +Status per phase as of the 2026-08-03 calibration (#4787). A phase is marked ✅ only for the part this repo can actually show; cloud-side work is marked as such rather than assumed. + +- **Phase 1** — ✅ **Implemented** (#4547). Land the shared per-type completeness predicate in `@objectstack/spec`; re-home cloud#687's summary/formula rules onto it; add `validate-functional-completeness` to `@objectstack/lint` so `os build`/`os validate` enforce the field-level core (closing the path asymmetry for the shapes already shipped in cloud). Shipped as `packages/spec/src/kernel/functional-completeness.ts` (`checkFieldCompleteness`, `checkViewCompleteness`, and the pinned `FUNCTIONAL_COMPLETENESS_RULES` id list) plus `packages/lint/src/validate-functional-completeness.ts`, registered as author-time rule 29. Rules shipped: `field/summary-without-operations`, `field/formula-without-expression`, `field/relationship-without-reference`, `field/choice-without-options` (error for `select`/`radio`, warning for `checkboxes`), `view/layout-without-binding` (warning; `kanban`/`calendar`/`gantt`). Its first run against a real app found `showcase_field_zoo.f_summary` — a bare roll-up in the object whose whole job is to demonstrate field types. +- **Phase 2** — 🟡 **Framework half implemented (#4577); cloud half open, cloud-owned.** The cloud authoring-path config-drop fix (§3) — spread-not-allow-list, blueprint field-shape slots, real `FieldType` enums — with the matrix of dropped keys from the audit. Landed here: the `BlueprintFieldSchema.expression` slot and the strict/lenient key-parity pin. Not verifiable here: the `objectBody`/`editBuildFieldDef` allow-list rewrite and the granular-tool enums, which live in the `cloud` repo. +- **Phase 3** — ✅ **Implemented as far as verification supports** (#4565). Verify-then-enforce the Tier-A remainder (action-locations, relationship-reference, date-view binding) and the Tier-B set, one verified shape at a time, across both `@objectstack/lint` and graph-lint. Outcome: `webhook/without-triggers` shipped (error, both `triggers: []` and an omitted key); action-`locations` and approval-approvers were found **already shipped** as their own validators (`validate-action-locations.ts`, `validate-approval-approvers.ts`, the former already exempting the documented headless `locations: []`); relationship-reference, date-view binding and choice-options shipped in Phase 1; nav targets of type page/report/url/component/action turned out to be **reference resolvability (ADR-0072), not completeness** and landed in that module instead (#4574). Deliberately **not shipped, unverified**: `dataset` with zero measures (no runtime consumer in this repo — the dataset compiler lives elsewhere) and schedule-trigger cron validity (`normalizeSchedule` accepts any non-empty string; the scheduler's behaviour on an invalid one was never traced). Three further Tier-B entries (write-side referential integrity, `unique: true` on the memory driver, composite/repeater sub-field constraints) are **runtime/driver gaps with no metadata omission to detect** — not authoring-lint items at all. The graph-lint half of this phase remains open with §2's re-homing. +- **Phase 4** — ✅ **Decided and half built** (#4599); the deferral was resolved, not extended. The two halves got **opposite** verdicts: + - *Registration-time diagnostics (§4)* — **built now**, because the evidence was already in hand rather than pending (see §4's landing note). + - *A generative completeness pass* (compose with ADR-0054's `@objectstack/verify` `deriveCrudCases`) — **rejected, not deferred.** A generator can enumerate candidates ("which optional keys might be load-bearing?") but cannot verify runtime skip sites, and by §6 a rule without its skip-site citation is a false prescription. The route is structurally wrong, not early; no amount of accumulated data fixes it. Recorded here so it is not re-proposed as "we finally have enough data". ## Consequences - **Positive.** Closes the gate-1 instance-completeness hole and the path asymmetry: one predicate, enforced on every authoring surface, so a hand/CLI/MCP author gets the same protection the cloud build agent does. Silent inertness becomes a *loud, self-correctable* error (ADR-0038 loop) instead of a false "done". The three gates compose into an honest chain — *valid* → *complete* → *has a live consumer* → *runs correctly*. §3 makes a correct authoring un-strippable. - **Negative / cost.** A shared predicate plus dual wiring is more than a cloud-only rule; mitigated by reusing the `isIncoherentAggregate` pattern and the existing `@objectstack/lint` harness. The audit is a candidate list — each rule costs a verification pass before it ships (deliberately; see the sharing-rule reversal). Cloud `.objectstack-sha` must bump when graph-lint re-homes onto the shared predicate. -- **Follow-up.** (1) Define the authoritative Tier-A predicate set. (2) cloud#687 is the first instance — its rules re-home onto the shared predicate in Phase 1. (3) Each Tier-B shape files its verification note before becoming a rule. +- **Follow-up.** (1) ✅ Define the authoritative Tier-A predicate set — done; it is `FUNCTIONAL_COMPLETENESS_RULES` in `functional-completeness.ts`, pinned by tests so ids cannot drift. (2) ✅ cloud#687 is the first instance — its rules re-homed onto the shared predicate in Phase 1. (3) ✅ Each Tier-B shape files its verification note before becoming a rule — the discipline is now §6, and two Tier-B candidates were left unshipped under it rather than written on the audit's stated confidence. (4) ⬜ **Open:** cloud graph-lint still has to consume the shared predicate (§2), and the cloud half of §3 (see Phasing) is unlanded here. Until (4) closes, "one predicate, every surface" is true of the framework surfaces only. ## Non-goals - **Hard Zod refinement that rejects existing metadata.** Strictness is lint + diagnostic (observable, non-breaking), not a registration-time crash — existing apps with benign-incomplete instances must keep loading. - **Linting all ~60 audit candidates now.** The ratchet targets verified, high-AI-likelihood, fully-inert shapes; the by-design/perf tail (precision/scale, warn-only fail-open predicates) and roadmapped types (`vector`) are marked `experimental`, not enforced. +- **A generative rule sweep.** *Rejected* in Phase 4, not deferred: generation cannot produce the one thing §6 requires of a rule — the runtime line that makes it true. Enumerating candidates is not the bottleneck; verifying them is, and that is the step a generator structurally skips. - **Re-litigating surface routing.** Which surface a behavior belongs to is ADR-0077; this ADR assumes the instance is on the right surface and asks only whether it is complete enough to run. - **Replacing prove-it-runs (ADR-0054).** Completeness is an *author-time* gate over *static* structure; it does not assert integration correctness. A complete instance can still run wrong — that is gate 3's job. They compose. diff --git a/scripts/adr-anchors.json b/scripts/adr-anchors.json index 2ce6d6b7d0..2efe46e27e 100644 --- a/scripts/adr-anchors.json +++ b/scripts/adr-anchors.json @@ -85,6 +85,11 @@ "file": "packages/metadata-protocol/src/protocol.ts", "adrs": ["ADR-0119"], "invariant": "`batchData`'s `atomic` is REAL or REFUSED, never silent best-effort. An explicitly atomic batch runs inside ONE `engine.transaction()`, so a failure rolls back every prior write and the response reports zero successes (rows marked ROLLED_BACK / NOT_ATTEMPTED — reporting a rolled-back row as `success: true` was the original bug, not merely the missing transaction). A runtime that cannot roll back gets 501 NOT_IMPLEMENTED; degrading to best-effort is how the flag came to lie." + }, + { + "file": "packages/spec/src/kernel/functional-completeness.ts", + "adrs": ["ADR-0078"], + "invariant": "Every rule here cites the runtime line that silently skips the instance, and every deliberate NON-rule cites the evidence that exempts it (ADR-0078 §6). `multiselect` without `options` is NOT flagged — `record-validator.ts` blesses it verbatim as free-form tags, which is §1 case (3) genuinely-optional; `user` relationships and `timeline`/`tree` views are exempt for their own stated reasons. A rule added without its skip-site citation, or an exemption 'fixed', is a false prescription: it tells an AI author to change working metadata, which is the failure this gate exists to prevent." } ] }