diff --git a/.changeset/react-blocks-declaration-parity-not-conformance.md b/.changeset/react-blocks-declaration-parity-not-conformance.md new file mode 100644 index 0000000000..b1a9d00708 --- /dev/null +++ b/.changeset/react-blocks-declaration-parity-not-conformance.md @@ -0,0 +1,50 @@ +--- +'@objectstack/spec': patch +--- + +`check:react-conformance` → `check:react-declaration-parity` — the gate compares two declarations, and said it compared a declaration to an implementation. + +Its header opened by claiming it "confirms the objectui components **ACTUALLY implement** +the props the spec protocol declares". It never could. Both sides of its diff are +declarations: the spec zod schema's props on the left, and on the right the `inputs` the +objectui *registry config* declares — copied verbatim into `sdui.manifest.json` by +`manifestFromConfigs`. No renderer appears anywhere in the chain. So a prop **both sides +declare and nothing reads** is, to this gate, perfect agreement. + +That is not hypothetical. #4413's four blocks (`record:details` / `record:highlights` / +`record:related_list` / `record:path`) published `objectName`/`recordId` that no renderer +read, rendered a "bind a record to preview" placeholder on a `kind:'react'` page, and sat +behind `{ "frontendOnly": [], "missing": false }` in the committed baseline for the whole +life of the defect. A human reading the objectui renderers found it. A gate reporting +green on a promise it cannot keep is worse than no gate — without one, someone checks by +hand. + +Prime Directive #10 (declared ≠ enforced), landing on the thing whose job is to catch it. +Same shape as #1475's "spec declares 9 validation rules, the executor honors 3". + +- **Renamed to what it does**, name and header together, because the name was load-bearing + in the misreading: `check-react-blocks-declaration-parity.ts`, + `react-declaration-parity.baseline.json`, and `frontendOnly` → `registryOnly` in the + baseline ("the registry *declared* it", not "the frontend *implements* it"). +- **The scope caveat is emitted on every run, clean ones included.** Whoever forms a + belief about this gate is reading a CI log, not a source header. +- **It actually gates now.** `gen-sdui-manifest.sh` ran it without `--strict` and swallowed + the exit code behind a `⚠`, so even the divergence it *could* see was recorded and never + stopped (#4472 secondary finding 1). The ratchet fires only on divergence new since the + accepted baseline, so a failure is always a deliberate registry change. +- **The claim is pinned by a test.** `check-react-blocks-declaration-parity.test.ts` + asserts both directions of what the gate can see, that the caveat rides along, and that + the implementation claim does not come back. + +What it sees is unchanged and still worth having — `spec-only` (palette gap, soft), +`registry-only` (undocumented extension, ratcheted), `missing` (not registered / not +public). Exactly one class is invisible: both sides declare it, nothing reads it. + +Evidence about the render path has to come from the render path, which is objectui's side. +`public-block-binding-reach.test.tsx` there mounts every public block declaring an +`objectName` under a recording `dataSource` and asserts the binding reaches it; its first +run separated five bound blocks from three unbound and surfaced two real defects of the +same shape (objectui#3144) — the confirmation this evidence was never obtainable here. +ADR-0082 carries the addendum; the 2026-06 audit carries a correction banner over the +assumption that carried the mistake ("the component reads its full config from the spec +schema at render" — an expectation, never measured). diff --git a/AGENTS.md b/AGENTS.md index 0b132ddf61..132a3a87c1 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -315,10 +315,20 @@ removals" this way while writing this section; `check:generated` now prints this inline when that gate is the one failing.) `check:liveness`, `check:empty-state`, `check:skill-examples`, -`check:react-conformance` and `check:exported-any` are pure checks with no generator — a -failure there is a real finding to fix, not an artifact to regenerate. `check:generated` -names them as deliberately not run, so its "all up to date" never reads as "everything -passed". +`check:react-declaration-parity` and `check:exported-any` are pure checks with no +generator — a failure there is a real finding to fix, not an artifact to regenerate. +`check:generated` names them as deliberately not run, so its "all up to date" never reads +as "everything passed". + +⚠️ **`check:react-declaration-parity` compares two DECLARATIONS, not a declaration against +an implementation.** Left: the props a block's spec zod schema declares. Right: the inputs +the objectui *registry config* declares. Both are declarations — `manifestFromConfigs` +copies `config.inputs` verbatim — so a prop **both sides declare and no renderer reads** +is, to this gate, perfect agreement. It was named `check:react-conformance` and opened by +claiming it confirmed the components "ACTUALLY implement" the spec props; it never could, +and #4413 shipped four dead blocks straight through a green run of it. Renamed and +re-scoped in #4472. The gate is still worth having (`spec-only`, `registry-only` and +`missing` are real signals) — just don't read it as proof anything renders. `check:exported-any` is the one of those that also reads the built `dist/*.d.ts`, so the stale-`dist` caveat above applies to it too. It asks the other half of the diff --git a/docs/adr/0082-react-component-contract-governance.md b/docs/adr/0082-react-component-contract-governance.md index 99e7343e3b..848fc3bc28 100644 --- a/docs/adr/0082-react-component-contract-governance.md +++ b/docs/adr/0082-react-component-contract-governance.md @@ -3,7 +3,7 @@ **Status**: Accepted (2026-06-30) **Deciders**: ObjectStack Protocol Architects **Builds on**: [ADR-0080](./0080-ai-authored-ui-jsx-source.md) (AI authors UI; the component registry `inputs` are the contract; **capability ≠ contract** — curate a small public surface, not the full capability set), [ADR-0081](./0081-trusted-react-page-tier.md) (the `kind:'react'` tier executes real React; its safety boundary is **trust + review**, and its prop ceiling is the **injected scope**), [ADR-0033](./0033-ai-assisted-metadata-authoring.md) (AI writes metadata via draft-gated review), [ADR-0054](./0054-runtime-proof-for-authorable-surface.md) (ratchet a snapshot; flag regressions, not the accepted baseline), [ADR-0078](./0078-no-silently-inert-metadata.md) (no silently-inert metadata — a prop the author writes must be honored or rejected, never silently dropped). -**Consumers**: `@objectstack/spec` (`packages/spec/src/ui/react-blocks.ts` — the block→schema index + React overlay; `scripts/build-react-blocks-contract.ts` — the generator; `scripts/check-react-blocks-conformance.ts` + `react-conformance.baseline.json` — the ratchet), `@objectstack/lint` (`validate-react-page-props.ts` — the authoring prop gate), `@objectstack/cli` (`os validate` wires the gate), `scripts/build-console.sh` (runs the ratchet at console-build time), `../objectui` (the component registry `inputs` are the projected surface the ratchet checks against). +**Consumers**: `@objectstack/spec` (`packages/spec/src/ui/react-blocks.ts` — the block→schema index + React overlay; `scripts/build-react-blocks-contract.ts` — the generator; `scripts/check-react-blocks-declaration-parity.ts` + `react-declaration-parity.baseline.json` — the ratchet), `@objectstack/lint` (`validate-react-page-props.ts` — the authoring prop gate), `@objectstack/cli` (`os validate` wires the gate), `scripts/build-console.sh` (runs the ratchet at console-build time), `../objectui` (the component registry `inputs` are the projected surface the ratchet checks against). **Premise**: ADR-0081 gave authors (and AI) a `kind:'react'` page tier whose blocks are the curated public data components (``, ``, charts, record:* panels). For AI to author those blocks *correctly* it must know each block's props — and for that knowledge to be trustworthy, the props must come from an authoritative, machine-readable, **non-drifting** source. The problem: **there is no single such source.** Three prop surfaces exist for the same components, and nothing keeps them in lockstep: @@ -24,9 +24,9 @@ They drift silently: a component can accept a prop the spec never declared (an u 1. **[source of truth] The spec zod schema is the protocol.** The AI-facing component contract is **generated** from the spec schemas (`z.toJSONSchema`) plus a thin React-interaction overlay — never hand-authored. Generated ⇒ it cannot drift into fiction. 2. **[registry is a subset] Registry `inputs` are the designer palette, not the protocol.** A prop the spec declares but the registry doesn't expose is a *soft* signal (panel gap), not a violation. A prop the component exposes that the spec doesn't declare is the *actionable* signal (undocumented extension). 3. **[overlay] React-interaction props live in a thin overlay, not the spec.** Callbacks (`onSuccess`, `onRowClick`), controlled props (`recordId`, `mode`, `filters`), and binding escape-hatches (`objectName`, a chart's static `data`, a list's `fields`/`options`) are real React surface the *view metadata* schema neither models nor should. They are declared in `react-blocks.ts`'s overlay so the contract documents them. -4. **[conformance = ratchet, not per-PR gate] Frontend↔spec conformance is checked where the manifest is free.** The registry-inputs manifest only exists at console-build time (the registry is a browser app). So conformance runs **inside `build-console.sh`**, warn-only, as a **baseline ratchet** (ADR-0054 shape): it flags only NEW frontend-only props or vanished blocks against a committed baseline — not the accepted divergence, and not every PR. +4. **[declaration parity = ratchet, not per-PR gate] Spec↔registry parity is checked where the manifest is free.** The registry-inputs manifest only exists at console-build time (the registry is a browser app). So it runs **inside `build-console.sh`** as a **baseline ratchet** (ADR-0054 shape): it flags only NEW registry-only inputs or vanished blocks against a committed baseline — not the accepted divergence, and not every PR. *(Amended by #4472: this said "conformance", ran warn-only, and was read as confirming the components implement the spec props. It compares two declarations and now runs `--strict`. See the addendum.)* 5. **[authoring = a hard gate] `os validate` enforces correct prop *usage*.** A separate `validate-react-page-props` gate parses each `kind:'react'` page's real JSX and checks block usage against the contract: a missing **required binding** is an error; a near-miss **prop typo** is a warning; arbitrary unknown props are *not* flagged (the contract's data props are a curated subset, so false positives stay near zero). -6. **[the chain] Five links, each with one job.** protocol source (spec) → generated contract (`react-blocks.md`) → conformance ratchet (build-console.sh) → authoring prop gate (os validate) → a dogfood golden page proving the loop closes. +6. **[the chain] Five links, each with one job.** protocol source (spec) → generated contract (`react-blocks.md`) → declaration-parity ratchet (build-console.sh) → authoring prop gate (os validate) → a dogfood golden page proving the loop closes. **No link in this chain observes a render** — see the addendum for what that costs and where the missing evidence now comes from. --- @@ -55,17 +55,19 @@ The spec UI schemas are *view metadata* — declarative, serializable configurat These are declared in the `react-blocks.ts` overlay with a `kind` of `binding`/`controlled`/`callback`. Declaring a genuine binding in the overlay is how a "frontend-only" prop is *closed* — the divergence was "the component accepts a prop the contract doesn't document," and the fix is to document it, not to leave it as accepted noise (framework #2488 took every block to **0 frontend-only** this way). -### 4. Conformance is a build-time baseline ratchet, not a per-PR gate +### 4. Declaration parity is a build-time baseline ratchet, not a per-PR gate -`scripts/check-react-blocks-conformance.ts` compares the spec props (per block, via `z.toJSONSchema`) against the registry-inputs manifest (`sdui.manifest.json`). The manifest **only exists at console-build time** — the registry is a browser app pulling browser-only deps, so a framework PR has no manifest to check against. Running conformance on every PR is therefore not worth it. +> **Corrected by #4472 — see the addendum.** This decision was written and implemented as "conformance": the check was named `check:react-conformance`, and its script header claimed it confirmed the components "ACTUALLY implement" the spec props. It does not and never did — it compares **two declarations**, and it was **warn-only** besides. The mechanism below is real and kept; the words for it are now `check:react-declaration-parity`, and the gate now runs `--strict`. -Instead, conformance runs **inside `build-console.sh`**, immediately after it dumps the manifest (near-zero marginal cost), as a **baseline ratchet** modeled on ADR-0054: +`scripts/check-react-blocks-declaration-parity.ts` compares the spec props (per block, via `z.toJSONSchema`) against the registry-inputs manifest (`sdui.manifest.json`). The manifest **only exists at console-build time** — the registry is a browser app pulling browser-only deps, so a framework PR has no manifest to check against. Running it on every PR is therefore not worth it. -- `react-conformance.baseline.json` stores each block's accepted frontend-only prop *set* + whether it is missing. -- `--baseline` reports **only regressions**: a block exposing a NEW frontend-only prop, or a previously-present block that vanished. The soft spec-only signal is not gated. -- It is **warn-only** in the console build (never fails it). `--strict` exits non-zero on regression for intentional gating; `--update` re-accepts the current state after a deliberate frontend change. +Instead, it runs **inside `build-console.sh`**, immediately after it dumps the manifest (near-zero marginal cost), as a **baseline ratchet** modeled on ADR-0054: -Because the baseline was driven to **0 frontend-only** (decision 3), the ratchet is noise-free: any future frontend-only prop is a real, actionable signal rather than one sitting in an accepted baseline. +- `react-declaration-parity.baseline.json` stores each block's accepted registry-only input *set* + whether it is missing. +- `--baseline` reports **only regressions**: a block declaring a NEW registry-only input, or a previously-present block that vanished. The soft spec-only signal is not gated. +- It runs `--strict` in the console build, so a regression **fails** it. `--update` re-accepts the current state after a deliberate registry change. + +Because the baseline was driven to **0 registry-only** (decision 3), the ratchet is noise-free: any future registry-only input is a real, actionable signal rather than one sitting in an accepted baseline. ### 5. Authoring correctness is a hard gate at `os validate` @@ -83,8 +85,8 @@ This is the ADR-0078 boundary applied to react pages: a prop the author writes i spec zod schema ──gen──► react-blocks.md (AI reads it — decisions 1–3) (protocol) (generated contract) │ - registry inputs ──────► conformance ratchet (build-console.sh — decision 4) - (designer subset) (warn-only baseline) + registry inputs ──────► declaration-parity ratchet (build-console.sh — decision 4) + (designer subset) (strict baseline; two declarations, no renderer) │ ▼ prop gate (os validate — decision 5) @@ -110,3 +112,30 @@ spec zod schema ──gen──► react-blocks.md (AI reads it — decis - **Hand-author the contract.** Rejected: it drifts into fiction (an earlier Phase-1 hand-authored contract did exactly this). Spec-as-source is zero-drift. - **Treat the registry `inputs` as the source of truth.** Rejected: `inputs` are a curated *subset* (the panel), not the full protocol; sourcing the contract from them would under-document what components actually accept. - **Sandbox/typecheck the React source against generated `.d.ts` for full prop typing.** Out of scope here (and partially covered by ADR-0080's codegen path for the `html` tier); the prop gate's required-binding + typo checks are the pragmatic 80% for `react` authoring without a full type-check harness over executed source. + +--- + +## Addendum (2026-08-01, #4472) — the ratchet compares two declarations; it was named and described as if it compared a declaration to an implementation + +**What was wrong.** Decision 4 shipped as `check:react-conformance`, and the script's header opened by saying it "confirms the objectui components **ACTUALLY implement** the props the spec protocol declares. The spec is the protocol; the frontend must conform." Both halves of its comparison are declarations: + +| left | right | +|---|---| +| the props a block's **spec zod schema** declares (`z.toJSONSchema`) | the inputs the objectui **registry config** declares (`sdui.manifest.json`) | + +The right-hand side comes from objectui's `manifestFromConfigs`, which copies `config.inputs` verbatim. Nothing in the chain looks at a renderer. So a prop **both sides declare and no renderer reads** is, to this gate, perfect agreement — neither declaration is individually false, and the falsehood lives one layer below, in a layer the gate cannot see. + +**What it cost.** #4413: `record:details` / `record:highlights` / `record:related_list` / `record:path` each published `objectName` + `recordId` that no renderer consumed (they take the record from the record page's shared context), so on a `kind:'react'` page all four rendered a "bind a record to preview" placeholder. The committed baseline recorded `{ frontendOnly: [], missing: false }` for all four, and the ratchet stayed green for the defect's entire lifetime. It was found by a human reading the objectui renderers. + +This is Prime Directive #10 (declared ≠ enforced) landing on a gate — the same shape as #1475's "spec declares 9 validation rules, the executor honors 3", except the thing overstating its coverage was the thing whose job is to catch that. **A gate that reports green on a promise it cannot keep is worse than no gate**: without one, someone checks by hand. + +**Corrections.** + +1. **Renamed to what it does** — `check:react-declaration-parity`, `check-react-blocks-declaration-parity.ts`, `react-declaration-parity.baseline.json`, and `frontendOnly` → `registryOnly` in the baseline (the old name implied the frontend *implemented* the prop; it means the registry *declared* it). The name was load-bearing in the misreading, so it had to change with the header. +2. **The scope caveat rides in the output, on every run** — including a clean one. Whoever forms a belief about this gate is reading a CI log, not a source header. +3. **It actually gates.** `gen-sdui-manifest.sh` ran it without `--strict` and swallowed the exit code behind a `⚠`, so even the divergence it *could* see was only ever recorded, never stopped. It now runs `--strict`; the ratchet fires only on divergence new since the accepted baseline, so a failure is always a deliberate registry change needing a spec/overlay edit or an explicit `--update`. +4. **The claim is pinned by a test.** `check-react-blocks-declaration-parity.test.ts` asserts both directions of what the gate *can* see, that the caveat is emitted, and that the implementation claim does not come back — the executable half of Prime Directive #10. + +**What is still true.** Decision 2's signal taxonomy is unchanged and worth keeping: `spec-only` (palette gap, soft), `registry-only` (undocumented extension, ratcheted), `missing` (not registered / not public). Exactly one class is invisible: both sides declare it, nothing reads it. + +**Where the missing evidence now comes from.** Evidence about the render path has to be taken from the render path, which lives in objectui. `apps/console/src/__tests__/public-block-binding-reach.test.tsx` (objectui) mounts every public block that declares an `objectName` input through `SchemaRenderer` with nothing but that binding, under a provider whose `dataSource` records every call, and asserts some call carried the object name. Deliberately narrow — "is this binding wired", not "is every declared input consumed", which is not decidable from outside without heuristics — and every non-reaching block carries a written reason in a ledger asserted to equal the observed set in both directions. Its first run separated five bound blocks from three unbound ones and surfaced two real defects of the #4413 shape (objectui#3144), which is the confirmation that this evidence was never obtainable from here. diff --git a/docs/audits/2026-06-react-blocks-conformance.md b/docs/audits/2026-06-react-blocks-conformance.md index 344e181856..0380e841ec 100644 --- a/docs/audits/2026-06-react-blocks-conformance.md +++ b/docs/audits/2026-06-react-blocks-conformance.md @@ -1,19 +1,43 @@ # Spec ↔ frontend conformance — react blocks (2026-06) +> ## ⚠️ Correction (2026-08-01, #4472) +> +> **This audit did not answer the question it was asked.** The question was +> whether the components *implement* the spec's props. What the check measures — +> then and now — is whether two **declarations** agree: the spec zod schema's +> props, and the `inputs` the objectui *registry config* declares. Both sides are +> declarations (`manifestFromConfigs` copies `config.inputs` verbatim); no +> renderer is involved anywhere in it. +> +> The assumption below that carried the mistake is stated in "How to read this": +> *"The component reads its full config from the spec schema at render."* Nothing +> here established that, and for the `record:*` family it was false — #4413 found +> four blocks publishing `objectName`/`recordId` that no renderer read, which +> this check reported as zero divergence for the whole life of the defect. +> +> The check is now `check:react-declaration-parity` and says so in its own output. +> The findings table below is still accurate **as a declaration diff** — read +> "frontend-only" as "the registry declared an input the spec did not", not as +> "the component accepts it". Evidence about the render path comes from +> objectui's `public-block-binding-reach.test.tsx`; see the ADR-0082 addendum. + **Question** (raised in review): we can't guarantee the frontend (objectui) components actually implement the props the backend spec protocol declares — should we confirm it? -**Answer**: confirmed — they diverge. Below is the first run of the conformance -check (`packages/spec/scripts/check-react-blocks-conformance.ts`), comparing the -spec schemas referenced by `REACT_BLOCKS` against the live objectui -registry-inputs manifest (`sdui.manifest.json`). +**Answer**: confirmed — they diverge. Below is the first run of the check +(`packages/spec/scripts/check-react-blocks-declaration-parity.ts`, then named +`check-react-blocks-conformance.ts`), comparing the spec schemas referenced by +`REACT_BLOCKS` against the live objectui registry-inputs manifest +(`sdui.manifest.json`). ## How to read this The registry `inputs` are the **designer palette** — a curated subset the visual editor exposes — NOT the component's full prop surface. The component reads its -full config from the spec schema at render. So: +full config from the spec schema at render. *(⚠️ That last sentence is the +assumption #4472 retracted — see the correction above. It was an expectation, not +something this audit measured.)* So: - **frontend-only** props (component declares an input the spec does not) are the reliable, actionable divergence: the spec is missing them or they are an @@ -55,7 +79,7 @@ full config from the spec schema at render. So: ``` # produce a manifest from the live registry (objectui), then: -MANIFEST=/path/to/sdui.manifest.json pnpm --filter @objectstack/spec check:react-conformance +MANIFEST=/path/to/sdui.manifest.json pnpm --filter @objectstack/spec check:react-declaration-parity # add --strict to fail on divergence (once triaged). ``` @@ -66,19 +90,21 @@ console-build time. So the conformance check is wired in as a **baseline ratchet at the one place the manifest is produced for free: `scripts/build-console.sh`, right after it dumps `sdui.manifest.json` from the freshly-built console registry. -- The accepted state lives in `packages/spec/react-conformance.baseline.json` - (per block: the frontend-only prop set + whether the block is missing). +- The accepted state lives in `packages/spec/react-declaration-parity.baseline.json` + (per block: the registry-only input set + whether the block is missing). - `--baseline ` compares the current dump against it and reports **only - regressions**: a component exposing a NEW undocumented prop, or a + regressions**: a registry config declaring a NEW undocumented input, or a previously-present block vanishing. The soft `spec-only` signal is not gated. -- In `build-console.sh` it runs **warn-only** (never fails the console build). - Use `--strict` to gate intentionally (exit 1 on regression). +- It runs `--strict`, so a regression fails the run. *(It shipped warn-only — + the exit code swallowed behind a `⚠` — which #4472 corrected along with the + name: "divergence recorded" and "divergence stopped" were being read as the + same green build.)* ``` -# accept the current frontend state as the new baseline (after an intentional change): +# accept the current registry state as the new baseline (after an intentional change): MANIFEST=/path/to/sdui.manifest.json \ - pnpm --filter @objectstack/spec check:react-conformance \ - --baseline react-conformance.baseline.json --update + pnpm --filter @objectstack/spec check:react-declaration-parity \ + --baseline react-declaration-parity.baseline.json --update ``` When the ratchet flags a new frontend-only prop, the fix is one of: declare it in diff --git a/packages/spec/package.json b/packages/spec/package.json index e432888031..3c56f0214c 100644 --- a/packages/spec/package.json +++ b/packages/spec/package.json @@ -213,7 +213,7 @@ "check:strictness-ledger": "tsx scripts/check-strictness-ledger.mts", "gen:react-blocks": "tsx scripts/build-react-blocks-contract.ts", "check:react-blocks": "tsx scripts/build-react-blocks-contract.ts --check", - "check:react-conformance": "tsx scripts/check-react-blocks-conformance.ts", + "check:react-declaration-parity": "tsx scripts/check-react-blocks-declaration-parity.ts", "check:skill-examples": "tsx scripts/check-skill-examples.ts", "typecheck": "tsc --noEmit" }, diff --git a/packages/spec/react-conformance.baseline.json b/packages/spec/react-conformance.baseline.json deleted file mode 100644 index d5f100be08..0000000000 --- a/packages/spec/react-conformance.baseline.json +++ /dev/null @@ -1,17 +0,0 @@ -{ - "_comment": "Accepted spec\u2194frontend conformance baseline (react blocks). Per block: the frontend-only prop set (component exposes, spec does not declare) and whether the block is missing. Regenerate with: MANIFEST=\u2026 check:react-conformance --baseline --update. The ratchet flags only NEW frontend-only props or newly-missing blocks.", - "blocks": { - "ObjectForm": { - "frontendOnly": [], - "missing": false - }, - "ListView": { - "frontendOnly": [], - "missing": false - }, - "ObjectChart": { - "frontendOnly": [], - "missing": false - } - } -} diff --git a/packages/spec/react-declaration-parity.baseline.json b/packages/spec/react-declaration-parity.baseline.json new file mode 100644 index 0000000000..51a6586aee --- /dev/null +++ b/packages/spec/react-declaration-parity.baseline.json @@ -0,0 +1,17 @@ +{ + "_comment": "Accepted spec↔registry DECLARATION-PARITY baseline (react blocks). Per block: the registry-only input set (the registry config declares it, the spec does not) and whether the block is missing. Regenerate with: MANIFEST=… check:react-declaration-parity --baseline --update. The ratchet flags only NEW registry-only inputs or newly-missing blocks. It compares two declarations and inspects no renderer, so a prop both sides declare and nothing reads records as agreement here (#4413/#4472).", + "blocks": { + "ObjectForm": { + "registryOnly": [], + "missing": false + }, + "ListView": { + "registryOnly": [], + "missing": false + }, + "ObjectChart": { + "registryOnly": [], + "missing": false + } + } +} diff --git a/packages/spec/scripts/check-generated.ts b/packages/spec/scripts/check-generated.ts index 7c5c3cba8c..dd83f94078 100644 --- a/packages/spec/scripts/check-generated.ts +++ b/packages/spec/scripts/check-generated.ts @@ -61,7 +61,10 @@ const GATED: ReadonlyArray<{ check: string; gen: string; artifact: string; reads const NO_GENERATOR: ReadonlyArray<{ check: string; why: string }> = [ { check: 'check:liveness', why: 'audits whether declared spec properties have a reader — no artifact' }, { check: 'check:empty-state', why: 'audits empty-state coverage — no artifact' }, - { check: 'check:react-conformance', why: 'audits react blocks against their contract — no artifact' }, + { + check: 'check:react-declaration-parity', + why: 'compares the spec schema props against the registry-declared inputs — two declarations, no artifact (and no renderer: #4472)', + }, { check: 'check:skill-examples', why: 'validates skill examples parse — no artifact' }, // Landed in #4177 while this ledger landed in #4183 — neither PR could see the // other, so `main` carried an unclassified script and this reconciliation was diff --git a/packages/spec/scripts/check-react-blocks-conformance.ts b/packages/spec/scripts/check-react-blocks-conformance.ts deleted file mode 100644 index 5e4d419e16..0000000000 --- a/packages/spec/scripts/check-react-blocks-conformance.ts +++ /dev/null @@ -1,169 +0,0 @@ -// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. -// -// Spec ↔ frontend conformance report (ADR-0081 follow-up). Confirms the -// objectui components ACTUALLY implement the props the spec protocol declares -// for each curated react block. The spec is the protocol; the frontend must -// conform. This surfaces (and can ratchet) the divergence. -// -// - spec-only : the spec schema declares a prop the component does NOT expose -// as a registry input → frontend hasn't implemented the protocol. -// - frontend-only: the component exposes an input the spec does NOT declare → -// undocumented extension (or the spec is behind). -// -// The frontend side is the objectui registry-inputs manifest (sdui.manifest.json, -// produced from the live registry — see objectui scripts/dump-public-manifest.mjs). -// Provide it with MANIFEST=/path/to/sdui.manifest.json. Without it, the check -// reports "manifest unavailable" and exits 0 (same manifest-optional posture as -// the html-tier gate). -// -// Run: MANIFEST=… pnpm --filter @objectstack/spec check:react-conformance -// -// Baseline ratchet (cheap CI posture). The full spec↔frontend divergence has an -// accepted baseline (some props are designer-palette-curated, some spec-only are -// soft). Running this on every PR is not worth it — the manifest only exists at -// console-build time. So we instead RATCHET at that point: store the accepted -// per-block frontend-only set, and warn/fail only on NEW divergence. -// -// --baseline compare current state against a committed baseline and -// report only regressions (a block exposes a NEW -// undocumented prop, or a previously-present block vanished). -// --update with --baseline, (re)write the baseline from the current -// manifest instead of comparing. Run after an intentional -// frontend change to accept the new state. -// --strict exit 1 on divergence (plain mode) or regression (baseline). - -process.env.OS_EAGER_SCHEMAS = '1'; - -import fs from 'fs'; -import { z } from 'zod'; -import { REACT_BLOCKS } from '../src/ui/react-blocks'; - -const MANIFEST = process.env.MANIFEST; -const FAIL_ON_DIVERGENCE = process.argv.includes('--strict'); -const UPDATE_BASELINE = process.argv.includes('--update'); -function argValue(flag: string): string | undefined { - const i = process.argv.indexOf(flag); - if (i >= 0 && process.argv[i + 1] && !process.argv[i + 1].startsWith('--')) return process.argv[i + 1]; - const inline = process.argv.find((a) => a.startsWith(`${flag}=`)); - return inline ? inline.slice(flag.length + 1) : undefined; -} -const BASELINE = argValue('--baseline'); - -function specProps(schema: any): string[] { - try { - let js: any = z.toJSONSchema(schema, { unrepresentable: 'any' } as any); - if (js?.$ref && js?.$defs) js = js.$defs[String(js.$ref).split('/').pop()!] ?? js; - return Object.keys(js?.properties ?? {}).filter((k) => !['aria', 'type', 'id', 'className', 'style'].includes(k)); - } catch { - return []; - } -} - -function manifestInputs(manifest: any, schemaType: string): string[] | null { - const comps = manifest?.components ?? manifest ?? {}; - // keys may be bare ('object-form') or namespaced ('plugin-form:object-form'). - const entry = - comps[schemaType] ?? - Object.entries(comps).find(([k]) => k === schemaType || k.endsWith(`:${schemaType}`))?.[1]; - if (!entry) return null; - const inputs = (entry as any).inputs ?? []; - return inputs.map((i: any) => i?.name).filter(Boolean); -} - -if (!MANIFEST || !fs.existsSync(MANIFEST)) { - console.log('⚠ react-blocks conformance: manifest unavailable (set MANIFEST=…) — skipping.'); - process.exit(0); -} - -const manifest = JSON.parse(fs.readFileSync(MANIFEST, 'utf8')); -let totalSpecOnly = 0; -let totalMissingComp = 0; -const overlay = (b: (typeof REACT_BLOCKS)[number]) => new Set(b.interactions.map((i) => i.name)); - -// Per-block snapshot of the actionable signal we ratchet on: the frontend-only -// prop set (component exposes, spec does not declare) and whether the block is -// missing from the manifest entirely. -type BlockState = { frontendOnly: string[]; missing: boolean }; -const current: Record = {}; - -console.log('# Spec ↔ frontend conformance (react blocks)\n'); -for (const b of REACT_BLOCKS) { - if (!b.schema) continue; - const spec = new Set(specProps(b.schema)); - const inputs = manifestInputs(manifest, b.schemaType); - if (inputs === null) { - console.log(`✗ <${b.tag}> (${b.schemaType}): NO component in the manifest — not registered or not public.`); - totalMissingComp++; - current[b.tag] = { frontendOnly: [], missing: true }; - continue; - } - const inputSet = new Set(inputs); - const ov = overlay(b); - const specOnly = [...spec].filter((p) => !inputSet.has(p) && !ov.has(p)); - const frontendOnly = [...inputSet].filter((p) => !spec.has(p) && !ov.has(p)); - const matched = [...spec].filter((p) => inputSet.has(p)); - totalSpecOnly += specOnly.length; - current[b.tag] = { frontendOnly: frontendOnly.slice().sort(), missing: false }; - const status = specOnly.length === 0 ? '✓' : '⚠'; - console.log(`${status} <${b.tag}> (${b.schemaType}): ${matched.length} matched, ${specOnly.length} spec-only, ${frontendOnly.length} frontend-only`); - if (specOnly.length) console.log(` spec declares but component lacks: ${specOnly.join(', ')}`); - if (frontendOnly.length) console.log(` component exposes but spec lacks: ${frontendOnly.join(', ')}`); -} -console.log(`\nSummary: ${totalSpecOnly} spec-only divergences, ${totalMissingComp} blocks missing from the frontend.`); - -// ── Baseline ratchet ───────────────────────────────────────────────────────── -if (BASELINE) { - type Baseline = { blocks: Record }; - if (UPDATE_BASELINE) { - const out: Baseline = { blocks: current }; - fs.writeFileSync( - BASELINE, - JSON.stringify( - { - _comment: - 'Accepted spec↔frontend conformance baseline (react blocks). Per block: the frontend-only prop set (component exposes, spec does not declare) and whether the block is missing. Regenerate with: MANIFEST=… check:react-conformance --baseline --update. The ratchet flags only NEW frontend-only props or newly-missing blocks.', - ...out, - }, - null, - 2, - ) + '\n', - 'utf8', - ); - console.log(`\n✓ wrote conformance baseline → ${BASELINE} (${Object.keys(current).length} blocks)`); - process.exit(0); - } - - if (!fs.existsSync(BASELINE)) { - console.error(`\n✗ baseline not found: ${BASELINE} — generate it with --update first.`); - process.exit(FAIL_ON_DIVERGENCE ? 1 : 0); - } - const baseline: Baseline = JSON.parse(fs.readFileSync(BASELINE, 'utf8')); - const regressions: string[] = []; - for (const [tag, state] of Object.entries(current)) { - const base = baseline.blocks?.[tag]; - const baseFO = new Set(base?.frontendOnly ?? []); - const newFO = state.frontendOnly.filter((p) => !baseFO.has(p)); - if (newFO.length) regressions.push(`<${tag}>: new frontend-only prop(s) not in baseline: ${newFO.join(', ')}`); - if (state.missing && base && !base.missing) regressions.push(`<${tag}>: block vanished from the manifest (was present in baseline).`); - } - // A brand-new block in the registry that isn't in the baseline is fine (purely - // additive coverage); we only ratchet against accepted blocks regressing. - console.log('\n## Baseline ratchet'); - if (!regressions.length) { - console.log('✓ no new divergence vs accepted baseline.'); - process.exit(0); - } - console.log('⚠ NEW divergence vs accepted baseline:'); - for (const r of regressions) console.log(` - ${r}`); - console.log( - '\n → If intentional (frontend added a prop / the spec is meant to follow), either declare it in the spec\n' + - ' schema, add it to the block overlay in packages/spec/src/ui/react-blocks.ts, or accept it by\n' + - ' rerunning with --update.', - ); - process.exit(FAIL_ON_DIVERGENCE ? 1 : 0); -} - -if (FAIL_ON_DIVERGENCE && (totalSpecOnly > 0 || totalMissingComp > 0)) { - console.error('Conformance check failed (--strict).'); - process.exit(1); -} diff --git a/packages/spec/scripts/check-react-blocks-declaration-parity.test.ts b/packages/spec/scripts/check-react-blocks-declaration-parity.test.ts new file mode 100644 index 0000000000..012a5a0ac6 --- /dev/null +++ b/packages/spec/scripts/check-react-blocks-declaration-parity.test.ts @@ -0,0 +1,157 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. +// +// Pins WHAT `check-react-blocks-declaration-parity` claims, not only what it +// computes (#4472). +// +// The script's defect was never its arithmetic — the set difference it prints +// was always right. The defect was the sentence wrapped around it: it opened by +// saying it "confirms the objectui components ACTUALLY implement the props the +// spec protocol declares", which is a statement about renderers, and it has +// never read a renderer. It reads two DECLARATIONS: the spec zod schema on one +// side, and on the other the objectui registry config's `inputs` — copied +// verbatim into the manifest by `manifestFromConfigs`, so also a declaration. +// +// A claim that outruns the capability is worse than no claim: with no gate a +// human checks by hand, and #4413's four dead `record:*` blocks were in fact +// found by hand — while this check reported `{ frontendOnly: [], missing: false }` +// for every one of them, for that defect's entire lifetime, because both +// declarations agreed and neither was lying. Only the renderer was, and the +// renderer is not in scope here. +// +// So these tests assert three things in one process run: +// 1. the signals it CAN see, in both directions (spec-only / registry-only); +// 2. the scope caveat rides along with EVERY report, success included — +// whoever forms a belief from this gate is reading a CI log, not a header; +// 3. the implementation claim stays gone. This is the executable half of +// Prime Directive #10 ("never advertise a capability the runtime doesn't +// deliver"): the wording that caused #4472 fails a test if it comes back. + +import { describe, it, expect } from 'vitest'; +import { execFileSync } from 'node:child_process'; +import fs from 'node:fs'; +import os from 'node:os'; +import path from 'node:path'; +import { fileURLToPath } from 'node:url'; + +const HERE = path.dirname(fileURLToPath(import.meta.url)); +const PKG = path.resolve(HERE, '..'); +const SCRIPT = path.join(HERE, 'check-react-blocks-declaration-parity.ts'); +const TSX = path.join(PKG, 'node_modules', '.bin', 'tsx'); + +type ManifestInput = { name: string }; +type Manifest = { components: Record }; + +/** Build a manifest declaring exactly `inputs` for `type`. */ +const manifestFor = (type: string, inputs: string[]): Manifest => ({ + components: { [type]: { type, inputs: inputs.map((name) => ({ name })) } }, +}); + +/** Run the real script against a synthetic manifest; return stdout+stderr. */ +function run(manifest: Manifest, args: string[] = []): string { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'react-parity-')); + const file = path.join(dir, 'sdui.manifest.json'); + fs.writeFileSync(file, JSON.stringify(manifest), 'utf8'); + try { + return execFileSync(TSX, [SCRIPT, ...args], { + cwd: PKG, + env: { ...process.env, MANIFEST: file }, + encoding: 'utf8', + stdio: ['ignore', 'pipe', 'pipe'], + }); + } catch (e: any) { + // Non-zero exit (e.g. --strict on divergence) still carries the report. + return `${e?.stdout ?? ''}${e?.stderr ?? ''}`; + } finally { + fs.rmSync(dir, { recursive: true, force: true }); + } +} + +// `object-form` is a real REACT_BLOCKS entry backed by FormViewSchema. Two of +// its schema props are enough to exercise both directions; the rest of the +// schema simply shows up as spec-only, which is the soft signal. +const SCHEMA_PROP = 'layout'; +const NOT_A_SCHEMA_PROP = 'zzzNotInTheFormViewSchema'; + +describe('check:react-declaration-parity — the signals it can see', () => { + it('reports a registry-declared input the spec does not declare as registry-only', () => { + const out = run(manifestFor('object-form', [SCHEMA_PROP, NOT_A_SCHEMA_PROP])); + expect(out).toMatch(/registry declares, spec does not: .*zzzNotInTheFormViewSchema/); + }); + + it('reports a spec-declared prop the registry does not declare as spec-only', () => { + const out = run(manifestFor('object-form', [])); + expect(out).toMatch(new RegExp(`spec declares, registry does not: .*${SCHEMA_PROP}`)); + }); + + it('reports a block absent from the manifest as missing', () => { + const out = run(manifestFor('something-else', [])); + expect(out).toMatch(/ \(object-form\): NO component in the manifest/); + }); +}); + +describe('check:react-declaration-parity — the blind spot is stated, every run (#4413/#4472)', () => { + /** + * The #4413 shape, reconstructed: both sides declare the same prop, so the + * report is clean. Nothing in this run touched a renderer — if `object-form`'s + * renderer stopped reading `layout` tomorrow, this output would not move. That + * is precisely how four `record:*` blocks that rendered "bind a record to + * preview" sat behind a green ratchet. + * + * The assertion is therefore not "it catches this" (it cannot) but "it says so + * while reporting the agreement". + */ + it('calls agreeing declarations agreement — and prints the caveat alongside it', () => { + const out = run(manifestFor('object-form', [SCHEMA_PROP])); + expect(out).toMatch(/declared by both/); + expect(out).not.toMatch(/registry declares, spec does not/); + expect(out).toMatch(/compares two DECLARATIONS/); + expect(out).toMatch(/No renderer is inspected/); + expect(out).toMatch(/#4413/); + }); + + it('carries the caveat on a clean baseline ratchet too, where it is easiest to over-read', () => { + const baseline = path.join(PKG, 'react-declaration-parity.baseline.json'); + // Every baselined block must be present, or the run reports them vanished + // instead of clean. Each declares only spec props, so registry-only is empty + // — the committed baseline's accepted state. + const manifest: Manifest = { + components: { + ...manifestFor('object-form', [SCHEMA_PROP]).components, + ...manifestFor('list-view', []).components, + ...manifestFor('object-chart', []).components, + }, + }; + const out = run(manifest, ['--baseline', baseline]); + expect(out).toMatch(/no new DECLARATION divergence/); + expect(out).toMatch(/compares two DECLARATIONS/); + }); +}); + +describe('check:react-declaration-parity — the retired claim stays retired (Prime Directive #10)', () => { + /** + * Guards the wording, in the script AND in what it prints. "conformance", + * "implements", "honors" all assert something about the render path; this gate + * observes none of it. If a future edit reaches for them again, #4472 recurs — + * a gate whose name promises more than it checks, trusted accordingly. + * + * Scoped to the prose the reader forms a belief from: the file's own + * explanations of what it cannot do are allowed to name the retired claim (and + * do), so the check runs against the report, plus the script's leading header + * block minus the lines that quote the old claim to correct it. + */ + const CLAIM_WORDS = /\b(actually implements?|conforms? to|conformance)\b/i; + + it('the report never claims the frontend implements anything', () => { + const out = run(manifestFor('object-form', [SCHEMA_PROP, NOT_A_SCHEMA_PROP])); + expect(out).not.toMatch(CLAIM_WORDS); + }); + + it('the script advertises its scope before its first line of code', () => { + const src = fs.readFileSync(SCRIPT, 'utf8'); + const header = src.slice(0, src.indexOf('process.env.OS_EAGER_SCHEMAS')); + expect(header).toMatch(/DECLARATION PARITY/); + expect(header).toMatch(/It never looks at a renderer|never (?:reads|inspects) a renderer/i); + // The one blind spot, named in the header rather than left to be rediscovered. + expect(header).toMatch(/BOTH SIDES declare and NO RENDERER READS/); + }); +}); diff --git a/packages/spec/scripts/check-react-blocks-declaration-parity.ts b/packages/spec/scripts/check-react-blocks-declaration-parity.ts new file mode 100644 index 0000000000..17c0bbc17b --- /dev/null +++ b/packages/spec/scripts/check-react-blocks-declaration-parity.ts @@ -0,0 +1,227 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. +// +// Spec ↔ registry DECLARATION PARITY for the curated react blocks (ADR-0081/ +// ADR-0082 follow-up; corrected by #4472). +// +// ⚠️ READ THIS BEFORE TRUSTING A GREEN RUN — what this does NOT do. +// +// This compares TWO DECLARATIONS. It never looks at a renderer: +// +// left the props the SPEC zod schema declares for a block (`z.toJSONSchema`) +// right the inputs the objectui REGISTRY CONFIG declares for the same block +// — read out of `sdui.manifest.json`, which objectui builds with +// `manifestFromConfigs(getPublicConfigs())`; that function copies +// `config.inputs` verbatim, so the right-hand side is a *declaration* +// too, not evidence that anything consumes the prop. +// +// So a prop that BOTH SIDES declare and NO RENDERER READS is, to this script, +// perfect agreement. Neither declaration is lying on its own; the lie is one +// layer below, where the render happens — and that layer is outside this +// script's field of view entirely. +// +// That blind spot is not hypothetical. This file used to open by claiming it +// "confirms the objectui components ACTUALLY implement the props the spec +// protocol declares". It never could. #4413 is what that cost: four blocks +// (`record:details` / `record:highlights` / `record:related_list` / +// `record:path`) published `objectName` / `recordId` that no renderer read — +// those renderers take the record from the record page's shared context — so +// the blocks rendered a "bind a record to preview" placeholder on a +// `kind:'react'` page. Both declarations agreed the whole time, so the baseline +// recorded `{ registryOnly: [], missing: false }` for all four and this check +// stayed green for #4413's entire lifetime. The defect was found by a human +// reading the objectui renderers, not by this script. #4472 is the correction. +// +// WHAT IT DOES SEE (real signals — don't discount them): +// +// - spec-only : the spec schema declares a prop the registry config does +// NOT declare as an input → the designer palette can't +// configure it. A soft signal per ADR-0082 §2 (the palette +// is deliberately a subset), reported but never ratcheted. +// - registry-only : the registry config declares an input the spec does NOT +// declare → undocumented extension, or the spec is behind. +// This is what the baseline ratchets on. +// - missing : no component for this block in the manifest at all → not +// registered, or not public. Real signal. +// +// WHAT IT CANNOT SEE is exactly one class: both sides declare it, nothing reads +// it. Catching that needs evidence from the render path (a behavioural probe or +// a renderer-side usage pass), which lives in objectui, not here. +// `check-react-blocks-declaration-parity.test.ts` pins the limitation with a +// fixture that stays green while its "renderer" ignores everything, so the +// capability cannot be re-assumed by the next reader. +// +// The frontend side is the objectui registry-inputs manifest (sdui.manifest.json +// — see objectui scripts/dump-public-manifest.mjs). Provide it with +// MANIFEST=/path/to/sdui.manifest.json. Without it, the check reports "manifest +// unavailable" and exits 0 (same manifest-optional posture as the html-tier gate). +// +// Run: MANIFEST=… pnpm --filter @objectstack/spec check:react-declaration-parity +// +// Baseline ratchet (cheap CI posture). The full spec↔registry divergence has an +// accepted baseline (some props are designer-palette-curated, some spec-only are +// soft). Running this on every PR is not worth it — the manifest only exists at +// console-build time. So we instead RATCHET at that point: store the accepted +// per-block registry-only set, and fail only on NEW divergence. +// +// --baseline compare current state against a committed baseline and +// report only regressions (a block declares a NEW +// undocumented input, or a previously-present block vanished). +// --update with --baseline, (re)write the baseline from the current +// manifest instead of comparing. Run after an intentional +// registry change to accept the new state. +// --strict exit 1 on divergence (plain mode) or regression (baseline). + +process.env.OS_EAGER_SCHEMAS = '1'; + +import fs from 'fs'; +import { z } from 'zod'; +import { REACT_BLOCKS } from '../src/ui/react-blocks'; + +const MANIFEST = process.env.MANIFEST; +const FAIL_ON_DIVERGENCE = process.argv.includes('--strict'); +const UPDATE_BASELINE = process.argv.includes('--update'); +function argValue(flag: string): string | undefined { + const i = process.argv.indexOf(flag); + if (i >= 0 && process.argv[i + 1] && !process.argv[i + 1].startsWith('--')) return process.argv[i + 1]; + const inline = process.argv.find((a) => a.startsWith(`${flag}=`)); + return inline ? inline.slice(flag.length + 1) : undefined; +} +const BASELINE = argValue('--baseline'); + +/** + * The scope caveat the report carries on EVERY run, success included. + * + * Deliberately in the OUTPUT and not only in this header: whoever forms a belief + * about what "✓ no new divergence" means is reading a CI log, not this file. + * #4472's finding is that the claim travelled further than the capability, so + * the correction has to travel with the result. + */ +const SCOPE_NOTE = + 'Scope: compares two DECLARATIONS — spec zod schema props vs registry-declared inputs.\n' + + ' No renderer is inspected. A prop BOTH sides declare and NO renderer reads counts\n' + + ' as agreement here; that blind spot is what #4413 shipped through (see #4472).'; + +function specProps(schema: any): string[] { + try { + let js: any = z.toJSONSchema(schema, { unrepresentable: 'any' } as any); + if (js?.$ref && js?.$defs) js = js.$defs[String(js.$ref).split('/').pop()!] ?? js; + return Object.keys(js?.properties ?? {}).filter((k) => !['aria', 'type', 'id', 'className', 'style'].includes(k)); + } catch { + return []; + } +} + +function manifestInputs(manifest: any, schemaType: string): string[] | null { + const comps = manifest?.components ?? manifest ?? {}; + // keys may be bare ('object-form') or namespaced ('plugin-form:object-form'). + const entry = + comps[schemaType] ?? + Object.entries(comps).find(([k]) => k === schemaType || k.endsWith(`:${schemaType}`))?.[1]; + if (!entry) return null; + const inputs = (entry as any).inputs ?? []; + return inputs.map((i: any) => i?.name).filter(Boolean); +} + +if (!MANIFEST || !fs.existsSync(MANIFEST)) { + console.log('⚠ react-blocks declaration parity: manifest unavailable (set MANIFEST=…) — skipping.'); + process.exit(0); +} + +const manifest = JSON.parse(fs.readFileSync(MANIFEST, 'utf8')); +let totalSpecOnly = 0; +let totalMissingComp = 0; +const overlay = (b: (typeof REACT_BLOCKS)[number]) => new Set(b.interactions.map((i) => i.name)); + +// Per-block snapshot of the actionable signal we ratchet on: the registry-only +// input set (the registry config declares it, the spec does not) and whether the +// block is missing from the manifest entirely. +type BlockState = { registryOnly: string[]; missing: boolean }; +const current: Record = {}; + +console.log('# Spec ↔ registry declaration parity (react blocks)\n'); +console.log(SCOPE_NOTE + '\n'); +for (const b of REACT_BLOCKS) { + if (!b.schema) continue; + const spec = new Set(specProps(b.schema)); + const inputs = manifestInputs(manifest, b.schemaType); + if (inputs === null) { + console.log(`✗ <${b.tag}> (${b.schemaType}): NO component in the manifest — not registered or not public.`); + totalMissingComp++; + current[b.tag] = { registryOnly: [], missing: true }; + continue; + } + const inputSet = new Set(inputs); + const ov = overlay(b); + const specOnly = [...spec].filter((p) => !inputSet.has(p) && !ov.has(p)); + const registryOnly = [...inputSet].filter((p) => !spec.has(p) && !ov.has(p)); + const declaredByBoth = [...spec].filter((p) => inputSet.has(p)); + totalSpecOnly += specOnly.length; + current[b.tag] = { registryOnly: registryOnly.slice().sort(), missing: false }; + const status = specOnly.length === 0 ? '✓' : '⚠'; + console.log( + `${status} <${b.tag}> (${b.schemaType}): ${declaredByBoth.length} declared by both, ${specOnly.length} spec-only, ${registryOnly.length} registry-only`, + ); + if (specOnly.length) console.log(` spec declares, registry does not: ${specOnly.join(', ')}`); + if (registryOnly.length) console.log(` registry declares, spec does not: ${registryOnly.join(', ')}`); +} +console.log( + `\nSummary: ${totalSpecOnly} spec-only divergences, ${totalMissingComp} blocks missing from the registry.`, +); +console.log(' "declared by both" is a statement about the two declarations, not about the renderer.'); + +// ── Baseline ratchet ───────────────────────────────────────────────────────── +if (BASELINE) { + type Baseline = { blocks: Record }; + if (UPDATE_BASELINE) { + const out: Baseline = { blocks: current }; + fs.writeFileSync( + BASELINE, + JSON.stringify( + { + _comment: + 'Accepted spec↔registry DECLARATION-PARITY baseline (react blocks). Per block: the registry-only input set (the registry config declares it, the spec does not) and whether the block is missing. Regenerate with: MANIFEST=… check:react-declaration-parity --baseline --update. The ratchet flags only NEW registry-only inputs or newly-missing blocks. It compares two declarations and inspects no renderer, so a prop both sides declare and nothing reads records as agreement here (#4413/#4472).', + ...out, + }, + null, + 2, + ) + '\n', + 'utf8', + ); + console.log(`\n✓ wrote declaration-parity baseline → ${BASELINE} (${Object.keys(current).length} blocks)`); + process.exit(0); + } + + if (!fs.existsSync(BASELINE)) { + console.error(`\n✗ baseline not found: ${BASELINE} — generate it with --update first.`); + process.exit(FAIL_ON_DIVERGENCE ? 1 : 0); + } + const baseline: Baseline = JSON.parse(fs.readFileSync(BASELINE, 'utf8')); + const regressions: string[] = []; + for (const [tag, state] of Object.entries(current)) { + const base = baseline.blocks?.[tag]; + const baseRO = new Set(base?.registryOnly ?? []); + const newRO = state.registryOnly.filter((p) => !baseRO.has(p)); + if (newRO.length) regressions.push(`<${tag}>: new registry-only input(s) not in baseline: ${newRO.join(', ')}`); + if (state.missing && base && !base.missing) regressions.push(`<${tag}>: block vanished from the manifest (was present in baseline).`); + } + // A brand-new block in the registry that isn't in the baseline is fine (purely + // additive coverage); we only ratchet against accepted blocks regressing. + console.log('\n## Baseline ratchet'); + if (!regressions.length) { + console.log('✓ no new DECLARATION divergence vs accepted baseline (see the scope note above).'); + process.exit(0); + } + console.log('⚠ NEW declaration divergence vs accepted baseline:'); + for (const r of regressions) console.log(` - ${r}`); + console.log( + '\n → If intentional (the registry added an input / the spec is meant to follow), either declare it\n' + + ' in the spec schema, add it to the block overlay in packages/spec/src/ui/react-blocks.ts, or\n' + + ' accept it by rerunning with --update.', + ); + process.exit(FAIL_ON_DIVERGENCE ? 1 : 0); +} + +if (FAIL_ON_DIVERGENCE && (totalSpecOnly > 0 || totalMissingComp > 0)) { + console.error('Declaration-parity check failed (--strict).'); + process.exit(1); +} diff --git a/scripts/build-console.sh b/scripts/build-console.sh index f185f616c7..0c5f9e4bf1 100755 --- a/scripts/build-console.sh +++ b/scripts/build-console.sh @@ -185,10 +185,10 @@ echo "✓ Bundle canary '${BUNDLE_CANARY}' present — framework client is in th BYTES="$(du -sk "$TARGET" 2>/dev/null | awk '{print $1}')" echo "✓ @objectstack/console dist ready (${BYTES} KB) from objectui@${PINNED_SHA:0:12}" -# ADR-0080/0081: the public-tier SDUI manifest and the spec↔frontend react-block -# conformance ratchet are intentionally NOT generated here — they require a real -# browser (Playwright) to enumerate the console registry, and the console build -# must not drag in a browser dependency. Regenerate them on demand instead: +# ADR-0080/0081: the public-tier SDUI manifest and the spec↔registry react-block +# declaration-parity ratchet are intentionally NOT generated here — they require a +# real browser (Playwright) to enumerate the console registry, and the console +# build must not drag in a browser dependency. Regenerate them on demand instead: # pnpm sdui:manifest (see scripts/gen-sdui-manifest.sh) -echo "ℹ SDUI manifest + conformance ratchet are decoupled from the console build." +echo "ℹ SDUI manifest + declaration-parity ratchet are decoupled from the console build." echo " Run 'pnpm sdui:manifest' on demand to regenerate (requires Playwright)." diff --git a/scripts/gen-sdui-manifest.sh b/scripts/gen-sdui-manifest.sh index f9661f31d9..15ce9857cd 100755 --- a/scripts/gen-sdui-manifest.sh +++ b/scripts/gen-sdui-manifest.sh @@ -67,13 +67,22 @@ else fi popd > /dev/null -# ADR-0081: ratchet the spec↔frontend react-block conformance against the -# committed baseline. Warn-only here — run check:react-conformance --strict to -# gate intentionally. -if [[ -f "${FRAMEWORK_ROOT}/packages/spec/react-conformance.baseline.json" ]]; then - echo "→ Ratcheting spec↔frontend react-block conformance (ADR-0081)..." +# ADR-0081/0082: ratchet the spec↔registry react-block DECLARATION PARITY against +# the committed baseline. +# +# `--strict`, i.e. this now GATES. It used to run without it and swallow the exit +# code behind a `⚠`, so "divergence recorded" and "divergence stopped" were two +# very different things wearing the same green build (#4472, secondary finding 1). +# The ratchet only fires on divergence NEW since the accepted baseline, so a +# failure here is a deliberate registry change that needs either a spec/overlay +# edit or an explicit `--update` to accept — never pre-existing noise. +# +# Scope, since a green line here is easy to over-read: this compares two +# DECLARATIONS (spec zod props vs registry-declared inputs) and inspects no +# renderer. See the header of check-react-blocks-declaration-parity.ts. +if [[ -f "${FRAMEWORK_ROOT}/packages/spec/react-declaration-parity.baseline.json" ]]; then + echo "→ Ratcheting spec↔registry react-block declaration parity (ADR-0082)..." ( cd "${FRAMEWORK_ROOT}" && MANIFEST="${TARGET}/sdui.manifest.json" \ - pnpm --filter @objectstack/spec check:react-conformance \ - --baseline react-conformance.baseline.json ) || \ - echo "⚠ conformance ratchet reported new divergence — run check:react-conformance --strict to gate." + pnpm --filter @objectstack/spec check:react-declaration-parity \ + --baseline react-declaration-parity.baseline.json --strict ) fi