Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 68 additions & 0 deletions .changeset/authoring-rule-command-coverage-registry.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
---
"@objectstack/cli": minor
---

fix(cli): every author-time rule that can gate runs on all three commands (#4409)

`os validate`, `os build` and `os lint` each hand-wired their own subset of the
author-time rules. Nothing connected the three lists, so "which rules run here?"
was answerable only by diffing three 800-line files by eye — and the answer
drifted every time a rule landed. The audit found 23 of 26 rules running on some
strict subset, nine of them able to emit `error`.

The worst direction was the least obvious. `os build` — the command that
PUBLISHES — was the weakest gate of the three: a flow whose expression approver
does not parse (`approval-expression-invalid`) built and published green, and
only `os lint` stopped it, while CI usually runs the other two. `os lint`
disagreed in *both* directions at once, running one gating rule neither other
command ran and missing six that both of them ran, which is worse than no
pre-flight — the remaining options are re-verifying everything or learning to
distrust the signal.

This is the same failure mode's fifth appearance (#3583, #3782, #4384/#4394,
#4402). Each earlier repair removed an instance and left the MODE: a rule's
command coverage was whatever its author remembered to type, and forgetting was
silent. #4402's guard could not catch the rest — it filtered on the current
member names of one suite, so a rule hand-wired into two commands from outside
that suite passed it without a word. A name list only guards the names on it.

**The registry.** `AUTHORING_RULES` declares all 26 rules as data: tier
(`gating`/`advisory`), which stack tier they read (pre-parse `normalized` vs
`parsed`), which commands run them, and a written reason for the one narrowing.
All three commands consume it through `runAuthoringRules()`, so adding a rule is
a one-line edit that reaches every command at once. The three command files
shrink by ~1000 lines between them.

**The ratchet.** The wiring guard is no longer a name list: a `gating` rule on
fewer than three commands fails, a narrowed rule with no reason fails, a command
that calls or imports a registry rule directly fails, and an `advisory` claim is
checked against the rule's own source — so a gate cannot wear an advisory label
to buy itself partial coverage. That last check is the one #3760 needed, having
promoted a `lintFlowPatterns` rule from advisory to gating with nothing anywhere
asking whether its coverage should follow. Remaining direct calls are listed
with reasons, and a stale entry fails too, so the ratchet cannot rot into a
permanent permission slip.

**The verdict, not just the wiring.** A separate test plants one defect per
previously-blind gating rule and asserts all three commands gate on it, plus the
issue's own repro driven end-to-end through the real CLI: exit 1 on all three
where it was 1/0/0.

Two behaviour changes fall out of reporting every failing rule in one run
instead of exiting at the first failing gate: an author with three unrelated
problems now sees all three in one pass, and `--strict` covers every advisory
rather than the roughly half that happened to be printed inline.

Also closes the same hole one gate over: `collectAndLintDocs` failed `os build`
and never ran on `os validate`, invisible because the parity guard keyed on the
`lint*`/`validate*` naming convention and that gate is called `collect*`. The
guard now names each shared non-registry gate explicitly instead of
pattern-matching for them.

Cost is not what argued against any of this. The heavy dependencies
(`typescript` ~9 MB, `sucrase`) are already lazy and load only when a stack
carries the metadata that needs them, and the heaviest rule of the set has run
on all three commands as a reference-integrity suite member since #4340 without
anyone noticing. The one narrowed rule, `lintUniqueDeclarations`, is scoped
because `os lint` already reports it through `lintDataModel` — coverage
recorded, not coverage missing.
40 changes: 32 additions & 8 deletions content/docs/deployment/cli.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -435,6 +435,14 @@ os validate path/to/config # Validate specific file
`dataset` / `dimensions` / `values` resolves to a declared dataset/field, so
a dangling binding fails here instead of rendering an empty chart.

…and every other author-time rule the three commands share — view shape,
name/action/filter references, page sources, approval approvers, security
posture, the autonumber and view-reference lints. All of them come from one
registry, so the list is the same on `os build` and `os lint`; see
[The one gate, three entry points](/docs/deployment/validating-metadata#the-one-gate-three-entry-points)
for the full matrix. Every failing rule is reported in a single run rather than
stopping at the first, so one pass shows the whole hole.

**Options:**
- `--strict` — Treat warnings as errors (exit code 1)
- `--json` — Output results as JSON
Expand All @@ -444,13 +452,18 @@ os validate path/to/config # Validate specific file
- Missing `manifest.namespace` (required for multi-app hosting)
- No objects defined
- No apps or plugins defined
- Every advisory the rule registry raised (dangling `stageField` /
`highlightFields` pointers, replay-unsafe seeds, ambiguous flow status,
deprecated visibility aliases, …)

<Callout type="tip">
`os validate` and `os build` share one validator, so a config that passes
`os validate` will not fail the build on schema/predicate/binding grounds. In a
scaffolded project these are wired as `npm run validate` and `npm run build`;
your `AGENTS.md` tells coding agents to run `npm run validate` after editing
metadata. See [Validating metadata](/docs/deployment/validating-metadata).
`os validate`, `os build` and `os lint` share one rule registry, so a config that
passes any of them will not fail another on schema/predicate/binding grounds — a
CLI test fails the build if a rule that can gate runs on fewer than all three
(#4409). In a scaffolded project these are wired as `npm run validate` and
`npm run build`; your `AGENTS.md` tells coding agents to run `npm run validate`
after editing metadata. See
[Validating metadata](/docs/deployment/validating-metadata).
</Callout>

#### `os info`
Expand Down Expand Up @@ -819,21 +832,32 @@ os create example my-app # Create examples/my-app

| Command | Description |
|---------|-------------|
| `os lint [config]` | Check metadata for style and convention issues (beyond `validate`'s hard gates) |
| `os lint [config]` | Every author-time gate `validate`/`build` run, plus style and convention checks |
| `os test [files]` | Run Quality Protocol test scenarios against a running server |
| `os doctor` | Check development environment health |

#### `os lint`

Style and convention checks on top of `os validate` — naming, labels, translation coverage — with a 0-100 quality score:
The cheapest of the three author-time commands. It runs the same rule registry
`os validate` and `os build` run — so anything that can fail a build fails here
too — and adds its own style rubric: naming, labels, namespace prefixes,
data-model conventions, translation coverage, with a 0-100 quality score.

```bash
os lint # Style / convention checks
os lint # Author-time rules + style / convention checks
os lint --score # Append a 0-100 metadata quality score (letter-graded)
os lint --fix # Show what would be fixed (dry-run)
os lint --json # JSON output for CI
```

It does not replace `os validate`: `os lint` never parses the stack against the
Zod schema (a schema error is `os validate`'s verdict to give), and it emits no
artifact. What it does guarantee is the direction that matters for a pre-flight
— a green `os lint` is not followed by a red `os build`. That was not true
before #4409: `os lint` ran one gating rule neither other command ran and missed
six that both of them ran, so it disagreed with the build in **both**
directions.

#### `os test`

Runs Quality Protocol test scenarios (JSON-based BDD) against a running ObjectStack server.
Expand Down
130 changes: 69 additions & 61 deletions content/docs/deployment/validating-metadata.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -318,51 +318,67 @@ Skipped, to keep false positives at zero: the same set as §8 — non-static
values, `{...spread}` usages, relationship paths, system fields, and objects
another package defines.

## The one gate, two entry points

`os validate` and `os build` (alias of `os compile`) run the **same** validator:

| | `os validate` | `os build` |
|---|---|---|
| Protocol schema (Zod) | ✓ | ✓ |
| CEL / predicate validation | ✓ | ✓ |
| Widget-binding integrity | ✓ | ✓ |
| Dashboard action/route references (ADR-0049) | ✓ | ✓ |
| Object & action name references (#3583) | ✓ | ✓ |
| Page-component field bindings (#3583) | ✓ | ✓ |
| React page block field bindings — §10 (#4340) | ✓ | ✓ |
| Chart bindings outside dashboards (#3583) | ✓ | ✓ |
| Navigation vs. granted access (ADR-0090 D6) | ✓ | ✓ |
| Security posture (ADR-0090 — e.g. every custom object declares `sharingModel`) | ✓ | ✓ |
| Autonumber `{field}` interpolation | ✓ | ✓ |
| View references — form targets, view-key collisions (#2554) | ✓ | ✓ |
| Flow authoring anti-patterns (#1874) | ✓ | ✓ |
| Liveness author-warnings | ✓ | ✓ |
| Undeclared authoring keys — every metadata collection (#3786) and the stack's own top-level keys (#4167) | ✓ | ✓ |
| Emits `dist/objectstack.json` | — | ✓ |

So `os validate` is the fast inner-loop check (no artifact); `os build` is what
you run when you need the deployable artifact. A config that passes `os validate`
will not fail `os build` on schema/predicate/binding grounds — a test in the CLI
asserts that every gate `os build` runs is also run by `os validate`, so the two
cannot drift apart again (#3782). Both entry points
also check SDUI styling (ADR-0065), and `os validate` additionally runs a set of
view- and page-SHAPE checks — list-view navigation modes (ADR-0053), view
container shape, and whether a JSX/React page source parses at all
(ADR-0080/0081) — that catch UI metadata which would otherwise be silently
dropped.

The field bindings INSIDE a react page source are a different matter: they are
reference-integrity, so they run wherever the suite runs. `os lint` gets them
too — it shares the same `REFERENCE_INTEGRITY_RULES` list, which is why the
table's reference rows are the ones a cheap pre-flight can rely on. That was not
always true: the react-page prop gate was hand-wired into `os validate` alone
until #4340's follow-up, so `os lint` and `os build` accepted a page whose every
field binding was stale — the same divergence #4394 closed for readonly flow
writes. A CLI test now asserts no command reaches for a suite member directly
(#4384).

A clean run walks each gate and reports timing:
## The one gate, three entry points

`os validate`, `os build` (alias of `os compile`) and `os lint` run the **same**
author-time rules, from one table — `AUTHORING_RULES` in
`packages/cli/src/lint/authoring-rules.ts`:

| | `os validate` | `os build` | `os lint` |
|---|---|---|---|
| Protocol schema (Zod) | ✓ | ✓ | — |
| CEL / predicate validation (ADR-0032) | ✓ | ✓ | ✓ |
| List-view navigation modes (ADR-0053) | ✓ | ✓ | ✓ |
| View container shape | ✓ | ✓ | ✓ |
| Widget-binding integrity (ADR-0021) | ✓ | ✓ | ✓ |
| Dashboard action/route references (ADR-0049) | ✓ | ✓ | ✓ |
| Filter placeholder resolvability (#3574) | ✓ | ✓ | ✓ |
| Object & action name references (#3583) | ✓ | ✓ | ✓ |
| Page-component field bindings (#3583) | ✓ | ✓ | ✓ |
| React page block field bindings — §10 (#4340) | ✓ | ✓ | ✓ |
| Chart bindings outside dashboards (#3583) | ✓ | ✓ | ✓ |
| Navigation vs. granted access (ADR-0090 D6) | ✓ | ✓ | ✓ |
| SDUI scoped styling (ADR-0065) | ✓ | ✓ | ✓ |
| JSX / React page source parses (ADR-0080/0081) | ✓ | ✓ | ✓ |
| Approval-node approvers (ADR-0090 D3) | ✓ | ✓ | ✓ |
| Security posture (ADR-0090 — e.g. every custom object declares `sharingModel`) | ✓ | ✓ | ✓ |
| Organization-axis red lines (ADR-0105 D6) | ✓ | ✓ | ✓ |
| Autonumber `{field}` interpolation | ✓ | ✓ | ✓ |
| View references — form targets, view-key collisions (#2554) | ✓ | ✓ | ✓ |
| Flow authoring anti-patterns (#1874) | ✓ | ✓ | ✓ |
| Advisory: flow trigger wiring, record titles, semantic field pointers (ADR-0085), seed replay/state safety, capability references, liveness, visibility aliases | ✓ | ✓ | ✓ |
| Package docs — flatness, prefixes, links (ADR-0046) | ✓ | ✓ | ✓ |
| Undeclared authoring keys — every metadata collection (#3786) and the stack's own top-level keys (#4167) | ✓ | ✓ | — |
| Naming, labels, data-model conventions, i18n coverage | — | — | ✓ |
| Emits `dist/objectstack.json` | — | ✓ | — |

So `os validate` is the fast inner-loop check (no artifact), `os build` is what
you run when you need the deployable artifact, and `os lint` adds its own style
rubric on top. **Any rule that can fail a build runs on all three**, so a green
`os lint` means the build's gates are green too, and a stack cannot be published
through the one command that happens to skip a check.

Two rows are deliberately not universal, and both are one-directional (neither
lets a stack through a gate another command enforces): the Zod parse and the
undeclared-key diff need the pre-parse tier and the schema, which only the two
commands that parse actually have; and `os lint`'s own rubric — snake_case
names, missing labels, data-model conventions — is a lint verdict, not a publish
gate. `os build` has never rejected a camelCase object name.

That invariant is enforced, not merely documented. Each rule declares its command
coverage as data, and a CLI test fails if a rule that can emit `error` runs on
fewer than all three, if a narrowed rule carries no written reason, or if any
command reaches for a rule directly instead of going through the registry.

The enforcement exists because the contract drifted four separate times, and the
last audit (#4409) found 23 of 26 rules running on some strict subset of the
three — nine of them able to fail a build. The worst direction was the least
obvious: **`os build` was the weakest of the three gates**, so it emitted an
artifact for stacks the other two refuse. A flow whose expression approver did
not parse built and published green; only `os lint` stopped it, and CI usually
runs the other two.

A clean run walks the registry and reports timing:

```
◆ Validate
Expand All @@ -371,19 +387,9 @@ A clean run walks each gate and reports timing:
Config: /path/to/support-desk/objectstack.config.ts
Load time: 21ms
→ Validating against ObjectStack Protocol...
→ Validating expressions (ADR-0032)...
→ Checking list-view navigation modes (ADR-0053)...
→ Checking view container shape...
→ Checking dashboard widget bindings (ADR-0021)...
→ Checking dashboard action references (ADR-0049)...
→ Checking SDUI styling (ADR-0065)...
→ Checking JSX-source pages (ADR-0080)...
→ Checking React-source pages (ADR-0081)...
→ Checking source-page styling (ADR-0065)...
→ Checking capability references (ADR-0066)...
→ Checking flow trigger wiring...
→ Running authoring lints (#3782)...
→ Checking security posture (ADR-0090 D7)...
→ Running author-time rules (26)...
→ Checking capability providers (#3366)...
→ Checking package docs (ADR-0046)...

✓ Validation passed (64ms)

Expand All @@ -397,9 +403,11 @@ see [the gate in action](/docs/getting-started/build-with-claude-code#4-the-gate
for the bare-reference example verbatim.

<Callout type="info">
`os lint` is a **separate** pass — style and convention checks (snake_case
naming, required labels, namespace prefixes, data-model patterns). Run it too,
but it does not replace `os validate`, and `os validate` does not replace it.
`os lint` runs every gate above **plus** its own style rubric (snake_case
naming, required labels, namespace prefixes, data-model patterns, translation
coverage). It does not replace `os validate` — it never parses against the Zod
schema, so a schema error is `os validate`'s verdict to give — but a rule that
can fail the build fails `os lint` too.
</Callout>

## The workflow
Expand Down
Loading
Loading