Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions .changeset/doc-tags-connects-book-tag-include.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
---
'@objectstack/spec': minor
---

feat(spec): declare `doc.tags`, so a book group's `include: { tag }` can finally match something (#4509)

`BookGroup.include` has always accepted two shapes — a glob over doc names, or
`{ tag: '<t>' }`. The tag variant could never match a single doc in any stack,
and not because the matcher was missing. Everything downstream already existed:

- `matchesInclude` compares `doc.tags` against the rule (`book.zod.ts`)
- the book route already forwards `tags: d.tags` into the resolver (`rest-server.ts`)
- `ResolverDoc` already declares `tags?: string[]` — annotated `(P3d; absent today)`

The gap was one line at the *authoring* end: `DocSchema` is `.strict()` and had
no `tags` key, so writing `tags:` on a doc was a parse error. Every doc therefore
reached the resolver with `tags === undefined`, and the variant matched nothing,
forever.

This is the enforce half of ADR-0049 enforce-or-remove. Removal was the
alternative and was rejected on two grounds: a union member has no clean
tombstone (`retiredKey` covers object keys), so authors would have received a
bare union error carrying no prescription — and it would have discarded a
working matcher to fix a declaration.

```ts
defineDoc({ name: 'crm_guide_lead', content: '# Leads', tags: ['tutorial'] })
defineBook({ name: 'crm', groups: [{ key: 'tut', label: 'Tutorials', include: { tag: 'tutorial' } }] })
```

Prefer a name convention (`include: 'crm_guide_*'`) where one exists — tags earn
their place when membership cuts *across* naming, e.g. a `tutorial` tag spanning
several feature prefixes, which no glob can collect.

Additive: `DocSchema` previously rejected `tags`, so nothing that parsed before
parses differently now.
63 changes: 63 additions & 0 deletions .changeset/mapping-context-selector-unwarnable-keys-retired.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
---
'@objectstack/spec': major
'@objectstack/platform-objects': patch
---

feat(spec)!: retire the five keys the advisory lint could never have warned about — mapping `extractQuery`/`errorPolicy`/`batchSize`, contextSelector `includeAll`/`placement` (#4509)

Five authorable keys parsed, stored, and controlled nothing. What groups them is
not the type they sit on but **why they had to go out in a major rather than
after a deprecation cycle**: four of the five carry schema DEFAULTS, and a
default materialises at parse time — so the liveness advisory lint cannot tell a
value the author wrote from one the schema supplied. Marking them would have
warned on every mapping and every selector in existence, which is why the ledger
recorded them as `_authorWarnSkipped` instead. For a key in that state, removal
is not the escalation after a warning. It is the only channel that ever reaches
the author.

**The retirement kit:**

| FROM | TO | Fix |
|---|---|---|
| `mapping.extractQuery` | *(removed)* | Delete the key. Exports run through the ordinary query API (`POST /api/v1/data/:object/query`) — no exporter has ever read a mapping artifact. |
| `mapping.errorPolicy` | *(removed)* | Delete the key. Error handling on the import path belongs to the import REQUEST's own options, not the stored mapping. |
| `mapping.batchSize` | *(removed)* | Delete the key. The write path sizes its own batches. **Do not relocate the value** — see below. |
| `app.contextSelectors[].includeAll` | *(removed)* | Delete the key. Selectors are mandatory-scope; widen `optionsSource.filter` to widen the choices. |
| `app.contextSelectors[].placement` | *(removed)* | Delete the key. Selectors always render in the sidebar header; `'topbar'` placed nothing. |

Run `os migrate meta --from 16` to rewrite existing sources automatically.

**`includeAll` is the one worth reading twice.** It was not unread — it was
deliberately *disobeyed*, and for a security reason. A context selector is a
mandatory scope, so an "All" row would clear the scope on a surface that exists
to be scoped; on Studio's package selector that means listing the platform's own
system/cloud kernel packages to a developer who scoped to their own package. The
renderer never offered an All row regardless of the flag, so `includeAll: false`
hardened nothing and `includeAll: true` unlocked nothing. `STUDIO_APP` shipped
authoring `includeAll: true` against a renderer that ignored it — that authoring
site goes with the key in this change.

**`batchSize` deliberately offers no rename.** `bulkActionDef.batchSize`,
`connector.batchSize`, `sync.batchSize`, `offline.batchSize`, the seed loader's
and the NoSQL driver cursor's are all LIVE and enforced — but each is a
different key on a different type sizing its own path, and none of them sizes a
mapping import. The rejection says so explicitly, because "removed" plus a
familiar name one line away is exactly how a dead setting gets laundered into a
live-looking one. Same trap `datasource.retryPolicy` had to defuse against
`hook`/`job` `retryPolicy` (which spell the delay `backoffMs`) one issue
earlier.

Both schemas are `.strict()`, so the keys are deleted from the shape and
rejected with a `guidance` prescription rather than tombstoned; their liveness
rows are deleted rather than kept. The retired ALIAS spellings (`query`,
`onError`, `errorHandling`, `errorMode`, `batch`, `chunkSize`, `skipErrors`,
`showall`, `location`) route to the same prescriptions instead of suggesting a
rename onto a key that is also gone.

Registered as the ADR-0087 D2 conversion `mapping-inert-keys-removed` and an
extension of `app-dead-authoring-keys-removed`, both wired into the protocol-17
D3 chain step. The mapping conversion is scoped to the `mappings` collection
deliberately — a stack-wide strip would delete an enforced `batchSize` from
connector, sync, bulk-action and offline shapes.

`datasource` reached zero dead keys in #4583; `mapping` reaches zero here.
3 changes: 0 additions & 3 deletions content/docs/references/data/mapping.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -60,9 +60,6 @@ const result = FieldMappingSchema.parse(data);
| **fieldMapping** | `{ source: string \| string[]; target: string \| string[]; transform: Enum<'none' \| 'constant' \| 'lookup' \| 'split' \| 'join' \| 'javascript' \| 'map'>; params?: object }[]` | ✅ | |
| **mode** | `Enum<'insert' \| 'update' \| 'upsert'>` | ✅ | |
| **upsertKey** | `string[]` | optional | Fields to match for upsert (e.g. email) |
| **extractQuery** | `{ object: string; fields?: string[]; where?: any; search?: string \| { query: string; fields?: string[]; fuzzy: boolean; operator: Enum<'and' \| 'or'>; … }; … }` | optional | Query to run for export only |
| **errorPolicy** | `Enum<'skip' \| 'abort' \| 'retry'>` | ✅ | |
| **batchSize** | `number` | ✅ | |
| **_lock** | `Enum<'none' \| 'no-overlay' \| 'no-delete' \| 'full'>` | optional | Item-level lock — controls overlay & delete (ADR-0010). |
| **_lockReason** | `string` | optional | Human-readable reason shown when a write is refused by _lock. |
| **_lockSource** | `Enum<'artifact' \| 'package' \| 'env-forced'>` | optional | Layer that set _lock (artifact \| package \| env-forced). |
Expand Down
1 change: 1 addition & 0 deletions content/docs/references/system/doc.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@ const result = DocSchema.parse(data);
| **content** | `string` | ✅ | Raw Markdown content (CommonMark + GFM) |
| **order** | `number` | optional | Sort key within a book group (ADR-0046 §6) |
| **group** | `string` | optional | Explicit book-group key (ADR-0046 §6); rules usually suffice |
| **tags** | `string[]` | optional | Membership tags matched by a book group's `include: { tag }` rule (ADR-0046 §5) |
| **translations** | `Record<string, { label?: string; description?: string; content: string }>` | optional | Per-locale `{label?,description?,content}` variants; the base doc is the fallback |
| **_lock** | `Enum<'none' \| 'no-overlay' \| 'no-delete' \| 'full'>` | optional | Item-level lock — controls overlay & delete (ADR-0010). |
| **_lockReason** | `string` | optional | Human-readable reason shown when a write is refused by _lock. |
Expand Down
4 changes: 1 addition & 3 deletions content/docs/references/ui/app.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -128,10 +128,8 @@ const result = ActionNavItemSchema.parse(data);
| **label** | `string` | ✅ | Dropdown label |
| **icon** | `string` | optional | Icon name |
| **optionsSource** | `{ endpoint: string; valueKey: string; labelKey: string; filter?: { key: string; op: Enum<'eq' \| 'ne' \| 'in' \| 'nin'>; value: string \| string[] }[] }` | ✅ | Option data source |
| **includeAll** | `boolean` | ✅ | Prepend an "All" option that clears the scope |
| **allValue** | `string` | ✅ | Template value when "All" is selected (empty = no filter) |
| **allValue** | `string` | ✅ | Sentinel value meaning "no concrete selection yet" (empty string is almost always right) |
| **persist** | `Enum<'query' \| 'session' \| 'none'>` | ✅ | Persist selection via URL query, sessionStorage, or not at all |
| **placement** | `Enum<'sidebar_header' \| 'topbar'>` | ✅ | Render location in the app chrome |


---
Expand Down
5 changes: 4 additions & 1 deletion docs/protocol-upgrade-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,8 @@ The `script` flow node converges on its one real path (#4343). It had four ways

The same audit reaches the driver contract itself: `IDataDriver.findStream` is removed (#4484). It was REQUIRED — every driver and every test double had to implement it — and documented as the read "optimized for large datasets to avoid memory overflow", while two of its three implementations awaited `find()` for the whole result set and then yielded it row by row, reaching exactly the peak it promised to avoid; the third streamed for real but was the one read in that driver that skipped `buildFindOptions`, so it dropped `query.fields`. Nothing anywhere called it, which is why a contract method could carry an inverted guarantee for this long and why ~20 test doubles could satisfy it by throwing `not implemented`. Paged `find()` is the read that exists and is enforced (its total-order guarantee is checked by the shared pagination-conformance cases); a cursor-based read is worth building when a caller asks for one, which is the honest order. A TS/API surface, never stored — one semantic TODO for driver authors, no source rewrite, and no tombstone: `DriverInterfaceSchema` describes a contract that code IMPLEMENTS and nothing ever `.parse()`d a driver, so tsc is the only channel that could carry the prescription, and it carries it where it matters — at a call site.

Finally, five keys retire because the advisory lint could never have warned about them (#4509): mapping `extractQuery` / `errorPolicy` / `batchSize`, and app `contextSelectors[].includeAll` / `.placement`. Four of the five carry schema DEFAULTS, and a default materialises at parse time — so the liveness lint cannot tell a value the author wrote from one the schema supplied, and marking them would have warned on every mapping and every selector in existence. For a key in that state removal is not the escalation after a warning; it is the only channel that ever reaches the author, which is why they ship inside the 17.0.0 window rather than after a deprecation cycle. What they claimed: `extractQuery` promised an export path no exporter implements (exports go through the ordinary query API); `errorPolicy` offered skip/abort/retry where error handling belongs to the import REQUEST; `batchSize` sized batches the write path sizes itself; `placement` offered a topbar that places nothing. `includeAll` is the one worth reading twice — it was not unread but deliberately DISOBEYED, because context selectors are mandatory-scope and an "All" row would clear the scope: on Studio's package selector that means listing the platform's own system/cloud kernel packages to a developer who scoped to their package. `STUDIO_APP` authored `includeAll: true` against a renderer that ignored it. The mapping prescription for `batchSize` deliberately offers no rename: bulk-action, connector, sync, offline, seed-loader and NoSQL-cursor `batchSize` are all live, but each is a different key sizing its own path — the same trap `datasource.retryPolicy` vs `hook`/`job` `retryPolicy` had to defuse one issue earlier.

### Mechanical (applied for you)

| Conversion | Surface | Change | Load window |
Expand All @@ -181,7 +183,7 @@ The same audit reaches the driver contract itself: `IDataDriver.findStream` is r
| `flow-node-script-config-aliases` | `flow.node.script.config` | script flow-node config keys 'functionName' → 'function', 'input' → 'inputs' (#3796) | live — protocol 17 loader accepts the old shape |
| `permission-rls-priority-removed` | `permission.rowLevelSecurity.priority` | RLS-policy key 'priority' removed (#3896 audit — policies OR-combine, so the promised conflict-resolution semantics cannot exist; dropping it changes no outcome) | retired — `migrate meta` only |
| `tool-inert-authoring-keys-removed` | `tool.category / tool.permissions / tool.active / tool.builtIn` | tool keys 'category'/'permissions'/'active'/'builtIn' removed (#3896 close-out — authorable and inert; permissions gated nothing, active:false withdrew nothing) | retired — `migrate meta` only |
| `app-dead-authoring-keys-removed` | `app.version / app.aria / app.objects / app.apis / app.sharing / app.embed / app.mobileNavigation` | app keys 'version'/'aria'/'objects'/'apis'/'sharing'/'embed'/'mobileNavigation' removed (2026-06 liveness audit — never read; sharing/embed declared a public surface no route enforced, mobileNavigation was fully unimplemented) | retired — `migrate meta` only |
| `app-dead-authoring-keys-removed` | `app.version / app.aria / app.objects / app.apis / app.sharing / app.embed / app.mobileNavigation / app.contextSelectors.includeAll / app.contextSelectors.placement` | app keys 'version'/'aria'/'objects'/'apis'/'sharing'/'embed'/'mobileNavigation' plus contextSelectors 'includeAll'/'placement' removed (liveness audits #4001, #4509 — never read; sharing/embed declared a public surface no route enforced, mobileNavigation was fully unimplemented, and includeAll was deliberately disobeyed because an 'All' row would clear a mandatory scope) | retired — `migrate meta` only |
| `field-required-notnull-explicit` | `object.fields.*.required / object.fields.*.storage.notNull` | required fields gain explicit 'storage.notNull: true' (ADR-0113 — pre-17 'required' implied the column constraint; post-17 it is only the write contract) | retired — `migrate meta` only |
| `action-inert-keys-removed` | `action.shortcut / action.bulkEnabled` | action keys 'shortcut'/'bulkEnabled' removed (#3896 close-out — no keydown path dispatches shortcuts; the multi-select toolbar reads the view's bulkActions) | retired — `migrate meta` only |
| `flow-inert-keys-removed` | `flow.active / flow.template / flow.nodes[].outputSchema / flow.errorHandling.fallbackNodeId` | flow keys 'active'/'template', node 'outputSchema' and errorHandling 'fallbackNodeId' removed (#3896 close-out — active:false never stopped a flow; status is the enforced lifecycle) | retired — `migrate meta` only |
Expand All @@ -194,6 +196,7 @@ The same audit reaches the driver contract itself: `IDataDriver.findStream` is r
| `datasource-read-replicas-removed` | `datasource.readReplicas` | datasource key 'readReplicas' removed (#4468 — no driver opened a replica connection and no query path splits reads from writes; front replicas behind one endpoint and point `config` at it) | retired — `migrate meta` only |
| `datasource-capabilities-removed` | `datasource.capabilities` | datasource key 'capabilities' removed (#4583 — eleven flags no code read; pushdown comes from the driver's own supports.*, and `readOnly` never made anything read-only) | retired — `migrate meta` only |
| `datasource-inert-blocks-removed` | `datasource.retryPolicy / datasource.healthCheck / datasource.external.label / datasource.external.requirePermission` | datasource keys 'retryPolicy'/'healthCheck' and external 'label'/'requirePermission' removed (#4583 — nothing retried, nothing probed on a schedule, and the federation label/permission were read by nobody) | retired — `migrate meta` only |
| `mapping-inert-keys-removed` | `mapping.extractQuery / mapping.errorPolicy / mapping.batchSize` | mapping keys 'extractQuery'/'errorPolicy'/'batchSize' removed (#4509 — no exporter reads a mapping, error handling belongs to the import request, and the write path sizes its own batches) | retired — `migrate meta` only |
| `datasource-config-driver-key-aliases` | `datasource.config` | datasource config keys → canonical per driver: sqlite 'file'/'database' → 'filename', postgres/mysql 'connectionString' → 'url' and 'user' → 'username', mongo 'uri' → 'url' and 'user' → 'username' (#4456 — driver-factory `??` fallback graduation) | retired — `migrate meta` only |
| `flow-node-script-branch-keys-removed` | `flow.node.script.config.actionType / flow.node.script.config.template / flow.node.script.config.recipients / flow.node.script.config.variables / flow.node.script.config.script` | script flow-node config keys 'actionType' (→ 'function' when it was shorthand for one; otherwise removed — 'email'/'slack' were logger-backed stubs that delivered nothing), plus 'template' / 'recipients' / 'variables' (fed those stubs) and 'script' (inline JS the runtime never executed) (#4343) | retired — `migrate meta` only |

Expand Down
Loading
Loading