diff --git a/.changeset/datasource-capabilities-retired.md b/.changeset/datasource-capabilities-retired.md new file mode 100644 index 0000000000..04bf1bf0ce --- /dev/null +++ b/.changeset/datasource-capabilities-retired.md @@ -0,0 +1,59 @@ +--- +"@objectstack/spec": major +"@objectstack/example-crm": patch +--- + +feat(spec)!: retire `datasource.capabilities` — eleven flags nothing read, one of them a safety claim (#4583) + +`DatasourceCapabilities` declared eleven booleans — `transactions`, seven `query*` +flags, `joins`, `fullTextSearch`, `readOnly`, `dynamicSchema` — all strict-guarded, +all read by nothing. Pushdown is decided by the runtime driver's own `supports.*` +object, a different mechanism entirely, so a datasource declaring +`queryAggregations: false` never once changed which engine path ran. The block is +removed rather than bridged: there was nothing on the other side to connect it to. + +**`readOnly` is why this is not tidy-up.** It reads as a safety property and was +authored as one — the shipped CRM example labelled a datasource "CRM Analytics Read +Replica" on the strength of it, while the datasource accepted writes exactly like the +primary. The key had already been MOVED twice toward somewhere it might be enforced, +out of `config` in #4410 and into `capabilities` in #4465, and was inert at every +address. This removes it instead of moving it a third time. + +**Removing it does not hand you a working replacement, and the rejection says so.** +The one enforced datasource-wide write gate is `external.allowWrites: false`, and it +applies only to a FEDERATED datasource — `assertWriteAllowed` returns early for a +`managed` (or unset-`schemaMode`) datasource, so that key would be equally inert for a +local database. **A managed datasource has no read-only gate at all**; that gap is +#4584, deliberately not invented here. Until it is answered, enforce read-only where +it is real: grant the connection SELECT-only at the database. + +FROM → TO: + +```ts +// before — parsed cleanly, changed nothing +defineDatasource({ + name: 'analytics', driver: 'sqlite', config: { filename: ':memory:' }, + capabilities: { readOnly: true, queryAggregations: true }, +}) + +// after — delete the block; for a FEDERATED datasource the enforced gate is: +defineDatasource({ + name: 'warehouse', driver: 'postgres', config: { … }, + schemaMode: 'external', + external: { allowWrites: false }, +}) +``` + +`os migrate meta --from 16` rewrites it automatically (ADR-0087 conversion +`datasource-capabilities-removed`). Both `DatasourceSchema` and +`DriverDefinitionSchema` are `.strict()`, so a leftover key is a loud rejection +carrying the prescription — never a silent strip. + +Also fixed: `READ_ONLY_BELONGS_ON_DATASOURCE`, the prescription every SQL driver +shares for a `readOnly` written inside `config`, was still sending authors *to* the +removed key. It now names the enforced gate and states plainly where that gate does +not apply — a prescription that lands on an inert key manufactures exactly the belief +it was meant to correct. + +The `datasource` liveness ledger drops from 20 dead properties to 9 (remaining: +`healthCheck` ×3, `retryPolicy` ×4, `external` ×2 — batches B/C/D of #4583). diff --git a/content/docs/getting-started/quick-reference.mdx b/content/docs/getting-started/quick-reference.mdx index 71f2ba9438..a11646456a 100644 --- a/content/docs/getting-started/quick-reference.mdx +++ b/content/docs/getting-started/quick-reference.mdx @@ -23,7 +23,7 @@ Core business logic and data modeling schemas. | **[Query](/docs/references/data/query)** | `query.zod.ts` | Query, QueryAST | Query AST with joins, aggregations | | **[Filter](/docs/references/data/filter)** | `filter.zod.ts` | QueryFilter, FilterCondition | Advanced filtering operators | | **[Validation](/docs/references/data/validation)** | `validation.zod.ts` | ValidationRule | Business validation rules | -| **[Datasource](/docs/references/data/datasource)** | `datasource.zod.ts` | Datasource, DatasourceCapabilities | Database connection configs | +| **[Datasource](/docs/references/data/datasource)** | `datasource.zod.ts` | Datasource, DriverDefinition | Database connection configs | | **[Analytics](/docs/references/data/analytics)** | `analytics.zod.ts` | Analytics | Data analytics and aggregation | | **[Mapping](/docs/references/data/mapping)** | `mapping.zod.ts` | FieldMapping | Field transformation mappings | | **[Hook](/docs/references/data/hook)** | `hook.zod.ts` | Hook, HookEvent | Lifecycle event hooks | diff --git a/content/docs/references/data/datasource.mdx b/content/docs/references/data/datasource.mdx index ce941d041b..93dbc1eb24 100644 --- a/content/docs/references/data/datasource.mdx +++ b/content/docs/references/data/datasource.mdx @@ -16,7 +16,7 @@ Can be a built-in driver or a plugin-contributed driver (e.g., "com.vendor.snowf ## TypeScript Usage ```typescript -import { DatasourceSchema, DatasourceCapabilities, DriverDefinitionSchema, DriverType, ExternalDatasourceSettingsSchema } from '@objectstack/spec/data'; +import { DatasourceSchema, DriverDefinitionSchema, DriverType, ExternalDatasourceSettingsSchema } from '@objectstack/spec/data'; import type { Datasource, DriverDefinition, ExternalDatasourceSettings } from '@objectstack/spec/data'; // Validate data @@ -36,7 +36,6 @@ const result = DatasourceSchema.parse(data); | **driver** | `string` | ✅ | Underlying driver type | | **config** | `Record` | ✅ | Driver specific configuration | | **pool** | `{ min: number; max: number; idleTimeoutMillis: number; connectionTimeoutMillis: number }` | optional | Connection pool settings | -| **capabilities** | `{ transactions: boolean; queryFilters: boolean; queryAggregations: boolean; querySorting: boolean; … }` | optional | Capability overrides | | **healthCheck** | `{ enabled: boolean; intervalMs: number; timeoutMs: number }` | optional | Datasource health check configuration | | **ssl** | `{ enabled: boolean; rejectUnauthorized: boolean; ca?: string; cert?: string; … }` | optional | SSL/TLS configuration for secure database connections | | **retryPolicy** | `{ maxRetries: number; baseDelayMs: number; maxDelayMs: number; backoffMultiplier: number }` | optional | Connection retry policy for transient failures | @@ -55,27 +54,6 @@ const result = DatasourceSchema.parse(data); | **_lockDocsUrl** | `string` | optional | Optional documentation link surfaced next to _lockReason. | ---- - -## DatasourceCapabilities - -### Properties - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **transactions** | `boolean` | ✅ | | -| **queryFilters** | `boolean` | ✅ | | -| **queryAggregations** | `boolean` | ✅ | | -| **querySorting** | `boolean` | ✅ | | -| **queryPagination** | `boolean` | ✅ | | -| **queryWindowFunctions** | `boolean` | ✅ | | -| **querySubqueries** | `boolean` | ✅ | | -| **joins** | `boolean` | ✅ | | -| **fullTextSearch** | `boolean` | ✅ | | -| **readOnly** | `boolean` | ✅ | | -| **dynamicSchema** | `boolean` | ✅ | | - - --- ## DriverDefinition @@ -89,7 +67,6 @@ const result = DatasourceSchema.parse(data); | **description** | `string` | optional | | | **icon** | `string` | optional | | | **configSchema** | `Record` | ✅ | JSON Schema for connection configuration | -| **capabilities** | `{ transactions: boolean; queryFilters: boolean; queryAggregations: boolean; querySorting: boolean; … }` | optional | | --- diff --git a/docs/audits/2026-07-unknown-key-strictness-ledger.md b/docs/audits/2026-07-unknown-key-strictness-ledger.md index 0976563cba..8e8abf2a22 100644 --- a/docs/audits/2026-07-unknown-key-strictness-ledger.md +++ b/docs/audits/2026-07-unknown-key-strictness-ledger.md @@ -454,7 +454,7 @@ tightening (the #4001 "sharing-rule lesson": candidates, not verdicts). | `notification.zod.ts` / `offline.zod.ts` / `report.zod.ts` | 3 ea | authorable (p) | | | `sharing.zod.ts` | 2 | authorable (p) | public-sharing config | -### `data/` — 165 sites +### `data/` — 164 sites | File | Sites | Class | Note | |---|---|---|---| @@ -465,7 +465,7 @@ tightening (the #4001 "sharing-rule lesson": candidates, not verdicts). | `field.zod.ts` | 11 | authorable | partially strict | | `filter.zod.ts` / `query.zod.ts` | 11+5 | open | query dialect — user data flows through; validated semantically elsewhere. `query.zod.ts` dropped one site in #4196: `FieldNodeSchema`'s nested-select object form was declared-but-inert and narrowed to `z.string()`, so the union's second member is gone. Four more left in #4286 with the `joins`/`windowFunctions` removals: `JoinNodeBaseSchema`, `WindowFunctionNodeSchema`, and `WindowSpecSchema`'s two blocks (outer + `frame`) were deleted with their clusters. Class unchanged | | `driver-nosql.zod.ts` / `driver.zod.ts` / `driver-sql.zod.ts` | 10+9+2 | wire | driver capability contracts | -| `datasource.zod.ts` | 9 | authorable | **strict as of #4001 data step** — all 9: `DatasourceSchema` (+ `pool` / `healthCheck` / `ssl` / `retryPolicy`), `ExternalDatasourceSettingsSchema` (+ `validation`), `DatasourceCapabilities`, `DriverDefinitionSchema`. `config` stays `z.record` **at this level** by construction (per-driver shapes), but is no longer unchecked: **#4410** made `DatasourceSchema`'s refinement parse it against the contract for the declared driver (`driver/config-registry.zod.ts`), so the openness here is a shape this level cannot express rather than the absence of one. This row used to add "the driver's own `configSchema` validates them", which was false until #4410 landed the parse site it names. #4410 extended the same parse to each `readReplicas` entry; **#4468 retired that key** — no driver ever opened a replica connection and no query path splits reads from writes, so the entries were being checked against a contract nothing would apply. Strictness makes a dropped key loud; it cannot make a slot live, and a *precisely validated* dead slot is the more convincing lie | +| `datasource.zod.ts` | 8 | authorable | **strict as of #4001 data step** — all 8: `DatasourceSchema` (+ `pool` / `healthCheck` / `ssl` / `retryPolicy`), `ExternalDatasourceSettingsSchema` (+ `validation`), `DriverDefinitionSchema`. `config` stays `z.record` **at this level** by construction (per-driver shapes), but is no longer unchecked: **#4410** made `DatasourceSchema`'s refinement parse it against the contract for the declared driver (`driver/config-registry.zod.ts`), so the openness here is a shape this level cannot express rather than the absence of one. This row used to add "the driver's own `configSchema` validates them", which was false until #4410 landed the parse site it names. #4410 extended the same parse to each `readReplicas` entry; **#4468 retired that key** — no driver ever opened a replica connection and no query path splits reads from writes, so the entries were being checked against a contract nothing would apply. Strictness makes a dropped key loud; it cannot make a slot live, and a *precisely validated* dead slot is the more convincing lie | **#4583 dropped the ninth site**: `DatasourceCapabilities` is gone — eleven flags no code read, on a block whose strictness was the clearest case of this row's own closing sentence. `readOnly` in particular was *precisely validated* and completely inert, and had been relocated twice (#4410, #4465) toward somewhere it might be enforced; the shipped CRM example called a datasource a read replica on the strength of it while writes went through. Class unchanged | `driver/memory.zod.ts` / `driver/mongo.zod.ts` / `driver/postgres.zod.ts` | 6+1+1 | authorable | The per-driver shapes for the `config` slot — what an author actually writes under `datasource.config` (`host`, `port`, `filename`). **Undeclared here until the coverage walk went recursive** (see below): a subdirectory was invisible to the gate, so these sites sat outside the map while the map reported full coverage. **Strict as of #4410**, which is also what unblocked them: this row previously read "strictness here would enforce nothing" because nothing parsed `datasource.config` against these schemas and both `*DriverSpec.configSchema` literals were `{}`. Now `DatasourceSchema` parses `config` against them, and the same schemas project onto `configSchema` and onto the Studio connection form. (#4410 also ran the parse over each `readReplicas` entry; #4468 retired that key outright — see the row above.) `postgres.zod.ts` drops a site: its `ssl` was a `boolean | {ca, cert, key, …}` union, and the object arm is gone — certificates now live in the datasource-level `ssl` block (declared, strict, and until #4410 read by nobody), leaving `config.ssl` as the on/off shorthand. That narrowing is forced by the same projection: the Studio form renders anything that is not boolean/enum/number as a TEXT INPUT, so a union here would have produced a wizard whose every `ssl` value the new gate rejects. `memory.zod.ts` keeps 6 but loses two KEYS — `indexes` / `maxRecordsPerObject`, which `InMemoryDriverConfig` has no field for, removed under ADR-0049 rather than blessed by the new gate | | `driver/mysql.zod.ts` / `driver/sqlite.zod.ts` | 1+2 | authorable | The rest of the `config` contract, added by #4410. `mysql.zod.ts` and `sqlite.zod.ts` (sqlite + sqlite-wasm) are shapes that **never existed** — both driver ids were offered by the connection form and buildable by the shared factory, with no config contract anywhere, so `driver: 'sqlite'` + a misspelled `filename` was an ephemeral `:memory:` database reported as configured. All three sites strict, same error factory as the rest of the campaign. (Their sibling `driver/common.zod.ts` holds shared enums and prescription strings and has no `z.object(` site, so the coverage gate skips it) | | `analytics.zod.ts` | 8 | mixed (p) | | diff --git a/docs/protocol-upgrade-guide.md b/docs/protocol-upgrade-guide.md index 9dfa080f4e..284d3b3747 100644 --- a/docs/protocol-upgrade-guide.md +++ b/docs/protocol-upgrade-guide.md @@ -188,6 +188,7 @@ The `script` flow node converges on its one real path (#4343). It had four ways | `stack-api-require-auth-removed` | `stack.api.requireAuth` | stack key 'api.requireAuth' removed — anonymous access is always denied; publish public surfaces by declaration (#3963) | retired — `migrate meta` only | | `flow-node-wait-timeout-keys-removed` | `flow.node.waitEventConfig` | waitEventConfig keys 'timeoutMs' (→ 'timerDuration', stringified — its only reader used it as the duration) and 'onTimeout' (removed — zero readers, so no timeout ever fired) (#4158) | retired — `migrate meta` only | | `datasource-read-replicas-removed` | `datasource.readReplicas` | datasource key 'readReplicas' removed (#4468 — no driver opened a replica connection and no query path splits reads from writes; front replicas behind one endpoint and point `config` at it) | retired — `migrate meta` only | +| `datasource-capabilities-removed` | `datasource.capabilities` | datasource key 'capabilities' removed (#4583 — eleven flags no code read; pushdown comes from the driver's own supports.*, and `readOnly` never made anything read-only) | retired — `migrate meta` only | | `flow-node-script-branch-keys-removed` | `flow.node.script.config.actionType / flow.node.script.config.template / flow.node.script.config.recipients / flow.node.script.config.variables / flow.node.script.config.script` | script flow-node config keys 'actionType' (→ 'function' when it was shorthand for one; otherwise removed — 'email'/'slack' were logger-backed stubs that delivered nothing), plus 'template' / 'recipients' / 'variables' (fed those stubs) and 'script' (inline JS the runtime never executed) (#4343) | retired — `migrate meta` only | ### Semantic (delegated to you, with acceptance criteria) diff --git a/examples/app-crm/src/datasources/crm.datasource.ts b/examples/app-crm/src/datasources/crm.datasource.ts index 14b5b58253..a3af9cc680 100644 --- a/examples/app-crm/src/datasources/crm.datasource.ts +++ b/examples/app-crm/src/datasources/crm.datasource.ts @@ -21,21 +21,26 @@ export const CrmDatasource = defineDatasource({ }); /** - * Read-replica for analytics queries — demonstrates datasource routing. + * Second datasource for analytics queries — demonstrates datasource routing. * - * `readOnly` is a datasource CAPABILITY, not sqlite config. It sat inside - * `config` here until #4410 gave that slot a gate — a key no driver read, so - * the "read replica" was writable while every signal said it was not. + * This used to declare `capabilities: { readOnly: true }` and call itself a read + * replica. It was neither: the key had no reader, so the "read replica" accepted + * writes exactly like the primary — the third spelling of the same defect, after + * the same claim sat inertly in `config` (#4410) and then in `capabilities` + * (#4465). #4583 removed the key rather than move it a fourth time. + * + * The label no longer promises read-only, because nothing here can deliver it: + * `external.allowWrites: false` is the one enforced write gate and it applies + * only to FEDERATED datasources, while this one is local and managed. Whether a + * managed datasource should have a read-only gate at all is #4584 — until that + * is answered, the honest demo is routing, not a safety claim. */ export const CrmAnalyticsDatasource = defineDatasource({ name: 'crm_analytics', - label: 'CRM Analytics Read Replica', + label: 'CRM Analytics', driver: 'sqlite', config: { filename: ':memory:', }, - capabilities: { - readOnly: true, - }, active: true, }); diff --git a/packages/cli/src/utils/lint-liveness-properties.test.ts b/packages/cli/src/utils/lint-liveness-properties.test.ts index ec70d3e388..6eb6af7211 100644 --- a/packages/cli/src/utils/lint-liveness-properties.test.ts +++ b/packages/cli/src/utils/lint-liveness-properties.test.ts @@ -195,42 +195,47 @@ describe('lintLivenessProperties', () => { // the type that most needed it: 20 of its 43 props have no runtime consumer, // and until #4487 nothing told an author so. - it('warns on the dead datasource blocks — capabilities / healthCheck / retryPolicy (#4487)', () => { + it('warns on the dead datasource blocks that remain — healthCheck / retryPolicy (#4487)', () => { + // `capabilities` left this list in #4583: the block was REMOVED from the + // schema, so an author who writes it now gets a hard parse rejection with a + // prescription — a stronger signal than a lint warning, and the reason its + // ledger rows are gone rather than flipped. healthCheck / retryPolicy are + // still authorable and still dead (batches B and C of #4583). const findings = lintLivenessProperties({ datasources: [{ name: 'warehouse', driver: 'postgres', config: { host: 'db.internal', database: 'analytics' }, - capabilities: { transactions: true, queryAggregations: true }, healthCheck: { enabled: true, intervalMs: 30000 }, retryPolicy: { maxRetries: 5, baseDelayMs: 1000 }, }], }); const msgs = paths(findings); - expect(msgs.some((m) => m.includes('capabilities.transactions'))).toBe(true); - expect(msgs.some((m) => m.includes('capabilities.queryAggregations'))).toBe(true); expect(msgs.some((m) => m.includes('healthCheck.enabled'))).toBe(true); expect(msgs.some((m) => m.includes('healthCheck.intervalMs'))).toBe(true); expect(msgs.some((m) => m.includes('retryPolicy.maxRetries'))).toBe(true); expect(msgs.some((m) => m.includes('retryPolicy.baseDelayMs'))).toBe(true); + // The removed block must no longer be reported by the lint at all. + expect(msgs.some((m) => m.includes('capabilities'))).toBe(false); }); - // The entry the whole audit was worth doing for. `capabilities.readOnly` reads - // as a safety switch and gates nothing, and two shipped prescriptions pointed - // authors AT it until #4487. The hint has to name the gate that IS enforced, - // or the warning just relocates the author's confusion. - it('warns on capabilities.readOnly and names the real write gate (#4487)', () => { + // The entry the whole audit was worth doing for. `capabilities.readOnly` read + // as a safety switch and gated nothing — a datasource labelled a read replica + // took writes like any other. #4583 REMOVED it rather than warn about it for + // another release, so the check moved up a level: the lint no longer has an + // opinion because the schema refuses the key outright. The prescription that + // replaces the hint is asserted in `packages/spec` (datasource.test.ts), where + // it can also assert the part a hint could not carry — that the enforced gate + // does NOT cover managed datasources (#4584). + it('no longer warns on capabilities.readOnly — the key is gone, not merely flagged (#4583)', () => { const findings = lintLivenessProperties({ datasources: [{ name: 'reporting', driver: 'postgres', config: { host: 'ro.internal', database: 'reporting' }, - capabilities: { readOnly: true }, }], }); - const hit = findings.find((f) => f.message.includes('capabilities.readOnly')); - expect(hit).toBeDefined(); - expect(hit!.hint).toMatch(/allowWrites/); + expect(findings.some((f) => f.message.includes('capabilities'))).toBe(false); }); it('stays silent on a datasource that only sets live properties (#4487)', () => { diff --git a/packages/spec/api-surface.json b/packages/spec/api-surface.json index 1a9b2fb098..0053e3eb12 100644 --- a/packages/spec/api-surface.json +++ b/packages/spec/api-surface.json @@ -290,8 +290,6 @@ "DataTypeMapping (type)", "DataTypeMappingSchema (const)", "Datasource (type)", - "DatasourceCapabilities (const)", - "DatasourceCapabilitiesType (type)", "DatasourceInput (type)", "DatasourceSchema (const)", "DateGranularity (const)", diff --git a/packages/spec/authorable-surface.json b/packages/spec/authorable-surface.json index 1822757c91..565e43d577 100644 --- a/packages/spec/authorable-surface.json +++ b/packages/spec/authorable-surface.json @@ -1,5 +1,5 @@ { - "description": "Ratchet of every AUTHORABLE key in the spec — what a metadata author may write, which for this platform IS the third-party API. Auto-updated on additions (commit the change). A key that disappears without a tombstone fails gen:schema, because these schemas are not .strict() and Zod would silently strip it. \"[RETIRED]\" marks a tombstoned key that still rejects with an upgrade prescription. See #3855, ADR-0059 §5.", + "description": "Ratchet of every AUTHORABLE key in the spec \u2014 what a metadata author may write, which for this platform IS the third-party API. Auto-updated on additions (commit the change). A key that disappears without a tombstone fails gen:schema, because these schemas are not .strict() and Zod would silently strip it. \"[RETIRED]\" marks a tombstoned key that still rejects with an upgrade prescription. See #3855, ADR-0059 \u00a75.", "keys": [ "ai/AIModelConfig:maxTokens", "ai/AIModelConfig:model", @@ -3234,7 +3234,6 @@ "data/Datasource:_provenance", "data/Datasource:active", "data/Datasource:autoConnect", - "data/Datasource:capabilities", "data/Datasource:config", "data/Datasource:description", "data/Datasource:driver", @@ -3247,17 +3246,6 @@ "data/Datasource:retryPolicy", "data/Datasource:schemaMode", "data/Datasource:ssl", - "data/DatasourceCapabilities:dynamicSchema", - "data/DatasourceCapabilities:fullTextSearch", - "data/DatasourceCapabilities:joins", - "data/DatasourceCapabilities:queryAggregations", - "data/DatasourceCapabilities:queryFilters", - "data/DatasourceCapabilities:queryPagination", - "data/DatasourceCapabilities:querySorting", - "data/DatasourceCapabilities:querySubqueries", - "data/DatasourceCapabilities:queryWindowFunctions", - "data/DatasourceCapabilities:readOnly", - "data/DatasourceCapabilities:transactions", "data/Dimension:description", "data/Dimension:granularities", "data/Dimension:label", @@ -3332,7 +3320,6 @@ "data/DriverConfig:name", "data/DriverConfig:poolConfig", "data/DriverConfig:type", - "data/DriverDefinition:capabilities", "data/DriverDefinition:configSchema", "data/DriverDefinition:description", "data/DriverDefinition:icon", diff --git a/packages/spec/docs-import-surface.baseline.json b/packages/spec/docs-import-surface.baseline.json index 2bab818b39..b2ea186696 100644 --- a/packages/spec/docs-import-surface.baseline.json +++ b/packages/spec/docs-import-surface.baseline.json @@ -43,7 +43,6 @@ "data/DataEngineInsertRequest — no type export", "data/DataEngineUpdateRequest — no type export", "data/DataEngineVectorFindRequest — no type export", - "data/DatasourceCapabilities — no type export", "data/DateGranularity — no type export", "data/DateMacroPlaceholder — no type export", "data/DimensionType — no type export", diff --git a/packages/spec/json-schema.manifest.json b/packages/spec/json-schema.manifest.json index 3717c980ad..b463a48906 100644 --- a/packages/spec/json-schema.manifest.json +++ b/packages/spec/json-schema.manifest.json @@ -1,5 +1,5 @@ { - "description": "Ratchet manifest of every JSON Schema emitted by scripts/build-schemas.ts. Auto-appended when new schemas are added (commit the change). A listed schema that a build no longer emits fails gen:schema — remove a key ONLY for a deliberate retirement. See #2978.", + "description": "Ratchet manifest of every JSON Schema emitted by scripts/build-schemas.ts. Auto-appended when new schemas are added (commit the change). A listed schema that a build no longer emits fails gen:schema \u2014 remove a key ONLY for a deliberate retirement. See #2978.", "schemas": [ "ai/AIModelConfig", "ai/AIUsageRecord", @@ -718,7 +718,6 @@ "data/DataEngineVectorFindRequest", "data/DataTypeMapping", "data/Datasource", - "data/DatasourceCapabilities", "data/DateGranularity", "data/DateMacroPlaceholder", "data/DateMacroToken", diff --git a/packages/spec/liveness/README.md b/packages/spec/liveness/README.md index 929ee9e667..edc084c42a 100644 --- a/packages/spec/liveness/README.md +++ b/packages/spec/liveness/README.md @@ -506,7 +506,7 @@ for t, v in r['types'].items(): | report | 21 | 0 | 0 | – | dataset-bound (ADR-0021); the aria/performance LEDGER entries were stale — the keys left the schema in the report-liveness close-out; deleted 2026-07-30 as hygiene. Audit-era `chart` DEAD superseded (framework#1890 / #3441) | | dashboard | 18 | 0 | 2 | – | ADR-0021 dataset widgets (#3251; DashboardWidgetSchema `.strict()`); `aria`/`performance` (and widget `performance` + PerformanceConfigSchema) REMOVED 2026-07-30 (#3896 close-out sweep — no renderer applied any of them); audit-era `globalFilters`/`dateRange` DEAD superseded (framework#2501) | | query | 16 | 1 | 4 | – | **not a metadata type** — the REQUEST surface (`QuerySchema`: client SDK QueryBuilder output; the `POST /data/:object/query` body), governed via `SPEC_ONLY_SCHEMAS` (#4286). The gate's one-level walk resolves 1 experimental; the 7 marker-experimental search affordances sit one level deeper, below the walk — resolved from `[EXPERIMENTAL — not enforced]` describe markers, not ledger entries (search `fuzzy`/`operator`/`boost`/`minScore`/`language`/`highlight` + `aggregations[].filter` — declared engine affordances no executor receives). The #4286 sweep closed out same-release: `having` ENFORCED 2026-07-31 (engine-side post-aggregation filter, both paths; was finding 1); dead 4 = the tombstoned removals `joins`/`windowFunctions`/`cursor`/`distinct` — REMOVED 2026-07-31 (retiredKey keeps each in the walked shape so the rows stay; protocol-17 semantic migrations; the JoinNode + WindowFunctionNode clusters and the `QueryBuilder.cursor()`/`.distinct()` producers deleted with their keys; `distinct`'s mis-wired REST count suppression deleted too — finding 2) | -| datasource | 23 | – | 20 | – | seeded 2026-08-01 (#4487) — the **highest dead ratio of any governed type** (20 of 43), and it was ungoverned until now, which is not a coincidence: #4410/#4465/#4481 found six inert keys here by hand, two security-shaped (`schemaMode` left an external DB constructible as `managed` with DDL ungated; `ssl` configured nothing while looking configured). Dead set = `capabilities.*` (all 11 — the engine gates pushdown on the runtime driver's `supports.*` object, a non-overlapping vocabulary), `healthCheck.*` (3 — nothing schedules a datasource probe; the 20 `healthCheck` hits in the repo all belong to the PLUGIN health monitor and other surfaces), `retryPolicy.*` (4 — `retryPolicy` IS enforced on `hook` and `job`, which is what makes this one read alive; the shapes differ), `external.label`, `external.requirePermission`. **`capabilities.readOnly` is the one to know**: it reads as a safety switch, gates nothing, and two shipped prescriptions pointed authors at it until #4487 — `external.allowWrites: false` is the enforced write gate. `config` is a `z.record`, so its per-driver keys sit outside the walk (recorded in the entry's note, not silently skipped) | +| datasource | 30 | 0 | 9 | 0 | seeded 2026-08-01 (#4487) — the **highest dead ratio of any governed type** (20 of 43), and it was ungoverned until now, which is not a coincidence: #4410/#4465/#4481 found six inert keys here by hand, two security-shaped (`schemaMode` left an external DB constructible as `managed` with DDL ungated; `ssl` configured nothing while looking configured). Dead set = `capabilities.*` (all 11 — the engine gates pushdown on the runtime driver's `supports.*` object, a non-overlapping vocabulary), `healthCheck.*` (3 — nothing schedules a datasource probe; the 20 `healthCheck` hits in the repo all belong to the PLUGIN health monitor and other surfaces), `retryPolicy.*` (4 — `retryPolicy` IS enforced on `hook` and `job`, which is what makes this one read alive; the shapes differ), `external.label`, `external.requirePermission`. **`capabilities.readOnly` is the one to know**: it reads as a safety switch, gates nothing, and two shipped prescriptions pointed authors at it until #4487 — `external.allowWrites: false` is the enforced write gate. `config` is a `z.record`, so its per-driver keys sit outside the walk (recorded in the entry's note, not silently skipped) **批 A CLOSED 2026-08-02 (#4583)**: the `capabilities` block — 11 flags, every one dead and authorWarn'd — was REMOVED rather than bridged; pushdown comes from the runtime driver's own `supports.*`, so there was nothing to connect it to. Its rows are deleted (strict-removal route), which is why dead falls 20 → 9. `readOnly` was the reason the audit was worth doing: it read as a safety switch, gated nothing, and had already been MOVED twice toward somewhere it might be enforced (#4410, #4465) — the shipped CRM example called a datasource a read replica on the strength of it while the datasource took writes. Removing it does NOT hand the author a working alternative: `external.allowWrites` only gates FEDERATED datasources, so a managed one has no read-only gate at all (#4584). Remaining 9 = healthCheck ×3 + retryPolicy ×4 + external ×2, batches B/C/D of #4583 | | webhook | 11 | 0 | 0 | – | **not a registered metadata type** — governed via the gate's spec-only schema override (`SPEC_ONLY_SCHEMAS`), not `getMetadataTypeSchema`; folding it onto the registry is the #3490 reassessment. This row once read 0/1/16 ("the ENTIRE authoring surface is dead", #3461) and both halves of that were CLOSED same-quarter: #3489 built the materializer bridge (authored `webhooks:` entries now land as `sys_webhook` dispatcher rows) and #3494 pruned the aspirational props outright — so the surviving surface is fully live. Kept in the table as the worked example that a dead verdict is a worklist entry, not a tombstone: enforce-or-remove resolved this one by ENFORCING | | app | 45 | – | 14 | – | seeded 2026-08-01 (#4488). Dead 14 = the seven #4142 `retiredKey` tombstones (version/aria/objects/apis/sharing/embed/mobileNavigation — rows stay while the tombstones hold the keys in the walked shape) + `homePageId` (the landing IS the first nav item; root landing follows `isDefault` routing) + the **fail-open area gates** `areas.visible` / `areas.requiredPermissions` (nothing evaluates them, while the per-ITEM siblings are enforced server- and client-side — the audit's most important app finding, both authorWarn'd) + `areas.order`/`description` + selector `includeAll` (deliberately ignored: selectors are mandatory-scope; an "All" would leak system metadata) and `placement`. Nav walk covers the union's `object` variant; other variants hand-verified live except the `actionDef` dispatch gap (renders, but no shipped shell passes `onAction`) — #4509 | | book | 13 | – | 2 | – | seeded 2026-08-01 (#4488). ADR-0046 §6 spine; `audience` is ENFORCED and fail-closed (tree 401/403 + per-doc effective-audience union on both list and tree). Dead 2 = BOTH inline `translations` maps (book-level and per-group): no resolver reads them and the bundle translator doesn't cover `book` — the trap is that `doc.translations` two files over works on every read path. Also recorded: the `include: { tag }` rule variant can never match (DocSchema declares no `tags`) | diff --git a/packages/spec/liveness/datasource.json b/packages/spec/liveness/datasource.json index 3b962257d0..70f23db9a9 100644 --- a/packages/spec/liveness/datasource.json +++ b/packages/spec/liveness/datasource.json @@ -1,6 +1,6 @@ { "type": "datasource", - "_note": "DatasourceSchema. Consumers: @objectstack/service-datasource (DatasourceConnectionService.toSpec → DatasourceConnectionSpec → createDefaultDatasourceDriverFactory), @objectstack/objectql (engine.ts federation write gate), @objectstack/runtime (external-validation-plugin). Seeded 2026-08-01 (#4487) after #4465/#4481 found six inert keys BY HAND on a type no gate governed. Method: the authoritative boundary is what crosses into `ConnectableDatasource` (datasource-connection-service.ts:45-74) and `DatasourceConnectionSpec` (contracts/datasource-driver-factory.ts:25-59) — a block on neither reaches no driver. objectui's DatasourcePreview renders `pool`/`ssl`/`retryPolicy`/`healthCheck` as SideBlocks and is NOT counted as evidence for any entry (see README, 'An authoring/preview renderer is NOT a runtime consumer' — the #4481 precedent was exactly this). Framework provenance/lock fields auto-live.", + "_note": "DatasourceSchema. Consumers: @objectstack/service-datasource (DatasourceConnectionService.toSpec → DatasourceConnectionSpec → createDefaultDatasourceDriverFactory), @objectstack/objectql (engine.ts federation write gate), @objectstack/runtime (external-validation-plugin). Seeded 2026-08-01 (#4487) after #4465/#4481 found six inert keys BY HAND on a type no gate governed. Method: the authoritative boundary is what crosses into `ConnectableDatasource` (datasource-connection-service.ts:45-74) and `DatasourceConnectionSpec` (contracts/datasource-driver-factory.ts:25-59) — a block on neither reaches no driver. objectui's DatasourcePreview renders `pool`/`ssl`/`retryPolicy`/`healthCheck` as SideBlocks and is NOT counted as evidence for any entry (see README, 'An authoring/preview renderer is NOT a runtime consumer' — the #4481 precedent was exactly this). Framework provenance/lock fields auto-live. RETIREMENT 2026-08-02 (#4583): the whole `capabilities` block (11 flags, every one dead and authorWarn'd) was REMOVED from the schema rather than bridged — pushdown is decided by the runtime driver's own `supports.*`, a different mechanism, so there was nothing to connect it to. Its rows are deleted rather than flipped, per the strict-removal route (the keys left the walked shape, so a kept row would read as an ORPHAN). `readOnly` is called out separately in the tombstone: deleting it does NOT hand the author a working alternative, because `external.allowWrites` only gates FEDERATED datasources — a managed datasource has no read-only gate at all, which is #4584 rather than something this removal invented.", "props": { "name": { "status": "live", @@ -45,32 +45,23 @@ } } }, - "capabilities": { + "healthCheck": { "children": { - "transactions": { "status": "dead", "authorWarn": true, "authorHint": "Delete it. The engine gates pushdown on the runtime driver's own `supports.*` object (objectql/src/engine.ts:3671, :4529, :4810 — `autonumber`, `queryDateGranularity`, `batchSchemaSync`), which is a different mechanism with a different vocabulary. Nothing reads this block." }, - "queryFilters": { "status": "dead", "authorWarn": true, "authorHint": "Delete it — see `capabilities.transactions`. Filter support is not negotiated through datasource metadata." }, - "queryAggregations": { "status": "dead", "authorWarn": true, "authorHint": "Delete it — see `capabilities.transactions`. Whether aggregation runs in SQL or in memory is decided by the driver's own code path, never by this flag." }, - "querySorting": { "status": "dead", "authorWarn": true, "authorHint": "Delete it — see `capabilities.transactions`." }, - "queryPagination": { "status": "dead", "authorWarn": true, "authorHint": "Delete it — see `capabilities.transactions`." }, - "queryWindowFunctions": { "status": "dead", "authorWarn": true, "authorHint": "Delete it — see `capabilities.transactions`. Window functions were themselves retired from the query surface in #4286." }, - "querySubqueries": { "status": "dead", "authorWarn": true, "authorHint": "Delete it — see `capabilities.transactions`." }, - "joins": { "status": "dead", "authorWarn": true, "authorHint": "Delete it — see `capabilities.transactions`. `query.joins` was retired in #4286; related-record retrieval is `expand`." }, - "fullTextSearch": { "status": "dead", "authorWarn": true, "authorHint": "Delete it — see `capabilities.transactions`. Search capability is deployment/locale-gated in the search companion, not declared here." }, - "readOnly": { + "enabled": { "status": "dead", "authorWarn": true, - "authorHint": "Delete it. It does NOT make a datasource read-only — no write path consults it. The enforced datasource-wide write gate is `external.allowWrites: false` (objectql/src/engine.ts:620), which requires `schemaMode !== 'managed'`.", - "note": "The most dangerous entry in this ledger and the reason it was seeded. `readOnly` reads as a safety property, and until this PR TWO shipped prescriptions pointed authors AT it: the `externalSettingsUnknownKeyError` guidance in datasource.zod.ts and the #4465 changeset's relocation table both offered `capabilities.readOnly` as the place to 'describe the driver'. Both were corrected in #4487. An author following that advice believed they had marked a datasource non-writable and had not." + "authorHint": "Delete it. No health-check loop reads this block. Connection liveness is probed on demand via the driver handle's `ping()` / `checkHealth()` (contracts/datasource-driver-factory.ts:88-92), which the admin service calls for `testConnection` — not on any interval this could enable." }, - "dynamicSchema": { "status": "dead", "authorWarn": true, "authorHint": "Delete it — see `capabilities.transactions`. Whether a driver is schemaless is a property of the driver, and the drivers that are (mongo, memory) behave that way unconditionally." } - }, - "note": "All 11 dead, verified 2026-08-01 by closing the call graph in both directions: (1) `capabilities` is absent from ConnectableDatasource AND DatasourceConnectionSpec, so it cannot reach a driver; (2) grepping each flag name across the monorepo returns only packages/spec — the schema's own declaration, its alias table, and the `*DriverSpec` literals in data/driver/*.zod.ts. The engine's real capability seam is `driver.supports?.*` on the runtime driver OBJECT, whose keys (`autonumber`, `batchSchemaSync`, `queryDateGranularity`) do not overlap with this vocabulary at all. `having-filter.ts:13` states the position plainly in a comment: 'SQL pushdown can come later behind a driver capability flag' — i.e. the mechanism this block describes is not built." - }, - "healthCheck": { - "children": { - "enabled": { "status": "dead", "authorWarn": true, "authorHint": "Delete it. No health-check loop reads this block. Connection liveness is probed on demand via the driver handle's `ping()` / `checkHealth()` (contracts/datasource-driver-factory.ts:88-92), which the admin service calls for `testConnection` — not on any interval this could enable." }, - "intervalMs": { "status": "dead", "authorWarn": true, "authorHint": "Delete it. Nothing schedules a datasource health check, so there is no interval to set. The only recurring datasource timer is `external.validation.checkIntervalMs` (schema-drift checking, a different concern)." }, - "timeoutMs": { "status": "dead", "authorWarn": true, "authorHint": "Delete it. See `healthCheck.intervalMs` — there is no probe loop for this to bound." } + "intervalMs": { + "status": "dead", + "authorWarn": true, + "authorHint": "Delete it. Nothing schedules a datasource health check, so there is no interval to set. The only recurring datasource timer is `external.validation.checkIntervalMs` (schema-drift checking, a different concern)." + }, + "timeoutMs": { + "status": "dead", + "authorWarn": true, + "authorHint": "Delete it. See `healthCheck.intervalMs` — there is no probe loop for this to bound." + } }, "note": "All 3 dead, verified 2026-08-01. `healthCheck` is absent from ConnectableDatasource and DatasourceConnectionSpec. Every `healthCheck` hit in the monorepo belongs to a DIFFERENT surface — the PLUGIN health monitor (core/src/health-monitor.ts, core/src/plugin-loader.ts:316), the AI model registry, the integration connector, `StartupOrchestratorOptions.healthCheck`. Name collision, not a consumer. Easy to mis-verify: a bare grep for 'healthCheck' returns 20 hits and none of them is this block." }, @@ -102,10 +93,26 @@ }, "retryPolicy": { "children": { - "maxRetries": { "status": "dead", "authorWarn": true, "authorHint": "Delete it. No connect or query path retries on this block. Connection failure handling is the boot policy in datasource-connection-service.ts (degraded boot / `bootCritical` fail-fast), which does not retry on a schedule. Do not confuse this with `hook.retryPolicy` (enforced, objectql/src/hook-wrappers.ts:105) or `job.retryPolicy` (enforced, runtime/src/app-plugin.ts:791) — same key name, different types, different shapes." }, - "baseDelayMs": { "status": "dead", "authorWarn": true, "authorHint": "Delete it — see `retryPolicy.maxRetries`. Note this key does not even exist on the two retryPolicy blocks that ARE enforced: `hook.retryPolicy` spells its delay `backoffMs`." }, - "maxDelayMs": { "status": "dead", "authorWarn": true, "authorHint": "Delete it — see `retryPolicy.maxRetries`." }, - "backoffMultiplier": { "status": "dead", "authorWarn": true, "authorHint": "Delete it — see `retryPolicy.maxRetries`." } + "maxRetries": { + "status": "dead", + "authorWarn": true, + "authorHint": "Delete it. No connect or query path retries on this block. Connection failure handling is the boot policy in datasource-connection-service.ts (degraded boot / `bootCritical` fail-fast), which does not retry on a schedule. Do not confuse this with `hook.retryPolicy` (enforced, objectql/src/hook-wrappers.ts:105) or `job.retryPolicy` (enforced, runtime/src/app-plugin.ts:791) — same key name, different types, different shapes." + }, + "baseDelayMs": { + "status": "dead", + "authorWarn": true, + "authorHint": "Delete it — see `retryPolicy.maxRetries`. Note this key does not even exist on the two retryPolicy blocks that ARE enforced: `hook.retryPolicy` spells its delay `backoffMs`." + }, + "maxDelayMs": { + "status": "dead", + "authorWarn": true, + "authorHint": "Delete it — see `retryPolicy.maxRetries`." + }, + "backoffMultiplier": { + "status": "dead", + "authorWarn": true, + "authorHint": "Delete it — see `retryPolicy.maxRetries`." + } }, "note": "All 4 dead, verified 2026-08-01. Absent from ConnectableDatasource and DatasourceConnectionSpec. The trap here is the name: `retryPolicy` IS enforced on `hook` and on `job`, so a grep for the key looks alive and a reader who stops there concludes the datasource one works too. The shapes differ — hook uses `{maxRetries, backoffMs}`, this declares `{maxRetries, baseDelayMs, maxDelayMs, backoffMultiplier}` — which is itself the tell that nothing reads both." }, diff --git a/packages/spec/spec-changes.json b/packages/spec/spec-changes.json index e94bfffc0a..32530b25fd 100644 --- a/packages/spec/spec-changes.json +++ b/packages/spec/spec-changes.json @@ -212,6 +212,12 @@ "conversionId": "datasource-read-replicas-removed", "toMajor": 17 }, + { + "surface": "datasource.capabilities", + "to": "datasource key 'capabilities' removed (#4583 — eleven flags no code read; pushdown comes from the driver's own supports.*, and `readOnly` never made anything read-only)", + "conversionId": "datasource-capabilities-removed", + "toMajor": 17 + }, { "surface": "flow.node.script.config.actionType / flow.node.script.config.template / flow.node.script.config.recipients / flow.node.script.config.variables / flow.node.script.config.script", "to": "script flow-node config keys 'actionType' (→ 'function' when it was shorthand for one; otherwise removed — 'email'/'slack' were logger-backed stubs that delivered nothing), plus 'template' / 'recipients' / 'variables' (fed those stubs) and 'script' (inline JS the runtime never executed) (#4343)", @@ -707,6 +713,12 @@ "conversionId": "datasource-read-replicas-removed", "toMajor": 17 }, + { + "surface": "datasource.capabilities", + "to": "datasource key 'capabilities' removed (#4583 — eleven flags no code read; pushdown comes from the driver's own supports.*, and `readOnly` never made anything read-only)", + "conversionId": "datasource-capabilities-removed", + "toMajor": 17 + }, { "surface": "flow.node.script.config.actionType / flow.node.script.config.template / flow.node.script.config.recipients / flow.node.script.config.variables / flow.node.script.config.script", "to": "script flow-node config keys 'actionType' (→ 'function' when it was shorthand for one; otherwise removed — 'email'/'slack' were logger-backed stubs that delivered nothing), plus 'template' / 'recipients' / 'variables' (fed those stubs) and 'script' (inline JS the runtime never executed) (#4343)", diff --git a/packages/spec/src/conversions/registry.ts b/packages/spec/src/conversions/registry.ts index 80b3091129..22013cf159 100644 --- a/packages/spec/src/conversions/registry.ts +++ b/packages/spec/src/conversions/registry.ts @@ -2230,6 +2230,63 @@ const flowNodeWaitTimeoutKeysRemoved: MetadataConversion = { }, }; +/** + * `datasource.capabilities` removed (protocol 17, #4583). + * + * Eleven boolean flags, declared and strict-guarded, read by nothing. Pushdown + * is decided by the runtime driver's own `supports.*` object — a different + * mechanism entirely — so a datasource declaring `queryAggregations: false` + * never once changed which engine path ran. + * + * `readOnly` is why this one is not merely tidy-up. It reads as a safety + * property and was authored as one: the shipped CRM example labelled a + * datasource "Read Replica" on the strength of it, while the datasource + * accepted writes exactly like the primary. The key had already been MOVED + * twice toward somewhere it might be enforced — out of `config` in #4410, into + * `capabilities` in #4465 — and was inert at every address. This removes it + * instead of moving it a third time. + * + * Deliberately NOT converted to `external.allowWrites: false`, which is the + * enforced gate and the obvious-looking target: it applies only to FEDERATED + * datasources (`schemaMode` other than `managed`), so rewriting a managed + * datasource that way would produce a key that is equally inert for that author + * — the exact defect being retired, laundered through a migration. A managed + * datasource has no read-only gate at all (#4584); the honest conversion is a + * delete plus a rejection message that says so. + * + * `retiredFromLoadPath`: both shapes are `.strict()` and reject the key with + * its prescription (`RETIRED_CAPABILITIES`). + */ +const datasourceCapabilitiesRemoved: MetadataConversion = { + id: 'datasource-capabilities-removed', + toMajor: 17, + retiredFromLoadPath: true, + surface: 'datasource.capabilities', + summary: "datasource key 'capabilities' removed (#4583 — eleven flags no code read; pushdown comes from the driver's own supports.*, and `readOnly` never made anything read-only)", + apply(stack, emit) { + return mapCollection(stack, 'datasources', (ds, path) => stripKeys(ds, ['capabilities'], emit, path)); + }, + fixture: { + before: { + datasources: [{ + name: 'analytics', + driver: 'sqlite', + config: { filename: ':memory:' }, + capabilities: { readOnly: true, queryAggregations: true }, + }], + }, + // One notice per datasource, not per flag: the block is what was removed. + after: { + datasources: [{ + name: 'analytics', + driver: 'sqlite', + config: { filename: ':memory:' }, + }], + }, + expectedNotices: 1, + }, +}; + /** * `datasource.readReplicas` — replica connections nothing ever opened (#4468). * @@ -2476,6 +2533,7 @@ export const CONVERSIONS_BY_MAJOR: Readonly { @@ -18,77 +16,51 @@ describe('DriverType', () => { }); }); -describe('DatasourceCapabilities', () => { - it('should accept empty capabilities with defaults', () => { - const capabilities = DatasourceCapabilities.parse({}); - - expect(capabilities.transactions).toBe(false); - expect(capabilities.queryFilters).toBe(false); - expect(capabilities.queryAggregations).toBe(false); - expect(capabilities.querySorting).toBe(false); - expect(capabilities.queryPagination).toBe(false); - expect(capabilities.queryWindowFunctions).toBe(false); - expect(capabilities.querySubqueries).toBe(false); - expect(capabilities.joins).toBe(false); - expect(capabilities.fullTextSearch).toBe(false); - expect(capabilities.readOnly).toBe(false); - expect(capabilities.dynamicSchema).toBe(false); - }); - - it('should accept full capabilities for SQL database', () => { - const capabilities = DatasourceCapabilities.parse({ - transactions: true, - queryFilters: true, - queryAggregations: true, - querySorting: true, - queryPagination: true, - queryWindowFunctions: true, - querySubqueries: true, - joins: true, - fullTextSearch: true, - readOnly: false, - dynamicSchema: false, - }); - - expect(capabilities.transactions).toBe(true); - expect(capabilities.queryWindowFunctions).toBe(true); - }); - - it('should accept limited capabilities for NoSQL database', () => { - const capabilities = DatasourceCapabilities.parse({ - transactions: false, - queryFilters: true, - queryAggregations: true, - querySorting: true, - queryPagination: true, - joins: false, - dynamicSchema: true, +describe('datasource.capabilities — RETIRED (#4583)', () => { + // These used to be five tests asserting the eleven flags parsed and defaulted. + // They did, faithfully, for a block no runtime ever read — the shape #4001 + // named: a schema that is loose (or here, merely unread) eventually grows a + // test asserting that state, and the assertion reads like intent. + // + // The replacement asserts the removal, and that the author is told where the + // enforced gate actually is. + it('rejects a `capabilities` block on a datasource', () => { + const result = DatasourceSchema.safeParse({ + name: 'analytics', + driver: 'sqlite', + config: { filename: ':memory:' }, + capabilities: { readOnly: true }, }); - expect(capabilities.joins).toBe(false); - expect(capabilities.dynamicSchema).toBe(true); + expect(result.success).toBe(false); }); - it('should accept read-only capabilities', () => { - const capabilities = DatasourceCapabilities.parse({ + it('tells a `readOnly` author that no managed read-only gate exists, rather than renaming the key', () => { + const result = DatasourceSchema.safeParse({ + name: 'analytics', + driver: 'sqlite', + config: { filename: ':memory:' }, readOnly: true, - queryFilters: true, - querySorting: true, }); - expect(capabilities.readOnly).toBe(true); + expect(result.success).toBe(false); + const msg = JSON.stringify(result.error?.issues ?? []); + // The prescription must name the enforced gate AND its limit. Naming only + // `external.allowWrites` would repeat the defect this removal closes: the + // author's datasource is managed, so that key would not gate it either. + expect(msg).toContain('external.allowWrites'); + expect(msg).toMatch(/managed/); }); - it('should accept capabilities for Excel/CSV', () => { - const capabilities = DatasourceCapabilities.parse({ - transactions: false, - queryFilters: true, - querySorting: true, - joins: false, - readOnly: false, + it('rejects `capabilities` on a driver definition too', () => { + const result = DriverDefinitionSchema.safeParse({ + id: 'sqlite', + label: 'SQLite', + configSchema: {}, + capabilities: { transactions: true }, }); - expect(capabilities.transactions).toBe(false); + expect(result.success).toBe(false); }); }); @@ -128,15 +100,9 @@ describe('DriverDefinitionSchema', () => { password: { type: 'string' }, }, }, - capabilities: { - transactions: true, - queryFilters: true, - queryAggregations: true, - }, }); expect(driver.description).toBe('PostgreSQL database driver'); - expect(driver.capabilities).toBeDefined(); }); it('should accept MongoDB driver definition', () => { @@ -229,18 +195,12 @@ describe('DatasourceSchema', () => { password: '${DB_PASSWORD}', ssl: true, }, - capabilities: { - transactions: true, - queryFilters: true, - queryAggregations: true, - }, description: 'Main production PostgreSQL database', active: true, }); expect(datasource.label).toBe('Production Database'); expect(datasource.description).toBeDefined(); - expect(datasource.capabilities).toBeDefined(); }); it('should accept PostgreSQL datasource', () => { @@ -364,19 +324,6 @@ describe('DatasourceSchema', () => { expect(datasource.active).toBe(false); }); - it('should accept datasource with capability overrides', () => { - const datasource = DatasourceSchema.parse({ - name: 'custom_db', - driver: 'postgres', - config: { database: 'mydb' }, - capabilities: { - queryWindowFunctions: false, - querySubqueries: false, - }, - }); - - expect(datasource.capabilities?.queryWindowFunctions).toBe(false); - }); it('should accept datasource with environment variables in config', () => { const datasource = DatasourceSchema.parse({ diff --git a/packages/spec/src/data/datasource.zod.ts b/packages/spec/src/data/datasource.zod.ts index a17ddf5572..8e4ed38bd5 100644 --- a/packages/spec/src/data/datasource.zod.ts +++ b/packages/spec/src/data/datasource.zod.ts @@ -41,7 +41,7 @@ import { validateDriverConfig } from './driver/config-registry.zod'; */ /** Keys {@link DriverDefinitionSchema} declares (drift-guarded by datasource.test.ts). */ -const DRIVER_DEFINITION_KEYS = ['id', 'label', 'description', 'icon', 'configSchema', 'capabilities'] as const; +const DRIVER_DEFINITION_KEYS = ['id', 'label', 'description', 'icon', 'configSchema'] as const; /** Keys {@link ExternalDatasourceSettingsSchema} declares (drift-guarded by datasource.test.ts). */ const EXTERNAL_SETTINGS_KEYS = [ @@ -54,7 +54,7 @@ const EXTERNAL_VALIDATION_KEYS = ['onMismatch', 'checkOnBoot', 'checkIntervalMs' /** Keys {@link DatasourceSchema} declares (drift-guarded by datasource.test.ts). */ const DATASOURCE_KEYS = [ - 'name', 'label', 'driver', 'config', 'pool', 'capabilities', + 'name', 'label', 'driver', 'config', 'pool', 'healthCheck', 'ssl', 'retryPolicy', 'description', 'active', 'autoConnect', 'schemaMode', 'external', 'origin', ] as const; @@ -71,6 +71,35 @@ const SSL_KEYS = ['enabled', 'rejectUnauthorized', 'ca', 'cert', 'key'] as const /** Keys the datasource `retryPolicy` block declares (drift-guarded by datasource.test.ts). */ const DATASOURCE_RETRY_POLICY_KEYS = ['maxRetries', 'baseDelayMs', 'maxDelayMs', 'backoffMultiplier'] as const; +const CAPABILITIES_REMOVED_PREFIX = + '`datasource.capabilities` was removed in @objectstack/spec 17.0.0 (#4583, ADR-0049) — ' + + 'all eleven flags were declared, strict-guarded and read by nobody. '; + +/** + * Tombstone for the retired `capabilities` block (#4583). + * + * Each flag gets its own prescription rather than one shared line, because the + * mechanism that actually decides the behaviour differs per flag — and a + * prescription that names the wrong mechanism is worse than none (#4001 landed + * four of those before the sweep caught them). + */ +const RETIRED_CAPABILITIES: Record = { + capabilities: + CAPABILITIES_REMOVED_PREFIX + + 'Pushdown is decided by the runtime driver\'s own `supports.*` object, not by datasource ' + + 'metadata, so declaring a capability here never changed which engine path ran. Delete the ' + + 'block. If you wrote `readOnly: true`, read its note below — it did NOT make anything ' + + 'read-only. Run `os migrate meta --from 16` to rewrite it automatically.', + readOnly: + CAPABILITIES_REMOVED_PREFIX + + '`readOnly` in particular NEVER made a datasource read-only: no write path consulted it, ' + + 'so a datasource labelled a read replica accepted writes exactly like any other. The one ' + + 'enforced datasource-wide write gate is `external.allowWrites: false`, and it applies ONLY ' + + 'to a federated datasource (`schemaMode` other than `managed`) — for a managed datasource ' + + 'there is currently no read-only gate at all, so delete the key rather than trusting it. ' + + 'Tracked in #4584.', +}; + /** * A connection detail written one level too high — it belongs inside `config`. * @@ -109,7 +138,10 @@ const driverDefinitionUnknownKeyError = strictUnknownKeyError({ title: 'label', config: 'configSchema', schema: 'configSchema', - capability: 'capabilities', + }, + guidance: { + capabilities: RETIRED_CAPABILITIES.capabilities, + capability: RETIRED_CAPABILITIES.capabilities, }, history: 'Until #4001 these were dropped silently — the driver still registered.', }); @@ -223,6 +255,8 @@ const datasourceUnknownKeyError = strictUnknownKeyError({ + '`external.credentialsRef`.', readReplicas: RETIRED_READ_REPLICAS, replicas: RETIRED_READ_REPLICAS, + capabilities: RETIRED_CAPABILITIES.capabilities, + readOnly: RETIRED_CAPABILITIES.readOnly, }, history: 'Until #4001 these were dropped silently — a connection key written one level too high ' @@ -303,35 +337,7 @@ const datasourceRetryPolicyUnknownKeyError = strictUnknownKeyError({ + 'retryPolicy spells its delay `backoffMs`; a datasource spells it `baseDelayMs`.', }); -const capabilitiesUnknownKeyError = strictUnknownKeyError({ - surface: 'these datasource capabilities', - knownKeys: [ - 'transactions', 'queryFilters', 'queryAggregations', 'querySorting', - 'queryPagination', 'queryWindowFunctions', 'querySubqueries', 'joins', - 'fullTextSearch', 'readOnly', 'dynamicSchema', - ], - aliases: { - transaction: 'transactions', - filters: 'queryFilters', - filtering: 'queryFilters', - aggregations: 'queryAggregations', - aggregation: 'queryAggregations', - sorting: 'querySorting', - sort: 'querySorting', - pagination: 'queryPagination', - windowfunctions: 'queryWindowFunctions', - subqueries: 'querySubqueries', - join: 'joins', - fulltext: 'fullTextSearch', - search: 'fullTextSearch', - readonly: 'readOnly', - schemaless: 'dynamicSchema', - }, - history: - 'Until #4001 these were dropped silently — and a capability that fails to register ' - + 'reads as FALSE, so the engine quietly stopped pushing that work down to the driver ' - + 'and recomputed it in memory instead.', -}); + export const DriverType = z.string().describe('Underlying driver identifier'); @@ -363,68 +369,11 @@ export const DriverDefinitionSchema = lazySchema(() => z.object({ */ configSchema: z.record(z.string(), z.unknown()).describe('JSON Schema for connection configuration'), - /** - * Default Capabilities - * What this driver supports out-of-the-box. - */ - capabilities: z.lazy(() => DatasourceCapabilities).optional(), }, { error: driverDefinitionUnknownKeyError }).strict()); /** A driver definition — {@link DriverDefinitionSchema}'s parsed shape. */ export type DriverDefinition = z.infer; -/** - * Datasource Capabilities Schema - * Declares what this datasource naturally supports. - * The ObjectQL engine uses this to determine what logic to push down - * and what to compute in memory. - */ -export const DatasourceCapabilities = z.object({ - // ============================================================================ - // Transaction & Connection Management - // ============================================================================ - - /** Can handle ACID transactions? */ - transactions: z.boolean().default(false), - - // ============================================================================ - // Query Operations - // ============================================================================ - - /** Can execute WHERE clause filters natively? */ - queryFilters: z.boolean().default(false), - - /** Can perform aggregation (group by, sum, avg)? */ - queryAggregations: z.boolean().default(false), - - /** Can perform ORDER BY sorting? */ - querySorting: z.boolean().default(false), - - /** Can perform LIMIT/OFFSET pagination? */ - queryPagination: z.boolean().default(false), - - /** Can perform window functions? */ - queryWindowFunctions: z.boolean().default(false), - - /** Can perform subqueries? */ - querySubqueries: z.boolean().default(false), - - /** Can execute SQL-like joins natively? */ - joins: z.boolean().default(false), - - // ============================================================================ - // Advanced Features - // ============================================================================ - - /** Can perform full-text search? */ - fullTextSearch: z.boolean().default(false), - - /** Is read-only? */ - readOnly: z.boolean().default(false), - - /** Is scheme-less (needs schema inference)? */ - dynamicSchema: z.boolean().default(false), -}, { error: capabilitiesUnknownKeyError }).strict(); /** * Schema Ownership Mode (ADR-0015) @@ -541,7 +490,6 @@ export const DatasourceSchema = lazySchema(() => z.object({ * Capability Overrides * Manually override what the driver claims to support. */ - capabilities: DatasourceCapabilities.optional().describe('Capability overrides'), /** Health Check */ healthCheck: z.object({ @@ -650,7 +598,6 @@ export const DatasourceSchema = lazySchema(() => z.object({ export type Datasource = z.infer; /** Authoring input for {@link Datasource} — defaulted fields are optional. */ export type DatasourceInput = z.input; -export type DatasourceCapabilitiesType = z.infer; /** * Type-safe factory for an external data connection (datasource). Validates at authoring time via diff --git a/packages/spec/src/data/driver/common.zod.ts b/packages/spec/src/data/driver/common.zod.ts index d026221e45..fbb7646512 100644 --- a/packages/spec/src/data/driver/common.zod.ts +++ b/packages/spec/src/data/driver/common.zod.ts @@ -42,10 +42,22 @@ export const SCHEMA_MODE_BELONGS_ON_DATASOURCE = + "(`schemaMode: 'external'`) — the connection service now carries it down to the driver, so " + 'the copy inside `config` is gone rather than duplicated.'; -/** `readOnly` written inside `config`. Shared by every driver. */ +/** + * `readOnly` written inside `config`. Shared by every driver. + * + * This line used to send authors to `capabilities: { readOnly: true }` — a key + * #4583 removed because nothing read it, so the advice manufactured exactly the + * belief it was meant to correct: an author moved the key, the parse went + * green, and the datasource stayed writable. The prescription now names the one + * gate that is enforced, and says plainly where it does NOT apply rather than + * leaving the reader to assume it covers their case (#4584). + */ export const READ_ONLY_BELONGS_ON_DATASOURCE = - '`readOnly` is not driver config. Use `capabilities: { readOnly: true }` on the datasource to ' - + 'declare the connection read-only, or `external.allowWrites: false` for a federated database.'; + '`readOnly` is not driver config, and there is no datasource key that makes a connection ' + + 'read-only. For a FEDERATED datasource use `external.allowWrites: false`, which the ObjectQL ' + + 'engine enforces before every write. For a managed (local) datasource there is currently no ' + + 'read-only gate — grant the connection SELECT-only at the database instead, which is a real ' + + 'boundary rather than an application-layer flag (#4584).'; /** * TLS on/off for a SQL driver — the shorthand, and deliberately ONLY the diff --git a/packages/spec/src/data/driver/memory.test.ts b/packages/spec/src/data/driver/memory.test.ts index 5a2c8a351c..0a6f6c3984 100644 --- a/packages/spec/src/data/driver/memory.test.ts +++ b/packages/spec/src/data/driver/memory.test.ts @@ -441,30 +441,4 @@ describe('MemoryDriverSpec', () => { it('should have an icon', () => { expect(MemoryDriverSpec.icon).toBe('memory'); }); - - it('should have capabilities defined', () => { - expect(MemoryDriverSpec.capabilities).toBeDefined(); - }); - - it('should support transactions', () => { - expect(MemoryDriverSpec.capabilities!.transactions).toBe(true); - }); - - it('should support core query features', () => { - expect(MemoryDriverSpec.capabilities!.queryFilters).toBe(true); - expect(MemoryDriverSpec.capabilities!.queryAggregations).toBe(true); - expect(MemoryDriverSpec.capabilities!.querySorting).toBe(true); - expect(MemoryDriverSpec.capabilities!.queryPagination).toBe(true); - }); - - it('should not support advanced query features', () => { - expect(MemoryDriverSpec.capabilities!.joins).toBe(false); - expect(MemoryDriverSpec.capabilities!.queryWindowFunctions).toBe(false); - expect(MemoryDriverSpec.capabilities!.querySubqueries).toBe(false); - expect(MemoryDriverSpec.capabilities!.fullTextSearch).toBe(false); - }); - - it('should support dynamic schema', () => { - expect(MemoryDriverSpec.capabilities!.dynamicSchema).toBe(true); - }); }); diff --git a/packages/spec/src/data/driver/memory.zod.ts b/packages/spec/src/data/driver/memory.zod.ts index 321fe233f5..6d5b44e998 100644 --- a/packages/spec/src/data/driver/memory.zod.ts +++ b/packages/spec/src/data/driver/memory.zod.ts @@ -307,8 +307,7 @@ export const getMemoryConfigJsonSchema = driverConfigJsonSchema(MemoryConfigSche // ========================================================================== /** - * The static definition of the Memory driver's capabilities and default - * metadata, satisfying the `DriverDefinitionSchema` contract (proved by + * The static definition of the Memory driver's default metadata, satisfying the `DriverDefinitionSchema` contract (proved by * `memory.test.ts`, which parses this constant). * * `configSchema` was `{}` here — a declared slot that nothing filled and @@ -324,24 +323,6 @@ export const MemoryDriverSpec = { get configSchema() { return getMemoryConfigJsonSchema(); }, - capabilities: { - transactions: true, - // Query - queryFilters: true, - queryAggregations: true, - querySorting: true, - queryPagination: true, - // No join, window function, or subquery support - joins: false, - queryWindowFunctions: false, - querySubqueries: false, - // No full-text search (linear scan) - fullTextSearch: false, - // Not read-only - readOnly: false, - // Dynamic schema (no DDL needed) - dynamicSchema: true, - }, } satisfies DriverDefinition; // ========================================================================== diff --git a/packages/spec/src/data/driver/mongo.test.ts b/packages/spec/src/data/driver/mongo.test.ts index 6fdf63ff24..e45bccc82b 100644 --- a/packages/spec/src/data/driver/mongo.test.ts +++ b/packages/spec/src/data/driver/mongo.test.ts @@ -111,27 +111,4 @@ describe('MongoDriverSpec', () => { it('should have an icon', () => { expect(MongoDriverSpec.icon).toBe('database'); }); - - it('should have capabilities defined', () => { - expect(MongoDriverSpec.capabilities).toBeDefined(); - }); - - it('should support transactions', () => { - expect(MongoDriverSpec.capabilities!.transactions).toBe(true); - }); - - it('should support query features', () => { - expect(MongoDriverSpec.capabilities!.queryFilters).toBe(true); - expect(MongoDriverSpec.capabilities!.queryAggregations).toBe(true); - expect(MongoDriverSpec.capabilities!.querySorting).toBe(true); - expect(MongoDriverSpec.capabilities!.queryPagination).toBe(true); - }); - - it('should support full text search', () => { - expect(MongoDriverSpec.capabilities!.fullTextSearch).toBe(true); - }); - - it('should support dynamic schema', () => { - expect(MongoDriverSpec.capabilities!.dynamicSchema).toBe(true); - }); }); diff --git a/packages/spec/src/data/driver/mongo.zod.ts b/packages/spec/src/data/driver/mongo.zod.ts index a41922dbb2..b719f97370 100644 --- a/packages/spec/src/data/driver/mongo.zod.ts +++ b/packages/spec/src/data/driver/mongo.zod.ts @@ -137,8 +137,7 @@ export const getMongoConfigJsonSchema = driverConfigJsonSchema(MongoConfigSchema // ========================================================================== /** - * The static definition of the Mongo driver's capabilities and default - * metadata, satisfying the `DriverDefinitionSchema` contract (proved by + * The static definition of the Mongo driver's default metadata, satisfying the `DriverDefinitionSchema` contract (proved by * `mongo.test.ts`, which parses this constant). * * `configSchema` is a getter so the JSON-Schema projection is computed on first @@ -158,21 +157,6 @@ export const MongoDriverSpec = { get configSchema() { return getMongoConfigJsonSchema(); }, - capabilities: { - transactions: true, - // Query - queryFilters: true, - queryAggregations: true, - querySorting: true, - queryPagination: true, - queryWindowFunctions: false, - querySubqueries: false, - joins: false, - fullTextSearch: true, - readOnly: false, - // Schema - dynamicSchema: true, - }, } satisfies DriverDefinition; /** diff --git a/packages/spec/src/migrations/registry.ts b/packages/spec/src/migrations/registry.ts index 75cfd3872b..4e31d8813c 100644 --- a/packages/spec/src/migrations/registry.ts +++ b/packages/spec/src/migrations/registry.ts @@ -586,6 +586,7 @@ const step17: MigrationStep = { 'agent-knowledge-removed', 'skill-trigger-phrases-removed', 'stack-api-require-auth-removed', + 'datasource-capabilities-removed', 'flow-node-wait-timeout-keys-removed', 'datasource-read-replicas-removed', 'flow-node-script-branch-keys-removed',