Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 59 additions & 0 deletions .changeset/datasource-capabilities-retired.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
---
"@objectstack/spec": major
"@objectstack/example-crm": patch
---

feat(spec)!: retire `datasource.capabilities` — eleven flags nothing read, one of them a safety claim (#4583)

`DatasourceCapabilities` declared eleven booleans — `transactions`, seven `query*`
flags, `joins`, `fullTextSearch`, `readOnly`, `dynamicSchema` — all strict-guarded,
all read by nothing. Pushdown is decided by the runtime driver's own `supports.*`
object, a different mechanism entirely, so a datasource declaring
`queryAggregations: false` never once changed which engine path ran. The block is
removed rather than bridged: there was nothing on the other side to connect it to.

**`readOnly` is why this is not tidy-up.** It reads as a safety property and was
authored as one — the shipped CRM example labelled a datasource "CRM Analytics Read
Replica" on the strength of it, while the datasource accepted writes exactly like the
primary. The key had already been MOVED twice toward somewhere it might be enforced,
out of `config` in #4410 and into `capabilities` in #4465, and was inert at every
address. This removes it instead of moving it a third time.

**Removing it does not hand you a working replacement, and the rejection says so.**
The one enforced datasource-wide write gate is `external.allowWrites: false`, and it
applies only to a FEDERATED datasource — `assertWriteAllowed` returns early for a
`managed` (or unset-`schemaMode`) datasource, so that key would be equally inert for a
local database. **A managed datasource has no read-only gate at all**; that gap is
#4584, deliberately not invented here. Until it is answered, enforce read-only where
it is real: grant the connection SELECT-only at the database.

FROM → TO:

```ts
// before — parsed cleanly, changed nothing
defineDatasource({
name: 'analytics', driver: 'sqlite', config: { filename: ':memory:' },
capabilities: { readOnly: true, queryAggregations: true },
})

// after — delete the block; for a FEDERATED datasource the enforced gate is:
defineDatasource({
name: 'warehouse', driver: 'postgres', config: { … },
schemaMode: 'external',
external: { allowWrites: false },
})
```

`os migrate meta --from 16` rewrites it automatically (ADR-0087 conversion
`datasource-capabilities-removed`). Both `DatasourceSchema` and
`DriverDefinitionSchema` are `.strict()`, so a leftover key is a loud rejection
carrying the prescription — never a silent strip.

Also fixed: `READ_ONLY_BELONGS_ON_DATASOURCE`, the prescription every SQL driver
shares for a `readOnly` written inside `config`, was still sending authors *to* the
removed key. It now names the enforced gate and states plainly where that gate does
not apply — a prescription that lands on an inert key manufactures exactly the belief
it was meant to correct.

The `datasource` liveness ledger drops from 20 dead properties to 9 (remaining:
`healthCheck` ×3, `retryPolicy` ×4, `external` ×2 — batches B/C/D of #4583).
2 changes: 1 addition & 1 deletion content/docs/getting-started/quick-reference.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ Core business logic and data modeling schemas.
| **[Query](/docs/references/data/query)** | `query.zod.ts` | Query, QueryAST | Query AST with joins, aggregations |
| **[Filter](/docs/references/data/filter)** | `filter.zod.ts` | QueryFilter, FilterCondition | Advanced filtering operators |
| **[Validation](/docs/references/data/validation)** | `validation.zod.ts` | ValidationRule | Business validation rules |
| **[Datasource](/docs/references/data/datasource)** | `datasource.zod.ts` | Datasource, DatasourceCapabilities | Database connection configs |
| **[Datasource](/docs/references/data/datasource)** | `datasource.zod.ts` | Datasource, DriverDefinition | Database connection configs |
| **[Analytics](/docs/references/data/analytics)** | `analytics.zod.ts` | Analytics | Data analytics and aggregation |
| **[Mapping](/docs/references/data/mapping)** | `mapping.zod.ts` | FieldMapping | Field transformation mappings |
| **[Hook](/docs/references/data/hook)** | `hook.zod.ts` | Hook, HookEvent | Lifecycle event hooks |
Expand Down
25 changes: 1 addition & 24 deletions content/docs/references/data/datasource.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ Can be a built-in driver or a plugin-contributed driver (e.g., "com.vendor.snowf
## TypeScript Usage

```typescript
import { DatasourceSchema, DatasourceCapabilities, DriverDefinitionSchema, DriverType, ExternalDatasourceSettingsSchema } from '@objectstack/spec/data';
import { DatasourceSchema, DriverDefinitionSchema, DriverType, ExternalDatasourceSettingsSchema } from '@objectstack/spec/data';
import type { Datasource, DriverDefinition, ExternalDatasourceSettings } from '@objectstack/spec/data';

// Validate data
Expand All @@ -36,7 +36,6 @@ const result = DatasourceSchema.parse(data);
| **driver** | `string` | ✅ | Underlying driver type |
| **config** | `Record<string, any>` | ✅ | Driver specific configuration |
| **pool** | `{ min: number; max: number; idleTimeoutMillis: number; connectionTimeoutMillis: number }` | optional | Connection pool settings |
| **capabilities** | `{ transactions: boolean; queryFilters: boolean; queryAggregations: boolean; querySorting: boolean; … }` | optional | Capability overrides |
| **healthCheck** | `{ enabled: boolean; intervalMs: number; timeoutMs: number }` | optional | Datasource health check configuration |
| **ssl** | `{ enabled: boolean; rejectUnauthorized: boolean; ca?: string; cert?: string; … }` | optional | SSL/TLS configuration for secure database connections |
| **retryPolicy** | `{ maxRetries: number; baseDelayMs: number; maxDelayMs: number; backoffMultiplier: number }` | optional | Connection retry policy for transient failures |
Expand All @@ -55,27 +54,6 @@ const result = DatasourceSchema.parse(data);
| **_lockDocsUrl** | `string` | optional | Optional documentation link surfaced next to _lockReason. |


---

## DatasourceCapabilities

### Properties

| Property | Type | Required | Description |
| :--- | :--- | :--- | :--- |
| **transactions** | `boolean` | ✅ | |
| **queryFilters** | `boolean` | ✅ | |
| **queryAggregations** | `boolean` | ✅ | |
| **querySorting** | `boolean` | ✅ | |
| **queryPagination** | `boolean` | ✅ | |
| **queryWindowFunctions** | `boolean` | ✅ | |
| **querySubqueries** | `boolean` | ✅ | |
| **joins** | `boolean` | ✅ | |
| **fullTextSearch** | `boolean` | ✅ | |
| **readOnly** | `boolean` | ✅ | |
| **dynamicSchema** | `boolean` | ✅ | |


---

## DriverDefinition
Expand All @@ -89,7 +67,6 @@ const result = DatasourceSchema.parse(data);
| **description** | `string` | optional | |
| **icon** | `string` | optional | |
| **configSchema** | `Record<string, any>` | ✅ | JSON Schema for connection configuration |
| **capabilities** | `{ transactions: boolean; queryFilters: boolean; queryAggregations: boolean; querySorting: boolean; … }` | optional | |


---
Expand Down
4 changes: 2 additions & 2 deletions docs/audits/2026-07-unknown-key-strictness-ledger.md
Original file line number Diff line number Diff line change
Expand Up @@ -454,7 +454,7 @@ tightening (the #4001 "sharing-rule lesson": candidates, not verdicts).
| `notification.zod.ts` / `offline.zod.ts` / `report.zod.ts` | 3 ea | authorable (p) | |
| `sharing.zod.ts` | 2 | authorable (p) | public-sharing config |

### `data/` — 165 sites
### `data/` — 164 sites

| File | Sites | Class | Note |
|---|---|---|---|
Expand All @@ -465,7 +465,7 @@ tightening (the #4001 "sharing-rule lesson": candidates, not verdicts).
| `field.zod.ts` | 11 | authorable | partially strict |
| `filter.zod.ts` / `query.zod.ts` | 11+5 | open | query dialect — user data flows through; validated semantically elsewhere. `query.zod.ts` dropped one site in #4196: `FieldNodeSchema`'s nested-select object form was declared-but-inert and narrowed to `z.string()`, so the union's second member is gone. Four more left in #4286 with the `joins`/`windowFunctions` removals: `JoinNodeBaseSchema`, `WindowFunctionNodeSchema`, and `WindowSpecSchema`'s two blocks (outer + `frame`) were deleted with their clusters. Class unchanged |
| `driver-nosql.zod.ts` / `driver.zod.ts` / `driver-sql.zod.ts` | 10+9+2 | wire | driver capability contracts |
| `datasource.zod.ts` | 9 | authorable | **strict as of #4001 data step** — all 9: `DatasourceSchema` (+ `pool` / `healthCheck` / `ssl` / `retryPolicy`), `ExternalDatasourceSettingsSchema` (+ `validation`), `DatasourceCapabilities`, `DriverDefinitionSchema`. `config` stays `z.record` **at this level** by construction (per-driver shapes), but is no longer unchecked: **#4410** made `DatasourceSchema`'s refinement parse it against the contract for the declared driver (`driver/config-registry.zod.ts`), so the openness here is a shape this level cannot express rather than the absence of one. This row used to add "the driver's own `configSchema` validates them", which was false until #4410 landed the parse site it names. #4410 extended the same parse to each `readReplicas` entry; **#4468 retired that key** — no driver ever opened a replica connection and no query path splits reads from writes, so the entries were being checked against a contract nothing would apply. Strictness makes a dropped key loud; it cannot make a slot live, and a *precisely validated* dead slot is the more convincing lie |
| `datasource.zod.ts` | 8 | authorable | **strict as of #4001 data step** — all 8: `DatasourceSchema` (+ `pool` / `healthCheck` / `ssl` / `retryPolicy`), `ExternalDatasourceSettingsSchema` (+ `validation`), `DriverDefinitionSchema`. `config` stays `z.record` **at this level** by construction (per-driver shapes), but is no longer unchecked: **#4410** made `DatasourceSchema`'s refinement parse it against the contract for the declared driver (`driver/config-registry.zod.ts`), so the openness here is a shape this level cannot express rather than the absence of one. This row used to add "the driver's own `configSchema` validates them", which was false until #4410 landed the parse site it names. #4410 extended the same parse to each `readReplicas` entry; **#4468 retired that key** — no driver ever opened a replica connection and no query path splits reads from writes, so the entries were being checked against a contract nothing would apply. Strictness makes a dropped key loud; it cannot make a slot live, and a *precisely validated* dead slot is the more convincing lie | **#4583 dropped the ninth site**: `DatasourceCapabilities` is gone — eleven flags no code read, on a block whose strictness was the clearest case of this row's own closing sentence. `readOnly` in particular was *precisely validated* and completely inert, and had been relocated twice (#4410, #4465) toward somewhere it might be enforced; the shipped CRM example called a datasource a read replica on the strength of it while writes went through. Class unchanged
| `driver/memory.zod.ts` / `driver/mongo.zod.ts` / `driver/postgres.zod.ts` | 6+1+1 | authorable | The per-driver shapes for the `config` slot — what an author actually writes under `datasource.config` (`host`, `port`, `filename`). **Undeclared here until the coverage walk went recursive** (see below): a subdirectory was invisible to the gate, so these sites sat outside the map while the map reported full coverage. **Strict as of #4410**, which is also what unblocked them: this row previously read "strictness here would enforce nothing" because nothing parsed `datasource.config` against these schemas and both `*DriverSpec.configSchema` literals were `{}`. Now `DatasourceSchema` parses `config` against them, and the same schemas project onto `configSchema` and onto the Studio connection form. (#4410 also ran the parse over each `readReplicas` entry; #4468 retired that key outright — see the row above.) `postgres.zod.ts` drops a site: its `ssl` was a `boolean | {ca, cert, key, …}` union, and the object arm is gone — certificates now live in the datasource-level `ssl` block (declared, strict, and until #4410 read by nobody), leaving `config.ssl` as the on/off shorthand. That narrowing is forced by the same projection: the Studio form renders anything that is not boolean/enum/number as a TEXT INPUT, so a union here would have produced a wizard whose every `ssl` value the new gate rejects. `memory.zod.ts` keeps 6 but loses two KEYS — `indexes` / `maxRecordsPerObject`, which `InMemoryDriverConfig` has no field for, removed under ADR-0049 rather than blessed by the new gate |
| `driver/mysql.zod.ts` / `driver/sqlite.zod.ts` | 1+2 | authorable | The rest of the `config` contract, added by #4410. `mysql.zod.ts` and `sqlite.zod.ts` (sqlite + sqlite-wasm) are shapes that **never existed** — both driver ids were offered by the connection form and buildable by the shared factory, with no config contract anywhere, so `driver: 'sqlite'` + a misspelled `filename` was an ephemeral `:memory:` database reported as configured. All three sites strict, same error factory as the rest of the campaign. (Their sibling `driver/common.zod.ts` holds shared enums and prescription strings and has no `z.object(` site, so the coverage gate skips it) |
| `analytics.zod.ts` | 8 | mixed (p) | |
Expand Down
1 change: 1 addition & 0 deletions docs/protocol-upgrade-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -188,6 +188,7 @@ The `script` flow node converges on its one real path (#4343). It had four ways
| `stack-api-require-auth-removed` | `stack.api.requireAuth` | stack key 'api.requireAuth' removed — anonymous access is always denied; publish public surfaces by declaration (#3963) | retired — `migrate meta` only |
| `flow-node-wait-timeout-keys-removed` | `flow.node.waitEventConfig` | waitEventConfig keys 'timeoutMs' (→ 'timerDuration', stringified — its only reader used it as the duration) and 'onTimeout' (removed — zero readers, so no timeout ever fired) (#4158) | retired — `migrate meta` only |
| `datasource-read-replicas-removed` | `datasource.readReplicas` | datasource key 'readReplicas' removed (#4468 — no driver opened a replica connection and no query path splits reads from writes; front replicas behind one endpoint and point `config` at it) | retired — `migrate meta` only |
| `datasource-capabilities-removed` | `datasource.capabilities` | datasource key 'capabilities' removed (#4583 — eleven flags no code read; pushdown comes from the driver's own supports.*, and `readOnly` never made anything read-only) | retired — `migrate meta` only |
| `flow-node-script-branch-keys-removed` | `flow.node.script.config.actionType / flow.node.script.config.template / flow.node.script.config.recipients / flow.node.script.config.variables / flow.node.script.config.script` | script flow-node config keys 'actionType' (→ 'function' when it was shorthand for one; otherwise removed — 'email'/'slack' were logger-backed stubs that delivered nothing), plus 'template' / 'recipients' / 'variables' (fed those stubs) and 'script' (inline JS the runtime never executed) (#4343) | retired — `migrate meta` only |

### Semantic (delegated to you, with acceptance criteria)
Expand Down
21 changes: 13 additions & 8 deletions examples/app-crm/src/datasources/crm.datasource.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,21 +21,26 @@ export const CrmDatasource = defineDatasource({
});

/**
* Read-replica for analytics queries — demonstrates datasource routing.
* Second datasource for analytics queries — demonstrates datasource routing.
*
* `readOnly` is a datasource CAPABILITY, not sqlite config. It sat inside
* `config` here until #4410 gave that slot a gate — a key no driver read, so
* the "read replica" was writable while every signal said it was not.
* This used to declare `capabilities: { readOnly: true }` and call itself a read
* replica. It was neither: the key had no reader, so the "read replica" accepted
* writes exactly like the primary — the third spelling of the same defect, after
* the same claim sat inertly in `config` (#4410) and then in `capabilities`
* (#4465). #4583 removed the key rather than move it a fourth time.
*
* The label no longer promises read-only, because nothing here can deliver it:
* `external.allowWrites: false` is the one enforced write gate and it applies
* only to FEDERATED datasources, while this one is local and managed. Whether a
* managed datasource should have a read-only gate at all is #4584 — until that
* is answered, the honest demo is routing, not a safety claim.
*/
export const CrmAnalyticsDatasource = defineDatasource({
name: 'crm_analytics',
label: 'CRM Analytics Read Replica',
label: 'CRM Analytics',
driver: 'sqlite',
config: {
filename: ':memory:',
},
capabilities: {
readOnly: true,
},
active: true,
});
31 changes: 18 additions & 13 deletions packages/cli/src/utils/lint-liveness-properties.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -195,42 +195,47 @@ describe('lintLivenessProperties', () => {
// the type that most needed it: 20 of its 43 props have no runtime consumer,
// and until #4487 nothing told an author so.

it('warns on the dead datasource blocks — capabilities / healthCheck / retryPolicy (#4487)', () => {
it('warns on the dead datasource blocks that remain — healthCheck / retryPolicy (#4487)', () => {
// `capabilities` left this list in #4583: the block was REMOVED from the
// schema, so an author who writes it now gets a hard parse rejection with a
// prescription — a stronger signal than a lint warning, and the reason its
// ledger rows are gone rather than flipped. healthCheck / retryPolicy are
// still authorable and still dead (batches B and C of #4583).
const findings = lintLivenessProperties({
datasources: [{
name: 'warehouse',
driver: 'postgres',
config: { host: 'db.internal', database: 'analytics' },
capabilities: { transactions: true, queryAggregations: true },
healthCheck: { enabled: true, intervalMs: 30000 },
retryPolicy: { maxRetries: 5, baseDelayMs: 1000 },
}],
});
const msgs = paths(findings);
expect(msgs.some((m) => m.includes('capabilities.transactions'))).toBe(true);
expect(msgs.some((m) => m.includes('capabilities.queryAggregations'))).toBe(true);
expect(msgs.some((m) => m.includes('healthCheck.enabled'))).toBe(true);
expect(msgs.some((m) => m.includes('healthCheck.intervalMs'))).toBe(true);
expect(msgs.some((m) => m.includes('retryPolicy.maxRetries'))).toBe(true);
expect(msgs.some((m) => m.includes('retryPolicy.baseDelayMs'))).toBe(true);
// The removed block must no longer be reported by the lint at all.
expect(msgs.some((m) => m.includes('capabilities'))).toBe(false);
});

// The entry the whole audit was worth doing for. `capabilities.readOnly` reads
// as a safety switch and gates nothing, and two shipped prescriptions pointed
// authors AT it until #4487. The hint has to name the gate that IS enforced,
// or the warning just relocates the author's confusion.
it('warns on capabilities.readOnly and names the real write gate (#4487)', () => {
// The entry the whole audit was worth doing for. `capabilities.readOnly` read
// as a safety switch and gated nothing — a datasource labelled a read replica
// took writes like any other. #4583 REMOVED it rather than warn about it for
// another release, so the check moved up a level: the lint no longer has an
// opinion because the schema refuses the key outright. The prescription that
// replaces the hint is asserted in `packages/spec` (datasource.test.ts), where
// it can also assert the part a hint could not carry — that the enforced gate
// does NOT cover managed datasources (#4584).
it('no longer warns on capabilities.readOnly — the key is gone, not merely flagged (#4583)', () => {
const findings = lintLivenessProperties({
datasources: [{
name: 'reporting',
driver: 'postgres',
config: { host: 'ro.internal', database: 'reporting' },
capabilities: { readOnly: true },
}],
});
const hit = findings.find((f) => f.message.includes('capabilities.readOnly'));
expect(hit).toBeDefined();
expect(hit!.hint).toMatch(/allowWrites/);
expect(findings.some((f) => f.message.includes('capabilities'))).toBe(false);
});

it('stays silent on a datasource that only sets live properties (#4487)', () => {
Expand Down
2 changes: 0 additions & 2 deletions packages/spec/api-surface.json
Original file line number Diff line number Diff line change
Expand Up @@ -290,8 +290,6 @@
"DataTypeMapping (type)",
"DataTypeMappingSchema (const)",
"Datasource (type)",
"DatasourceCapabilities (const)",
"DatasourceCapabilitiesType (type)",
"DatasourceInput (type)",
"DatasourceSchema (const)",
"DateGranularity (const)",
Expand Down
Loading
Loading