From 5371a85d300a5178eea519b390473b771b0e3590 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 31 Jul 2026 03:55:20 +0000 Subject: [PATCH] refactor(spec,plugin-security)!: QueryAST stops declaring what no executor runs (#4286) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Tombstone `query.joins` and `query.windowFunctions` (retiredKey — authoring either is a tsc error and the parse carries the prescription), and delete the orphaned Join and WindowFunction schema clusters with their keys: `expand` is the live spelling for related records, and the one live window-function door (`SqlDriver.findWithWindowFunctions()`) consumes its own flat driver shape that the spec vocabulary never matched. Both removals register as protocol-18 semantic migrations (`query-joins-retired`, `query-window-functions-retired`) — QueryAST is a request shape with no stored source to rewrite (#4196 precedent). Both names stay reserved at the REST boundary while the tombstones live (retiredKey keeps them in `keyof QueryAST`). Mark the six search sub-flags and `AggregationNode.filter` `[EXPERIMENTAL — not enforced]` — declared engine affordances the ADR-0061 expansion and the aggregation paths never receive. Close the gate that let the class stay invisible: `QuerySchema` joins the liveness ledger through SPEC_ONLY_SCHEMAS as governed type `query` (liveness/query.json — 15 live with evidence, 7 experimental via describe markers, 5 dead). `cursor` / `distinct` / `having` are recorded dead with evidence pending their #4286 step-3/4 dispositions, not unilaterally removed — `distinct` is mis-wired (its only effect is REST count-suppression) and `having` is the strongest enforce candidate (engine.aggregate() currently rebuilds the driver AST without it). The FLS predicate guard drops its windowFunctions walk (the clause no longer exists to leak through); the `having` and `aggregations[].filter` walks stay while those members stay declared. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_012K2BX3WQwPBA2ZBehqJUBX --- .changeset/query-ast-inert-request-surface.md | 80 + content/docs/data-modeling/queries.mdx | 115 +- content/docs/protocol/objectql/index.mdx | 2 +- .../docs/protocol/objectql/query-syntax.mdx | 78 +- content/docs/protocol/objectql/schema.mdx | 2 +- content/docs/protocol/objectql/types.mdx | 2 +- content/docs/references/api/contract.mdx | 4 +- content/docs/references/data/query.mdx | 110 +- .../2026-07-unknown-key-strictness-ledger.md | 4 +- packages/metadata-protocol/src/protocol.ts | 10 + packages/objectql/src/protocol-data.test.ts | 5 + .../src/predicate-guard.test.ts | 10 +- .../plugin-security/src/predicate-guard.ts | 34 +- packages/spec/api-surface.json | 10 - packages/spec/authorable-surface.json | 17 +- packages/spec/json-schema.manifest.json | 8 +- packages/spec/liveness/README.md | 11 +- packages/spec/liveness/query.json | 117 ++ .../spec/scripts/liveness/check-liveness.mts | 14 +- packages/spec/src/data/query.test.ts | 1452 ++--------------- packages/spec/src/data/query.zod.ts | 419 +---- packages/spec/src/migrations/registry.ts | 55 +- .../src/recursive-schema-input-assertions.ts | 21 +- skills/objectstack-query/SKILL.md | 39 +- skills/objectstack-query/evals/README.md | 2 +- skills/objectstack-query/rules/aggregation.md | 15 +- 26 files changed, 640 insertions(+), 1996 deletions(-) create mode 100644 .changeset/query-ast-inert-request-surface.md create mode 100644 packages/spec/liveness/query.json diff --git a/.changeset/query-ast-inert-request-surface.md b/.changeset/query-ast-inert-request-surface.md new file mode 100644 index 0000000000..4c74675ccd --- /dev/null +++ b/.changeset/query-ast-inert-request-surface.md @@ -0,0 +1,80 @@ +--- +"@objectstack/spec": major +"@objectstack/plugin-security": patch +--- + +refactor(data)!: the QueryAST request surface stops declaring what no executor runs — `joins` and `windowFunctions` removed, six search flags and `aggregations[].filter` marked experimental, and the liveness ledger now governs the query surface (#4286) + +#4196 removed one declared-but-inert member from `FieldNode`. Applying the same +method to the rest of the request surface (#4286) found 12 more members of +`QueryAST` that no executor runs — `packages/objectql`'s `engine.ts` contains +zero reads of any of them on the query path. This change dispositions the +mechanical tiers and closes the gate that let the class stay invisible. + +**Removed (tombstoned): `query.joins` and `query.windowFunctions`.** + +- `joins` — no engine or driver ever read it; a query carrying it silently ran + as a single-table query. Related-record retrieval already has a live + spelling: `expand`. The orphaned `JoinNode` / `JoinNodeInput` / + `JoinNodeSchema` / `JoinType` / `JoinStrategy` exports are deleted with the + key (`data/JoinNode`, `data/JoinType`, `data/JoinStrategy` leave the + published JSON schemas). +- `windowFunctions` — `find()` never applied it, so every OVER clause it + declared was silently dropped. The one live door is the SQL driver's own + `findWithWindowFunctions(object, query)` (driver-level, not on the + `IDataDriver` contract), and its input is a flat driver shape the spec + vocabulary never matched — `WindowFunctionNodeSchema` declared `field` / + `over` / `frame` members that door never read. The `WindowFunction` / + `WindowSpec` / `WindowFunctionNode` exports are deleted with the key. + +**FROM → TO** + +| Was | Now | +| :--- | :--- | +| `joins: [{ type: 'inner', object: 'customer', on: … }]` | `expand: { customer_id: { object: 'customer', fields: ['name'] } }` | +| `joins` for one related column | `fields: ['customer_id.name']` (dotted path) | +| `windowFunctions: [{ function: 'rank', … }]` in a query | `aggregations` + `groupBy`, or rankings in report/dashboard metadata | +| OVER-clause SQL from an embedder | `sqlDriver.findWithWindowFunctions(object, { windowFunctions: [{ function, alias, partitionBy?, orderBy? }] })` | + +The one-line fix: **delete the key**. Both are `retiredKey()` tombstones on the +non-strict `BaseQuerySchema`, so authoring either fails `tsc` (input type +`never`) and a query still carrying one — even as an empty array — fails to +parse with the prescription itself. `QueryAST` is a request shape, never stored +in stack metadata, so there is no `os migrate meta` step: the removals are +registered as protocol-18 **semantic** migrations (`query-joins-retired`, +`query-window-functions-retired`), the #4196 precedent. + +Compat note for the REST boundary: both names remain **reserved** list-query +parameters while the tombstones live (`retiredKey()` keeps a key in +`keyof QueryAST`, which feeds `RESERVED_LIST_QUERY_PARAMS`), so nothing changes +for objects with fields named `joins`/`windowFunctions` — the un-reservation +happens when the tombstones age out, and is called out in +`metadata-protocol`'s `QUERY_AST_KEYS` comment for whoever does it. + +**Marked `[EXPERIMENTAL — not enforced]` (no wire or compat impact):** +`search.fuzzy` / `operator` / `boost` / `minScore` / `language` / `highlight` +(the ADR-0061 expansion reads only `query` + `fields`) and +`AggregationNode.filter` (a SQL `FILTER (WHERE …)` affordance neither the SQL +builders nor the in-memory fallback applies). Authoring one is now a +declaration, not a silent no-op. + +**Deliberately NOT dispositioned here** (they want a maintainer call, #4286 +steps 3–4): `having` (the strongest enforce candidate — `engine.aggregate()` +currently rebuilds the driver AST without it), and `cursor` / `distinct` +(shipped SDK producers `QueryBuilder.cursor()` / `.distinct()`; `distinct` is +mis-wired — its only observable effect is suppressing the REST list count). +All three are recorded `dead` with evidence in the new ledger. + +**The gate:** `QuerySchema` joins the liveness ledger through the gate's +`SPEC_ONLY_SCHEMAS` override (the `webhook` precedent) as governed type +`query` — the first governance of what *callers* write into a query rather +than what authors write into metadata files. `packages/spec/liveness/query.json` +classifies all 27 walked members (15 live with evidence, 7 experimental via +describe markers, 5 dead), so the next declared-but-inert request member fails +CI instead of needing a person to notice it. + +`@objectstack/plugin-security` (patch): the FLS predicate guard's +`windowFunctions` walk is pruned — the clause no longer exists to leak through. +The `having` and `aggregations[].filter` walks stay, deliberately: those +members remain declared, and the guard being ready is what makes enforcing +them later safe. diff --git a/content/docs/data-modeling/queries.mdx b/content/docs/data-modeling/queries.mdx index 219f0313eb..b3ce6ba09d 100644 --- a/content/docs/data-modeling/queries.mdx +++ b/content/docs/data-modeling/queries.mdx @@ -1,6 +1,6 @@ --- title: Query Syntax Cheat Sheet -description: One-page reference for ObjectStack QuerySchema — filters, sorts, pagination, aggregations, and joins +description: One-page reference for ObjectStack QuerySchema — filters, sorts, pagination, aggregations, and expand --- # Query Syntax Cheat Sheet @@ -387,47 +387,27 @@ on it either. Post-filter grouped results on the client until this is wired up. --- -## Joins +## Joins — removed -`joins` is defined in `QuerySchema` (`JoinNodeSchema`) and the SQL driver advertises -`supports.joins: true`, but no driver's `find()` actually executes a join today — the SQL, -in-memory, and MongoDB drivers all ignore the `joins` array (there is no `.join()`/`.leftJoin()` -call in the SQL driver's query builder). Use `expand` for relationship traversal instead. +`query.joins` was **removed in `@objectstack/spec` 18** (#4286, ADR-0049 +enforce-or-remove): no driver's `find()` ever executed a join — the SQL, in-memory, and +MongoDB drivers all ignored the array, so it only ever declared a capability that did +not run. The key is tombstoned: authoring it is a `tsc` error, and a query carrying it +(even `joins: []`) fails to parse with the upgrade prescription. The +`JoinNode`/`JoinType`/`JoinStrategy` exports left with it. -### Join Types - -| Type | Description | -|:---|:---| -| `inner` | Records with matches in both objects | -| `left` | All records from left + matching from right | -| `right` | All records from right + matching from left | -| `full` | All records from both objects | - -### Join Strategies - -| Strategy | Description | -|:---|:---| -| `auto` | Let the engine choose the best strategy | -| `database` | Push join to the database level | -| `hash` | In-memory hash join | -| `loop` | Nested loop join | - -### Join Example +Use `expand` for relationship traversal — the live spelling for related records — or a +dotted `fields` path (`'customer.name'`) for a single related column: ```typescript { object: 'order', - fields: ['id', 'total', 'customer.name', 'customer.email'], - joins: [ - { - type: 'inner', - object: 'customer', - on: ['order.customer_id', '=', 'customer.id'], - strategy: 'auto' - } - ] + fields: ['id', 'total'], + expand: { + customer_id: { object: 'customer', fields: ['name', 'email'] } + } } ``` @@ -440,13 +420,7 @@ call in the SQL driver's query builder). Use `expand` for relationship traversal object: 'article', search: { query: 'kubernetes deployment', - fields: ['title', 'body', 'tags'], - fuzzy: true, - operator: 'and', - boost: { title: 2.0, body: 1.0 }, - minScore: 0.5, - language: 'english', - highlight: true + fields: ['title', 'body', 'tags'] } } ``` @@ -455,19 +429,20 @@ call in the SQL driver's query builder). Use `expand` for relationship traversal |:---|:---|:---| | `query` | `string` | Search text | | `fields` | `string[]` | Fields to search (optional — defaults to all searchable) | -| `fuzzy` | `boolean` | Enable fuzzy matching for typo tolerance | -| `operator` | `'and' \| 'or'` | How to combine search terms | -| `boost` | `Record` | Field relevance weights | -| `minScore` | `number` | Minimum relevance score (0–1) | -| `language` | `string` | Language for stemming/stopwords | -| `highlight` | `boolean` | Return highlighted matches | +| `fuzzy` | `boolean` | `[EXPERIMENTAL — not enforced]` Fuzzy matching for typo tolerance | +| `operator` | `'and' \| 'or'` | `[EXPERIMENTAL — not enforced]` How to combine search terms | +| `boost` | `Record` | `[EXPERIMENTAL — not enforced]` Field relevance weights | +| `minScore` | `number` | `[EXPERIMENTAL — not enforced]` Minimum relevance score (0–1) | +| `language` | `string` | `[EXPERIMENTAL — not enforced]` Language for stemming/stopwords | +| `highlight` | `boolean` | `[EXPERIMENTAL — not enforced]` Return highlighted matches | Only `query` and `fields` are implemented. The engine expands `search` into a driver-agnostic `$and`-of-`$or`-of-`$contains` filter (ADR-0061) — `fuzzy`, `operator`, `boost`, `minScore`, `language`, and `highlight` are accepted by `QuerySchema` but read nowhere in -`expandSearchToFilter()` / `normalizeSearch()`, so they have no effect. Multiple search terms -are always AND-ed regardless of `operator`. +`expandSearchToFilter()` / `normalizeSearch()`, so they have no effect; since #4286 their +`.describe()` markers say so. Multiple search terms are always AND-ed regardless of +`operator`. ### Pinyin recall (Chinese deployments) @@ -496,46 +471,36 @@ by `@objectstack/plugin-pinyin-search`) recomputes the column on demand. --- -## Window Functions +## Window Functions — removed from the request surface -`windowFunctions` is only reachable by calling the SQL driver's `findWithWindowFunctions()` -method directly. `ObjectQL.find()` / `.aggregate()` and the `POST /api/v1/data/:object/query` -route never call it, so sending `windowFunctions` through the standard client/REST query path -has no effect. +`query.windowFunctions` was **removed in `@objectstack/spec` 18** (#4286): +`ObjectQL.find()` / `.aggregate()` and the `POST /api/v1/data/:object/query` route never +routed it anywhere, so sending it had no effect. The key is tombstoned — a query carrying +it fails to parse with the upgrade prescription — and the +`WindowFunction`/`WindowSpec`/`WindowFunctionNode` exports left with it. -| Function | Description | -|:---|:---| -| `row_number` | Sequential row number within partition | -| `rank` | Rank with gaps for ties | -| `dense_rank` | Rank without gaps for ties | -| `percent_rank` | Relative rank as percentage | -| `lag` | Access previous row value | -| `lead` | Access next row value | -| `first_value` | First value in window | -| `last_value` | Last value in window | -| `sum` / `avg` / `count` / `min` / `max` | Running aggregations | - -### Window Function Example +Window functions remain a **SQL-driver door**: `findWithWindowFunctions()`, callable +directly on a SQL driver instance (it is not on the `IDataDriver` contract). Its input +is the driver's own flat shape: ```typescript -{ - object: 'employee', - fields: ['name', 'department', 'salary'], +const ranked = await sqlDriver.findWithWindowFunctions('employee', { windowFunctions: [ { function: 'rank', alias: 'salary_rank', - over: { - partitionBy: ['department'], - orderBy: [{ field: 'salary', order: 'desc' }] - } + partitionBy: ['department'], + orderBy: [{ field: 'salary', order: 'desc' }] } ] -} +}); ``` +For request-level analytics, use `aggregations` + `groupBy`, or model rankings in +report/dashboard metadata. + --- ## Distinct & Group By diff --git a/content/docs/protocol/objectql/index.mdx b/content/docs/protocol/objectql/index.mdx index ba7947d872..0cbd3c76ec 100644 --- a/content/docs/protocol/objectql/index.mdx +++ b/content/docs/protocol/objectql/index.mdx @@ -396,6 +396,6 @@ See [Security Protocol](/docs/protocol/objectql/security) for details. diff --git a/content/docs/protocol/objectql/query-syntax.mdx b/content/docs/protocol/objectql/query-syntax.mdx index 3eb59416f2..55deed504a 100644 --- a/content/docs/protocol/objectql/query-syntax.mdx +++ b/content/docs/protocol/objectql/query-syntax.mdx @@ -70,11 +70,9 @@ interface QueryAST { offset?: number; // Skip records (OFFSET) top?: number; // Alias for limit (OData compat) cursor?: Record; // Keyset pagination cursor - joins?: JoinNode[]; // Explicit JOINs aggregations?: AggregationNode[]; // Aggregation functions groupBy?: GroupByNode[]; // GROUP BY targets (string | object) having?: FilterCondition; // HAVING clause - windowFunctions?: WindowFunctionNode[]; // Window functions (OVER) distinct?: boolean; // SELECT DISTINCT expand?: Record; // Recursive relation loading } @@ -92,14 +90,22 @@ on the `find()` path: | Member | Status | |:-------|:-------| -| `joins` | No driver reads it — there is no `query.joins` / `ast.joins` consumer anywhere in `packages/` | | `having` | Never read: `engine.aggregate()` forwards only `object`/`where`/`groupBy`/`aggregations`, and the in-memory fallback has no HAVING stage | | `cursor` | Accepted by `EngineQueryOptions`, but no driver implements keyset pagination | | `distinct` | Not applied by `find()`; the SQL and in-memory drivers expose a separate `distinct(object, field, filters?)` method instead | -| `windowFunctions` | Not applied by `find()`; `SqlDriver.findWithWindowFunctions()` is a separate method that is not on the `IDataDriver` contract | -| `search.fuzzy` / `boost` / `operator` / `minScore` / `language` / `highlight` | Parsed by the schema but ignored: only `query` and `fields` drive the expansion | +| `aggregations[].filter` | `[EXPERIMENTAL — not enforced]` — a SQL `FILTER (WHERE …)` affordance neither the SQL builders nor the in-memory fallback applies | +| `search.fuzzy` / `boost` / `operator` / `minScore` / `language` / `highlight` | `[EXPERIMENTAL — not enforced]` — only `query` and `fields` drive the expansion | `top` is the exception that *is* honored — the engine normalises it to `limit`. + +Two members left this table by being **removed** (#4286, ADR-0049 enforce-or-remove): +`joins` and `windowFunctions` are tombstoned in `@objectstack/spec` 18 — a query +carrying either fails to parse with the upgrade prescription, and authoring one is a +`tsc` error. Related records are read through `expand`; window functions remain a +SQL-driver door (`SqlDriver.findWithWindowFunctions()`). The inert members that +*remain* declared (`having`, `cursor`, `distinct`, and the experimental flags above) +are tracked in the liveness ledger (`packages/spec/liveness/query.json`) pending +their #4286 dispositions. ### Key Types @@ -118,7 +124,7 @@ interface AggregationNode { field?: string; // optional for COUNT(*) alias: string; // result column alias distinct?: boolean; // DISTINCT before aggregation — in-memory path only - filter?: FilterCondition; // FILTER WHERE clause — protocol only, never applied + filter?: FilterCondition; // [EXPERIMENTAL — not enforced] FILTER WHERE clause — never applied } // FieldNode — one entry of the select list. A field name, optionally dotted to @@ -742,40 +748,34 @@ expansion emits a plain `$contains`, which `SqlDriver` compiles to a parameteris `LIKE '%…%'` with no case folding — so the dialect's own `LIKE`/collation rules decide — while the in-memory driver matches with a case-insensitive regex. Only `select` / `status` option *labels* are matched case-insensitively by the expansion itself. -`fuzzy`, `boost`, `operator`, `minScore`, `language`, and `highlight` are accepted by -the schema but ignored by the expansion. - -### Joins (protocol only) +`fuzzy`, `boost`, `operator`, `minScore`, `language`, and `highlight` carry +`[EXPERIMENTAL — not enforced]` markers (#4286): the schema accepts them, the +expansion ignores them. -`QuerySchema` defines a `joins` array, but **no driver reads it** — there is no -`query.joins` consumer anywhere in `packages/`, so a query carrying `joins` silently -runs as a single-table query. Use `expand` (§4) for relationship loading, or two -queries joined in application code. +### Joins — removed (#4286) -For reference, the protocol shape is: +`query.joins` was **removed in `@objectstack/spec` 18** (#4286, ADR-0049 +enforce-or-remove): no driver ever read it, so a query carrying `joins` silently ran +as a single-table query. The key is tombstoned — authoring it is a `tsc` error, and a +query that still carries it (even as an empty array) fails to parse with the upgrade +prescription. The `JoinNode` / `JoinType` / `JoinStrategy` exports left with it. -```typescript -const query: QueryAST = { - object: 'order', - fields: ['id', 'amount'], - joins: [ - { - type: 'inner', - object: 'customer', - alias: 'c', - on: { 'order.customer_id': { $eq: { $field: 'c.id' } } }, - }, - ], -}; -``` +Use `expand` (§4) for relationship loading — the live spelling for related records — +a dotted `fields` path (`'owner.name'`) for a single related column, or two queries +joined in application code. -**Join types:** `inner`, `left`, `right`, `full` +### Window Functions — removed from the request surface (#4286) -### Window Functions +`query.windowFunctions` was **removed in `@objectstack/spec` 18** (#4286): `find()` +never applied it, so every OVER clause it declared was silently dropped. The key is +tombstoned, and the `WindowFunction` / `WindowSpec` / `WindowFunctionNode` exports +left with it — they declared `field` / `over` / `frame` members that no executor ever +read. -`windowFunctions` on a `QueryAST` is ignored by `find()`. The SQL drivers implement -window functions through a separate `findWithWindowFunctions()` method, which is not on -the `IDataDriver` contract and is not surfaced by `IDataEngine`: +Window functions remain a **SQL-driver door**: `findWithWindowFunctions()`, which is +not on the `IDataDriver` contract and is not surfaced by `IDataEngine`. Its input is +the driver's own flat shape — function name, alias, and optional flat +`partitionBy` / `orderBy`: ```typescript const ranked = await driver.findWithWindowFunctions('order', { @@ -783,10 +783,8 @@ const ranked = await driver.findWithWindowFunctions('order', { { function: 'row_number', alias: 'rank', - over: { - partitionBy: ['customer_id'], - orderBy: [{ field: 'amount', order: 'desc' }], - }, + partitionBy: ['customer_id'], + orderBy: [{ field: 'amount', order: 'desc' }], }, ], }); @@ -796,7 +794,9 @@ const ranked = await driver.findWithWindowFunctions('order', { ``` That method always selects `*` plus the window columns — it does not honor a `fields` -projection. +projection, and niladic rendering means argument-taking functions (`LAG(field)`) +emit without their argument. For request-level analytics use `aggregations` + +`groupBy` (§5). --- diff --git a/content/docs/protocol/objectql/schema.mdx b/content/docs/protocol/objectql/schema.mdx index e3d74f254e..42519167ac 100644 --- a/content/docs/protocol/objectql/schema.mdx +++ b/content/docs/protocol/objectql/schema.mdx @@ -968,7 +968,7 @@ indexes: ` | optional | Cursor for keyset pagination | -| **joins** | `{ type: Enum<'inner' \| 'left' \| 'right' \| 'full'>; strategy?: Enum<'auto' \| 'database' \| 'hash' \| 'loop'>; object: string; alias?: string; … }[]` | optional | Explicit Table Joins | +| **joins** | `any` | optional | [REMOVED] `query.joins` was removed in @objectstack/spec 18 (#4286, ADR-0049) — no engine or driver ever read it: a query carrying `joins` behaved exactly as if the key were absent, while its name squatted on the reserved REST parameter set. Delete the key. Related records are read through `expand` — `expand: { owner: { object: 'user', fields: ['name'] } }` — which the engine resolves via batch $in queries, and a single related column is a dotted `fields` path (`fields: ['owner.name']`). | | **aggregations** | `{ function: Enum<'count' \| 'sum' \| 'avg' \| 'min' \| 'max' \| 'count_distinct' \| 'array_agg' \| 'string_agg'>; field?: string; alias: string; distinct?: boolean; … }[]` | optional | Aggregation functions | | **groupBy** | `string \| { field: string; dateGranularity?: Enum<'day' \| 'week' \| 'month' \| 'quarter' \| 'year'>; alias?: string }[]` | optional | GROUP BY targets (strings or `{field, dateGranularity?}` objects for date bucketing) | | **having** | `any` | optional | HAVING clause for aggregation filtering | -| **windowFunctions** | `{ function: Enum<'row_number' \| 'rank' \| 'dense_rank' \| 'percent_rank' \| 'lag' \| 'lead' \| 'first_value' \| 'last_value' \| 'sum' \| 'avg' \| 'count' \| 'min' \| 'max'>; field?: string; alias: string; over: object }[]` | optional | Window functions with OVER clause | +| **windowFunctions** | `any` | optional | [REMOVED] `query.windowFunctions` was removed in @objectstack/spec 18 (#4286, ADR-0049) — `find()` never applied it: no engine or driver read the key on the query path, so every OVER clause it declared was silently dropped. Delete the key. Window functions are a SQL-driver capability behind `SqlDriver.findWithWindowFunctions(object, query)` (embedder-level; not on the `IDataDriver` contract or the REST surface); request-level analytics are `aggregations` + `groupBy`. | | **distinct** | `boolean` | optional | SELECT DISTINCT flag | | **expand** | `Record` | optional | Recursive relation loading map. Keys are lookup/master_detail field names; values are nested QueryAST objects that control select (`fields`) and filter (`where`, AND-merged with the batch $in), plus further expansion on the related object. The engine resolves expand via batch $in queries (driver-agnostic) with a default max depth of 3; per-parent `limit`/`offset`/`orderBy` are NOT applied on this path. | diff --git a/content/docs/references/data/query.mdx b/content/docs/references/data/query.mdx index 8a62809d65..633a9138b4 100644 --- a/content/docs/references/data/query.mdx +++ b/content/docs/references/data/query.mdx @@ -16,8 +16,8 @@ Represents "Order By". ## TypeScript Usage ```typescript -import { AggregationFunction, AggregationNode, DateGranularity, FieldNode, FullTextSearch, GroupByNode, JoinNode, JoinStrategy, JoinType, Query, SortNode, WindowFunction, WindowFunctionNode, WindowSpec } from '@objectstack/spec/data'; -import type { AggregationFunction, AggregationNode, DateGranularity, FieldNode, FullTextSearch, GroupByNode, JoinNode, JoinStrategy, JoinType, Query, SortNode, WindowFunction, WindowFunctionNode, WindowSpec } from '@objectstack/spec/data'; +import { AggregationFunction, AggregationNode, DateGranularity, FieldNode, FullTextSearch, GroupByNode, Query, SortNode } from '@objectstack/spec/data'; +import type { AggregationFunction, AggregationNode, DateGranularity, FieldNode, FullTextSearch, GroupByNode, Query, SortNode } from '@objectstack/spec/data'; // Validate data const result = AggregationFunction.parse(data); @@ -51,7 +51,7 @@ const result = AggregationFunction.parse(data); | **field** | `string` | optional | Field to aggregate (optional for COUNT(*)) | | **alias** | `string` | ✅ | Result column alias | | **distinct** | `boolean` | optional | Apply DISTINCT before aggregation | -| **filter** | `any` | optional | Filter/Condition to apply to the aggregation (FILTER WHERE clause) | +| **filter** | `any` | optional | [EXPERIMENTAL — not enforced] Per-aggregation filter (SQL FILTER (WHERE …)). Neither the SQL builders nor the in-memory fallback applies it (#4286); filter the whole query with `where` instead. | --- @@ -80,12 +80,12 @@ const result = AggregationFunction.parse(data); | :--- | :--- | :--- | :--- | | **query** | `string` | ✅ | Search query text | | **fields** | `string[]` | optional | Fields to search in (if not specified, searches all text fields) | -| **fuzzy** | `boolean` | ✅ | Enable fuzzy matching (tolerates typos) | -| **operator** | `Enum<'and' \| 'or'>` | ✅ | Logical operator between terms | -| **boost** | `Record` | optional | Field-specific relevance boosting (field name -> boost factor) | -| **minScore** | `number` | optional | Minimum relevance score threshold | -| **language** | `string` | optional | Language for text analysis (e.g., "en", "zh", "es") | -| **highlight** | `boolean` | ✅ | Enable search result highlighting | +| **fuzzy** | `boolean` | ✅ | [EXPERIMENTAL — not enforced] Fuzzy matching (tolerate typos). The ADR-0061 expansion reads only `query` + `fields`; no executor receives this flag (#4286). | +| **operator** | `Enum<'and' \| 'or'>` | ✅ | [EXPERIMENTAL — not enforced] Logical operator between terms. The ADR-0061 expansion applies its own term semantics; no executor receives this flag (#4286). | +| **boost** | `Record` | optional | [EXPERIMENTAL — not enforced] Field-specific relevance boosting (field name -> boost factor). No executor scores results (#4286). | +| **minScore** | `number` | optional | [EXPERIMENTAL — not enforced] Minimum relevance score threshold. No executor scores results (#4286). | +| **language** | `string` | optional | [EXPERIMENTAL — not enforced] Language for text analysis (e.g., "en", "zh", "es"). No executor selects an analyzer (#4286). | +| **highlight** | `boolean` | ✅ | [EXPERIMENTAL — not enforced] Search result highlighting. No executor emits highlights (#4286). | --- @@ -115,46 +115,6 @@ Type: `string` --- ---- - -## JoinNode - -### Properties - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **type** | `Enum<'inner' \| 'left' \| 'right' \| 'full'>` | ✅ | Join type | -| **strategy** | `Enum<'auto' \| 'database' \| 'hash' \| 'loop'>` | optional | Execution strategy hint | -| **object** | `string` | ✅ | Object/table to join | -| **alias** | `string` | optional | Table alias | -| **on** | `any` | ✅ | Join condition | -| **subquery** | `{ object: string; fields?: string[]; where?: any; search?: object; … }` | optional | Subquery instead of object | - - ---- - -## JoinStrategy - -### Allowed Values - -* `auto` -* `database` -* `hash` -* `loop` - - ---- - -## JoinType - -### Allowed Values - -* `inner` -* `left` -* `right` -* `full` - - --- ## Query @@ -172,11 +132,11 @@ Type: `string` | **offset** | `number` | optional | Records to skip (OFFSET) | | **top** | `number` | optional | Alias for limit (OData compatibility) | | **cursor** | `Record` | optional | Cursor for keyset pagination | -| **joins** | `{ type: Enum<'inner' \| 'left' \| 'right' \| 'full'>; strategy?: Enum<'auto' \| 'database' \| 'hash' \| 'loop'>; object: string; alias?: string; … }[]` | optional | Explicit Table Joins | +| **joins** | `any` | optional | [REMOVED] `query.joins` was removed in @objectstack/spec 18 (#4286, ADR-0049) — no engine or driver ever read it: a query carrying `joins` behaved exactly as if the key were absent, while its name squatted on the reserved REST parameter set. Delete the key. Related records are read through `expand` — `expand: { owner: { object: 'user', fields: ['name'] } }` — which the engine resolves via batch $in queries, and a single related column is a dotted `fields` path (`fields: ['owner.name']`). | | **aggregations** | `{ function: Enum<'count' \| 'sum' \| 'avg' \| 'min' \| 'max' \| 'count_distinct' \| 'array_agg' \| 'string_agg'>; field?: string; alias: string; distinct?: boolean; … }[]` | optional | Aggregation functions | | **groupBy** | `string \| { field: string; dateGranularity?: Enum<'day' \| 'week' \| 'month' \| 'quarter' \| 'year'>; alias?: string }[]` | optional | GROUP BY targets (strings or `{field, dateGranularity?}` objects for date bucketing) | | **having** | `any` | optional | HAVING clause for aggregation filtering | -| **windowFunctions** | `{ function: Enum<'row_number' \| 'rank' \| 'dense_rank' \| 'percent_rank' \| 'lag' \| 'lead' \| 'first_value' \| 'last_value' \| 'sum' \| 'avg' \| 'count' \| 'min' \| 'max'>; field?: string; alias: string; over: object }[]` | optional | Window functions with OVER clause | +| **windowFunctions** | `any` | optional | [REMOVED] `query.windowFunctions` was removed in @objectstack/spec 18 (#4286, ADR-0049) — `find()` never applied it: no engine or driver read the key on the query path, so every OVER clause it declared was silently dropped. Delete the key. Window functions are a SQL-driver capability behind `SqlDriver.findWithWindowFunctions(object, query)` (embedder-level; not on the `IDataDriver` contract or the REST surface); request-level analytics are `aggregations` + `groupBy`. | | **distinct** | `boolean` | optional | SELECT DISTINCT flag | | **expand** | `Record` | optional | Recursive relation loading map. Keys are lookup/master_detail field names; values are nested QueryAST objects that control select (`fields`) and filter (`where`, AND-merged with the batch $in), plus further expansion on the related object. The engine resolves expand via batch $in queries (driver-agnostic) with a default max depth of 3; per-parent `limit`/`offset`/`orderBy` are NOT applied on this path. | @@ -195,51 +155,3 @@ Type: `string` --- -## WindowFunction - -### Allowed Values - -* `row_number` -* `rank` -* `dense_rank` -* `percent_rank` -* `lag` -* `lead` -* `first_value` -* `last_value` -* `sum` -* `avg` -* `count` -* `min` -* `max` - - ---- - -## WindowFunctionNode - -### Properties - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **function** | `Enum<'row_number' \| 'rank' \| 'dense_rank' \| 'percent_rank' \| 'lag' \| 'lead' \| 'first_value' \| 'last_value' \| 'sum' \| 'avg' \| 'count' \| 'min' \| 'max'>` | ✅ | Window function name | -| **field** | `string` | optional | Field to operate on (for aggregate window functions) | -| **alias** | `string` | ✅ | Result column alias | -| **over** | `{ partitionBy?: string[]; orderBy?: { field: string; order: Enum<'asc' \| 'desc'> }[]; frame?: object }` | ✅ | Window specification (OVER clause) | - - ---- - -## WindowSpec - -### Properties - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **partitionBy** | `string[]` | optional | PARTITION BY fields | -| **orderBy** | `{ field: string; order: Enum<'asc' \| 'desc'> }[]` | optional | ORDER BY specification | -| **frame** | `{ type?: Enum<'rows' \| 'range'>; start?: string; end?: string }` | optional | Window frame specification | - - ---- - diff --git a/docs/audits/2026-07-unknown-key-strictness-ledger.md b/docs/audits/2026-07-unknown-key-strictness-ledger.md index 769dc5054b..f6bf6bb4ac 100644 --- a/docs/audits/2026-07-unknown-key-strictness-ledger.md +++ b/docs/audits/2026-07-unknown-key-strictness-ledger.md @@ -146,7 +146,7 @@ tightening (the #4001 "sharing-rule lesson": candidates, not verdicts). | `notification.zod.ts` / `offline.zod.ts` / `report.zod.ts` | 3 ea | authorable (p) | | | `sharing.zod.ts` | 2 | authorable (p) | public-sharing config | -### `data/` — 153 sites +### `data/` — 149 sites | File | Sites | Class | Note | |---|---|---|---| @@ -155,7 +155,7 @@ tightening (the #4001 "sharing-rule lesson": candidates, not verdicts). | `external-lookup.zod.ts` | 12 | mixed (p) | authored config + wire results | | `seed-loader.zod.ts` | 12 | mixed (p) | seed file shapes are authored; loader state is runtime | | `field.zod.ts` | 11 | authorable | partially strict | -| `filter.zod.ts` / `query.zod.ts` | 11+9 | open | query dialect — user data flows through; validated semantically elsewhere. `query.zod.ts` dropped one site in #4196: `FieldNodeSchema`'s nested-select object form was declared-but-inert and narrowed to `z.string()`, so the union's second member is gone. Class unchanged | +| `filter.zod.ts` / `query.zod.ts` | 11+5 | open | query dialect — user data flows through; validated semantically elsewhere. `query.zod.ts` dropped one site in #4196: `FieldNodeSchema`'s nested-select object form was declared-but-inert and narrowed to `z.string()`, so the union's second member is gone. Four more left in #4286 with the `joins`/`windowFunctions` removals: `JoinNodeBaseSchema`, `WindowFunctionNodeSchema`, and `WindowSpecSchema`'s two blocks (outer + `frame`) were deleted with their clusters. Class unchanged | | `driver-nosql.zod.ts` / `driver.zod.ts` / `driver-sql.zod.ts` | 10+9+2 | wire | driver capability contracts | | `datasource.zod.ts` | 9 | authorable | **strict as of #4001 data step** — all 9: `DatasourceSchema` (+ `pool` / `healthCheck` / `ssl` / `retryPolicy`), `ExternalDatasourceSettingsSchema` (+ `validation`), `DatasourceCapabilities`, `DriverDefinitionSchema`. `config` + `readReplicas` stay `z.record` by construction (per-driver shapes; the driver's own `configSchema` validates them) — which is precisely why the top level had to close: a connection key written one level too high was stripped, and the datasource then connected on driver defaults instead of failing | | `analytics.zod.ts` | 8 | mixed (p) | | diff --git a/packages/metadata-protocol/src/protocol.ts b/packages/metadata-protocol/src/protocol.ts index 63a108f2b2..e60a44b08f 100644 --- a/packages/metadata-protocol/src/protocol.ts +++ b/packages/metadata-protocol/src/protocol.ts @@ -776,6 +776,16 @@ function mergeDroppedFieldEvents(events: DroppedFieldsEvent[]): DroppedFieldsEve * decides what is a query parameter and what is a field filter — silently * drifting from the AST would resurrect exactly the #4134 failure for whatever * key was added. + * + * The #4286 tombstones (`joins`, `windowFunctions`) still count: `retiredKey()` + * keeps a retired key in `keyof QueryAST`, so both stay listed — and therefore + * deliberately stay RESERVED at this boundary while the tombstone lives. That + * is the right compat posture: a caller still sending one belongs with the + * prescription, not with a silent `where.joins` filter. When a tombstone ages + * out (~two majors) and the key leaves the spec, the excess-property error + * here is the reminder that deleting its line UN-reserves the name — an object + * field genuinely called `joins` would start resolving as an implicit filter, + * which is a behavior change to call out in that changeset. */ const QUERY_AST_KEYS: Readonly> = { object: true, fields: true, where: true, search: true, orderBy: true, diff --git a/packages/objectql/src/protocol-data.test.ts b/packages/objectql/src/protocol-data.test.ts index ff9c728592..ec7b6d255c 100644 --- a/packages/objectql/src/protocol-data.test.ts +++ b/packages/objectql/src/protocol-data.test.ts @@ -948,6 +948,11 @@ describe('ObjectStackProtocolImplementation - Data Operations', () => { // `client.data.query(object, query)` posts a `Partial`, // whose documented shape includes `object`. Treated as a field // filter it would become `where.object` and match zero rows. + // `joins`/`windowFunctions` are tombstoned since #4286 but stay in + // `keyof QueryAST` (and so in this reserved set) while the + // tombstone lives: a caller still sending one must meet the + // prescription wherever the schema parses, never a silent + // `where.joins` filter. const { protocol, engine } = makeProtocol(); await protocol.findData({ object: 'showcase_task', diff --git a/packages/plugins/plugin-security/src/predicate-guard.test.ts b/packages/plugins/plugin-security/src/predicate-guard.test.ts index ce2465b26f..bfa211b8f1 100644 --- a/packages/plugins/plugin-security/src/predicate-guard.test.ts +++ b/packages/plugins/plugin-security/src/predicate-guard.test.ts @@ -29,19 +29,19 @@ describe('collectConditionFields', () => { }); describe('collectQueryFields', () => { - it('covers where / orderBy / groupBy / having / aggregations / window functions', () => { + // `windowFunctions` left this walk with the `QueryAST` key (#4286) — the + // tombstone refuses it wherever the schema parses, and no executor ever ran + // one off the query path, so there is no clause left to leak through. + it('covers where / orderBy / groupBy / having / aggregations', () => { const fields = collectQueryFields({ where: { status: 'open' }, orderBy: [{ field: 'salary', order: 'desc' }], groupBy: ['department', { field: 'hired_at', dateGranularity: 'month' }], having: { headcount: { $gt: 3 } }, aggregations: [{ function: 'sum', field: 'bonus', alias: 'total', filter: { region: 'emea' } }], - windowFunctions: [ - { function: 'row_number', alias: 'r', over: { partitionBy: ['team'], orderBy: [{ field: 'score', order: 'desc' }] } }, - ], }); expect([...fields].sort()).toEqual([ - 'bonus', 'department', 'headcount', 'hired_at', 'region', 'salary', 'score', 'status', 'team', + 'bonus', 'department', 'headcount', 'hired_at', 'region', 'salary', 'status', ]); }); diff --git a/packages/plugins/plugin-security/src/predicate-guard.ts b/packages/plugins/plugin-security/src/predicate-guard.ts index 3af48b422e..797bddef5c 100644 --- a/packages/plugins/plugin-security/src/predicate-guard.ts +++ b/packages/plugins/plugin-security/src/predicate-guard.ts @@ -52,11 +52,17 @@ export function collectConditionFields(condition: unknown, out: Set = ne /** * Collect every field referenced by the query's row-shaping clauses: - * where / orderBy / groupBy / having / aggregations (field + FILTER) / - * window functions (field + partitionBy + over.orderBy). `fields` - * (projection) is intentionally NOT collected — selecting a hidden field is - * harmless because FieldMasker strips it from the result; only predicates - * leak. + * where / orderBy / groupBy / having / aggregations (field + FILTER). + * `fields` (projection) is intentionally NOT collected — selecting a hidden + * field is harmless because FieldMasker strips it from the result; only + * predicates leak. + * + * `windowFunctions` was walked here too until the key was removed from + * `QueryAST` (#4286) — the tombstone refuses it wherever the schema parses, + * and no executor ever ran a window function off the query path, so there is + * no clause left to leak through. `having` and `aggregations[].filter` stay + * walked while they stay declared (#4286 leaves `having`'s enforce-or-remove + * call open): this guard being ready is what makes enforcing them later safe. */ export function collectQueryFields(ast: Record): Set { const out = new Set(); @@ -90,24 +96,6 @@ export function collectQueryFields(ast: Record): Set { } } - const windowFunctions = ast.windowFunctions; - if (Array.isArray(windowFunctions)) { - for (const w of windowFunctions) { - const field = (w as { field?: unknown })?.field; - if (typeof field === 'string') out.add(field.split('.')[0]); - const over = (w as { over?: { partitionBy?: unknown; orderBy?: unknown } })?.over; - if (Array.isArray(over?.partitionBy)) { - for (const p of over.partitionBy) if (typeof p === 'string') out.add(p.split('.')[0]); - } - if (Array.isArray(over?.orderBy)) { - for (const s of over.orderBy) { - const f = (s as { field?: unknown })?.field; - if (typeof f === 'string') out.add(f.split('.')[0]); - } - } - } - } - return out; } diff --git a/packages/spec/api-surface.json b/packages/spec/api-surface.json index 03d2d0cc73..2141d7c539 100644 --- a/packages/spec/api-surface.json +++ b/packages/spec/api-surface.json @@ -397,11 +397,6 @@ "InstantValueSchema (const)", "JSONValidation (type)", "JSONValidationSchema (const)", - "JoinNode (type)", - "JoinNodeInput (type)", - "JoinNodeSchema (const)", - "JoinStrategy (const)", - "JoinType (const)", "LEGACY_API_METHODS (const)", "LEGACY_API_METHOD_GUIDANCE (const)", "LIFECYCLE_DURATION_REGEX (const)", @@ -581,11 +576,6 @@ "ValidationRuleSchema (const)", "ValueForm (type)", "ValueShapeFieldDef (interface)", - "WindowFunction (const)", - "WindowFunctionNode (type)", - "WindowFunctionNodeSchema (const)", - "WindowSpec (type)", - "WindowSpecSchema (const)", "canonicalAstOperator (function)", "canonicalizeSqlType (function)", "classifyFilterToken (function)", diff --git a/packages/spec/authorable-surface.json b/packages/spec/authorable-surface.json index 9c94819925..e1b4c4759e 100644 --- a/packages/spec/authorable-surface.json +++ b/packages/spec/authorable-surface.json @@ -3557,12 +3557,6 @@ "data/JSONValidation:severity", "data/JSONValidation:tags", "data/JSONValidation:type", - "data/JoinNode:alias", - "data/JoinNode:object", - "data/JoinNode:on", - "data/JoinNode:strategy", - "data/JoinNode:subquery", - "data/JoinNode:type", "data/Lifecycle:archive", "data/Lifecycle:class", "data/Lifecycle:reclaim", @@ -3736,7 +3730,7 @@ "data/Query:fields", "data/Query:groupBy", "data/Query:having", - "data/Query:joins", + "data/Query:joins [RETIRED]", "data/Query:limit", "data/Query:object", "data/Query:offset", @@ -3744,7 +3738,7 @@ "data/Query:search", "data/Query:top", "data/Query:where", - "data/Query:windowFunctions", + "data/Query:windowFunctions [RETIRED]", "data/QueryFilter:where", "data/ReferenceResolution:field", "data/ReferenceResolution:fieldType", @@ -3864,13 +3858,6 @@ "data/StringOperator:$startsWith", "data/TenancyConfig:enabled", "data/TenancyConfig:tenantField", - "data/WindowFunctionNode:alias", - "data/WindowFunctionNode:field", - "data/WindowFunctionNode:function", - "data/WindowFunctionNode:over", - "data/WindowSpec:frame", - "data/WindowSpec:orderBy", - "data/WindowSpec:partitionBy", "identity/Account:accessToken", "identity/Account:createdAt", "identity/Account:expiresAt", diff --git a/packages/spec/json-schema.manifest.json b/packages/spec/json-schema.manifest.json index 266ead7820..862edf9a14 100644 --- a/packages/spec/json-schema.manifest.json +++ b/packages/spec/json-schema.manifest.json @@ -1,5 +1,5 @@ { - "description": "Ratchet manifest of every JSON Schema emitted by scripts/build-schemas.ts. Auto-appended when new schemas are added (commit the change). A listed schema that a build no longer emits fails gen:schema — remove a key ONLY for a deliberate retirement. See #2978.", + "description": "Ratchet manifest of every JSON Schema emitted by scripts/build-schemas.ts. Auto-appended when new schemas are added (commit the change). A listed schema that a build no longer emits fails gen:schema \u2014 remove a key ONLY for a deliberate retirement. See #2978.", "schemas": [ "ai/AIModelConfig", "ai/AIUsageRecord", @@ -776,9 +776,6 @@ "data/Index", "data/InstantValue", "data/JSONValidation", - "data/JoinNode", - "data/JoinStrategy", - "data/JoinType", "data/Lifecycle", "data/LifecycleClass", "data/LocationCoordinates", @@ -837,9 +834,6 @@ "data/TransformType", "data/UniqueScope", "data/ValidationRule", - "data/WindowFunction", - "data/WindowFunctionNode", - "data/WindowSpec", "identity/Account", "identity/ApiKey", "identity/EvalUser", diff --git a/packages/spec/liveness/README.md b/packages/spec/liveness/README.md index 55eb900cc5..cb07b6ac43 100644 --- a/packages/spec/liveness/README.md +++ b/packages/spec/liveness/README.md @@ -34,6 +34,14 @@ disconnected from the `sys_webhook` dispatcher (#3461). Since being off the regi disconnect like `webhook`'s is resolved (materializer built or surface retired), fold the type back onto the registry and drop the override. +`query` is the second override, and a permanent one (#4286): `QuerySchema` is not +metadata at all but the **request surface** — the client SDK QueryBuilder's output and +the `POST /data/:object/query` body. It is authorable by every API caller yet never +stored as stack metadata, which is exactly why 12 declared-but-inert members survived +every other gate: this ledger governed what authors write into metadata files, and +nothing governed what callers write into a query. Unlike `webhook`, there is no +registry to fold it back onto — the override *is* its governance. + ## Status vocabulary | Status | Meaning | @@ -440,7 +448,7 @@ The governed set is `GOVERNED` at the top of `check-liveness.mts`. To add a type RecordDetailView had been gating the History tab on it the whole time (#2707). 4. Add the type to `GOVERNED`; confirm the gate is green. -## Current state — 16 governed types +## Current state — 17 governed types Counts include drilled `children` entries; regenerate with the snippet below rather than hand-editing (this table drifted badly once — field was listed 34/39 while the @@ -479,6 +487,7 @@ EOF | report | 13 | 0 | 0 | – | dataset-bound (ADR-0021); the aria/performance LEDGER entries were stale — the keys left the schema in the report-liveness close-out; deleted 2026-07-30 as hygiene. Audit-era `chart` DEAD superseded (framework#1890 / #3441) | | dashboard | 10 | 0 | 2 | – | ADR-0021 dataset widgets (#3251; DashboardWidgetSchema `.strict()`); `aria`/`performance` (and widget `performance` + PerformanceConfigSchema) REMOVED 2026-07-30 (#3896 close-out sweep — no renderer applied any of them); audit-era `globalFilters`/`dateRange` DEAD superseded (framework#2501) | +| query | 15 | 7 | 5 | – | **not a metadata type** — the REQUEST surface (`QuerySchema`: client SDK QueryBuilder output; the `POST /data/:object/query` body), governed via `SPEC_ONLY_SCHEMAS` (#4286). The 7 experimental resolve from `[EXPERIMENTAL — not enforced]` describe markers, not ledger entries (search `fuzzy`/`operator`/`boost`/`minScore`/`language`/`highlight` + `aggregations[].filter` — declared engine affordances no executor receives). dead = the #4286 worklist: `joins`/`windowFunctions` REMOVED 2026-07-31 (tombstoned; protocol-18 semantic migrations `query-joins-retired`/`query-window-functions-retired`; the JoinNode and WindowFunctionNode clusters deleted with them), `cursor`/`distinct` (shipped SDK producers, no executor — `distinct` is MIS-WIRED: its only effect is REST count-suppression, protocol.ts:3816) and `having` (never reaches a driver — `engine.aggregate()` rebuilds the AST without it) pending their #4286 step-3/4 dispositions | | webhook | 0 | 1 | 16 | – | **not a registered metadata type** — governed via the gate's spec-only schema override (`SPEC_ONLY_SCHEMAS`), not `getMetadataTypeSchema` (#3461/#3462). The ENTIRE authoring surface is dead: nothing materializes an authored `webhooks:` entry into a `sys_webhook` dispatcher row (#3461, enforce-or-remove pending). `url` carries the single per-webhook `authorWarn` (one no-op heads-up per artifact, not per-prop); `authentication` experimental (HMAC-`secret`-only); `isActive` unmarked (default(true)). Notes cite the sys_webhook column map as the future materializer's mapping table | The `dead` set across types is the enforce-or-remove worklist (ADR-0049); every diff --git a/packages/spec/liveness/query.json b/packages/spec/liveness/query.json new file mode 100644 index 0000000000..8b244f02c5 --- /dev/null +++ b/packages/spec/liveness/query.json @@ -0,0 +1,117 @@ +{ + "type": "query", + "_note": "QuerySchema (BaseQuerySchema + expand) — packages/spec/src/data/query.zod.ts. NOT a registered metadata type: QueryAST is the REQUEST surface (the client SDK QueryBuilder's output; the POST /data/:object/query body), governed via the gate's SPEC_ONLY_SCHEMAS override like webhook (#4286). It is authorable by every API caller yet never stored as stack metadata, which is why no other gate reached it: this ledger read the metadata-type registry, check:authorable-surface walks the same registry, and check:exported-any is a type-resolution gate. Seeded 2026-07-31 from the #4286 sweep — joins/windowFunctions REMOVED (tombstoned; protocol-18 semantic migrations), search sub-flags + aggregations.filter marked experimental at the schema, cursor/distinct/having dead pending their #4286 dispositions. The main executor evidence below: objectql engine find/aggregate and the drivers' find paths.", + "props": { + "object": { + "status": "live", + "verifiedAt": "2026-07-31", + "evidence": "packages/objectql/src/engine.ts (find/aggregate resolve the target object from ast.object); packages/metadata-protocol/src/protocol.ts:780 (QUERY_AST_KEYS reserves it at the REST boundary so a POST body's `object` is never read as a field filter)" + }, + "fields": { + "status": "live", + "verifiedAt": "2026-07-31", + "evidence": "packages/plugins/driver-sql/src/sql-driver.ts:1353-1354 (select projection); driver-memory projectFields; objectql formula projection + known-field filters (closed end-to-end in the #4196 verification)" + }, + "where": { + "status": "live", + "verifiedAt": "2026-07-31", + "evidence": "packages/plugins/driver-sql/src/sql-driver.ts:1330-1331 (applyFilters); every driver's find() applies it" + }, + "search": { + "note": "Container. `query` + `fields` are the executed surface (ADR-0061); the six engine-affordance flags (fuzzy/operator/boost/minScore/language/highlight) resolve `experimental` from their `[EXPERIMENTAL — not enforced]` describe markers (#4286 step 1 — declared search-engine affordances no executor receives).", + "children": { + "query": { + "status": "live", + "verifiedAt": "2026-07-31", + "evidence": "packages/objectql/src/engine.ts:2860-2882 (ADR-0061: expanded into a server-resolved cross-field $or via expandSearchToFilter)" + }, + "fields": { + "status": "live", + "verifiedAt": "2026-07-31", + "evidence": "packages/objectql/src/engine.ts:2867-2870 (scopes the ADR-0061 expansion to the named columns)" + } + } + }, + "orderBy": { + "status": "live", + "verifiedAt": "2026-07-31", + "evidence": "packages/plugins/driver-sql/src/sql-driver.ts:1335-1342; every driver's find() sorts by it" + }, + "limit": { + "status": "live", + "verifiedAt": "2026-07-31", + "evidence": "packages/plugins/driver-sql/src/sql-driver.ts:1345" + }, + "offset": { + "status": "live", + "verifiedAt": "2026-07-31", + "evidence": "packages/plugins/driver-sql/src/sql-driver.ts:1344" + }, + "top": { + "status": "live", + "verifiedAt": "2026-07-31", + "evidence": "packages/objectql/src/engine.ts:2850-2853 (top → limit normalization, OData compatibility); packages/metadata-protocol/src/protocol.ts:3484-3486 (same fold on the REST list path)" + }, + "cursor": { + "status": "dead", + "verifiedAt": "2026-07-31", + "note": "#4286 Tier A: a public SDK method mints it — QueryBuilder.cursor() (packages/client/src/query-builder.ts:294), serialized to the wire at 5 sites in client/src/index.ts, and re-declared on EngineQueryOptionsSchema — but no driver implements keyset pagination, so it has no effect. Disposition PENDING (#4286 step 4): removal is ADR-0087 conversion/tombstone work that must take the SDK producer and the wire sites with it." + }, + "joins": { + "status": "dead", + "verifiedAt": "2026-07-31", + "note": "REMOVED 2026-07-31 (#4286) — tombstoned at the schema (retiredKey carries the prescription; authoring it is a tsc error and a parse error). No conversion strips it: QueryAST is a request surface, never stored in stack metadata, so the removal is the protocol-18 semantic migration `query-joins-retired`. The entry stays because retiredKey keeps the key in the walked shape (the rls.priority precedent). Related records are read through `expand` (batch $in resolution); the orphaned JoinNode/JoinType/JoinStrategy cluster left with the key." + }, + "aggregations": { + "note": "Container. `filter` resolves `experimental` from its `[EXPERIMENTAL — not enforced]` describe marker (#4286 — a SQL FILTER (WHERE …) affordance neither the SQL builders nor the in-memory fallback applies; the FLS predicate guard walks it, predicate-guard.ts:89).", + "children": { + "function": { + "status": "live", + "verifiedAt": "2026-07-31", + "evidence": "packages/objectql/src/engine.ts aggregate() forwards aggregations to the driver or the fallback; packages/objectql/src/in-memory-aggregation.ts:167 applies each function" + }, + "field": { + "status": "live", + "verifiedAt": "2026-07-31", + "evidence": "packages/objectql/src/in-memory-aggregation.ts:204-206 (collectValues reads rows[field]); SQL builders render FUNC(field)" + }, + "alias": { + "status": "live", + "verifiedAt": "2026-07-31", + "evidence": "result column naming in both the SQL builders and packages/objectql/src/in-memory-aggregation.ts" + }, + "distinct": { + "status": "live", + "verifiedAt": "2026-07-31", + "evidence": "packages/objectql/src/in-memory-aggregation.ts:167,204-206 (dedupe before applying the function)" + } + } + }, + "groupBy": { + "status": "live", + "verifiedAt": "2026-07-31", + "evidence": "packages/objectql/src/engine.ts aggregate() forwards groupBy; packages/objectql/src/in-memory-aggregation.ts buckets by it (including dateGranularity fallback)" + }, + "having": { + "status": "dead", + "verifiedAt": "2026-07-31", + "note": "Never delivered: engine.aggregate() rebuilds the AST it hands the driver with exactly object/where/groupBy/aggregations, so a driver that DID implement HAVING would never receive it (#4286 finding 1), and the in-memory fallback has no HAVING stage. The strongest ENFORCE candidate of the #4286 set — a genuinely expected SQL capability the fallback could apply in a few lines — so it is deliberately left declared pending that call (#4286 step 3). The FLS predicate guard already walks it (predicate-guard.ts:64), so enforcing it later leaks nothing." + }, + "windowFunctions": { + "status": "dead", + "verifiedAt": "2026-07-31", + "note": "REMOVED 2026-07-31 (#4286) — tombstoned at the schema; the protocol-18 semantic migration `query-window-functions-retired` carries it (request surface, no source to rewrite). The entry stays because retiredKey keeps the key in the walked shape (the rls.priority precedent). find() never applied a window function; the live door is SqlDriver.findWithWindowFunctions() (driver-level, its own flat input shape), and the WindowFunctionNode cluster — which declared field/over/frame members that door never read — left with the key. The FLS guard's window walk was pruned in the same change." + }, + "distinct": { + "status": "dead", + "verifiedAt": "2026-07-31", + "authorWarn": true, + "note": "#4286 finding 2 — MIS-WIRED, not merely dead (ADR-0078 treats this more harshly): no driver renders SELECT DISTINCT, but metadata-protocol/src/protocol.ts:3816 treats `options.distinct != null` as not-countable, so a distinct query silently skips engine.count() and degrades the REST total/hasMore to a page-local estimate — the caller gets non-distinct rows AND worse pagination metadata. QueryBuilder.distinct() (packages/client/src/query-builder.ts:302) mints it; also declared on EngineQueryOptionsSchema. Disposition PENDING (#4286 step 4): removal or enforcement must handle the count-suppression side effect deliberately either way." + }, + "expand": { + "status": "live", + "verifiedAt": "2026-07-31", + "evidence": "packages/objectql/src/engine.ts:2519+ (post-process: batch $in resolution of lookup/master_detail expands, default max depth 3)" + } + } +} diff --git a/packages/spec/scripts/liveness/check-liveness.mts b/packages/spec/scripts/liveness/check-liveness.mts index 12c8b633ce..653fc9ebac 100644 --- a/packages/spec/scripts/liveness/check-liveness.mts +++ b/packages/spec/scripts/liveness/check-liveness.mts @@ -49,6 +49,7 @@ import { fileURLToPath } from 'node:url'; import { dirname, join, resolve } from 'node:path'; import { getMetadataTypeSchema, listMetadataTypeSchemaTypes } from '../../src/kernel/metadata-type-schemas'; import { WebhookSchema } from '../../src/automation/webhook.zod'; +import { QuerySchema } from '../../src/data/query.zod'; import { BOUND_PROOF_PATHS, HIGH_RISK_CLASSES, @@ -71,7 +72,8 @@ const repoRoot = resolve(specRoot, '../..'); const ledgerRoot = join(specRoot, 'liveness'); // Governed metadata types, rolled out highest-frequency / highest-risk first. -const GOVERNED = ['object', 'field', 'flow', 'action', 'hook', 'permission', 'position', 'agent', 'tool', 'skill', 'dataset', 'page', 'view', 'report', 'dashboard', 'webhook']; +// (`query` is not a metadata type — see SPEC_ONLY_SCHEMAS below.) +const GOVERNED = ['object', 'field', 'flow', 'action', 'hook', 'permission', 'position', 'agent', 'tool', 'skill', 'dataset', 'page', 'view', 'report', 'dashboard', 'webhook', 'query']; // Spec-only override: governed types whose canonical schema is NOT (yet) in the // metadata-type registry, so they can't be resolved via getMetadataTypeSchema. @@ -87,8 +89,18 @@ const GOVERNED = ['object', 'field', 'flow', 'action', 'hook', 'permission', 'po // webhook.json), and whether to fold `webhook` back onto the metadata-type // registry is the reassessment tracked in #3490. The gate only needs to WALK // the schema, not register it — so we resolve it directly. +// +// `query` is not a metadata type at all — `QueryAST` is the REQUEST surface: +// the client SDK QueryBuilder's output and the `POST /data/:object/query` +// body, authorable by every API caller yet never stored as stack metadata. +// #4286 found 12 declared members no executor ran, and the reason no gate had +// noticed: this ledger read only the metadata-type registry, so it governed +// what authors write into metadata files while nothing governed what callers +// write into a query. Walking `QuerySchema` here closes that class; +// `query.json` carries the request-surface verdicts. const SPEC_ONLY_SCHEMAS: Record = { webhook: WebhookSchema, + query: QuerySchema, }; // ADR-0010 provenance/lock overlay fields — system-stamped, on every type; auto-live. diff --git a/packages/spec/src/data/query.test.ts b/packages/spec/src/data/query.test.ts index 8ccbf22c22..99d73e4567 100644 --- a/packages/spec/src/data/query.test.ts +++ b/packages/spec/src/data/query.test.ts @@ -3,12 +3,7 @@ import { QuerySchema, FieldNodeSchema, AggregationFunction, - JoinType, - WindowFunction, type QueryAST, - type AggregationNode, - type JoinNode, - type WindowFunctionNode, } from './query.zod'; describe('AggregationFunction', () => { @@ -29,34 +24,6 @@ describe('AggregationFunction', () => { }); }); -describe('JoinType', () => { - it('should accept valid join types', () => { - expect(() => JoinType.parse('inner')).not.toThrow(); - expect(() => JoinType.parse('left')).not.toThrow(); - expect(() => JoinType.parse('right')).not.toThrow(); - expect(() => JoinType.parse('full')).not.toThrow(); - }); - - it('should reject invalid join types', () => { - expect(() => JoinType.parse('INNER')).toThrow(); - expect(() => JoinType.parse('cross')).toThrow(); - }); -}); - -describe('WindowFunction', () => { - it('should accept valid window functions', () => { - const validFunctions = [ - 'row_number', 'rank', 'dense_rank', 'percent_rank', - 'lag', 'lead', 'first_value', 'last_value', - 'sum', 'avg', 'count', 'min', 'max' - ]; - - validFunctions.forEach(fn => { - expect(() => WindowFunction.parse(fn)).not.toThrow(); - }); - }); -}); - describe('QuerySchema - Basic', () => { it('should accept simple query', () => { const query: QueryAST = { @@ -492,1346 +459,219 @@ describe('QuerySchema - Aggregations', () => { }); }); -describe('QuerySchema - Joins', () => { - // ============================================================================ - // INNER JOIN Tests - // ============================================================================ - - it('should accept query with INNER JOIN', () => { - const query: QueryAST = { - object: 'order', - fields: ['id', 'amount'], - joins: [ - { - type: 'inner', - object: 'customer', - alias: 'c', - on: { 'order.customer_id': { $eq: { $field: 'c.id' } } }, - }, - ], - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); - }); - - it('should accept INNER JOIN without alias', () => { - const query: QueryAST = { - object: 'order', - fields: ['id'], - joins: [ - { - type: 'inner', - object: 'customer', - on: { 'order.customer_id': { $eq: { $field: 'customer.id' } } }, - }, - ], - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); - }); - - it('should accept INNER JOIN with complex ON condition', () => { - const query: QueryAST = { +/** + * [#4286] `query.joins` and `query.windowFunctions` were declared-but-inert: + * no engine or driver read either on the query path, so every join and OVER + * clause a caller declared was silently dropped (ADR-0078; ADR-0049 + * enforce-or-remove). Both are tombstoned rather than deleted — + * `BaseQuerySchema` is not `.strict()`, so a plain deletion would have + * silently STRIPPED the keys (the ADR-0104 class); `retiredKey()` keeps the + * removal audible in both channels (`tsc` and the parse). + */ +describe('QueryAST.joins — REMOVED (#4286)', () => { + it('rejects a query carrying `joins` with the prescription, even as an empty array', () => { + expect(() => QuerySchema.parse({ object: 'order', joins: [] })) + .toThrow(/query\.joins.*removed.*expand/s); + expect(() => QuerySchema.parse({ object: 'order', - fields: ['id'], - joins: [ - { - type: 'inner', - object: 'customer', - alias: 'c', - on: { - $and: [ - { 'order.customer_id': { $eq: { $field: 'c.id' } } }, - { 'order.status': 'active' }, - ], - }, - }, - ], - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); - }); - - // ============================================================================ - // LEFT JOIN Tests - // ============================================================================ - - it('should accept query with LEFT JOIN', () => { - const query: QueryAST = { - object: 'customer', - fields: ['name'], - joins: [ - { - type: 'left', - object: 'order', - on: { 'customer.id': { $eq: { $field: 'order.customer_id' } } }, - }, - ], - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); - }); - - it('should accept LEFT JOIN with alias', () => { - const query: QueryAST = { - object: 'customer', - fields: ['id', 'name'], - joins: [ - { - type: 'left', - object: 'order', - alias: 'o', - on: { 'customer.id': { $eq: { $field: 'o.customer_id' } } }, - }, - ], - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); + joins: [{ + type: 'inner', + object: 'customer', + on: { 'order.customer_id': { $eq: { $field: 'customer.id' } } }, + }], + })).toThrow(/query\.joins.*removed/s); }); - it('should accept LEFT JOIN to find unmatched records', () => { - const query: QueryAST = { - object: 'customer', - fields: ['id', 'name'], - joins: [ - { - type: 'left', - object: 'order', - alias: 'o', - on: { 'customer.id': { $eq: { $field: 'o.customer_id' } } }, - }, - ], - filters: ['o.id', 'is_null', null], - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); + it('still accepts absence — `undefined` is not authoring', () => { + expect(() => QuerySchema.parse({ object: 'order', joins: undefined })).not.toThrow(); }); - // ============================================================================ - // RIGHT JOIN Tests - // ============================================================================ - - it('should accept query with RIGHT JOIN', () => { - const query: QueryAST = { + it('the replacement parses: related records through `expand`, a related column as a dotted path', () => { + const parsed = QuerySchema.parse({ object: 'order', - fields: ['id'], - joins: [ - { - type: 'right', - object: 'customer', - alias: 'c', - on: { 'order.customer_id': { $eq: { $field: 'c.id' } } }, - }, - ], - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); + fields: ['id', 'customer_id.name'], + expand: { customer_id: { object: 'customer', fields: ['name'] } }, + }); + expect(parsed).not.toHaveProperty('joins'); }); +}); - it('should accept RIGHT JOIN without alias', () => { - const query: QueryAST = { +describe('QueryAST.windowFunctions — REMOVED (#4286)', () => { + it('rejects a query carrying `windowFunctions` with the prescription naming the door', () => { + expect(() => QuerySchema.parse({ object: 'order', - fields: ['id', 'amount'], - joins: [ - { - type: 'right', - object: 'customer', - on: { 'order.customer_id': { $eq: { $field: 'customer.id' } } }, - }, - ], - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); - }); - - // ============================================================================ - // FULL OUTER JOIN Tests - // ============================================================================ - - it('should accept query with FULL OUTER JOIN', () => { - const query: QueryAST = { - object: 'customer', - fields: ['id', 'name'], - joins: [ - { - type: 'full', - object: 'order', - alias: 'o', - on: { 'customer.id': { $eq: { $field: 'o.customer_id' } } }, - }, - ], - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); + windowFunctions: [{ function: 'row_number', alias: 'rn', over: {} }], + })).toThrow(/query\.windowFunctions.*removed.*findWithWindowFunctions/s); + expect(() => QuerySchema.parse({ object: 'order', windowFunctions: [] })) + .toThrow(/query\.windowFunctions.*removed/s); }); - it('should accept FULL JOIN to find all unmatched records', () => { - const query: QueryAST = { - object: 'customer', - fields: ['id'], - joins: [ - { - type: 'full', - object: 'order', - alias: 'o', - on: { 'customer.id': { $eq: { $field: 'o.customer_id' } } }, - }, - ], - filters: [['customer.id', 'is_null', null], 'or', ['o.id', 'is_null', null]], - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); + it('still accepts absence — `undefined` is not authoring', () => { + expect(() => QuerySchema.parse({ object: 'order', windowFunctions: undefined })).not.toThrow(); }); +}); - // ============================================================================ - // Multiple Joins Tests - // ============================================================================ - - it('should accept query with multiple INNER JOINs', () => { +describe('QuerySchema - Complex Queries', () => { + it('should accept complex query with aggregations, HAVING, ordering and paging', () => { const query: QueryAST = { object: 'order', - fields: ['id'], - joins: [ - { - type: 'inner', - object: 'customer', - alias: 'c', - on: { 'order.customer_id': { $eq: { $field: 'c.id' } } }, - }, - { - type: 'inner', - object: 'product', - alias: 'p', - on: { 'order.product_id': { $eq: { $field: 'p.id' } } }, - }, + fields: ['customer_id'], + aggregations: [ + { function: 'sum', field: 'amount', alias: 'total_amount' }, + { function: 'count', alias: 'order_count' }, ], + groupBy: ['customer_id'], + having: { order_count: { $gt: 5 } }, + orderBy: [{ field: 'total_amount', order: 'desc' }], + top: 100, }; expect(() => QuerySchema.parse(query)).not.toThrow(); }); - it('should accept query with mixed join types', () => { + // [#4286] This case used to stack `joins` + `windowFunctions` on top — both + // parsed and then did nothing. The composition that actually executes is + // expand + aggregation + HAVING declaration + ordering + paging. + it('should accept query combining expand with the other declared features', () => { const query: QueryAST = { object: 'order', - fields: ['id'], - joins: [ - { - type: 'inner', - object: 'customer', - alias: 'c', - on: { 'order.customer_id': { $eq: { $field: 'c.id' } } }, - }, - { - type: 'left', - object: 'product', - alias: 'p', - on: { 'order.product_id': { $eq: { $field: 'p.id' } } }, - }, - { - type: 'left', - object: 'shipment', - alias: 's', - on: { 'order.id': { $eq: { $field: 's.order_id' } } }, - }, + fields: ['id', 'customer_id', 'amount'], + distinct: true, + where: { status: 'completed' }, + expand: { customer_id: { object: 'customer', fields: ['name'] } }, + aggregations: [ + { function: 'avg', field: 'amount', alias: 'avg_amount' }, ], + groupBy: ['customer_id'], + having: { avg_amount: { $gt: 500 } }, + orderBy: [{ field: 'avg_amount', order: 'desc' }], + top: 50, + offset: 0, }; expect(() => QuerySchema.parse(query)).not.toThrow(); }); +}); - it('should accept query with 4+ table joins', () => { +describe('QuerySchema - Edge Cases and Null Handling', () => { + it('should handle null values in filter expressions', () => { const query: QueryAST = { - object: 'order', - fields: ['id', 'total'], - joins: [ - { - type: 'inner', - object: 'customer', - alias: 'c', - on: { 'order.customer_id': { $eq: { $field: 'c.id' } } }, - }, - { - type: 'inner', - object: 'order_item', - alias: 'oi', - on: { 'order.id': { $eq: { $field: 'oi.order_id' } } }, - }, - { - type: 'inner', - object: 'product', - alias: 'p', - on: { 'oi.product_id': { $eq: { $field: 'p.id' } } }, - }, - { - type: 'left', - object: 'category', - alias: 'cat', - on: { 'p.category_id': { $eq: { $field: 'cat.id' } } }, - }, - ], + object: 'account', + fields: ['name'], + filters: ['deleted_at', 'is_null', null], }; expect(() => QuerySchema.parse(query)).not.toThrow(); }); - // ============================================================================ - // Self-Join Tests - // ============================================================================ - - it('should accept self-join query', () => { + it('should handle undefined for optional fields', () => { const query: QueryAST = { - object: 'employee', - fields: ['id', 'name'], - joins: [ - { - type: 'left', - object: 'employee', - alias: 'manager', - on: { 'employee.manager_id': { $eq: { $field: 'manager.id' } } }, - }, - ], + object: 'account', + fields: undefined, + filters: undefined, + sort: undefined, + aggregations: undefined, + joins: undefined, + groupBy: undefined, + having: undefined, + windowFunctions: undefined, }; expect(() => QuerySchema.parse(query)).not.toThrow(); }); - it('should accept hierarchical self-join', () => { + it('should handle empty arrays', () => { + // `joins: []` / `windowFunctions: []` moved to the retirement pins — the + // tombstones refuse even an empty array (#4286). const query: QueryAST = { - object: 'category', - fields: ['id', 'name'], - joins: [ - { - type: 'left', - object: 'category', - alias: 'parent', - on: { 'category.parent_id': { $eq: { $field: 'parent.id' } } }, - }, - ], + object: 'account', + fields: [], + aggregations: [], + groupBy: [], + orderBy: [], }; expect(() => QuerySchema.parse(query)).not.toThrow(); }); - // ============================================================================ - // Join with Filters Tests - // ============================================================================ - - it('should accept join with WHERE clause on main table', () => { - const query: QueryAST = { - object: 'order', - fields: ['id'], - filters: ['order.status', '=', 'completed'], - joins: [ - { - type: 'inner', - object: 'customer', - alias: 'c', - on: { 'order.customer_id': { $eq: { $field: 'c.id' } } }, - }, - ], - }; + it('should handle zero and negative values in pagination', () => { + const queries = [ + { object: 'account', top: 0, skip: 0 }, + { object: 'account', top: 1, skip: 0 }, + { object: 'account', top: 100, skip: 1000 }, + ]; - expect(() => QuerySchema.parse(query)).not.toThrow(); + queries.forEach(query => { + expect(() => QuerySchema.parse(query)).not.toThrow(); + }); }); - it('should accept join with ON clause containing multiple conditions', () => { + it('should handle complex nested null filters', () => { const query: QueryAST = { object: 'order', fields: ['id'], - joins: [ - { - type: 'inner', - object: 'customer', - alias: 'c', - on: { - $and: [ - { 'order.customer_id': { $eq: { $field: 'c.id' } } }, - { 'c.status': 'active' }, - ], - }, - }, + filters: [ + ['approved_at', 'is_null', null], + 'and', + ['rejected_at', 'is_null', null], ], }; expect(() => QuerySchema.parse(query)).not.toThrow(); }); - // ============================================================================ - // Join with Aggregations Tests - // ============================================================================ + // [#4196] This case used to assert the opposite — that `{ field, fields, + // alias }` entries parse. They did, and then nothing honoured them: the pin + // proved the DECLARATION, which is exactly how a declared-but-inert shape + // survives a green suite. The `alias` half was the emptiest of all; no + // projection anywhere was ever named by it. + it('refuses relationship field nodes, with or without an alias', () => { + expect(() => QuerySchema.parse({ + object: 'account', + fields: ['name', { field: 'owner', fields: ['name', 'email'] }], + })).toThrow(/nested-select object form.*removed/s); + expect(() => QuerySchema.parse({ + object: 'account', + fields: ['name', { field: 'manager', fields: ['name'], alias: 'mgr' }], + })).toThrow(/`alias` has no replacement here/); + }); - it('should accept join with GROUP BY and aggregations', () => { + it('should handle aggregation without field for COUNT', () => { const query: QueryAST = { object: 'order', - fields: ['customer_id'], - joins: [ - { - type: 'inner', - object: 'customer', - alias: 'c', - on: { 'order.customer_id': { $eq: { $field: 'c.id' } } }, - }, - ], aggregations: [ - { function: 'count', alias: 'order_count' }, - { function: 'sum', field: 'amount', alias: 'total_amount' }, + { function: 'count', alias: 'total_count' }, ], - groupBy: ['customer_id'], }; expect(() => QuerySchema.parse(query)).not.toThrow(); }); - // ============================================================================ - // Subquery Join Tests - // ============================================================================ - - it('should accept query with subquery join', () => { + it('should handle optional distinct flag in aggregation', () => { const query: QueryAST = { object: 'order', - fields: ['id', 'amount'], - joins: [ - { - type: 'inner', - object: 'customer', - alias: 'high_value_customers', - on: { 'order.customer_id': { $eq: { $field: 'high_value_customers.id' } } }, - subquery: { - object: 'customer', - fields: ['id'], - filters: ['total_spent', '>', 10000], - }, - }, + aggregations: [ + { function: 'count', field: 'customer_id', alias: 'unique_customers', distinct: true }, + { function: 'sum', field: 'amount', alias: 'total_amount' }, // distinct undefined ], + groupBy: ['region'], }; expect(() => QuerySchema.parse(query)).not.toThrow(); }); - it('should accept LEFT JOIN with aggregated subquery', () => { - const query: QueryAST = { - object: 'customer', - fields: ['id', 'name'], - joins: [ - { - type: 'left', - object: 'order', - alias: 'order_summary', - on: { 'customer.id': { $eq: { $field: 'order_summary.customer_id' } } }, - subquery: { - object: 'order', - fields: ['customer_id'], - aggregations: [ - { function: 'count', alias: 'order_count' }, - { function: 'sum', field: 'amount', alias: 'total_spent' }, - ], - groupBy: ['customer_id'], - }, - }, - ], - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); + it('should reject invalid object type', () => { + expect(() => QuerySchema.parse({ + object: 123, // Should be string + fields: ['name'], + })).toThrow(); }); - // ============================================================================ - // Real-World Join Examples (SOQL Comparisons) - // ============================================================================ - - it('should accept Salesforce-style relationship query (SOQL: SELECT Name, (SELECT Name FROM Contacts) FROM Account)', () => { - const query: QueryAST = { + it('should reject invalid field types in array', () => { + expect(() => QuerySchema.parse({ object: 'account', - fields: ['id', 'name'], - joins: [ - { - type: 'left', - object: 'contact', - on: { 'account.id': { $eq: { $field: 'contact.account_id' } } }, - }, - ], - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); - }); - - it('should accept complex multi-table join for reporting', () => { - const query: QueryAST = { - object: 'order', - fields: ['id', 'order_date'], - joins: [ - { - type: 'inner', - object: 'customer', - alias: 'c', - on: { 'order.customer_id': { $eq: { $field: 'c.id' } } }, - }, - { - type: 'inner', - object: 'order_item', - alias: 'oi', - on: { 'order.id': { $eq: { $field: 'oi.order_id' } } }, - }, - { - type: 'inner', - object: 'product', - alias: 'p', - on: { 'oi.product_id': { $eq: { $field: 'p.id' } } }, - }, - ], - aggregations: [ - { function: 'sum', field: 'oi.quantity', alias: 'total_quantity' }, - { function: 'sum', field: 'oi.line_total', alias: 'order_total' }, - ], - groupBy: ['order.id', 'order.order_date'], - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); - }); - - it('should accept customer order history join', () => { - const query: QueryAST = { - object: 'customer', - fields: ['id', 'name', 'email'], - joins: [ - { - type: 'left', - object: 'order', - alias: 'o', - on: { 'customer.id': { $eq: { $field: 'o.customer_id' } } }, - }, - ], - aggregations: [ - { function: 'count', field: 'o.id', alias: 'total_orders' }, - { function: 'sum', field: 'o.amount', alias: 'lifetime_value' }, - { function: 'max', field: 'o.created_at', alias: 'last_order_date' }, - ], - groupBy: ['customer.id', 'customer.name', 'customer.email'], - orderBy: [{ field: 'lifetime_value', order: 'desc' }], - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); - }); -}); - -describe('QuerySchema - Window Functions', () => { - // ============================================================================ - // ROW_NUMBER Tests - // ============================================================================ - - it('should accept query with ROW_NUMBER window function', () => { - const query: QueryAST = { - object: 'order', - fields: ['id', 'customer_id', 'amount'], - windowFunctions: [ - { - function: 'row_number', - alias: 'row_num', - over: { - partitionBy: ['customer_id'], - orderBy: [{ field: 'amount', order: 'desc' }], - }, - }, - ], - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); - }); - - it('should accept ROW_NUMBER without partition', () => { - const query: QueryAST = { - object: 'student', - fields: ['name', 'score'], - windowFunctions: [ - { - function: 'row_number', - alias: 'rank', - over: { - orderBy: [{ field: 'score', order: 'desc' }], - }, - }, - ], - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); - }); - - it('should accept ROW_NUMBER with multiple partition fields', () => { - const query: QueryAST = { - object: 'sales', - fields: ['region', 'product', 'revenue'], - windowFunctions: [ - { - function: 'row_number', - alias: 'row_num', - over: { - partitionBy: ['region', 'product'], - orderBy: [{ field: 'revenue', order: 'desc' }], - }, - }, - ], - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); - }); - - // ============================================================================ - // RANK and DENSE_RANK Tests - // ============================================================================ - - it('should accept query with RANK window function', () => { - const query: QueryAST = { - object: 'student', - fields: ['name', 'score'], - windowFunctions: [ - { - function: 'rank', - alias: 'rank', - over: { - orderBy: [{ field: 'score', order: 'desc' }], - }, - }, - ], - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); - }); - - it('should accept query with DENSE_RANK window function', () => { - const query: QueryAST = { - object: 'employee', - fields: ['name', 'salary'], - windowFunctions: [ - { - function: 'dense_rank', - alias: 'salary_rank', - over: { - partitionBy: ['department'], - orderBy: [{ field: 'salary', order: 'desc' }], - }, - }, - ], - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); - }); - - it('should accept query with PERCENT_RANK window function', () => { - const query: QueryAST = { - object: 'student', - fields: ['name', 'score'], - windowFunctions: [ - { - function: 'percent_rank', - alias: 'percentile', - over: { - orderBy: [{ field: 'score', order: 'desc' }], - }, - }, - ], - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); - }); - - // ============================================================================ - // LAG and LEAD Tests - // ============================================================================ - - it('should accept query with LAG window function', () => { - const query: QueryAST = { - object: 'sales', - fields: ['month', 'revenue'], - windowFunctions: [ - { - function: 'lag', - field: 'revenue', - alias: 'prev_month_revenue', - over: { - orderBy: [{ field: 'month', order: 'asc' }], - }, - }, - ], - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); - }); - - it('should accept query with LEAD window function', () => { - const query: QueryAST = { - object: 'sales', - fields: ['month', 'revenue'], - windowFunctions: [ - { - function: 'lead', - field: 'revenue', - alias: 'next_month_revenue', - over: { - orderBy: [{ field: 'month', order: 'asc' }], - }, - }, - ], - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); - }); - - it('should accept LAG and LEAD together', () => { - const query: QueryAST = { - object: 'stock_price', - fields: ['date', 'price'], - windowFunctions: [ - { - function: 'lag', - field: 'price', - alias: 'prev_day_price', - over: { - orderBy: [{ field: 'date', order: 'asc' }], - }, - }, - { - function: 'lead', - field: 'price', - alias: 'next_day_price', - over: { - orderBy: [{ field: 'date', order: 'asc' }], - }, - }, - ], - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); - }); - - // ============================================================================ - // FIRST_VALUE and LAST_VALUE Tests - // ============================================================================ - - it('should accept query with FIRST_VALUE window function', () => { - const query: QueryAST = { - object: 'order', - fields: ['customer_id', 'order_date', 'amount'], - windowFunctions: [ - { - function: 'first_value', - field: 'amount', - alias: 'first_order_amount', - over: { - partitionBy: ['customer_id'], - orderBy: [{ field: 'order_date', order: 'asc' }], - }, - }, - ], - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); - }); - - it('should accept query with LAST_VALUE window function', () => { - const query: QueryAST = { - object: 'order', - fields: ['customer_id', 'order_date', 'amount'], - windowFunctions: [ - { - function: 'last_value', - field: 'amount', - alias: 'last_order_amount', - over: { - partitionBy: ['customer_id'], - orderBy: [{ field: 'order_date', order: 'asc' }], - }, - }, - ], - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); - }); - - // ============================================================================ - // Aggregate Window Function Tests - // ============================================================================ - - it('should accept query with SUM aggregate window function', () => { - const query: QueryAST = { - object: 'order', - fields: ['id', 'amount'], - windowFunctions: [ - { - function: 'sum', - field: 'amount', - alias: 'running_total', - over: { - orderBy: [{ field: 'created_at', order: 'asc' }], - }, - }, - ], - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); - }); - - it('should accept query with AVG aggregate window function', () => { - const query: QueryAST = { - object: 'sales', - fields: ['month', 'revenue'], - windowFunctions: [ - { - function: 'avg', - field: 'revenue', - alias: 'moving_avg', - over: { - orderBy: [{ field: 'month', order: 'asc' }], - frame: { - type: 'rows', - start: '2 PRECEDING', - end: 'CURRENT ROW', - }, - }, - }, - ], - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); - }); - - it('should accept query with COUNT aggregate window function', () => { - const query: QueryAST = { - object: 'event', - fields: ['timestamp', 'user_id'], - windowFunctions: [ - { - function: 'count', - alias: 'running_count', - over: { - partitionBy: ['user_id'], - orderBy: [{ field: 'timestamp', order: 'asc' }], - }, - }, - ], - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); - }); - - it('should accept query with MIN/MAX aggregate window functions', () => { - const query: QueryAST = { - object: 'temperature', - fields: ['date', 'value'], - windowFunctions: [ - { - function: 'min', - field: 'value', - alias: 'min_so_far', - over: { - orderBy: [{ field: 'date', order: 'asc' }], - }, - }, - { - function: 'max', - field: 'value', - alias: 'max_so_far', - over: { - orderBy: [{ field: 'date', order: 'asc' }], - }, - }, - ], - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); - }); - - // ============================================================================ - // Window Frame Specification Tests - // ============================================================================ - - it('should accept query with ROWS frame specification', () => { - const query: QueryAST = { - object: 'order', - fields: ['id', 'amount'], - windowFunctions: [ - { - function: 'sum', - field: 'amount', - alias: 'running_total', - over: { - orderBy: [{ field: 'created_at', order: 'asc' }], - frame: { - type: 'rows', - start: 'UNBOUNDED PRECEDING', - end: 'CURRENT ROW', - }, - }, - }, - ], - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); - }); - - it('should accept query with RANGE frame specification', () => { - const query: QueryAST = { - object: 'sales', - fields: ['date', 'amount'], - windowFunctions: [ - { - function: 'sum', - field: 'amount', - alias: 'total_in_range', - over: { - orderBy: [{ field: 'date', order: 'asc' }], - frame: { - type: 'range', - start: '7 PRECEDING', - end: 'CURRENT ROW', - }, - }, - }, - ], - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); - }); - - it('should accept query with window frame FOLLOWING', () => { - const query: QueryAST = { - object: 'sales', - fields: ['month', 'revenue'], - windowFunctions: [ - { - function: 'avg', - field: 'revenue', - alias: 'centered_avg', - over: { - orderBy: [{ field: 'month', order: 'asc' }], - frame: { - type: 'rows', - start: '1 PRECEDING', - end: '1 FOLLOWING', - }, - }, - }, - ], - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); - }); - - // ============================================================================ - // Multiple Window Functions Tests - // ============================================================================ - - it('should accept query with multiple window functions', () => { - const query: QueryAST = { - object: 'order', - fields: ['customer_id', 'amount', 'created_at'], - windowFunctions: [ - { - function: 'row_number', - alias: 'row_num', - over: { - partitionBy: ['customer_id'], - orderBy: [{ field: 'created_at', order: 'desc' }], - }, - }, - { - function: 'rank', - alias: 'amount_rank', - over: { - partitionBy: ['customer_id'], - orderBy: [{ field: 'amount', order: 'desc' }], - }, - }, - { - function: 'sum', - field: 'amount', - alias: 'running_total', - over: { - partitionBy: ['customer_id'], - orderBy: [{ field: 'created_at', order: 'asc' }], - }, - }, - ], - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); - }); - - // ============================================================================ - // Real-World Window Function Examples - // ============================================================================ - - it('should accept query for top N per group (SQL: ROW_NUMBER() OVER (PARTITION BY ...))', () => { - const query: QueryAST = { - object: 'product', - fields: ['category_id', 'name', 'price'], - windowFunctions: [ - { - function: 'row_number', - alias: 'rank_in_category', - over: { - partitionBy: ['category_id'], - orderBy: [{ field: 'price', order: 'desc' }], - }, - }, - ], - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); - }); - - it('should accept running total query', () => { - const query: QueryAST = { - object: 'transaction', - fields: ['date', 'amount'], - windowFunctions: [ - { - function: 'sum', - field: 'amount', - alias: 'running_balance', - over: { - orderBy: [{ field: 'date', order: 'asc' }], - frame: { - type: 'rows', - start: 'UNBOUNDED PRECEDING', - end: 'CURRENT ROW', - }, - }, - }, - ], - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); - }); - - it('should accept moving average query', () => { - const query: QueryAST = { - object: 'stock_price', - fields: ['date', 'close_price'], - windowFunctions: [ - { - function: 'avg', - field: 'close_price', - alias: 'ma_7_day', - over: { - orderBy: [{ field: 'date', order: 'asc' }], - frame: { - type: 'rows', - start: '6 PRECEDING', - end: 'CURRENT ROW', - }, - }, - }, - ], - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); - }); - - it('should accept year-over-year comparison query', () => { - const query: QueryAST = { - object: 'monthly_sales', - fields: ['month', 'revenue'], - windowFunctions: [ - { - function: 'lag', - field: 'revenue', - alias: 'prev_year_revenue', - over: { - orderBy: [{ field: 'month', order: 'asc' }], - }, - }, - ], - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); - }); - - it('should accept employee ranking within department', () => { - const query: QueryAST = { - object: 'employee', - fields: ['department', 'name', 'salary'], - windowFunctions: [ - { - function: 'rank', - alias: 'salary_rank', - over: { - partitionBy: ['department'], - orderBy: [{ field: 'salary', order: 'desc' }], - }, - }, - { - function: 'percent_rank', - alias: 'salary_percentile', - over: { - partitionBy: ['department'], - orderBy: [{ field: 'salary', order: 'desc' }], - }, - }, - ], - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); - }); -}); - -describe('QuerySchema - Complex Queries', () => { - it('should accept complex query with joins, aggregations, and window functions', () => { - const query: QueryAST = { - object: 'order', - fields: ['customer_id'], - joins: [ - { - type: 'inner', - object: 'customer', - alias: 'c', - on: { 'order.customer_id': { $eq: { $field: 'c.id' } } }, - }, - ], - aggregations: [ - { function: 'sum', field: 'amount', alias: 'total_amount' }, - { function: 'count', alias: 'order_count' }, - ], - groupBy: ['customer_id'], - having: { order_count: { $gt: 5 } }, - orderBy: [{ field: 'total_amount', order: 'desc' }], - top: 100, - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); - }); - - it('should accept query with all features', () => { - const query: QueryAST = { - object: 'order', - fields: ['id', 'customer_id', 'amount'], - distinct: true, - filters: ['status', '=', 'completed'], - joins: [ - { - type: 'inner', - object: 'customer', - on: { 'order.customer_id': { $eq: { $field: 'customer.id' } } }, - }, - ], - aggregations: [ - { function: 'avg', field: 'amount', alias: 'avg_amount' }, - ], - windowFunctions: [ - { - function: 'rank', - alias: 'customer_rank', - over: { - partitionBy: ['customer_id'], - orderBy: [{ field: 'amount', order: 'desc' }], - }, - }, - ], - groupBy: ['customer_id'], - having: { avg_amount: { $gt: 500 } }, - orderBy: [{ field: 'avg_amount', order: 'desc' }], - top: 50, - skip: 0, - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); - }); -}); - -describe('QuerySchema - Edge Cases and Null Handling', () => { - it('should handle null values in filter expressions', () => { - const query: QueryAST = { - object: 'account', - fields: ['name'], - filters: ['deleted_at', 'is_null', null], - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); - }); - - it('should handle undefined for optional fields', () => { - const query: QueryAST = { - object: 'account', - fields: undefined, - filters: undefined, - sort: undefined, - aggregations: undefined, - joins: undefined, - groupBy: undefined, - having: undefined, - windowFunctions: undefined, - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); - }); - - it('should handle empty arrays', () => { - const query: QueryAST = { - object: 'account', - fields: [], - aggregations: [], - joins: [], - windowFunctions: [], - groupBy: [], - orderBy: [], - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); - }); - - it('should handle zero and negative values in pagination', () => { - const queries = [ - { object: 'account', top: 0, skip: 0 }, - { object: 'account', top: 1, skip: 0 }, - { object: 'account', top: 100, skip: 1000 }, - ]; - - queries.forEach(query => { - expect(() => QuerySchema.parse(query)).not.toThrow(); - }); - }); - - it('should handle complex nested null filters', () => { - const query: QueryAST = { - object: 'order', - fields: ['id'], - filters: [ - ['approved_at', 'is_null', null], - 'and', - ['rejected_at', 'is_null', null], - ], - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); - }); - - // [#4196] This case used to assert the opposite — that `{ field, fields, - // alias }` entries parse. They did, and then nothing honoured them: the pin - // proved the DECLARATION, which is exactly how a declared-but-inert shape - // survives a green suite. The `alias` half was the emptiest of all; no - // projection anywhere was ever named by it. - it('refuses relationship field nodes, with or without an alias', () => { - expect(() => QuerySchema.parse({ - object: 'account', - fields: ['name', { field: 'owner', fields: ['name', 'email'] }], - })).toThrow(/nested-select object form.*removed/s); - expect(() => QuerySchema.parse({ - object: 'account', - fields: ['name', { field: 'manager', fields: ['name'], alias: 'mgr' }], - })).toThrow(/`alias` has no replacement here/); - }); - - it('should handle aggregation without field for COUNT', () => { - const query: QueryAST = { - object: 'order', - aggregations: [ - { function: 'count', alias: 'total_count' }, - ], - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); - }); - - it('should handle optional distinct flag in aggregation', () => { - const query: QueryAST = { - object: 'order', - aggregations: [ - { function: 'count', field: 'customer_id', alias: 'unique_customers', distinct: true }, - { function: 'sum', field: 'amount', alias: 'total_amount' }, // distinct undefined - ], - groupBy: ['region'], - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); - }); - - it('should handle optional properties in window functions', () => { - const query: QueryAST = { - object: 'sales', - fields: ['amount'], - windowFunctions: [ - { - function: 'row_number', - alias: 'row_num', - over: { - // partitionBy and orderBy are optional - }, - }, - { - function: 'sum', - field: 'amount', - alias: 'total', - over: { - partitionBy: ['region'], - // orderBy is optional - }, - }, - ], - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); - }); - - it('should handle optional frame in window specification', () => { - const query: QueryAST = { - object: 'transactions', - fields: ['amount'], - windowFunctions: [ - { - function: 'sum', - field: 'amount', - alias: 'running_total', - over: { - orderBy: [{ field: 'date', order: 'asc' }], - frame: { - type: 'rows', - start: 'UNBOUNDED PRECEDING', - end: 'CURRENT ROW', - }, - }, - }, - ], - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); - }); - - it('should handle optional subquery in joins', () => { - const query: QueryAST = { - object: 'customer', - joins: [ - { - type: 'left', - object: 'order', - on: { 'customer.id': { $eq: { $field: 'order.customer_id' } } }, - }, - { - type: 'inner', - object: 'filtered_orders', - alias: 'fo', - on: { 'customer.id': { $eq: { $field: 'fo.customer_id' } } }, - subquery: { - object: 'order', - fields: ['customer_id', 'amount'], - filters: ['amount', '>', 1000], - }, - }, - ], - }; - - expect(() => QuerySchema.parse(query)).not.toThrow(); - }); - - it('should reject invalid object type', () => { - expect(() => QuerySchema.parse({ - object: 123, // Should be string - fields: ['name'], - })).toThrow(); - }); - - it('should reject invalid field types in array', () => { - expect(() => QuerySchema.parse({ - object: 'account', - fields: [123, 456], // Should be strings or objects - })).toThrow(); + fields: [123, 456], // Should be strings or objects + })).toThrow(); }); it('should reject invalid aggregation function', () => { @@ -1843,32 +683,6 @@ describe('QuerySchema - Edge Cases and Null Handling', () => { })).toThrow(); }); - it('should reject invalid join type', () => { - expect(() => QuerySchema.parse({ - object: 'order', - joins: [ - { - type: 'invalid_join', - object: 'customer', - on: { 'order.customer_id': { $eq: { $field: 'customer.id' } } }, - }, - ], - })).toThrow(); - }); - - it('should reject invalid window function', () => { - expect(() => QuerySchema.parse({ - object: 'sales', - windowFunctions: [ - { - function: 'invalid_window_func', - alias: 'test', - over: {}, - }, - ], - })).toThrow(); - }); - it('should reject invalid sort order', () => { expect(() => QuerySchema.parse({ object: 'account', diff --git a/packages/spec/src/data/query.zod.ts b/packages/spec/src/data/query.zod.ts index ded5b51a4e..0e000d8b9d 100644 --- a/packages/spec/src/data/query.zod.ts +++ b/packages/spec/src/data/query.zod.ts @@ -8,6 +8,7 @@ import { FilterConditionSchema } from './filter.zod'; * Represents "Order By". */ import { lazySchema } from '../shared/lazy-schema'; +import { retiredKey } from '../shared/retired-key'; export const SortNodeSchema = lazySchema(() => z.object({ field: z.string(), order: z.enum(['asc', 'desc']).default('asc') @@ -128,325 +129,28 @@ export const AggregationNodeSchema = lazySchema(() => z.object({ field: z.string().optional().describe('Field to aggregate (optional for COUNT(*))'), alias: z.string().describe('Result column alias'), distinct: z.boolean().optional().describe('Apply DISTINCT before aggregation'), - filter: FilterConditionSchema.optional().describe('Filter/Condition to apply to the aggregation (FILTER WHERE clause)'), + filter: FilterConditionSchema.optional().describe('[EXPERIMENTAL — not enforced] Per-aggregation filter (SQL FILTER (WHERE …)). Neither the SQL builders nor the in-memory fallback applies it (#4286); filter the whole query with `where` instead.'), })); -/** - * Join Type Enum - * Standard SQL join types for combining tables. - * - * Join Types: - * - **inner**: Returns only matching rows from both tables (SQL: INNER JOIN) - * - **left**: Returns all rows from left table, matching rows from right (SQL: LEFT JOIN) - * - **right**: Returns all rows from right table, matching rows from left (SQL: RIGHT JOIN) - * - **full**: Returns all rows from both tables (SQL: FULL OUTER JOIN) - * - * @example - * // SQL: SELECT * FROM orders INNER JOIN customers ON orders.customer_id = customers.id - * { - * object: 'order', - * joins: [ - * { - * type: 'inner', - * object: 'customer', - * on: ['order.customer_id', '=', 'customer.id'] - * } - * ] - * } - * - * @example - * // Salesforce SOQL-style: Find all customers and their orders (if any) - * { - * object: 'customer', - * joins: [ - * { - * type: 'left', - * object: 'order', - * on: ['customer.id', '=', 'order.customer_id'] - * } - * ] - * } - */ -export const JoinType = z.enum(['inner', 'left', 'right', 'full']); - -/** - * Join Execution Strategy - * Hints to the query engine on how to execute the join. - * - * Strategies: - * - **auto**: Engine decides best strategy (Default). - * - **database**: Push down join to the database (Requires same datasource). - * - **hash**: Load both sets into memory and hash join (Cross-datasource, memory intensive). - * - **loop**: Nested loop lookup (N+1 safe version). (Good for small right-side lookups). - */ -export const JoinStrategy = z.enum(['auto', 'database', 'hash', 'loop']); - -/** Non-recursive half of {@link JoinNodeSchema} — every key except `subquery`. */ -const JoinNodeBaseSchema = lazySchema(() => z.object({ - type: JoinType.describe('Join type'), - strategy: JoinStrategy.optional().describe('Execution strategy hint'), - object: z.string().describe('Object/table to join'), - alias: z.string().optional().describe('Table alias'), - on: FilterConditionSchema.describe('Join condition'), -})); - -/** - * A single join — the TYPE half of {@link JoinNodeSchema}. - * - * `subquery` is what makes the schema recursive (through {@link QuerySchema}), - * so it is declared here rather than inferred: `z.lazy()` needs an annotation, - * and the `z.ZodType` this carried before #4171 made the exported - * `JoinNode` — and `QueryAST['joins']` with it — resolve to `any`. - */ -export type JoinNode = z.infer & { - /** Join against a derived dataset instead of a plain object/table. */ - subquery?: QueryAST; -}; - -/** - * The authoring shape of a join — the INPUT half of {@link JoinNodeSchema} - * (#4195), the same relationship {@link QueryInput} has to {@link QueryAST}. - * - * Kept as its own type rather than reusing `JoinNode` because the recursive - * knot differs: a nested `subquery` is authored, so it is a `QueryInput`, not - * the parsed `QueryAST`. - */ -export type JoinNodeInput = z.input & { - /** Join against a derived dataset instead of a plain object/table. */ - subquery?: QueryInput; -}; - -/** - * Join Node - * Represents table joins for combining data from multiple objects. - * - * Joins connect related data across multiple tables using ON conditions. - * Supports both direct object joins and subquery joins. - * - * @example - * // SQL: SELECT o.*, c.name FROM orders o INNER JOIN customers c ON o.customer_id = c.id - * { - * object: 'order', - * fields: ['id', 'amount'], - * joins: [ - * { - * type: 'inner', - * object: 'customer', - * alias: 'c', - * on: ['order.customer_id', '=', 'c.id'] - * } - * ] - * } - * - * @example - * // SQL: Multi-table join - * // SELECT * FROM orders o - * // INNER JOIN customers c ON o.customer_id = c.id - * // LEFT JOIN shipments s ON o.id = s.order_id - * { - * object: 'order', - * joins: [ - * { - * type: 'inner', - * object: 'customer', - * alias: 'c', - * on: ['order.customer_id', '=', 'c.id'] - * }, - * { - * type: 'left', - * object: 'shipment', - * alias: 's', - * on: ['order.id', '=', 's.order_id'] - * } - * ] - * } - * - * @example - * // Salesforce SOQL: SELECT Name, (SELECT LastName FROM Contacts) FROM Account - * { - * object: 'account', - * fields: ['name'], - * joins: [ - * { - * type: 'left', - * object: 'contact', - * on: ['account.id', '=', 'contact.account_id'] - * } - * ] - * } - * - * @example - * // Subquery Join: Join with a filtered/aggregated dataset - * { - * object: 'customer', - * joins: [ - * { - * type: 'left', - * object: 'order', - * alias: 'high_value_orders', - * on: ['customer.id', '=', 'high_value_orders.customer_id'], - * subquery: { - * object: 'order', - * fields: ['customer_id', 'total'], - * filters: ['total', '>', 1000] - * } - * } - * ] - * } - */ -export const JoinNodeSchema: z.ZodType = z.lazy(() => - JoinNodeBaseSchema.extend({ - subquery: z.lazy(() => QuerySchema).optional().describe('Subquery instead of object'), - }) -); - -/** - * Window Function Enum - * Advanced analytical functions for row-based calculations. - * - * Window Functions: - * - **row_number**: Sequential number within partition (SQL: ROW_NUMBER() OVER (...)) - * - **rank**: Rank with gaps for ties (SQL: RANK() OVER (...)) - * - **dense_rank**: Rank without gaps (SQL: DENSE_RANK() OVER (...)) - * - **percent_rank**: Relative rank as percentage (SQL: PERCENT_RANK() OVER (...)) - * - **lag**: Access previous row value (SQL: LAG(field) OVER (...)) - * - **lead**: Access next row value (SQL: LEAD(field) OVER (...)) - * - **first_value**: First value in window (SQL: FIRST_VALUE(field) OVER (...)) - * - **last_value**: Last value in window (SQL: LAST_VALUE(field) OVER (...)) - * - **sum/avg/count/min/max**: Aggregates over window (SQL: SUM(field) OVER (...)) - * - * @example - * // SQL: SELECT *, ROW_NUMBER() OVER (PARTITION BY customer_id ORDER BY amount DESC) as rank - * // FROM orders - * { - * object: 'order', - * fields: ['id', 'customer_id', 'amount'], - * windowFunctions: [ - * { - * function: 'row_number', - * alias: 'rank', - * over: { - * partitionBy: ['customer_id'], - * orderBy: [{ field: 'amount', order: 'desc' }] - * } - * } - * ] - * } - * - * @example - * // SQL: Running total with SUM() OVER (...) - * { - * object: 'transaction', - * fields: ['date', 'amount'], - * windowFunctions: [ - * { - * function: 'sum', - * field: 'amount', - * alias: 'running_total', - * over: { - * orderBy: [{ field: 'date', order: 'asc' }], - * frame: { - * type: 'rows', - * start: 'UNBOUNDED PRECEDING', - * end: 'CURRENT ROW' - * } - * } - * } - * ] - * } - */ -export const WindowFunction = z.enum([ - 'row_number', 'rank', 'dense_rank', 'percent_rank', - 'lag', 'lead', 'first_value', 'last_value', - 'sum', 'avg', 'count', 'min', 'max' -]); - -/** - * Window Specification - * Defines PARTITION BY and ORDER BY for window functions. - * - * Window specifications control how window functions compute values: - * - **partitionBy**: Divide rows into groups (like GROUP BY but without collapsing rows) - * - **orderBy**: Define order for ranking and offset functions - * - **frame**: Specify which rows to include in aggregate calculations - * - * @example - * // Partition by department, order by salary - * { - * partitionBy: ['department'], - * orderBy: [{ field: 'salary', order: 'desc' }] - * } - * - * @example - * // Moving average with frame specification - * { - * orderBy: [{ field: 'date', order: 'asc' }], - * frame: { - * type: 'rows', - * start: '6 PRECEDING', - * end: 'CURRENT ROW' - * } - * } - */ -export const WindowSpecSchema = lazySchema(() => z.object({ - partitionBy: z.array(z.string()).optional().describe('PARTITION BY fields'), - orderBy: z.array(SortNodeSchema).optional().describe('ORDER BY specification'), - frame: z.object({ - type: z.enum(['rows', 'range']).optional(), - start: z.string().optional().describe('Frame start (e.g., "UNBOUNDED PRECEDING", "1 PRECEDING")'), - end: z.string().optional().describe('Frame end (e.g., "CURRENT ROW", "1 FOLLOWING")'), - }).optional().describe('Window frame specification'), -})); +// ─── Joins: REMOVED (#4286, ADR-0049) ──────────────────────────────────────── +// The whole join cluster — `JoinType`, `JoinStrategy`, the internal +// `JoinNodeBaseSchema`, `JoinNodeSchema` and the `JoinNode`/`JoinNodeInput` +// types — was deleted together with the `query.joins` tombstone below: no +// engine or driver ever read a query's `joins`, and an exported schema with no +// consumer reads as a capability (the #3950 precedent). Related records are +// read through `expand`; a single related column is a dotted `fields` path +// (`fields: ['owner.name']`). -/** - * Window Function Node - * Represents window function with OVER clause. - * - * Window functions perform calculations across a set of rows related to the current row, - * without collapsing the result set (unlike GROUP BY aggregations). - * - * @example - * // SQL: Top 3 products per category - * // SELECT *, ROW_NUMBER() OVER (PARTITION BY category ORDER BY sales DESC) as rank - * // FROM products - * { - * object: 'product', - * fields: ['name', 'category', 'sales'], - * windowFunctions: [ - * { - * function: 'row_number', - * alias: 'category_rank', - * over: { - * partitionBy: ['category'], - * orderBy: [{ field: 'sales', order: 'desc' }] - * } - * } - * ] - * } - * - * @example - * // SQL: Year-over-year comparison with LAG - * { - * object: 'monthly_sales', - * fields: ['month', 'revenue'], - * windowFunctions: [ - * { - * function: 'lag', - * field: 'revenue', - * alias: 'prev_year_revenue', - * over: { - * orderBy: [{ field: 'month', order: 'asc' }] - * } - * } - * ] - * } - */ -export const WindowFunctionNodeSchema = lazySchema(() => z.object({ - function: WindowFunction.describe('Window function name'), - field: z.string().optional().describe('Field to operate on (for aggregate window functions)'), - alias: z.string().describe('Result column alias'), - over: WindowSpecSchema.describe('Window specification (OVER clause)'), -})); +// ─── Window functions: REMOVED (#4286, ADR-0049) ───────────────────────────── +// The window cluster — `WindowFunction`, `WindowSpecSchema`, +// `WindowFunctionNodeSchema` and the `WindowFunctionNode`/`WindowSpec` types — +// was deleted together with the `query.windowFunctions` tombstone below. +// `find()` never applied a window function, and the one live door — +// `SqlDriver.findWithWindowFunctions(object, query)` — consumes its own flat +// driver-level shape (`{ function, alias, partitionBy?, orderBy? }`), NOT this +// vocabulary: the spec node declared `field`, `over` and `frame` members the +// door never read, so keeping the schemas would have documented an input no +// executor accepts (the #3950 orphaned-schema precedent, one layer over). /** * One entry of a select list: a field name. @@ -500,33 +204,56 @@ export const FieldNodeSchema = z.string({ : undefined, }); +/** + * The prescriptions for the two AST members removed in #4286. Like + * {@link FIELD_NODE_OBJECT_FORM_REMOVED}, the rejection is where an author + * meets a retirement, so each message carries the FROM → TO mapping. `QueryAST` + * is a request shape, never stored in stack metadata, so there is no + * `os migrate meta` step — callers rewrite their own queries (the protocol-18 + * semantic migrations `query-joins-retired` / `query-window-functions-retired`). + */ +const QUERY_JOINS_REMOVED = + '`query.joins` was removed in @objectstack/spec 18 (#4286, ADR-0049) — no engine or driver ' + + 'ever read it: a query carrying `joins` behaved exactly as if the key were absent, while ' + + 'its name squatted on the reserved REST parameter set. Delete the key. Related records are ' + + "read through `expand` — `expand: { owner: { object: 'user', fields: ['name'] } }` — which " + + 'the engine resolves via batch $in queries, and a single related column is a dotted ' + + "`fields` path (`fields: ['owner.name']`)."; + +const QUERY_WINDOW_FUNCTIONS_REMOVED = + '`query.windowFunctions` was removed in @objectstack/spec 18 (#4286, ADR-0049) — `find()` ' + + 'never applied it: no engine or driver read the key on the query path, so every OVER ' + + 'clause it declared was silently dropped. Delete the key. Window functions are a ' + + 'SQL-driver capability behind `SqlDriver.findWithWindowFunctions(object, query)` ' + + '(embedder-level; not on the `IDataDriver` contract or the REST surface); request-level ' + + 'analytics are `aggregations` + `groupBy`.'; + /** * Full-Text Search Configuration * Defines full-text search parameters for text queries. - * - * Supports: - * - Multi-field search - * - Relevance scoring - * - Fuzzy matching - * - Language-specific analyzers - * + * + * What actually executes (ADR-0061): the engine expands `search` into a + * server-resolved cross-field `$or` filter, reading exactly two members — + * `query` (the text) and `fields` (which columns to match). The remaining + * six flags are declared search-engine affordances no executor receives; + * they carry `[EXPERIMENTAL — not enforced]` markers so authoring one is a + * declaration, not a silent no-op (#4286, ADR-0078). + * * @example * { * query: "John Smith", - * fields: ["name", "email", "description"], - * fuzzy: true, - * boost: { "name": 2.0, "email": 1.5 } + * fields: ["name", "email", "description"] * } */ export const FullTextSearchSchema = lazySchema(() => z.object({ query: z.string().describe('Search query text'), fields: z.array(z.string()).optional().describe('Fields to search in (if not specified, searches all text fields)'), - fuzzy: z.boolean().optional().default(false).describe('Enable fuzzy matching (tolerates typos)'), - operator: z.enum(['and', 'or']).optional().default('or').describe('Logical operator between terms'), - boost: z.record(z.string(), z.number()).optional().describe('Field-specific relevance boosting (field name -> boost factor)'), - minScore: z.number().optional().describe('Minimum relevance score threshold'), - language: z.string().optional().describe('Language for text analysis (e.g., "en", "zh", "es")'), - highlight: z.boolean().optional().default(false).describe('Enable search result highlighting'), + fuzzy: z.boolean().optional().default(false).describe('[EXPERIMENTAL — not enforced] Fuzzy matching (tolerate typos). The ADR-0061 expansion reads only `query` + `fields`; no executor receives this flag (#4286).'), + operator: z.enum(['and', 'or']).optional().default('or').describe('[EXPERIMENTAL — not enforced] Logical operator between terms. The ADR-0061 expansion applies its own term semantics; no executor receives this flag (#4286).'), + boost: z.record(z.string(), z.number()).optional().describe('[EXPERIMENTAL — not enforced] Field-specific relevance boosting (field name -> boost factor). No executor scores results (#4286).'), + minScore: z.number().optional().describe('[EXPERIMENTAL — not enforced] Minimum relevance score threshold. No executor scores results (#4286).'), + language: z.string().optional().describe('[EXPERIMENTAL — not enforced] Language for text analysis (e.g., "en", "zh", "es"). No executor selects an analyzer (#4286).'), + highlight: z.boolean().optional().default(false).describe('[EXPERIMENTAL — not enforced] Search result highlighting. No executor emits highlights (#4286).'), })); export type FullTextSearch = z.infer; @@ -566,14 +293,12 @@ export type FullTextSearch = z.infer; * } * * @example - * // Full-text search + * // Full-text search (ADR-0061: expanded into a cross-field $or) * { * object: 'article', * search: { * query: "machine learning", - * fields: ["title", "content"], - * fuzzy: true, - * boost: { "title": 2.0 } + * fields: ["title", "content"] * }, * limit: 10 * } @@ -600,9 +325,9 @@ const BaseQuerySchema = z.object({ top: z.number().optional().describe('Alias for limit (OData compatibility)'), cursor: z.record(z.string(), z.unknown()).optional().describe('Cursor for keyset pagination'), - /** Joins */ - joins: z.array(JoinNodeSchema).optional().describe('Explicit Table Joins'), - + /** Joins — REMOVED (#4286): `expand` is the one spelling for related records. */ + joins: retiredKey(QUERY_JOINS_REMOVED), + /** Aggregations */ aggregations: z.array(AggregationNodeSchema).optional().describe('Aggregation functions'), @@ -612,9 +337,9 @@ const BaseQuerySchema = z.object({ /** Having Clause */ having: FilterConditionSchema.optional().describe('HAVING clause for aggregation filtering'), - /** Window Functions */ - windowFunctions: z.array(WindowFunctionNodeSchema).optional().describe('Window functions with OVER clause'), - + /** Window functions — REMOVED from the request surface (#4286); the capability's door is `SqlDriver.findWithWindowFunctions()`. */ + windowFunctions: retiredKey(QUERY_WINDOW_FUNCTIONS_REMOVED), + /** Subquery flag */ distinct: z.boolean().optional().describe('SELECT DISTINCT flag'), }); @@ -665,9 +390,7 @@ export type SortNode = z.infer; export type AggregationNode = z.infer; export type GroupByNode = z.infer; export type DateGranularityValue = z.infer; -// `JoinNode` is declared next to its schema — it IS that schema's annotation, so -// it cannot be inferred back out of it (#4171). `FieldNode` sits there too, but -// for a different reason since #4196: it is no longer recursive, it is just the -// name the docs and the engine give to "one entry of a select list". -export type WindowFunctionNode = z.infer; -export type WindowSpec = z.infer; +// `FieldNode` is declared next to its schema rather than here: since #4196 it +// is no longer recursive, it is just the name the docs and the engine give to +// "one entry of a select list". (`JoinNode` and `WindowFunctionNode`/`WindowSpec` +// used to sit here too — removed with their clusters, #4286.) diff --git a/packages/spec/src/migrations/registry.ts b/packages/spec/src/migrations/registry.ts index 5f09171706..f53c71c59c 100644 --- a/packages/spec/src/migrations/registry.ts +++ b/packages/spec/src/migrations/registry.ts @@ -587,7 +587,16 @@ const step18: MigrationStep = { + 'MongoDB, and refused by the REST ingress as an unknown field of that name. `expand` is the ' + 'one spelling for nested selection (ADR-0049 enforce-or-remove; Prime Directive #12: one ' + 'capability, one contract). Like the two above it is a request shape, never stored, so the ' - + 'chain has no source to rewrite.', + + 'chain has no source to rewrite.\n\n' + + 'The #4286 sweep applies the same method to the rest of the request surface: `query.joins` ' + + 'and `query.windowFunctions` are tombstoned — no engine or driver ever read either on the ' + + 'query path, so every join and OVER clause a caller declared was silently dropped. Joins ' + + 'were the second, broken spelling of related-record retrieval (`expand` is the live one; ' + + 'the whole JoinNode cluster goes with the key), and window functions only ever ran behind ' + + '`SqlDriver.findWithWindowFunctions()`, a driver-level door whose flat input shape the ' + + 'spec vocabulary never matched (it declared `field`/`over`/`frame` members the door never ' + + 'read — that cluster goes too). Request shapes again: two semantic TODOs, no source ' + + 'rewrite.', conversionIds: ['stack-api-require-auth-removed'], semantic: [ { @@ -623,6 +632,50 @@ const step18: MigrationStep = { 'No /batch, /updateMany or /deleteMany call sends `options.validateOnly`; a request that ' + 'includes it answers 400 VALIDATION_FAILED with the retirement prescription.', }, + { + id: 'query-joins-retired', + surface: 'data.query.joins', + replacement: + "expand (`expand: { owner: { object: 'user', fields: ['name'] } }`), or a dotted " + + "`fields` path for a single related column (`fields: ['owner.name']`)", + reason: + 'The `joins` array was declared-but-inert: no engine or driver read `query.joins` ' + + 'anywhere on the query path, so a query carrying it behaved exactly as if the key were ' + + 'absent — while the name squatted on the reserved REST parameter set. Related-record ' + + 'retrieval already has a live spelling (`expand`, resolved by the engine via batch ' + + '`$in` queries), so the removal deletes the second, broken spelling rather than the ' + + 'capability, and the orphaned `JoinNode`/`JoinType`/`JoinStrategy` cluster goes with ' + + 'the key. A REQUEST surface — `QueryAST` is never stored in stack metadata — so there ' + + 'is no source for the chain to rewrite; callers move their own queries. ' + + 'ADR-0049 / ADR-0078, #4286.', + acceptanceCriteria: + 'No caller sends `joins`; related records are read through `expand` and single related ' + + 'columns through dotted `fields` paths. A query that still carries `joins` fails to ' + + 'parse with the removal prescription (even as an empty array), and authoring it is a ' + + '`tsc` error at the call site.', + }, + { + id: 'query-window-functions-retired', + surface: 'data.query.windowFunctions', + replacement: + '`aggregations` + `groupBy` for request-level analytics; ' + + '`SqlDriver.findWithWindowFunctions(object, query)` for embedders on a SQL datasource', + reason: + 'The `windowFunctions` array was declared-but-inert on the query path: `find()` never ' + + 'applied a window function, so every OVER clause a caller declared was silently ' + + 'dropped. The capability only ever ran behind `SqlDriver.findWithWindowFunctions()`, ' + + 'a driver-level door that is not on the `IDataDriver` contract and whose flat input ' + + 'shape (`{ function, alias, partitionBy?, orderBy? }`) the spec vocabulary never ' + + 'matched — `WindowFunctionNodeSchema` declared `field`/`over`/`frame` members the door ' + + 'never read, so that cluster is removed with the key rather than left as a false ' + + 'affordance. A REQUEST surface, never stored; no source to rewrite. ' + + 'ADR-0049 / ADR-0078, #4286.', + acceptanceCriteria: + 'No caller sends `windowFunctions` in a query; request-level analytics use ' + + '`aggregations` + `groupBy`, and embedders needing OVER-clause SQL call the SQL ' + + "driver's `findWithWindowFunctions` door directly. A query that still carries the key " + + 'fails to parse with the removal prescription naming that door.', + }, ], }; diff --git a/packages/spec/src/recursive-schema-input-assertions.ts b/packages/spec/src/recursive-schema-input-assertions.ts index 5e8808baaa..0e9490dc61 100644 --- a/packages/spec/src/recursive-schema-input-assertions.ts +++ b/packages/spec/src/recursive-schema-input-assertions.ts @@ -62,8 +62,6 @@ import type { import type { FieldNode, FieldNodeSchema, - JoinNodeInput, - JoinNodeSchema, QueryInput, QuerySchema, } from './data/query.zod'; @@ -83,19 +81,9 @@ export const queryNotAString: QueryInput = 'not a query at all'; // @ts-expect-error — `object` is required; nothing else can stand in for it export const queryNeedsObject: QueryInput = { fields: ['name'] }; -/** A join carries its ON condition and may join a derived subquery. */ -export const joinInput: JoinNodeInput = { - type: 'inner', - object: 'contact', - on: { account_id: 'id' }, - subquery: { object: 'account' }, -}; - -// @ts-expect-error — a join node is not a number -export const joinNotANumber: JoinNodeInput = 42; - -// @ts-expect-error — `type` must be one of the four declared join kinds -export const joinBadType: JoinNodeInput = { type: 'sideways', object: 'c', on: {} }; +// `JoinNodeSchema` probes removed with the join cluster (#4286): `query.joins` +// is tombstoned and the schema — the package's other recursive knot through +// `QuerySchema` — is gone, so there is no input half left to pin. /** * A select entry is a field name, optionally dotted through a relationship. @@ -217,9 +205,6 @@ export const formFieldNeedsField: FormFieldInput = { type: 'text' }; // @ts-expect-error — QuerySchema's input is checked, not `unknown` export const wiredQuery: z.input = 42; -// @ts-expect-error — JoinNodeSchema's input is checked -export const wiredJoin: z.input = 42; - // @ts-expect-error — FieldNodeSchema's input is checked export const wiredFieldNode: z.input = 42; diff --git a/skills/objectstack-query/SKILL.md b/skills/objectstack-query/SKILL.md index 05f7d24a7c..f8f095b420 100644 --- a/skills/objectstack-query/SKILL.md +++ b/skills/objectstack-query/SKILL.md @@ -445,10 +445,12 @@ Load related records through lookup/master_detail fields: ## Joins -> ⚠️ **Schema-reserved — NOT executed by the engine yet.** `joins` (and the -> `JoinStrategy` hints) exist only in the `QueryAST` schema — no engine or -> driver code consumes them, regardless of a driver advertising -> `supports.joins`. A query carrying `joins` behaves as if they were absent. +> ⛔ **REMOVED in `@objectstack/spec` 18 (#4286, ADR-0049).** `query.joins` +> (and the `JoinNode`/`JoinType`/`JoinStrategy` vocabulary) is gone from the +> `QueryAST` schema — no engine or driver ever consumed it, so it only ever +> declared a capability that did not run. The key is tombstoned: authoring it +> is a `tsc` error, and a query carrying it (even `joins: []`) fails to parse +> with the upgrade prescription. Do not emit `joins`. **Working alternatives** (both implemented): - **`expand`** — load related records through lookup / master_detail fields @@ -495,20 +497,24 @@ fields match by option *label*, mapped to stored values. Omit `fields` to search the object's declared `searchableFields` (or an auto-default of name/title + short-text fields), resolved server-side. -> ⚠️ **Schema-reserved — NOT executed by the engine yet:** `fuzzy`, `boost`, +> ⚠️ **`[EXPERIMENTAL — not enforced]` (#4286):** `fuzzy`, `boost`, > `operator`, `minScore`, `language`, and `highlight` validate against the -> schema but are never read. Terms are always AND-ed; there is no relevance -> scoring or highlighting. +> schema but are never read — their `.describe()` markers now say so. Terms +> are always AND-ed; there is no relevance scoring or highlighting. --- ## Window Functions (Analytics) -> ⚠️ **Schema-reserved — NOT executed by the engine yet.** `windowFunctions` -> exists in the `QueryAST` schema, but the engine never routes it to any -> driver — the property is silently dropped from ordinary queries. (Even the -> SQL driver's internal builder drops the `field` argument, so `lag(revenue)` -> would render as `LAG()`.) Do not emit `windowFunctions`. +> ⛔ **REMOVED from the request surface in `@objectstack/spec` 18 (#4286).** +> `query.windowFunctions` is gone from the `QueryAST` schema — the engine +> never routed it to any driver, so every OVER clause it declared was +> silently dropped. The key is tombstoned (a query carrying it fails to +> parse with the prescription), and the `WindowFunction`/`WindowSpec`/ +> `WindowFunctionNode` exports left with it. Do not emit `windowFunctions`. +> The one live door is the SQL driver's own `findWithWindowFunctions()` +> method (driver-level, its own flat input shape — and even there the +> builder drops the `field` argument, so `lag(revenue)` renders as `LAG()`). **Working alternatives:** - **Ranking / top-N per group and running totals:** model them in @@ -517,13 +523,6 @@ auto-default of name/title + short-text fields), resolved server-side. - **Ad-hoc analysis:** fetch the ordered rows (`orderBy` + `limit`) and compute ranks or running sums in application code. -### Window Function Enum (schema-reserved) - -For completeness, the full `WindowFunction` enum declared by the schema: -`row_number`, `rank`, `dense_rank`, `percent_rank`, `lag`, `lead`, -`first_value`, `last_value`, `sum`, `avg`, `count`, `min`, `max`. -None of these execute today. - --- ## Common Patterns @@ -536,7 +535,7 @@ None of these execute today. | Filter parent by child conditions | Nested relation filter | | Simple parent→child navigation | `expand` | | Paginate/sort a parent's related records | Query the related object directly | -| Analytical queries across objects | Report/dashboard metadata, or separate queries combined in app code (`joins` is schema-reserved — see above) | +| Analytical queries across objects | Report/dashboard metadata, or separate queries combined in app code (`joins` was removed in #4286 — see above) | ### Pagination Pattern for APIs diff --git a/skills/objectstack-query/evals/README.md b/skills/objectstack-query/evals/README.md index e270f31daa..b34b5beac9 100644 --- a/skills/objectstack-query/evals/README.md +++ b/skills/objectstack-query/evals/README.md @@ -12,7 +12,7 @@ subset the engine actually executes. instead of `$null`. 2. **Nested relation filter** — "Find orders where the customer's country is US." Expect a nested relation filter (`customer: { country: 'US' }`), - not a `joins` array (schema-reserved). + not a `joins` array (removed in #4286). 3. **Pagination pattern** — "Implement infinite scroll for a feed." Expect manual keyset pagination (`where` on the sort key + `orderBy` + `limit`); fail if the answer uses the schema-reserved `cursor` property. diff --git a/skills/objectstack-query/rules/aggregation.md b/skills/objectstack-query/rules/aggregation.md index eacc8b2974..5b09784a3e 100644 --- a/skills/objectstack-query/rules/aggregation.md +++ b/skills/objectstack-query/rules/aggregation.md @@ -160,13 +160,14 @@ const [active] = await engine.aggregate('user', { ## Window Functions -> ⚠️ **Schema-reserved — NOT executed by the engine yet.** The `QueryAST` -> schema declares `windowFunctions` (enum: `row_number`, `rank`, -> `dense_rank`, `percent_rank`, `lag`, `lead`, `first_value`, `last_value`, -> `sum`, `avg`, `count`, `min`, `max`), but the engine never routes the -> property to any driver — it is silently dropped. Even the SQL driver's -> internal builder drops the `field` argument (`lag(revenue)` would render -> as `LAG()`). Do not emit `windowFunctions` in queries. +> ⛔ **REMOVED in `@objectstack/spec` 18 (#4286, ADR-0049).** The `QueryAST` +> schema no longer declares `windowFunctions` — the engine never routed the +> property to any driver, so it was silently dropped. The key is tombstoned: +> a query carrying it fails to parse with the upgrade prescription. The one +> live door is the SQL driver's own `findWithWindowFunctions()` (driver-level, +> its own flat input shape; even there the builder drops the `field` argument, +> so `lag(revenue)` renders as `LAG()`). Do not emit `windowFunctions` in +> queries. **Working alternatives:**