Skip to content

Commit 8675db6

Browse files
os-zhuangclaude
andauthored
refactor(data)!: a select-list entry is a field name — the nested-select object form is removed (#4196) (#4268)
`FieldNodeSchema` declared two forms for one entry of `QueryAST['fields']`: a string, and `{ field, fields?, alias? }` for a nested select on a relationship. The object form was declared-but-inert. Nothing produced it, and nothing read `.fields` or `.alias`: - `objectql/engine.ts` — formula projection and both known-field filters treat the list as `string[]`; one already reached for `String(f).split('.')[0]`, which is what defensiveness against an unhandled shape looks like. - `driver-sql` `select()`, `driver-memory` `projectFields` — same. - `driver-mongodb` keyed its projection with the entry itself, so an object entry asked for a column literally named "[object Object]". #4194 taught it to read `.field` because the newly-precise type made the old line a compile error — defensive handling of a shape nothing sends, not evidence of life. - The REST ingress stringified each entry before comparing it to the field map, so the same entry came back as `400 INVALID_FIELD: Unknown field '[object Object]'` — a rejection naming something the caller never wrote. So `fields: [{ field: 'owner', fields: ['name'] }]` was accepted by validation and then dropped or mangled depending on the driver (ADR-0078 silently-inert declaration; ADR-0049 enforce-or-remove). The capability it described is already served by `expand`, resolved through batch `$in` queries. Removing the second spelling rather than lowering it into the first is Prime Directive #12: one capability, one contract. `FieldNode` is now `string`; `FieldNodeSchema` is `z.string()` with an error map that answers an object entry with the FROM → TO prescription instead of zod's "expected string, received object" — and only an object, since telling the author of `fields: [42]` that their entry "was removed" would misinform. `z.input` becomes `string`, so `tsc` fails at the authoring site first. `assertProjectionFieldsExist` gains the shape axis, judged BEFORE the object's field map: the entry is wrong about its shape, not about this object, and a registry-less host (which the name gate gives up on) would otherwise hand it to a driver that cannot read it. Registered as an ADR-0087 D3 semantic migration (`query-field-node-object-form-retired`) rather than a D2 conversion: `QueryAST` is a request shape, never stored in stack metadata, so the chain has no source to rewrite. Same treatment as `EnhancedApiError.fieldErrors` and `BatchOptions.validateOnly` on this step. Two spec tests that pinned the object form as parseable are inverted — they proved the DECLARATION, which is exactly how a declared-but-inert shape survives a green suite. New pins cover the prescription, the non-object mistake keeping zod's own message, and the `expand` replacement; the list path gets three more in the #4226 conformance suite. Merging `main` also caught a jointly-wrong pair with no git conflict: #4252's recursive-schema input pins assert the retired object form is assignable. Those two probes move to the negative side (`FieldNode` is no longer recursive, so `FieldNodeSchema` infers both halves itself) — the AGENTS.md §10 case. No runtime behaviour changes for anything that ever worked; the defensive unwrapping the drivers had grown against a shape nothing sends goes with it. Closes #4196. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 parent 3ec8186 commit 8675db6

19 files changed

Lines changed: 338 additions & 122 deletions

File tree

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
---
2+
"@objectstack/spec": major
3+
"@objectstack/objectql": patch
4+
"@objectstack/metadata-protocol": patch
5+
"@objectstack/driver-mongodb": patch
6+
"@objectstack/driver-sql": patch
7+
"@objectstack/driver-memory": patch
8+
---
9+
10+
refactor(data)!: a select-list entry is a field name — the nested-select object form is removed (#4196)
11+
12+
`FieldNode` declared two forms for one entry of `QueryAST['fields']`:
13+
14+
```ts
15+
type FieldNode =
16+
| string // "name"
17+
| { field: string; fields?: FieldNode[]; alias?: string }; // nested select
18+
```
19+
20+
The object form was **declared-but-inert**. Nothing produced it, and nothing
21+
read `.fields` or `.alias` — every consumer on the path treats the list as
22+
`string[]`: `objectql`'s formula projection and its two known-field filters,
23+
`driver-sql`'s `select()`, `driver-memory`'s `projectFields`. `driver-mongodb`
24+
keyed its projection with the entry itself, so an object entry asked for a
25+
column literally named `"[object Object]"`, and the REST ingress stringified
26+
each entry before comparing it to the field map, so the same entry came back as
27+
`400 INVALID_FIELD: Unknown field '[object Object]'` — a rejection naming
28+
something the caller never wrote. An author who wrote
29+
`fields: [{ field: 'owner', fields: ['name'] }]` got it accepted by validation
30+
and then dropped or mangled, depending on the driver (ADR-0078 silently-inert
31+
declaration; ADR-0049 enforce-or-remove).
32+
33+
The capability the object form described is already served, by a different key.
34+
Removing the second spelling rather than lowering it into the first is Prime
35+
Directive #12: one capability, one contract.
36+
37+
**FROM → TO**
38+
39+
| Was | Now |
40+
| :--- | :--- |
41+
| `fields: [{ field: 'owner', fields: ['name'] }]` | `expand: { owner: { object: 'user', fields: ['name'] } }` |
42+
| `fields: [{ field: 'owner' }]` | `fields: ['owner']` |
43+
| `fields: [{ field: 'owner', fields: ['name'] }]`, one column only | `fields: ['owner.name']` (dotted path) |
44+
| `fields: [{ field: 'total', alias: 't' }]` | `aggregations` / `windowFunctions` — they carry the live `alias` |
45+
46+
The one-line fix: **a `fields[]` entry is a string.** Move nested selection to
47+
`expand`, which the engine resolves through batch `$in` queries (default max
48+
depth 3).
49+
50+
There is no `os migrate meta` step, and deliberately so: `QueryAST` is a request
51+
shape, never stored in stack metadata, so the chain has no source to rewrite. It
52+
is registered as an ADR-0087 D3 **semantic** migration
53+
(`query-field-node-object-form-retired`) on the protocol-18 step instead — the
54+
`EnhancedApiError.fieldErrors` / `BatchOptions.validateOnly` precedent. Callers
55+
move their own select lists, and both channels tell them how:
56+
57+
- **The parse.** `FieldNodeSchema` narrows to `z.string()` with an error map that
58+
answers an object entry with the prescription above, not "expected string,
59+
received object". `z.input` becomes `string`, so `tsc` fails at the authoring
60+
site first.
61+
- **The ingress.** `assertProjectionFieldsExist` judges the entry's *shape*
62+
before consulting the object's field map — it is wrong about the shape, not
63+
about this object, and a registry-less host would otherwise pass it to a driver
64+
that cannot read it. The 400 now names the retired form instead of the field
65+
`"[object Object]"`.
66+
67+
No runtime behaviour changes for anything that ever worked; the defensive
68+
unwrapping the drivers had grown against a shape nothing sends goes with it.

content/docs/kernel/contracts/data-engine.mdx

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -85,7 +85,7 @@ Defined by `EngineQueryOptionsSchema` in `@objectstack/spec`:
8585
```typescript
8686
interface EngineQueryOptions {
8787
where?: FilterCondition; // WHERE clause — MongoDB-style $op
88-
fields?: FieldNode[]; // SELECT fields
88+
fields?: FieldNode[]; // SELECT — field names ('name', 'owner.name')
8989
orderBy?: SortNode[]; // ORDER BY
9090
limit?: number; // LIMIT
9191
offset?: number; // OFFSET
@@ -387,7 +387,7 @@ The following legacy parameter names are accepted by the RPC layer for backward
387387
| Legacy (Deprecated) | Canonical | Notes |
388388
|:---------------------|:----------|:------|
389389
| `filter` | `where` | FilterCondition object |
390-
| `select` | `fields` | Array of FieldNode |
390+
| `select` | `fields` | Array of FieldNode (field-name strings) |
391391
| `sort` | `orderBy` | Array of `{ field, order }` |
392392
| `skip` | `offset` | Number |
393393
| `populate` | `expand` | Record of field → QueryAST |

content/docs/protocol/objectql/query-syntax.mdx

Lines changed: 12 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -62,7 +62,7 @@ import type { QueryAST } from '@objectstack/spec/data';
6262
// QueryAST — full structure
6363
interface QueryAST {
6464
object: string; // Target object (required)
65-
fields?: FieldNode[]; // Projection (SELECT)
65+
fields?: FieldNode[]; // Projection (SELECT) — field names
6666
where?: FilterCondition; // Filtering (WHERE) — MongoDB-style $op
6767
search?: FullTextSearch; // Full-text search ($search)
6868
orderBy?: SortNode[]; // Ordering (ORDER BY)
@@ -97,7 +97,6 @@ on the `find()` path:
9797
| `cursor` | Accepted by `EngineQueryOptions`, but no driver implements keyset pagination |
9898
| `distinct` | Not applied by `find()`; the SQL and in-memory drivers expose a separate `distinct(object, field, filters?)` method instead |
9999
| `windowFunctions` | Not applied by `find()`; `SqlDriver.findWithWindowFunctions()` is a separate method that is not on the `IDataDriver` contract |
100-
| Object `FieldNode`s (`{ field, fields }`) | The engine drops non-string entries from `fields` — use `expand` to project related records |
101100
| `search.fuzzy` / `boost` / `operator` / `minScore` / `language` / `highlight` | Parsed by the schema but ignored: only `query` and `fields` drive the expansion |
102101

103102
`top` is the exception that *is* honored — the engine normalises it to `limit`.
@@ -122,12 +121,16 @@ interface AggregationNode {
122121
filter?: FilterCondition; // FILTER WHERE clause — protocol only, never applied
123122
}
124123

125-
// FieldNode — field selection
126-
type FieldNode = string | {
127-
field: string;
128-
fields?: FieldNode[]; // nested select — protocol only, dropped by the engine
129-
alias?: string;
130-
};
124+
// FieldNode — one entry of the select list. A field name, optionally dotted to
125+
// reach through a relationship ('owner.name'). Related *records* come from
126+
// `expand`, not from inside this list.
127+
//
128+
// The `{ field, fields, alias }` nested-select member this union used to carry
129+
// was REMOVED in protocol 18 (#4196): nothing produced it and nothing read
130+
// `.fields`/`.alias`, so it was dropped by the SQL and memory drivers,
131+
// projected as a column named "[object Object]" by MongoDB, and refused as an
132+
// unknown field by the REST ingress. `expand` is the one spelling.
133+
type FieldNode = string;
131134

132135
// GroupByNode — GROUP BY target
133136
type GroupByNode = string | {
@@ -980,7 +983,7 @@ Similarly, the following legacy field names should not be used in new code:
980983
| `aggregate` (object map) | `aggregations` (array) | Array of `AggregationNode` objects |
981984
| `expand` (string array) | `expand` (Record) | Map of field name → nested QueryAST |
982985
| `skip` | `offset` | Number |
983-
| `select` | `fields` | Array of FieldNode |
986+
| `select` | `fields` | Array of FieldNode (field-name strings) |
984987
| `populate` | `expand` | Map of field name → nested QueryAST |
985988
</Callout>
986989

content/docs/references/api/contract.mdx

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -147,7 +147,7 @@ const result = ApiError.parse(data);
147147
| Property | Type | Required | Description |
148148
| :--- | :--- | :--- | :--- |
149149
| **object** | `string` || Object name (e.g. account) |
150-
| **fields** | `string \| { field: string; fields?: object[]; alias?: string }[]` | optional | Fields to retrieve |
150+
| **fields** | `string[]` | optional | Fields to retrieve — field names, optionally dotted to reach through a relationship (`owner.name`). Related *records* are selected with `expand`, not from inside this list. |
151151
| **where** | `any` | optional | Filtering criteria (WHERE) |
152152
| **search** | `{ query: string; fields?: string[]; fuzzy: boolean; operator: Enum<'and' \| 'or'>; … }` | optional | Full-text search configuration ($search parameter) |
153153
| **orderBy** | `{ field: string; order: Enum<'asc' \| 'desc'> }[]` | optional | Sorting instructions (ORDER BY) |
@@ -161,7 +161,7 @@ const result = ApiError.parse(data);
161161
| **having** | `any` | optional | HAVING clause for aggregation filtering |
162162
| **windowFunctions** | `{ function: Enum<'row_number' \| 'rank' \| 'dense_rank' \| 'percent_rank' \| 'lag' \| 'lead' \| 'first_value' \| 'last_value' \| 'sum' \| 'avg' \| 'count' \| 'min' \| 'max'>; field?: string; alias: string; over: object }[]` | optional | Window functions with OVER clause |
163163
| **distinct** | `boolean` | optional | SELECT DISTINCT flag |
164-
| **expand** | `Record<string, { object: string; fields?: string \| { field: string; fields?: object[]; alias?: string }[]; where?: any; search?: object; … }>` | optional | Recursive relation loading map. Keys are lookup/master_detail field names; values are nested QueryAST objects that control select (`fields`) and filter (`where`, AND-merged with the batch $in), plus further expansion on the related object. The engine resolves expand via batch $in queries (driver-agnostic) with a default max depth of 3; per-parent `limit`/`offset`/`orderBy` are NOT applied on this path. |
164+
| **expand** | `Record<string, { object: string; fields?: string[]; where?: any; search?: object; … }>` | optional | Recursive relation loading map. Keys are lookup/master_detail field names; values are nested QueryAST objects that control select (`fields`) and filter (`where`, AND-merged with the batch $in), plus further expansion on the related object. The engine resolves expand via batch $in queries (driver-agnostic) with a default max depth of 3; per-parent `limit`/`offset`/`orderBy` are NOT applied on this path. |
165165

166166

167167
---

content/docs/references/api/protocol.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -529,7 +529,7 @@ const result = AiAgentCapabilities.parse(data);
529529
| Property | Type | Required | Description |
530530
| :--- | :--- | :--- | :--- |
531531
| **object** | `string` || The unique machine name of the object to query (e.g. "account"). |
532-
| **query** | `{ object: string; fields?: string \| { field: string; fields?: object[]; alias?: string }[]; where?: any; search?: object; … }` | optional | Structured query definition (filter, sort, select, pagination). |
532+
| **query** | `{ object: string; fields?: string[]; where?: any; search?: object; … }` | optional | Structured query definition (filter, sort, select, pagination). |
533533

534534

535535
---

content/docs/references/data/data-engine.mdx

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -186,7 +186,7 @@ Reference: any
186186
| :--- | :--- | :--- | :--- |
187187
| **method** | `'findOne'` || |
188188
| **object** | `string` || |
189-
| **query** | `{ context?: object; where?: Record<string, any> \| any; fields?: string \| { field: string; fields?: object[]; alias?: string }[]; orderBy?: { field: string; order: Enum<'asc' \| 'desc'> }[]; … }` | optional | |
189+
| **query** | `{ context?: object; where?: Record<string, any> \| any; fields?: string[]; orderBy?: { field: string; order: Enum<'asc' \| 'desc'> }[]; … }` | optional | |
190190

191191

192192
---
@@ -199,7 +199,7 @@ Reference: any
199199
| :--- | :--- | :--- | :--- |
200200
| **method** | `'find'` || |
201201
| **object** | `string` || |
202-
| **query** | `{ context?: object; where?: Record<string, any> \| any; fields?: string \| { field: string; fields?: object[]; alias?: string }[]; orderBy?: { field: string; order: Enum<'asc' \| 'desc'> }[]; … }` | optional | |
202+
| **query** | `{ context?: object; where?: Record<string, any> \| any; fields?: string[]; orderBy?: { field: string; order: Enum<'asc' \| 'desc'> }[]; … }` | optional | |
203203

204204

205205
---
@@ -268,7 +268,7 @@ This schema accepts one of the following structures:
268268
| :--- | :--- | :--- | :--- |
269269
| **method** | `'find'` || |
270270
| **object** | `string` || |
271-
| **query** | `{ context?: object; where?: Record<string, any> \| any; fields?: string \| { field: string; fields?: object[]; alias?: string }[]; orderBy?: { field: string; order: Enum<'asc' \| 'desc'> }[]; … }` | optional | |
271+
| **query** | `{ context?: object; where?: Record<string, any> \| any; fields?: string[]; orderBy?: { field: string; order: Enum<'asc' \| 'desc'> }[]; … }` | optional | |
272272

273273
---
274274

@@ -280,7 +280,7 @@ This schema accepts one of the following structures:
280280
| :--- | :--- | :--- | :--- |
281281
| **method** | `'findOne'` || |
282282
| **object** | `string` || |
283-
| **query** | `{ context?: object; where?: Record<string, any> \| any; fields?: string \| { field: string; fields?: object[]; alias?: string }[]; orderBy?: { field: string; order: Enum<'asc' \| 'desc'> }[]; … }` | optional | |
283+
| **query** | `{ context?: object; where?: Record<string, any> \| any; fields?: string[]; orderBy?: { field: string; order: Enum<'asc' \| 'desc'> }[]; … }` | optional | |
284284

285285
---
286286

@@ -540,14 +540,14 @@ QueryAST-aligned query options for IDataEngine.find() operations
540540
| :--- | :--- | :--- | :--- |
541541
| **context** | `{ userId?: string; actor?: string; email?: string; tenantId?: string; … }` | optional | |
542542
| **where** | `Record<string, any> \| any` | optional | |
543-
| **fields** | `string \| { field: string; fields?: object[]; alias?: string }[]` | optional | |
543+
| **fields** | `string[]` | optional | |
544544
| **orderBy** | `{ field: string; order: Enum<'asc' \| 'desc'> }[]` | optional | |
545545
| **limit** | `number` | optional | |
546546
| **offset** | `number` | optional | |
547547
| **top** | `number` | optional | |
548548
| **cursor** | `Record<string, any>` | optional | |
549549
| **search** | `{ query: string; fields?: string[]; fuzzy: boolean; operator: Enum<'and' \| 'or'>; … }` | optional | |
550-
| **expand** | `Record<string, { object: string; fields?: string \| { field: string; fields?: object[]; alias?: string }[]; where?: any; search?: object; … }>` | optional | |
550+
| **expand** | `Record<string, { object: string; fields?: string[]; where?: any; search?: object; … }>` | optional | |
551551
| **distinct** | `boolean` | optional | |
552552

553553

content/docs/references/data/mapping.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,7 @@ const result = FieldMapping.parse(data);
5252
| **fieldMapping** | `{ source: string \| string[]; target: string \| string[]; transform: Enum<'none' \| 'constant' \| 'lookup' \| 'split' \| 'join' \| 'javascript' \| 'map'>; params?: object }[]` || |
5353
| **mode** | `Enum<'insert' \| 'update' \| 'upsert'>` || |
5454
| **upsertKey** | `string[]` | optional | Fields to match for upsert (e.g. email) |
55-
| **extractQuery** | `{ object: string; fields?: string \| { field: string; fields?: object[]; alias?: string }[]; where?: any; search?: object; … }` | optional | Query to run for export only |
55+
| **extractQuery** | `{ object: string; fields?: string[]; where?: any; search?: object; … }` | optional | Query to run for export only |
5656
| **errorPolicy** | `Enum<'skip' \| 'abort' \| 'retry'>` || |
5757
| **batchSize** | `number` || |
5858

content/docs/references/data/query.mdx

Lines changed: 2 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -67,30 +67,6 @@ const result = AggregationFunction.parse(data);
6767
* `year`
6868

6969

70-
---
71-
72-
## FieldNode
73-
74-
### Union Options
75-
76-
This schema accepts one of the following structures:
77-
78-
#### Option 1
79-
80-
Type: `string`
81-
82-
---
83-
84-
#### Option 2
85-
86-
### Properties
87-
88-
| Property | Type | Required | Description |
89-
| :--- | :--- | :--- | :--- |
90-
| **field** | `string` || |
91-
| **fields** | `[FieldNode](#fieldnode)[]` | optional | |
92-
| **alias** | `string` | optional | |
93-
9470
---
9571

9672

@@ -152,7 +128,7 @@ Type: `string`
152128
| **object** | `string` || Object/table to join |
153129
| **alias** | `string` | optional | Table alias |
154130
| **on** | `any` || Join condition |
155-
| **subquery** | `{ object: string; fields?: string \| { field: string; fields?: object[]; alias?: string }[]; where?: any; search?: object; … }` | optional | Subquery instead of object |
131+
| **subquery** | `{ object: string; fields?: string[]; where?: any; search?: object; … }` | optional | Subquery instead of object |
156132

157133

158134
---
@@ -188,7 +164,7 @@ Type: `string`
188164
| Property | Type | Required | Description |
189165
| :--- | :--- | :--- | :--- |
190166
| **object** | `string` || Object name (e.g. account) |
191-
| **fields** | `string \| { field: string; fields?: object[]; alias?: string }[]` | optional | Fields to retrieve |
167+
| **fields** | `string[]` | optional | Fields to retrieve — field names, optionally dotted to reach through a relationship (`owner.name`). Related *records* are selected with `expand`, not from inside this list. |
192168
| **where** | `any` | optional | Filtering criteria (WHERE) |
193169
| **search** | `{ query: string; fields?: string[]; fuzzy: boolean; operator: Enum<'and' \| 'or'>; … }` | optional | Full-text search configuration ($search parameter) |
194170
| **orderBy** | `{ field: string; order: Enum<'asc' \| 'desc'> }[]` | optional | Sorting instructions (ORDER BY) |

docs/audits/2026-07-unknown-key-strictness-ledger.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -146,7 +146,7 @@ tightening (the #4001 "sharing-rule lesson": candidates, not verdicts).
146146
| `notification.zod.ts` / `offline.zod.ts` / `report.zod.ts` | 3 ea | authorable (p) | |
147147
| `sharing.zod.ts` | 2 | authorable (p) | public-sharing config |
148148

149-
### `data/`154 sites
149+
### `data/`153 sites
150150

151151
| File | Sites | Class | Note |
152152
|---|---|---|---|
@@ -155,7 +155,7 @@ tightening (the #4001 "sharing-rule lesson": candidates, not verdicts).
155155
| `external-lookup.zod.ts` | 12 | mixed (p) | authored config + wire results |
156156
| `seed-loader.zod.ts` | 12 | mixed (p) | seed file shapes are authored; loader state is runtime |
157157
| `field.zod.ts` | 11 | authorable | partially strict |
158-
| `filter.zod.ts` / `query.zod.ts` | 11+10 | open | query dialect — user data flows through; validated semantically elsewhere |
158+
| `filter.zod.ts` / `query.zod.ts` | 11+9 | open | query dialect — user data flows through; validated semantically elsewhere. `query.zod.ts` dropped one site in #4196: `FieldNodeSchema`'s nested-select object form was declared-but-inert and narrowed to `z.string()`, so the union's second member is gone. Class unchanged |
159159
| `driver-nosql.zod.ts` / `driver.zod.ts` / `driver-sql.zod.ts` | 10+9+2 | wire | driver capability contracts |
160160
| `datasource.zod.ts` | 9 | authorable | **strict as of #4001 data step** — all 9: `DatasourceSchema` (+ `pool` / `healthCheck` / `ssl` / `retryPolicy`), `ExternalDatasourceSettingsSchema` (+ `validation`), `DatasourceCapabilities`, `DriverDefinitionSchema`. `config` + `readReplicas` stay `z.record` by construction (per-driver shapes; the driver's own `configSchema` validates them) — which is precisely why the top level had to close: a connection key written one level too high was stripped, and the datasource then connected on driver defaults instead of failing |
161161
| `analytics.zod.ts` | 8 | mixed (p) | |

0 commit comments

Comments
 (0)