Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 68 additions & 0 deletions .changeset/query-field-node-object-form-removed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
---
"@objectstack/spec": major
"@objectstack/objectql": patch
"@objectstack/metadata-protocol": patch
"@objectstack/driver-mongodb": patch
"@objectstack/driver-sql": patch
"@objectstack/driver-memory": patch
---

refactor(data)!: a select-list entry is a field name — the nested-select object form is removed (#4196)

`FieldNode` declared two forms for one entry of `QueryAST['fields']`:

```ts
type FieldNode =
| string // "name"
| { field: string; fields?: FieldNode[]; alias?: string }; // nested select
```

The object form was **declared-but-inert**. Nothing produced it, and nothing
read `.fields` or `.alias` — every consumer on the path treats the list as
`string[]`: `objectql`'s formula projection and its two known-field filters,
`driver-sql`'s `select()`, `driver-memory`'s `projectFields`. `driver-mongodb`
keyed its projection with the entry itself, so an object entry asked for a
column literally named `"[object Object]"`, and the REST ingress stringified
each entry before comparing it to the field map, so the same entry came back as
`400 INVALID_FIELD: Unknown field '[object Object]'` — a rejection naming
something the caller never wrote. An author who wrote
`fields: [{ field: 'owner', fields: ['name'] }]` got it accepted by validation
and then dropped or mangled, depending on the driver (ADR-0078 silently-inert
declaration; ADR-0049 enforce-or-remove).

The capability the object form described is already served, by a different key.
Removing the second spelling rather than lowering it into the first is Prime
Directive #12: one capability, one contract.

**FROM → TO**

| Was | Now |
| :--- | :--- |
| `fields: [{ field: 'owner', fields: ['name'] }]` | `expand: { owner: { object: 'user', fields: ['name'] } }` |
| `fields: [{ field: 'owner' }]` | `fields: ['owner']` |
| `fields: [{ field: 'owner', fields: ['name'] }]`, one column only | `fields: ['owner.name']` (dotted path) |
| `fields: [{ field: 'total', alias: 't' }]` | `aggregations` / `windowFunctions` — they carry the live `alias` |

The one-line fix: **a `fields[]` entry is a string.** Move nested selection to
`expand`, which the engine resolves through batch `$in` queries (default max
depth 3).

There is no `os migrate meta` step, and deliberately so: `QueryAST` is a request
shape, never stored in stack metadata, so the chain has no source to rewrite. It
is registered as an ADR-0087 D3 **semantic** migration
(`query-field-node-object-form-retired`) on the protocol-18 step instead — the
`EnhancedApiError.fieldErrors` / `BatchOptions.validateOnly` precedent. Callers
move their own select lists, and both channels tell them how:

- **The parse.** `FieldNodeSchema` narrows to `z.string()` with an error map that
answers an object entry with the prescription above, not "expected string,
received object". `z.input` becomes `string`, so `tsc` fails at the authoring
site first.
- **The ingress.** `assertProjectionFieldsExist` judges the entry's *shape*
before consulting the object's field map — it is wrong about the shape, not
about this object, and a registry-less host would otherwise pass it to a driver
that cannot read it. The 400 now names the retired form instead of the field
`"[object Object]"`.

No runtime behaviour changes for anything that ever worked; the defensive
unwrapping the drivers had grown against a shape nothing sends goes with it.
4 changes: 2 additions & 2 deletions content/docs/kernel/contracts/data-engine.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ Defined by `EngineQueryOptionsSchema` in `@objectstack/spec`:
```typescript
interface EngineQueryOptions {
where?: FilterCondition; // WHERE clause — MongoDB-style $op
fields?: FieldNode[]; // SELECT fields
fields?: FieldNode[]; // SELECT — field names ('name', 'owner.name')
orderBy?: SortNode[]; // ORDER BY
limit?: number; // LIMIT
offset?: number; // OFFSET
Expand Down Expand Up @@ -387,7 +387,7 @@ The following legacy parameter names are accepted by the RPC layer for backward
| Legacy (Deprecated) | Canonical | Notes |
|:---------------------|:----------|:------|
| `filter` | `where` | FilterCondition object |
| `select` | `fields` | Array of FieldNode |
| `select` | `fields` | Array of FieldNode (field-name strings) |
| `sort` | `orderBy` | Array of `{ field, order }` |
| `skip` | `offset` | Number |
| `populate` | `expand` | Record of field → QueryAST |
Expand Down
21 changes: 12 additions & 9 deletions content/docs/protocol/objectql/query-syntax.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ import type { QueryAST } from '@objectstack/spec/data';
// QueryAST — full structure
interface QueryAST {
object: string; // Target object (required)
fields?: FieldNode[]; // Projection (SELECT)
fields?: FieldNode[]; // Projection (SELECT) — field names
where?: FilterCondition; // Filtering (WHERE) — MongoDB-style $op
search?: FullTextSearch; // Full-text search ($search)
orderBy?: SortNode[]; // Ordering (ORDER BY)
Expand Down Expand Up @@ -97,7 +97,6 @@ on the `find()` path:
| `cursor` | Accepted by `EngineQueryOptions`, but no driver implements keyset pagination |
| `distinct` | Not applied by `find()`; the SQL and in-memory drivers expose a separate `distinct(object, field, filters?)` method instead |
| `windowFunctions` | Not applied by `find()`; `SqlDriver.findWithWindowFunctions()` is a separate method that is not on the `IDataDriver` contract |
| Object `FieldNode`s (`{ field, fields }`) | The engine drops non-string entries from `fields` — use `expand` to project related records |
| `search.fuzzy` / `boost` / `operator` / `minScore` / `language` / `highlight` | Parsed by the schema but ignored: only `query` and `fields` drive the expansion |

`top` is the exception that *is* honored — the engine normalises it to `limit`.
Expand All @@ -122,12 +121,16 @@ interface AggregationNode {
filter?: FilterCondition; // FILTER WHERE clause — protocol only, never applied
}

// FieldNode — field selection
type FieldNode = string | {
field: string;
fields?: FieldNode[]; // nested select — protocol only, dropped by the engine
alias?: string;
};
// FieldNode — one entry of the select list. A field name, optionally dotted to
// reach through a relationship ('owner.name'). Related *records* come from
// `expand`, not from inside this list.
//
// The `{ field, fields, alias }` nested-select member this union used to carry
// was REMOVED in protocol 18 (#4196): nothing produced it and nothing read
// `.fields`/`.alias`, so it was dropped by the SQL and memory drivers,
// projected as a column named "[object Object]" by MongoDB, and refused as an
// unknown field by the REST ingress. `expand` is the one spelling.
type FieldNode = string;

// GroupByNode — GROUP BY target
type GroupByNode = string | {
Expand Down Expand Up @@ -980,7 +983,7 @@ Similarly, the following legacy field names should not be used in new code:
| `aggregate` (object map) | `aggregations` (array) | Array of `AggregationNode` objects |
| `expand` (string array) | `expand` (Record) | Map of field name → nested QueryAST |
| `skip` | `offset` | Number |
| `select` | `fields` | Array of FieldNode |
| `select` | `fields` | Array of FieldNode (field-name strings) |
| `populate` | `expand` | Map of field name → nested QueryAST |
</Callout>

Expand Down
4 changes: 2 additions & 2 deletions content/docs/references/api/contract.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -147,7 +147,7 @@ const result = ApiError.parse(data);
| Property | Type | Required | Description |
| :--- | :--- | :--- | :--- |
| **object** | `string` | ✅ | Object name (e.g. account) |
| **fields** | `string \| { field: string; fields?: object[]; alias?: string }[]` | optional | Fields to retrieve |
| **fields** | `string[]` | optional | Fields to retrieve — field names, optionally dotted to reach through a relationship (`owner.name`). Related *records* are selected with `expand`, not from inside this list. |
| **where** | `any` | optional | Filtering criteria (WHERE) |
| **search** | `{ query: string; fields?: string[]; fuzzy: boolean; operator: Enum<'and' \| 'or'>; … }` | optional | Full-text search configuration ($search parameter) |
| **orderBy** | `{ field: string; order: Enum<'asc' \| 'desc'> }[]` | optional | Sorting instructions (ORDER BY) |
Expand All @@ -161,7 +161,7 @@ const result = ApiError.parse(data);
| **having** | `any` | optional | HAVING clause for aggregation filtering |
| **windowFunctions** | `{ function: Enum<'row_number' \| 'rank' \| 'dense_rank' \| 'percent_rank' \| 'lag' \| 'lead' \| 'first_value' \| 'last_value' \| 'sum' \| 'avg' \| 'count' \| 'min' \| 'max'>; field?: string; alias: string; over: object }[]` | optional | Window functions with OVER clause |
| **distinct** | `boolean` | optional | SELECT DISTINCT flag |
| **expand** | `Record<string, { object: string; fields?: string \| { field: string; fields?: object[]; alias?: string }[]; where?: any; search?: object; … }>` | optional | Recursive relation loading map. Keys are lookup/master_detail field names; values are nested QueryAST objects that control select (`fields`) and filter (`where`, AND-merged with the batch $in), plus further expansion on the related object. The engine resolves expand via batch $in queries (driver-agnostic) with a default max depth of 3; per-parent `limit`/`offset`/`orderBy` are NOT applied on this path. |
| **expand** | `Record<string, { object: string; fields?: string[]; where?: any; search?: object; … }>` | optional | Recursive relation loading map. Keys are lookup/master_detail field names; values are nested QueryAST objects that control select (`fields`) and filter (`where`, AND-merged with the batch $in), plus further expansion on the related object. The engine resolves expand via batch $in queries (driver-agnostic) with a default max depth of 3; per-parent `limit`/`offset`/`orderBy` are NOT applied on this path. |


---
Expand Down
2 changes: 1 addition & 1 deletion content/docs/references/api/protocol.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -529,7 +529,7 @@ const result = AiAgentCapabilities.parse(data);
| Property | Type | Required | Description |
| :--- | :--- | :--- | :--- |
| **object** | `string` | ✅ | The unique machine name of the object to query (e.g. "account"). |
| **query** | `{ object: string; fields?: string \| { field: string; fields?: object[]; alias?: string }[]; where?: any; search?: object; … }` | optional | Structured query definition (filter, sort, select, pagination). |
| **query** | `{ object: string; fields?: string[]; where?: any; search?: object; … }` | optional | Structured query definition (filter, sort, select, pagination). |


---
Expand Down
12 changes: 6 additions & 6 deletions content/docs/references/data/data-engine.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -186,7 +186,7 @@ Reference: any
| :--- | :--- | :--- | :--- |
| **method** | `'findOne'` | ✅ | |
| **object** | `string` | ✅ | |
| **query** | `{ context?: object; where?: Record<string, any> \| any; fields?: string \| { field: string; fields?: object[]; alias?: string }[]; orderBy?: { field: string; order: Enum<'asc' \| 'desc'> }[]; … }` | optional | |
| **query** | `{ context?: object; where?: Record<string, any> \| any; fields?: string[]; orderBy?: { field: string; order: Enum<'asc' \| 'desc'> }[]; … }` | optional | |


---
Expand All @@ -199,7 +199,7 @@ Reference: any
| :--- | :--- | :--- | :--- |
| **method** | `'find'` | ✅ | |
| **object** | `string` | ✅ | |
| **query** | `{ context?: object; where?: Record<string, any> \| any; fields?: string \| { field: string; fields?: object[]; alias?: string }[]; orderBy?: { field: string; order: Enum<'asc' \| 'desc'> }[]; … }` | optional | |
| **query** | `{ context?: object; where?: Record<string, any> \| any; fields?: string[]; orderBy?: { field: string; order: Enum<'asc' \| 'desc'> }[]; … }` | optional | |


---
Expand Down Expand Up @@ -268,7 +268,7 @@ This schema accepts one of the following structures:
| :--- | :--- | :--- | :--- |
| **method** | `'find'` | ✅ | |
| **object** | `string` | ✅ | |
| **query** | `{ context?: object; where?: Record<string, any> \| any; fields?: string \| { field: string; fields?: object[]; alias?: string }[]; orderBy?: { field: string; order: Enum<'asc' \| 'desc'> }[]; … }` | optional | |
| **query** | `{ context?: object; where?: Record<string, any> \| any; fields?: string[]; orderBy?: { field: string; order: Enum<'asc' \| 'desc'> }[]; … }` | optional | |

---

Expand All @@ -280,7 +280,7 @@ This schema accepts one of the following structures:
| :--- | :--- | :--- | :--- |
| **method** | `'findOne'` | ✅ | |
| **object** | `string` | ✅ | |
| **query** | `{ context?: object; where?: Record<string, any> \| any; fields?: string \| { field: string; fields?: object[]; alias?: string }[]; orderBy?: { field: string; order: Enum<'asc' \| 'desc'> }[]; … }` | optional | |
| **query** | `{ context?: object; where?: Record<string, any> \| any; fields?: string[]; orderBy?: { field: string; order: Enum<'asc' \| 'desc'> }[]; … }` | optional | |

---

Expand Down Expand Up @@ -540,14 +540,14 @@ QueryAST-aligned query options for IDataEngine.find() operations
| :--- | :--- | :--- | :--- |
| **context** | `{ userId?: string; actor?: string; email?: string; tenantId?: string; … }` | optional | |
| **where** | `Record<string, any> \| any` | optional | |
| **fields** | `string \| { field: string; fields?: object[]; alias?: string }[]` | optional | |
| **fields** | `string[]` | optional | |
| **orderBy** | `{ field: string; order: Enum<'asc' \| 'desc'> }[]` | optional | |
| **limit** | `number` | optional | |
| **offset** | `number` | optional | |
| **top** | `number` | optional | |
| **cursor** | `Record<string, any>` | optional | |
| **search** | `{ query: string; fields?: string[]; fuzzy: boolean; operator: Enum<'and' \| 'or'>; … }` | optional | |
| **expand** | `Record<string, { object: string; fields?: string \| { field: string; fields?: object[]; alias?: string }[]; where?: any; search?: object; … }>` | optional | |
| **expand** | `Record<string, { object: string; fields?: string[]; where?: any; search?: object; … }>` | optional | |
| **distinct** | `boolean` | optional | |


Expand Down
2 changes: 1 addition & 1 deletion content/docs/references/data/mapping.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ const result = FieldMapping.parse(data);
| **fieldMapping** | `{ source: string \| string[]; target: string \| string[]; transform: Enum<'none' \| 'constant' \| 'lookup' \| 'split' \| 'join' \| 'javascript' \| 'map'>; params?: object }[]` | ✅ | |
| **mode** | `Enum<'insert' \| 'update' \| 'upsert'>` | ✅ | |
| **upsertKey** | `string[]` | optional | Fields to match for upsert (e.g. email) |
| **extractQuery** | `{ object: string; fields?: string \| { field: string; fields?: object[]; alias?: string }[]; where?: any; search?: object; … }` | optional | Query to run for export only |
| **extractQuery** | `{ object: string; fields?: string[]; where?: any; search?: object; … }` | optional | Query to run for export only |
| **errorPolicy** | `Enum<'skip' \| 'abort' \| 'retry'>` | ✅ | |
| **batchSize** | `number` | ✅ | |

Expand Down
28 changes: 2 additions & 26 deletions content/docs/references/data/query.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -67,30 +67,6 @@ const result = AggregationFunction.parse(data);
* `year`


---

## FieldNode

### Union Options

This schema accepts one of the following structures:

#### Option 1

Type: `string`

---

#### Option 2

### Properties

| Property | Type | Required | Description |
| :--- | :--- | :--- | :--- |
| **field** | `string` | ✅ | |
| **fields** | `[FieldNode](#fieldnode)[]` | optional | |
| **alias** | `string` | optional | |

---


Expand Down Expand Up @@ -152,7 +128,7 @@ Type: `string`
| **object** | `string` | ✅ | Object/table to join |
| **alias** | `string` | optional | Table alias |
| **on** | `any` | ✅ | Join condition |
| **subquery** | `{ object: string; fields?: string \| { field: string; fields?: object[]; alias?: string }[]; where?: any; search?: object; … }` | optional | Subquery instead of object |
| **subquery** | `{ object: string; fields?: string[]; where?: any; search?: object; … }` | optional | Subquery instead of object |


---
Expand Down Expand Up @@ -188,7 +164,7 @@ Type: `string`
| Property | Type | Required | Description |
| :--- | :--- | :--- | :--- |
| **object** | `string` | ✅ | Object name (e.g. account) |
| **fields** | `string \| { field: string; fields?: object[]; alias?: string }[]` | optional | Fields to retrieve |
| **fields** | `string[]` | optional | Fields to retrieve — field names, optionally dotted to reach through a relationship (`owner.name`). Related *records* are selected with `expand`, not from inside this list. |
| **where** | `any` | optional | Filtering criteria (WHERE) |
| **search** | `{ query: string; fields?: string[]; fuzzy: boolean; operator: Enum<'and' \| 'or'>; … }` | optional | Full-text search configuration ($search parameter) |
| **orderBy** | `{ field: string; order: Enum<'asc' \| 'desc'> }[]` | optional | Sorting instructions (ORDER BY) |
Expand Down
4 changes: 2 additions & 2 deletions docs/audits/2026-07-unknown-key-strictness-ledger.md
Original file line number Diff line number Diff line change
Expand Up @@ -146,7 +146,7 @@ tightening (the #4001 "sharing-rule lesson": candidates, not verdicts).
| `notification.zod.ts` / `offline.zod.ts` / `report.zod.ts` | 3 ea | authorable (p) | |
| `sharing.zod.ts` | 2 | authorable (p) | public-sharing config |

### `data/` — 154 sites
### `data/` — 153 sites

| File | Sites | Class | Note |
|---|---|---|---|
Expand All @@ -155,7 +155,7 @@ tightening (the #4001 "sharing-rule lesson": candidates, not verdicts).
| `external-lookup.zod.ts` | 12 | mixed (p) | authored config + wire results |
| `seed-loader.zod.ts` | 12 | mixed (p) | seed file shapes are authored; loader state is runtime |
| `field.zod.ts` | 11 | authorable | partially strict |
| `filter.zod.ts` / `query.zod.ts` | 11+10 | open | query dialect — user data flows through; validated semantically elsewhere |
| `filter.zod.ts` / `query.zod.ts` | 11+9 | open | query dialect — user data flows through; validated semantically elsewhere. `query.zod.ts` dropped one site in #4196: `FieldNodeSchema`'s nested-select object form was declared-but-inert and narrowed to `z.string()`, so the union's second member is gone. Class unchanged |
| `driver-nosql.zod.ts` / `driver.zod.ts` / `driver-sql.zod.ts` | 10+9+2 | wire | driver capability contracts |
| `datasource.zod.ts` | 9 | authorable | **strict as of #4001 data step** — all 9: `DatasourceSchema` (+ `pool` / `healthCheck` / `ssl` / `retryPolicy`), `ExternalDatasourceSettingsSchema` (+ `validation`), `DatasourceCapabilities`, `DriverDefinitionSchema`. `config` + `readReplicas` stay `z.record` by construction (per-driver shapes; the driver's own `configSchema` validates them) — which is precisely why the top level had to close: a connection key written one level too high was stripped, and the datasource then connected on driver defaults instead of failing |
| `analytics.zod.ts` | 8 | mixed (p) | |
Expand Down
Loading
Loading