Skip to content

Commit e70b273

Browse files
committed
Merge remote-tracking branch 'origin/main' into claude/sys-secret-producer-conflict-157867
2 parents 7faf5ae + 8675db6 commit e70b273

25 files changed

Lines changed: 704 additions & 122 deletions

File tree

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
---
2+
"@objectstack/objectql": patch
3+
"@objectstack/cli": patch
4+
---
5+
6+
feat(migrate,objectql): the upgrade path names the data migrations that are still open here (#3438, ADR-0104 2026-07-30)
7+
8+
Both value-shape gates fail toward leniency: a deployment that never runs its
9+
migration keeps warning instead of rejecting, and keeps every released file
10+
forever. That default is right — and completely silent, so the gate could sit
11+
open for the life of a deployment without anyone learning that one command ends
12+
it. A gate nobody is told about is served by nobody.
13+
14+
Two announcements, each where an upgrade actually looks:
15+
16+
- **`os migrate meta --from 16`** now ends by naming the data migrations a
17+
chain crossing into 17 leaves behind — `files-to-references`, `value-shapes`
18+
— with what each unlocks, scoped to the field classes the author's own
19+
metadata declares (an app with no media field is never told about the file
20+
migration). `--json` carries the same list as `dataMigrations`. The command
21+
reads no database, so it reports what remains *to do*, never what a given
22+
deployment has *done*.
23+
- **The server logs one line per open gate at boot**, naming the command that
24+
closes it. Only the lax posture announces itself — a verified gate already
25+
logs that it is enforcing, and an app declaring neither class of field costs
26+
nothing and says nothing. This is the half that can speak to a deployment's
27+
actual data, because it is the half with the database.
28+
29+
Nothing about enforcement changes: same gates, same flags, same fail-toward-
30+
leniency default. The advisory runs on `kernel:bootstrapped` rather than
31+
`kernel:ready`, deliberately — the answer depends on the storage service's own
32+
ready handler, which registers `sys_migration` and may attest a store it just
33+
created, and racing it would tell a brand-new deployment its gates are open
34+
moments after they closed.
Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
---
2+
"@objectstack/spec": major
3+
"@objectstack/objectql": patch
4+
"@objectstack/metadata-protocol": patch
5+
"@objectstack/driver-mongodb": patch
6+
"@objectstack/driver-sql": patch
7+
"@objectstack/driver-memory": patch
8+
---
9+
10+
refactor(data)!: a select-list entry is a field name — the nested-select object form is removed (#4196)
11+
12+
`FieldNode` declared two forms for one entry of `QueryAST['fields']`:
13+
14+
```ts
15+
type FieldNode =
16+
| string // "name"
17+
| { field: string; fields?: FieldNode[]; alias?: string }; // nested select
18+
```
19+
20+
The object form was **declared-but-inert**. Nothing produced it, and nothing
21+
read `.fields` or `.alias` — every consumer on the path treats the list as
22+
`string[]`: `objectql`'s formula projection and its two known-field filters,
23+
`driver-sql`'s `select()`, `driver-memory`'s `projectFields`. `driver-mongodb`
24+
keyed its projection with the entry itself, so an object entry asked for a
25+
column literally named `"[object Object]"`, and the REST ingress stringified
26+
each entry before comparing it to the field map, so the same entry came back as
27+
`400 INVALID_FIELD: Unknown field '[object Object]'` — a rejection naming
28+
something the caller never wrote. An author who wrote
29+
`fields: [{ field: 'owner', fields: ['name'] }]` got it accepted by validation
30+
and then dropped or mangled, depending on the driver (ADR-0078 silently-inert
31+
declaration; ADR-0049 enforce-or-remove).
32+
33+
The capability the object form described is already served, by a different key.
34+
Removing the second spelling rather than lowering it into the first is Prime
35+
Directive #12: one capability, one contract.
36+
37+
**FROM → TO**
38+
39+
| Was | Now |
40+
| :--- | :--- |
41+
| `fields: [{ field: 'owner', fields: ['name'] }]` | `expand: { owner: { object: 'user', fields: ['name'] } }` |
42+
| `fields: [{ field: 'owner' }]` | `fields: ['owner']` |
43+
| `fields: [{ field: 'owner', fields: ['name'] }]`, one column only | `fields: ['owner.name']` (dotted path) |
44+
| `fields: [{ field: 'total', alias: 't' }]` | `aggregations` / `windowFunctions` — they carry the live `alias` |
45+
46+
The one-line fix: **a `fields[]` entry is a string.** Move nested selection to
47+
`expand`, which the engine resolves through batch `$in` queries (default max
48+
depth 3).
49+
50+
There is no `os migrate meta` step, and deliberately so: `QueryAST` is a request
51+
shape, never stored in stack metadata, so the chain has no source to rewrite. It
52+
is registered as an ADR-0087 D3 **semantic** migration
53+
(`query-field-node-object-form-retired`) on the protocol-18 step instead — the
54+
`EnhancedApiError.fieldErrors` / `BatchOptions.validateOnly` precedent. Callers
55+
move their own select lists, and both channels tell them how:
56+
57+
- **The parse.** `FieldNodeSchema` narrows to `z.string()` with an error map that
58+
answers an object entry with the prescription above, not "expected string,
59+
received object". `z.input` becomes `string`, so `tsc` fails at the authoring
60+
site first.
61+
- **The ingress.** `assertProjectionFieldsExist` judges the entry's *shape*
62+
before consulting the object's field map — it is wrong about the shape, not
63+
about this object, and a registry-less host would otherwise pass it to a driver
64+
that cannot read it. The 400 now names the retired form instead of the field
65+
`"[object Object]"`.
66+
67+
No runtime behaviour changes for anything that ever worked; the defensive
68+
unwrapping the drivers had grown against a shape nothing sends goes with it.

content/docs/deployment/cli.mdx

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -745,6 +745,22 @@ them temporarily, `OS_ALLOW_LAX_MEDIA_VALUES=1` (media) and
745745
per class, and re-running the corresponding migration re-establishes its flag
746746
from the data itself.
747747

748+
#### How you find out a gate is open
749+
750+
You are told, in the two places an upgrade actually looks.
751+
752+
**`os migrate meta --from 16`** — the metadata half of the upgrade — ends by
753+
naming the data migrations that remain, scoped to the field classes *your*
754+
metadata declares. It reads no database, so it reports what is left to do,
755+
never what this deployment has already done. The machine-readable output
756+
carries the same list under `dataMigrations`.
757+
758+
**The server, once per boot**, logs one line per gate that is still open here
759+
and the command that closes it. Only the lax posture announces itself: a
760+
closed gate logs that it is enforcing, and an app that declares neither class
761+
of field says nothing at all. So a running deployment always tells you the
762+
state of its own data — which is the question `os migrate meta` cannot answer.
763+
748764
### Scaffolding
749765

750766
| Command | Alias | Description |

content/docs/kernel/contracts/data-engine.mdx

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -85,7 +85,7 @@ Defined by `EngineQueryOptionsSchema` in `@objectstack/spec`:
8585
```typescript
8686
interface EngineQueryOptions {
8787
where?: FilterCondition; // WHERE clause — MongoDB-style $op
88-
fields?: FieldNode[]; // SELECT fields
88+
fields?: FieldNode[]; // SELECT — field names ('name', 'owner.name')
8989
orderBy?: SortNode[]; // ORDER BY
9090
limit?: number; // LIMIT
9191
offset?: number; // OFFSET
@@ -387,7 +387,7 @@ The following legacy parameter names are accepted by the RPC layer for backward
387387
| Legacy (Deprecated) | Canonical | Notes |
388388
|:---------------------|:----------|:------|
389389
| `filter` | `where` | FilterCondition object |
390-
| `select` | `fields` | Array of FieldNode |
390+
| `select` | `fields` | Array of FieldNode (field-name strings) |
391391
| `sort` | `orderBy` | Array of `{ field, order }` |
392392
| `skip` | `offset` | Number |
393393
| `populate` | `expand` | Record of field → QueryAST |

content/docs/protocol/objectql/query-syntax.mdx

Lines changed: 12 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -62,7 +62,7 @@ import type { QueryAST } from '@objectstack/spec/data';
6262
// QueryAST — full structure
6363
interface QueryAST {
6464
object: string; // Target object (required)
65-
fields?: FieldNode[]; // Projection (SELECT)
65+
fields?: FieldNode[]; // Projection (SELECT) — field names
6666
where?: FilterCondition; // Filtering (WHERE) — MongoDB-style $op
6767
search?: FullTextSearch; // Full-text search ($search)
6868
orderBy?: SortNode[]; // Ordering (ORDER BY)
@@ -97,7 +97,6 @@ on the `find()` path:
9797
| `cursor` | Accepted by `EngineQueryOptions`, but no driver implements keyset pagination |
9898
| `distinct` | Not applied by `find()`; the SQL and in-memory drivers expose a separate `distinct(object, field, filters?)` method instead |
9999
| `windowFunctions` | Not applied by `find()`; `SqlDriver.findWithWindowFunctions()` is a separate method that is not on the `IDataDriver` contract |
100-
| Object `FieldNode`s (`{ field, fields }`) | The engine drops non-string entries from `fields` — use `expand` to project related records |
101100
| `search.fuzzy` / `boost` / `operator` / `minScore` / `language` / `highlight` | Parsed by the schema but ignored: only `query` and `fields` drive the expansion |
102101

103102
`top` is the exception that *is* honored — the engine normalises it to `limit`.
@@ -122,12 +121,16 @@ interface AggregationNode {
122121
filter?: FilterCondition; // FILTER WHERE clause — protocol only, never applied
123122
}
124123

125-
// FieldNode — field selection
126-
type FieldNode = string | {
127-
field: string;
128-
fields?: FieldNode[]; // nested select — protocol only, dropped by the engine
129-
alias?: string;
130-
};
124+
// FieldNode — one entry of the select list. A field name, optionally dotted to
125+
// reach through a relationship ('owner.name'). Related *records* come from
126+
// `expand`, not from inside this list.
127+
//
128+
// The `{ field, fields, alias }` nested-select member this union used to carry
129+
// was REMOVED in protocol 18 (#4196): nothing produced it and nothing read
130+
// `.fields`/`.alias`, so it was dropped by the SQL and memory drivers,
131+
// projected as a column named "[object Object]" by MongoDB, and refused as an
132+
// unknown field by the REST ingress. `expand` is the one spelling.
133+
type FieldNode = string;
131134

132135
// GroupByNode — GROUP BY target
133136
type GroupByNode = string | {
@@ -980,7 +983,7 @@ Similarly, the following legacy field names should not be used in new code:
980983
| `aggregate` (object map) | `aggregations` (array) | Array of `AggregationNode` objects |
981984
| `expand` (string array) | `expand` (Record) | Map of field name → nested QueryAST |
982985
| `skip` | `offset` | Number |
983-
| `select` | `fields` | Array of FieldNode |
986+
| `select` | `fields` | Array of FieldNode (field-name strings) |
984987
| `populate` | `expand` | Map of field name → nested QueryAST |
985988
</Callout>
986989

content/docs/references/api/contract.mdx

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -147,7 +147,7 @@ const result = ApiError.parse(data);
147147
| Property | Type | Required | Description |
148148
| :--- | :--- | :--- | :--- |
149149
| **object** | `string` || Object name (e.g. account) |
150-
| **fields** | `string \| { field: string; fields?: object[]; alias?: string }[]` | optional | Fields to retrieve |
150+
| **fields** | `string[]` | optional | Fields to retrieve — field names, optionally dotted to reach through a relationship (`owner.name`). Related *records* are selected with `expand`, not from inside this list. |
151151
| **where** | `any` | optional | Filtering criteria (WHERE) |
152152
| **search** | `{ query: string; fields?: string[]; fuzzy: boolean; operator: Enum<'and' \| 'or'>; … }` | optional | Full-text search configuration ($search parameter) |
153153
| **orderBy** | `{ field: string; order: Enum<'asc' \| 'desc'> }[]` | optional | Sorting instructions (ORDER BY) |
@@ -161,7 +161,7 @@ const result = ApiError.parse(data);
161161
| **having** | `any` | optional | HAVING clause for aggregation filtering |
162162
| **windowFunctions** | `{ function: Enum<'row_number' \| 'rank' \| 'dense_rank' \| 'percent_rank' \| 'lag' \| 'lead' \| 'first_value' \| 'last_value' \| 'sum' \| 'avg' \| 'count' \| 'min' \| 'max'>; field?: string; alias: string; over: object }[]` | optional | Window functions with OVER clause |
163163
| **distinct** | `boolean` | optional | SELECT DISTINCT flag |
164-
| **expand** | `Record<string, { object: string; fields?: string \| { field: string; fields?: object[]; alias?: string }[]; where?: any; search?: object; … }>` | optional | Recursive relation loading map. Keys are lookup/master_detail field names; values are nested QueryAST objects that control select (`fields`) and filter (`where`, AND-merged with the batch $in), plus further expansion on the related object. The engine resolves expand via batch $in queries (driver-agnostic) with a default max depth of 3; per-parent `limit`/`offset`/`orderBy` are NOT applied on this path. |
164+
| **expand** | `Record<string, { object: string; fields?: string[]; where?: any; search?: object; … }>` | optional | Recursive relation loading map. Keys are lookup/master_detail field names; values are nested QueryAST objects that control select (`fields`) and filter (`where`, AND-merged with the batch $in), plus further expansion on the related object. The engine resolves expand via batch $in queries (driver-agnostic) with a default max depth of 3; per-parent `limit`/`offset`/`orderBy` are NOT applied on this path. |
165165

166166

167167
---

content/docs/references/api/protocol.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -529,7 +529,7 @@ const result = AiAgentCapabilities.parse(data);
529529
| Property | Type | Required | Description |
530530
| :--- | :--- | :--- | :--- |
531531
| **object** | `string` || The unique machine name of the object to query (e.g. "account"). |
532-
| **query** | `{ object: string; fields?: string \| { field: string; fields?: object[]; alias?: string }[]; where?: any; search?: object; … }` | optional | Structured query definition (filter, sort, select, pagination). |
532+
| **query** | `{ object: string; fields?: string[]; where?: any; search?: object; … }` | optional | Structured query definition (filter, sort, select, pagination). |
533533

534534

535535
---

content/docs/references/data/data-engine.mdx

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -186,7 +186,7 @@ Reference: any
186186
| :--- | :--- | :--- | :--- |
187187
| **method** | `'findOne'` || |
188188
| **object** | `string` || |
189-
| **query** | `{ context?: object; where?: Record<string, any> \| any; fields?: string \| { field: string; fields?: object[]; alias?: string }[]; orderBy?: { field: string; order: Enum<'asc' \| 'desc'> }[]; … }` | optional | |
189+
| **query** | `{ context?: object; where?: Record<string, any> \| any; fields?: string[]; orderBy?: { field: string; order: Enum<'asc' \| 'desc'> }[]; … }` | optional | |
190190

191191

192192
---
@@ -199,7 +199,7 @@ Reference: any
199199
| :--- | :--- | :--- | :--- |
200200
| **method** | `'find'` || |
201201
| **object** | `string` || |
202-
| **query** | `{ context?: object; where?: Record<string, any> \| any; fields?: string \| { field: string; fields?: object[]; alias?: string }[]; orderBy?: { field: string; order: Enum<'asc' \| 'desc'> }[]; … }` | optional | |
202+
| **query** | `{ context?: object; where?: Record<string, any> \| any; fields?: string[]; orderBy?: { field: string; order: Enum<'asc' \| 'desc'> }[]; … }` | optional | |
203203

204204

205205
---
@@ -268,7 +268,7 @@ This schema accepts one of the following structures:
268268
| :--- | :--- | :--- | :--- |
269269
| **method** | `'find'` || |
270270
| **object** | `string` || |
271-
| **query** | `{ context?: object; where?: Record<string, any> \| any; fields?: string \| { field: string; fields?: object[]; alias?: string }[]; orderBy?: { field: string; order: Enum<'asc' \| 'desc'> }[]; … }` | optional | |
271+
| **query** | `{ context?: object; where?: Record<string, any> \| any; fields?: string[]; orderBy?: { field: string; order: Enum<'asc' \| 'desc'> }[]; … }` | optional | |
272272

273273
---
274274

@@ -280,7 +280,7 @@ This schema accepts one of the following structures:
280280
| :--- | :--- | :--- | :--- |
281281
| **method** | `'findOne'` || |
282282
| **object** | `string` || |
283-
| **query** | `{ context?: object; where?: Record<string, any> \| any; fields?: string \| { field: string; fields?: object[]; alias?: string }[]; orderBy?: { field: string; order: Enum<'asc' \| 'desc'> }[]; … }` | optional | |
283+
| **query** | `{ context?: object; where?: Record<string, any> \| any; fields?: string[]; orderBy?: { field: string; order: Enum<'asc' \| 'desc'> }[]; … }` | optional | |
284284

285285
---
286286

@@ -540,14 +540,14 @@ QueryAST-aligned query options for IDataEngine.find() operations
540540
| :--- | :--- | :--- | :--- |
541541
| **context** | `{ userId?: string; actor?: string; email?: string; tenantId?: string; … }` | optional | |
542542
| **where** | `Record<string, any> \| any` | optional | |
543-
| **fields** | `string \| { field: string; fields?: object[]; alias?: string }[]` | optional | |
543+
| **fields** | `string[]` | optional | |
544544
| **orderBy** | `{ field: string; order: Enum<'asc' \| 'desc'> }[]` | optional | |
545545
| **limit** | `number` | optional | |
546546
| **offset** | `number` | optional | |
547547
| **top** | `number` | optional | |
548548
| **cursor** | `Record<string, any>` | optional | |
549549
| **search** | `{ query: string; fields?: string[]; fuzzy: boolean; operator: Enum<'and' \| 'or'>; … }` | optional | |
550-
| **expand** | `Record<string, { object: string; fields?: string \| { field: string; fields?: object[]; alias?: string }[]; where?: any; search?: object; … }>` | optional | |
550+
| **expand** | `Record<string, { object: string; fields?: string[]; where?: any; search?: object; … }>` | optional | |
551551
| **distinct** | `boolean` | optional | |
552552

553553

content/docs/references/data/mapping.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,7 @@ const result = FieldMapping.parse(data);
5252
| **fieldMapping** | `{ source: string \| string[]; target: string \| string[]; transform: Enum<'none' \| 'constant' \| 'lookup' \| 'split' \| 'join' \| 'javascript' \| 'map'>; params?: object }[]` || |
5353
| **mode** | `Enum<'insert' \| 'update' \| 'upsert'>` || |
5454
| **upsertKey** | `string[]` | optional | Fields to match for upsert (e.g. email) |
55-
| **extractQuery** | `{ object: string; fields?: string \| { field: string; fields?: object[]; alias?: string }[]; where?: any; search?: object; … }` | optional | Query to run for export only |
55+
| **extractQuery** | `{ object: string; fields?: string[]; where?: any; search?: object; … }` | optional | Query to run for export only |
5656
| **errorPolicy** | `Enum<'skip' \| 'abort' \| 'retry'>` || |
5757
| **batchSize** | `number` || |
5858

content/docs/references/data/query.mdx

Lines changed: 2 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -67,30 +67,6 @@ const result = AggregationFunction.parse(data);
6767
* `year`
6868

6969

70-
---
71-
72-
## FieldNode
73-
74-
### Union Options
75-
76-
This schema accepts one of the following structures:
77-
78-
#### Option 1
79-
80-
Type: `string`
81-
82-
---
83-
84-
#### Option 2
85-
86-
### Properties
87-
88-
| Property | Type | Required | Description |
89-
| :--- | :--- | :--- | :--- |
90-
| **field** | `string` || |
91-
| **fields** | `[FieldNode](#fieldnode)[]` | optional | |
92-
| **alias** | `string` | optional | |
93-
9470
---
9571

9672

@@ -152,7 +128,7 @@ Type: `string`
152128
| **object** | `string` || Object/table to join |
153129
| **alias** | `string` | optional | Table alias |
154130
| **on** | `any` || Join condition |
155-
| **subquery** | `{ object: string; fields?: string \| { field: string; fields?: object[]; alias?: string }[]; where?: any; search?: object; … }` | optional | Subquery instead of object |
131+
| **subquery** | `{ object: string; fields?: string[]; where?: any; search?: object; … }` | optional | Subquery instead of object |
156132

157133

158134
---
@@ -188,7 +164,7 @@ Type: `string`
188164
| Property | Type | Required | Description |
189165
| :--- | :--- | :--- | :--- |
190166
| **object** | `string` || Object name (e.g. account) |
191-
| **fields** | `string \| { field: string; fields?: object[]; alias?: string }[]` | optional | Fields to retrieve |
167+
| **fields** | `string[]` | optional | Fields to retrieve — field names, optionally dotted to reach through a relationship (`owner.name`). Related *records* are selected with `expand`, not from inside this list. |
192168
| **where** | `any` | optional | Filtering criteria (WHERE) |
193169
| **search** | `{ query: string; fields?: string[]; fuzzy: boolean; operator: Enum<'and' \| 'or'>; … }` | optional | Full-text search configuration ($search parameter) |
194170
| **orderBy** | `{ field: string; order: Enum<'asc' \| 'desc'> }[]` | optional | Sorting instructions (ORDER BY) |

0 commit comments

Comments
 (0)