Skip to content

Commit 4086e3e

Browse files
os-zhuangclaude
andcommitted
Merge main into the wait-timeout retirement branch
Resolves the protocol-18 rationale conflict in `migrations/registry.ts`: main added the `QueryAST.fields` paragraph (#4196) and this branch added the `wait` timeout pair (#4158). Both are kept and `conversionIds` is the union. Ordering is load-bearing. Main's paragraph says "Like the two above it is a request shape", which is only true while `requireAuth` and `validateOnly` are the two that precede it — so it stays third and the incoming paragraph goes fourth, renumbered from "The third of the same kind" to "The fourth". Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2 parents 7a245bb + c9c98c1 commit 4086e3e

414 files changed

Lines changed: 20153 additions & 4008 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
---
2+
"@objectstack/objectql": patch
3+
"@objectstack/cli": patch
4+
---
5+
6+
feat(migrate,objectql): the upgrade path names the data migrations that are still open here (#3438, ADR-0104 2026-07-30)
7+
8+
Both value-shape gates fail toward leniency: a deployment that never runs its
9+
migration keeps warning instead of rejecting, and keeps every released file
10+
forever. That default is right — and completely silent, so the gate could sit
11+
open for the life of a deployment without anyone learning that one command ends
12+
it. A gate nobody is told about is served by nobody.
13+
14+
Two announcements, each where an upgrade actually looks:
15+
16+
- **`os migrate meta --from 16`** now ends by naming the data migrations a
17+
chain crossing into 17 leaves behind — `files-to-references`, `value-shapes`
18+
— with what each unlocks, scoped to the field classes the author's own
19+
metadata declares (an app with no media field is never told about the file
20+
migration). `--json` carries the same list as `dataMigrations`. The command
21+
reads no database, so it reports what remains *to do*, never what a given
22+
deployment has *done*.
23+
- **The server logs one line per open gate at boot**, naming the command that
24+
closes it. Only the lax posture announces itself — a verified gate already
25+
logs that it is enforcing, and an app declaring neither class of field costs
26+
nothing and says nothing. This is the half that can speak to a deployment's
27+
actual data, because it is the half with the database.
28+
29+
Nothing about enforcement changes: same gates, same flags, same fail-toward-
30+
leniency default. The advisory runs on `kernel:bootstrapped` rather than
31+
`kernel:ready`, deliberately — the answer depends on the storage service's own
32+
ready handler, which registers `sys_migration` and may attest a store it just
33+
created, and racing it would tell a brand-new deployment its gates are open
34+
moments after they closed.
Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
---
2+
---
3+
4+
docs(adr-0104): record the fresh-datastore attestation seam as implemented — driver-reported created-vs-found table counts, not "sys_migration is missing" (which is the 16→17 trap) — and note that born-attested dogfood now exercises the #3459 collection path on every boot. Releases nothing.
Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
---
2+
"@objectstack/runtime": major
3+
---
4+
5+
feat(runtime)!: action params are enforced by default, and the opt-in is gone (#3438, ADR-0104 D2)
6+
7+
A request bag that violates an action's declared `params[]` — a missing
8+
`required` param, a value outside its `options`, a scalar where `multiple`
9+
declares an array, a non-id where a `reference` declares one, or a key the
10+
action never declared — is now **rejected before the handler runs**:
11+
`400 VALIDATION_FAILED` on REST, a thrown error on MCP. It used to be logged
12+
and passed through.
13+
14+
```diff
15+
- OS_ACTION_PARAMS_STRICT_ENABLED=1 # removed — enforcement is the default
16+
+ OS_ALLOW_LAX_ACTION_PARAMS=1 # escape hatch: warn and pass, as before
17+
```
18+
19+
**What breaks.** Only calls that were *already* wrong. The declaration was a
20+
complete contract that informed nothing but the client dialog, so a bag the
21+
server accepted could still have been silently ignored by the handler — which
22+
is exactly how a correctly-intended `reference: 'sys_user'` degraded into a
23+
paste-a-UUID box (#3405) with a success envelope on top. Those calls now fail
24+
loudly instead of quietly. Actions declaring no `params` are untouched, and the
25+
dispatcher's own `recordId` / `objectName` are allowlisted
26+
(`ACTION_PARAM_BUILTIN_KEYS`), so the keys dispatch itself merges in were never
27+
candidates for the unknown-key error.
28+
29+
**Fixing a rejection** takes one edit at the call site: the message names the
30+
offending param and the declared list. If an integration you cannot reach in
31+
time is affected, set `OS_ALLOW_LAX_ACTION_PARAMS=1` to restore the old
32+
pass-through — the violation still logs once per action, so the drift stays
33+
visible rather than becoming invisible again.
34+
35+
**Why 17.0 rather than a warn window in 17 and the flip in 18.** R3 asked for
36+
warn-then-error, and ADR-0104's 2026-07-30 addendum declined it on the merits
37+
rather than postponing. What a violation strands is a **caller**, not data: the
38+
rejection reaches a developer or an agent who can fix it in one edit, no stored
39+
row is made unwritable, and the escape hatch makes it reversible in a restart.
40+
Deferring that by a major would have charged every deployment a second upgrade
41+
ceremony — 16→17 is already a substantial, tested migration — to postpone a
42+
break that costs one edited call. v17 already carries harsher zero-window
43+
flips (`allowExport` unset now means denied; an undeclared action handler 404s
44+
with no opt-out at all), so holding the milder change to a stricter standard
45+
would have been inconsistent rather than cautious.
46+
47+
For AI and MCP callers specifically — the population D2 was built for — a 400
48+
is corrective feedback consumed in-loop, while a server-side warning is
49+
feedback nobody ever reads.
50+
51+
D1's value-shape half went the opposite way for the opposite reason: it rejects
52+
on the basis of **stored data**, which an author cannot edit their way out of,
53+
so it stays gated per deployment on that deployment's own migration evidence.

.changeset/adr-0104-d2-typed-action-handlers.md

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -25,12 +25,13 @@ read an untyped bag. D2 makes the declaration enforced and typed.
2525
required presence, per-type value shape, and unknown keys (the dispatcher's
2626
own `recordId` / `objectName` are allowlisted).
2727

28-
**Warn-first rollout (ADR-0104 R3).** A violation is **logged and passes** by
29-
default — params that were silently wrong before keep working while the drift
30-
becomes visible. Set `OS_ACTION_PARAMS_STRICT_ENABLED=1` to reject with a
31-
`400 VALIDATION` (REST) / an error (MCP). Actions that declare no `params` are
32-
untouched (nothing to validate against). The flip to strict-by-default rides a
33-
later minor once telemetry is quiet.
28+
**Enforced by default.** A violation is rejected with a `400 VALIDATION_FAILED`
29+
(REST) / an error (MCP) before the handler runs. Actions that declare no
30+
`params` are untouched — there is nothing to validate against. This landed
31+
warn-first behind an `OS_ACTION_PARAMS_STRICT_ENABLED` opt-in during the 17.0
32+
RC line; that variable does **not** exist in 17.0 — see the companion entry
33+
"action params are enforced by default, and the opt-in is gone" for the
34+
reasoning and the `OS_ALLOW_LAX_ACTION_PARAMS` escape hatch.
3435

3536
Not included: file/image params becoming `sys_file` references — that depends
3637
on file-as-reference (ADR-0104 D3). Per-name static typing of `ctx.params` from
Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
---
2+
"@objectstack/service-storage": minor
3+
---
4+
5+
feat(storage): released field files enter collection on deployments that verified their file migration — ADR-0104 D3 wave 2 PR-5b (#3459)
6+
7+
The gated, final step of the file-as-reference sequence. On a deployment whose
8+
`adr-0104-file-references` flag is verified (`os migrate files-to-references
9+
--apply`, #3617), releasing a field file's ownership — clearing the field, or
10+
deleting the owning record — now also tombstones the file
11+
(`status='deleted'` + `deleted_at`), which starts the `sys_file` lifecycle's
12+
declared 30-day grace window and, at its end, hands the row to the reap sweep.
13+
Re-referencing the id inside the window revives it, exactly like re-attaching
14+
an attachment.
15+
16+
**The two halves ship together, deliberately.** The same change extends the
17+
reap guard's sweep-time re-verify beyond `sys_attachment` join rows to the
18+
ownership columns: a tombstoned file whose `ref_*` columns name a current
19+
owner (re-claimed in the window, or a release/claim race) is un-tombstoned and
20+
vetoed. Tombstoning released files without that re-verify would have turned
21+
every release into a *guaranteed* byte delete — the guard's old check consults
22+
a table that is always empty for field files. This pairing was the standing
23+
hard constraint on #3459, locked by regression tests on both halves.
24+
25+
**Nothing changes for a deployment that has not migrated.** Release keeps
26+
clearing the ownership columns only, and released files are retained forever.
27+
Every way of not knowing — no flag row, an unreadable table, an engine that
28+
cannot be asked — reads as "not verified": the gate fails closed, toward
29+
retention. And the guard re-reads the flag *fresh* at sweep time (not the
30+
release path's memoized read), so a later failing migration run — a database
31+
that has drifted — closes the gate for already-written tombstones too, without
32+
a restart. Attachments-scope collection is unchanged and needs no flag.
33+
34+
The irreversible moment is therefore per deployment: day 30 after *that*
35+
deployment verified its migration and released a file — never the upgrade
36+
itself.
Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
---
2+
"@objectstack/objectql": minor
3+
---
4+
5+
feat(objectql): `engine.isFileReferencesMigrationVerified()` is public — one memoized flag read for both in-process consumers (#3459 PR-5b)
6+
7+
The memoized per-deployment read of the `adr-0104-file-references` migration
8+
flag was private to the engine's media value-shape enforcement. The storage
9+
service's release path now asks the same question — may a released field file
10+
be tombstoned? — so the method is public and the release hooks reach it as an
11+
optional duck-typed member (an older engine or a test fake reads as "not
12+
verified", failing closed). One read, one invalidation
13+
(`invalidateDataMigrationFlags()`), no way for the two consumers to see
14+
different answers.
Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
---
2+
"@objectstack/spec": minor
3+
"@objectstack/platform-objects": minor
4+
"@objectstack/objectql": minor
5+
"@objectstack/driver-memory": minor
6+
"@objectstack/driver-sql": minor
7+
"@objectstack/service-storage": minor
8+
---
9+
10+
feat(migrate): a datastore created from empty attests its data migrations at creation (#3438, ADR-0104 2026-07-30 addendum)
11+
12+
Deployment-level migration flags could only be recorded by running
13+
`os migrate`. That left a hole at the other end of a deployment's life: a
14+
database created on a version that already ships the migrations started **lax**
15+
and stayed lax until someone thought to run a command that, for them, converts
16+
nothing and finds nothing. Every new deployment re-entered the warn regime, so
17+
the warn regime would never die out — and, since #3459, every new deployment
18+
also kept every released file forever.
19+
20+
A store the platform **creates from empty** now records
21+
`adr-0104-file-references` and `adr-0104-value-shapes` at that moment. Nothing
22+
to run; enforcement and collection are live from the first boot.
23+
24+
**This is not version-gating in disguise.** The fact recorded — no legacy value
25+
is stored here — is *observed*: the store had no history at all. The platform
26+
attests only what it watched itself create, and the test is deliberately
27+
strict: every table made by this boot and **none found already present**. One
28+
pre-existing table anywhere, one datasource that was already there, one driver
29+
that cannot account for its schema sync — any of those and the deployment
30+
attests nothing and produces its evidence by scan, exactly as before. "Found
31+
empty" and "created empty" are not the same claim, and only the second is an
32+
observation.
33+
34+
**New surfaces.** `IDataDriver.getSchemaSyncStats?()` (optional, purely
35+
observational: tables created vs found since connect — implemented by the SQL
36+
and in-memory drivers), `engine.wasDatastoreCreatedFromEmpty()`,
37+
`attestFreshDatastore()` in `@objectstack/platform-objects/system`, and
38+
`VALUE_SHAPES_MIGRATION_ID` / `CREATION_ATTESTED_MIGRATION_IDS` in
39+
`@objectstack/spec/system`. Attestation never overwrites an existing flag row
40+
and never throws into a boot: a failure leaves the deployment lax, which a
41+
migration run can still fix.
42+
43+
**Upgrading changes nothing for an existing database.** It is non-empty when
44+
the platform reaches it, so it is never attested — run
45+
`os migrate files-to-references --apply` as before. Importing legacy values
46+
into an attested deployment is rejected loudly at the write path;
47+
`OS_ALLOW_LAX_MEDIA_VALUES=1` re-opens leniency while you diagnose.
Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
---
2+
"@objectstack/objectql": patch
3+
---
4+
5+
fix(objectql): the boot gate announcement stops firing where it is false, and stops counting fields nothing enforces (#3438)
6+
7+
The startup line that names an open value-shape gate (#4253) fired on every
8+
deployment there is, and said something untrue on any deployment that had
9+
already settled the question with an environment switch. Both are the same
10+
failure — an advisory that speaks where it does not apply is how readers learn
11+
to ignore it — and neither is reachable from the suite that shipped with it,
12+
because `engine.test.ts` mocks the registry away and a mocked registry hands
13+
the engine exactly the fields the test wrote.
14+
15+
`objectHasCoveredValueField` — the dormancy short-circuit that is supposed to
16+
spare an object with no covered field the flag query — tested raw type
17+
membership, while the real registry INJECTS covered-type fields into every
18+
object it registers: `organization_id` and `owner_id` (both `system`),
19+
`created_by` and `updated_by` (both in `SKIP_FIELDS`), four `lookup`s.
20+
`validateRecord` skips every one of them before it reaches the value-shape
21+
check, so the short-circuit answered `true` for literally every object, never
22+
fired, and its WeakMap memoized a constant. Counting is now by the validator's
23+
own `isScannableValueShapeField`, the predicate the scanner already imports —
24+
three readings of "a covered field" drifting by one clause is how a gate ends
25+
up governing fields nothing enforces.
26+
27+
The announcement also consulted no environment switch, while both postures it
28+
reports on short-circuit ahead of the deployment flag. Under
29+
`OS_DATA_VALUE_SHAPE_STRICT_ENABLED` enforcement is already on for both
30+
classes, so "checked but NOT enforced here" was simply false; under either
31+
opt-out the operator chose leniency deliberately, so naming a migration that
32+
cannot change what they get is noise. Each gate now consults its own pair
33+
(`mediaPostureSetByEnv` / `valueShapePostureSetByEnv`, siblings for the reason
34+
`mediaStrictEffective` and `valueShapeStrictEffective` are siblings), since the
35+
opt-outs are per-class while the opt-in opens both. Cheapest test first, so a
36+
kernel with nothing to say still reaches no flag query.
37+
38+
Enforcement is unchanged: same gates, same flags, same default. The flag read
39+
was already memoized per process, so what this corrects is the property
40+
ADR-0104 states, not a user-visible cost.
Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
11
---
22
---
33

4-
docs(adr): ADR-0104 addendum — evidence for the remaining #3438 strict flips (D1 references/structured JSON gate on a per-deployment `os migrate value-shapes` scan; fresh datastores attest both flags at creation; the D2 action-param flip rides the 18.0 major with an escape hatch) — releases nothing.
4+
docs(adr): ADR-0104 addendum — evidence for the remaining #3438 strict flips (D1 references/structured JSON gate on a per-deployment `os migrate value-shapes` scan, advertised on the 16→17 upgrade path; fresh datastores attest both flags at creation; D2 action params flip strict-by-default in 17.0, `OS_ACTION_PARAMS_STRICT_ENABLED` deleted before it reaches `latest`, `OS_ALLOW_LAX_ACTION_PARAMS` as the escape hatch) — releases nothing.
Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
---
2+
---
3+
4+
docs(adr): revise the ADR-0104 2026-07-30 addendum — D2 action params flip strict-by-default in 17.0 (opt-in env var deleted before it reaches `latest`, `OS_ALLOW_LAX_ACTION_PARAMS` escape hatch); D1 non-media keeps the per-deployment scan gate, now advertised on the 16→17 upgrade path — releases nothing.

0 commit comments

Comments
 (0)