fix(spec): automation/etl.zod.ts 的九个别名回到 X / XParsed house convention (#4963)

.changeset/etl-author-shape-aliases.md

@@ -0,0 +1,52 @@
+---
+"@objectstack/spec": major
+---
+
+**`automation/etl.zod.ts`'s nine type aliases now follow the house `X` / `XParsed`
+convention** (#4963): the bare name is `z.input` — what an author writes — and a new
+`XParsed` is `z.infer` — what a parse returns.
+
+Until 17 all nine were `z.infer` under the bare name with no `*Parsed` counterpart at
+all. On this file that was not a style detail. Five named keys carry `.default()`
+(`ETLDestination.writeMode`, `ETLTransformation.continueOnError`, `ETLPipeline.syncMode`
+/ `.enabled`, `ETLSource.incremental.enabled`), as does every key of
+`ETLPipeline.retry` and of `ETLPipelineRun.stats`, and `schedule` is a
+`CronExpressionInputSchema` transform whose *output* is the `{ dialect, source }`
+envelope. Under `z.infer` all of them were REQUIRED and a bare cron string was rejected — so the single use this file has, `const p: ETLPipeline = { … }`
+written by hand, did not compile. That is the whole authoring door: `etl.zod.ts` has no
+parse site in objectstack / objectui / cloud, so the exported schema and the exported
+type are the only surface an author touches.
+
+The evidence was checked in. `packages/spec/docs/SYNC_ARCHITECTURE.md` carried three
+`ETLPipeline` examples, none of which compiled, and both `ETL` factories were spelling
+out defaults and pre-wrapping their cron purely to satisfy their own return type.
+
+### Migration
+
+**Zero importers across objectstack, objectui and cloud** (re-measured against each
+repo's `origin/main` for this change), so the migration surface is empty. If you have a
+local consumer:
+
+| You wrote | Keep it if | Change it to |
+|:---|:---|:---|
+| `const p: ETLPipeline = { … }` | you are AUTHORING a pipeline literal | nothing — this is the case that now compiles |
+| `const p: ETLPipeline = ETLPipelineSchema.parse(raw)` | — | `const p: ETLPipelineParsed = …` |
+| `function run(p: ETLPipeline)` reading `p.syncMode` as always-present | — | `ETLPipelineParsed` |
+
+The same rename applies to each of the nine: `ETLEndpointType`, `ETLSource`,
+`ETLDestination`, `ETLTransformationType`, `ETLTransformation`, `ETLSyncMode`,
+`ETLPipeline`, `ETLRunStatus`, `ETLPipelineRun` — append `Parsed` wherever the annotated
+value came out of a `.parse()`. The four enum aliases are unaffected in practice
+(`z.input` and `z.infer` coincide for an enum); their pair exists so a reader never has
+to know which of the nine has defaults before choosing an annotation.
+
+Nothing at runtime moves: no schema, default, bound or key changed, and both `ETL`
+factories still produce documents that parse to the same result. `ETL.databaseSync` /
+`ETL.apiToDatabase` no longer restate `enabled: true` and no longer pre-wrap a bare cron
+string into `{ dialect: 'cron', source }` — the schema does that at parse, which is where
+it always belonged. Each helper still states what it DECIDES (`incremental` + `upsert`
+vs `full` + `append`), because that contrast is the reason the pair exists.
+
+`SYNC_ARCHITECTURE.md`'s three examples now compile, and a compiler-API test
+(`etl-author-shape.test.ts`) compiles them verbatim on every run — import line included —
+so they cannot rot again silently.

docs/audits/2026-07-unknown-key-strictness-ledger.md

@@ -691,7 +691,7 @@ column does not move and the `strip` column falls by the count of what left.
 | File | Class | Note |
 |---|---|---|
 | `flow.zod.ts` | authorable | **strict as of #4001** — the four outer authoring shapes at step 1, and **the six nested blocks at batch 11** (`FlowNode.connectorConfig` / `.position` / `.inputSchema` / `.waitEventConfig` / `.boundaryConfig`, `Flow.errorHandling`). The gap between those two dates is this campaign's own finding 17 inside its own file: closing the shells left the gate rejecting `nodee:` at node level while `connectorConfig: { connectorId, actionId, params: {…} }` parsed clean and the executor dispatched `input ?? {}` — a successful connector call carrying nothing. Worth recording precisely, because the obvious example is the wrong one: a slip on a REQUIRED key was always loud (it then reads as missing). What `.strip` swallowed here is the OPTIONAL half — the input map, the retry budget, `interrupting: false`, `required: true` — i.e. exactly the keys an author adds to CONSTRAIN behaviour, replaced by a permissive default without a word. `Flow.errorHandling` gained a second chapter at **#4964**: closing it in 批 11 revealed (rather than caused) that its retry keys were a THIRD encoding of the policy #4661 had converged — it spelled the base delay `retryDelayMs` where the shared declaration spells it `backoffMs` and tombstones the old word, so the strictness this row records was, for one release, rejecting an author for having read the newer file. The block now builds from `retryPolicyShape()`. Site count unchanged; only the vocabulary. Two things stay open and are now pinned in code with the reason, so a later sweep stops rather than "finishes" the file: the node `config` slot (ADR-0018 plugin namespace) and `FlowVersionHistorySchema` (the file's only WIRE shape — emitted on publish, never authored; its `definition` is `FlowSchema`, so the authored half inside a history record is gated anyway) |
-| `etl.zod.ts` | mixed | **7 strict as of #4001 批 12** — the authoring half (`ETLSource` + `.incremental`, `ETLDestination`, `ETLTransformation`, `ETLPipeline` + `.retry` + `.notifications`). The other 3 — `ETLPipelineRun` + `.stats` + `.error` — are **deliberately left open**: engine-emitted run state (an id it minted, a status it reached, counters it accumulated), same disposition and same reason as `FlowVersionHistorySchema` above and all of `execution.zod.ts`. The exemption is recorded on the schema itself, not only here, because a note only this file carries is a note the next sweep does not read. The old blanket `authorable (p)` was too wide; verification split it. ⚠️ **Read the classification caveat before reusing this verdict**: `etl.zod.ts` has NO parse site in objectstack / objectui / cloud, so neither half could be settled by pointing at a live call. The 7 are authorable because the exported schema and type ARE the door (`SYNC_ARCHITECTURE.md` and the module's `@example` both hand-write `const p: ETLPipeline = { … }`) — the `webhook.zod.ts` posture. The 3 are wire on the shape's semantics plus settled precedent, NOT on an emit site anyone can point at today; if an ETL engine ever lands and a run result turns out to be operator-authored, that verdict is the one to revisit. Two out-of-scope findings were filed rather than fixed here; **the first is now closed**: the `retry` block was a third retry-policy vocabulary #4661's convergence never reached (#4962 — converged onto `shared/RetryPolicySchema` in the v17 window, together with `flow.errorHandling` (#4964), the fourth. Both were anonymous inline blocks, so the dual-source instrument that drove #4661 could not see them: it asks how many declarations share an exported NAME, and neither has one. 批 12's five curated `retry` entries described that divergence and dissolved with it — the block's site count is unchanged, only its vocabulary). Still open: all nine type aliases export the parsed shape under the bare name, which is why the SYNC_ARCHITECTURE.md pipeline examples do not compile (#4963). **−12 at #4738**: `sync.zod.ts` (the L1 "Simple Sync" file — `DataSyncConfig`, its `ConflictResolution` enum and satellites, formerly this row's co-candidate) was deleted whole rather than hardened: three-repo zero importers, no parse site, defs unreachable from the metadata-type roots (#4650 gate), so there was no author for strictness to protect (#4535 C13+C15). The integration-side `ConflictResolution` → `ConnectorConflictResolution` rename in the same change is name-only and moves no sites |
+| `etl.zod.ts` | mixed | **7 strict as of #4001 批 12** — the authoring half (`ETLSource` + `.incremental`, `ETLDestination`, `ETLTransformation`, `ETLPipeline` + `.retry` + `.notifications`). The other 3 — `ETLPipelineRun` + `.stats` + `.error` — are **deliberately left open**: engine-emitted run state (an id it minted, a status it reached, counters it accumulated), same disposition and same reason as `FlowVersionHistorySchema` above and all of `execution.zod.ts`. The exemption is recorded on the schema itself, not only here, because a note only this file carries is a note the next sweep does not read. The old blanket `authorable (p)` was too wide; verification split it. ⚠️ **Read the classification caveat before reusing this verdict**: `etl.zod.ts` has NO parse site in objectstack / objectui / cloud, so neither half could be settled by pointing at a live call. The 7 are authorable because the exported schema and type ARE the door (`SYNC_ARCHITECTURE.md` and the module's `@example` both hand-write `const p: ETLPipeline = { … }`) — the `webhook.zod.ts` posture. The 3 are wire on the shape's semantics plus settled precedent, NOT on an emit site anyone can point at today; if an ETL engine ever lands and a run result turns out to be operator-authored, that verdict is the one to revisit. Two out-of-scope findings were filed rather than fixed here; **the first is now closed**: the `retry` block was a third retry-policy vocabulary #4661's convergence never reached (#4962 — converged onto `shared/RetryPolicySchema` in the v17 window, together with `flow.errorHandling` (#4964), the fourth. Both were anonymous inline blocks, so the dual-source instrument that drove #4661 could not see them: it asks how many declarations share an exported NAME, and neither has one. 批 12's five curated `retry` entries described that divergence and dissolved with it — the block's site count is unchanged, only its vocabulary). **The second is now closed too**: all nine type aliases exported the PARSED shape under the bare name with no `*Parsed` counterpart, so the authoring door this row's whole classification rests on — `const p: ETLPipeline = { … }` — did not actually compile, and the SYNC_ARCHITECTURE.md examples proving it were the evidence (#4963 — bare names flipped to `z.input`, nine `*Parsed` added, house convention per `shared/retry-policy.zod.ts`; three-repo zero importers made the migration surface empty). Worth carrying forward for the next classification: **"authorable because the exported type IS the door" is a claim about a type that must be checked, not assumed** — 批 12 read the door correctly and nobody compiled it. **−12 at #4738**: `sync.zod.ts` (the L1 "Simple Sync" file — `DataSyncConfig`, its `ConflictResolution` enum and satellites, formerly this row's co-candidate) was deleted whole rather than hardened: three-repo zero importers, no parse site, defs unreachable from the metadata-type roots (#4650 gate), so there was no author for strictness to protect (#4535 C13+C15). The integration-side `ConflictResolution` → `ConnectorConflictResolution` rename in the same change is name-only and moves no sites |
 | `execution.zod.ts` | wire | run-state envelopes — never strict. +5 at #4354 (the run-summary family: step metrics / skip reason / per-node / per-gate / the summary itself) — engine-emitted telemetry read by the Console and by operator queries, nobody authors them, so the `wire` verdict covers them unchanged |
 | `state-machine.zod.ts` | authorable | **strict as of #4001 批 10** — all six sites (`ActionRef` / `GuardRef` / `Transition` / `StateNode` + `.meta` / `StateMachine`). **The `(p)` was NOT a formality here.** ADR-0020 retired this XState shape as a *record-lifecycle* declaration — the top-level `workflow` metadata type and `object.stateMachines` are both gone, and a record's transitions live on the `state_machine` VALIDATION RULE instead — so had those been the only doors this file would be DEAD surface, and the correct action would have been to fix its class, not close it. One authoring door survives: `ai/agent.zod.ts`'s `lifecycle` is `StateMachineSchema`, and `agent` is a registered type, so `defineStack({ agents })` / meta REST / the Studio agent form all reach here through `AgentSchema.parse()`. Verified by parse: an agent whose lifecycle carried `stats`, a state with `onn` (one keystroke from `on`) and a `meta` with two unknown keys **parsed clean**, returning a machine with NO transitions at all — the declaration whose whole job is to deny undeclared transitions, silently emptied and reported valid. `.meta` was checked for the #4909 open-slot case and is CLOSED: the hand-written `StateNodeConfig` type declares exactly its four keys (passthrough would open the Zod while `tsc` stayed shut), nothing in the repo reads any `meta` key, and the prior behaviour was strip — an author's `meta` arrived as `{}` — so there was no openness to preserve. ⚠️ `ActionRef` / `GuardRef` are UNIONS: a strict branch's message does not reach the top (zod raises one `invalid_union` whose message is the literal `"Invalid input"`, with the real prescription nested in `issue.errors[]`), which `formatZodError` then flattens away — filed, not fixed here. **−1 at #4658**: the orphan `EventSchema` (`{ type, schema }`, an XState-style signal declaration nothing referenced — `StateMachineSchema` names event types as `on:` record keys) was deleted rather than converged with `kernel/events/core.zod.ts`'s envelope `EventSchema`, whose key set it did not intersect (#4535 C6). The remaining 6 sites and their verdict are unchanged |
 | `control-flow.zod.ts` | authorable | **strict as of #4001 批 10** — all five sites (`FlowRegion` / `Loop` / `ParallelBranch` / `Parallel` / `TryCatch`). The `(p)` resolves to authorable on the executors' own parse seam (`parseNodeConfig`, #4277) plus `validateControlFlow`'s region parse. **`validateControlFlow` is a sibling guard, not a key gate, and the two do not fight**: it answers single-entry / single-exit / acyclic, which no key check can decide, and the schema answers key membership, which no structural check can decide. They meet at exactly one seam — the guard `safeParse`s each region slot before analyzing it, so an undeclared region key now surfaces there as `<where>: invalid region — <the strictObject message>`, the guard's framing wrapping the schema's prescription. Nothing was duplicated and nothing removed; the guard simply stopped silently repairing its own input before judging it. Two curation entries had to be MEASURED rather than reasoned: the bare edit-distance fallback answers `itemVariable` with **`indexVariable`** — binding the loop INDEX where the author wanted the ITEM — so the alias exists to overrule a confidently wrong suggestion from this campaign's own helper (the `pii` → `min` shape, third instance); and `join`/`joinGateway` needed two DISTINCT prescriptions because `guidance` emits one bullet per key verbatim, so a shared string printed the same paragraph twice. Its test instrument also had to be rebuilt: `region-slots.test.ts` probed every construct with every candidate key at once and depended on `.strip` to discard the mismatches, so it returned "no schema accepts any region" the moment the shapes closed — it failed loudly, which is the only reason this is a footnote and not a fourth finding-3. Structural validation by `validateControlFlow` remains. **−1 at #4661**: `RetryPolicySchema` moved out to `shared/retry-policy.zod.ts` — `./automation` and `./system` published the same name for two different declarations (#4411), so the retry policy converged onto one. The site still exists and is still non-strict and authorable; it is simply no longer in a directory this ledger sections. ⚠️ That is a coverage gap worth knowing about: this audit sections `ui/` / `data/` / `automation/` / `security/` / `studio/` only, so a `shared/` shape is unaudited by construction. The tolerance is deliberate here — the `retryDelayMs` → `backoffMs` rename is tombstoned via `retiredKey()` precisely because a non-strict parent would otherwise swallow the old spelling. **#4964 widened that rename to `flow.errorHandling`**, which spelled the base delay the pre-17 way while the shared policy tombstoned it — so the two automation retry surfaces now teach the same word, and the tombstone's prescription names all four surfaces instead of the two #4661 could see |

packages/spec/api-surface.json

@@ -2106,22 +2106,31 @@
     "DeleteRecordConfigSchema (const)",
     "ETL (const)",
     "ETLDestination (type)",
+    "ETLDestinationParsed (type)",
     "ETLDestinationSchema (const)",
     "ETLEndpointType (type)",
+    "ETLEndpointTypeParsed (type)",
     "ETLEndpointTypeSchema (const)",
     "ETLPipeline (type)",
+    "ETLPipelineParsed (type)",
     "ETLPipelineRun (type)",
+    "ETLPipelineRunParsed (type)",
     "ETLPipelineRunSchema (const)",
     "ETLPipelineSchema (const)",
     "ETLRunStatus (type)",
+    "ETLRunStatusParsed (type)",
     "ETLRunStatusSchema (const)",
     "ETLSource (type)",
+    "ETLSourceParsed (type)",
     "ETLSourceSchema (const)",
     "ETLSyncMode (type)",
+    "ETLSyncModeParsed (type)",
     "ETLSyncModeSchema (const)",
     "ETLTransformation (type)",
+    "ETLTransformationParsed (type)",
     "ETLTransformationSchema (const)",
     "ETLTransformationType (type)",
+    "ETLTransformationTypeParsed (type)",
     "ETLTransformationTypeSchema (const)",
     "ExecutionError (type)",
     "ExecutionErrorParsed (type)",

packages/spec/docs/SYNC_ARCHITECTURE.md

@@ -77,8 +77,19 @@ Advanced data pipelines for complex transformations, multi-source aggregation, a
 
 ### Example
 
+> **`ETLPipeline` is the AUTHOR shape.** It is `z.input` of `ETLPipelineSchema`
+> (#4963, the house `X` / `XParsed` convention), so every key carrying a
+> `.default()` — `syncMode`, `enabled`, `destination.writeMode`, a
+> transformation's `continueOnError`, `source.incremental.enabled` — is optional
+> when you write a pipeline, and `schedule` takes the bare cron string the
+> schema wraps for you. Annotate the **result** of
+> `ETLPipelineSchema.parse(…)` with **`ETLPipelineParsed`**, where those same
+> keys are all present. The example below states them anyway, because it is a
+> tour of the surface; the Migration Guide's examples omit them, because that is
+> what ordinary authoring looks like.
+
 ```typescript
-import { ETLPipeline } from '@objectstack/spec/automation';
+import type { ETLPipeline } from '@objectstack/spec/automation';
 
 const dataWarehousePipeline: ETLPipeline = {
   name: 'customer_360_pipeline',
@@ -377,24 +388,36 @@ const connector: Connector = {
 
 **After (L2):**
 ```typescript
+import type { ETLPipeline } from '@objectstack/spec/automation';
+
 const pipeline: ETLPipeline = {
   name: 'order_analytics_pipeline',
-  source: { type: 'api', connector: 'orders' },
+  source: { type: 'api', connector: 'orders', config: { endpoint: '/orders' } },
   transformations: [
     { type: 'aggregate', config: { groupBy: ['customer_id'] } }
   ],
   destination: { type: 'database', config: { table: 'analytics_order' } }
 };
 ```
 
+Every endpoint carries a `config` bag — it is the one required key besides
+`type`, and it is where endpoint-specific settings (`table`, `endpoint`, `path`,
+`format`) live. `syncMode`, `enabled`, `destination.writeMode` and the
+transformation's `continueOnError` are omitted on purpose: they have defaults,
+and `ETLPipeline` is the author shape.
+
 ### From L2 to L3
 
 When your ETL pipeline needs webhooks, advanced auth, or rate limiting:
 
 **Before (L2):**
 ```typescript
+import type { ETLPipeline } from '@objectstack/spec/automation';
+
 const pipeline: ETLPipeline = {
-  source: { type: 'api', connector: 'external_api' }
+  name: 'external_api_ingest',
+  source: { type: 'api', connector: 'external_api', config: { endpoint: '/events' } },
+  destination: { type: 'database', config: { table: 'external_events' } }
 };
 ```