From cb7756ab29975e919734fc15776e5f3804b60b96 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 5 Aug 2026 14:26:39 +0000 Subject: [PATCH 1/3] =?UTF-8?q?fix(spec):=20`automation/etl.zod.ts`=20?= =?UTF-8?q?=E7=9A=84=E4=B9=9D=E4=B8=AA=E5=88=AB=E5=90=8D=E5=9B=9E=E5=88=B0?= =?UTF-8?q?=20`X`=20/=20`XParsed`=20house=20convention=20(#4963)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 裸名翻转为 `z.input`(作者写的形状),新增九个 `*Parsed` = `z.infer`(parse 之后的 形状),与 `shared/retry-policy.zod.ts` 记下的 house convention 一致。 翻转之前九个别名全是 `z.infer`,而本文件有六个带 `.default()` 的键,外加 `schedule` 是 `CronExpressionInputSchema` transform —— 于是 `const p: ETLPipeline = { … }`(三仓零 parse site,这就是本文件唯一的授权门) 根本编译不过。SYNC_ARCHITECTURE.md 的三段示例就是证据,同 PR 修到可编译并加 compiler-API 测试逐字编译它们。 两个 ETL factory 去掉为满足自身返回类型而写的 `enabled: true` 与 cron 预包装。 Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_018fxLGQdatPbBUvCgiVxg6D --- .changeset/etl-author-shape-aliases.md | 52 +++ .../2026-07-unknown-key-strictness-ledger.md | 2 +- packages/spec/api-surface.json | 9 + packages/spec/docs/SYNC_ARCHITECTURE.md | 29 +- .../src/automation/etl-author-shape.test.ts | 371 ++++++++++++++++++ packages/spec/src/automation/etl.zod.ts | 131 ++++++- 6 files changed, 576 insertions(+), 18 deletions(-) create mode 100644 .changeset/etl-author-shape-aliases.md create mode 100644 packages/spec/src/automation/etl-author-shape.test.ts diff --git a/.changeset/etl-author-shape-aliases.md b/.changeset/etl-author-shape-aliases.md new file mode 100644 index 0000000000..978fcee75c --- /dev/null +++ b/.changeset/etl-author-shape-aliases.md @@ -0,0 +1,52 @@ +--- +"@objectstack/spec": major +--- + +**`automation/etl.zod.ts`'s nine type aliases now follow the house `X` / `XParsed` +convention** (#4963): the bare name is `z.input` — what an author writes — and a new +`XParsed` is `z.infer` — what a parse returns. + +Until 17 all nine were `z.infer` under the bare name with no `*Parsed` counterpart at +all. On this file that was not a style detail. Six keys carry `.default()` +(`ETLDestination.writeMode`, `ETLTransformation.continueOnError`, `ETLPipeline.syncMode` +/ `.enabled`, `ETLSource.incremental.enabled`, and every key of `ETLPipeline.retry`) and +`schedule` is a `CronExpressionInputSchema` transform whose *output* is the +`{ dialect, source }` envelope. Under `z.infer` all of them were REQUIRED and a bare +cron string was rejected — so the single use this file has, `const p: ETLPipeline = { … }` +written by hand, did not compile. That is the whole authoring door: `etl.zod.ts` has no +parse site in objectstack / objectui / cloud, so the exported schema and the exported +type are the only surface an author touches. + +The evidence was checked in. `packages/spec/docs/SYNC_ARCHITECTURE.md` carried three +`ETLPipeline` examples, none of which compiled, and both `ETL` factories were spelling +out defaults and pre-wrapping their cron purely to satisfy their own return type. + +### Migration + +**Zero importers across objectstack, objectui and cloud** (re-measured against each +repo's `origin/main` for this change), so the migration surface is empty. If you have a +local consumer: + +| You wrote | Keep it if | Change it to | +|:---|:---|:---| +| `const p: ETLPipeline = { … }` | you are AUTHORING a pipeline literal | nothing — this is the case that now compiles | +| `const p: ETLPipeline = ETLPipelineSchema.parse(raw)` | — | `const p: ETLPipelineParsed = …` | +| `function run(p: ETLPipeline)` reading `p.syncMode` as always-present | — | `ETLPipelineParsed` | + +The same rename applies to each of the nine: `ETLEndpointType`, `ETLSource`, +`ETLDestination`, `ETLTransformationType`, `ETLTransformation`, `ETLSyncMode`, +`ETLPipeline`, `ETLRunStatus`, `ETLPipelineRun` — append `Parsed` wherever the annotated +value came out of a `.parse()`. The four enum aliases are unaffected in practice +(`z.input` and `z.infer` coincide for an enum); their pair exists so a reader never has +to know which of the nine has defaults before choosing an annotation. + +Nothing at runtime moves: no schema, default, bound or key changed, and both `ETL` +factories still produce documents that parse to the same result. `ETL.databaseSync` / +`ETL.apiToDatabase` no longer restate `enabled: true` and no longer pre-wrap a bare cron +string into `{ dialect: 'cron', source }` — the schema does that at parse, which is where +it always belonged. Each helper still states what it DECIDES (`incremental` + `upsert` +vs `full` + `append`), because that contrast is the reason the pair exists. + +`SYNC_ARCHITECTURE.md`'s three examples now compile, and a compiler-API test +(`etl-author-shape.test.ts`) compiles them verbatim on every run — import line included — +so they cannot rot again silently. diff --git a/docs/audits/2026-07-unknown-key-strictness-ledger.md b/docs/audits/2026-07-unknown-key-strictness-ledger.md index e37f8cc728..8d371ed495 100644 --- a/docs/audits/2026-07-unknown-key-strictness-ledger.md +++ b/docs/audits/2026-07-unknown-key-strictness-ledger.md @@ -691,7 +691,7 @@ column does not move and the `strip` column falls by the count of what left. | File | Class | Note | |---|---|---| | `flow.zod.ts` | authorable | **strict as of #4001** — the four outer authoring shapes at step 1, and **the six nested blocks at batch 11** (`FlowNode.connectorConfig` / `.position` / `.inputSchema` / `.waitEventConfig` / `.boundaryConfig`, `Flow.errorHandling`). The gap between those two dates is this campaign's own finding 17 inside its own file: closing the shells left the gate rejecting `nodee:` at node level while `connectorConfig: { connectorId, actionId, params: {…} }` parsed clean and the executor dispatched `input ?? {}` — a successful connector call carrying nothing. Worth recording precisely, because the obvious example is the wrong one: a slip on a REQUIRED key was always loud (it then reads as missing). What `.strip` swallowed here is the OPTIONAL half — the input map, the retry budget, `interrupting: false`, `required: true` — i.e. exactly the keys an author adds to CONSTRAIN behaviour, replaced by a permissive default without a word. `Flow.errorHandling` gained a second chapter at **#4964**: closing it in 批 11 revealed (rather than caused) that its retry keys were a THIRD encoding of the policy #4661 had converged — it spelled the base delay `retryDelayMs` where the shared declaration spells it `backoffMs` and tombstones the old word, so the strictness this row records was, for one release, rejecting an author for having read the newer file. The block now builds from `retryPolicyShape()`. Site count unchanged; only the vocabulary. Two things stay open and are now pinned in code with the reason, so a later sweep stops rather than "finishes" the file: the node `config` slot (ADR-0018 plugin namespace) and `FlowVersionHistorySchema` (the file's only WIRE shape — emitted on publish, never authored; its `definition` is `FlowSchema`, so the authored half inside a history record is gated anyway) | -| `etl.zod.ts` | mixed | **7 strict as of #4001 批 12** — the authoring half (`ETLSource` + `.incremental`, `ETLDestination`, `ETLTransformation`, `ETLPipeline` + `.retry` + `.notifications`). The other 3 — `ETLPipelineRun` + `.stats` + `.error` — are **deliberately left open**: engine-emitted run state (an id it minted, a status it reached, counters it accumulated), same disposition and same reason as `FlowVersionHistorySchema` above and all of `execution.zod.ts`. The exemption is recorded on the schema itself, not only here, because a note only this file carries is a note the next sweep does not read. The old blanket `authorable (p)` was too wide; verification split it. ⚠️ **Read the classification caveat before reusing this verdict**: `etl.zod.ts` has NO parse site in objectstack / objectui / cloud, so neither half could be settled by pointing at a live call. The 7 are authorable because the exported schema and type ARE the door (`SYNC_ARCHITECTURE.md` and the module's `@example` both hand-write `const p: ETLPipeline = { … }`) — the `webhook.zod.ts` posture. The 3 are wire on the shape's semantics plus settled precedent, NOT on an emit site anyone can point at today; if an ETL engine ever lands and a run result turns out to be operator-authored, that verdict is the one to revisit. Two out-of-scope findings were filed rather than fixed here; **the first is now closed**: the `retry` block was a third retry-policy vocabulary #4661's convergence never reached (#4962 — converged onto `shared/RetryPolicySchema` in the v17 window, together with `flow.errorHandling` (#4964), the fourth. Both were anonymous inline blocks, so the dual-source instrument that drove #4661 could not see them: it asks how many declarations share an exported NAME, and neither has one. 批 12's five curated `retry` entries described that divergence and dissolved with it — the block's site count is unchanged, only its vocabulary). Still open: all nine type aliases export the parsed shape under the bare name, which is why the SYNC_ARCHITECTURE.md pipeline examples do not compile (#4963). **−12 at #4738**: `sync.zod.ts` (the L1 "Simple Sync" file — `DataSyncConfig`, its `ConflictResolution` enum and satellites, formerly this row's co-candidate) was deleted whole rather than hardened: three-repo zero importers, no parse site, defs unreachable from the metadata-type roots (#4650 gate), so there was no author for strictness to protect (#4535 C13+C15). The integration-side `ConflictResolution` → `ConnectorConflictResolution` rename in the same change is name-only and moves no sites | +| `etl.zod.ts` | mixed | **7 strict as of #4001 批 12** — the authoring half (`ETLSource` + `.incremental`, `ETLDestination`, `ETLTransformation`, `ETLPipeline` + `.retry` + `.notifications`). The other 3 — `ETLPipelineRun` + `.stats` + `.error` — are **deliberately left open**: engine-emitted run state (an id it minted, a status it reached, counters it accumulated), same disposition and same reason as `FlowVersionHistorySchema` above and all of `execution.zod.ts`. The exemption is recorded on the schema itself, not only here, because a note only this file carries is a note the next sweep does not read. The old blanket `authorable (p)` was too wide; verification split it. ⚠️ **Read the classification caveat before reusing this verdict**: `etl.zod.ts` has NO parse site in objectstack / objectui / cloud, so neither half could be settled by pointing at a live call. The 7 are authorable because the exported schema and type ARE the door (`SYNC_ARCHITECTURE.md` and the module's `@example` both hand-write `const p: ETLPipeline = { … }`) — the `webhook.zod.ts` posture. The 3 are wire on the shape's semantics plus settled precedent, NOT on an emit site anyone can point at today; if an ETL engine ever lands and a run result turns out to be operator-authored, that verdict is the one to revisit. Two out-of-scope findings were filed rather than fixed here; **the first is now closed**: the `retry` block was a third retry-policy vocabulary #4661's convergence never reached (#4962 — converged onto `shared/RetryPolicySchema` in the v17 window, together with `flow.errorHandling` (#4964), the fourth. Both were anonymous inline blocks, so the dual-source instrument that drove #4661 could not see them: it asks how many declarations share an exported NAME, and neither has one. 批 12's five curated `retry` entries described that divergence and dissolved with it — the block's site count is unchanged, only its vocabulary). **The second is now closed too**: all nine type aliases exported the PARSED shape under the bare name with no `*Parsed` counterpart, so the authoring door this row's whole classification rests on — `const p: ETLPipeline = { … }` — did not actually compile, and the SYNC_ARCHITECTURE.md examples proving it were the evidence (#4963 — bare names flipped to `z.input`, nine `*Parsed` added, house convention per `shared/retry-policy.zod.ts`; three-repo zero importers made the migration surface empty). Worth carrying forward for the next classification: **"authorable because the exported type IS the door" is a claim about a type that must be checked, not assumed** — 批 12 read the door correctly and nobody compiled it. **−12 at #4738**: `sync.zod.ts` (the L1 "Simple Sync" file — `DataSyncConfig`, its `ConflictResolution` enum and satellites, formerly this row's co-candidate) was deleted whole rather than hardened: three-repo zero importers, no parse site, defs unreachable from the metadata-type roots (#4650 gate), so there was no author for strictness to protect (#4535 C13+C15). The integration-side `ConflictResolution` → `ConnectorConflictResolution` rename in the same change is name-only and moves no sites | | `execution.zod.ts` | wire | run-state envelopes — never strict. +5 at #4354 (the run-summary family: step metrics / skip reason / per-node / per-gate / the summary itself) — engine-emitted telemetry read by the Console and by operator queries, nobody authors them, so the `wire` verdict covers them unchanged | | `state-machine.zod.ts` | authorable | **strict as of #4001 批 10** — all six sites (`ActionRef` / `GuardRef` / `Transition` / `StateNode` + `.meta` / `StateMachine`). **The `(p)` was NOT a formality here.** ADR-0020 retired this XState shape as a *record-lifecycle* declaration — the top-level `workflow` metadata type and `object.stateMachines` are both gone, and a record's transitions live on the `state_machine` VALIDATION RULE instead — so had those been the only doors this file would be DEAD surface, and the correct action would have been to fix its class, not close it. One authoring door survives: `ai/agent.zod.ts`'s `lifecycle` is `StateMachineSchema`, and `agent` is a registered type, so `defineStack({ agents })` / meta REST / the Studio agent form all reach here through `AgentSchema.parse()`. Verified by parse: an agent whose lifecycle carried `stats`, a state with `onn` (one keystroke from `on`) and a `meta` with two unknown keys **parsed clean**, returning a machine with NO transitions at all — the declaration whose whole job is to deny undeclared transitions, silently emptied and reported valid. `.meta` was checked for the #4909 open-slot case and is CLOSED: the hand-written `StateNodeConfig` type declares exactly its four keys (passthrough would open the Zod while `tsc` stayed shut), nothing in the repo reads any `meta` key, and the prior behaviour was strip — an author's `meta` arrived as `{}` — so there was no openness to preserve. ⚠️ `ActionRef` / `GuardRef` are UNIONS: a strict branch's message does not reach the top (zod raises one `invalid_union` whose message is the literal `"Invalid input"`, with the real prescription nested in `issue.errors[]`), which `formatZodError` then flattens away — filed, not fixed here. **−1 at #4658**: the orphan `EventSchema` (`{ type, schema }`, an XState-style signal declaration nothing referenced — `StateMachineSchema` names event types as `on:` record keys) was deleted rather than converged with `kernel/events/core.zod.ts`'s envelope `EventSchema`, whose key set it did not intersect (#4535 C6). The remaining 6 sites and their verdict are unchanged | | `control-flow.zod.ts` | authorable | **strict as of #4001 批 10** — all five sites (`FlowRegion` / `Loop` / `ParallelBranch` / `Parallel` / `TryCatch`). The `(p)` resolves to authorable on the executors' own parse seam (`parseNodeConfig`, #4277) plus `validateControlFlow`'s region parse. **`validateControlFlow` is a sibling guard, not a key gate, and the two do not fight**: it answers single-entry / single-exit / acyclic, which no key check can decide, and the schema answers key membership, which no structural check can decide. They meet at exactly one seam — the guard `safeParse`s each region slot before analyzing it, so an undeclared region key now surfaces there as `: invalid region — `, the guard's framing wrapping the schema's prescription. Nothing was duplicated and nothing removed; the guard simply stopped silently repairing its own input before judging it. Two curation entries had to be MEASURED rather than reasoned: the bare edit-distance fallback answers `itemVariable` with **`indexVariable`** — binding the loop INDEX where the author wanted the ITEM — so the alias exists to overrule a confidently wrong suggestion from this campaign's own helper (the `pii` → `min` shape, third instance); and `join`/`joinGateway` needed two DISTINCT prescriptions because `guidance` emits one bullet per key verbatim, so a shared string printed the same paragraph twice. Its test instrument also had to be rebuilt: `region-slots.test.ts` probed every construct with every candidate key at once and depended on `.strip` to discard the mismatches, so it returned "no schema accepts any region" the moment the shapes closed — it failed loudly, which is the only reason this is a footnote and not a fourth finding-3. Structural validation by `validateControlFlow` remains. **−1 at #4661**: `RetryPolicySchema` moved out to `shared/retry-policy.zod.ts` — `./automation` and `./system` published the same name for two different declarations (#4411), so the retry policy converged onto one. The site still exists and is still non-strict and authorable; it is simply no longer in a directory this ledger sections. ⚠️ That is a coverage gap worth knowing about: this audit sections `ui/` / `data/` / `automation/` / `security/` / `studio/` only, so a `shared/` shape is unaudited by construction. The tolerance is deliberate here — the `retryDelayMs` → `backoffMs` rename is tombstoned via `retiredKey()` precisely because a non-strict parent would otherwise swallow the old spelling. **#4964 widened that rename to `flow.errorHandling`**, which spelled the base delay the pre-17 way while the shared policy tombstoned it — so the two automation retry surfaces now teach the same word, and the tombstone's prescription names all four surfaces instead of the two #4661 could see | diff --git a/packages/spec/api-surface.json b/packages/spec/api-surface.json index f902ec0006..cbb28aa677 100644 --- a/packages/spec/api-surface.json +++ b/packages/spec/api-surface.json @@ -2106,22 +2106,31 @@ "DeleteRecordConfigSchema (const)", "ETL (const)", "ETLDestination (type)", + "ETLDestinationParsed (type)", "ETLDestinationSchema (const)", "ETLEndpointType (type)", + "ETLEndpointTypeParsed (type)", "ETLEndpointTypeSchema (const)", "ETLPipeline (type)", + "ETLPipelineParsed (type)", "ETLPipelineRun (type)", + "ETLPipelineRunParsed (type)", "ETLPipelineRunSchema (const)", "ETLPipelineSchema (const)", "ETLRunStatus (type)", + "ETLRunStatusParsed (type)", "ETLRunStatusSchema (const)", "ETLSource (type)", + "ETLSourceParsed (type)", "ETLSourceSchema (const)", "ETLSyncMode (type)", + "ETLSyncModeParsed (type)", "ETLSyncModeSchema (const)", "ETLTransformation (type)", + "ETLTransformationParsed (type)", "ETLTransformationSchema (const)", "ETLTransformationType (type)", + "ETLTransformationTypeParsed (type)", "ETLTransformationTypeSchema (const)", "ExecutionError (type)", "ExecutionErrorParsed (type)", diff --git a/packages/spec/docs/SYNC_ARCHITECTURE.md b/packages/spec/docs/SYNC_ARCHITECTURE.md index e9faef7fb5..64d9f063b4 100644 --- a/packages/spec/docs/SYNC_ARCHITECTURE.md +++ b/packages/spec/docs/SYNC_ARCHITECTURE.md @@ -77,8 +77,19 @@ Advanced data pipelines for complex transformations, multi-source aggregation, a ### Example +> **`ETLPipeline` is the AUTHOR shape.** It is `z.input` of `ETLPipelineSchema` +> (#4963, the house `X` / `XParsed` convention), so every key carrying a +> `.default()` — `syncMode`, `enabled`, `destination.writeMode`, a +> transformation's `continueOnError`, `source.incremental.enabled` — is optional +> when you write a pipeline, and `schedule` takes the bare cron string the +> schema wraps for you. Annotate the **result** of +> `ETLPipelineSchema.parse(…)` with **`ETLPipelineParsed`**, where those same +> keys are all present. The example below states them anyway, because it is a +> tour of the surface; the Migration Guide's examples omit them, because that is +> what ordinary authoring looks like. + ```typescript -import { ETLPipeline } from '@objectstack/spec/automation'; +import type { ETLPipeline } from '@objectstack/spec/automation'; const dataWarehousePipeline: ETLPipeline = { name: 'customer_360_pipeline', @@ -377,9 +388,11 @@ const connector: Connector = { **After (L2):** ```typescript +import type { ETLPipeline } from '@objectstack/spec/automation'; + const pipeline: ETLPipeline = { name: 'order_analytics_pipeline', - source: { type: 'api', connector: 'orders' }, + source: { type: 'api', connector: 'orders', config: { endpoint: '/orders' } }, transformations: [ { type: 'aggregate', config: { groupBy: ['customer_id'] } } ], @@ -387,14 +400,24 @@ const pipeline: ETLPipeline = { }; ``` +Every endpoint carries a `config` bag — it is the one required key besides +`type`, and it is where endpoint-specific settings (`table`, `endpoint`, `path`, +`format`) live. `syncMode`, `enabled`, `destination.writeMode` and the +transformation's `continueOnError` are omitted on purpose: they have defaults, +and `ETLPipeline` is the author shape. + ### From L2 to L3 When your ETL pipeline needs webhooks, advanced auth, or rate limiting: **Before (L2):** ```typescript +import type { ETLPipeline } from '@objectstack/spec/automation'; + const pipeline: ETLPipeline = { - source: { type: 'api', connector: 'external_api' } + name: 'external_api_ingest', + source: { type: 'api', connector: 'external_api', config: { endpoint: '/events' } }, + destination: { type: 'database', config: { table: 'external_events' } } }; ``` diff --git a/packages/spec/src/automation/etl-author-shape.test.ts b/packages/spec/src/automation/etl-author-shape.test.ts new file mode 100644 index 0000000000..d9faf0f33e --- /dev/null +++ b/packages/spec/src/automation/etl-author-shape.test.ts @@ -0,0 +1,371 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +import { describe, it, expect } from 'vitest'; +import ts from 'typescript'; +import { readFileSync } from 'node:fs'; +import { dirname, resolve } from 'node:path'; +import { fileURLToPath } from 'node:url'; + +import { ETL, ETLPipelineSchema } from './etl.zod'; + +// ─── [#4963] `ETLPipeline` is the AUTHOR shape — proved with the compiler ─── +// +// Until 17 all nine `etl.zod.ts` type aliases were `z.infer` under the bare +// name with no `*Parsed` counterpart, against the house convention (bare name = +// `z.input` = what an author writes; `XParsed` = `z.infer` = what a parse +// returns — written up on `shared/retry-policy.zod.ts`, followed by every +// sibling automation config). On this file that was not cosmetic: six defaulted +// keys and a transform-typed `schedule` were all REQUIRED under `z.infer`, so +// `const p: ETLPipeline = { … }` — the file's only authoring door, there being +// no parse site in objectstack / objectui / cloud — did not compile. +// `SYNC_ARCHITECTURE.md` carried three examples that proved it. +// +// ## Why this file uses the compiler API instead of type-level pins +// +// #4642 established that a conditional-type pin in a `packages/spec` test is a +// NO-OP: `tsconfig.json` excludes `**/*.test.ts` (the package's measured +// `TEST_DEBT` entry in `scripts/check-type-check-coverage.mjs`) and vitest never +// enables `typecheck`. A `@ts-expect-error` or `expectTypeOf` written here would +// be read by nothing. So the pins below drive `ts.createProgram` themselves and +// assert on real diagnostics, the way `sync-retirement.test.ts` does — and, like +// it, they carry anti-vacuity guards, because a harness that resolves nothing +// reports zero errors and looks exactly like success. + +const SPEC_DIR = resolve(dirname(fileURLToPath(import.meta.url)), '../..'); +const SYNC_ARCHITECTURE = resolve(SPEC_DIR, 'docs/SYNC_ARCHITECTURE.md'); + +/** + * Compile a set of probe files against this package's real source and return + * each one's diagnostics, keyed by probe name. + * + * `@objectstack/spec/` is mapped through `paths` to the entry barrel in + * `src/`, which is what lets a documentation snippet be compiled VERBATIM — + * import line included — rather than rewritten into a relative import that no + * reader of the docs would ever type. + * + * `noUnusedLocals` is deliberately off: a documentation snippet declares a + * `const` and stops, and TS6133 is a lint opinion about the snippet's framing, + * not a statement about whether the pipeline literal is well-typed. Everything + * else runs at the repo's real strictness (`strict: true`). + */ +function compileProbes(probes: Readonly>): Map { + const dir = resolve(SPEC_DIR, 'src/__etl_author_shape_probes__'); + const paths = new Map(); + for (const [name, text] of Object.entries(probes)) paths.set(resolve(dir, `${name}.ts`), text); + + const options: ts.CompilerOptions = { + target: ts.ScriptTarget.ES2020, + module: ts.ModuleKind.ESNext, + moduleResolution: ts.ModuleResolutionKind.Bundler, + strict: true, + skipLibCheck: true, + noEmit: true, + noUnusedLocals: false, + noUnusedParameters: false, + baseUrl: SPEC_DIR, + paths: { '@objectstack/spec/*': [resolve(SPEC_DIR, 'src/*/index.ts')] }, + }; + + const host = ts.createCompilerHost(options, true); + const realGetSourceFile = host.getSourceFile.bind(host); + const realFileExists = host.fileExists.bind(host); + const realReadFile = host.readFile.bind(host); + host.getSourceFile = (fileName, languageVersion, onError, shouldCreate) => { + const overlay = paths.get(resolve(fileName)); + return overlay === undefined + ? realGetSourceFile(fileName, languageVersion, onError, shouldCreate) + : ts.createSourceFile(fileName, overlay, languageVersion, true); + }; + host.fileExists = (fileName) => paths.has(resolve(fileName)) || realFileExists(fileName); + host.readFile = (fileName) => paths.get(resolve(fileName)) ?? realReadFile(fileName); + + const program = ts.createProgram([...paths.keys()], options, host); + const out = new Map(); + for (const name of Object.keys(probes)) out.set(name, []); + for (const d of ts.getPreEmitDiagnostics(program)) { + const file = d.file?.fileName ? resolve(d.file.fileName) : undefined; + for (const [name] of Object.entries(probes)) { + if (file === resolve(dir, `${name}.ts`)) out.get(name)!.push(d); + } + } + return out; +} + +/** One diagnostic per line, `TS: `, for readable assertions. */ +function render(diagnostics: readonly ts.Diagnostic[]): string { + return diagnostics + .map((d) => `TS${d.code}: ${ts.flattenDiagnosticMessageText(d.messageText, ' ')}`) + .join('\n'); +} + +/** Every ```typescript fence in a markdown file. */ +function typescriptBlocks(markdown: string): string[] { + return [...markdown.matchAll(/```typescript\r?\n([\s\S]*?)```/g)].map((m) => m[1]); +} + +describe('[#4963] SYNC_ARCHITECTURE.md pipeline examples compile', () => { + const markdown = readFileSync(SYNC_ARCHITECTURE, 'utf8'); + const blocks = typescriptBlocks(markdown); + const pipelineBlocks = blocks.filter((b) => b.includes('ETLPipeline')); + + it('finds the examples this gate exists for, and counts the ones it skips', () => { + // Anti-vacuity: a selector that matched nothing would make the compile + // assertion below pass over an empty program — the way a gate goes dormant. + expect(pipelineBlocks.length, 'ETLPipeline examples in SYNC_ARCHITECTURE.md').toBe(3); + // The other three are the L3 `Connector` examples. They are out of this + // gate's scope — they belong to `integration/connector.zod.ts`, and two of + // them deliberately elide with a bare `...`, which is not TypeScript. The + // total is pinned rather than left open so that ADDING a block to this + // document is a decision someone has to make on purpose: a new ETL example + // is picked up automatically by the selector above, and anything else + // turns this red until it is classified here. + expect(blocks.length, 'total ```typescript blocks — classify any new one').toBe(6); + }); + + it('compiles all three verbatim, import line included, with zero diagnostics', () => { + const probes: Record = {}; + pipelineBlocks.forEach((block, i) => { probes[`doc-example-${i}`] = block; }); + // The harness's own control: a probe that MUST fail. Without it, a + // resolution failure (paths mapping wrong, host overlay not applied) would + // report zero diagnostics for every block and read as three green examples. + probes['harness-self-test'] = [ + "import type { ETLPipeline } from '@objectstack/spec/automation';", + "const broken: ETLPipeline = { name: 'no_source_no_destination' };", + ].join('\n'); + + const results = compileProbes(probes); + expect(render(results.get('harness-self-test')!), 'the harness must be able to report an error') + .toContain('TS2739'); + + for (const [name, diagnostics] of results) { + if (name === 'harness-self-test') continue; + expect(render(diagnostics), `${name} must compile clean`).toBe(''); + } + }); +}); + +describe('[#4963] the bare name is the author shape; `*Parsed` is the parse result', () => { + /** + * The load-bearing pin, written as ONE program so the positive and the + * negative share a document: the same literal, the same keys, only the + * annotation differs. A red `parsed-*` probe therefore cannot be a literal + * that was wrong for an unrelated reason — its `author-*` twin just compiled. + */ + const probes = { + // ── author shape: every defaulted key omitted, cron written bare ── + 'author-pipeline': ` + import type { ETLPipeline } from '@objectstack/spec/automation'; + const p: ETLPipeline = { + name: 'customer_360', + source: { type: 'api', connector: 'salesforce', config: { object: 'Account' } }, + destination: { type: 'warehouse', config: { table: 'customers' } }, + transformations: [{ type: 'filter', config: { condition: 'active' } }], + schedule: '0 2 * * *', + }; + `, + 'author-source': ` + import type { ETLSource } from '@objectstack/spec/automation'; + const s: ETLSource = { type: 'api', config: {}, incremental: { cursorField: 'updated_at' } }; + `, + 'author-destination': ` + import type { ETLDestination } from '@objectstack/spec/automation'; + const d: ETLDestination = { type: 'database', config: { table: 't' } }; + `, + 'author-transformation': ` + import type { ETLTransformation } from '@objectstack/spec/automation'; + const t: ETLTransformation = { type: 'map', config: {} }; + `, + 'author-run': ` + import type { ETLPipelineRun } from '@objectstack/spec/automation'; + const r: ETLPipelineRun = { + id: 'run-1', pipelineName: 'customer_360', status: 'succeeded', + startedAt: '2024-01-01T02:00:00Z', stats: {}, + }; + `, + + // ── parsed shape: the SAME literals, which must now be rejected ── + 'parsed-pipeline': ` + import type { ETLPipelineParsed } from '@objectstack/spec/automation'; + const p: ETLPipelineParsed = { + name: 'customer_360', + source: { type: 'api', connector: 'salesforce', config: { object: 'Account' } }, + destination: { type: 'warehouse', config: { table: 'customers' } }, + transformations: [{ type: 'filter', config: { condition: 'active' } }], + schedule: '0 2 * * *', + }; + `, + // Everything a parse WOULD have filled in is present except `syncMode` and + // `enabled`, so the top-level omission is the only thing left to report. + // The probe above cannot say this on its own: TypeScript reports the + // deepest mismatch it finds and stops, so with `writeMode` / + // `continueOnError` / the cron envelope also missing, the two top-level + // keys never appear in its message at all. + 'parsed-pipeline-top-level-only': ` + import type { ETLPipelineParsed } from '@objectstack/spec/automation'; + const p: ETLPipelineParsed = { + name: 'customer_360', + source: { + type: 'api', connector: 'salesforce', config: { object: 'Account' }, + incremental: { enabled: true, cursorField: 'updated_at' }, + }, + destination: { type: 'warehouse', config: { table: 'customers' }, writeMode: 'upsert' }, + transformations: [{ type: 'filter', config: { condition: 'active' }, continueOnError: false }], + schedule: { dialect: 'cron', source: '0 2 * * *' }, + }; + `, + 'parsed-destination': ` + import type { ETLDestinationParsed } from '@objectstack/spec/automation'; + const d: ETLDestinationParsed = { type: 'database', config: { table: 't' } }; + `, + 'parsed-transformation': ` + import type { ETLTransformationParsed } from '@objectstack/spec/automation'; + const t: ETLTransformationParsed = { type: 'map', config: {} }; + `, + 'parsed-run': ` + import type { ETLPipelineRunParsed } from '@objectstack/spec/automation'; + const r: ETLPipelineRunParsed = { + id: 'run-1', pipelineName: 'customer_360', status: 'succeeded', + startedAt: '2024-01-01T02:00:00Z', stats: {}, + }; + `, + } as const; + + const results = compileProbes(probes); + + it.each(['author-pipeline', 'author-source', 'author-destination', 'author-transformation', 'author-run'])( + '%s compiles with the defaulted keys left out', + (name) => { + expect(render(results.get(name)!)).toBe(''); + }, + ); + + it('rejects the same pipeline literal under `ETLPipelineParsed`', () => { + // The direction stated before it was run: under the PARSED alias the + // defaulted keys are facts about a document that has already been through + // `.parse()`, so omitting them is an error. Each missing key is named + // rather than asserting "some diagnostic", because a bare non-empty check + // would still pass if the alias were quietly pointed back at `z.input` and + // the literal broke for an unrelated reason. + const message = render(results.get('parsed-pipeline')!); + expect(message).toContain('writeMode'); + expect(message).toContain('continueOnError'); + // The transform half: a bare cron string is the input, never the output. + expect(message).toContain("Type 'string' is not assignable"); + }); + + it('requires `syncMode` and `enabled` on `ETLPipelineParsed` — the top-level defaults', () => { + const message = render(results.get('parsed-pipeline-top-level-only')!); + expect(message).toContain('TS2739'); + expect(message).toContain('syncMode'); + expect(message).toContain('enabled'); + // …and the author-shape twin of this exact literal is clean, which is what + // makes the red above a statement about the ANNOTATION and nothing else. + expect(render(results.get('author-pipeline')!)).toBe(''); + }); + + it('rejects the same destination / transformation / run literals under `*Parsed`', () => { + expect(render(results.get('parsed-destination')!)).toContain('writeMode'); + expect(render(results.get('parsed-transformation')!)).toContain('continueOnError'); + // `stats: {}` is complete under the input shape and missing four counters + // under the parsed one — the wire half follows the same rule as the seven + // authoring shapes, deliberately (see the alias note in `etl.zod.ts`). + expect(render(results.get('parsed-run')!)).toContain('recordsRead'); + }); + + it('accepts a bare cron string only on the author side', () => { + // `schedule` is the second half of why the old aliases did not compile: + // `CronExpressionInputSchema` is a transform, so `z.infer` is the + // `{ dialect, source }` envelope and the string every doc example writes + // was rejected. This is asserted through the pipeline probes above rather + // than a fourth pair, because `schedule` has no standalone ETL alias. + expect(render(results.get('author-pipeline')!)).toBe(''); + expect(render(results.get('parsed-pipeline')!)).toContain("Type 'string' is not assignable"); + }); +}); + +describe('[#4963] all nine aliases carry the pair', () => { + const NAMES = [ + 'ETLEndpointType', 'ETLSource', 'ETLDestination', 'ETLTransformationType', + 'ETLTransformation', 'ETLSyncMode', 'ETLPipeline', 'ETLRunStatus', 'ETLPipelineRun', + ] as const; + + it('exports `X` and `XParsed` from `@objectstack/spec/automation` for every one of them', () => { + const program = ts.createProgram([resolve(SPEC_DIR, 'src/automation/index.ts')], { + module: ts.ModuleKind.ESNext, + moduleResolution: ts.ModuleResolutionKind.Bundler, + skipLibCheck: true, + noEmit: true, + }); + const checker = program.getTypeChecker(); + const sf = program.getSourceFile(resolve(SPEC_DIR, 'src/automation/index.ts')); + const moduleSym = sf && checker.getSymbolAtLocation(sf); + // Without this the `toContain`s below would assert over an empty list. + expect(moduleSym, 'the ./automation barrel must resolve').toBeTruthy(); + const exported = checker.getExportsOfModule(moduleSym!).map((s) => s.getName()); + expect(exported.length).toBeGreaterThan(50); + + for (const name of NAMES) { + expect(exported, `${name} must still be exported`).toContain(name); + expect(exported, `${name}Parsed must exist — the pair is the convention`).toContain(`${name}Parsed`); + } + }); + + it('leaves the four enum aliases mutually assignable — the pair is a deliberate synonym', () => { + // Stated as a compile probe rather than a comment so that an enum which + // later gains a `.transform()` or `.catch()` turns this red instead of + // silently making `X` and `XParsed` disagree behind four identical-looking + // declarations. + const enums = ['ETLEndpointType', 'ETLTransformationType', 'ETLSyncMode', 'ETLRunStatus']; + const probes: Record = {}; + for (const name of enums) { + probes[`enum-${name}`] = [ + `import type { ${name}, ${name}Parsed } from '@objectstack/spec/automation';`, + `declare const a: ${name}; declare const b: ${name}Parsed;`, + `const toParsed: ${name}Parsed = a; const toInput: ${name} = b;`, + 'void toParsed; void toInput;', + ].join('\n'); + } + const results = compileProbes(probes); + for (const [name, diagnostics] of results) { + expect(render(diagnostics), `${name} must be assignable in both directions`).toBe(''); + } + }); +}); + +describe('[#4963] the ETL factories stopped working around their own return type', () => { + it('passes a bare cron string straight through instead of pre-wrapping it', () => { + // Pre-17 the helpers normalized `'0 * * * *'` into `{ dialect: 'cron', + // source }` because `z.infer` of `CronExpressionInputSchema` is the + // post-transform envelope and would not accept the string. Returning the + // AUTHOR shape moves that normalization back to where it belongs: the parse. + const pipeline = ETL.databaseSync({ + name: 'users_sync', sourceTable: 'src', destTable: 'dst', schedule: '0 * * * *', + }); + expect(pipeline.schedule).toBe('0 * * * *'); + + const parsed = ETLPipelineSchema.parse(pipeline); + expect(parsed.schedule).toEqual({ dialect: 'cron', source: '0 * * * *' }); + }); + + it('no longer spells out `enabled`, and the parse still supplies it', () => { + // `enabled: true` restated the schema's own default and existed only + // because `z.infer` made the key required. Dropping it must not change what + // a parsed pipeline says — that is the whole claim of the flip. + const pipeline = ETL.apiToDatabase({ name: 'api_ingest', apiConnector: 'stripe', destTable: 'payments' }); + expect(pipeline).not.toHaveProperty('enabled'); + expect(ETLPipelineSchema.parse(pipeline).enabled).toBe(true); + }); + + it('still states what each helper DECIDES, not what the schema defaults to', () => { + // The keys that survived are the ones carrying intent: the two helpers are + // a contrast (incremental+upsert vs full+append) and a reader must see it + // without looking up two defaults. + const sync = ETL.databaseSync({ name: 'users_sync', sourceTable: 'src', destTable: 'dst' }); + expect(sync.syncMode).toBe('incremental'); + expect(sync.destination.writeMode).toBe('upsert'); + const ingest = ETL.apiToDatabase({ name: 'api_ingest', apiConnector: 'stripe', destTable: 'payments' }); + expect(ingest.syncMode).toBe('full'); + expect(ingest.destination.writeMode).toBe('append'); + for (const p of [sync, ingest]) expect(ETLPipelineSchema.safeParse(p).success).toBe(true); + }); +}); diff --git a/packages/spec/src/automation/etl.zod.ts b/packages/spec/src/automation/etl.zod.ts index 7d00fea21f..65358afaf5 100644 --- a/packages/spec/src/automation/etl.zod.ts +++ b/packages/spec/src/automation/etl.zod.ts @@ -100,7 +100,46 @@ export const ETLEndpointTypeSchema = lazySchema(() => z.enum([ 'spreadsheet', // Google Sheets, Excel Online ])); -export type ETLEndpointType = z.infer; +// ─── `X` / `XParsed` — which shape the bare name means (#4963) ──────── +// +// House convention, stated once here because this file exports nine pairs of +// it: the **bare name is what an author writes** (`z.input` — defaults +// unapplied, every defaulted key optional), and `XParsed` is **what a parse +// returns** (`z.infer` — defaults applied, those same keys present). The +// clearest write-up is on `shared/retry-policy.zod.ts`; the sibling automation +// configs (`flow.zod.ts`, `io-node-config.zod.ts`, +// `builtin-node-config.zod.ts`, `control-flow.zod.ts`) all export the pair. +// +// Until 17 all nine aliases here were `z.infer` under the bare name with no +// `*Parsed` counterpart at all, and on this file that was not a style detail. +// Six keys across four shapes carry `.default()` — `ETLDestination.writeMode`, +// `ETLTransformation.continueOnError`, `ETLPipeline.syncMode` / `.enabled`, +// `ETLSource.incremental.enabled`, plus the whole `retry` block — and `schedule` +// is a `CronExpressionInputSchema` transform whose *output* is the `{ dialect, +// source }` envelope. Under `z.infer` all six were REQUIRED and a bare-string +// cron was rejected, so the one use this file actually has — +// `const p: ETLPipeline = { … }`, hand-written, which is the whole authoring +// door given there is no parse site in objectstack / objectui / cloud — did not +// compile. `packages/spec/docs/SYNC_ARCHITECTURE.md` carried three examples +// that proved it, and both ETL factories below were forced to spell defaults +// out and pre-wrap their cron just to satisfy their own return type. +// +// Flipping the bare names is breaking and was done in one step (#4963): the +// three-repo importer count was zero, so the migration surface is empty. A +// consumer that reads a PARSE RESULT — `const p = ETLPipelineSchema.parse(raw)` +// annotated by hand — renames its annotation to `ETLPipelineParsed`. +// +// The four enum aliases (`ETLEndpointType`, `ETLTransformationType`, +// `ETLSyncMode`, `ETLRunStatus`) get the pair too, even though `z.input` and +// `z.infer` are the same type for an enum. That is deliberate, not +// cargo-culting: the convention's value is that a reader never has to know +// WHICH of the nine has defaults before choosing an annotation, and a pair that +// exists today keeps costing nothing while an enum that later gains a +// `.transform()` or `.catch()` would otherwise reopen exactly this issue. + +export type ETLEndpointType = z.input; +/** @see {@link ETLEndpointType} — the enum has no transform, so this is the same type. */ +export type ETLEndpointTypeParsed = z.infer; // ─── Unknown-key strictness (#4001 批 12, ADR-0078) ─────────────────── // @@ -308,7 +347,10 @@ export const ETLSourceSchema = lazySchema(() => strictObject({ }).optional().describe('Incremental extraction config'), })); -export type ETLSource = z.infer; +/** What an author writes — `incremental.enabled` optional. */ +export type ETLSource = z.input; +/** The post-parse shape — `incremental.enabled` present. */ +export type ETLSourceParsed = z.infer; /** * ETL Destination Configuration @@ -349,7 +391,10 @@ export const ETLDestinationSchema = lazySchema(() => strictObject({ primaryKey: z.array(z.string()).optional().describe('Primary key fields'), })); -export type ETLDestination = z.infer; +/** What an author writes — `writeMode` optional (defaults to `append`). */ +export type ETLDestination = z.input; +/** The post-parse shape — `writeMode` present. */ +export type ETLDestinationParsed = z.infer; /** * ETL Transformation Type @@ -367,7 +412,9 @@ export const ETLTransformationTypeSchema = lazySchema(() => z.enum([ 'deduplicate', // Remove duplicates ])); -export type ETLTransformationType = z.infer; +export type ETLTransformationType = z.input; +/** @see {@link ETLTransformationType} — the enum has no transform, so this is the same type. */ +export type ETLTransformationTypeParsed = z.infer; /** * ETL Transformation Configuration @@ -408,7 +455,10 @@ export const ETLTransformationSchema = lazySchema(() => strictObject({ continueOnError: z.boolean().default(false).describe('Continue on error'), })); -export type ETLTransformation = z.infer; +/** What an author writes — `continueOnError` optional (defaults to `false`). */ +export type ETLTransformation = z.input; +/** The post-parse shape — `continueOnError` present. */ +export type ETLTransformationParsed = z.infer; /** * ETL Sync Mode @@ -419,7 +469,9 @@ export const ETLSyncModeSchema = lazySchema(() => z.enum([ 'cdc', // Change Data Capture - real-time streaming ])); -export type ETLSyncMode = z.infer; +export type ETLSyncMode = z.input; +/** @see {@link ETLSyncMode} — the enum has no transform, so this is the same type. */ +export type ETLSyncModeParsed = z.infer; /** * ETL Pipeline Schema @@ -567,7 +619,21 @@ export const ETLPipelineSchema = lazySchema(() => strictObject({ metadata: z.record(z.string(), z.unknown()).optional().describe('Custom metadata'), })); -export type ETLPipeline = z.infer; +/** + * What an author writes — the annotation for a hand-written pipeline literal. + * + * `syncMode`, `enabled`, `destination.writeMode`, each transformation's + * `continueOnError`, `source.incremental.enabled` and every key of `retry` are + * optional here, and `schedule` accepts the bare cron string + * (`'0 2 * * *'`) that `CronExpressionInputSchema` wraps at parse. + */ +export type ETLPipeline = z.input; +/** + * The post-parse shape — every defaulted key present and `schedule` normalized + * to its `{ dialect: 'cron', source }` envelope. Annotate the RESULT of + * `ETLPipelineSchema.parse(…)` with this, never the literal you pass in. + */ +export type ETLPipelineParsed = z.infer; /** * ETL Run Status @@ -581,7 +647,9 @@ export const ETLRunStatusSchema = lazySchema(() => z.enum([ 'timeout', // Timed out ])); -export type ETLRunStatus = z.infer; +export type ETLRunStatus = z.input; +/** @see {@link ETLRunStatus} — the enum has no transform, so this is the same type. */ +export type ETLRunStatusParsed = z.infer; /** * ETL Pipeline Run Result @@ -679,10 +747,47 @@ export const ETLPipelineRunSchema = lazySchema(() => z.object({ logs: z.array(z.string()).optional().describe('Execution logs'), })); -export type ETLPipelineRun = z.infer; +/** + * What a writer of a run result hands in — `stats`' four counters optional. + * + * A run result is engine-emitted, not authored (see the schema note above), so + * the pair here is about the READER's annotation, not an authoring door. It + * still follows the house convention rather than opting out, for the same + * reason `FlowVersionHistory` — the other wire shape in `automation/` — does: + * one rule for the whole namespace beats a per-shape exception nobody can + * predict from the outside. + */ +export type ETLPipelineRun = z.input; +/** The post-parse shape — `stats`' counters present, defaulted to 0. */ +export type ETLPipelineRunParsed = z.infer; /** - * Helper factory for creating ETL pipelines + * Helper factory for creating ETL pipelines. + * + * Both helpers return {@link ETLPipeline} — the AUTHOR shape. They construct a + * literal by hand, which is the same act an author performs, so the input type + * is the correct return type; the caller passes the result to + * `ETLPipelineSchema.parse` (or hands it to an engine that will) exactly as + * they would their own literal. + * + * The annotation is textually unchanged from pre-17 but its MEANING flipped + * with the alias (#4963), and that removed two workarounds these helpers were + * carrying purely to satisfy their own return type: + * + * - **`enabled: true` is gone from both.** It stated the schema's own default + * and was written only because `z.infer` made the key required. What each + * helper still spells out is what it actually DECIDES: + * `databaseSync` is `incremental` + `upsert` (neither is the default) and + * `apiToDatabase` is `full` + `append`. The second pair does coincide with + * the defaults, and is kept deliberately — the two helpers exist as a + * contrast, and a reader comparing them must be able to see which extraction + * and write posture each one picked without going to look up two defaults. + * - **`schedule` is passed straight through.** It used to be pre-wrapped + * (`typeof s === 'string' ? { dialect: 'cron', source: s } : s`) because + * `z.infer` of `CronExpressionInputSchema` is the post-transform envelope + * and would not accept the bare cron string these helpers advertise. The + * union IS the input type, so the normalization belongs where it always + * did — in the schema, at parse. */ export const ETL = { /** @@ -705,8 +810,7 @@ export const ETL = { writeMode: 'upsert', }, syncMode: 'incremental', - schedule: typeof params.schedule === 'string' ? { dialect: 'cron', source: params.schedule } : params.schedule, - enabled: true, + schedule: params.schedule, }), /** @@ -730,7 +834,6 @@ export const ETL = { writeMode: 'append', }, syncMode: 'full', - schedule: typeof params.schedule === 'string' ? { dialect: 'cron', source: params.schedule } : params.schedule, - enabled: true, + schedule: params.schedule, }), } as const; From c45ac973eb11d2c13d4ef7254801104bed60175c Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 5 Aug 2026 14:49:25 +0000 Subject: [PATCH 2/3] =?UTF-8?q?docs(spec):=20=E7=B2=BE=E7=A1=AE=E5=8C=96?= =?UTF-8?q?=20etl=20=E5=88=AB=E5=90=8D=E6=B3=A8=E9=87=8A=E4=B8=8E=20change?= =?UTF-8?q?set=20=E9=87=8C=E5=B8=A6=E9=BB=98=E8=AE=A4=E5=80=BC=E7=9A=84?= =?UTF-8?q?=E9=94=AE=E8=AE=A1=E6=95=B0?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 原文写「六个键」却列了五个具名键 + 一整个 retry 块,内部不自洽;改为逐类点名 (五个具名键 + retry 的五个 + stats 的四个),不再给一个含混的总数。 Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_018fxLGQdatPbBUvCgiVxg6D --- .changeset/etl-author-shape-aliases.md | 10 +++++----- packages/spec/src/automation/etl.zod.ts | 14 ++++++++------ 2 files changed, 13 insertions(+), 11 deletions(-) diff --git a/.changeset/etl-author-shape-aliases.md b/.changeset/etl-author-shape-aliases.md index 978fcee75c..52bfdf9322 100644 --- a/.changeset/etl-author-shape-aliases.md +++ b/.changeset/etl-author-shape-aliases.md @@ -7,12 +7,12 @@ convention** (#4963): the bare name is `z.input` — what an author writes — a `XParsed` is `z.infer` — what a parse returns. Until 17 all nine were `z.infer` under the bare name with no `*Parsed` counterpart at -all. On this file that was not a style detail. Six keys carry `.default()` +all. On this file that was not a style detail. Five named keys carry `.default()` (`ETLDestination.writeMode`, `ETLTransformation.continueOnError`, `ETLPipeline.syncMode` -/ `.enabled`, `ETLSource.incremental.enabled`, and every key of `ETLPipeline.retry`) and -`schedule` is a `CronExpressionInputSchema` transform whose *output* is the -`{ dialect, source }` envelope. Under `z.infer` all of them were REQUIRED and a bare -cron string was rejected — so the single use this file has, `const p: ETLPipeline = { … }` +/ `.enabled`, `ETLSource.incremental.enabled`), as does every key of +`ETLPipeline.retry` and of `ETLPipelineRun.stats`, and `schedule` is a +`CronExpressionInputSchema` transform whose *output* is the `{ dialect, source }` +envelope. Under `z.infer` all of them were REQUIRED and a bare cron string was rejected — so the single use this file has, `const p: ETLPipeline = { … }` written by hand, did not compile. That is the whole authoring door: `etl.zod.ts` has no parse site in objectstack / objectui / cloud, so the exported schema and the exported type are the only surface an author touches. diff --git a/packages/spec/src/automation/etl.zod.ts b/packages/spec/src/automation/etl.zod.ts index 65358afaf5..2986611f3c 100644 --- a/packages/spec/src/automation/etl.zod.ts +++ b/packages/spec/src/automation/etl.zod.ts @@ -112,12 +112,14 @@ export const ETLEndpointTypeSchema = lazySchema(() => z.enum([ // // Until 17 all nine aliases here were `z.infer` under the bare name with no // `*Parsed` counterpart at all, and on this file that was not a style detail. -// Six keys across four shapes carry `.default()` — `ETLDestination.writeMode`, -// `ETLTransformation.continueOnError`, `ETLPipeline.syncMode` / `.enabled`, -// `ETLSource.incremental.enabled`, plus the whole `retry` block — and `schedule` -// is a `CronExpressionInputSchema` transform whose *output* is the `{ dialect, -// source }` envelope. Under `z.infer` all six were REQUIRED and a bare-string -// cron was rejected, so the one use this file actually has — +// Five named keys across four shapes carry `.default()` — +// `ETLDestination.writeMode`, `ETLTransformation.continueOnError`, +// `ETLPipeline.syncMode` / `.enabled`, `ETLSource.incremental.enabled` — plus +// all five of `ETLPipeline.retry`'s (via `retryPolicyShape()`) and all four of +// `ETLPipelineRun.stats`'. And `schedule` is a `CronExpressionInputSchema` +// transform whose *output* is the `{ dialect, source }` envelope. Under +// `z.infer` every one of them was REQUIRED and a bare-string cron was +// rejected, so the one use this file actually has — // `const p: ETLPipeline = { … }`, hand-written, which is the whole authoring // door given there is no parse site in objectstack / objectui / cloud — did not // compile. `packages/spec/docs/SYNC_ARCHITECTURE.md` carried three examples From 5c0c5f956d0b41e0d63512388b670b6531c0f71a Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 5 Aug 2026 14:55:10 +0000 Subject: [PATCH 3/3] =?UTF-8?q?docs(spec):=20=E8=AF=B4=E6=B8=85=20L3=20?= =?UTF-8?q?=E5=9D=97=E8=A2=AB=E8=B7=B3=E8=BF=87=E7=9A=84=E7=9C=9F=E5=AE=9E?= =?UTF-8?q?=E7=90=86=E7=94=B1,=E5=B9=B6=E6=8C=87=E5=90=91=20#5515?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 原注释说三段 L3 块「两段用 ... 省略」,读起来像第三段没问题;实测第三段 (sapConnector)是完整字面量、报四条诊断,其中三条写的是 schema 会拒收的 键名/取值。已 file 为 #5515,注释直接点名,免得下一个读者以为那是已审阅的豁免。 Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_018fxLGQdatPbBUvCgiVxg6D --- .../spec/src/automation/etl-author-shape.test.ts | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/packages/spec/src/automation/etl-author-shape.test.ts b/packages/spec/src/automation/etl-author-shape.test.ts index d9faf0f33e..001dd8ef5d 100644 --- a/packages/spec/src/automation/etl-author-shape.test.ts +++ b/packages/spec/src/automation/etl-author-shape.test.ts @@ -112,10 +112,17 @@ describe('[#4963] SYNC_ARCHITECTURE.md pipeline examples compile', () => { // Anti-vacuity: a selector that matched nothing would make the compile // assertion below pass over an empty program — the way a gate goes dormant. expect(pipelineBlocks.length, 'ETLPipeline examples in SYNC_ARCHITECTURE.md').toBe(3); - // The other three are the L3 `Connector` examples. They are out of this - // gate's scope — they belong to `integration/connector.zod.ts`, and two of - // them deliberately elide with a bare `...`, which is not TypeScript. The - // total is pinned rather than left open so that ADDING a block to this + // The other three are the L3 `Connector` examples, out of this gate's scope + // because they belong to `integration/connector.zod.ts`. Two of them are + // Migration-Guide sketches that elide with a bare `...`, which is not + // TypeScript. The third — the full `sapConnector` example — is NOT exempt on + // its merits: run through this same harness it reports four diagnostics, and + // three of them are keys or values the schema REJECTS (`sourceField` / + // `targetField` for `source` / `target`, `transform.type: 'custom'`, + // `webhooks[].retryPolicy`). That is filed as #5515, not fixed here, because + // the fourth diagnostic is `Connector` being `z.infer` — this issue's twin + // on a file whose migration surface is NOT empty, so it needs its own ruling. + // The total is pinned rather than left open so that ADDING a block to this // document is a decision someone has to make on purpose: a new ETL example // is picked up automatically by the selector above, and anything else // turns this red until it is classified here.