feat(automation,migrate): os migrate meta --stored covers flow rows (#4454) - #4492
Merged
Merged
Conversation
… reusable off the load seam (#4454) `os migrate meta --stored` (#4327) covers every metadata type except `flow`, because flow-node conversions carry ADR-0078's open-namespace conflict guard and that needs the automation engine's live executor registry. This is the half both possible hosts need. `AutomationEngine.canonicalizeStoredFlow` is now the single policy, and `registerFlow` calls it — so the load seam and any stored-row migration can never disagree about what "canonical" means. It returns two shapes from one pass: - `parsed` for EXECUTION — FlowSchema.parse + the #4347 region pass, schema defaults materialized. What registerFlow runs. - `storable` for PERSISTENCE — conversions plus the `{dialect, source}` envelopes the schema derives for edge conditions, and deliberately nothing else. Excluding schema defaults is the load-bearing decision, and it was measured rather than assumed. Driving a pre-17 flow through all three steps shows parse + normalize REMOVE nothing (FlowSchema is strict since #4001, so an unknown key throws instead of being dropped — the graftNormalizedOperators precedent does not transfer) and ADD only defaults: `version`, `runAs`, per-edge `type` / `isDefault`. Persisting a default the author never wrote would pin every migrated row to today's value while untouched rows follow tomorrow's — two populations with different behaviour, which is the drift this pass exists to remove. A migration must not become a source of it. `migrateStoredMetadata` gains an optional `canonicalizeFlow` hook. Without it, flow rows keep reporting `skipped` with the reason. With it: conversions are applied and reported per row, a refused rename (the guard firing over a live third-party node type) fails the row loudly naming the token, and a flow that cannot canonicalize at all is reported rather than persisted as a guess. One subtlety the tests pin: the condition envelope is a schema transform, not a conversion, so it emits NO notice while still changing the body. Reading notices alone — correct for every other type — would call such a row canonical and leave it re-deriving on every boot. Both passes are copy-on-write, so identity (`storable === body`) is the exact test for flows. No host is wired yet: booting the automation plugin in the CLI would arm triggers and schedulers (registerFlow activates them), so that needs either an inert plugin mode or the admin route. Tracked in #4454. Refs #4327, #3903, #4001, #4347, ADR-0078, ADR-0087 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WoZPKPDqJ7WB7z84xk9y3f
…4454) The stored-metadata pass (#4327) skipped `flow` — the one type where the most stored dialect actually lives, since the graduated conversions `flow-node-crud-filter-alias`, `-object-alias`, `-notify-config-aliases` and `-script-config-aliases` are all flow-node entries. Flow-node conversions carry ADR-0078's open-namespace conflict guard, which must consult the LIVE executor registry to tell a rename from a clobber, and the metadata layer cannot obtain one. This wires the engine in without wiring in a server. `AutomationServicePluginOptions.armRuntime` (default true — every existing host is unaffected). With `false` the plugin brings up the engine and the COMPLETE node registry — built-ins plus whatever `automation:ready` contributes, because a partial registry would make the guard read a live custom node type as unowned and rewrite over it — then stops before anything is armed: - no flow registered (registerFlow calls activateFlowTrigger, so record triggers and scheduled jobs would go live); - no kernel:ready / metadata:reloaded re-sync (skipping only the boot pull would arm them a moment later); - no declarative connector materialized (an MCP provider spawns a process); - no suspended wait-timer re-armed (it RESUMES paused runs — a migration that silently continues someone's approval is indefensible). `os migrate meta --stored` boots it in that mode and passes `canonicalizeStoredFlow` as the protocol's `canonicalizeFlow` hook. A migration process must not become a second server. `IAutomationService.canonicalizeStoredFlow` is declared on the contract rather than only on the implementation, because the CLI consumes it through the `automation` slot — which is exactly what the slot-lookup lint rule asks for instead of erasing the lookup to `any`. Verified against a real database, not only unit tests: a legacy flow row (`config.filters`) seeded into examples/app-todo migrates preview(exit 1) → apply → re-run(exit 0), the row comes back with `config.filter`, a `sys_metadata_history` entry sourced `migrate-stored`, and — the load-bearing part — NO schema defaults written (`version`, `runAs`, per-edge `type` are absent), so the row is not frozen on today's default values. Boot logs confirm 17 executors registered and then "inert mode … no trigger or schedule armed". Refs #4327, #3903, #4001, #4347, ADR-0078, ADR-0087 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WoZPKPDqJ7WB7z84xk9y3f
…tion-stored-rewrite-jxlz2k
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
github-actions
Bot
added
documentation
Contributor
📓 Docs Drift CheckThis PR changes 4 package(s): 113 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:
|
…tion-stored-rewrite-jxlz2k
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #4454. Follow-up to #4327 (merged as #4464).
Why
os migrate meta --storedgave the stored-metadata conversion chain a finish line for every type exceptflow— which is the type where the most stored dialect actually lives: the graduated conversionsflow-node-crud-filter-alias,flow-node-crud-object-alias,flow-node-notify-config-aliasesandflow-node-script-config-aliasesare all flow-node entries. Flow-node conversions carry ADR-0078's open-namespace conflict guard, which must consult the live executor registry to tell a rename from a clobber, and the metadata layer has no way to obtain one. So flows were reportedskipped.This wires the engine in — without wiring in a server.
Two findings that shaped the design, both measured rather than assumed
1. "Canonical flow" is three steps, and writing back the third would fossilize defaults.
registerFlowrunsapplyConversionsToFlow→FlowSchema.parse→normalizeControlFlowRegions. Driving a pre-17 flow (legacyfiltersat top level and inside aloopbody, bare-string edge conditions in both) through all three:version,runAs, per-edgetype/isDefault— all schema defaultscondition{dialect:'cel', source}This corrected my own earlier analysis on the issue. The
graftNormalizedOperatorsprecedent does not transfer: it exists because the view parse strips Studio-only auxiliary keys, butFlowSchemais strict since #4001 — an unrecognized key throws rather than being dropped (verified with_uiPosition). There is nothing for a graft to protect. The real hazard runs the other way: persisting a default the author never wrote pins that row to today's value while untouched rows follow tomorrow's — two populations with different behaviour, which is exactly the drift this pass exists to remove. A migration must not become a source of the drift it cleans up.So
canonicalizeStoredFlowreturns two shapes from one pass:parsedfor execution (defaults materialized — whatregisterFlowruns) andstorablefor persistence (conversions +conditionenvelopes, no defaults).2. The CLI could not host this as-is.
AutomationServicePluginlooks cheap to boot, butregisterFlowcallsactivateFlowTrigger, andstart()pulls and registers every flow. Booting it for a one-shot migration would arm record triggers and scheduled jobs, materialize declarative connectors (an MCP provider spawns a child process), and re-arm suspended wait timers — which resumes paused runs. A migration that silently continues someone's pending approval is indefensible.What changed
AutomationServicePluginOptions.armRuntime(defaulttrue; every server, dev stack and test host is unaffected). Withfalse, the plugin brings up the engine and the complete node registry — built-ins plus whateverautomation:readycontributes, because a partial registry would make the guard read a live custom node type as unowned and rewrite over it — then stops before anything is armed. The early return is placed deliberately afterautomation:readyfor that reason.armRuntime: falsekernel:ready/metadata:reloadedre-syncregisterFlowarms triggers; skipping only the boot pull would arm them a moment laterAutomationEngine.canonicalizeStoredFlowis now the single policy andregisterFlowcalls it, so the load seam and the migration can never disagree about what canonical means.migrateStoredMetadatagains thecanonicalizeFlowhook;os migrate meta --storedboots the plugin inert and passes it.IAutomationService.canonicalizeStoredFlowis declared on the contract, not only the implementation — the CLI consumes it through theautomationslot, and that is precisely what theslot-lookup/no-any-assignmentrule asks for instead of erasing the lookup toany. (The rule caught my first attempt; the fix was to state the contract, not to silence it.)One subtlety the tests pin
The
conditionenvelope is a schema transform, not a conversion, so it emits no notice while still changing the body. Reading notices alone — correct for every other metadata type — would call such a row canonical and leave it re-deriving on every boot. Both passes are copy-on-write, so identity (storable === body) is the exact test for flows.Failure modes are loud, never silent
failed, naming the token and its owner. Never a skip, never a clobber.failedwith the parse message. Such a row cannot register today either, so the report is telling you about a flow already broken at runtime.Verification
canonicalize-stored-flow.test.ts(both nesting depths; defaults excluded whileparsedkeeps them; idempotent; node-configpredicate untouched; strict-key throw;registerFlowunchanged), 7 ininert-mode.test.ts(nothing armed,automation:readystill fires, registry populated, and the default path still arms — the control), plus 6 flow-hook cases inprotocol.stored-migration.test.tsincluding the no-notice envelope case and the conflict case.spec·service-automation·metadata-protocol·cli·objectql·runtime·plugin-approvalsall green after mergingmain(which carried 6 commits, three of them breaking!refactors touching these packages).pnpm lint,tsc --noEmit, and spec'sapi-surface/generated(all 8 artifacts) /exported-anygates pass.examples/app-todo→ preview exits 1 naming the conversion →--apply→ re-run exits 0. The row comes back withconfig.filter, asys_metadata_historyentry sourcedmigrate-stored, and no schema defaults written. Boot logs confirm 17 executors registered, theninert mode … no trigger or schedule armed.Docs: the
--storedsection incli.mdxno longer claims flows are uncovered, and ADR-0087 gains a second 2026-08-01 addendum recording all three decisions. Per the newly-added CLAUDE.md rule, this PR does not touchcontent/docs/releases/— the changeset carries the release-notes input.🤖 Generated with Claude Code
https://claude.ai/code/session_01WoZPKPDqJ7WB7z84xk9y3f
Generated by Claude Code