Skip to content

fix(automation,approvals): 审批决策不能在流程原地不动的情况下"成功" (#4420) - #4460

Merged
os-zhuang merged 5 commits into
mainfrom
claude/approval-decision-restart-bug-slcmhe
Aug 1, 2026
Merged

fix(automation,approvals): 审批决策不能在流程原地不动的情况下"成功" (#4420)#4460
os-zhuang merged 5 commits into
mainfrom
claude/approval-decision-restart-bug-slcmhe

Conversation

@os-zhuang

Copy link
Copy Markdown
Contributor

修复 #4420

现象

流程停在 approval 节点 → 发版重启 → 审批人点「批准」:请求行翻成 approved、UI 提示成功,但流程一步没动。下一级审批请求不创建、记录上镜像的状态冻结在半路、全链路零报错。审批流天然一挂就是几天,中间隔一次发版是常态而非边缘情况——也就是说每次发版都可能把在途审批悄悄变成僵尸,而审批人侧毫无感知。

定位:#1518 并不是缺失的那一环

issue 推断 #1518(挂起 run 持久化)没落地。实际查下来它已经落地并接好了suspended-run-store.tssys-automation-run.object.ts、引擎的冷启动 rehydrate、幂等 resume 都在,suspendedRuns 只是热缓存而不是唯一存储。真正造成该现象的是另外两个缺陷,两者叠加正好复现报告的全部症状。

一、装配漏洞:store 可以挂在一张根本没建的表上

对象注册在 init() 里走 manifest,store 激活在 start() 里只看 objectql——两个不同的服务、两个不同的生命周期阶段,而插件没有声明任何顺序。当它排在 ObjectQL 之前时:init() 拿不到 manifest,warn 一句继续走;start() 照样把 DB store 装上。此后每次挂起都以 no such table: sys_automation_run 失败在一条没人看的 warn 里,run 静默退回内存,下次重启全丢。这正是 issue 里「dist 有 sys_automation_run 标识符、运行库里却没这张表」的来源。

修复:

  • 声明 optionalDependencies: ['com.objectstack.engine.objectql'](ADR-0116 的 order-if-present)。不用 requiresServices——assertInitServiceRequirements 是无条件抛错,会硬失败所有不带 objectql 的组合(packages/verify/src/harness.ts 的 memory 模式、各 LiteKernel 单测、connector 插件)。这个插件没有引擎时确实能跑(纯内存挂起),所以只能是软边。
  • init 阶段漏掉的注册,在 start 阶段补一次(仍早于 ObjectQL 自己 start() 里的 schema sync,因此表照常创建)。
  • 注册没成功就绝不挂 store,并按 error 级别写明降级后果与修法。降级到内存是合法模式;一边降级一边报告「持久化已启用」不是。
  • 启动时探一次表,让 no such table 在启动时暴露一次,而不是运行期每次挂起各失败一次。
  • 挂起写盘失败从 warn 升为 error——一个本该持久的暂停悄悄只留在内存里,是尚未发生的数据丢失。

取舍说明:探测最初设计成「探不到就不挂 store」,但集成测试暴露出驱动可能在 bootstrap 之后才注册,那样会把本来可用的持久化关掉、反而制造出本 issue 的 bug。所以探测只作诊断(照常挂载 + error 日志),真正的门禁是「注册是否成功」这个确定性信号

二、resume 失败在每一层都被读成成功

AutomationEngine.resume() 对丢失的 run 是返回 { success: false } 而非抛错;ApprovalService.serviceResume() 把返回值丢掉;decide() 只把抛错算失败。于是"run 已经没了"这件事一路返回 resumed: true + HTTP 200。而死信清扫 releaseDeadRunRequests 只扫 pending 的请求,已翻成 approved 的僵尸对它完全不可见——没有任何一层看得见它。

修复:

  • resume 失败分类:新增 RUN_NOT_FOUND / STORE_UNAVAILABLE / RESUME_IN_PROGRESS(与既有 PERMISSION_DENIED / INVALID_SIGNAL 并列)。"彻底没了"和"库暂时读不到"必须可区分——两者补救方式相反,一次数据库抖动被读成死 run 就会永久废掉一个活着的 run。raw resume 路由相应映射 404 / 503 / 409。
  • 新增 AutomationEngine.hasSuspendedRun(runId):读挂起存储(getRun() 做不到——它读的是执行日志,重启后对活着的 run 也返回 null),库读不到时抛错而不是答 false
  • 审批侧用它在 decide / sendBack / resubmit 写第一条记录之前预检。于是"已审批 + run 悬空"的半状态不是被报告出来,而是压根不会被创建:抛 RESUME_TARGET_LOST(409),请求仍是 pending、仍可操作,审计里也不会留下一条没有生效的批准。
  • 决策已落库之后才失败的竞态无法回滚,但会抛 RESUME_FAILED(500) 并点名那个悬空的 run,而不是报成功。
  • 并发重复 resume 仍算良性(引擎的幂等保护正在生效),通过新增的可选 resumeError 字段返回。
  • 撤回类操作(recall、修订窗口内的 cancelRun)保持非致命——用户本就在放弃这个请求,本地终结和记录解锁必须成功——但改为 error 日志带原因,不再吞掉。
  • 未挂 automation 的组合、以及引擎没有 hasSuspendedRun 的情况,行为完全不变。

行为变更(供 API 消费方注意)

decide / sendBack / resubmit 现在会在 run 不可恢复时抛错(RESUME_TARGET_LOST → 409,RESUME_FAILED → 500),而不再返回 resumed: true。这正是 issue 的第 2 条期望(「approve 接口应失败或至少告警,不能落一半状态静默成功」)。Console 会显示 409 的说明文案。

测试

新增 packages/plugins/plugin-approvals/src/approval-restart-resume.test.ts——其中「进程 A 挂起 → 进程 B 审批」既是 #4420 的复现,也是 #1518 第一次真正穿过审批链路的验收(此前只有引擎侧自测):

  • 共享持久化 store ⇒ 决策成功、流程沿 approve 分支推进、请求变 approved
  • 状态丢失 ⇒ RESUME_TARGET_LOST,请求仍为 pending、无 approve 审计行、下游节点未执行;
  • 决策后竞态 ⇒ RESUME_FAILED 并点名 run;重复 resume ⇒ 良性,resumeError 有值;
  • store 抖动 ⇒ 不阻塞决策(故意 fail-open);无引擎组合 ⇒ 行为不变。

新增 packages/services/service-automation/src/plugin-suspended-run-wiring.test.ts 覆盖装配的五种组合(正常 / 漏洞 / manifest 晚到 / 探测失败 / memory 模式),外加引擎侧的失败码与 hasSuspendedRun 语义测试。

本地:service-automation 581、plugin-approvals 341、spec 7228、rest 539、runtime 1023 全绿,pnpm typecheck 122/122、pnpm lintcheck:error-code-casingcheck:service-providers 均通过;依赖包(triggers / connectors / verify / cli / objectql)也已跑过。

不在本次范围

已有部署里存量的僵尸请求(已 approved、run 悬空)本次不修复——releaseDeadRunRequests 只扫 pending,需要单独一个管理端清扫。本 PR 只保证不再产生新的。


Generated by Claude Code

claude added 3 commits August 1, 2026 08:00
… while its flow stays parked (#4420)

A flow paused at an `approval` node, a deploy, then an approver clicking
Approve: the request row flipped to `approved`, the UI toasted success — and
the flow never moved. No next-stage request, no error, the record's mirrored
status frozen mid-workflow. Approval flows pause for days by design, so a
restart mid-flight is the normal case, not the edge one.

Durable suspended runs (#1518) had shipped and were not the missing piece.
Two other things were.

The wiring could enable a store over a table nobody had created. Object
registration and store activation resolve different services in different
phases — `manifest` at init(), `objectql` at start() — and the plugin declared
no ordering. Composed ahead of ObjectQL, init() found no `manifest`, warned,
and continued; start() attached the DB-backed store anyway. Every suspend then
failed with `no such table: sys_automation_run` into a log line nobody read.
Now: `optionalDependencies: ['com.objectstack.engine.objectql']` (order-if-
present per ADR-0116 — an engine-less kernel must still boot), a missed
registration retried at start() while it still precedes schema sync, no store
attached when registration did not happen (at error level), a boot-time table
probe, and a failed durable write logged at error rather than warn.

A reported resume failure read as success. `AutomationEngine.resume()` answers
a lost run by RETURNING `{ success: false }`, never by throwing; approvals
discarded that value and counted only throws as failure. Resume failures are
now classified — RUN_NOT_FOUND, STORE_UNAVAILABLE, RESUME_IN_PROGRESS —
so a run gone for good is distinguishable from a store merely unreachable,
and the raw resume route maps them to 404 / 503 / 409.

Approvals acts on them. The new `hasSuspendedRun(runId)` — which reads the
suspension store, unlike `getRun()`, and throws rather than answering false
when the store is unreadable — pre-flights decide / sendBack / resubmit BEFORE
their first write, so the zombie half-state is never created rather than
merely reported: RESUME_TARGET_LOST (409), request left actionable. A resume
that fails after the decision is durable throws RESUME_FAILED (500) naming the
stranded run. A concurrent duplicate stays benign via the new `resumeError`
field; recall and revise-window cancellation stay non-fatal but log at error.
Compositions with no automation engine attached are unaffected.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EPRaZTNbACfvGKytYESX2h
@vercel

vercel Bot commented Aug 1, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
objectstack Ignored Ignored Aug 1, 2026 8:53am

Request Review

@github-actions github-actions Bot added size/xl documentation Improvements or additions to documentation tests tooling labels Aug 1, 2026
@github-actions

github-actions Bot commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 5 package(s): @objectstack/plugin-approvals, @objectstack/rest, @objectstack/runtime, @objectstack/service-automation, @objectstack/spec.

115 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:

  • content/docs/ai/agents.mdx (via @objectstack/spec)
  • content/docs/ai/connect-mcp.mdx (via @objectstack/rest)
  • content/docs/ai/skills-reference.mdx (via @objectstack/spec)
  • content/docs/ai/skills.mdx (via @objectstack/spec)
  • content/docs/api/client-sdk.mdx (via packages/runtime, @objectstack/spec)
  • content/docs/api/environment-routing.mdx (via @objectstack/spec)
  • content/docs/api/error-catalog.mdx (via @objectstack/spec)
  • content/docs/api/error-handling-client.mdx (via @objectstack/spec)
  • content/docs/api/error-handling-server.mdx (via @objectstack/rest, @objectstack/spec)
  • content/docs/api/index.mdx (via @objectstack/rest, @objectstack/runtime, @objectstack/spec)
  • content/docs/api/wire-format.mdx (via @objectstack/runtime)
  • content/docs/automation/approvals.mdx (via @objectstack/plugin-approvals, @objectstack/spec)
  • content/docs/automation/connectors.mdx (via @objectstack/spec)
  • content/docs/automation/flows.mdx (via @objectstack/service-automation, @objectstack/spec)
  • content/docs/automation/hook-bodies.mdx (via @objectstack/runtime, packages/spec)
  • content/docs/automation/hooks.mdx (via @objectstack/spec)
  • content/docs/automation/index.mdx (via @objectstack/spec)
  • content/docs/automation/webhooks.mdx (via @objectstack/spec)
  • content/docs/automation/workflows.mdx (via @objectstack/spec)
  • content/docs/concepts/architecture.mdx (via @objectstack/spec)
  • content/docs/concepts/design-principles.mdx (via packages/spec)
  • content/docs/concepts/index.mdx (via @objectstack/spec)
  • content/docs/concepts/metadata-driven.mdx (via @objectstack/spec)
  • content/docs/concepts/metadata-lifecycle.mdx (via @objectstack/runtime, packages/spec)
  • content/docs/concepts/north-star.mdx (via packages/runtime, packages/spec)
  • content/docs/data-modeling/analytics.mdx (via @objectstack/spec)
  • content/docs/data-modeling/drivers.mdx (via @objectstack/runtime, @objectstack/spec)
  • content/docs/data-modeling/external-datasources.mdx (via @objectstack/spec)
  • content/docs/data-modeling/field-types.mdx (via @objectstack/spec)
  • content/docs/data-modeling/fields.mdx (via @objectstack/spec)
  • content/docs/data-modeling/formulas.mdx (via @objectstack/spec)
  • content/docs/data-modeling/index.mdx (via @objectstack/spec)
  • content/docs/data-modeling/objects.mdx (via @objectstack/spec)
  • content/docs/data-modeling/queries.mdx (via @objectstack/spec)
  • content/docs/data-modeling/schema-design.mdx (via @objectstack/spec)
  • content/docs/data-modeling/seed-data.mdx (via @objectstack/spec)
  • content/docs/data-modeling/validation-rules.mdx (via @objectstack/spec)
  • content/docs/data-modeling/validation.mdx (via @objectstack/spec)
  • content/docs/deployment/cli.mdx (via @objectstack/spec)
  • content/docs/deployment/index.mdx (via @objectstack/runtime)
  • content/docs/deployment/production-readiness.mdx (via @objectstack/runtime)
  • content/docs/deployment/single-project-mode.mdx (via @objectstack/runtime)
  • content/docs/deployment/troubleshooting.mdx (via @objectstack/spec)
  • content/docs/deployment/validating-metadata.mdx (via @objectstack/spec)
  • content/docs/deployment/vercel.mdx (via @objectstack/runtime)
  • content/docs/getting-started/build-with-claude-code.mdx (via @objectstack/spec)
  • content/docs/getting-started/common-patterns.mdx (via @objectstack/spec)
  • content/docs/getting-started/examples.mdx (via @objectstack/spec)
  • content/docs/getting-started/quick-reference.mdx (via @objectstack/spec)
  • content/docs/getting-started/quick-start.mdx (via @objectstack/spec)
  • content/docs/getting-started/your-first-project.mdx (via @objectstack/runtime, @objectstack/spec)
  • content/docs/kernel/cluster.mdx (via @objectstack/runtime, @objectstack/spec)
  • content/docs/kernel/contracts/auth-service.mdx (via packages/spec)
  • content/docs/kernel/contracts/cache-service.mdx (via packages/spec)
  • content/docs/kernel/contracts/data-engine.mdx (via @objectstack/spec)
  • content/docs/kernel/contracts/index.mdx (via @objectstack/spec)
  • content/docs/kernel/contracts/metadata-service.mdx (via packages/spec)
  • content/docs/kernel/contracts/storage-service.mdx (via packages/spec)
  • content/docs/kernel/index.mdx (via packages/spec)
  • content/docs/kernel/runtime-services/email-service.mdx (via packages/spec)
  • content/docs/kernel/runtime-services/index.mdx (via packages/spec)
  • content/docs/kernel/runtime-services/queue-service.mdx (via packages/spec)
  • content/docs/kernel/runtime-services/sharing-service.mdx (via packages/spec)
  • content/docs/kernel/runtime-services/sms-service.mdx (via packages/spec)
  • content/docs/kernel/runtime-services/storage-service.mdx (via packages/spec)
  • content/docs/kernel/services-checklist.mdx (via @objectstack/plugin-approvals, @objectstack/service-automation, @objectstack/spec)
  • content/docs/kernel/services.mdx (via @objectstack/spec)
  • content/docs/permissions/authentication.mdx (via @objectstack/rest, @objectstack/runtime)
  • content/docs/permissions/authorization.mdx (via packages/runtime, @objectstack/spec)
  • content/docs/permissions/permission-sets.mdx (via @objectstack/spec)
  • content/docs/permissions/permissions-matrix.mdx (via @objectstack/spec)
  • content/docs/permissions/positions.mdx (via @objectstack/spec)
  • content/docs/permissions/rls.mdx (via @objectstack/spec)
  • content/docs/permissions/sharing-rules.mdx (via @objectstack/spec)
  • content/docs/plugins/adding-a-metadata-type.mdx (via @objectstack/spec)
  • content/docs/plugins/development.mdx (via @objectstack/spec)
  • content/docs/plugins/index.mdx (via @objectstack/rest, @objectstack/spec)
  • content/docs/plugins/packages.mdx (via @objectstack/plugin-approvals, @objectstack/rest, @objectstack/runtime, @objectstack/service-automation, @objectstack/spec)
  • content/docs/protocol/backward-compatibility.mdx (via @objectstack/spec)
  • content/docs/protocol/diagram.mdx (via packages/spec)
  • content/docs/protocol/kernel/config-resolution.mdx (via @objectstack/spec)
  • content/docs/protocol/kernel/http-protocol.mdx (via @objectstack/runtime)
  • content/docs/protocol/kernel/i18n-standard.mdx (via packages/rest, @objectstack/spec)
  • content/docs/protocol/kernel/index.mdx (via @objectstack/runtime, @objectstack/spec)
  • content/docs/protocol/kernel/lifecycle.mdx (via @objectstack/runtime, @objectstack/spec)
  • content/docs/protocol/kernel/plugin-spec.mdx (via @objectstack/spec)
  • content/docs/protocol/kernel/runtime-capabilities.mdx (via @objectstack/spec)
  • content/docs/protocol/knowledge.mdx (via @objectstack/spec)
  • content/docs/protocol/objectql/index.mdx (via @objectstack/spec)
  • content/docs/protocol/objectql/query-syntax.mdx (via @objectstack/spec)
  • content/docs/protocol/objectql/schema.mdx (via @objectstack/spec)
  • content/docs/protocol/objectql/security.mdx (via packages/spec)
  • content/docs/protocol/objectql/state-machine.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/actions.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/concept.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/index.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/layout-dsl.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/record-alert.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/widget-contract.mdx (via @objectstack/spec)
  • content/docs/releases/implementation-status.mdx (via @objectstack/plugin-approvals, @objectstack/rest, @objectstack/runtime, @objectstack/service-automation, @objectstack/spec)
  • content/docs/releases/index.mdx (via @objectstack/spec)
  • content/docs/releases/v12.mdx (via @objectstack/rest, @objectstack/spec)
  • content/docs/releases/v13.mdx (via @objectstack/spec)
  • content/docs/releases/v16.mdx (via @objectstack/spec)
  • content/docs/releases/v17.mdx (via @objectstack/rest, @objectstack/runtime, @objectstack/spec)
  • content/docs/releases/v9.mdx (via @objectstack/plugin-approvals, @objectstack/service-automation, @objectstack/spec)
  • content/docs/ui/actions.mdx (via @objectstack/spec)
  • content/docs/ui/create-vs-edit-form.mdx (via @objectstack/spec)
  • content/docs/ui/dashboards.mdx (via @objectstack/spec)
  • content/docs/ui/forms.mdx (via @objectstack/spec)
  • content/docs/ui/index.mdx (via @objectstack/spec)
  • content/docs/ui/public-data-collection.mdx (via @objectstack/spec)
  • content/docs/ui/setup-app.mdx (via @objectstack/spec)
  • content/docs/ui/translations.mdx (via @objectstack/spec)
  • content/docs/ui/views.mdx (via @objectstack/spec)

Advisory only. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs origin/main → pass the list as args.docs.

claude added 2 commits August 1, 2026 08:25
…es (#4420)

`check:docs` compares `content/docs/references/` against what
`gen:schema && gen:docs` produces. The five codes this branch registers in the
ADR-0112 ledger — RUN_NOT_FOUND, STORE_UNAVAILABLE, RESUME_IN_PROGRESS,
RESUME_TARGET_LOST, RESUME_FAILED — feed `ErrorCode`, which the generator
inlines into every response-envelope table. Hence eleven files, all of it the
same union widening: the ledger page gains the five entries and every page
that renders an ApiError picks them up.

Generated, not hand-edited:
  pnpm --filter @objectstack/spec gen:schema && pnpm --filter @objectstack/spec gen:docs

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EPRaZTNbACfvGKytYESX2h
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/xl tests tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants