Skip to content

fix(objectql,cli): 巡检随引擎一起停 —— unreadableObjects 不再被关停污染 (#4747) - #4815

Merged
os-zhuang merged 3 commits into
mainfrom
claude/issue-4747-integrity-audit-shutdown
Aug 3, 2026
Merged

fix(objectql,cli): 巡检随引擎一起停 —— unreadableObjects 不再被关停污染 (#4747)#4815
os-zhuang merged 3 commits into
mainfrom
claude/issue-4747-integrity-audit-shutdown

Conversation

@os-zhuang

Copy link
Copy Markdown
Contributor

Fixes #4747

采纳 issue 与 PM 裁定的 A + B:A 修时序,B 保证即使时序再次被打破,unreadableObjects 也不会被非发现污染。单独的 C(一次性命令不跑巡检)没有采用 —— 见下面「巡检在 CLI 场景仍然运行」一节。

根因:两处静默空转叠在一起

现场比 issue 猜的更基础一层。两个「看起来做了、其实没做」的调用:

  1. ObjectQLPlugin 把关停逻辑写在 stop() 里,而内核根本不调 stop()
    插件契约是 init / start / destroy(packages/core/src/types.ts),performShutdown() 只遍历 plugin.destroy。这件事仓里已经写明过了 —— DefaultDatasourcePlugin.destroy 的注释一字不差:

    Note the kernel's teardown phase is destroy()stop() exists nowhere in the Plugin contract and is never called.

    于是那行唯一会解除 ADR-0057 巡检定时器的 this.lifecycleService?.stop(),在任何宿主上都没有跑过,os serve 收 SIGTERM 时也一样。

  2. bootSchemaStack().shutdown() 调的是 (runtime as any).stop?.(),而 Runtime 没有 stop
    一个强制转型加一个 ?.,把「没有关停」伪装成了「关停过了」:内核状态始终是 running、没有任何插件收到 destroy(),而同一个函数的下一行已经把 driver 断开了。

叠起来的结果就是 issue 里那三行:启动 60 秒后巡检在仍然活着的进程里醒来,读一个自己宿主早已断开的连接池。scanned: 0,两个对象进 unreadableObjects

改了什么

文件 改动
packages/objectql/src/plugin.ts stopdestroy(内核真正调用的钩子)
packages/cli/src/utils/schema-migrate.ts kernel.shutdown() —— 与 os serve 收到 SIGTERM 时同一条路径;显式 disconnect() 保留为兜底
packages/objectql/src/lifecycle/lifecycle-service.ts stop() 不只清定时器:它把「引擎正在拆」这一位交给飞行中的 sweep,并在每个 leg 边界检查;停掉之后不再开始新 sweep。新增 get stopped
packages/objectql/src/integrity/dangling-reference-audit.ts 新增 signal;读之前先问「还算数吗」;因关停而失败的读被丢弃而不是归档;新增 DanglingReferenceReport.aborted

engine.tssrc/validation/** 零改动(#4794 的车道):inspectDanglingReferences 本来就原样转发 options,signal 顺着已有的通道流下去。

B 为什么不是字符串匹配

区分「引擎正在关停」与「数据源不可达」的信号是显式传下去的一位,不是对 Unable to acquire a connection 做消息嗅探。后者正是 Prime Directive #12 说的消费端宽容:它会把一个错误文案变成事实上的第二份契约,换个 driver 就失灵。LifecycleServiceOptions.referenceAudit 的类型也顺手收成了 Omit< DanglingReferenceAuditOptions, 'signal' > —— 巡检的寿命就是这个 service 的寿命,不该有第二个人来回答「现在还能读吗」。

aborted 是独立字段而不是 unreadableObjects 里的一条:「没人问」不是「数据源拒绝了」。它让「这次没跑完」依然是响的(报告永远不会被读成 clean bill of health),但不占用只该装发现的那个桶。

验收

1. 成功运行里不再有噪声 —— 真实 CLI,不是测试替身

修前(examples/app-crm,migrate recorded-by --json):

2026-08-03T07:30:14.388Z ERROR Find operation failed {"object":"sys_metadata", …}
2026-08-03T07:30:14.388Z WARN  [integrity] dangling-reference audit could not list an object …
2026-08-03T07:30:14.389Z ERROR Find operation failed {"object":"sys_view_definition", …}
2026-08-03T07:30:14.389Z WARN  [integrity] stored references that resolve to nothing (#4551)
      {"scanned":0,…,"unreadableObjects":["sys_metadata","sys_view_definition"],…}

修后,同一条命令:

{ "planId": "metadata.recorded-by-sentinel-to-null", "sentinel": "system", "pending": 0, "applied": false }
2026-08-03T07:54:21.546Z INFO Graceful shutdown started
2026-08-03T07:54:21.556Z INFO ✅ Graceful shutdown complete

grep -c "Find operation failed"0;grep -c integrity0;stderr 全空。注意 Graceful shutdown 这两行本身就是证据:关停第一次真的发生了。

2. 真正读不出来的对象照旧进桶 —— 两种情况被同一对测试钉死

dangling-reference-audit.test.ts 里新增的 describe 以这一对开头,因为修之前它们无法区分:

  • a REAL datasource fault still lands in unreadableObjects, loudly — 列表失败、没有 abort ⇒ unreadableObjects: ['…']aborted: false、warn 照发。
  • the SAME failure, raced by a teardown, is dropped instead of filed — 同一段抛错,只是 signal 在 find 中途翻转(连接池就是这样关的)⇒ unreadableObjects: []aborted: true、不发 warn。

外加:abort 在读之前 ⇒ 一次查询都不发(这是 ERROR Find operation failed 消失的机制);abort 在中途 ⇒ 已经证成的 dangling 全部保留、没读到的对象不进桶;探针在关停下抛错也不算 undetermined(同一个范畴错误,隔壁桶)。

3. 巡检在 CLI 场景仍然运行

这不是「一次性命令不跑巡检」。巡检在每个宿主上照旧接线、配置完全一致,唯一的边界是引擎活着才读。新的 schema-migrate.teardown.integration.test.ts 在真实 bootSchemaStack 上把两头都断言了:引擎活着时 sweep 真的读、报告 aborted: falseunreadableObjects: [];shutdown() 之后 sweep 一次读都不发。最后一条断言故意证明它不是空转 —— 直接对引擎发一次 find 会 reject,说明池确实关了,上面的沉默是修复的结果而不是「本来就没东西可读」。

pnpm --filter @objectstack/objectql exec vitest run   →  107 files / 1695 tests passed
pnpm --filter @objectstack/cli      exec vitest run   →   67 files /  586 tests passed
pnpm --filter @objectstack/runtime  exec vitest run   →   80 files / 1092 tests passed
pnpm --filter @objectstack/objectql --filter @objectstack/cli typecheck  →  Done

顺带的两件事

#4776 是同一族的镜像面。 那边是「判断下得太早(提供方还没注册)」,这边是「判断下得太晚(依赖已经拆了)」。共同点不是时序本身,而是检查与它所依赖的资源之间没有一份可被检查方看见的生命周期契约:两处的检查都只能观察到自己被调用了,观察不到「现在问这个问题还算不算数」。本 PR 在一条边上把它变成了显式的一位(signal / stopped),没有推广成通用机制 —— 如果要统一,自然的形状是让 PluginContext 暴露一个内核级的 readiness/teardown 信号,让所有周期性工作都从同一处取,而不是每个 service 各自发明。本 PR 不实现它,记在这里供裁定。

另发现一条,已单独开 #4813(未认领)。 内核的 initPluginWithTimeout / startPluginWithTimeout 创建的超时守卫定时器在插件赢下 race 之后既不 clearTimeout 也不 unref(),于是每个进程在活干完之后还要空转 startupTimeout(ObjectQLPlugin 是 120s)。实测一条 os migrate 3 秒干完活、120 秒才退出,8 根带 ref 的 Timeout 挂着。这正是 #4747 的使能条件 —— 一个干完就退出的进程永远碰不到那个 60 秒的定时器。本 PR 从生命周期契约那一侧修,不依赖 #4813;而同一个 kernel.tsshutdown() 自己的守卫已经写了 t.unref() 并附了注释,修法就在隔壁 30 行。

🤖 Generated with Claude Code

https://claude.ai/code/session_015Br2xsJsczFsTR9bvbh2Ny


Generated by Claude Code

)

Every `os migrate` subcommand ended a SUCCESSFUL run with two
`ERROR Find operation failed` lines and a #4551 report naming
`sys_metadata` / `sys_view_definition` as `unreadableObjects`. That
bucket exists to separate "I could not check" from "I checked and it
was fine"; non-empty on every healthy run, it separated nothing.

Two silent no-ops stacked up:

- `ObjectQLPlugin` put its teardown in `stop()`, but the kernel's Plugin
  contract is `init`/`start`/`destroy` — `stop()` is never called by
  anyone, so the ADR-0057 sweep timers were never disarmed on any host.
- `bootSchemaStack().shutdown()` called `(runtime as any).stop?.()` and
  `Runtime` has no `stop`. The optional call made "no teardown" look
  exactly like "teardown performed": the kernel stayed running with
  every timer armed while the command closed its driver.

60s after boot the sweep woke inside the still-alive process and read a
pool its own host had already disconnected.

- ObjectQLPlugin: `stop` -> `destroy`, the hook the kernel actually calls.
- bootSchemaStack: tear down via `kernel.shutdown()` — the same path
  `os serve` takes on SIGTERM, so one-shot and server exit alike.
- LifecycleService.stop(): raise an abort bit the in-flight sweep holds,
  not just clear timers; refuse to start a sweep once stopped.
- The audit takes that bit as `signal` and stops issuing reads. A read
  that fails *because* the run was called off is dropped rather than
  filed — it is not evidence about the datasource. The new
  `DanglingReferenceReport.aborted` keeps the incompleteness loud
  without spending the finding bucket on it.

A genuine datasource fault still lands in `unreadableObjects`, and the
audit still runs on CLI hosts: this is a liveness boundary, not the
"one-shot commands skip the audit" switch the issue rejected.

Fixes #4747

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015Br2xsJsczFsTR9bvbh2Ny
@vercel

vercel Bot commented Aug 3, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
objectstack Ignored Ignored Aug 3, 2026 8:36am

Request Review

@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 2 package(s): @objectstack/cli, @objectstack/objectql.

29 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:

  • content/docs/ai/skills-reference.mdx (via packages/cli)
  • content/docs/api/client-sdk.mdx (via @objectstack/cli)
  • content/docs/api/data-flow.mdx (via @objectstack/cli)
  • content/docs/api/environment-routing.mdx (via @objectstack/cli)
  • content/docs/api/error-catalog.mdx (via @objectstack/cli)
  • content/docs/automation/hook-bodies.mdx (via packages/cli)
  • content/docs/concepts/metadata-lifecycle.mdx (via @objectstack/objectql)
  • content/docs/data-modeling/formulas.mdx (via packages/objectql)
  • content/docs/deployment/backup-restore.mdx (via @objectstack/cli)
  • content/docs/deployment/cli.mdx (via @objectstack/cli)
  • content/docs/deployment/migration-from-objectql.mdx (via @objectstack/objectql)
  • content/docs/deployment/self-hosting.mdx (via @objectstack/cli)
  • content/docs/deployment/validating-metadata.mdx (via packages/cli)
  • content/docs/deployment/vercel.mdx (via @objectstack/objectql)
  • content/docs/getting-started/your-first-project.mdx (via @objectstack/cli)
  • content/docs/kernel/runtime-services/data-service.mdx (via packages/cli)
  • content/docs/kernel/runtime-services/index.mdx (via packages/cli)
  • content/docs/kernel/services-checklist.mdx (via @objectstack/objectql)
  • content/docs/kernel/services.mdx (via @objectstack/objectql)
  • content/docs/permissions/authentication.mdx (via @objectstack/cli, @objectstack/objectql)
  • content/docs/plugins/index.mdx (via @objectstack/cli, @objectstack/objectql)
  • content/docs/plugins/packages.mdx (via @objectstack/cli, @objectstack/objectql)
  • content/docs/protocol/kernel/index.mdx (via @objectstack/objectql)
  • content/docs/protocol/kernel/plugin-spec.mdx (via @objectstack/cli)
  • content/docs/protocol/kernel/realtime-protocol.mdx (via @objectstack/cli)
  • content/docs/protocol/objectql/query-syntax.mdx (via packages/objectql)
  • content/docs/protocol/objectql/state-machine.mdx (via @objectstack/objectql)
  • content/docs/releases/implementation-status.mdx (via @objectstack/cli, @objectstack/objectql)
  • content/docs/releases/v16.mdx (via @objectstack/cli)

Advisory only. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs origin/main → pass the list as args.docs.

@github-actions github-actions Bot added size/l documentation Improvements or additions to documentation tests tooling and removed size/l labels Aug 3, 2026
`AuditAbortSignal` 从包根导出,加上 `DanglingReferenceAuditOptions.signal`
与 `DanglingReferenceReport.aborted` 两个新字段:新增公开面按仓内先例
(#4791 因新增 `sealNodeTypeVocabulary()` 等公共 API 定 minor)是 minor,
不是 patch。`@objectstack/cli` 只动了内部的 schema-migrate.ts,维持 patch。

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015Br2xsJsczFsTR9bvbh2Ny
@github-actions github-actions Bot added the size/l label Aug 3, 2026

Copy link
Copy Markdown
Contributor Author

已按复核意见改定级:.changeset/hungry-donkeys-repeat.md@objectstack/objectqlpatch 改为 minor(commit a89f712)。

理由与复核一致,记在这里备查 —— 本 PR 给 objectql 新增了三处公开面:

  • AuditAbortSignal 类型,src/index.ts 显式从包根导出;
  • DanglingReferenceAuditOptions.signal;
  • DanglingReferenceReport.aborted

三者都是新增而非改签名,消费者无需改动即可升级,所以 minor 是对的 bar,与 #4791 一致。@objectstack/cli 维持 patch:它只改了内部的 utils/schema-migrate.ts(shutdown() 的实现),没有新增或改变任何导出。

顺带一提,LifecycleServiceOptions.referenceAudit 的类型从 DanglingReferenceAuditOptions & { enabled?: boolean } 收窄成了 Omit< DanglingReferenceAuditOptions, 'signal' > & { enabled?: boolean } —— 严格说这是一处收窄,但被收掉的键是本 PR 同一批新增的 signal,在 main 上从来不存在,所以对任何已发布版本的消费者都不是 breaking,不需要 major。收窄是刻意的:巡检的寿命就是 LifecycleService 的寿命,不该有第二个人回答「现在还能读吗」。

同时已把最新 main 合入(#4794 已落地,只碰 engine.ts / validation/** / index.ts,与本 PR 的 integrity/ + lifecycle/ 无重叠;index.ts 的导出块 git 自动合并干净),并按 AGENTS.md §9 重装依赖 + 重建后重跑了受影响包的 typecheck 与测试。


Generated by Claude Code

@os-zhuang
os-zhuang marked this pull request as ready for review August 3, 2026 08:44
@os-zhuang
os-zhuang added this pull request to the merge queue Aug 3, 2026
Merged via the queue into main with commit 127f091 Aug 3, 2026
21 checks passed
@os-zhuang
os-zhuang deleted the claude/issue-4747-integrity-audit-shutdown branch August 3, 2026 08:52
os-zhuang pushed a commit that referenced this pull request Aug 3, 2026
`fillEmptyGroups` 从包根 index.ts 导出,属于新增公共 API,按 #4815
(objectql / AuditAbortSignal)与 #4791(service-automation /
sealNodeTypeVocabulary)确立的同一条 bar,changeset 从 patch 改为 minor,
并在正文点名这个新导出。

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015Br2xsJsczFsTR9bvbh2Ny
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/l tests tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

每个 os migrate 子命令关停时,悬空引用巡检都会把 sys_metadata / sys_view_definition 报成 unreadableObjects(连接已关闭)

2 participants