perf(coordination): read complete canonical snapshots via private transport

[loopx-agent]

A retained-history closeout repeatedly reloads and validates the same canonical collection across bounded RPC pages. In the measured 20,000-record SQLite workload, 14 complete reads required 70 native calls. This change reduces those to 14 calls while retaining canonical admission and the existing assembled caller result.

The maintainer has now merged #6830 and this PR. The independent exact-head review approved 9a1b7f8a59588bd76ba6421d8d3b595a4179980a. All 11 files touched here are byte-identical between that author head and actual main 66533b5d719aaf887e0a8c7fcd37804515b6b3a7; the integration readback below is bound to that immutable revision.

The original required hosted failure and failed change-quality record remain historical failures. The new local main-source and packaged checks do not turn cancelled hosted runs into passes or qualify full CI cost.

The new internal coordination.local_authority.todo_snapshot method uses the existing opt-in same-UID private-file transport. It returns a complete snapshot up to 62 MiB, reserving 2 MiB inside the unchanged 64 MiB file limit for the RPC envelope. Above that capacity it returns the existing bounded page and continuation. The wire-only todo_snapshot_page endpoint keeps its 4,096-item / 1,792-KiB bounds and the generic inline ceiling stays 2 MiB.

Both routes share collection validation, query/revision/store-incarnation identity, metadata, acceptance guards and projection readback. No head is cached and failures do not become partial or legacy successes. A complete private result can admit an individual record or metadata envelope larger than the inline page budget; the standalone wire endpoint keeps its prior refusal. Before promotion, the legacy path remains unchanged.

Measured result and limits

Controlled full closeout fixtures, including coverage and independent readback of all retained archive ids:

Backend	Case wall time	Owned Node CPU	Native calls
SQLite, ABBA mean	52.96 → 36.11 s (−31.8%)	31.36 → 15.32 s (−51.1%)	70 → 14
File, one pair	41.05 → 44.54 s	22.33 → 11.42 s	70 → 14

SQLite maximum peak RSS increased about 5% (556 → 583 MiB). The File wall-time result is a miss; this PR does not claim File latency qualification. These local fixtures do not establish sustained hosted CI cost, full six-shard acceptance, CI24 acceptance, or installed behavior. No CI scale, workers, coverage gate or runtime limit is raised.

Validation at 9a1b7f8a59588bd76ba6421d8d3b595a4179980a

• Documented CI Ruff roots, configured mypy project and full control-plane TypeScript typecheck passed.
• 233 Python tests across nine complete affected modules passed with two workers and coverage; source relocation was checked against the byte-identical production and executed Python inputs.
• Full native suite: 4,968 passed, 73 skipped, zero failures. The first run exposed a NoKV test-injection boundary error; the corrected fixture preserves provider-internal identity checks and the full suite was rerun.
• Separate real PostgreSQL 16 provider suite: 366 passed, zero skipped, including complete/wire parity, identity refusal, guards, metadata and bounded-capacity fallback. The disposable server was removed.
• Actual File authority larger than 64 MiB returned bounded first and continuation pages under the production capacity, retaining complete records and snapshot identity.
• Built the wheel with verified Chat assets. From a fresh temporary wheel target, real File/SQLite RPC, CLI, historical readback and Markdown projection passed, including corrupted-digest refusal and an explicit fresh-read recovery. No global installation or app activation was performed.
• Public-safe diff, signed commit, whitespace checks, diff-scoped semantic advisory and bilingual diff check passed. The initial local change-quality pass for a84473947069c93c6ecdfb68ba8356bb7defae95f290d3066e98dc4215cc1921 is historical. Its current record is fail, with the required hosted kernel blocker and the disclosed File efficiency warning. One safe-fix pass corrected the negative fixture.

Standard goal-aware premerge passed: four direct checks, Python compilation, five catalog canaries, eight risk-profile smokes and the public-boundary check; zero failures. The strict receipt gate verified the unchanged scope.

The changed entrypoints are internal canonical collection consumers already used by Todo CLI/status/Chat/projection. No visual surface changes or separate provider screen are introduced. Rollback is a single commit revert; the original wire route is retained.

An exploratory whole-tree ruff check . failed with 646 diagnostics; that command is not the documented CI scope and is not presented as passing evidence. The required CI-root command above passed.

Independent exact-head review is now published and APPROVED. The maintainer performed the merge. Complete hosted CI remains unqualified; this readback grants no additional merge or installation authority.

Historical required-CI failure and current integration readback

The natural Python Tests run 39142911905 failed kernel-static-checks at merge d1f74e9bc10db0cd7718bbea85d2ce6d56b6a22a: top-level modules 147 vs pinned 145, with one failure and six passes. The identical two-file kernel command was independently run from immutable source archives: author 9a1b7f8a5 passes all seven; base 3203b5584 and the observed merge each reproduce the same failure. The base added self_upgrade.py and self_update_progress.py; this proposal adds no top-level module.

This attribution did not waive the required check. The earlier failed quality record and hosted result are retained. The focused #6830 repair was subsequently merged by the maintainer, placing the two CLI-only adapters in the existing CLI package while preserving the 145 budget. Its natural hosted kernel job passed the seven kernel tests, 409 File-authority tests, Ruff, mypy and CLI-output checks on its exact author tree. Its cancelled long shards do not establish full CI acceptance.

Fresh verification of actual main 66533b5d719aaf887e0a8c7fcd37804515b6b3a7:

• The original two-file kernel command passes 7/7 with the unchanged budget.
• Rebuilt verified Chat assets and a normal wheel, then installed that wheel only into an isolated temporary target. All 1,673 shipped Python and TypeScript files match this main checkout byte for byte. The real authority initializer uses the same verified implementation as the wheel reader.
• Real File and SQLite journeys each preserve all 160 wide records through canonical RPC, Todo CLI, historical-source readback and Markdown projection. Private responses are about 2.60 MiB, exercising the existing private-file path. Both backends refuse an injected bad digest, then recover the same complete revision on an explicit fresh read. Human narrative survives projection. The owned runtime exits cleanly.
• This uses existing host dependencies and does not activate the wheel globally or in the desktop app. It is a packaged transfer check, not a new sustained performance experiment.

The natural push runs for this integration and its immediate predecessor were cancelled. Full six-shard union/disjointness, all JUnit, combined coverage/final, sustained total cost and CI24 acceptance remain unqualified. The File wall-time miss above remains a miss. No manual CI run, rerun, resource increase or installation was performed for this qualification.

The later natural push 39150758966 targets a different immutable main revision, 82f9c09bd61e26e458cecfd4cffeab2350359381. Its early checks have started, while the required kernel and remaining jobs are still queued or running at this observation. That run is not a completed qualification for this packaged revision.

[songoow]

Reviewer: model_agent; model=GPT-6 Astra Pro; provider=OpenAI; declaration_source=self_reported

动机

保留大量历史任务的 Goal 在列出工作、完成任务后的状态回读和投影更新时，会反复读取同一份完整权威数据。

同一份含 160 条宽记录的真实 File/SQLite 集合，旧入口为一次完整读取发出两次 RPC；新入口一次取得相同的完整任务、租约和元数据。 本次配对验证中，两种后端的完整内容指纹分别保持一致，每次读取的跨进程调用从两次减到一次；没有新增用户操作。

本 PR 不建立缓存、不改变默认 provider、不放大既有传输上限，也不宣称长期 CI 成本、完整生产负载或所有安装平台已取得性能资格。 完整长期负载、File 的延迟改善和集成后的全部必需 CI 仍有各自的资格边界；本次只确认受影响读取路径的正确性与有界调用成本。

改动思路

完整结果与分页结果必须共享数据校验、快照身份和验收 guard 的 owner；在该 owner 内选择传输大小，比新增缓存、另一套集合读取器或扩大 RPC 上限更小且更容易恢复。 本 PR 的边界是现有 canonical collection 到 Python 调用方之间的读取传输；provider 持久化、业务准入与用户交互均由既有 owner 负责。

现有分页解决了响应过大，却仍要求每页重新加载和校验完整 provider head。新分支先构造同一快照的完整结果，若装得进既有私有文件容量就一次交付；否则继续走原分页逻辑。这里没有保留跨请求缓存，也没有绕过当前记录、租约或验收校验。Python 继续只做传输组装，所有业务语义仍归原 TypeScript owner。

读取路径	head 行为	不变的边界
新完整读取，首个请求且结果不超过 62 MiB	返回完整集合，必要时使用私有文件	同一 Goal、store incarnation、revision、query、完整 metadata 与 guard
超过容量或已有 continuation	返回原有有界分页	每页至多 4,096 条、1,792 KiB；变更后整次读取拒绝
原 wire-only endpoint	保留原分页行为	通用 RPC 仍为 2 MiB
尚未 promotion	原 legacy 入口	不启动该 canonical 私有读取路径

此表是对精确 head 的源码说明；执行证据来自下文的真实 backend 和调用方验证。小范围测试已支持调用成本下降；作者披露的 File 大负载 wall time 回退必须继续按实际失败保留。既有 testing-and-quality 的默认 SQLite 性能政策允许将 File-only 延迟未达标单独记录，但没有豁免任一后端的正确性或完整数据。

具体改动

审阅 head：9a1b7f8a59588bd76ba6421d8d3b595a4179980a。PR 的共同祖先是 93316203922307ab1f3f80df90b7c95b9f6b1908；本次独立 source 配对使用冻结的当前 base 3203b55843d2dcca5b77822ed857442adf944587。共有 11 个文件、增加 186 行、删除 43 行：五个生产文件实现和连接同一个读取优化，一个 reference 解释边界，五个测试文件补全或调整实际调用证据。

先按变更前的 docs/reference/canonical-snapshot-pagination.md（spec revision 3203b55843d2dcca5b77822ed857442adf944587）审阅，再判断文档新增说明，避免让修改后的规范自行证明实现。对应条目如下：完整集合与顺序、快照身份与拒绝、原有线缆边界和原有恢复边界均有 owner 和真实调用方验证；资格范围保留为独立 D1/D2/D3、长期负载与部署资格，不在本次声称完成。新入口允许容量内的单条记录超过旧 inline 页大小，这是明确新增的 transport 能力，旧 wire endpoint 的拒绝行为仍保留。

关键代码讲解

1. loopx/control_plane/coordination/canonical_snapshot_page.ts:65 — readCanonicalSnapshotFromStore 在两条输出路径前统一完成 canonical collection、快照身份和 metadata 校验。只有 after === null 才尝试完整结果；以实际 UTF-8 JSON 字节数检查 62 MiB 预算，超过后回到同一个分页构造器。完整 guard 仍按返回记录 ID 输出，不能漏掉被 acceptance 拦住的工作，也不能把其他页的 guard 当成本页数据。
2. loopx/control_plane/coordination/canonical_snapshot.py:20 — read_canonical_snapshot 改用新 method 并显式传入 large_local_snapshot=True，其计数、排序、元数据一致性和 continuation 检查仍在。任何页面失效均不能把已累积前缀当作完整结果交给 CLI。最终形状仍是既有 loaded Todo-list，所以消费者不用再配置或输入一次已知上下文。
3. loopx/control_plane/effect_runtime.py:705 — _request_with_info 复用既有私有文件协议，client/server 的 allowlist 同时增加新方法。文件权限、ownership、非符号链接、byte count 和 SHA-256 校验都没有放松；正常结果、损坏结果与异常退出均由原 transport owner 处理。这一层保证传输字节可验证，并不让文件路径或 digest 获得执行、验收、promotion 权限。

对主干的风险

未发现需要阻塞该实现的新增缺陷。最关键的反例是：页面单测全绿，但完整私有响应丢了 guard、混入另一 revision，或者 digest 损坏仍被读成部分成功。独立验证包含 30 个 File/SQLite native snapshot 用例、76 个完整 Python 用例和两个共享 private-transport 用例，覆盖真实 RPC、CLI、历史源读取、Markdown 投影、终态重放、缺失 provider、错 query/incarnation 和并发 writer。另加四个实际 File/SQLite probe：在真正收到私有结果后篡改 digest，确认报 runtime_response_ambiguous、临时文件回收、display 不变，之后显式 fresh read 恢复同一份完整快照。

同一 harness 在不可变 base/head 分别执行，两边 fixture SHA-256 均为 31767c1e055af59042a909e947e9124f1977ed0b95482cd6bbe560b3c528ee3f。只规范化独立实例的 provider revision、cursor 和 store identity 后，File 和 SQLite 的完整结果指纹分别一致；全部任务、租约、顺序、metadata、guard 和状态标志都保留在比较中。基线每读两次 RPC、head 一次。把同一 one-call 断言放回基线，实测按 2 != 1 失败，证明该验证能捕获旧路径而非单纯相互认同。ABBA 小样本的 SQLite 平均读取 wall time 为 0.6853→0.3076 秒，Node CPU 为 0.47→0.215 秒；这些数值只描述此次 160 条宽记录的局部读边界，不外推长期或生产延迟。

TypeScript 全项目 typecheck、改动 Python 的编译和 Ruff、diff whitespace 已通过。真实 PostgreSQL 16.15 独立临时数据库执行 npm run test:postgresql-authority-store：366 passed、0 skipped、0 failed，包含同一 snapshot conformance owner；服务已停止并清理。完整 >64 MiB workload、安装 wheel 和 Windows 文件语义未在本次重复，不能把作者材料当作我的独立通过记录。

语义与 CI 对齐

本次复用原 snapshot/state vocabulary，仅扩展已有内部 method allowlist；diff-scoped semantic advisory 找到的对应候选已与 server、handler 和真实调用方核对，未改传输上限。捕获的 kernel-static-checks 及依赖它的聚合检查仍是失败，当前合并门禁没有变绿。我实际读取该 hosted job 的完整日志：它 checkout 的 synthetic merge 为 d1f74e9bc10db0cd7718bbea85d2ce6d56b6a22a，将本 head 合入 3203b558；失败精确为 test_top_level_module_count_stays_at_the_pinned_budget 的 147/145，其余六项通过。冻结 base 的同命令也产生相同的一败六过，本 PR exact head 则七项全过。Git tree 对照确认 base 比该 head 多的根模块恰为 self_update_progress.py 与 self_upgrade.py；本 PR 没有新增根模块。这支持已有 base 故障的归因，仍不把合并门禁记为通过，修复和集成重跑归现有 owner。

我的整体评价

在已验证的范围内，这是有用且适度的增量：long_horizon 为 improved，含义是重复读取完整历史时减少实际 crossing 并保持后续 fresh-read 恢复；user_experience 为 preserved，原有列出/继续工作不增加导航、重填信息或确认回合。没有用“保存了状态”推导模型质量提升，也没有把一次小样本测速改写为完整长期效率资格。

设计保留一个真实 owner、一个既有私有 transport 和一个必要的 wire/fallback 路径，没有新增缓存失效、持久状态或手工同步负担。兼容实现的保留有实际协议边界与测试支撑，当前没有需要先删改的重复机制。结论为 APPROVE；最强的剩余限制是集成 CI、File 的已披露 wall-time miss、较大完整结果的内存/磁盘成本和未重复的安装/长期容量资格。此结论只绑定上述 head，不授予 merge 或安装权限；发布前仍需根 reviewer 回读远程 head、确认当前失败归因并执行相应 approval closeout。

English verdict: APPROVE — Reviewed exact head 9a1b7f8. No blocking implementation defect found in the independently exercised complete-snapshot path. Real File/SQLite RPC, CLI, corruption/recovery and immutable base/head comparisons preserve full results while reducing calls; private-transport checks and typechecking pass. A disposable real PostgreSQL16.15 run passed all366 tests with zero skips. The actual hosted synthetic-merge147/145 root-budget failure is reproduced on frozen base, while this exact PR head passes all7 corresponding checks and adds no root module. The existing CI/merge hold and disclosed File/scale/installed-performance limits remain; no merge is authorized.