From 09bb04ccc0559381d61ed71a5640f94ec89187a1 Mon Sep 17 00:00:00 2001 From: SHIN09-git Date: Thu, 12 Nov 2026 03:41:45 +0800 Subject: [PATCH 1/8] test(library): preserve a synthetic preview.20 upgrade baseline --- .gitattributes | 4 + ROADMAP.md | 22 +- .../LIBRARY_STABILITY_AND_PERFORMANCE_PLAN.md | 76 +++ scripts/fixtures/library-preview20/README.md | 38 ++ .../fixtures/library-preview20/backup.json | 598 ++++++++++++++++++ .../fixtures/library-preview20/library.sqlite | Bin 0 -> 73728 bytes .../fixtures/library-preview20/manifest.json | 43 ++ scripts/fixtures/library-recovery.mjs | 105 +++ 8 files changed, 875 insertions(+), 11 deletions(-) create mode 100644 docs/LIBRARY_STABILITY_AND_PERFORMANCE_PLAN.md create mode 100644 scripts/fixtures/library-preview20/README.md create mode 100644 scripts/fixtures/library-preview20/backup.json create mode 100644 scripts/fixtures/library-preview20/library.sqlite create mode 100644 scripts/fixtures/library-preview20/manifest.json create mode 100644 scripts/fixtures/library-recovery.mjs diff --git a/.gitattributes b/.gitattributes index b6d7da2..ee75643 100644 --- a/.gitattributes +++ b/.gitattributes @@ -13,3 +13,7 @@ *.docx binary *.woff binary *.woff2 binary + +# Byte-for-byte fixtures from preview.20; their published digests must survive checkout. +scripts/fixtures/library-preview20/library.sqlite binary +scripts/fixtures/library-preview20/backup.json -text diff --git a/ROADMAP.md b/ROADMAP.md index 3e8155b..43490f8 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -1,6 +1,6 @@ # 路线图 / Roadmap -更新日期:2026-11-11。这里只保留当前方向和下一步,不再重复各版本的施工日志、源码行数和已完成待办。历史变更见 [CHANGELOG](./CHANGELOG.md),包的实际状态见 [发行产物说明](./docs/RELEASE_ARTIFACTS.md)。 +更新日期:2026-11-12。这里只保留当前方向和下一步,不再重复各版本的施工日志、源码行数和已完成待办。历史变更见 [CHANGELOG](./CHANGELOG.md),包的实际状态见 [发行产物说明](./docs/RELEASE_ARTIFACTS.md)。 ## 当前方向 @@ -13,20 +13,20 @@ - 桌面和 Agent 共用生成、审稿、修订、候选决定及正式提交流程;项目权限可以撤回。 - 章节阅读、局部重写、任务编辑、候选历史和作者诊断摘要已有入口。 -2026-10-31:复用单次 Novelty 审计的分词结果,去掉 Prompt 追踪的重复渲染,简化 token 扫描;修订指令按类型输出。范围、实测和限制见 [本轮性能记录](./docs/REDUNDANCY_PERFORMANCE.md)。已纳入本地 Preview.19 安装包和 `release/win-unpacked`,尚未上传 GitHub。 +[Preview.20](https://github.com/SHIN09-git/novel/releases/tag/v0.1.6-preview.20) 已公开发布,包含[长篇完稿修复](./docs/LONG_NOVEL_COMPLETION_REPAIR_PLAN.md):超限保稿、成稿硬规则核查、跨章对账、按书统计用量和批量改名。用量不能反推供应商余额或缺失的历史账单。此前减少 Novelty 重复分词、Prompt 追踪重复渲染的实测见[性能记录](./docs/REDUNDANCY_PERFORMANCE.md)。 ## 下一步优先事项 -按[长篇完稿后的修复计划](./docs/LONG_NOVEL_COMPLETION_REPAIR_PLAN.md),源码已支持保留超限稿、成稿硬规则核查、全书进度与连写顺序、按章取余额和跨章事实核对。P2 增加按书统计模型用量、调用前估算、连接诊断、按起止章编排导向,以及可预览和追溯的批量改名。尚不能从用量反推供应商余额或旧书缺失的账单。下表保留原有维护事项;安装包要单独更新。 +执行[书库可靠性与长篇性能施工计划](./docs/LIBRARY_STABILITY_AND_PERFORMANCE_PLAN.md)。**当前只完成核查,尚未开始实施。** 从 S0 的干净主线工作区开始;下一版只需完成 S1、S2,不等所有性能工作做完才交付。 | 优先级 | 工作 | 完成条件 | | --- | --- | --- | -| 1 | 处理已报告的依赖安全告警 | 核对受影响调用,按兼容范围升级;完整回归和包内 SQLite 启动通过后再发包 | -| 2 | 继续压缩实际慢入口 | 复现具体操作,测量全量数据读取、跨进程传输或诊断耗时;只有可重复收益才新增定向查询或后台计算 | -| 3 | 用真实修订案例检查提示词 | 对比指定问题是否修好、无关段落是否保留;记录 Novelty 误报/漏报,避免继续堆全局禁令 | -| 4 | 修补高频操作中的阻碍 | 优先未保存输入恢复、采纳失败恢复、阅读和修订切换;按用户反馈选具体入口,不做全站重设计 | +| S1,下一版 | 书库打开失败时不切换到可写空库 | 原路径和文件不变;作者能重试、选择已有库或从备份恢复;正常首次建库不受影响 | +| S2,下一版 | 核对运行时依赖,用 Windows 安装器验证升级 | 干净安装、从 preview.20 升级后均能读写 SQLite;重启后作品和版本链仍在 | +| S3-A / S3-B | 先优化打开项目,再优化采纳与修订 | 更新时间不传整库;提交不读取无关项目历史。用虚构长篇对照耗时,保留并发、回滚和版本恢复能力 | +| S4,有依据再做 | 历史详情按需读取、减少新副本或压缩自动备份 | 不把局部数据当整库保存;不删除作者采纳来源;旧备份可恢复,不自动删手动备份 | -真实模型对照需要另行确认供应商、模型和费用。本地性能基准不消耗模型额度,不能替代成文质量评估。 +具体的修订质量、Novelty 误报和未保存输入仍按作者反馈跟进,不阻塞上述交付。本轮性能验收不消耗模型额度,也不替代成文质量评估。 ## 保留的必要校验 @@ -40,7 +40,7 @@ npm.cmd test npm.cmd run build ``` -更新安装包时另跑 `npm.cmd run smoke:packaged`;公开发布前做密钥扫描。每项新优化必须有行为回归或实测依据,不新增只检查文案和文件行数的门槛。 +发布候选包另跑 `npm.cmd run smoke:release` 检查解包程序,再用 Setup 验证实际安装和升级;两者不能互相替代。公开发布前做密钥扫描。每项新优化必须有行为回归或实测依据,不新增只检查文案和文件行数的门槛。 ## 暂缓 @@ -56,8 +56,8 @@ npm.cmd run build ## English -Updated 2026-11-11. The [long-novel repair plan](./docs/LONG_NOVEL_COMPLETION_REPAIR_PLAN.md) now includes project-level usage, single-request cost estimates, connection diagnoses, custom story-direction ranges and auditable chapter renames, in addition to draft retention, canon checks and cross-chapter continuity. Provider balances and unrecorded historical usage remain unknown. This describes the working tree, not a new installer. +Updated 2026-11-12. The [preview.20 prerelease](https://github.com/SHIN09-git/novel/releases/tag/v0.1.6-preview.20) includes the [long-novel fixes](./docs/LONG_NOVEL_COMPLETION_REPAIR_PLAN.md). Provider balances and unrecorded historical usage remain unknown. -Next: verify dependency advisories; profile concrete slow operations before redesigning storage or threading; evaluate task-specific revision instructions with real cases; repair reported editing and acceptance friction. The [current performance record](./docs/REDUNDANCY_PERFORMANCE.md) covers the optimizations now packaged locally as Preview.19, not a claim about model quality or a GitHub publication. +Next, follow the [library reliability and performance plan](./docs/LIBRARY_STABILITY_AND_PERFORMANCE_PLAN.md). No implementation is claimed yet. The next release candidate must distinguish an unavailable library from an empty one and pass a real Windows Setup upgrade from preview.20. Then measure and narrow project opening and chapter commits. History compaction and automatic backup compression are conditional, not release blockers. Keep project isolation, version conflict checks, idempotent commits, report provenance, rollback, backups, authorization and credential redaction. Full CRUD rewrites, FTS, graphs, cloud sync and generic extension systems remain deferred. Historical plans and measurements are references, not additional mandatory work. diff --git a/docs/LIBRARY_STABILITY_AND_PERFORMANCE_PLAN.md b/docs/LIBRARY_STABILITY_AND_PERFORMANCE_PLAN.md new file mode 100644 index 0000000..523ec5c --- /dev/null +++ b/docs/LIBRARY_STABILITY_AND_PERFORMANCE_PLAN.md @@ -0,0 +1,76 @@ +# 书库可靠性与长篇性能施工计划 / Library Reliability and Performance + +拟定于 2026-11-12。**状态:已完成问题核查,S0 至 S4 均未实施。** 本计划承接 [ROADMAP](../ROADMAP.md),不重新排期已发布的长篇完稿修复。 + +下一版先让作者确信“打开的是原来的书库”,并用真正的 Windows 安装器验证升级。之后再缩短打开一本书、采纳或修订一章的等待时间。不要求先重构全库,也不拿删除作者历史来换取性能。 + +## 顺序与交付 + +| 批次 | 交付 | 启动条件 | +| --- | --- | --- | +| S0 准备 | 从核实过的远端主线建立独立工作区,准备虚构旧书库 | 首个工程步骤;不改动原工作区 | +| S1 书库打开失败 | 保留原路径,显示恢复入口,不悄悄换到另一份可写书库 | S0 后;下一版必做 | +| S2 Windows 升级验收 | 在受支持的 Electron 上验证安装、升级、SQLite 读写和重启 | 可与 S1 并行准备环境;交付候选包前必须合并并验过 S1 | +| S3-A 打开项目 | 只更新当前项目的最近打开时间;保留排序 | 用 S2 同一套依赖先测基线,可单独合并 | +| S3-B 采纳和修订 | 只读取这次提交真正需要的记录,保留原有事务校验 | 沿用 S3-A 的样本;采纳、修订分别对照 | +| S4 历史与备份 | 按瓶颈选做:按需读历史、减少新记录副本或压缩自动备份 | 不阻挡前面发版;每项须有测量或明确的磁盘需求 | + +## S0:选对起点 + +- 核查时 GitHub `main` 是 `78550db`。按 Git 的文件内容比对,当时本地 901 个非忽略项目文件均与 `v0.1.6-preview.20` 一致;与主线的差异只有 CI、依赖清单和锁文件。这是**拟定本计划前**的结果,不代表后来新增的文件都已发布。 +- 开工前重新核对远端、未提交和未跟踪文件。从最新已核实的 `origin/main` 建独立工作区,带入这份计划及经确认的后续变更。保留作者原工作区,不硬重置,不用旧锁文件覆盖主线。 +- 只用临时目录、虚构书库和模型替身。准备一份由 preview.20 写出的样本,至少含两本书、一章已采纳正文、一次修订和一条尚可续跑的任务。记录项目 ID、正文摘要值、版本链和已采纳草稿 ID,供后续逐项核对。不要复制真实书库做测试。 + +## S1:打开失败,不冒充空库 + +**现状。** [存储工厂](../src/storage/SqliteStorageService.ts)在原生 SQLite 模块加载失败时会返回 JSON 后端;[主进程启动](../src/main/index.ts)可能随之改写配置路径。[界面加载](../src/renderer/src/hooks/useAppData.ts)遇到读取错误也会设置空数据。只改一条错误文案不能区分“库是空的”和“库没读出来”。 + +| 情形 | 作者应看到 / 程序应做什么 | +| --- | --- | +| 真正首次使用,尚无书库 | 按现有流程创建默认库;已有合法旧 JSON 的首次迁移仍保留原件和备份 | +| 已选书库因磁盘未连接、路径失效、原生模块不匹配或读取失败而无法打开 | 显示出错的书库路径、简短原因及下一步;不改配置,不创建替代库,不开放对空数据的保存 | +| 用户要恢复 | 提供“重试”“选择已有书库”“从备份恢复”入口。先在所选位置验证能读,再切换;恢复优先写新位置。选择失败或取消,仍指向原库 | +| 用户明确选择使用 JSON | 保留 JSON 的独立使用及导入导出能力,说明写入哪个文件;不暗示新写内容会自动合回 SQLite | + +不能只凭“文件不存在”认定是新用户。旧配置不足以区分首次使用和磁盘离线时,先请作者选择已有书库或明确新建。不要在判断之前调用会初始化空库的读取方法。 + +**验收:** 分别模拟模块不可用、已有库损坏及已配置的库暂时找不到;确认原数据库、配置和同目录 JSON 未被改写,也没有新建空库。恢复正常条件后点“重试”,两本书及版本链都在。另跑新用户首次启动、旧 JSON 迁移和显式恢复的正向用例。复用[存储验证](../scripts/validate-sqlite-storage.mjs),并从成品窗口验证作者看到的状态,而不只看日志。 + +## S2:发布候选包必须过安装升级 + +preview.20 实包使用 Electron 39.8.10 / better-sqlite3 12.9.0;核查时主线锁定的是 44.9.1 / 13.0.3。按候选提交的锁文件和 [Electron 支持日程](https://releases.electronjs.org/schedule)复核,不把源码升级当成安装包升级。区分会随应用运行的依赖和只影响构建的依赖,不为清零告警一次合并所有大版本。 + +1. 在独立工作区安装锁定依赖,跑类型检查、全量测试和生产构建。从同一提交生成 **新版本号** 的 Windows x64 安装包,核对版本、SQLite 原生模块与 SHA-256。保留已公开的 preview.20 标签及附件。 +2. 保留 CI 预览启动检查,并用 `npm.cmd run smoke:release` 检查实际 `win-unpacked`。后者不等于运行 Setup。另在干净 Windows 环境用 Setup 安装、启动,再验证卸载;第二条路径先装 preview.20、写入 S0 样本,再原地升级候选版。两条测试均不使用作者的用户目录。 +3. 升级后核对原书库路径、项目 ID、正文摘要值和版本链;编辑样本一章、关闭再重开,确认写回同一 SQLite。独立验证 Node/Agent 仍能读取同一份样本。保留验收记录;任何一项失败均不发布,不用 JSON 回退冒充通过。 + +**回退:** 保留上一已验证安装包和升级前样本备份。先在隔离环境验证旧程序能否读取新版写过的样本;不能直接降级时,使用旧程序和升级前备份恢复到另一位置,不覆盖升级后的作者数据。S2 验收通过即可交付候选包,无须等待性能批次;对外发布另行确认。 + +## S3:按作者操作测量和修改 + +**先测,再改。** 沿用 [500 章基线及计时定义](./PREVIEW14_LONG_NOVEL_PERFORMANCE.md)和[虚构样本](../scripts/fixtures/long-novel-performance.mjs)。增加一档多项目、每章多次生成、采纳并非最后一次成功、含失败续跑与多轮修订的样本;目标约 150–200 MiB,以实际紧凑 UTF-8 JSON 字节数为准。这个数字是测试目标,不是真实作品统计,也不拿带缩进的备份大小与它直接相比。 + +在同一机器、同一依赖和同一份虚构库上分别跑改前与改后。记录启动到项目列表、点击“进入”到工作台可操作、采纳和修订到提交成功的耗时;同时记录每次操作的整库读取次数、跨进程传输字节和实体变更数。桌面操作建议先热身,再至少测 5 次,报告中位数、最慢一次和样本数;备份到期与不需要备份分开测。毫秒数用于对照,不设跨机器 CI 时限。 + +| 操作 | 改动入口与边界 | 可判定的完成条件 | +| --- | --- | --- | +| S3-A 打开项目 | [项目列表](../src/renderer/src/views/HomeView.tsx)当前为更新时间而等待整库保存。改为单项目更新,复用保存队列与版本冲突处理;导入后进入项目也走同一入口 | 在非备份日,打开已有项目不再为时间戳发送完整 AppData;除该项目和必要的存储版本标识外不改其他记录。重启仍按最近打开排序,并发编辑不被覆盖 | +| S3-B 采纳草稿、提交修订 | [SQLite 提交路径](../src/storage/SqliteStorageService.ts)目前两者都在事务内读完整库;生成步骤已有按任务读取的实现,不重写。列出校验实际用到的章节、版本、报告和回执,在原事务内按 ID 核对存在性及项目归属;保留独立的 Agent 授权校验 | 新旧实现对相同样本给出相同的接受或拒绝结果;常规提交不再读取无关项目历史。与当前正文不匹配的报告、过期版本和跨项目引用仍被拒绝;重复提交不增版本,中途报错不留半次修改,恢复后的正文一致 | + +给每批保留一份同机对照:如果整库读取或传输没有下降、交互没有可重复的收益,先排查,再决定是否保留改动。不要因此删除校验、关闭备份或跳过存储版本检查。 + +## S4:有依据再减容 + +- **历史按需读取:先分读写,再减首屏。** [全量保存](../src/storage/sqlite/sqliteSnapshotWriter.ts)把输入缺少的实体视作删除。项目列表或历史摘要只能作为只读结果,不能补空数组后送回全量保存。先试一个入口:列表只拿概要,打开某章历史才取详情。回归场景是“不打开历史、编辑一章、保存、重启”,未加载的步骤、追踪、其他书的版本必须原样存在;全量 JSON 导入导出仍保留原语义。 +- **优先减少新副本,不批量改旧回执。** 只在占用统计证实值得做时,让新回执引用不可变版本和对应报告;缺少可靠引用时保留快照。作者采纳的不一定是最后生成的那一稿,旧章节的修订前正文还可能只在[提交记录](../src/services/ChapterVersionChainService.ts)里。测试要能恢复旧版、查到当时的报告、续跑未完任务;不设“每章只留最后成功”规则。 +- **备份先可恢复,再压缩。** [现有备份](../src/main/BackupService.ts)是未压缩 JSON。先让列表、识别和恢复兼容旧 `.json` 与拟用的 `.json.gz`,验证完整还原、坏文件和空间不足,再给**新自动备份**启用压缩。沿用自动备份保留 7 份的规则,只在新备份成功后轮替。保留旧版能读的普通 JSON 回退副本;不改手动备份和作品 JSON 导出,不自动清历史发行包。库外的 `.usage` 用量收据不随单个库文件恢复;密钥和授权不随作品备份搬运。 + +## 验证与交接 + +每个修改程序的批次运行 `npm.cmd run typecheck`、`npm.cmd test`、`npm.cmd run build`。S3 额外保留 [采纳](../scripts/validate-chapter-commit-bundle.mjs)、[修订](../scripts/validate-p2c-revision-commit-bundle.mjs)、[版本恢复](../scripts/validate-version-chain-restore.mjs)的对照;S4 压缩若启动,还要验证旧 JSON 导入导出、备份列表和恢复。性能记录只提交样本配方、指标与复现方式,不含私人路径、正文或密钥。不调用付费模型。 + +暂不安排流水线引擎搬目录、所有页面改为增删改查接口、清理作者文件或按文件行数重构。**下一次施工从 S0 → S1 开始**,交付的是能复现、能恢复的书库打开失败用例。 + +## English summary + +Start from a clean worktree based on verified upstream `main`. First distinguish an unavailable library from an empty one; preserve the original path and require an explicit recovery choice. Qualify a new Windows candidate with both a clean Setup installation and an in-place upgrade from preview.20 using synthetic data. Next benchmark and narrow two operations: updating the last-opened project and committing an accepted or revised chapter. Lazy history reads, smaller new receipts and compressed automatic backups are conditional; never submit a partial view as a full snapshot, delete an author's only recoverable version, or rotate manual backups automatically. diff --git a/scripts/fixtures/library-preview20/README.md b/scripts/fixtures/library-preview20/README.md new file mode 100644 index 0000000..9398c57 --- /dev/null +++ b/scripts/fixtures/library-preview20/README.md @@ -0,0 +1,38 @@ +# S0:preview.20 虚构旧库 + +`library.sqlite`(73,728 字节)是 Windows x64 公开版 Novel Director +`0.1.6-preview.20`、Electron `39.8.10`、better-sqlite3 `12.9.0` 写出的书库。 +`backup.json` 是同一版本通过“手动备份”接口产生的完整 JSON,不是手工拼出的数据库副本。 +`manifest.json` 记录这两份文件的 SHA-256,以及从窗口回读的正文、版本、回执、任务检查点。 +正文按原始 UTF-8 字节计算 SHA-256,保留段间换行。清单不依赖本机文件路径。 + +两本书和全部正文均为虚构测试内容。没有作者项目、API 密钥、工作台授权或真实用户目录。 +模型配置只指向回环地址;测试程序占用该端口,拒绝并统计任何请求,不代理请求。 +样本本身不是运行中的服务。若端口被其他程序占用,复测会在启动 Electron 前退出。 + +**来源。** 在全新的临时用户目录中,通过公开版 preload 写入两本书、两份未采纳草稿和任务检查点。 +在《纸灯码头》的生产流水线确认“未完成审稿,直接采纳”,再在连贯阅读页手动编辑并点“保存本章”。 +两条正式提交回执都是旧版窗口生成的。用同一版本更改保存位置、创建手动备份并退出; +冷启动核对原路径和版本链,关闭应用且确认没有待写 WAL 后才复制 SQLite。库不是从真实用户目录裁剪而来。 + +| 项目 | 固定检查项 | +| --- | --- | +| 纸灯码头 `s0-quay` | `s0-quay-draft` 已采纳;`generated_draft` → `manual_revision`,第二版引用第一版;当前正文与修订后版本相同。 | +| 潮表留白 `s0-tide` | `s0-tide-job` 的写稿步骤已完成,`await_user_confirmation` 为失败步骤。检查点和草稿仍在;只在工作副本点击“重试失败步骤”,同一任务可无模型请求地完成。 | + +**复测。** 从仓库根目录运行。传给两个版本的路径都是 **win-unpacked 目录**,不是 exe、Setup 或源码目录。 + +```powershell +# 用固定样本复现公开版问题;省略 --preview20 时读取当前仓库的 release/win-unpacked。 +node scripts/qa-library-recovery.mjs s0 --preview20 '<公开版 win-unpacked 目录>' --fixture scripts/fixtures/library-preview20 + +# 重新制作一份等价样本;正式提交会生成新的版本 ID,既有样本不会被覆盖。 +node scripts/qa-library-recovery.mjs s0 --preview20 '<公开版 win-unpacked 目录>' + +# 用此工作区已构建的候选成品验收;默认使用本目录里的固定样本。 +node scripts/qa-library-recovery.mjs s1 --candidate '<候选 win-unpacked 目录>' +``` + +每次运行使用独立的 `tmp/library-s0-*` 或 `tmp/library-s1-*`,保留报告、窗口截图和隔离用户目录。 +新制作的样本在该次 S0 运行的 `fixture` 子目录。可给 S1 传 `--fixture '<该 fixture 子目录>'` 来验同一份新样本。 +S2 安装升级测试应复制 SQLite 或 JSON 到它自己的临时书库,不直接打开此目录中的基准文件。 diff --git a/scripts/fixtures/library-preview20/backup.json b/scripts/fixtures/library-preview20/backup.json new file mode 100644 index 0000000..21f3fc4 --- /dev/null +++ b/scripts/fixtures/library-preview20/backup.json @@ -0,0 +1,598 @@ +{ + "schemaVersion": 3, + "projects": [ + { + "id": "s0-quay", + "name": "纸灯码头", + "genre": "虚构测试", + "description": "", + "targetReaders": "", + "coreAppeal": "", + "style": "", + "createdAt": "2026-01-08T06:00:00.000Z", + "updatedAt": "2026-01-08T06:00:00.000Z", + "lastOpenedAt": "2026-11-11T19:26:04.217Z" + }, + { + "id": "s0-tide", + "name": "潮表留白", + "genre": "虚构测试", + "description": "", + "targetReaders": "", + "coreAppeal": "", + "style": "", + "createdAt": "2026-01-08T06:00:00.000Z", + "updatedAt": "2026-01-08T06:00:00.000Z", + "lastOpenedAt": null + } + ], + "storyBibles": [], + "chapters": [ + { + "id": "4c78e007-7e23-47c5-81f1-dcabb412eb90", + "projectId": "s0-quay", + "order": 1, + "title": "回程票", + "body": "末班渡船离岸后,许遥才看见绳桩旁那盏纸灯。\n\n她掀开灯罩,里面压着一张回程船票,背面写着明早六点。", + "summary": "", + "newInformation": "", + "characterChanges": "", + "newForeshadowing": "", + "resolvedForeshadowing": "", + "endingHook": "", + "riskWarnings": "", + "includedInStageSummary": false, + "archivedAt": null, + "createdAt": "2026-11-11T19:26:04.944Z", + "updatedAt": "2026-11-11T19:26:05.601Z" + } + ], + "chapterTitleReceipts": [], + "characters": [], + "characterStateLogs": [], + "characterStateFacts": [], + "characterStateTransactions": [], + "characterStateChangeCandidates": [], + "foreshadowings": [], + "timelineEvents": [], + "stageSummaries": [], + "promptVersions": [], + "promptContextSnapshots": [], + "storyDirectionGuides": [], + "hardCanonPacks": [ + { + "id": "hard-canon-pack-s0-quay", + "projectId": "s0-quay", + "title": "不可违背设定包", + "description": "这里放不能被 AI 改写的硬设定。不要放长篇剧情回顾。", + "items": [], + "maxPromptTokens": 900, + "createdAt": "2026-11-11T19:26:03.819Z", + "updatedAt": "2026-11-11T19:26:03.819Z", + "schemaVersion": 1 + }, + { + "id": "hard-canon-pack-s0-tide", + "projectId": "s0-tide", + "title": "不可违背设定包", + "description": "这里放不能被 AI 改写的硬设定。不要放长篇剧情回顾。", + "items": [], + "maxPromptTokens": 900, + "createdAt": "2026-11-11T19:26:03.819Z", + "updatedAt": "2026-11-11T19:26:03.819Z", + "schemaVersion": 1 + } + ], + "contextNeedPlans": [], + "chapterContinuityBridges": [], + "chapterGenerationJobs": [ + { + "id": "s0-quay-job", + "projectId": "s0-quay", + "targetChapterOrder": 1, + "pipelineMode": "standard", + "promptContextSnapshotId": null, + "contextSource": "auto", + "aiRunConfig": { + "apiProvider": "local", + "baseUrl": "http://127.0.0.1:57760/v1", + "modelName": "s0-no-model", + "codexCliPath": "", + "codexCliModel": "", + "temperature": 0.8, + "maxTokens": 8000, + "retryEnabled": false, + "maxRetries": 0, + "requestTimeoutMs": 300000, + "schemaVersion": 1, + "roles": { + "planner": { + "apiProvider": "local", + "baseUrl": "http://127.0.0.1:57760/v1", + "modelName": "s0-no-model", + "codexCliPath": "", + "codexCliModel": "", + "temperature": 0.8, + "maxTokens": 8000, + "retryEnabled": false, + "maxRetries": 0, + "requestTimeoutMs": 300000 + }, + "prose": { + "apiProvider": "local", + "baseUrl": "http://127.0.0.1:57760/v1", + "modelName": "s0-no-model", + "codexCliPath": "", + "codexCliModel": "", + "temperature": 0.8, + "maxTokens": 8000, + "retryEnabled": false, + "maxRetries": 0, + "requestTimeoutMs": 300000 + }, + "extraction": { + "apiProvider": "local", + "baseUrl": "http://127.0.0.1:57760/v1", + "modelName": "s0-no-model", + "codexCliPath": "", + "codexCliModel": "", + "temperature": 0.8, + "maxTokens": 8000, + "retryEnabled": false, + "maxRetries": 0, + "requestTimeoutMs": 300000 + }, + "reviewer": { + "apiProvider": "local", + "baseUrl": "http://127.0.0.1:57760/v1", + "modelName": "s0-no-model", + "codexCliPath": "", + "codexCliModel": "", + "temperature": 0.8, + "maxTokens": 8000, + "retryEnabled": false, + "maxRetries": 0, + "requestTimeoutMs": 300000 + }, + "revision": { + "apiProvider": "local", + "baseUrl": "http://127.0.0.1:57760/v1", + "modelName": "s0-no-model", + "codexCliPath": "", + "codexCliModel": "", + "temperature": 0.8, + "maxTokens": 8000, + "retryEnabled": false, + "maxRetries": 0, + "requestTimeoutMs": 300000 + } + } + }, + "status": "completed", + "currentStep": "await_user_confirmation", + "errorMessage": "", + "createdAt": "2026-01-08T06:00:00.000Z", + "updatedAt": "2026-01-08T06:00:00.000Z", + "chapterTaskSnapshot": null, + "taskEdit": null, + "pipelineRecipeId": "standard", + "pipelineRecipeVersion": 1, + "pipelineRecipe": null + }, + { + "id": "s0-tide-job", + "projectId": "s0-tide", + "targetChapterOrder": 1, + "pipelineMode": "standard", + "promptContextSnapshotId": null, + "contextSource": "auto", + "aiRunConfig": { + "apiProvider": "local", + "baseUrl": "http://127.0.0.1:57760/v1", + "modelName": "s0-no-model", + "codexCliPath": "", + "codexCliModel": "", + "temperature": 0.8, + "maxTokens": 8000, + "retryEnabled": false, + "maxRetries": 0, + "requestTimeoutMs": 300000, + "schemaVersion": 1, + "roles": { + "planner": { + "apiProvider": "local", + "baseUrl": "http://127.0.0.1:57760/v1", + "modelName": "s0-no-model", + "codexCliPath": "", + "codexCliModel": "", + "temperature": 0.8, + "maxTokens": 8000, + "retryEnabled": false, + "maxRetries": 0, + "requestTimeoutMs": 300000 + }, + "prose": { + "apiProvider": "local", + "baseUrl": "http://127.0.0.1:57760/v1", + "modelName": "s0-no-model", + "codexCliPath": "", + "codexCliModel": "", + "temperature": 0.8, + "maxTokens": 8000, + "retryEnabled": false, + "maxRetries": 0, + "requestTimeoutMs": 300000 + }, + "extraction": { + "apiProvider": "local", + "baseUrl": "http://127.0.0.1:57760/v1", + "modelName": "s0-no-model", + "codexCliPath": "", + "codexCliModel": "", + "temperature": 0.8, + "maxTokens": 8000, + "retryEnabled": false, + "maxRetries": 0, + "requestTimeoutMs": 300000 + }, + "reviewer": { + "apiProvider": "local", + "baseUrl": "http://127.0.0.1:57760/v1", + "modelName": "s0-no-model", + "codexCliPath": "", + "codexCliModel": "", + "temperature": 0.8, + "maxTokens": 8000, + "retryEnabled": false, + "maxRetries": 0, + "requestTimeoutMs": 300000 + }, + "revision": { + "apiProvider": "local", + "baseUrl": "http://127.0.0.1:57760/v1", + "modelName": "s0-no-model", + "codexCliPath": "", + "codexCliModel": "", + "temperature": 0.8, + "maxTokens": 8000, + "retryEnabled": false, + "maxRetries": 0, + "requestTimeoutMs": 300000 + } + } + }, + "status": "failed", + "currentStep": "await_user_confirmation", + "errorMessage": "虚构样本:等待确认步骤尚未交接。", + "createdAt": "2026-01-08T06:00:00.000Z", + "updatedAt": "2026-01-08T06:00:00.000Z", + "chapterTaskSnapshot": null, + "taskEdit": null, + "pipelineRecipeId": "standard", + "pipelineRecipeVersion": 1, + "pipelineRecipe": null + } + ], + "chapterGenerationSteps": [ + { + "id": "s0-quay-job-await_user_confirmation", + "jobId": "s0-quay-job", + "type": "await_user_confirmation", + "status": "completed", + "inputSnapshot": "{\"targetChapterOrder\":1,\"pipelineMode\":\"standard\"}", + "output": "等待确认", + "errorMessage": "", + "createdAt": "2026-01-08T06:00:00.000Z", + "updatedAt": "2026-01-08T06:00:00.000Z" + }, + { + "id": "s0-quay-job-generate_chapter_draft", + "jobId": "s0-quay-job", + "type": "generate_chapter_draft", + "status": "completed", + "inputSnapshot": "{\"targetChapterOrder\":1,\"pipelineMode\":\"standard\"}", + "output": "{\"title\":\"回程票\",\"body\":\"末班渡船离岸后,许遥才看见绳桩旁那盏纸灯。\\n\\n她掀开灯罩,里面压着一张没有盖章的船票。\",\"checkpoint\":{\"kind\":\"chapter_draft_checkpoint\",\"draftId\":\"s0-quay-draft\"}}", + "errorMessage": "", + "createdAt": "2026-01-08T06:00:00.000Z", + "updatedAt": "2026-01-08T06:00:00.000Z" + }, + { + "id": "s0-tide-job-await_user_confirmation", + "jobId": "s0-tide-job", + "type": "await_user_confirmation", + "status": "failed", + "inputSnapshot": "{\"targetChapterOrder\":1,\"pipelineMode\":\"standard\"}", + "output": "", + "errorMessage": "虚构样本:等待确认步骤尚未交接。", + "createdAt": "2026-01-08T06:00:00.000Z", + "updatedAt": "2026-01-08T06:00:00.000Z" + }, + { + "id": "s0-tide-job-generate_chapter_draft", + "jobId": "s0-tide-job", + "type": "generate_chapter_draft", + "status": "completed", + "inputSnapshot": "{\"targetChapterOrder\":1,\"pipelineMode\":\"standard\"}", + "output": "{\"title\":\"铅字\",\"body\":\"排字房里剩下一块铅字。阿澄把它排进空栏,潮表上多了一个零。\",\"checkpoint\":{\"kind\":\"chapter_draft_checkpoint\",\"draftId\":\"s0-tide-draft\"}}", + "errorMessage": "", + "createdAt": "2026-01-08T06:00:00.000Z", + "updatedAt": "2026-01-08T06:00:00.000Z" + } + ], + "generatedChapterDrafts": [ + { + "id": "s0-quay-draft", + "projectId": "s0-quay", + "jobId": "s0-quay-job", + "chapterId": "4c78e007-7e23-47c5-81f1-dcabb412eb90", + "title": "回程票", + "body": "末班渡船离岸后,许遥才看见绳桩旁那盏纸灯。\n\n她掀开灯罩,里面压着一张没有盖章的船票。", + "summary": "", + "status": "accepted", + "tokenEstimate": 48, + "createdAt": "2026-01-08T06:00:00.000Z", + "updatedAt": "2026-11-11T19:26:04.944Z" + }, + { + "id": "s0-tide-draft", + "projectId": "s0-tide", + "jobId": "s0-tide-job", + "chapterId": null, + "title": "铅字", + "body": "排字房里剩下一块铅字。阿澄把它排进空栏,潮表上多了一个零。", + "summary": "", + "status": "draft", + "tokenEstimate": 48, + "createdAt": "2026-01-08T06:00:00.000Z", + "updatedAt": "2026-01-08T06:00:00.000Z" + } + ], + "memoryUpdateCandidates": [], + "candidateDecisionReceipts": [], + "worldManagementReceipts": [], + "consistencyReviewReports": [], + "contextBudgetProfiles": [], + "qualityGateReports": [], + "generationRunTraces": [], + "runTraceAuthorSummaries": [], + "redundancyReports": [], + "editorialVerdicts": [], + "revisionCandidates": [], + "revisionSessions": [], + "revisionRequests": [], + "revisionVersions": [], + "quickRewriteDrafts": [], + "chapterVersions": [ + { + "id": "4675cd2a-c41b-4b9d-9d98-fca0daa2aaa2", + "projectId": "s0-quay", + "chapterId": "4c78e007-7e23-47c5-81f1-dcabb412eb90", + "source": "manual_revision", + "title": "回程票", + "body": "末班渡船离岸后,许遥才看见绳桩旁那盏纸灯。\n\n她掀开灯罩,里面压着一张回程船票,背面写着明早六点。", + "note": "在连贯阅读页明确保存正文修改。", + "createdAt": "2026-11-11T19:26:05.601Z", + "linkedRevisionCommitId": "a2593e36-1d20-4f02-95c9-a310c2b7ed4f", + "linkedGenerationRunTraceId": null, + "linkedChapterCommitId": "323c8ce2-dc7f-4e7a-9f86-507d17c5aed4", + "baseChapterVersionId": "8886a2a2-65ae-439b-a395-bb888ce8e6c9" + }, + { + "id": "8886a2a2-65ae-439b-a395-bb888ce8e6c9", + "projectId": "s0-quay", + "chapterId": "4c78e007-7e23-47c5-81f1-dcabb412eb90", + "source": "generated_draft", + "title": "回程票", + "body": "末班渡船离岸后,许遥才看见绳桩旁那盏纸灯。\n\n她掀开灯罩,里面压着一张没有盖章的船票。", + "note": "作者明确选择未完成审稿即采纳,已有诊断仍保留。", + "createdAt": "2026-11-11T19:26:04.944Z", + "linkedChapterCommitId": "323c8ce2-dc7f-4e7a-9f86-507d17c5aed4", + "linkedGenerationRunTraceId": null, + "baseChapterVersionId": null + } + ], + "chapterCommitBundles": [ + { + "schemaVersion": 1, + "id": "323c8ce2-dc7f-4e7a-9f86-507d17c5aed4", + "commitId": "323c8ce2-dc7f-4e7a-9f86-507d17c5aed4", + "projectId": "s0-quay", + "chapterId": "4c78e007-7e23-47c5-81f1-dcabb412eb90", + "jobId": "s0-quay-job", + "generatedDraftId": "s0-quay-draft", + "acceptedAt": "2026-11-11T19:26:04.944Z", + "acceptedBy": "user", + "acceptanceReview": { + "mode": "unreviewed", + "draftContentHash": "draft-v1-afac445a97c0c957-43", + "editorialRequired": false, + "qualityGateReportId": null, + "editorialVerdictId": null, + "qualityStatus": "not_available", + "editorialStatus": "not_available" + }, + "chapter": { + "id": "4c78e007-7e23-47c5-81f1-dcabb412eb90", + "projectId": "s0-quay", + "order": 1, + "title": "回程票", + "body": "末班渡船离岸后,许遥才看见绳桩旁那盏纸灯。\n\n她掀开灯罩,里面压着一张没有盖章的船票。", + "summary": "", + "newInformation": "", + "characterChanges": "", + "newForeshadowing": "", + "resolvedForeshadowing": "", + "endingHook": "", + "riskWarnings": "", + "includedInStageSummary": false, + "archivedAt": null, + "createdAt": "2026-11-11T19:26:04.944Z", + "updatedAt": "2026-11-11T19:26:04.944Z" + }, + "chapterVersion": { + "id": "8886a2a2-65ae-439b-a395-bb888ce8e6c9", + "projectId": "s0-quay", + "chapterId": "4c78e007-7e23-47c5-81f1-dcabb412eb90", + "source": "generated_draft", + "title": "回程票", + "body": "末班渡船离岸后,许遥才看见绳桩旁那盏纸灯。\n\n她掀开灯罩,里面压着一张没有盖章的船票。", + "note": "作者明确选择未完成审稿即采纳,已有诊断仍保留。", + "createdAt": "2026-11-11T19:26:04.944Z", + "linkedChapterCommitId": "323c8ce2-dc7f-4e7a-9f86-507d17c5aed4", + "linkedGenerationRunTraceId": null, + "baseChapterVersionId": null + }, + "generatedDraft": { + "id": "s0-quay-draft", + "projectId": "s0-quay", + "jobId": "s0-quay-job", + "chapterId": "4c78e007-7e23-47c5-81f1-dcabb412eb90", + "title": "回程票", + "body": "末班渡船离岸后,许遥才看见绳桩旁那盏纸灯。\n\n她掀开灯罩,里面压着一张没有盖章的船票。", + "summary": "", + "status": "accepted", + "tokenEstimate": 48, + "createdAt": "2026-01-08T06:00:00.000Z", + "updatedAt": "2026-11-11T19:26:04.944Z" + }, + "acceptedMemoryUpdateCandidates": [], + "acceptedCharacterStateChangeCandidates": [], + "appliedCharacterStateFacts": [], + "appliedCharacterStateTransactions": [], + "appliedForeshadowingUpdates": [], + "appliedTimelineEvents": [], + "qualityGateReportId": null, + "consistencyReviewReportId": null, + "generationRunTraceId": null, + "qualityGateReports": [], + "consistencyReviewReports": [], + "redundancyReports": [], + "commitNote": "作者明确选择未完成审稿即采纳,已有诊断仍保留。" + } + ], + "revisionCommitBundles": [ + { + "schemaVersion": 1, + "id": "a2593e36-1d20-4f02-95c9-a310c2b7ed4f", + "revisionCommitId": "a2593e36-1d20-4f02-95c9-a310c2b7ed4f", + "projectId": "s0-quay", + "chapterId": "4c78e007-7e23-47c5-81f1-dcabb412eb90", + "baseChapterVersionId": "8886a2a2-65ae-439b-a395-bb888ce8e6c9", + "newChapterVersionId": "4675cd2a-c41b-4b9d-9d98-fca0daa2aaa2", + "revisionSessionId": null, + "revisionVersionId": null, + "revisedAt": "2026-11-11T19:26:05.601Z", + "revisedBy": "user", + "beforeText": "末班渡船离岸后,许遥才看见绳桩旁那盏纸灯。\n\n她掀开灯罩,里面压着一张没有盖章的船票。", + "afterText": "末班渡船离岸后,许遥才看见绳桩旁那盏纸灯。\n\n她掀开灯罩,里面压着一张回程船票,背面写着明早六点。", + "chapter": { + "id": "4c78e007-7e23-47c5-81f1-dcabb412eb90", + "projectId": "s0-quay", + "order": 1, + "title": "回程票", + "body": "末班渡船离岸后,许遥才看见绳桩旁那盏纸灯。\n\n她掀开灯罩,里面压着一张回程船票,背面写着明早六点。", + "summary": "", + "newInformation": "", + "characterChanges": "", + "newForeshadowing": "", + "resolvedForeshadowing": "", + "endingHook": "", + "riskWarnings": "", + "includedInStageSummary": false, + "archivedAt": null, + "createdAt": "2026-11-11T19:26:04.944Z", + "updatedAt": "2026-11-11T19:26:05.601Z" + }, + "chapterVersion": { + "id": "4675cd2a-c41b-4b9d-9d98-fca0daa2aaa2", + "projectId": "s0-quay", + "chapterId": "4c78e007-7e23-47c5-81f1-dcabb412eb90", + "source": "manual_revision", + "title": "回程票", + "body": "末班渡船离岸后,许遥才看见绳桩旁那盏纸灯。\n\n她掀开灯罩,里面压着一张回程船票,背面写着明早六点。", + "note": "在连贯阅读页明确保存正文修改。", + "createdAt": "2026-11-11T19:26:05.601Z", + "linkedRevisionCommitId": "a2593e36-1d20-4f02-95c9-a310c2b7ed4f", + "linkedGenerationRunTraceId": null, + "linkedChapterCommitId": "323c8ce2-dc7f-4e7a-9f86-507d17c5aed4", + "baseChapterVersionId": "8886a2a2-65ae-439b-a395-bb888ce8e6c9" + }, + "revisionReason": "阅读页手动编辑", + "revisionNote": "在连贯阅读页明确保存正文修改。", + "linkedGenerationRunTraceId": null, + "linkedChapterCommitId": "323c8ce2-dc7f-4e7a-9f86-507d17c5aed4", + "affectedCharacterIds": [], + "affectedForeshadowingIds": [], + "affectedTimelineEventIds": [] + } + ], + "chapterWritebackFailures": [], + "agentRuns": [], + "agentActionPreviews": [], + "settings": { + "apiProvider": "local", + "apiKey": "", + "hasApiKey": false, + "baseUrl": "http://127.0.0.1:57760/v1", + "modelName": "s0-no-model", + "codexCliPath": "codex", + "codexCliModel": "", + "temperature": 0.8, + "maxTokens": 8000, + "retryEnabled": false, + "maxRetries": 0, + "requestTimeoutMs": 300000, + "pipelineModelRoles": { + "planner": { + "apiProvider": "local", + "baseUrl": "http://127.0.0.1:57760/v1", + "modelName": "s0-no-model", + "codexCliPath": "", + "codexCliModel": "", + "maxRetries": 0, + "retryEnabled": false + }, + "prose": { + "apiProvider": "local", + "baseUrl": "http://127.0.0.1:57760/v1", + "modelName": "s0-no-model", + "codexCliPath": "", + "codexCliModel": "", + "maxRetries": 0, + "retryEnabled": false + }, + "extraction": { + "apiProvider": "local", + "baseUrl": "http://127.0.0.1:57760/v1", + "modelName": "s0-no-model", + "codexCliPath": "", + "codexCliModel": "", + "maxRetries": 0, + "retryEnabled": false + }, + "reviewer": { + "apiProvider": "local", + "baseUrl": "http://127.0.0.1:57760/v1", + "modelName": "s0-no-model", + "codexCliPath": "", + "codexCliModel": "", + "maxRetries": 0, + "retryEnabled": false + }, + "revision": { + "apiProvider": "local", + "baseUrl": "http://127.0.0.1:57760/v1", + "modelName": "s0-no-model", + "codexCliPath": "", + "codexCliModel": "", + "maxRetries": 0, + "retryEnabled": false + } + }, + "enableAutoSummary": false, + "enableChapterDiagnostics": false, + "defaultTokenBudget": 16000, + "defaultPromptMode": "standard", + "theme": "system" + } +} \ No newline at end of file diff --git a/scripts/fixtures/library-preview20/library.sqlite b/scripts/fixtures/library-preview20/library.sqlite new file mode 100644 index 0000000000000000000000000000000000000000..716a67ee5ce5ed730392218d9965072d2c2b8ee4 GIT binary patch literal 73728 zcmeHQdu$s=dZ#2xvSf;DUoIykf*O^Ho47V*QldmrsqU&cN}@P^$@1mSaUiUAN7hE< zF0;Fot#buPmX-LC*l8|~-$!3fOXyH7K|s{7pou94-x zUH)w4-&F2^2>PN3Py{Ff6ak6=MSvne5x7MN{LEEe9jYxFT_cD*+9U9LdXT6Js(_Rp zuAh}zPDQd2QnkLTqrJPMu5;7l9Y3kFr>(5hUYj@78L1xOq;ME&!fR1b%h@ z^73jglzmTj*)K@FMtWE7Qh{oo@_+)DG`ga)ym~7X^`q>f=Bn2di}I|B_#XH}KvTvlGa0qPdY zTeqkzy#S0gu&OAiRwU6p^&wOm@2bnotJke78hu`6d&5ZK`cZ`Ki3q!77WXM7D_|L^ zM|b;U&vevfMb|w-${s|r0vSMe?(TT9qpNO9SLep|uAkOz==kYcLc&I)VL|H8{(f8Gg4v^Nbd~df%qf^6J{!qWzLqGXn?Xf0?BoMyOf*T>r7=ZImtW_Th8TXz>QQ z!9kMD2-XKcCZ#^E0o}}qxxQw=Vq^qDA&e~KbDh}|{DK0QTF^4GNCVg+6kverfUYYm zt6pDQ zA!eNi(2x$aO{E9fa7<@cMyGwaxUBlodvjF}=ilSUe@kG3S%xQ>D)=p7{=~uj2Xhww z(HBL4B0v$K2v7tl0u%v?07ZZzKoOt_Py{Ffw-|wkN=hDRBU*-bwZYfW?DhNM-R%#x zHZ-^Tn(Bki4LdrEowcLoG8z=HJ}VcMx2dlQwKN6$ysSSE@&+1%E#8nXz6`~kiN z`B+~|U}aIs!;G#n`?0PbYpjGel`#M0V1C7X4gctiB0v$K2v7tl0u%v?07ZZzKoOt_ zPy{Ff6oI=0fvS=RicGaaiL0cx$j}}XuPAxA!lD4c_Wz>FUpe4SUlaj~07ZZzKoOt_ zPy{Ff6oDmxz@eI=(wd*|ZR1fN8w;y@J#19iB1?k;xZHbMJz>j0syx&oVt;ww(@JUxR9z4bk9I&ZO!Hz4DQdS6K~hZJ2!V}|7N?F1Q4|(Lq?Oqv zfzQ3Cb+0EHW*xJ@#6~p6=@^29iDX97K=n!8H5ZOvwF;W{UM;o1$LJxDXPH1fFuM&W@*F!u^7J# zsh(DUvo?%L6iBj2RK;T%%N&&zc~<6OaH;SdjTSVdfJE^O=kgoC_WvU0za8+VFNy#~ zfFeKOX|seifC z%{^LL<_bPo1I1Q2+9ry~c^6M2&`D|sfjlksDxgw*F~$yg;nWNAQPXPp{9a#6w+|ke z@WHztv?DwAdIZpnVjZlJ08}F6XpPRa7_2B^Wl*aS9n5BY4hyJQtXfj5W(BeVjmvsT zR6)PDO=P1=zog{*+Hy4px^@uRk!Tpf(O@8_ zxtNSkMBApKC>GwkSwZcIDR4v=$n`z}ACil~0j0hyOY%mfDC{mQ<#9?G;p4^HK}GM$ zH7@~~>6n+Fj7)&ttTJHCF~$s|LRbecKsox1V;JoU;#>}hnqnA_H zCsJdtq^7UMr%%RT8PB#mG}gEHL&gl9BS*j~W46ZSK&W<36(DC@7Q{nkI9Z75HaAb| z`ia?>UrD@i4aietS5s%-uWRqDOT2L<{?du$$^FU65A=HedSoA_cy|;?XOCS^UVb@# z@V&&?c>L7cvy;~#7IKEusv^)?o_k(1$m(Vn!z>%ZcGWEYTb||ATMhYS@!y^-a}ou{ zI2Xhiyzn9GA6u{%&KR5{G$4a9B=oz>nEj~4Rl8kVxjxW#pqdO^CBOyYsas#r8$=C_ z-awFR^0xT<{9c}8dwT=^2Gkq!5&l3oOP)WRoI0P)lcxG+pWhfPg=CqNLD~_6+PF7Z zWflu*dLz|hm#W{ zH)bH}j9xM>W9mH4eM}>OF*z}kcW7tUlawZS zjgu^45ijn-?hJ(T2}wpEvv_GY92uvFLVy$=M7&KD5_uqdN|FW$sh|uz%gQ1?ZcYyo zL@pfT5#K3pQ$g%!T60;M>lYwzJOhxuK<1QS8S8=iP#~}a8kJup=v}}wysx&{RjUbR z$EvomPBIMfrU=W%#MSB4W6Uh@-bIm?;yCNb@Mcq{;JuOsc7ZS1Wi6?d9Wa91p ziBArvE+2znt7wG)edpALSlBZWgg8LFJsL&eQKKr3|@_m7w<_gC&eXMVDIn==` z@)wRblX(hf@)vHV&QZAghtA-3^8xSmQY0d%kHthDp6g&6nnI1J5l)rk8+_hCpRd6i zYT`m(w$bn78hV3>5A;>FIh(c{O}C_OY-r?KIMe_dD>zjyfP$Kk;zKRoK92SAEMy74m5(?>4;LA1-)|%W{jaeSZ)_ssWr#`d4!5Qam%1u6@~`G zsv;=Q9aVOZbG0KwDNt$?=7b%;nY%LY{@?YLqw3!nr~4nP2CE*ca=X5AGwwfPj=BZ+ z1j8{w=3kgksxH$0OA(+5Py{Ff6ak6=MSvne5ugZA1Qsg-jm}V8_6AEcTIV_dZskp+ zS#c9-n43uBZg2+M?Dw(uzTFKws{GC-69UWrHdhMQTIU)wg|Y9=Ce}|bTwWNletu!4 z)@1vAaxQjM%$LUIIK^ceOtz-?T*Z_xJW1Ia=jyhMeRwuZ`kAk#3!rtba@LyoE7>{f zSe9s1#)JI1npOM|W^4YfSt9UoO^hPU}Sx zpa@U|C;}7#iU37`B0v$K2v7tl0=Eo-KX8U@PRPqVTybS#WGnKMVJ~)k;2u*LnLCep z{~>3SS#68@gFFa;i&=oBU#pP427@@};|weElNrs!_J7mEAlv?*BWfY`e>02u?f-7- z|9{KI3eAWjKoOt_Py{Ff6ak6=MSvne5ugZA1PB7u{~zB~uvqs0`27De#{q}?@2V~? zf2`udvK8>0e zAlc`|G3G6nN@m{%Zcwyx#dq@LQmf$uL-0WzW72yJwrHuewEe>5x0p>4dK0_;^|H~|=-?hEO zn<}G00qy<{^jzFfMY~#gNpo}B&yi90Wmr5Tmm{I z%aXhiDGFQ!O+;)Jc(5^r3I!(HPi_a`SmNKIdhPoIQ~#xcda zqd+=)?0WL@%khKnCC0|#p7PnrYx*_h0?eDxS)O|yCSZ0COx~~v=#~bM2!SD=4|+}} z|1I9Bx-TfyF#l!WCRyer3XE|sh%tD1Lsm()U}Za(>SdFq6TZXB%dw&)dZR2i;3Zi> z12Pz6ETaTfyXo7utR>Bu{iwuMyIotkKG1fc+BkbFzy({7&lmItQA49Q02iitTl{@~ zxJ`}i?G5-FP;babK!t9WJbySjbv~OXy2CHwNg-M0WRPSoZH+Nx77J;5C4LCRSmN~i z$)lGNGn1(USCa339{+GA{@RhRzj!4zJu^Exl{oln^7P@Pq6sNc@YD zaTi`_63QpwHhQI><)z(%2&--!3IS4h5b-uqNaTU+DM=b2q=GW=EGvr;NU{_}E*#?# z-zjcWLF{icD%Qt_6$DFsnd=uI?{-r%=NAdO7d=!ezev!#fMW&nJ(aNS?Sl|E`EgQ*yt3dRnLsn7Cy;r=?DuOuW56@yX%T`kKCv`> zu!z$8X!6Wx{M^U$<3ofR!6!ZKU5Sq;Td)M`8~ni?o?*QIcNPshmTf4jcOHNb^mXSD zcxhRsYt8nW?NH(aZHLOq)L}e#Q^<9gWCkf@W-!Hlc4x58G6mei38TP%89#+8TAG~m5<}hCPOdQwgfbFwyT~A!wpEz_VK0TH= z@_Op}spN&v5@(LWYDk}rZ_FHupF4TuvzH*@jhTzHr~blbHDvB!jmD^WL(-wB0<4dq zD+XF_?Yuj@|Ndo|OWT1<54641l#@$%{?rj(fPf;T7;AIDXUi&m)s&&&HQJxSOfrq9 zF;lBp7Bf8siYkNMX7$oIerHzBi4P|erw=Aiy_tOf4D4X_C3L=?!Ezje;fz%qx3F_q zBotZ44j*kOPgCJPM+VEBzafx}hjSq6W}VAU6*MD$P-Mn{#0 zIz+ZN47_h8I}}h{bV0a)_edJei!r3A-9iLOF?AyZHbPpQI#c@q`B4@Ak~7s%75>sQ zzX4Cp1@zLkKUq}xeXM{59V~ZlPJd5y9cMmGoc`eJFHR;e9gJTahc$a@`ds4D)a=D` z@jpA6IDPTPXXg?}rgALX?KKqzDm2J!dB^qrc>iBq{5wb0R`^3-6oKy&0&i?}xtc)# z0^1o!+k4B&7_>YpRLI`<=Jq(*Tbri(U~|Kcy*8^a0oHJyLaKr#pcIJ+s9N}dc8I|gpvczj_IPH{%F^6+MD;0F!45K7kay7UW&2nz5NKjULC(j8 znu6XyBcwoJWR+xr4R@gzV}hKvkcQTU1$F32*kyO2s3dEx(p3UxBG7^G0+$v;5`pbN zH#>v6sD~Ydi3=~%(~HYN0aH8S32vf*yF`I}v1jmNvIrHrul)|%e~j(5$$is?PdY(C z3brgo*UgV|Q(>gtrFKPt|QP1AEdO9^SuB&8cM-CR4yLEn zdz6QHWj#2Y3it!4Ww6tUQ6cC_y0sh65?J33ChsiTXf!M!-b(re*vO5%?Bc-~AS${$ zpOwsFE7vP$CBOJSja(QWU%PzHrgva787+oQuMkUC$5*&Q+Z@=c z=x7`7AU*j>@Eege2cpa)EW9}!K8dXYo2Cl4SLDhJI!4;eYM4XtL^Bk{yOtbv00MJ% zGSyvz7>$_^Awb%_JBeEk$bM|a)vO^qu@nAIkKyl$p9O%Ro#sOgJ3YgYIhX(-FUSKv zXLz@M%i6Kw@s*4#w3V>&0GpGIcuk9W5g19(OUI0=W6AfYrB}}xh z0mPd;gnhc{b$mP6h`R#zpU_xsP82&m&=atgS_}TB{(j)_SD43NuKk@>9e7@Z8;DAR zs6wTn+5>{fYt^%G4|q982940xuAMre!C5T~x-;(28GRT+vc&N>(Aa{*f;&yLl+52J za|~;v>I>_XIWmJr=IxcalijgynXX#D{akwe_o`e$-NVY!b~a>~Ml346Li}e4mkQCh zWG}_$(d!jbJ|JbSyS5rDFy$><*4b}0Jq4~VL3guWpOw~EYKx7mw<$(!^R!SceXbA zbWxy#HW7WMa-onx0?o4Tm5WI!xgZoQYJ~5gn$mDn &?aG=|F3)07KY7a{NO6_ba zrU2A6j9p9AuXJwSga;KANd$>L=azf$aQx8uF~Py{Ff6ak6= zMSvne5ugZA1SkUE1_Jx}m1#CF createHash('sha256').update(value).digest('hex') +const at = '2026-01-08T06:00:00.000Z' + +// Only projects, unaccepted drafts and pipeline checkpoints are seeded. The packaged +// renderer must create both formal receipts, the chapter and its versions itself. +export function seed(baseUrl) { + assert.match(baseUrl, /^http:\/\/127\.0\.0\.1:\d+\/v1$/) + const model = { apiProvider: 'local', baseUrl, modelName: 's0-no-model', retryEnabled: false, + maxRetries: 0, codexCliPath: '', codexCliModel: '' } + const roles = Object.fromEntries(['planner', 'prose', 'extraction', 'reviewer', 'revision'].map(k => [k, model])) + const jobs = books.map(({ id }, i) => ({ + id: id + '-job', projectId: id, targetChapterOrder: 1, pipelineMode: 'standard', + promptContextSnapshotId: null, contextSource: 'auto', aiRunConfig: { ...model, roles, schemaVersion: 1 }, + status: i ? 'failed' : 'completed', currentStep: waitStep, + errorMessage: i ? '虚构样本:等待确认步骤尚未交接。' : '', createdAt: at, updatedAt: at + })) + const drafts = books.map(({ id }, i) => ({ + id: id + '-draft', projectId: id, jobId: jobs[i].id, chapterId: null, title: i ? '铅字' : '回程票', + body: i ? waitBody : acceptedBody, summary: '', status: 'draft', tokenEstimate: 48, + createdAt: at, updatedAt: at + })) + return { + projects: books.map(book => ({ ...book, genre: '虚构测试', description: '', targetReaders: '', + coreAppeal: '', style: '', createdAt: at, updatedAt: at })), + chapterGenerationJobs: jobs, generatedChapterDrafts: drafts, + chapterGenerationSteps: jobs.flatMap((job, i) => ['generate_chapter_draft', waitStep].map((type, n) => ({ + id: job.id + '-' + type, jobId: job.id, type, status: n ? job.status : 'completed', + inputSnapshot: JSON.stringify({ targetChapterOrder: 1, pipelineMode: 'standard' }), + output: n ? (i ? '' : '等待确认') : JSON.stringify({ title: drafts[i].title, body: drafts[i].body, + checkpoint: { kind: 'chapter_draft_checkpoint', draftId: drafts[i].id } }), + errorMessage: n ? job.errorMessage : '', createdAt: at, updatedAt: at + }))), + settings: { ...model, apiKey: '', hasApiKey: false, enableAutoSummary: false, + enableChapterDiagnostics: false, pipelineModelRoles: roles } + } +} + +// Verify relationships and content against the recipe BEFORE using a readback as an oracle. +export function signature(data) { + const ids = data.projects.map(p => p.id).sort() + assert.deepEqual(ids, books.map(p => p.id)) + assert.equal(data.chapters.length, 1) + assert.equal(data.chapterCommitBundles.length, 1) + assert.equal(data.revisionCommitBundles.length, 1) + assert.equal(data.chapterVersions.length, 2) + assert.equal(data.chapterGenerationJobs.length, 2) + assert.equal(data.generatedChapterDrafts.length, 2) + const [chapter] = data.chapters, [accept] = data.chapterCommitBundles, [revise] = data.revisionCommitBundles + const draft = data.generatedChapterDrafts.find(d => d.id === draftId) + const first = data.chapterVersions.find(v => v.source === 'generated_draft') + const second = data.chapterVersions.find(v => v.source === 'manual_revision') + assert(first && second && draft) + for (const row of [draft, first, second, accept, revise]) { + assert.equal(row.projectId, books[0].id) + assert.equal(row.chapterId, chapter.id) + } + assert.equal(chapter.projectId, books[0].id) + assert.equal(draft.status, 'accepted') + assert.equal(accept.generatedDraftId, draft.id) + assert.equal(accept.jobId, draft.jobId) + assert.equal(first.linkedChapterCommitId, accept.commitId) + assert.equal(second.linkedChapterCommitId, accept.commitId) + assert.equal(second.linkedRevisionCommitId, revise.revisionCommitId) + assert.equal(revise.linkedChapterCommitId, accept.commitId) + assert.equal(first.baseChapterVersionId ?? null, null) + assert.equal(revise.baseChapterVersionId, first.id) + assert.equal(second.baseChapterVersionId, first.id) + assert.equal(revise.newChapterVersionId, second.id) + for (const text of [draft.body, first.body, accept.chapter.body, revise.beforeText]) assert.equal(text, acceptedBody) + for (const text of [chapter.body, second.body, revise.afterText]) assert.equal(text, revisedBody) + const job = data.chapterGenerationJobs.find(j => j.id === waitJob) + const steps = data.chapterGenerationSteps.filter(s => s.jobId === waitJob) + const generated = steps.find(s => s.type === 'generate_chapter_draft') + const waiting = steps.find(s => s.type === waitStep) + const pending = data.generatedChapterDrafts.find(d => d.jobId === waitJob) + assert.equal(job.projectId, books[1].id) + assert.equal(job.status, 'failed') + assert.equal(job.currentStep, waitStep) + assert.equal(waiting.status, 'failed') + assert.equal(generated.status, 'completed') + assert.equal(JSON.parse(generated.output).checkpoint.draftId, pending.id) + assert.equal(pending.status, 'draft') + assert.equal(pending.body, waitBody) + return { projectIds: ids, chapterId: chapter.id, chapterSha256: digest(chapter.body), + acceptedDraftId: draft.id, acceptanceId: accept.commitId, revisionId: revise.revisionCommitId, + versions: [first, second].map(v => ({ id: v.id, base: v.baseChapterVersionId ?? null, + source: v.source, bodySha256: digest(v.body) })), + unfinished: { jobId: job.id, projectId: job.projectId, status: job.status, stepId: waiting.id, + stepType: waiting.type, draftId: pending.id, draftSha256: digest(pending.body), + checkpointSha256: digest(generated.output) } } +} From 804f73858110b66ddb979d03e4339ff925144de2 Mon Sep 17 00:00:00 2001 From: SHIN09-git Date: Thu, 12 Nov 2026 03:57:40 +0800 Subject: [PATCH 2/8] fix(storage): keep unavailable libraries closed until explicit recovery --- README.md | 12 +- ROADMAP.md | 6 +- .../LIBRARY_STABILITY_AND_PERFORMANCE_PLAN.md | 20 +- package.json | 2 + scripts/qa-library-recovery.mjs | 794 ++++++++++++++++++ scripts/run-tests.mjs | 3 + scripts/validate-agent-candidate-undo.mjs | 19 +- scripts/validate-agent-library-choice.mjs | 193 +++++ scripts/validate-agent-revision-tools.mjs | 33 +- scripts/validate-library-session.mjs | 208 +++++ scripts/validate-library-storage.mjs | 306 +++++++ scripts/validate-secure-credentials.mjs | 3 +- scripts/validate-sqlite-storage.mjs | 2 +- src/agent/AgentProjectOverviewRuntime.ts | 16 +- src/agent/AgentRuntime.ts | 184 ++-- src/agent/agentLibraryChoice.ts | 71 ++ src/agent/agentPipelineWritebackGuard.ts | 7 +- src/agent/revision/agentRevisionGeneration.ts | 4 +- src/agent/tools/agentBookProjection.ts | 27 +- src/agent/tools/agentChapterTitleTools.ts | 26 +- src/agent/tools/agentWorldTools.ts | 4 +- src/main/AppConfigService.ts | 43 +- src/main/BackupService.ts | 5 +- src/main/LibrarySession.ts | 154 ++++ src/main/dataMerge/dataFileStorage.ts | 13 +- src/main/index.ts | 29 +- src/main/ipc/backupIpcHandlers.ts | 2 +- src/main/ipc/dataIpcHandlers.ts | 17 +- src/main/ipc/libraryRecoveryIpc.ts | 55 ++ src/main/ipc/storageDataOperations.ts | 25 +- src/main/ipc/storageManagementIpcHandlers.ts | 10 +- src/main/libraryFiles.ts | 77 ++ src/preload/index.ts | 6 + src/renderer/src/App.tsx | 19 +- src/renderer/src/hooks/useAppData.ts | 58 +- src/renderer/src/hooks/useLibraryOpen.ts | 80 ++ src/renderer/src/styles/app-shell.css | 60 ++ .../src/views/LibraryRecoveryView.tsx | 35 + .../tests/library-recovery-window.mjs | 154 ++++ .../tests/validate-library-recovery.mjs | 34 + src/shared/ipc/ipcChannels.ts | 1 + src/shared/ipc/ipcTypes.ts | 7 + src/storage/JsonStorageService.ts | 69 +- src/storage/SqliteStorageService.ts | 48 +- src/storage/jsonReadonly.ts | 7 +- src/storage/jsonWriteLock.ts | 4 +- src/storage/libraryFile.ts | 10 + src/storage/libraryValidation.ts | 75 ++ src/storage/sqlite/sqliteInitialization.ts | 24 +- src/storage/sqlite/sqliteLibrary.ts | 66 ++ src/storage/sqlite/sqliteSchema.ts | 28 + src/storage/sqlite/sqliteSnapshot.ts | 31 +- src/storage/sqlite/sqliteTypes.ts | 1 + 53 files changed, 2861 insertions(+), 326 deletions(-) create mode 100644 scripts/qa-library-recovery.mjs create mode 100644 scripts/validate-agent-library-choice.mjs create mode 100644 scripts/validate-library-session.mjs create mode 100644 scripts/validate-library-storage.mjs create mode 100644 src/agent/agentLibraryChoice.ts create mode 100644 src/main/LibrarySession.ts create mode 100644 src/main/ipc/libraryRecoveryIpc.ts create mode 100644 src/main/libraryFiles.ts create mode 100644 src/renderer/src/hooks/useLibraryOpen.ts create mode 100644 src/renderer/src/views/LibraryRecoveryView.tsx create mode 100644 src/renderer/tests/library-recovery-window.mjs create mode 100644 src/renderer/tests/validate-library-recovery.mjs create mode 100644 src/storage/libraryFile.ts create mode 100644 src/storage/libraryValidation.ts create mode 100644 src/storage/sqlite/sqliteLibrary.ts diff --git a/README.md b/README.md index 5af567a..31f4727 100644 --- a/README.md +++ b/README.md @@ -14,7 +14,7 @@ Novel Director 是一个实验性的、本地优先的 AI 长篇小说工作台 - 实验性本地优先桌面应用。 - Windows 是主要开发和验证平台。 -- 项目数据默认保存到本地 SQLite 数据库;JSON 导入、导出和 fallback 仍然保留。 +- 项目默认保存到本地 SQLite。仍可导入、导出 JSON,也可明确选择一份 JSON 作为独立书库。 - AI 功能是可选的,取决于你配置的模型供应商。 - 源码仓库不跟踪预构建 `.exe` 或打包产物。 @@ -95,7 +95,7 @@ Novel Director 使用 `better-sqlite3` 作为 main process 的 SQLite 后端。 - 跑 Agent CLI、验证脚本或 Node.js 侧 SQLite 测试前,运行 `npm.cmd run native:node`,把 `better-sqlite3` 恢复为本机 Node.js ABI。 - 打包使用 `npm.cmd run dist:win`,脚本会使用仓库内 `.electron-cache` 和 `.electron-builder-cache`,减少 C 盘缓存压力。 - 打包流程会在生成 `release/win-unpacked/Novel Director.exe` 后运行 `npm.cmd run smoke:packaged`,检查图标、preload API、SQLite 默认路径、导入导出 API 和无 API Key 状态。 -- 如果 `better-sqlite3` rebuild 失败,先安装 Visual Studio C++ Build Tools;应用代码仍保留 JsonStorageService fallback,但正式安装包应优先修复 native 绑定后再发布。 +- 如果 SQLite 组件无法加载,桌面版会显示“书库未打开”,不会自动切换到 JSON。修复安装后点“重试”。明确选择独立 JSON 时,之后的修改只写入该文件。 单独运行打包后烟测: @@ -109,7 +109,7 @@ npm.cmd run smoke:packaged 排查建议: -- 如果 SQLite 数据库损坏,先备份当前 `novel-director-data.sqlite`,再尝试从设置页恢复最近备份或导入旧 JSON。 +- 如果看到“书库未打开”,先核对错误页的路径。外接磁盘断开时,接回原盘并点“重试”。文件损坏时,点“从备份恢复到新文件”,选择 JSON 备份和一个未用过的文件名,原库不会被覆盖。不需要先建空项目。 - 如果 JSON 导入失败,确认文件来自 Novel Director 导出或旧版 `novel-director-data.json`,不要直接导入章节 TXT/Markdown。 - 如果迁移或合并失败,不要删除源文件;检查自动生成的 `.bak` 或迁移前备份。 - 如果备份恢复失败,保留失败文件和日志,通过 Settings 的日志导出或 issue 模板提交脱敏信息。 @@ -219,7 +219,7 @@ Start with the [beginner tutorial](./docs/BEGINNER_TUTORIAL.md) or [quickstart]( - Experimental local-first desktop app. - Windows is the primary development target. -- Data is stored locally in SQLite by default; JSON import, export, and fallback remain available. +- SQLite is the default local store. JSON import, export, and explicitly selected standalone JSON libraries remain available. - AI features are optional and depend on the provider you configure. - No prebuilt binaries are tracked in this source repository. @@ -289,7 +289,7 @@ Recommendations: - Before running Agent CLI, validation scripts, or Node-side SQLite tests, run `npm.cmd run native:node` to restore the `better-sqlite3` binding for the local Node.js ABI. - Use `npm.cmd run dist:win` for packaging. The script uses repository-local `.electron-cache` and `.electron-builder-cache` directories to reduce C drive pressure. - The packaging flow runs `npm.cmd run smoke:packaged` after creating `release/win-unpacked/Novel Director.exe`; the smoke test checks the icon, preload API, SQLite default path, import/export API, and no-key state. -- If `better-sqlite3` rebuild fails, install Visual Studio C++ Build Tools first. The app retains JsonStorageService fallback in code, but release packages should be shipped only after the native binding is healthy. +- If the SQLite component cannot load, the desktop app shows a recovery screen instead of switching to JSON. Repair the installation and retry. Changes made to an explicitly selected JSON library stay in that file. Run packaged smoke test separately: @@ -303,7 +303,7 @@ If the project list is empty on first launch, click `导入旧数据 JSON` and s Troubleshooting: -- If the SQLite database is corrupted, back up `novel-director-data.sqlite` first, then restore a recent backup or import old JSON from Settings. +- On the library recovery screen, check the displayed path. Reconnect the original disk and retry, or restore a JSON backup to a new filename. Recovery does not overwrite the original library or require an empty project first. - If JSON import fails, make sure the file is a Novel Director export or legacy `novel-director-data.json`, not a chapter TXT/Markdown export. - If migration or merge fails, do not delete the source file; check the generated `.bak` files or pre-migration backups. - If backup restore fails, keep the failed file and logs, then share redacted diagnostics through the security or issue process. diff --git a/ROADMAP.md b/ROADMAP.md index 43490f8..2f940f3 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -17,11 +17,11 @@ ## 下一步优先事项 -执行[书库可靠性与长篇性能施工计划](./docs/LIBRARY_STABILITY_AND_PERFORMANCE_PLAN.md)。**当前只完成核查,尚未开始实施。** 从 S0 的干净主线工作区开始;下一版只需完成 S1、S2,不等所有性能工作做完才交付。 +执行[书库可靠性与长篇性能施工计划](./docs/LIBRARY_STABILITY_AND_PERFORMANCE_PLAN.md)。**S0 已固定旧版样本,S1 已实现书库恢复。下一项是 S2:实际运行 Windows 安装器,验证干净安装和从 preview.20 升级。** 这些更改未包含在已发布的 preview.20 中。 | 优先级 | 工作 | 完成条件 | | --- | --- | --- | -| S1,下一版 | 书库打开失败时不切换到可写空库 | 原路径和文件不变;作者能重试、选择已有库或从备份恢复;正常首次建库不受影响 | +| S0 / S1,本地完成 | 保留 preview.20 虚构库;区分书库未打开和空书库 | 从错误页重试、选择已有库、恢复至新文件或明确新建;不自动切换独立 JSON | | S2,下一版 | 核对运行时依赖,用 Windows 安装器验证升级 | 干净安装、从 preview.20 升级后均能读写 SQLite;重启后作品和版本链仍在 | | S3-A / S3-B | 先优化打开项目,再优化采纳与修订 | 更新时间不传整库;提交不读取无关项目历史。用虚构长篇对照耗时,保留并发、回滚和版本恢复能力 | | S4,有依据再做 | 历史详情按需读取、减少新副本或压缩自动备份 | 不把局部数据当整库保存;不删除作者采纳来源;旧备份可恢复,不自动删手动备份 | @@ -58,6 +58,6 @@ npm.cmd run build Updated 2026-11-12. The [preview.20 prerelease](https://github.com/SHIN09-git/novel/releases/tag/v0.1.6-preview.20) includes the [long-novel fixes](./docs/LONG_NOVEL_COMPLETION_REPAIR_PLAN.md). Provider balances and unrecorded historical usage remain unknown. -Next, follow the [library reliability and performance plan](./docs/LIBRARY_STABILITY_AND_PERFORMANCE_PLAN.md). No implementation is claimed yet. The next release candidate must distinguish an unavailable library from an empty one and pass a real Windows Setup upgrade from preview.20. Then measure and narrow project opening and chapter commits. History compaction and automatic backup compression are conditional, not release blockers. +The [library reliability and performance plan](./docs/LIBRARY_STABILITY_AND_PERFORMANCE_PLAN.md) now has a synthetic preview.20 library and an explicit desktop recovery flow. Neither changes the existing public release. Next, verify a clean Windows Setup installation and an in-place upgrade from preview.20. Then measure and narrow project opening and chapter commits. History compaction and automatic backup compression are conditional, not release blockers. Keep project isolation, version conflict checks, idempotent commits, report provenance, rollback, backups, authorization and credential redaction. Full CRUD rewrites, FTS, graphs, cloud sync and generic extension systems remain deferred. Historical plans and measurements are references, not additional mandatory work. diff --git a/docs/LIBRARY_STABILITY_AND_PERFORMANCE_PLAN.md b/docs/LIBRARY_STABILITY_AND_PERFORMANCE_PLAN.md index 523ec5c..4ece2c9 100644 --- a/docs/LIBRARY_STABILITY_AND_PERFORMANCE_PLAN.md +++ b/docs/LIBRARY_STABILITY_AND_PERFORMANCE_PLAN.md @@ -1,6 +1,6 @@ # 书库可靠性与长篇性能施工计划 / Library Reliability and Performance -拟定于 2026-11-12。**状态:已完成问题核查,S0 至 S4 均未实施。** 本计划承接 [ROADMAP](../ROADMAP.md),不重新排期已发布的长篇完稿修复。 +拟定于 2026-11-12。本计划承接 [ROADMAP](../ROADMAP.md),不重新排期已发布的长篇完稿修复。**S0 样本已固定,S1 已实现;Windows Setup 安装、升级及发布仍属 S2。** 下一版先让作者确信“打开的是原来的书库”,并用真正的 Windows 安装器验证升级。之后再缩短打开一本书、采纳或修订一章的等待时间。不要求先重构全库,也不拿删除作者历史来换取性能。 @@ -23,7 +23,7 @@ ## S1:打开失败,不冒充空库 -**现状。** [存储工厂](../src/storage/SqliteStorageService.ts)在原生 SQLite 模块加载失败时会返回 JSON 后端;[主进程启动](../src/main/index.ts)可能随之改写配置路径。[界面加载](../src/renderer/src/hooks/useAppData.ts)遇到读取错误也会设置空数据。只改一条错误文案不能区分“库是空的”和“库没读出来”。 +**preview.20 的问题。** SQLite 模块加载失败时,旧版可能切换为独立 JSON 并改写路径;文件缺失或损坏时,旧界面还可能显示可编辑的空项目列表。S1 将[打开与恢复](../src/main/LibrarySession.ts)分开:先完整读出所选库,再交给保存功能。读取已有库不建表,不用旁边的文件修复它;只有首次建库、明确新建或恢复才初始化文件。 | 情形 | 作者应看到 / 程序应做什么 | | --- | --- | @@ -36,6 +36,20 @@ **验收:** 分别模拟模块不可用、已有库损坏及已配置的库暂时找不到;确认原数据库、配置和同目录 JSON 未被改写,也没有新建空库。恢复正常条件后点“重试”,两本书及版本链都在。另跑新用户首次启动、旧 JSON 迁移和显式恢复的正向用例。复用[存储验证](../scripts/validate-sqlite-storage.mjs),并从成品窗口验证作者看到的状态,而不只看日志。 +**复测入口。** [preview.20 虚构旧库](../scripts/fixtures/library-preview20/README.md)含两个项目、一次正式采纳、一次手动修订、一个可续跑的任务,以及原始 SQLite 和 JSON 备份的校验值。旧版和候选版只打开它的临时副本。用 `npm.cmd test -- validate-library validate-agent-library-choice` 检查选库、原件保护和 Agent 读写;用 `npm.cmd run qa:library:ui` 检查恢复页面和保存队列。成品窗口复测命令见样本说明,不能用开发预览或网页代替。 + +**S0 / S1 验收(2026-11-12,Windows x64):** + +| 验证范围 | 结果 | +| --- | --- | +| S0 起点 | 核对远端主线 `78550db`;在独立工作区保留两份计划文档,使用主线锁文件。未修改原工作区或作者书库。 | +| 旧版实证 | 在 preview.20 的独立窗口复现了 4 种误导性空列表:书库目录缺失、文件缺失、损坏且旁有 JSON、仅有损坏文件。 | +| 新版故障恢复 | 同一份样本分别模拟上述 4 种故障和原生 SQLite 组件不可用。出错时数据读写被拒绝,配置、原库和旁边 JSON 未被改写。恢复原条件后在同一进程重试,两个项目、正文、版本链和待续跑任务均与清单一致。 | +| 首次使用与作者操作 | 成品窗口分别验证首次建库、合法旧 JSON 迁移、选择已有库、将备份恢复至新文件、明确选择独立 JSON、明确新建;取消选择和选错文件不切库。所有窗口仅用虚构数据,未请求模型。 | +| 防止空备份覆盖 | 旧设置页的备份恢复及覆盖导入也检查完整书库结构。空对象、无效字节不能替代整库;有实际记录的旧项目片段仍可合并,支持带 UTF-8 BOM 的备份。 | +| 成品凭据 | 最终验过的候选 `app.asar` SHA-256 为 `4b235d3221f59c64f69a8dd7182a521f2e12e9eb296fb41053f8f65f0a9614ae`。11/11 种场景、23 个进程会话,模型请求为 0。 | +| 交付边界 | 这是未发布的解包程序验收,不是 Setup 安装、卸载或原地升级。公开 preview.20 安装包未替换。 | + ## S2:发布候选包必须过安装升级 preview.20 实包使用 Electron 39.8.10 / better-sqlite3 12.9.0;核查时主线锁定的是 44.9.1 / 13.0.3。按候选提交的锁文件和 [Electron 支持日程](https://releases.electronjs.org/schedule)复核,不把源码升级当成安装包升级。区分会随应用运行的依赖和只影响构建的依赖,不为清零告警一次合并所有大版本。 @@ -69,7 +83,7 @@ preview.20 实包使用 Electron 39.8.10 / better-sqlite3 12.9.0;核查时主 每个修改程序的批次运行 `npm.cmd run typecheck`、`npm.cmd test`、`npm.cmd run build`。S3 额外保留 [采纳](../scripts/validate-chapter-commit-bundle.mjs)、[修订](../scripts/validate-p2c-revision-commit-bundle.mjs)、[版本恢复](../scripts/validate-version-chain-restore.mjs)的对照;S4 压缩若启动,还要验证旧 JSON 导入导出、备份列表和恢复。性能记录只提交样本配方、指标与复现方式,不含私人路径、正文或密钥。不调用付费模型。 -暂不安排流水线引擎搬目录、所有页面改为增删改查接口、清理作者文件或按文件行数重构。**下一次施工从 S0 → S1 开始**,交付的是能复现、能恢复的书库打开失败用例。 +暂不安排流水线引擎搬目录、所有页面改为增删改查接口、清理作者文件或按文件行数重构。**下一批是 S2**:用真正的 Windows Setup 做干净安装和原地升级;本次解包窗口验收不是安装升级结论。 ## English summary diff --git a/package.json b/package.json index a5e1d62..4473432 100644 --- a/package.json +++ b/package.json @@ -20,6 +20,8 @@ "typecheck": "tsc --noEmit", "preview": "electron-vite preview", "qa:hig": "node scripts/qa-hig-workspace.mjs", + "qa:library:ui": "node src/renderer/tests/validate-library-recovery.mjs", + "qa:library": "node scripts/qa-library-recovery.mjs", "preview:sqlite": "npm run build && npm run native:electron && electron-vite preview", "native:electron": "node scripts/manage-sqlite-native.mjs electron", "native:node": "node scripts/manage-sqlite-native.mjs node", diff --git a/scripts/qa-library-recovery.mjs b/scripts/qa-library-recovery.mjs new file mode 100644 index 0000000..9c5b674 --- /dev/null +++ b/scripts/qa-library-recovery.mjs @@ -0,0 +1,794 @@ +#!/usr/bin/env node +import assert from 'node:assert/strict' +import { execFile } from 'node:child_process' +import { constants } from 'node:fs' +import { copyFile, cp, mkdir, mkdtemp, readFile, readdir, realpath, rename, rm, stat, writeFile } from 'node:fs/promises' +import { createServer } from 'node:http' +import { once } from 'node:events' +import { createRequire } from 'node:module' +import { basename, dirname, join, relative, resolve } from 'node:path' +import { pathToFileURL } from 'node:url' +import { parseArgs, promisify } from 'node:util' +import { repoRoot } from './utils/repo-root.mjs' +import { button, click, closeElectron, fill, waitFor } from './utils/qa-agent-authorization-driver.mjs' +import { connect, evaluate, launch, sha256, within } from './utils/long-novel-ui-driver.mjs' +import { acceptedBody, books, digest, draftId, revisedBody, seed, signature, tag, waitJob, waitStep } from './fixtures/library-recovery.mjs' + +const options = { preview20: { type: 'string' }, candidate: { type: 'string' }, fixture: { type: 'string' }, group: { type: 'string' }, + 'publish-fixture': { type: 'boolean' } } +const { values, positionals } = parseArgs({ options, allowPositionals: true }) +const [phase, previousRun] = positionals +if (phase === 'clean-native') assert(positionals.length === 2 && !Object.keys(values).length) +else { + assert(positionals.length === 1 && ['s0', 's1'].includes(phase), 'Expected s0 [--preview20 ] or s1 --candidate [--fixture ]') + assert(phase === 's0' ? !values.candidate && !(values.fixture && values['publish-fixture']) : values.candidate && !values.preview20 && !values['publish-fixture']) +} +assert(!values.group || phase === 's1' && ['retry', 'explicit'].includes(values.group)) +assert.equal(process.platform, 'win32') +const publicDir = resolve(values.preview20 ?? join(repoRoot, 'release', 'win-unpacked')) +const publicAsar = 'ee65587c7164fae65824446c691f9998c2841203fe4bb36378439df766d63ba5' +const publicNative = 'fd54166b0bc9655d8dad4c6f5d39fc62dad48f2f4c3b2aa93edb8531737385c6' +const publicExe = 'f632a57f512ccd47fca3eb687456ede06575b9f0f867aa76a158e77306bfd875' +const sqliteRoot = join('resources', 'app.asar.unpacked', 'node_modules', 'better-sqlite3') +const portable = join(repoRoot, 'scripts', 'fixtures', 'library-preview20') +const tmp = join(repoRoot, 'tmp'), dbName = 'novel-director-data.sqlite', jsonName = 'novel-director-data.json' +const j = JSON.stringify, now = () => new Date().toISOString() +const json = async path => JSON.parse(await readFile(path, 'utf8')) +const save = (path, data) => writeFile(path, j(data, null, 2) + '\n', 'utf8') +const copyNew = (a, b) => copyFile(a, b, constants.COPYFILE_EXCL) +const exists = p => stat(p).then(() => true, e => { if (e.code === 'ENOENT') return false; throw e }) +const q = selector => 'document.querySelector(' + j(selector) + ')' +const api = 'window.novelDirector', loadData = api + '.data.load()' +const btn = (label, scope) => '[...document.querySelectorAll(' + j(scope ? scope + ' button' : 'button') + + ')].find(b=>b.textContent.trim()===' + j(label) + ')' +const until = (c, expr) => waitFor(c, '(async()=>Boolean(await (' + expr + ')))()', 20000) +const press = async (c, element) => { + await until(c, '(' + element + ')?.disabled === false') + await click(c, element) +} +const call = (c, expr) => evaluate(c, api + '.' + expr) +const view = c => evaluate(c, 'document.body.innerText') +const row = name => '[...document.querySelectorAll(".project-row")].find(r=>r.innerText.includes(' + j(name) + '))' + +await mkdir(tmp, { recursive: true }) +assert.equal(await realpath(tmp), tmp) +if (phase === 'clean-native') { + const run = resolve(previousRun) + await checkedRun(run) + const prior = await json(join(run, 'report.json')) + assert(prior.finishedAt && ['passed', 'failed'].includes(prior.status), 'Refusing a run without a final report') + const receipt = await cleanNativeCopy(run, prior) + console.log(receipt ? 'Removed private runtime: ' + receipt.path : 'No private runtime to remove') + process.exit(0) +} +const work = await mkdtemp(join(tmp, 'library-' + phase + '-')) +assert.equal(await realpath(work), work) +console.log('Evidence: ' + work) +const report = { phase, group: values.group ?? 'all', status: 'running', startedAt: now(), work, sessions: [], checks: [], modelRequests: [] } +let sink, manifest, fixtureDir + +async function target(dir, old) { + assert.equal(await realpath(dir), dir) + assert((await stat(dir)).isDirectory(), 'Pass the unpacked package directory, not an executable') + const executable = join(dir, 'Novel Director.exe'), archive = join(dir, 'resources', 'app.asar') + const layouts = [join(sqliteRoot, 'build', 'Release', 'better_sqlite3.node'), join(sqliteRoot, 'prebuilds', 'win32-x64.node')] + const found = [] + for (const path of layouts) if (await exists(join(dir, path))) found.push(path) + assert.equal(found.length, 1, 'Unknown/ambiguous packaged SQLite binding: ' + found.join(', ')) + const [binding] = found + for (const path of [executable, archive, join(dir, binding)]) assert.equal(await realpath(path), path) + const identity = { executable, archive, binding, exeSha256: await sha256(executable), + asarSha256: await sha256(archive), sqliteSha256: await sha256(join(dir, binding)) } + if (old) { + assert.equal(identity.exeSha256, publicExe) + assert.equal(identity.asarSha256, publicAsar) + assert.equal(identity.sqliteSha256, publicNative) + Object.assign(identity, { version: '0.1.6-preview.20', electronVersion: '39.8.10' }) + } else { + assert.notEqual(identity.asarSha256, publicAsar, 'Candidate is just preview.20') + const asar = createRequire(import.meta.url)('@electron/asar') + const packaged = JSON.parse(asar.extractFile(archive, 'package.json').toString('utf8')) + assert.equal(packaged.main, 'out/main/index.js') + identity.version = packaged.version + const out = join(repoRoot, 'out') + const fingerprints = {} + for (const entry of asar.listPackage(archive)) { + const rel = entry.replaceAll('\\', '/').replace(/^\/+/, '') + if (!rel.startsWith('out/') || 'files' in asar.statFile(archive, join(...rel.split('/')))) continue + const local = join(repoRoot, rel) + const hash = digest(asar.extractFile(archive, join(...rel.split('/')))) + assert.equal(await sha256(local), hash, 'Packaged file differs from this worktree build: ' + rel) + fingerprints[rel] = hash + } + for (const entry of ['out/main/index.js', 'out/preload/index.cjs', 'out/renderer/index.html']) assert(fingerprints[entry]) + const files = (await readdir(out, { recursive: true, withFileTypes: true })).filter(e => e.isFile()) + assert.equal(Object.keys(fingerprints).length, files.length, 'Packaged out file set differs from the worktree build') + identity.matchedBuildFiles = files.length + await save(join(work, 'candidate-build.json'), fingerprints) + } + report.target = identity + return { ...identity, executable, mode: 'packaged', runtimeRoot: dir } +} + +async function startSink(port = 0) { + const server = createServer((req, res) => { + report.modelRequests.push({ method: req.method, at: now() }) + res.writeHead(503, { Connection: 'close' }).end() + }) + await new Promise((ok, fail) => { server.once('error', fail); server.listen(port, '127.0.0.1', ok) }) + sink = server + return 'http://127.0.0.1:' + server.address().port + '/v1' +} + +async function screen(c, name) { + const { data } = await c.send('Page.captureScreenshot', { format: 'png', fromSurface: true }) + await writeFile(join(work, name + '.png'), Buffer.from(data, 'base64')) + await writeFile(join(work, name + '.txt'), await view(c), 'utf8') +} + +async function profile(name, configured = true) { + const dir = join(work, name), user = join(dir, 'user'), library = join(dir, 'volume', dbName) + await mkdir(user, { recursive: true }) + assert.equal(await realpath(user), user) + const result = { dir, user, library, defaultDb: join(user, dbName), config: join(user, 'app-config.json') } + if (configured) { + await mkdir(dirname(library)) + await copyNew(join(fixtureDir, 'library.sqlite'), library) + await writeFile(result.config, j({ storagePath: library }, null, 2), { flag: 'wx' }) + assert.equal(await sha256(library), manifest.sqliteSha256) + } + return result +} + +async function windowRun(t, p, name, test) { + assert(within(work, p.user) && await realpath(p.user) === p.user) + if (await exists(p.config)) { + const config = await json(p.config) + assert(within(p.dir, resolve(config.storagePath)), 'Refusing profile pointing outside this case') + } + const s = await launch(t, p.user) + const session = { name, userData: p.user, pid: s.child.pid, port: s.port } + report.sessions.push(session) + try { + await connect(s) + assert(s.processIds.includes(s.child.pid), 'CDP did not connect to the child we started') + const c = s.client + await until(c, api + '?.app?.getRuntimeInfo') + const info = await call(c, 'app.getRuntimeInfo()') + assert.equal(info.mode, 'packaged') + assert.equal(resolve(info.execPath), t.executable) + assert.equal(await evaluate(c, 'location.href'), pathToFileURL(join(t.archive, 'out', 'renderer', 'index.html')).href) + if (t.version) assert.equal(info.version, t.version) + if (t.electronVersion) assert.equal(info.electronVersion, t.electronVersion) + session.runtime = info + await test(c, session) + assert.equal(await call(c, 'ai.getCallProgress({})'), null) + assert.deepEqual(s.networkRequests, []) + assert.deepEqual(s.exceptions.map(e => e.exception?.description ?? e.text), []) + } catch (e) { + session.error = e.message + if (s.client) await screen(s.client, name + '-failure').catch(() => {}) + throw e + } finally { + if (!s.browser && s.child.pid && s.child.exitCode === null) { + await promisify(execFile)('taskkill.exe', ['/PID', String(s.child.pid), '/T', '/F'], { windowsHide: true }).catch(() => {}) + } + if (s.browser?.socket.readyState === WebSocket.OPEN) { + const current = await s.browser.send('SystemInfo.getProcessInfo').catch(() => null) + if (current) s.processIds = [...new Set([...s.processIds, ...current.processInfo.map(p => p.id)])] + } + const exited = s.child.exitCode === null && s.child.signalCode === null && s.child.pid ? once(s.child, 'exit') : null + session.closed = await closeElectron(s.child, s.client, s.browser, s.processIds) + if (exited) await exited + session.exitCode = s.child.exitCode + session.networkRequests = s.networkRequests + await writeFile(join(work, name + '.log'), s.logs.join(''), 'utf8') + assert.deepEqual(session.closed.remainingProcessIds, []) + if (!session.error) assert.equal(session.exitCode, 0, 'Electron did not exit cleanly') + assert.deepEqual(report.modelRequests, []) + await save(join(work, 'report.json'), report) + } +} + +async function expectPath(c, p, path) { + const paths = await call(c, 'app.getStoragePath()') + assert.equal(paths.defaultStoragePath, p.defaultDb) + assert.equal(paths.storagePath, path) +} + +async function openProject(c, book) { + await until(c, row(book.name)) + await press(c, button('进入', '(' + row(book.name) + ')')) + await until(c, q('.dashboard-view')) +} + +async function healthy(c, p, path, expected = manifest.signature) { + await until(c, 'document.querySelectorAll(".project-row").length === 2') + for (const book of books) assert(await evaluate(c, 'Boolean(' + row(book.name) + ')')) + await expectPath(c, p, path) + const loaded = await evaluate(c, loadData) + assert.equal(loaded.storagePath, path) + assert.deepEqual(signature(loaded.data), expected) + assert.equal(loaded.data.settings.baseUrl, manifest.baseUrl) + assert(!loaded.data.settings.hasApiKey && !loaded.data.settings.apiKey) + return { storagePath: path, storageRevision: loaded.revision, signature: signature(loaded.data) } +} + +async function createFixture(t, baseUrl) { + const p = await profile('author', false) + fixtureDir = join(work, 'fixture') + await mkdir(fixtureDir) + const data = seed(baseUrl) + let backup + await windowRun(t, p, 'author', async (c, session) => { + await until(c, q('.home-empty')) + await expectPath(c, p, p.defaultDb) + const empty = await evaluate(c, loadData) + assert.equal(empty.data.projects.length, 0) + assert(!empty.data.settings.hasApiKey && !empty.data.settings.apiKey) + const saved = await call(c, 'data.save(' + j({ ...empty.data, ...data, settings: { ...empty.data.settings, ...data.settings } }) + ')') + assert(saved.ok && saved.storagePath === p.defaultDb) + await c.send('Page.reload', { ignoreCache: true }) + await openProject(c, books[0]) + await press(c, btn('生产流水线', '.sidebar')) + await press(c, btn('草稿', '.pipeline-artifact-tabs')) + await until(c, q('.pipeline-draft-body') + '?.value === ' + j(acceptedBody)) + await press(c, btn('未完成审稿,直接采纳', '.pipeline-draft-actions')) + await press(c, btn('确认未审稿采纳', '.confirm-dialog')) + await until(c, loadData + '.then(r=>r.data.chapterCommitBundles.length===1)') + const accepted = (await evaluate(c, loadData)).data + assert.equal(accepted.chapterVersions.length, 1) + assert.equal(accepted.revisionCommitBundles.length, 0) + assert.equal(accepted.chapterCommitBundles[0].generatedDraftId, draftId) + assert.equal(accepted.chapters[0].body, acceptedBody) + session.afterAccept = { draftId, sha256: digest(accepted.chapters[0].body), version: accepted.chapterVersions[0].id } + await press(c, btn('章节', '.sidebar')) + await press(c, btn('连贯阅读')) + await press(c, btn('编辑本章')) + await fill(c, q('.reader-inline-editor textarea'), revisedBody) + await press(c, btn('保存本章')) + await until(c, q('.reader-message') + '?.innerText.includes("已保存本章正文")') + const read = await evaluate(c, loadData) + const expected = signature(read.data) + await screen(c, 'formal-revision') + const moved = await call(c, 'app.migrateStoragePath(' + j(p.library) + ',' + j(read.data) + ')') + assert(moved.ok, j(moved)) + await expectPath(c, p, p.library) + assert.equal((await json(p.config)).storagePath, p.library) + const manual = await call(c, 'backup.create()') + backup = manual.backupPath + assert(backup && within(join(p.user, 'backups'), backup) && await realpath(backup) === backup) + assert.deepEqual(signature(await json(backup)), expected) + manifest = { format: tag, version: t.version, appAsarSha256: t.asarSha256, baseUrl, signature: expected } + }) + await windowRun(t, p, 'cold-preview20', async (c, s) => { + s.readback = await healthy(c, p, p.library) + await screen(c, 'cold-preview20') + }) + assert(!await exists(p.library + '-wal') || (await stat(p.library + '-wal')).size === 0, 'Uncheckpointed WAL') + await copyNew(p.library, join(fixtureDir, 'library.sqlite')) + await copyNew(backup, join(fixtureDir, 'backup.json')) + manifest.sqliteSha256 = await sha256(p.library) + manifest.backupSha256 = await sha256(backup) + await save(join(fixtureDir, 'manifest.json'), manifest) + report.fixture = manifest +} + +async function verifyFixture(dir) { + assert.equal(await realpath(dir), dir) + for (const f of ['manifest.json', 'backup.json', 'library.sqlite']) assert.equal(await realpath(join(dir, f)), join(dir, f)) + const m = await json(join(dir, 'manifest.json')) + assert.equal(m.format, tag) + assert.equal(m.version, '0.1.6-preview.20') + assert.equal(m.appAsarSha256, publicAsar) + assert.equal(await sha256(join(dir, 'library.sqlite')), m.sqliteSha256) + assert.equal(await sha256(join(dir, 'backup.json')), m.backupSha256) + assert.match(m.baseUrl, /^http:\/\/127\.0\.0\.1:\d+\/v1$/) + const backup = await json(join(dir, 'backup.json')) + assert.deepEqual(signature(backup), m.signature) + assert(!backup.settings.apiKey && !backup.settings.hasApiKey) + for (const model of [backup.settings, ...Object.values(backup.settings.pipelineModelRoles), + ...backup.chapterGenerationJobs.flatMap(job => [job.aiRunConfig, ...Object.values(job.aiRunConfig.roles)])]) { + assert.equal(model.baseUrl, m.baseUrl) + assert.equal(model.apiProvider, 'local') + } + for (const name of ['manifest.json', 'backup.json', 'library.sqlite']) { + const bytes = (await readFile(join(dir, name))).toString('latin1') + assert(!/\b[A-Za-z]:[\\/]|\/(?:Users|home)\/|"(?:apiKey|api_key)"\s*:\s*"[^"]+"|"hasApiKey"\s*:\s*true/i.test(bytes), + 'Machine path or populated credential field in portable ' + name) + } + if (!report.portableReadback) { + // Some SQLite builds create WAL sidecars even on read-only open. Never open + // the distributed file itself; inspect a separate copy containing no sidecars. + const copy = join(work, 'readonly-copy.sqlite') + await copyNew(join(dir, 'library.sqlite'), copy) + const { DatabaseSync } = await import('node:sqlite') + const db = new DatabaseSync(copy, { readOnly: true }) + try { + assert.equal(db.prepare('PRAGMA integrity_check').get().integrity_check, 'ok') + const rows = db.prepare('SELECT collection,id,json FROM entities ORDER BY collection,id').all() + const arrays = Object.entries(backup).filter(([, list]) => Array.isArray(list)) + assert.equal(rows.length, arrays.reduce((n, [, list]) => n + list.length, 0)) + for (const row of rows) { + const entity = backup[row.collection]?.find(item => (item.id ?? item.projectId) === row.id) + assert.deepEqual(JSON.parse(row.json), entity, 'Different backup entity: ' + row.collection + '/' + row.id) + } + const settings = db.prepare('SELECT json FROM app_settings').all() + assert.equal(settings.length, 1) + assert.deepEqual(JSON.parse(settings[0].json), backup.settings) + assert.equal(Number(db.prepare("SELECT value FROM meta WHERE key='schemaVersion'").get().value), backup.schemaVersion) + report.portableReadback = { integrity: 'ok', matchedEntityRows: rows.length, matchedSettings: true, + schemaVersion: backup.schemaVersion, revision: db.prepare("SELECT value FROM meta WHERE key='revision'").get().value } + } finally { db.close() } + } + return m +} + +async function proveResume(t) { + const p = await profile('resume-copy') + await windowRun(t, p, 'resume', async (c, s) => { + await healthy(c, p, p.library) + await openProject(c, books[1]) + await press(c, btn('生产流水线', '.sidebar')) + await press(c, btn('重试失败步骤', '.pipeline-top-status-inner')) + await until(c, loadData + '.then(r=>r.data.chapterGenerationJobs.find(x=>x.id===' + j(waitJob) + ')?.status==="completed")') + const { data } = await evaluate(c, loadData) + const job = data.chapterGenerationJobs.find(x => x.id === waitJob) + const step = data.chapterGenerationSteps.find(x => x.id === manifest.signature.unfinished.stepId) + assert.equal(job.currentStep, waitStep) + assert.equal(step.status, 'completed') + assert.equal(data.chapterGenerationJobs.length, 2) + assert.equal(data.generatedChapterDrafts.length, 2) + const checkpoint = data.chapterGenerationSteps.find(x => x.jobId === waitJob && x.type === 'generate_chapter_draft') + assert.equal(digest(checkpoint.output), manifest.signature.unfinished.checkpointSha256) + assert.equal(data.chapterCommitBundles.length, 1) + assert.equal(data.revisionCommitBundles.length, 1) + s.resumed = { jobId: job.id, status: job.status, stepId: step.id, output: step.output, checkpointSha256: digest(checkpoint.output) } + await screen(c, 'resume-completed') + }) +} + +async function inventory(p) { + async function files(dir, filter = () => true) { + if (!await exists(dir)) return null + const found = {} + for (const e of await readdir(dir, { withFileTypes: true })) { + if (!filter(e.name)) continue + assert(!e.isSymbolicLink()) + found[e.name] = e.isFile() ? await sha256(join(dir, e.name)) : await files(join(dir, e.name)) + } + return found + } + return { configuredPath: (await json(p.config)).storagePath, + profile: await files(p.user, n => /^(novel-director-data|app-config)(\.|-)/.test(n)), + backups: await files(join(p.user, 'backups')) ?? {}, + volume: await files(dirname(p.library)), parked: await files(join(p.dir, 'parked')) } +} + +async function fault(name, caseName = name) { + const p = await profile(caseName) + const parked = join(p.dir, 'parked'), volume = dirname(p.library) + const neighbor = join(volume, jsonName) + if (name !== 'corrupt-alone') await writeFile(neighbor, j({ schemaVersion: 3, projects: [], chapters: [], storyBibles: [], + settings: { ...seed(manifest.baseUrl).settings, modelName: 's0-sibling-never-choose' } }), { flag: 'wx' }) + if (name === 'missing-volume') await rename(volume, parked) + else if (name !== 'native') { + await mkdir(parked) + await rename(p.library, join(parked, dbName)) + if (name.startsWith('corrupt')) await writeFile(p.library, 'Not a database: synthetic damage only.\n', { flag: 'wx' }) + } + return p +} + +async function observeOld(t, name) { + const p = await fault(name), entry = { name, before: await inventory(p) } + report.checks.push(entry) + await windowRun(t, p, 'old-' + name, async (c, s) => { + await until(c, q('.home-empty')) + await expectPath(c, p, p.library) + entry.ui = await view(c) + assert.match(entry.ui, /0 个项目/) + assert.match(entry.ui, /还没有小说项目/) + assert(await evaluate(c, '(' + btn('新建项目') + ')?.disabled === false')) + await screen(c, 'old-' + name) + try { const r = await evaluate(c, loadData); s.load = { projects: r.data.projects.length, path: r.storagePath } } + catch (e) { s.load = { error: e.message } } + if (name === 'corrupt-alone') assert(s.load.error) + else assert.equal(s.load.projects, 0) + }) + entry.after = await inventory(p) + assert.equal(entry.after.parked[dbName], manifest.sqliteSha256) + if (name !== 'corrupt-alone') assert(entry.after.volume[dbName] && entry.before.volume?.[dbName] !== entry.after.volume[dbName]) + entry.status = 'reproduced' +} + +const actions = { retry: '重试', choose: '选择已有书库', restore: '从备份恢复到新文件', create: '新建书库' } +const recovery = '.library-recovery' +const feedback = q('.library-recovery-feedback') + '?.innerText' +const recover = (c, action) => press(c, btn(actions[action], recovery)) + +async function blocked(c, p) { + await until(c, q(recovery)) + const alert = await evaluate(c, q('.library-recovery-error') + '.innerText') + assert(alert.includes(p.library), alert) + assert(alert.replace(p.library, '').trim().length > 8) + await expectPath(c, p, p.library) + const controls = await evaluate(c, '[...document.querySelectorAll("button")].filter(b=>b.getClientRects().length && !b.disabled).map(b=>b.textContent.trim())') + for (const label of Object.values(actions)) assert(controls.includes(label), j(controls)) + assert(!await evaluate(c, 'Boolean(document.querySelector(".home,.project-row,.sidebar,.home-empty"))')) + // A valid empty payload distinguishes an unavailable library from payload-validation errors. + const empty = await json(join(fixtureDir, 'backup.json')) + for (const [k, v] of Object.entries(empty)) if (Array.isArray(v)) empty[k] = [] + const refusal = expr => evaluate(c, expr + '.then(()=>({refused:false}),e=>({refused:true,reason:e.message}))') + const read = await refusal(loadData), write = await refusal(api + '.data.save(' + j(empty) + ')') + for (const r of [read, write]) assert(r.refused && r.reason.includes(p.library), j(r)) + return { alert, controls, readRefusal: read.reason, writeRefusal: write.reason } +} + +async function withoutBinding(t) { + const dir = join(work, 'native-package') + assert(!within(t.runtimeRoot, dir)) + await cp(t.runtimeRoot, dir, { recursive: true, force: false, errorOnExist: true, + filter: path => relative(t.runtimeRoot, path) !== t.binding }) + const executable = join(dir, 'Novel Director.exe'), archive = join(dir, 'resources', 'app.asar') + assert.equal(await sha256(executable), t.exeSha256) + assert.equal(await sha256(archive), t.asarSha256) + assert(!await exists(join(dir, t.binding))) + return { ...t, runtimeRoot: dir, executable, archive } +} + +async function checkedRun(run) { + assert.equal(dirname(run), tmp, 'Not a direct child of this worktree tmp') + assert.match(basename(run), /^library-s1-[a-zA-Z0-9]{6}$/) + assert.equal(await realpath(run), run, 'Refusing a redirected run directory') +} + +async function cleanNativeCopy(run, history) { + await checkedRun(run) + assert.equal(history.work, run) + assert.equal(history.phase, 's1') + const path = join(run, 'native-package') + if (!await exists(path)) return null + assert.equal(await realpath(path), path) + const supplied = dirname(history.target.executable) + assert(path !== supplied && !within(path, supplied) && !within(supplied, path)) + assert(history.sessions.every(s => s.closed?.remainingProcessIds?.length === 0 && Number.isInteger(s.exitCode)), + 'Leaving the private package in place: not all owned processes have confirmed exits') + const command = '$ErrorActionPreference="Stop"; $prefix=$env:NOVEL_QA_PRIVATE_RUNTIME; ' + + '$p=@(Get-CimInstance Win32_Process -Property ProcessId,ExecutablePath | Where-Object { $_.ExecutablePath -and ' + + '$_.ExecutablePath.StartsWith($prefix,[StringComparison]::OrdinalIgnoreCase) } | ForEach-Object { $_.ProcessId }); ' + + 'ConvertTo-Json -Compress -InputObject $p' + const { stdout } = await promisify(execFile)(process.env.NOVEL_QA_PWSH || 'pwsh.exe', + ['-NoProfile', '-NonInteractive', '-Command', command], + { env: { ...process.env, NOVEL_QA_PRIVATE_RUNTIME: path + '\\' }, windowsHide: true, timeout: 15000 }) + const running = JSON.parse(stdout.trim()) + assert.deepEqual(running, [], 'A process still uses the private package') + async function sum(dir) { + let bytes = 0 + for (const entry of await readdir(dir, { withFileTypes: true })) { + const file = join(dir, entry.name) + assert(!entry.isSymbolicLink() && within(path, file) && await realpath(file) === file, 'Refusing a redirected package entry') + if (entry.isDirectory()) bytes += await sum(file) + else { assert(entry.isFile()); bytes += (await stat(file)).size } + } + return bytes + } + const bytes = await sum(path), hashes = {} + for (const file of ['Novel Director.exe', join('resources', 'app.asar'), history.target.binding]) { + assert(within(path, resolve(path, file))) + hashes[file] = await exists(join(path, file)) ? await sha256(join(path, file)) : null + } + const receipt = { path, bytes, hashes, running, removed: false } + const receiptPath = join(run, 'native-package-cleanup.json') + await save(receiptPath, receipt) + assert.equal(await realpath(path), join(await realpath(run), 'native-package')) + await rm(path, { recursive: true, maxRetries: 3, retryDelay: 200 }) + assert(!await exists(path)) + Object.assign(receipt, { removed: true, at: now() }) + await save(receiptPath, receipt) + return receipt +} + +async function retryCase(t, name) { + const p = await fault(name) + const entry = { name, before: await inventory(p) } + report.checks.push(entry) + const runtime = name === 'native' ? await withoutBinding(t) : t + await windowRun(runtime, p, 'blocked-' + name, async (c, s) => { + entry.blocked = await blocked(c, p) + const reason = name === 'native' ? /SQLite|组件|\.node|module/i : name.startsWith('corrupt') ? /损坏|SQLite|格式/i : /找不到|不存在|连接/ + assert.match(entry.blocked.alert, reason) + await screen(c, 'blocked-' + name) + await recover(c, 'retry') + await until(c, feedback + '?.includes("失败")') + entry.retryWhileUnavailable = await evaluate(c, feedback) + entry.afterRefusals = await inventory(p) + assert.deepEqual(entry.afterRefusals, entry.before, 'Unavailable library or its neighbors changed') + if (name === 'native') { + await copyNew(join(t.runtimeRoot, t.binding), join(runtime.runtimeRoot, t.binding)) + entry.restoredBinding = { layout: t.binding, sha256: await sha256(join(runtime.runtimeRoot, t.binding)) } + assert.equal(entry.restoredBinding.sha256, t.sqliteSha256) + } else if (name === 'missing-volume') { + await rename(join(p.dir, 'parked'), dirname(p.library)) + } else { + if (name.startsWith('corrupt')) await rename(p.library, join(p.dir, 'damaged-kept.bin')) + await rename(join(p.dir, 'parked', dbName), p.library) + } + assert.equal(await sha256(p.library), manifest.sqliteSha256) + await recover(c, 'retry') + entry.sameProcess = { pid: s.pid, ...await healthy(c, p, p.library) } + await screen(c, 'retry-ok-' + name) + assert.equal(await sha256(p.config), entry.before.profile['app-config.json']) + }) + await windowRun(t, p, 'reopen-' + name, async (c, s) => { + entry.reopened = { pid: s.pid, ...await healthy(c, p, p.library) } + assert.notEqual(entry.reopened.pid, entry.sameProcess.pid) + assert.equal(entry.reopened.storageRevision, entry.sameProcess.storageRevision) + }) + entry.afterRestart = await inventory(p) + assert.deepEqual(entry.afterRestart.profile, entry.before.profile) + const sibling = (name === 'missing-volume' ? entry.before.parked : entry.before.volume)?.[jsonName] + assert.equal(entry.afterRestart.volume[jsonName], sibling) + assert.equal(entry.afterRestart.volume[dbName], manifest.sqliteSha256) + entry.status = 'passed' +} + +async function firstOrLegacy(t, legacy) { + const name = legacy ? 'legacy-migration' : 'first-use', p = await profile(name, false) + const entry = { name, before: { config: await exists(p.config), database: await exists(p.defaultDb) } } + report.checks.push(entry) + assert.deepEqual(entry.before, { config: false, database: false }) + const source = join(p.user, jsonName) + if (legacy) await copyNew(join(fixtureDir, 'backup.json'), source) + for (const stage of ['start', 'restart']) await windowRun(t, p, name + '-' + stage, async c => { + if (legacy) { + entry[stage] = await healthy(c, p, p.defaultDb) + const backups = (await readdir(p.user)).filter(f => f.startsWith(jsonName + '.before-sqlite-migrate.')) + assert.equal(backups.length, 1) + entry.sourceSha256 = await sha256(source) + entry.migrationBackupSha256 = await sha256(join(p.user, backups[0])) + assert.equal(entry.sourceSha256, manifest.backupSha256) + assert.equal(entry.migrationBackupSha256, manifest.backupSha256) + } else { + await until(c, q('.home-empty')) + assert(!await evaluate(c, 'Boolean(' + q(recovery) + ')')) + await expectPath(c, p, p.defaultDb) + const loaded = await evaluate(c, loadData) + assert.equal(loaded.data.projects.length, 0) + entry[stage] = { storagePath: loaded.storagePath, storageRevision: loaded.revision, projectCount: 0 } + assert.equal(loaded.storagePath, p.defaultDb) + assert(!await exists(source)) + } + assert.equal((await json(p.config)).storagePath, p.defaultDb) + if (stage === 'start') await screen(c, name) + }) + assert.equal(entry.start.storageRevision, entry.restart.storageRevision) + entry.status = 'passed' +} + +// Operate only a named native dialog belonging to this Electron PID. Do not read +// the shell's item list, use foreground keystrokes or substitute dialog return values. +const pickerDriver = String.raw` +$ErrorActionPreference = 'Stop' +$ErrorView = 'NormalView' +[Console]::OutputEncoding = [Text.UTF8Encoding]::new($false) +Add-Type -AssemblyName UIAutomationTypes,UIAutomationClient +$a = [Windows.Automation.AutomationElement] +$scope = [Windows.Automation.TreeScope] +$spec = $env:NOVEL_QA_PICKER | ConvertFrom-Json +function prop($p,$v) { [Windows.Automation.PropertyCondition]::new($p,$v) } +$owned = prop $a::ProcessIdProperty ([int]$spec.pid) +$match = [Windows.Automation.AndCondition]::new($owned,(prop $a::NameProperty $spec.title)) +$end = [DateTime]::UtcNow.AddSeconds(10) +do { + $dialog = $a::RootElement.FindFirst($scope::Children,$match) + if ($dialog) { break } + Start-Sleep -Milliseconds 70 +} while ([DateTime]::UtcNow -lt $end) +if (!$dialog) { + $names = $a::RootElement.FindAll($scope::Children,$owned) | ForEach-Object { $_.Current.Name } + throw "Owned dialog missing: $($spec.title); owned windows: $($names -join ', ')" +} +Add-Type -TypeDefinition @' +using System; +using System.Text; +using System.Linq; +using System.Collections.Generic; +using System.Runtime.InteropServices; +public static class QaPicker { + delegate bool Visitor(IntPtr hwnd, IntPtr unused); + [DllImport("user32")] static extern bool EnumChildWindows(IntPtr root, Visitor visit, IntPtr unused); + [DllImport("user32")] static extern int GetDlgCtrlID(IntPtr h); + [DllImport("user32")] static extern int GetWindowThreadProcessId(IntPtr h, out int pid); + [DllImport("user32")] static extern bool IsWindowEnabled(IntPtr h); + [DllImport("user32", CharSet=CharSet.Unicode)] static extern int GetClassName(IntPtr h, StringBuilder name, int size); + [DllImport("user32", CharSet=CharSet.Unicode)] static extern IntPtr SendMessage(IntPtr h, int msg, IntPtr wp, StringBuilder text); + [DllImport("user32", CharSet=CharSet.Unicode)] static extern IntPtr SendMessage(IntPtr h, int msg, IntPtr wp, string text); + [DllImport("user32")] static extern bool PostMessage(IntPtr h, int msg, IntPtr wp, IntPtr lp); + static string Kind(IntPtr h) { var s=new StringBuilder(100); GetClassName(h,s,s.Capacity); return s.ToString(); } + static bool Owned(IntPtr h,int pid) { int owner; GetWindowThreadProcessId(h,out owner); return owner==pid; } + static string Label(IntPtr h) { var s=new StringBuilder(256); SendMessage(h,0x000D,(IntPtr)s.Capacity,s); return s.ToString().Replace("&",""); } + public static void Operate(IntPtr root,int pid,int id,string label,string path) { + if (!Owned(root,pid) || Kind(root)!="#32770") throw new Exception("Unexpected dialog owner or type"); + var controls=new List(); + EnumChildWindows(root,(h,x)=>{ if (Owned(h,pid)) controls.Add(h); return true; },IntPtr.Zero); + var buttons=controls.Where(h=>Kind(h)=="Button" && IsWindowEnabled(h)).ToArray(); + var button=buttons.SingleOrDefault(h=>String.IsNullOrEmpty(label) ? GetDlgCtrlID(h)==id : Label(h)==label); + if(button==IntPtr.Zero) throw new Exception("No matching native button; IDs: "+String.Join(",",buttons.Select(GetDlgCtrlID))); + if(!String.IsNullOrEmpty(path)) { + var edits=controls.Where(h=>Kind(h)=="Edit" && IsWindowEnabled(h)).ToArray(); + var edit=edits.SingleOrDefault(h=>GetDlgCtrlID(h)==1148); + if(edit==IntPtr.Zero) edit=edits.SingleOrDefault(h=>GetDlgCtrlID(h)==1001); + if(edit==IntPtr.Zero) throw new Exception("No filename edit; IDs: "+String.Join(",",edits.Select(GetDlgCtrlID))); + SendMessage(edit,0x000C,IntPtr.Zero,path); + var actual=new StringBuilder(path.Length+2); + SendMessage(edit,0x000D,(IntPtr)actual.Capacity,actual); + if(actual.ToString()!=path) throw new Exception("Filename field did not take the isolated path"); + } + if(!PostMessage(button,0x00F5,IntPtr.Zero,IntPtr.Zero)) throw new Exception("Native button click failed"); + } +} +'@ +[QaPicker]::Operate([IntPtr]$dialog.Current.NativeWindowHandle,[int]$spec.pid,[int]$spec.button,$spec.label,$spec.file) +ConvertTo-Json -Compress @{ title=$spec.title; pid=$spec.pid; file=$spec.file; action=$spec.action; shell=(Get-Process -Id $PID).Path } +` + +async function picker(s, p, title, action, file) { + if (file) { + assert(within(p.dir, file) && await realpath(dirname(file)) === dirname(file)) + if (await exists(file)) assert.equal(await realpath(file), file) + } + const spec = { pid: s.pid, title, action, file, + label: action === 'confirm-json' ? '使用这个 JSON 书库' : action === 'cancel-json' ? '取消' : null, + button: action === 'cancel' ? '2' : '1' } + const { stdout } = await promisify(execFile)(process.env.NOVEL_QA_PWSH || 'pwsh.exe', + ['-NoLogo', '-NoProfile', '-NonInteractive', '-OutputFormat', 'Text', '-EncodedCommand', Buffer.from(pickerDriver, 'utf16le').toString('base64')], + { env: { ...process.env, NOVEL_QA_PICKER: j(spec) }, cwd: p.dir, timeout: 19000, windowsHide: true } + ).catch(e => { throw Error('Native dialog ' + title + ' (' + action + '): ' + (e.stderr?.trim().slice(-1400) || e.code || 'timeout')) }) + s.nativeDialogs ??= [] + s.nativeDialogs.push(JSON.parse(stdout.trim())) +} + +async function explicitRecovery(t) { + const open = '选择已有书库', src = '选择要恢复的 JSON 备份', dest = '将备份恢复到新文件(不覆盖原书库)' + const independent = '使用独立 JSON 书库' + for (const mode of ['choose', 'restore', 'explicit-json', 'create']) { + const p = await fault('corrupt', mode) + const name = mode === 'restore' ? 'restored.sqlite' : mode === 'explicit-json' ? 'independent.json' : mode + '.sqlite' + const alternative = join(p.dir, name), bad = join(p.dir, 'invalid.sqlite') + const manual = join(p.user, 'backups', 'novel-director-100-manual-s0.json') + if (mode === 'restore') { + await mkdir(dirname(manual)) + await copyNew(join(fixtureDir, 'backup.json'), manual) + } else if (mode !== 'create') await copyNew(join(fixtureDir, mode === 'choose' ? 'library.sqlite' : 'backup.json'), alternative) + if (mode === 'choose') await writeFile(bad, 'Invalid synthetic library.\n', { flag: 'wx' }) + const entry = { name: mode, before: await inventory(p) } + report.checks.push(entry) + async function verifyChosen(c) { + await expectPath(c, p, alternative) + assert.equal((await json(p.config)).storagePath, alternative) + if (mode !== 'create') return healthy(c, p, alternative) + await until(c, q('.home-empty')) + const { data, storagePath, revision } = await evaluate(c, loadData) + assert.equal(data.projects.length, 0) + assert.equal(storagePath, alternative) + return { storagePath, storageRevision: revision, projectCount: 0 } + } + await windowRun(t, p, 'explicit-' + mode, async (c, s) => { + entry.blocked = await blocked(c, p) + const choice = mode === 'explicit-json' ? 'choose' : mode + const operate = async cancel => { + await recover(c, choice) + if (mode === 'choose') await picker(s, p, open, cancel ? 'cancel' : 'select', cancel ? undefined : alternative) + if (mode === 'explicit-json') { + await picker(s, p, open, 'select', alternative) + await picker(s, p, independent, cancel ? 'cancel-json' : 'confirm-json') + } + if (mode === 'restore') { + await picker(s, p, src, 'select', manual) + await picker(s, p, dest, cancel ? 'cancel' : 'save', cancel ? undefined : alternative) + } + if (mode === 'create') await picker(s, p, '新建独立书库', cancel ? 'cancel' : 'save', cancel ? undefined : alternative) + } + await operate(true) + await until(c, feedback + '?.includes("取消")') + entry.cancelFeedback = await evaluate(c, feedback) + entry.afterCancel = await inventory(p) + assert.deepEqual(entry.afterCancel, entry.before) + assert(!await exists(p.defaultDb)) + if (mode === 'restore' || mode === 'create') assert(!await exists(alternative)) + await screen(c, mode + '-cancelled') + if (mode === 'choose') { + const badHash = await sha256(bad) + await recover(c, choice) + await picker(s, p, open, 'select', bad) + await until(c, feedback + '?.includes("失败")') + entry.invalidFeedback = await evaluate(c, feedback) + assert.deepEqual(await inventory(p), entry.before) + assert.equal(await sha256(bad), badHash) + await screen(c, 'choose-invalid') + } + await operate(false) + await until(c, q('.home')) + entry.chosen = await verifyChosen(c) + if (mode === 'explicit-json') { + assert.equal((await json(p.config)).explicitJson, true) + const saved = await evaluate(c, '(async()=>{const {data}=await ' + loadData + ';data.projects[0].description="s1-json-only";return await ' + api + '.data.save(data)})()') + assert(saved.ok && saved.storagePath === alternative) + } + await screen(c, mode + '-success') + }) + await windowRun(t, p, mode + '-restart', async c => { + await until(c, q('.home')) + entry.reopened = await verifyChosen(c) + if (mode === 'explicit-json') assert((await evaluate(c, loadData)).data.projects.some(b => b.description === 's1-json-only')) + }) + entry.afterRestart = await inventory(p) + assert.deepEqual(entry.afterRestart.volume, entry.before.volume) + assert.deepEqual(entry.afterRestart.parked, entry.before.parked) + assert(!await exists(p.defaultDb)) + if (mode === 'restore') assert.equal(await sha256(manual), manifest.backupSha256) + entry.status = 'passed' + } +} + +async function runS1(t) { + if (values.group !== 'explicit') { + for (const name of ['missing-volume', 'missing-file', 'corrupt', 'corrupt-alone', 'native']) await retryCase(t, name) + await firstOrLegacy(t, false) + await firstOrLegacy(t, true) + await proveResume(t) + } + if (values.group !== 'retry') await explicitRecovery(t) +} + +async function useFixture() { + fixtureDir = resolve(values.fixture ?? portable) + assert(fixtureDir === portable || /^library-s0-[A-Za-z0-9]+[\\/]fixture$/.test(relative(tmp, fixtureDir)), + 'Use only the supplied fixture or the fixture directory from an isolated S0 run') + manifest = await verifyFixture(fixtureDir) + report.fixture = manifest + assert.equal(await startSink(Number(new URL(manifest.baseUrl).port)), manifest.baseUrl) +} + +try { + if (phase === 's0') { + const t = await target(publicDir, true) + if (values.fixture) await useFixture() + else await createFixture(t, await startSink()) + await verifyFixture(fixtureDir) + await proveResume(t) + for (const name of ['missing-volume', 'missing-file', 'corrupt', 'corrupt-alone']) await observeOld(t, name) + if (values['publish-fixture']) { + assert.equal(await realpath(dirname(portable)), dirname(portable)) + await mkdir(portable, { recursive: true }) + for (const f of ['library.sqlite', 'backup.json', 'manifest.json']) assert(!await exists(join(portable, f)), 'Refusing to replace the shared fixture') + for (const f of ['library.sqlite', 'backup.json', 'manifest.json']) await copyNew(join(fixtureDir, f), join(portable, f)) + await verifyFixture(portable) + report.publishedFixture = portable + } + } else { + await useFixture() + await runS1(await target(resolve(values.candidate), false)) + } + assert.deepEqual(await verifyFixture(fixtureDir), manifest) + assert.equal(await sha256(report.target.archive), report.target.asarSha256) + assert.equal(await sha256(report.target.executable), report.target.exeSha256) + assert.equal(await sha256(join(dirname(report.target.executable), report.target.binding)), report.target.sqliteSha256) + assert.deepEqual(report.modelRequests, []) + report.status = 'passed' +} catch (e) { + report.status = 'failed' + report.error = e.stack + process.exitCode = 1 + console.error(report.error) +} finally { + if (sink) await new Promise(ok => sink.close(ok)) + if (phase === 's1') { + try { report.nativePackageCleanup = await cleanNativeCopy(work, report) } + catch (e) { report.status = 'failed'; report.cleanupError = e.message; process.exitCode = 1; console.error(e.message) } + } + report.finishedAt = now() + await save(join(work, 'report.json'), report) + console.log(report.status + ': ' + join(work, 'report.json')) +} diff --git a/scripts/run-tests.mjs b/scripts/run-tests.mjs index f982c19..277fc0a 100644 --- a/scripts/run-tests.mjs +++ b/scripts/run-tests.mjs @@ -60,6 +60,9 @@ const tests = [ ['validate-renderer-save-consistency.mjs'], ['validate-buffered-form-persistence.mjs'], ['validate-app-config-safety.mjs'], + ['validate-library-session.mjs'], + ['validate-library-storage.mjs'], + ['validate-agent-library-choice.mjs'], ['validate-runtime-info.mjs'], ['validate-context-budget-consistency.mjs'], ['validate-context-relevance-scoring.mjs'], diff --git a/scripts/validate-agent-candidate-undo.mjs b/scripts/validate-agent-candidate-undo.mjs index e660f09..e6551a9 100644 --- a/scripts/validate-agent-candidate-undo.mjs +++ b/scripts/validate-agent-candidate-undo.mjs @@ -10,7 +10,8 @@ import { repoRoot } from './utils/repo-root.mjs' const tempRoot = join(repoRoot, 'tmp') await mkdir(tempRoot, { recursive: true }) const outDir = await mkdtemp(join(tempRoot, 'agent-candidate-undo-')) -const nativePaths = new Set(), handles = new Set() +const jsonPaths = new Set(), nativePaths = new Set(), handles = new Set() +globalThis.__candidateUndoJsonPaths = jsonPaths globalThis.__candidateUndoNativePaths = nativePaths globalThis.__candidateUndoHandles = handles globalThis.fetch = async () => { throw new Error('Network forbidden in candidate undo fixtures.') } @@ -59,22 +60,24 @@ try { export const executeAgentChapterPipeline = () => { throw new Error('AI forbidden'); }; export const createHeadlessTransport = () => { throw new Error('AI forbidden'); }; ` : ` - import { SqliteStorageService, loadSqliteSnapshotReadonly as readNative, + import { createExistingLibraryStorage as openExisting, loadSqliteSnapshotReadonly as readNative, loadSqliteCollectionsReadonly as readNativeCollections } from 'real-undo-sqlite'; - export { SqliteStorageService }; + export { SqliteStorageService, checkSqliteBindingAvailable } from 'real-undo-sqlite'; + export const createNewLibraryStorage = () => { throw new Error('Unexpected new library in undo fixture'); }; export const loadChapterTitlesReadonly = () => { throw new Error('Unexpected title query in undo test'); }; export const loadSqliteUsageHistoryReadonly = () => { throw new Error('Unexpected usage query in undo test'); }; function check(path) { if (!globalThis.__candidateUndoNativePaths.has(path)) throw new Error('Unregistered SQLite path forbidden: ' + path); } - export const createStorageService = (path) => { + export const createExistingLibraryStorage = (path) => { + if (globalThis.__candidateUndoJsonPaths.has(path)) return openExisting(path); check(path); - const storage = new SqliteStorageService(path); + const storage = openExisting(path); globalThis.__candidateUndoHandles.add(storage); const close = storage.close.bind(storage); storage.close = () => { close(); globalThis.__candidateUndoHandles.delete(storage); }; return storage; }; export const loadSqliteSnapshotReadonly = (path) => { check(path); return readNative(path); }; - export const loadSqliteCollectionsReadonly = (path, collections) => { check(path); return readNativeCollections(path, collections); }; + export const loadSqliteCollectionsReadonly = (path, collections, selector) => { check(path); return readNativeCollections(path, collections, selector); }; `, loader: 'js' })) } }] }) @@ -101,7 +104,8 @@ try { const dir = join(outDir, `fixture-${++fixtureId}`) await mkdir(dir) const storagePath = join(dir, kind === 'json' ? 'data.json' : 'data.sqlite') - if (kind !== 'json') nativePaths.add(storagePath) + if (kind === 'json') jsonPaths.add(storagePath) + else nativePaths.add(storagePath) const withStorage = async (fn) => { const storage = kind === 'json' ? new api.JsonStorageService(storagePath) : new api.SqliteStorageService(storagePath) if (kind !== 'json') handles.add(storage) @@ -396,6 +400,7 @@ try { console.log(NativeDatabase ? `Native Node ${process.version} ABI ${process.versions.modules}; all handles closed.` : `Native SKIP: ${skipReason}`) } finally { for (const handle of handles) { try { handle.close() } finally { handles.delete(handle) } } + delete globalThis.__candidateUndoJsonPaths delete globalThis.__candidateUndoNativePaths delete globalThis.__candidateUndoHandles const child = relative(tempRoot, outDir) diff --git a/scripts/validate-agent-library-choice.mjs b/scripts/validate-agent-library-choice.mjs new file mode 100644 index 0000000..cf08cff --- /dev/null +++ b/scripts/validate-agent-library-choice.mjs @@ -0,0 +1,193 @@ +import assert from 'node:assert/strict' +import { spawnSync } from 'node:child_process' +import { createHash } from 'node:crypto' +import { mkdir, mkdtemp, readdir, readFile, rename, rm, writeFile } from 'node:fs/promises' +import { createRequire } from 'node:module' +import { dirname, join } from 'node:path' +import { pathToFileURL } from 'node:url' +import Database from 'better-sqlite3' +import { build } from 'esbuild' +import { repoRoot } from './utils/repo-root.mjs' + +const tmp = join(repoRoot, 'tmp') +await mkdir(tmp, { recursive: true }) +const work = await mkdtemp(join(tmp, 'agent-library-choice-')) +const profile = join(work, 'profile'), books = join(work, 'books') +const config = join(profile, 'app-config.json') +const json = join(books, 'selected.JSON'), sqlite = join(books, 'novel-director-data.sqlite') +const legacy = join(books, 'novel-director-data.json') +const opts = { userDataPath: profile }, projectId = 'same-project' +const select = (storagePath, explicitJson) => writeFile(config, JSON.stringify({ storagePath, explicitJson })) +const digest = async path => createHash('sha256').update(await readFile(path)).digest('hex') +async function fingerprint(dir) { + const result = {} + for (const entry of await readdir(dir, { withFileTypes: true })) { + const path = join(dir, entry.name) + result[entry.name] = entry.isDirectory() ? await fingerprint(path) : await digest(path) + } + return result +} + +const buildOpts = { absWorkingDir: repoRoot, bundle: true, platform: 'node', target: 'node22', format: 'esm', + external: ['electron', 'better-sqlite3'], logLevel: 'silent' } +const noNetwork = "globalThis.fetch = () => { throw new Error('Network disabled for this test') }" +const fetchBefore = globalThis.fetch +function invoke(entry, args, input = '') { + const child = spawnSync(process.execPath, [join(work, entry + '.mjs'), ...args], { + cwd: work, encoding: 'utf8', input, timeout: 20000, windowsHide: true, + env: { SystemRoot: process.env.SystemRoot, PATH: process.env.PATH, + HOME: work, USERPROFILE: work, APPDATA: work, LOCALAPPDATA: work, XDG_CONFIG_HOME: work } + }) + assert.ifError(child.error) + assert.equal(child.status, 0, child.stderr || child.stdout) + return JSON.parse(child.stdout) +} +function mcp(name, args) { + return invoke('mcp', [], JSON.stringify({ jsonrpc: '2.0', id: 1, method: 'tools/call', + params: { name, arguments: { ...opts, ...args } } }) + '\n') +} + +const titleEdit = { projectId, operationId: 'rename', prefixMode: 'manual', edits: [{ chapterId: 'ch', title: 'Renamed' }] } +const reads = [ + ['agent.listProjects', {}], ['agent.getChapterText', { projectId, chapterOrder: 1, detail: 'full' }], + ['agent.getBookProgress', { projectId }], ['agent.getBookContinuityReport', { projectId }], + ['agent.getProjectUsage', { projectId }], ['agent.getChapterTitleHistory', { projectId }], + ['agent.previewChapterTitleRename', titleEdit] +] +const start = ['agent.startAgentRun', { projectId, chapterOrder: 2, goal: 'Prepare only' }] + +try { + globalThis.fetch = () => { throw new Error('Network disabled for this test') } + await mkdir(profile); await mkdir(books) + await build({ ...buildOpts, stdin: { resolveDir: repoRoot, contents: [ + "export * from './src/agent/AgentRuntime'", "export { AgentToolService } from './src/agent/tools/AgentToolService'", + "export { normalizeAppData } from './src/shared/defaults'", "export { SqliteStorageService } from './src/storage/SqliteStorageService'" + ].join('\n') }, outfile: join(work, 'api.mjs') }) + await build({ ...buildOpts, entryPoints: { cli: 'src/agent/cli.ts', mcp: 'src/agent/mcp/server.ts' }, + outdir: work, outExtension: { '.js': '.mjs' }, banner: { js: noNetwork } }) + const apiUrl = pathToFileURL(join(work, 'api.mjs')).href + const api = await import(apiUrl) + const fixture = label => api.normalizeAppData({ projects: [{ id: projectId, name: label }], + chapters: [{ id: 'ch', projectId, order: 1, title: label, body: label + ' prose.' }] }) + const call = (name, args) => api.AgentToolService.callTool({ name, arguments: { ...opts, ...args } }) + const database = new api.SqliteStorageService(sqlite) + try { await database.save(fixture('SQLite book')) } finally { database.close() } + const db = new Database(sqlite, { fileMustExist: true }) + try { db.pragma('journal_mode = DELETE') } finally { db.close() } + await writeFile(json, JSON.stringify(fixture('JSON book'))) + await writeFile(legacy, JSON.stringify(fixture('Old default JSON'))) + await select(json, true) + const selectedConfig = await digest(config), neighbors = [await digest(sqlite), await digest(legacy)] + const beforeReads = await fingerprint(books) + const oldJson = await api.loadAgentRuntimeData(opts) + assert.equal(oldJson.data.projects[0].name, 'JSON book') + for (const [name, args] of reads) { + const reply = await call(name, args) + assert.deepEqual(reply.storage, { ...opts, storagePath: json, source: 'json', revision: oldJson.revision }, name) + if (name === 'agent.listProjects') assert.equal(reply.data[0].name, 'JSON book') + if (name === 'agent.getChapterText') assert.equal(reply.data.text, 'JSON book prose.') + if (name === 'agent.previewChapterTitleRename') assert.equal(reply.data.rows[0].oldTitle, 'JSON book') + } + assert.deepEqual(await fingerprint(books), beforeReads) + assert.deepEqual(await readdir(profile), ['app-config.json']) + const prepared = mcp(...start) + assert.equal(prepared.result?.isError, false, JSON.stringify(prepared.error)) + assert.equal(prepared.result.structuredContent.storage.storagePath, json) + assert.equal(JSON.parse(await readFile(json, 'utf8')).agentRuns[0].id, prepared.result.structuredContent.data.agentRun.id) + await assert.rejects(api.saveAgentRuntimeData(oldJson.data, oldJson), { code: 'STORAGE_REVISION_CONFLICT' }) + assert.deepEqual([await digest(sqlite), await digest(legacy)], neighbors) + assert.equal(await digest(config), selectedConfig) + const override = invoke('cli', ['list-projects', '--user-data', profile, '--storage', sqlite]) + assert.equal(override.data[0].name, 'SQLite book') + assert.equal(override.storage.storagePath, sqlite) + console.log('PASS selected standalone JSON: full/scoped reads, actual MCP write, revision guard and CLI override') + + const require = createRequire(import.meta.url) + const nativeModule = require.cache[require.resolve('better-sqlite3')], nativeExport = nativeModule.exports + try { + nativeModule.exports = class { constructor() { throw new Error('Test: SQLite binding unavailable') } } + const unavailable = await import(apiUrl + '?no-native-binding') + assert.equal((await unavailable.loadAgentRuntimeData(opts)).storagePath, json) + await select(sqlite) + const before = await fingerprint(books), beforeConfig = await digest(config) + await assert.rejects(unavailable.AgentToolService.callTool({ name: 'agent.listProjects', arguments: opts }), /binding unavailable/) + await assert.rejects(unavailable.saveAgentRuntimeData(oldJson.data, opts), /binding unavailable/) + assert.deepEqual(await fingerprint(books), before) + assert.equal(await digest(config), beforeConfig) + } finally { nativeModule.exports = nativeExport } + console.log('PASS unavailable native binding cannot redirect a configured SQLite read or write to JSON') + + for (const path of [json, sqlite]) { + await select(path, path === json ? true : undefined) + const loaded = await api.loadAgentRuntimeData(opts) + const moved = join(work, 'disconnected') + await rename(path, moved) + const before = await fingerprint(books), beforeConfig = await fingerprint(profile) + for (const [name, args] of [...reads, start]) { + await assert.rejects(call(name, args), error => error.message.includes(path), name) + } + await assert.rejects(api.reloadAgentRuntimeData(loaded)) + await assert.rejects(api.saveAgentRuntimeData(loaded.data, loaded)) + await assert.rejects(api.saveAgentRuntimeData(loaded.data, opts)) + await assert.rejects(api.saveAgentChapterCommitBundle({}, loaded)) + await assert.rejects(api.saveAgentRevisionCommitBundle({}, loaded)) + await assert.rejects(api.saveAgentCandidateDecision({ actor: { kind: 'agent' } }, loaded)) + assert.deepEqual(await fingerprint(books), before, 'Must not recreate a missing file or touch siblings/sidecars') + assert.deepEqual(await fingerprint(profile), beforeConfig) + await rename(moved, path) + assert.equal((await api.loadAgentRuntimeData(opts)).revision, loaded.revision) + } + console.log('PASS missing configured JSON and SQLite, including loss after read and retry on original file') + + await select(json) + await assert.rejects(call('agent.listProjects', {}), /SQLite/) + await writeFile(config, '{') + await assert.rejects(call('agent.listProjects', {})) + assert.equal((await call('agent.listProjects', { storagePath: json })).data[0].name, 'JSON book') + assert.equal(await readFile(config, 'utf8'), '{') + + const fresh = join(work, 'unused-profile'), freshOpts = { userDataPath: fresh } + const empty = await api.loadAgentRuntimeData(freshOpts) + assert.equal(empty.source, 'empty') + await assert.rejects(api.saveAgentRuntimeData(empty.data, empty)) + await assert.rejects(readdir(fresh), { code: 'ENOENT' }) + await mkdir(fresh) + const oldPath = join(fresh, 'novel-director-data.json'), alias = join(fresh, 'novel-director-data.sqlite') + await writeFile(oldPath, JSON.stringify(fixture('Legacy only'))) + const oldFiles = await fingerprint(fresh) + for (const args of [freshOpts, { ...freshOpts, storagePath: fresh }, { ...freshOpts, storagePath: alias }]) { + assert.equal((await call('agent.listProjects', args)).storage.storagePath, oldPath) + } + await assert.rejects(call(start[0], { ...start[1], ...freshOpts, storagePath: alias })) + assert.deepEqual(await fingerprint(fresh), oldFiles) + await writeFile(join(fresh, 'app-config.json'), JSON.stringify({ storagePath: alias })) + await assert.rejects(call('agent.listProjects', { ...freshOpts, storagePath: alias })) + console.log('PASS ambiguous/broken config, empty profile without file creation and read-only legacy CLI alias') + + const partialPath = join(books, 'cli-snapshot.json'), partial = fixture('Old CLI snapshot') + delete partial.storyBibles + await writeFile(partialPath, JSON.stringify(partial)) + await select(partialPath, true) + const beforePartial = await fingerprint(books) + await assert.rejects(call(...start)) + assert.deepEqual(await fingerprint(books), beforePartial, 'Desktop choice must use the strict writer') + const direct = { ...opts, storagePath: partialPath } + const loaded = await api.loadAgentRuntimeData(direct) + await rename(partialPath, join(work, 'offline-cli-snapshot')) + await assert.rejects(api.saveAgentRuntimeData(loaded.data, loaded)) + await assert.rejects(readFile(partialPath), { code: 'ENOENT' }) + await writeFile(partialPath, JSON.stringify({ ...partial, projects: [{ id: projectId, name: 'Later edit' }] })) + await assert.rejects(api.saveAgentRuntimeData(loaded.data, loaded), { code: 'STORAGE_REVISION_CONFLICT' }) + const preparedOld = await call(start[0], { ...start[1], ...direct }) + assert.equal(preparedOld.storage.storagePath, partialPath) + const upgraded = JSON.parse(await readFile(partialPath, 'utf8')) + assert.deepEqual(upgraded.storyBibles, []) + assert.equal(upgraded.projects[0].name, 'Later edit') + assert.equal(upgraded.agentRuns[0].id, preparedOld.data.agentRun.id) + assert.deepEqual([await digest(sqlite), await digest(legacy)], neighbors) + console.log('PASS only an explicitly named, present, revision-matched old CLI JSON snapshot can be upgraded') +} finally { + globalThis.fetch = fetchBefore + assert.equal(dirname(work), tmp) + await rm(work, { recursive: true, force: true, maxRetries: 3 }) +} diff --git a/scripts/validate-agent-revision-tools.mjs b/scripts/validate-agent-revision-tools.mjs index 08395ed..010880d 100644 --- a/scripts/validate-agent-revision-tools.mjs +++ b/scripts/validate-agent-revision-tools.mjs @@ -17,9 +17,11 @@ const routedRegistry = join(outDir, 'routed-api-registry.json') const revisionRoute = { baseUrl: 'https://revision.invalid/v1', keyEnv: 'NOVEL_REVISION_TEST_API_KEY', backupKeyEnv: 'NOVEL_REVISION_TEST_BACKUP_API_KEY' } const databases = new Map() +const jsonPaths = new Set() const nativePaths = new Set() const nativeHandles = new Set() globalThis.__agentRevisionDatabases = databases +globalThis.__agentRevisionJsonPaths = jsonPaths globalThis.__agentRevisionNativePaths = nativePaths globalThis.__agentRevisionNativeHandles = nativeHandles let networkAttempts = 0 @@ -85,29 +87,34 @@ try { export const executeAgentChapterPipeline = () => { throw new Error('Pipeline forbidden in revision fixture'); }; export const createHeadlessTransport = () => ({ transport: { chatCompletion: (request) => globalThis.__agentRevisionAI(request) } }); ` : ` - import { SqliteStorageService, loadSqliteSnapshotReadonly as readNative, + import { createExistingLibraryStorage as openExisting, loadSqliteSnapshotReadonly as readNative, loadSqliteCollectionsReadonly as readNativeCollections } from 'real-revision-sqlite'; - export { SqliteStorageService }; + export { SqliteStorageService, checkSqliteBindingAvailable } from 'real-revision-sqlite'; + export const createNewLibraryStorage = () => { throw new Error('Unexpected new library in revision fixture'); }; export const loadChapterTitlesReadonly = () => { throw new Error('Unexpected title query in revision test'); }; export const loadSqliteUsageHistoryReadonly = () => { throw new Error('Unexpected usage query in revision test'); }; - export const createStorageService = (path) => { + function simulated(path) { + const storage = globalThis.__agentRevisionDatabases.get(path); + if (!storage) throw new Error('Unregistered simulated SQLite path: ' + path); + return storage; + } + export const createExistingLibraryStorage = (path) => { + if (globalThis.__agentRevisionJsonPaths.has(path)) return openExisting(path); if (globalThis.__agentRevisionNativePaths.has(path)) { - // No JSON fallback: this branch must construct and use the real native service. - const storage = new SqliteStorageService(path); + const storage = openExisting(path); globalThis.__agentRevisionNativeHandles.add(storage); const close = storage.close.bind(storage); storage.close = () => { close(); globalThis.__agentRevisionNativeHandles.delete(storage); }; return storage; } - const storage = globalThis.__agentRevisionDatabases.get(path); - if (!storage) throw new Error('Native/unregistered SQLite path forbidden: ' + path); - return storage; + return simulated(path); }; export const loadSqliteSnapshotReadonly = (path) => globalThis.__agentRevisionNativePaths.has(path) - ? readNative(path) : createStorageService(path).loadSnapshot(); - export const loadSqliteCollectionsReadonly = async (path, collections) => { - if (globalThis.__agentRevisionNativePaths.has(path)) return readNativeCollections(path, collections); - const snapshot = await createStorageService(path).loadSnapshot(); + ? readNative(path) : simulated(path).loadSnapshot(); + export const loadSqliteCollectionsReadonly = async (path, collections, selector) => { + if (globalThis.__agentRevisionNativePaths.has(path)) return readNativeCollections(path, collections, selector); + if (selector !== undefined) throw new Error('Unexpected project-scoped query in revision fixture'); + const snapshot = await simulated(path).loadSnapshot(); return { revision: snapshot.revision, data: Object.fromEntries(collections.map(key => [key, snapshot.data[key]])) }; }; @@ -184,6 +191,7 @@ try { let boundary let storage if (kind === 'json') { + jsonPaths.add(storagePath) storage = new api.JsonStorageService(storagePath) await storage.save(initial) } else if (kind === 'sqlite-native') { @@ -735,6 +743,7 @@ try { mock.restoreAll() for (const handle of nativeHandles) { try { handle.close() } finally { nativeHandles.delete(handle) } } delete globalThis.__agentRevisionDatabases + delete globalThis.__agentRevisionJsonPaths delete globalThis.__agentRevisionNativePaths delete globalThis.__agentRevisionNativeHandles delete globalThis.__agentRevisionAI diff --git a/scripts/validate-library-session.mjs b/scripts/validate-library-session.mjs new file mode 100644 index 0000000..190ce9d --- /dev/null +++ b/scripts/validate-library-session.mjs @@ -0,0 +1,208 @@ +import assert from 'node:assert/strict' +import { existsSync } from 'node:fs' +import { mkdir, mkdtemp, readFile, readdir, rename, rm, writeFile } from 'node:fs/promises' +import { join } from 'node:path' +import { pathToFileURL } from 'node:url' +import { build } from 'esbuild' +import Database from 'better-sqlite3' +import { repoRoot } from './utils/repo-root.mjs' + +const root = join(repoRoot, 'tmp') +await mkdir(root, { recursive: true }) +const dir = await mkdtemp(join(root, 's1-session-')) +const sessions = [] +let passed = 0 +try { + const outfile = join(dir, 'session.mjs') + await build({ stdin: { resolveDir: repoRoot, contents: [ + 'main/LibrarySession', 'main/AppConfigService', 'main/libraryFiles', 'main/BackupService', + 'storage/libraryValidation', 'shared/defaults' + ].map(name => `export * from './src/${name}'`).join('\n') }, outfile, + format: 'esm', platform: 'node', target: 'node22', bundle: true, external: ['better-sqlite3'] }) + const { LibrarySession, AppConfigService, BackupService, normalizeImportedLibrary, EMPTY_APP_DATA, libraryStores } = + await import(pathToFileURL(outfile)) + const json = JSON.stringify({ ...EMPTY_APP_DATA, projects: [{ id: 'book-1', name: '纸灯' }, { id: 'book-2', name: '回潮' }] }) + const ids = s => s.snapshot.data.projects.map(p => p.id) + async function setup(name, stores = libraryStores) { + const home = join(dir, name) + await mkdir(home) + const config = new AppConfigService(home) + function session(previousLaunch = false) { + const s = new LibrarySession(config, stores, undefined, previousLaunch) + sessions.push(s) + return s + } + return { home, config, cfg: await config.getConfigPath(), db: config.getDefaultStoragePath(), + old: join(home, 'novel-director-data.json'), session } + } + async function scenario(name, fn) { await fn(); passed++; console.log('PASS:', name) } + const off = () => { throw new Error('NODE_MODULE_VERSION mismatch') } + const unavailable = { checkSqlite: off, existing: p => /\.json$/i.test(p) ? libraryStores.existing(p) : off(), + create: (p, data) => /\.json$/i.test(p) ? libraryStores.create(p, data) : off() } + await scenario('首次启动:两个并发读取只建一个空库;重启仍指向同一文件', async () => { + const f = await setup('fresh'), s = f.session() + assert.throws(() => s.getStorage()) + const [a, b] = await Promise.all([s.load(), s.load()]) + assert.equal(a.storage, b.storage) + assert.deepEqual(ids(a), []) + assert.equal(JSON.parse(await readFile(f.cfg)).storagePath, f.db) + assert.ok(!existsSync(f.old)) + s.adopt(libraryStores.existing(f.db)) + await assert.rejects(a.storage.loadSnapshot(), /已关闭/) + s.close() + assert.equal((await f.session().load()).snapshot.revision, a.snapshot.revision) + }) + await scenario('旧 JSON 及其常规备份迁入 SQLite;原件及迁移备份逐字节不变', async () => { + const f = await setup('legacy') + await writeFile(f.old, json) + await writeFile(f.old + '.bak', json) + const s = f.session() + assert.deepEqual(ids(await s.load()), ['book-1', 'book-2']) + assert.equal(s.path, f.db) + const backups = (await readdir(f.home)).filter(n => n.includes('before-sqlite-migrate.')) + assert.equal(backups.length, 1) + assert.equal(await readFile(join(f.home, backups[0]), 'utf8'), json) + assert.equal(await readFile(f.old, 'utf8'), json) + s.close() + assert.deepEqual(ids(await f.session().load()), ['book-1', 'book-2']) + }) + await scenario('已配置书库离线:阻止读取和写入;重试仍打开原库', async () => { + const f = await setup('offline') + await writeFile(f.old, json) + const initial = f.session() + await initial.load() + initial.close() + const savedConfig = await readFile(f.cfg) + await rename(f.db, f.db + '.offline') + const s = f.session() + await assert.rejects(s.load(), { code: 'LIBRARY_UNAVAILABLE' }) + assert.equal(s.path, f.db) + assert.throws(() => s.getStorage()) + await assert.rejects(s.recover('retry'), /找不到原书库/) + await assert.rejects(s.recover('restore', f.db, f.old), /另选/) + assert.deepEqual(await readFile(f.cfg), savedConfig) + assert.ok(!existsSync(f.db)) + assert.equal(await readFile(f.old, 'utf8'), json) + await rename(f.db + '.offline', f.db) + await s.recover('retry') + assert.deepEqual(ids(await s.load()), ['book-1', 'book-2']) + }) + await scenario('损坏的 SQLite 与旁边的 JSON 均不被修理、替换或另存', async () => { + const f = await setup('corrupt') + await f.config.setStoragePath(f.db) + await writeFile(f.db, 'not sqlite') + await writeFile(f.old, json) + const files = await readdir(f.home), config = await readFile(f.cfg) + const s = f.session() + await assert.rejects(s.load(), { code: 'LIBRARY_UNAVAILABLE' }) + await assert.rejects(s.recover('choose', f.db)) + assert.deepEqual(await readdir(f.home), files) + assert.deepEqual(await readFile(f.cfg), config) + assert.equal(await readFile(f.db, 'utf8'), 'not sqlite') + assert.equal(await readFile(f.old, 'utf8'), json) + }) + await scenario('SQLite 组件不可用:可明确选择独立 JSON;重启不会改读同目录 SQLite', async () => { + const f = await setup('native', unavailable) + await writeFile(f.db, 'untouched') + await writeFile(f.old, json) + await f.config.setStoragePath(f.db) + const s = f.session() + await assert.rejects(s.load(), /组件/) + assert.equal(JSON.parse(await readFile(f.cfg)).storagePath, f.db) + await s.recover('choose', f.old) + assert.deepEqual(ids(await s.load()), ['book-1', 'book-2']) + assert.equal(JSON.parse(await readFile(f.cfg)).explicitJson, true) + s.close() + assert.equal((await f.session().load()).storage.getStoragePath(), f.old) + assert.equal(await readFile(f.db, 'utf8'), 'untouched') + }) + await scenario('无法区分旧 JSON 自动回退和作者选择时,不猜测', async () => { + const f = await setup('ambiguous') + await writeFile(f.db, 'old sqlite') + await writeFile(f.old, json) + await f.config.setStoragePath(f.old) + const cfg = await readFile(f.cfg) + await assert.rejects(f.session().load(), /明确选择/) + assert.deepEqual(await readFile(f.cfg), cfg) + }) + await scenario('恢复到新位置:拒绝坏备份及已存在的目标,恢复后能重启', async () => { + const f = await setup('restore'), backup = join(f.home, 'backup.json') + const invalid = join(f.home, 'invalid.json'), target = join(f.home, 'restored.sqlite') + await f.config.setStoragePath(f.db) + const saved = JSON.parse(json) + saved.settings.apiKey = 'SYNTHETIC_RECOVERY_CREDENTIAL' + const raw = JSON.stringify(saved) + await writeFile(backup, raw) + await writeFile(invalid, '{}') + const s = f.session(), cfg = await readFile(f.cfg) + await assert.rejects(s.load()) + await assert.rejects(s.recover('restore', target, invalid)) + await assert.rejects(s.recover('restore', backup, backup)) + assert.deepEqual(await readFile(f.cfg), cfg) + assert.ok(!existsSync(target)) + await s.recover('restore', target, backup) + assert.deepEqual(ids(await s.load()), ['book-1', 'book-2']) + assert.equal(s.path, target) + assert.equal(await readFile(backup, 'utf8'), raw) + assert.ok(!existsSync(f.db)) + s.close() + const db = new Database(target, { readonly: true, fileMustExist: true }) + try { assert.ok(!db.prepare('SELECT json FROM app_settings').get().json.includes(saved.settings.apiKey)) } + finally { db.close() } + assert.equal((await f.session().load()).storage.getStoragePath(), target) + }) + await scenario('旧 JSON 无效或配置损坏时,不创建默认库或自行重写配置', async () => { + const f = await setup('invalid') + await writeFile(f.old, '{') + await assert.rejects(f.session().load()) + assert.deepEqual(await readdir(f.home), ['novel-director-data.json']) + await writeFile(f.cfg, '{') + await assert.rejects(f.session().load(), /上次选择/) + assert.equal(await readFile(f.cfg, 'utf8'), '{') + assert.ok(!existsSync(f.db)) + const priorConfig = JSON.stringify({ storagePath: f.db }) + await writeFile(f.cfg + '.bak', priorConfig) + await writeFile(f.old, json) + const recovery = f.session() + await assert.rejects(recovery.load()) + await recovery.recover('choose', f.old) + assert.equal(await readFile(f.cfg + '.bak', 'utf8'), priorConfig) + }) + await scenario('有旧备份但没有配置:允许显式新建,不覆盖已有文件或遗留 WAL', async () => { + const f = await setup('witness', unavailable) + await mkdir(join(f.home, 'backups')) + await writeFile(join(f.home, 'backups', 'previous.json'), json) + const s = f.session(), target = join(f.home, 'new.json') + await assert.rejects(s.load(), /旧书库记录/) + await writeFile(target + '-wal', 'preserve') + await assert.rejects(s.recover('create', target), /旧书库文件/) + assert.ok(!existsSync(target)) + const newTarget = join(f.home, 'choice.json') + await s.recover('create', newTarget) + assert.deepEqual(ids(await s.load()), []) + assert.ok(!existsSync(f.db)) + }) + await scenario('已启动过的旧桌面配置即便没有书库和备份,也不冒充首次使用', async () => { + const f = await setup('previous-launch') + const s = f.session(true) + await assert.rejects(s.load(), /旧书库记录/) + assert.ok(!existsSync(f.db) && !existsSync(f.cfg)) + }) + await scenario('坏备份和空对象不能用于覆盖;部分旧项目可合并,Windows UTF-8 备份可读', async () => { + const f = await setup('import'), backups = new BackupService(f.home) + await backups.ensureBackupDir() + const backup = join(backups.getBackupDir(), 'test.json') + await writeFile(backup, '{}') + await assert.rejects(backups.loadBackup(backup), /完整/) + for (const strategy of ['replace', 'merge']) assert.throws(() => normalizeImportedLibrary({}, strategy), /完整/) + const partial = { projects: JSON.parse(json).projects } + assert.throws(() => normalizeImportedLibrary(partial, 'replace'), /完整/) + assert.equal(normalizeImportedLibrary(partial, 'merge').projects.length, 2) + await writeFile(backup, '\ufeff' + json) + assert.equal((await backups.loadBackup(backup)).projects.length, 2) + }) + console.log(`S1: ${passed} library session scenarios passed.`) +} finally { + for (const s of sessions) s.close() + await rm(dir, { recursive: true, force: true }) +} diff --git a/scripts/validate-library-storage.mjs b/scripts/validate-library-storage.mjs new file mode 100644 index 0000000..8977956 --- /dev/null +++ b/scripts/validate-library-storage.mjs @@ -0,0 +1,306 @@ +import assert from 'node:assert/strict' +import { closeSync, existsSync, mkdirSync, mkdtempSync, openSync, readFileSync, readdirSync, + rmSync, unlinkSync, writeFileSync } from 'node:fs' +import { createRequire } from 'node:module' +import { dirname, join } from 'node:path' +import { after, test } from 'node:test' +import Database from 'better-sqlite3' +import { build } from 'esbuild' +import { repoRoot } from './utils/repo-root.mjs' + +const require = createRequire(import.meta.url) +const base = join(repoRoot, 'tmp') +mkdirSync(base, { recursive: true }) +const root = mkdtempSync(join(base, 'library-storage-')) +after(() => { assert.equal(dirname(root), base); rmSync(root, { recursive: true }) }) +const compiled = await build({ stdin: { resolveDir: repoRoot, contents: ` + export * from './src/storage/SqliteStorageService' + export { EMPTY_APP_DATA } from './src/shared/defaults' +` }, bundle: true, write: false, platform: 'node', packages: 'external', format: 'cjs', logLevel: 'silent' }) + +function api(binding = Database, json = JSON) { + const localRequire = id => id === 'better-sqlite3' ? binding + : id === 'node:module' ? { ...require(id), createRequire: () => localRequire } : require(id) + const mod = { exports: {} } + new Function('require', 'module', 'exports', 'JSON', compiled.outputFiles[0].text)( + localRequire, mod, mod.exports, json) + return mod.exports +} +const { EMPTY_APP_DATA, SqliteStorageService, createNewLibraryStorage: create, createExistingLibraryStorage: existing } = api() +const seeded = { ...EMPTY_APP_DATA, projects: [{ id: 'test-book', name: 'fixture-first' }], + settings: { ...EMPTY_APP_DATA.settings, apiKey: 'SYNTHETIC_DO_NOT_STORE' } } +function directory() { return mkdtempSync(join(root, 'case-')) } +function reserve(path) { closeSync(openSync(path, 'wx')) } +function modify(path, f) { + const db = new Database(path, { fileMustExist: true }) + try { return f(db) } finally { db.close() } +} +function seededFile(dir, name = 'book.sqlite', data = seeded) { + const path = join(dir, name) + reserve(path) + create(path, data).close?.() + return path +} + +for (const suffix of ['.SQLite', '.DB', '.JSON']) { + test(suffix + ': exact path, empty vs missing, initial payload, write and reopen', async () => { + const dir = directory(), path = join(dir, 'selected' + suffix) + const sibling = join(dir, 'novel-director-data.json'), other = JSON.stringify(seeded) + writeFileSync(sibling, other) + assert.throws(() => existing(path)) + assert.throws(() => create(path)) + assert(!existsSync(path)) + reserve(path) + assert.throws(() => existing(path)) + const service = create(path) + try { + assert.equal(service.getStoragePath(), path) + const empty = await service.loadSnapshot() + assert.deepEqual(empty.data.projects, []) + const saved = await service.saveIfCurrent(seeded, empty.revision) + assert.notEqual(saved.revision, empty.revision) + await assert.rejects(service.saveIfCurrent(EMPTY_APP_DATA, empty.revision), { code: 'STORAGE_REVISION_CONFLICT' }) + } finally { service.close?.() } + const reopened = existing(path) + try { + const loaded = await reopened.load() + assert.equal(loaded.projects[0].id, 'test-book') + assert.equal(loaded.settings.apiKey, '') + await reopened.save(EMPTY_APP_DATA) + } finally { reopened.close?.() } + const again = existing(path) + try { assert.deepEqual((await again.load()).projects, []) } finally { again.close?.() } + const ready = existing(seededFile(dir, 'seeded' + suffix)) + try { assert.equal((await ready.load()).projects[0].name, 'fixture-first') } finally { ready.close?.() } + assert.equal(readFileSync(sibling, 'utf8'), other) + }) +} + +test('new requires an untouched reservation, rejects stale SQLite sidecars and unknown suffixes', () => { + const dir = directory() + for (const suffix of ['.db', '.json', '.bak', '', '.sqlite3']) { + const path = join(dir, 'existing' + suffix) + writeFileSync(path, 'do not change') + assert.throws(() => create(path)) + assert.throws(() => existing(path)) + assert.equal(readFileSync(path, 'utf8'), 'do not change') + } + const path = join(dir, 'reserved.db') + reserve(path) + for (const suffix of ['-shm', '-wal', '-journal']) { + writeFileSync(path + suffix, 'unowned') + assert.throws(() => create(path)) + assert.equal(readFileSync(path).length, 0) + assert.equal(readFileSync(path + suffix, 'utf8'), 'unowned') + unlinkSync(path + suffix) + } +}) + +test('failed initial payload rolls back schema as well as data; no published empty library', () => { + for (const extension of ['.json', '.db']) { + const path = join(directory(), 'new' + extension) + reserve(path) + assert.throws(() => create(path, { ...seeded, projects: [null] })) + assert.throws(() => existing(path)) + if (extension === '.json') assert.equal(readFileSync(path).length, 0) + else modify(path, db => assert.equal(db.prepare('SELECT 1 FROM sqlite_schema LIMIT 1').get(), undefined)) + } +}) + +test('an insert failure after the first project rolls back the entire new library', () => { + const path = join(directory(), 'interrupted.sqlite') + reserve(path) + const injected = api(function Observed(file, options) { + const db = new Database(file, options) + if (file === path) { + const exec = db.exec.bind(db) + db.exec = sql => { + const result = exec(sql) + if (sql.includes('CREATE TABLE IF NOT EXISTS entities')) exec(`CREATE TEMP TRIGGER reject_second + BEFORE INSERT ON entities WHEN new.collection='projects' AND new.id='second' + BEGIN SELECT RAISE(ABORT, 'synthetic mid-write failure'); END;`) + return result + } + } + return db + }) + assert.throws(() => injected.createNewLibraryStorage(path, { ...seeded, projects: [...seeded.projects, { id: 'second' }] }), + /synthetic mid-write failure/) + assert.throws(() => existing(path)) + modify(path, db => assert.deepEqual(db.prepare('SELECT name FROM sqlite_schema').all(), [])) +}) + +test('JSON rejection precedes normalization, creation, backup or recovery', () => { + const dir = directory(), path = join(dir, 'selected.json') + for (const input of ['', '{', 'null', '[]', '{}', JSON.stringify({ settings: {}, projects: [] }), + ...[{ settings: null }, { schemaVersion: 0 }, { schemaVersion: 4 }, { schemaVersion: '3' }, + { projects: {} }, { chapters: [null] }, { storyBibles: [{}] }, { projects: [{ name: 'no id' }] }, + { chapters: [{ id: 'c' }] }, { projects: [{ id: 'p' }, { id: 'p' }] }, + { candidateDecisionReceipts: [{ id: 'bad', projectId: 'test-book' }] } + ].map(change => JSON.stringify({ ...seeded, ...change }))]) { + writeFileSync(path, input) + assert.throws(() => existing(path)) + assert.equal(readFileSync(path, 'utf8'), input) + assert.deepEqual(readdirSync(dir), ['selected.json']) + } +}) + +test('JSON checks current bytes on first and later reads, writes and backups', async () => { + const dir = directory(), path = join(dir, 'legacy.json') + const legacy = { schemaVersion: 1, projects: [{ id: 'p' }], storyBibles: [{ projectId: 'p' }], + chapters: [], settings: { theme: 'dark' } } + const raw = JSON.stringify(legacy) + writeFileSync(path, raw) + let decodes = 0 + const strict = api(Database, { ...JSON, stringify: JSON.stringify, parse: text => { + if (text === raw) decodes++ + return JSON.parse(text) + } }).createExistingLibraryStorage(path) + assert.equal((await strict.load()).storyBibles[0].projectId, 'p') + assert.equal(decodes, 1, 'opening and first load must not decode the same document twice') + const other = existing(path) + writeFileSync(path, '{}') + await assert.rejects(other.load()) // changed before first load + await assert.rejects(strict.loadSnapshot()) + await assert.rejects(strict.save(seeded)) + await assert.rejects(strict.backupTo(join(dir, 'new-backup.json'))) + assert.deepEqual(readdirSync(dir), ['legacy.json']) + assert.equal(readFileSync(path, 'utf8'), '{}') + unlinkSync(path) + await assert.rejects(strict.load()) + await assert.rejects(strict.save(seeded)) + assert.deepEqual(readdirSync(dir), []) +}) + +test('invalid UTF-8 inside an otherwise valid JSON string is not replaced with missing prose', () => { + const path = join(directory(), 'invalid-text.json') + const bytes = Buffer.from(JSON.stringify(seeded)) + bytes[bytes.indexOf('fixture-first')] = 0xff + writeFileSync(path, bytes) + assert.throws(() => existing(path), /UTF-8/) + assert.deepEqual(readFileSync(path), bytes) +}) + +test('legacy entity keys agree with the original SQLite writer, including explicit story bible ids', async () => { + const dir = directory(), sqlite = join(dir, 'older.db'), json = join(dir, 'export.json') + const old = new SqliteStorageService(sqlite) + const data = { ...seeded, storyBibles: [{ id: 'separate-id', projectId: 'test-book', corePremise: 'fixture' }], + promptVersions: [{ projectId: 'test-book', label: 'legacy without id' }] } + try { await old.save(data) } finally { old.close() } + writeFileSync(json, JSON.stringify(data)) + for (const path of [sqlite, json]) { + const store = existing(path) + try { + const { data: loaded, revision } = await store.loadSnapshot() + assert.equal(loaded.storyBibles[0].id, 'separate-id') + assert.equal(loaded.promptVersions[0].label, 'legacy without id') + await store.saveIfCurrent(loaded, revision) + } finally { store.close?.() } + } + modify(sqlite, db => assert.deepEqual(db.prepare("SELECT id FROM entities WHERE collection IN ('promptVersions', 'storyBibles') ORDER BY collection").all(), + [{ id: 'promptVersions:0' }, { id: 'separate-id' }])) +}) + +test('unrelated, truncated, incomplete and unsupported SQLite files are never adopted or repaired', () => { + const dir = directory(), sibling = join(dir, 'novel-director-data.json'), config = join(dir, 'app-config.json') + const other = JSON.stringify(seeded) + writeFileSync(sibling, other); writeFileSync(config, '{"unmodified":true}') + const unrelated = join(dir, 'unrelated.sqlite') + const wrong = new Database(unrelated) + wrong.exec('CREATE TABLE unrelated (id INTEGER)'); wrong.close() + const mislabeled = join(dir, 'mislabeled.db') + writeFileSync(mislabeled, other) + const truncated = join(dir, 'truncated.db') + writeFileSync(truncated, readFileSync(unrelated).subarray(0, 165)) + function rejected(path) { + const original = readFileSync(path) + assert.throws(() => existing(path)) + assert.deepEqual(readFileSync(path), original) + } + for (const path of [unrelated, mislabeled, truncated]) rejected(path) + for (const [i, sql] of [ + 'DROP TABLE app_settings', 'ALTER TABLE entities RENAME COLUMN chapter_order TO wrong', + 'DELETE FROM schema_migrations', "INSERT INTO schema_migrations VALUES (2, 'future')", + "DELETE FROM meta WHERE key = 'revision'", "UPDATE meta SET value='5' WHERE key='schemaVersion'", + 'DELETE FROM app_settings', "UPDATE app_settings SET json='null'", "UPDATE app_settings SET json='{'", + "UPDATE entities SET json='{'", "UPDATE entities SET json='[]'", "UPDATE entities SET json='null'", + "UPDATE entities SET json='{}'", "UPDATE entities SET json='{\"id\":\"different\"}'", + "UPDATE entities SET collection='future' WHERE collection='projects'" + ].entries()) { + const path = seededFile(dir, 'case' + i + '.sqlite') + modify(path, db => { db.pragma('journal_mode = DELETE'); db.exec(sql) }) + rejected(path) + } + assert.equal(readFileSync(sibling, 'utf8'), other) + assert.equal(readFileSync(config, 'utf8'), '{"unmodified":true}') + assert(!readdirSync(dir).some(n => /before-|invalid-sqlite|\.bak|corrupt/.test(n))) +}) + +test('compatible extra columns and missing performance indexes do not prevent normal writes', async () => { + const path = seededFile(directory()) + modify(path, db => db.exec(`PRAGMA journal_mode=DELETE; DROP INDEX idx_entities_job; + ALTER TABLE entities ADD COLUMN optional_column TEXT; + ALTER TABLE entities ADD COLUMN with_default INTEGER NOT NULL DEFAULT 0;`)) + const store = existing(path) + try { await store.saveIfCurrent(seeded, (await store.loadSnapshot()).revision) } finally { store.close() } + modify(path, db => { + assert.equal(db.pragma('journal_mode', { simple: true }), 'delete') + assert.equal(db.prepare("SELECT 1 FROM sqlite_schema WHERE name='idx_entities_job'").get(), undefined) + }) +}) + +test('live WAL is authoritative; readonly preflight and first load use one payload scan', async () => { + const path = seededFile(directory()), opens = [], sql = [] + const peer = new Database(path, { fileMustExist: true }) + const strict = api(function Observed(file, options) { + opens.push({ file, options }) + return new Database(file, { ...options, verbose: query => sql.push(query) }) + }) + const change = name => peer.prepare(`UPDATE entities SET json=json_set(json, '$.name', ?) + WHERE collection='projects'`).run(name) + try { + peer.pragma('wal_autocheckpoint = 0') + const main = readFileSync(path) + change('in WAL') + assert.deepEqual(readFileSync(path), main) + assert.ok(readFileSync(path + '-wal').length) + const store = strict.createExistingLibraryStorage(path) + try { + assert.equal((await store.loadSnapshot()).data.projects[0].name, 'in WAL') + assert.equal(sql.filter(q => q.includes('FROM entities ORDER BY collection, id')).length, 1) + assert.deepEqual(opens.filter(o => o.file !== ':memory:'), [ + { file: path, options: { readonly: true, fileMustExist: true } }, { file: path, options: { fileMustExist: true } } + ]) + assert(!sql.some(q => /integrity_check|journal_mode\s*=|checkpoint|CREATE (TABLE|INDEX)/i.test(q))) + } finally { store.close() } + const another = existing(path) + try { + change('newer WAL without revision bump') + assert.equal((await another.load()).projects[0].name, 'newer WAL without revision bump') + peer.exec("UPDATE entities SET json='not JSON' WHERE collection='projects'") + await assert.rejects(another.load()) + } finally { another.close() } + } finally { peer.close() } +}) + +test('binding check touches only memory, is cached on success, and cannot fall back to JSON', async () => { + const opens = [], dir = directory(), path = seededFile(dir), reserved = join(dir, 'new.db') + reserve(reserved) + const before = readFileSync(path), names = readdirSync(dir) + const missing = api(function Unavailable(file) { opens.push(file); throw new Error('native unavailable') }) + assert.throws(missing.checkSqliteBindingAvailable, /native unavailable/) + assert.throws(() => missing.createExistingLibraryStorage(path), /native unavailable/) + assert.throws(() => missing.createNewLibraryStorage(reserved), /native unavailable/) + assert.deepEqual(readdirSync(dir), names) + assert.deepEqual(readFileSync(path), before) + assert.equal(readFileSync(reserved).length, 0) + const json = join(dir, 'new.json') + reserve(json) + assert.deepEqual((await missing.createNewLibraryStorage(json).load()).projects, []) + assert.deepEqual(opens, [':memory:', ':memory:', ':memory:']) + let probes = 0 + const available = api(function Traced(file) { assert.equal(file, ':memory:'); probes++; return new Database(file) }) + available.checkSqliteBindingAvailable() + available.checkSqliteBindingAvailable() + assert.equal(probes, 1) +}) diff --git a/scripts/validate-secure-credentials.mjs b/scripts/validate-secure-credentials.mjs index 76e2475..f365a18 100644 --- a/scripts/validate-secure-credentials.mjs +++ b/scripts/validate-secure-credentials.mjs @@ -146,7 +146,8 @@ async function main() { checks.push( assert( dataIpc.includes('storage.saveIfCurrent(secured.data') && - storageOps.includes('nextStorage.saveIfCurrent(secured.data)') && + storageOps.includes('next.storage.saveIfCurrent(secured.data, revision)') && + storageOps.includes('writeNewLibrary(targetPath, libraryStores, secured.data)') && storageOps.includes('backupFileForOverwrite(targetPath)'), 'storage migration overwrite path saves sanitized data and backs up target data' ) diff --git a/scripts/validate-sqlite-storage.mjs b/scripts/validate-sqlite-storage.mjs index 5b6deaa..70dfadd 100644 --- a/scripts/validate-sqlite-storage.mjs +++ b/scripts/validate-sqlite-storage.mjs @@ -357,7 +357,7 @@ async function main() { storageServiceSource.includes('backupTo(targetPath: string): Promise') && sqliteSource.includes('await db.backup(targetPath)') && sqliteSource.includes('close(): void') && - mainIndexSource.includes('storage.close?.()'), + mainIndexSource.includes('library.adopt(nextStorage)'), 'storage switching closes stale SQLite connections and exposes WAL-safe backup support' ) ) diff --git a/src/agent/AgentProjectOverviewRuntime.ts b/src/agent/AgentProjectOverviewRuntime.ts index 76b5702..2ffaf78 100644 --- a/src/agent/AgentProjectOverviewRuntime.ts +++ b/src/agent/AgentProjectOverviewRuntime.ts @@ -1,10 +1,8 @@ -import { existsSync } from 'node:fs' import { normalizeAppData } from '../shared/defaults' import type { AppData } from '../shared/types' import { loadSqliteCollectionsReadonly } from '../storage/SqliteStorageService' -import { resolveSqliteStoragePath } from '../storage/StorageService' import { - loadAgentRuntimeData, resolveAgentRuntimeStoragePath, + loadResolvedAgentRuntime, resolveAgentRuntimeStoragePath, type AgentRuntimeData, type AgentRuntimeLoadOptions } from './AgentRuntime' import { AGENT_PROJECT_OVERVIEW_COLLECTIONS, type AgentProjectOverviewData } from './agentProjectOverviewData' @@ -20,17 +18,15 @@ function pickOverview(data: AppData): AgentProjectOverviewData { export async function loadAgentProjectOverview( options: AgentRuntimeLoadOptions = {} ): Promise { - const { storagePath, userDataPath } = await resolveAgentRuntimeStoragePath(options) - const sqlitePath = resolveSqliteStoragePath(storagePath) - if (existsSync(sqlitePath)) { - const snapshot = await loadSqliteCollectionsReadonly(sqlitePath, AGENT_PROJECT_OVERVIEW_COLLECTIONS) + const choice = await resolveAgentRuntimeStoragePath(options, 'overview') + if (choice.source === 'sqlite') { + const snapshot = await loadSqliteCollectionsReadonly(choice.storagePath, AGENT_PROJECT_OVERVIEW_COLLECTIONS) return { - data: pickOverview(normalizeAppData(snapshot.data)), - source: 'sqlite', storagePath: sqlitePath, userDataPath, revision: snapshot.revision + ...choice, data: pickOverview(normalizeAppData(snapshot.data)), revision: snapshot.revision } } // JSON has no indexed collections. Preserve the legacy reader's parsing, path, // revision and empty-directory behavior, but return only the overview shape. - const runtime = await loadAgentRuntimeData({ storagePath, userDataPath }) + const runtime = await loadResolvedAgentRuntime(choice) return { ...runtime, data: pickOverview(runtime.data) } } diff --git a/src/agent/AgentRuntime.ts b/src/agent/AgentRuntime.ts index 1af28d1..8b7fbcb 100644 --- a/src/agent/AgentRuntime.ts +++ b/src/agent/AgentRuntime.ts @@ -1,8 +1,6 @@ -import { loadJsonSnapshotReadonly } from '../storage/jsonReadonly' -import { existsSync } from 'node:fs' -import { readFile } from 'node:fs/promises' -import { homedir } from 'node:os' -import { dirname, join, resolve } from 'node:path' +import { constants } from 'node:fs' +import { access, readFile } from 'node:fs/promises' +import { resolve } from 'node:path' import type { AppData, ChapterCommitBundle, RevisionCommitBundle } from '../shared/types' import type { StorageWriteResult } from '../shared/ipc/ipcTypes' import type { CandidateDecisionCommand } from '../shared/types/candidateDecision' @@ -11,16 +9,17 @@ import { previewCandidateDecisionUndo } from '../services/CandidateDecisionUndoS import { observeFormalChapterWrites } from '../services/ChapterWritebackFailureService' import type { FormalChapterCommitWriter, StorageService } from '../storage/StorageService' import { normalizeAppData } from '../shared/defaults' +import { loadJsonSnapshotReadonly } from '../storage/jsonReadonly' import { JsonStorageService } from '../storage/JsonStorageService' -import { createStorageService, loadSqliteSnapshotReadonly } from '../storage/SqliteStorageService' -import { JSON_DATA_FILE_NAME, SQLITE_DATA_FILE_NAME, resolveJsonStoragePath, resolveSqliteStoragePath, StorageRevisionConflictError } from '../storage/StorageService' +import { normalizeLibraryData, parseLibraryJsonValue } from '../storage/libraryValidation' +import { createExistingLibraryStorage, loadSqliteSnapshotReadonly } from '../storage/SqliteStorageService' +import { StorageRevisionConflictError } from '../storage/StorageService' +import { libraryStores, writeNewLibrary } from '../main/libraryFiles' import { AgentAuthorizationService } from '../main/services/AgentAuthorizationService' import { candidateDecisionChapterOrders } from './agentAuthorizationScope' +import { resolveAgentRuntimeStoragePath, storageSourceForPath, type AgentLibraryChoice, type AgentRuntimeLoadOptions } from './agentLibraryChoice' -export interface AgentRuntimeLoadOptions { - storagePath?: string - userDataPath?: string -} +export { resolveAgentRuntimeStoragePath, type AgentRuntimeLoadOptions } from './agentLibraryChoice' export interface AgentRuntimeData { data: AppData @@ -28,6 +27,7 @@ export interface AgentRuntimeData { userDataPath: string source: 'sqlite' | 'json' | 'empty' revision: string + namedJsonInput?: true } function isLoadedAgentRuntime( @@ -36,10 +36,6 @@ function isLoadedAgentRuntime( return 'source' in runtime && 'revision' in runtime && 'userDataPath' in runtime } -function storageSourceForPath(storagePath: string): 'sqlite' | 'json' { - return storagePath.toLowerCase().endsWith('.json') ? 'json' : 'sqlite' -} - function adoptRuntimeWriteResult( runtime: AgentRuntimeData | AgentRuntimeLoadOptions, result: Pick, @@ -52,69 +48,54 @@ function adoptRuntimeWriteResult( if (data) runtime.data = data } -function defaultUserDataPath(): string { - if (process.platform === 'win32') { - return join(process.env.APPDATA || join(homedir(), 'AppData', 'Roaming'), 'Novel Director') - } - if (process.platform === 'darwin') { - return join(homedir(), 'Library', 'Application Support', 'Novel Director') +export async function loadAgentRuntimeData(options: AgentRuntimeLoadOptions = {}): Promise { + const choice = await resolveAgentRuntimeStoragePath(options) + const runtime = await loadResolvedAgentRuntime(choice) + if (choice.source === 'json' && options.storagePath && resolve(options.storagePath) === choice.storagePath) { + runtime.namedJsonInput = true } - return join(process.env.XDG_CONFIG_HOME || join(homedir(), '.config'), 'Novel Director') + return runtime } -async function readConfiguredStoragePath(userDataPath: string): Promise { - try { - const config = JSON.parse(await readFile(join(userDataPath, 'app-config.json'), 'utf-8')) as { storagePath?: unknown } - return typeof config.storagePath === 'string' && config.storagePath.trim() ? config.storagePath : null - } catch { - return null - } +export async function loadResolvedAgentRuntime(choice: AgentLibraryChoice): Promise { + if (choice.source === 'empty') return { ...choice, data: normalizeAppData({}), revision: '0' } + if (choice.source === 'json') return { ...choice, ...await loadJsonSnapshotReadonly(choice.storagePath) } + const sqlitePath = choice.storagePath + return { ...choice, ...await loadSqliteSnapshotReadonly(sqlitePath) } } -export async function resolveAgentRuntimeStoragePath(options: AgentRuntimeLoadOptions = {}): Promise<{ - storagePath: string - userDataPath: string -}> { - const userDataPath = resolve(options.userDataPath ?? defaultUserDataPath()) - const configuredPath = options.storagePath ?? (await readConfiguredStoragePath(userDataPath)) - const storagePath = configuredPath ? resolve(configuredPath) : join(userDataPath, SQLITE_DATA_FILE_NAME) - return { storagePath, userDataPath } +function boundLibrary(runtime: AgentRuntimeData): AgentLibraryChoice { + const { storagePath, userDataPath, source } = runtime + if (!storagePath || source === 'empty') throw new Error('没有已打开的书库。请先在桌面选择已有书库。') + if (source !== storageSourceForPath(storagePath)) throw new Error(`书库路径与已打开的格式不一致:${storagePath}`) + return { storagePath, userDataPath, source } } -export async function loadAgentRuntimeData(options: AgentRuntimeLoadOptions = {}): Promise { - const { storagePath, userDataPath } = await resolveAgentRuntimeStoragePath(options) - const sqlitePath = resolveSqliteStoragePath(storagePath) - const jsonPath = resolveJsonStoragePath(storagePath) - - if (existsSync(sqlitePath)) { - const snapshot = await loadSqliteSnapshotReadonly(sqlitePath) - return { - data: snapshot.data, - storagePath: sqlitePath, - userDataPath, - source: 'sqlite', - revision: snapshot.revision - } - } +export async function reloadAgentRuntimeData(runtime: AgentRuntimeData): Promise { + const fresh = await loadResolvedAgentRuntime(boundLibrary(runtime)) + if (runtime.namedJsonInput) fresh.namedJsonInput = true + return fresh +} - const siblingJsonPath = existsSync(jsonPath) ? jsonPath : join(dirname(sqlitePath), JSON_DATA_FILE_NAME) - if (existsSync(siblingJsonPath)) { - const snapshot = await loadJsonSnapshotReadonly(siblingJsonPath) - return { - data: snapshot.data, - storagePath: siblingJsonPath, - userDataPath, - source: 'json', - revision: snapshot.revision - } - } +async function openAgentStorage(runtime: AgentRuntimeData | AgentRuntimeLoadOptions) { + const choice = isLoadedAgentRuntime(runtime) ? boundLibrary(runtime) : await resolveAgentRuntimeStoragePath(runtime) + if (choice.source === 'empty') throw new Error(`没有可写入的书库:${choice.storagePath}`) + return createExistingLibraryStorage(choice.storagePath) +} - return { - data: normalizeAppData({}), - storagePath, - userDataPath, - source: 'empty', - revision: storagePath.toLowerCase().endsWith('.json') ? 'missing' : '0' +async function openSnapshotWriter(runtime: AgentRuntimeData | AgentRuntimeLoadOptions): Promise> { + try { + return await openAgentStorage(runtime) + } catch (error) { + if (!isLoadedAgentRuntime(runtime) || !runtime.namedJsonInput || runtime.source !== 'json' || + !runtime.storagePath || storageSourceForPath(runtime.storagePath) !== 'json' || !/^[a-f0-9]{64}$/.test(runtime.revision)) throw error + // The explicit CLI accepts older snapshots without storyBibles. Only a full + // save with the digest of the exact input can upgrade one; never call its legacy load(). + const raw = parseLibraryJsonValue(await readFile(runtime.storagePath, 'utf8')) + if (!raw || typeof raw !== 'object' || Array.isArray(raw) || 'storyBibles' in raw) throw error + normalizeLibraryData({ ...raw, storyBibles: [] }) + await access(runtime.storagePath, constants.W_OK) + return new JsonStorageService(runtime.storagePath) } } @@ -123,31 +104,18 @@ export async function saveAgentRuntimeData(data: AppData, runtime: AgentRuntimeD source: 'sqlite' | 'json' revision: string }> { - const resolved = isLoadedAgentRuntime(runtime) - ? { - storagePath: runtime.storagePath ?? (await resolveAgentRuntimeStoragePath({ userDataPath: runtime.userDataPath })).storagePath, - userDataPath: runtime.userDataPath, - source: runtime.source - } - : { - ...(await resolveAgentRuntimeStoragePath(runtime)), - source: 'sqlite' as const - } - - const rawPath = resolved.storagePath - const isJson = resolved.source === 'json' || rawPath.toLowerCase().endsWith('.json') - if (isJson) { - const jsonPath = resolveJsonStoragePath(rawPath) - const result = await new JsonStorageService(jsonPath).saveIfCurrent( - data, - isLoadedAgentRuntime(runtime) ? runtime.revision : undefined - ) - adoptRuntimeWriteResult(runtime, result, data) - return { storagePath: result.storagePath, source: 'json', revision: result.revision } + // Legacy programmatic create-only request, never produced by a CLI/MCP read of a missing file. + if (isLoadedAgentRuntime(runtime) && runtime.source === 'json' && runtime.storagePath && + storageSourceForPath(runtime.storagePath) === 'json' && runtime.revision === 'missing') { + const { storage, snapshot } = await writeNewLibrary(runtime.storagePath, libraryStores, data) + try { + const result = { storagePath: storage.getStoragePath(), source: 'json' as const, revision: snapshot.revision } + adoptRuntimeWriteResult(runtime, result, data) + return result + } finally { storage.close?.() } } - const sqlitePath = resolveSqliteStoragePath(rawPath) - const storage = createStorageService(sqlitePath) + const storage = await openSnapshotWriter(runtime) try { const result = await storage.saveIfCurrent( data, @@ -164,33 +132,11 @@ export async function saveAgentRuntimeData(data: AppData, runtime: AgentRuntimeD } } -async function resolveAgentRuntimeSaveTarget(runtime: AgentRuntimeData | AgentRuntimeLoadOptions): Promise<{ - storagePath: string - userDataPath: string - source: 'sqlite' | 'json' | 'empty' -}> { - if (isLoadedAgentRuntime(runtime)) { - return { - storagePath: runtime.storagePath ?? (await resolveAgentRuntimeStoragePath({ userDataPath: runtime.userDataPath })).storagePath, - userDataPath: runtime.userDataPath, - source: runtime.source - } - } - return { - ...(await resolveAgentRuntimeStoragePath(runtime)), - source: 'sqlite' - } -} - async function saveAgentFormalCommit( runtime: AgentRuntimeData | AgentRuntimeLoadOptions, write: (storage: FormalChapterCommitWriter, revision?: string) => Promise ): Promise { - const resolved = await resolveAgentRuntimeSaveTarget(runtime) - const isJson = resolved.source === 'json' || resolved.storagePath.toLowerCase().endsWith('.json') - const storage: StorageService = isJson - ? new JsonStorageService(resolveJsonStoragePath(resolved.storagePath)) - : createStorageService(resolveSqliteStoragePath(resolved.storagePath)) + const storage = await openAgentStorage(runtime) try { const result = await write(observeFormalChapterWrites(storage), isLoadedAgentRuntime(runtime) ? runtime.revision : undefined) adoptRuntimeWriteResult(runtime, result) @@ -220,11 +166,7 @@ export async function saveAgentCandidateDecision( expectedRevision = runtime.revision ) { if (command.actor.kind !== 'agent') throw new Error('Agent candidate commands must use the agent actor.') - const resolved = await resolveAgentRuntimeSaveTarget(runtime) - const isJson = resolved.source === 'json' || resolved.storagePath.toLowerCase().endsWith('.json') - const storage = isJson - ? new JsonStorageService(resolveJsonStoragePath(resolved.storagePath)) - : createStorageService(resolveSqliteStoragePath(resolved.storagePath)) + const storage = await openAgentStorage(runtime) try { const current = await storage.loadSnapshot() const previous = current.data.candidateDecisionReceipts.find((receipt) => receipt.id === command.id) @@ -270,6 +212,6 @@ export async function saveAgentCandidateDecision( } return result } finally { - if ('close' in storage) storage.close?.() + storage.close?.() } } diff --git a/src/agent/agentLibraryChoice.ts b/src/agent/agentLibraryChoice.ts new file mode 100644 index 0000000..440288d --- /dev/null +++ b/src/agent/agentLibraryChoice.ts @@ -0,0 +1,71 @@ +import { homedir } from 'node:os' +import { dirname, extname, join, relative, resolve } from 'node:path' +import { AppConfigService } from '../main/AppConfigService' +import { exists, hasLibraryTraces, libraryFile } from '../main/libraryFiles' +import { JSON_DATA_FILE_NAME, SQLITE_DATA_FILE_NAME } from '../storage/StorageService' + +export interface AgentRuntimeLoadOptions { + storagePath?: string + userDataPath?: string +} + +export interface AgentLibraryChoice { + storagePath: string + userDataPath: string + source: 'sqlite' | 'json' | 'empty' +} + +export function storageSourceForPath(storagePath: string): 'sqlite' | 'json' { + return storagePath.toLowerCase().endsWith('.json') ? 'json' : 'sqlite' +} + +function defaultUserDataPath(): string { + if (process.platform === 'win32') { + return join(process.env.APPDATA || join(homedir(), 'AppData', 'Roaming'), 'Novel Director') + } + if (process.platform === 'darwin') return join(homedir(), 'Library', 'Application Support', 'Novel Director') + return join(process.env.XDG_CONFIG_HOME || join(homedir(), '.config'), 'Novel Director') +} + +function missingLibrary(path: string): Error { + return new Error(`Project database not found. 找不到原书库:${path}。请检查路径,或在桌面重新选择书库。`) +} + +export async function resolveAgentRuntimeStoragePath( + options: AgentRuntimeLoadOptions = {}, mode?: 'overview' +): Promise { + const userDataPath = resolve(options.userDataPath ?? defaultUserDataPath()) + const explicit = options.storagePath ? resolve(options.storagePath) : undefined + const namedFile = explicit && extname(explicit) ? libraryFile(explicit) : undefined + const directory = explicit && !namedFile ? explicit : userDataPath + const sqlite = join(directory, SQLITE_DATA_FILE_NAME) + const choice = (storagePath: string): AgentLibraryChoice => + ({ storagePath, userDataPath, source: storageSourceForPath(storagePath) }) + + if (namedFile) { + if (await exists(namedFile)) return choice(namedFile) + // Old overview callers probe an unconfigured profile using its default SQLite filename. + if (mode !== 'overview' || relative(sqlite, namedFile) !== '') throw missingLibrary(namedFile) + } + + const selected = await new AppConfigService(directory).readLibrarySelection() + if (selected) { + if (namedFile) throw missingLibrary(namedFile) + const path = libraryFile(selected.storagePath) + if (!await exists(path)) throw missingLibrary(path) + if (!selected.explicitJson && storageSourceForPath(path) === 'json' && await hasLibraryTraces(dirname(path), 'sqlite')) { + throw new Error(`旧配置指向 ${path},但同目录留有 SQLite 书库记录。请在桌面明确选择书库。`) + } + return choice(path) + } + + if (await exists(directory)) { + if (await hasLibraryTraces(directory, 'config')) throw new Error(`无法确认 ${directory} 上次选择的书库,请在桌面重新选择。`) + if (await exists(sqlite)) return choice(sqlite) + const json = join(directory, JSON_DATA_FILE_NAME) + const hasJson = await exists(json) + if (await hasLibraryTraces(directory, hasJson ? 'sqlite' : 'any')) throw missingLibrary(sqlite) + if (hasJson) return choice(json) + } + return { storagePath: sqlite, userDataPath, source: 'empty' } +} diff --git a/src/agent/agentPipelineWritebackGuard.ts b/src/agent/agentPipelineWritebackGuard.ts index bf9045e..947b838 100644 --- a/src/agent/agentPipelineWritebackGuard.ts +++ b/src/agent/agentPipelineWritebackGuard.ts @@ -3,14 +3,11 @@ import { normalizeAppData } from '../shared/defaults' import { BOOK_PROGRESS_COLLECTIONS } from '../services/BookProgressService' import { loadSqliteCollectionsReadonly } from '../storage/SqliteStorageService' import { StorageRevisionConflictError } from '../storage/StorageService' -import { loadAgentRuntimeData, saveAgentRuntimeData, type AgentRuntimeData } from './AgentRuntime' +import { reloadAgentRuntimeData, saveAgentRuntimeData, type AgentRuntimeData } from './AgentRuntime' import { checkPipelineRunTarget, isMultiChapterRun } from './agentBatchWriteback' export async function reopenPipelineRuntime(runtime: AgentRuntimeData) { - if (!runtime.storagePath) throw new Error('没有可重读的书库,无法核对章节任务。') - const fresh = await loadAgentRuntimeData({ storagePath: runtime.storagePath, userDataPath: runtime.userDataPath }) - if (fresh.source === 'empty' || fresh.storagePath !== runtime.storagePath) throw new Error('原书库已不存在或位置已变化,请重新打开任务。') - return fresh + return reloadAgentRuntimeData(runtime) } export function batchModelRequestGuard(runtime: AgentRuntimeData, run: AgentRun, jobId: ID) { diff --git a/src/agent/revision/agentRevisionGeneration.ts b/src/agent/revision/agentRevisionGeneration.ts index 95da1e1..f2a63b9 100644 --- a/src/agent/revision/agentRevisionGeneration.ts +++ b/src/agent/revision/agentRevisionGeneration.ts @@ -8,7 +8,7 @@ import { findRewriteRange } from '../../renderer/src/components/aiRewriteResultM import { looksLikeFullChapterRevision } from '../../renderer/src/utils/revisionMerge' import { createHeadlessTransport } from '../AgentPipelineExecutor' import { agentApiKey, apiEndpoint, readApiRegistry } from '../../main/services/ApiRegistry' -import { loadAgentRuntimeData, saveAgentRuntimeData, type AgentRuntimeData } from '../AgentRuntime' +import { reloadAgentRuntimeData, saveAgentRuntimeData, type AgentRuntimeData } from '../AgentRuntime' import { findProjectId, requiredString } from '../tools/agentToolArguments' import { readRevisionVersion } from './agentRevisionReads' import { @@ -85,7 +85,7 @@ export async function generateAgentRevisionVersion( aiRunId, modelConfigHash: revisionFingerprint(normalizePipelineAIModelOverride(settings)) } // Reload after the asynchronous model call. Never save the pre-request AppData snapshot. - const current = await loadAgentRuntimeData({ storagePath: runtime.storagePath ?? undefined, userDataPath: runtime.userDataPath }) + const current = await reloadAgentRuntimeData(runtime) const replay = replayRevisionOperation(current.data, projectId, identity) if (replay?.versionId) return { ...readRevisionVersion(current.data, projectId, replay.versionId), replayed: true } const currentSession = revisionSession(current.data, projectId, session.id) diff --git a/src/agent/tools/agentBookProjection.ts b/src/agent/tools/agentBookProjection.ts index 7753d9b..2ed130f 100644 --- a/src/agent/tools/agentBookProjection.ts +++ b/src/agent/tools/agentBookProjection.ts @@ -1,31 +1,20 @@ -import { existsSync } from 'node:fs' import type { AppDataArrayKey } from '../../shared/appDataCollections' import { normalizeAppData } from '../../shared/defaults' import { loadSqliteCollectionsReadonly } from '../../storage/SqliteStorageService' -import { loadJsonSnapshotReadonly } from '../../storage/jsonReadonly' -import { resolveSqliteStoragePath } from '../../storage/StorageService' -import { loadAgentRuntimeData, resolveAgentRuntimeStoragePath } from '../AgentRuntime' +import { loadResolvedAgentRuntime, resolveAgentRuntimeStoragePath } from '../AgentRuntime' import { findProjectId, stringArg } from './agentToolArguments' export async function loadAgentBookProjection(args: Record, collections: readonly AppDataArrayKey[]) { const selector = stringArg(args, 'projectId') ?? stringArg(args, 'project') if (!selector) throw new Error('Missing projectId or project.') - const explicit = stringArg(args, 'storagePath') - const paths = await resolveAgentRuntimeStoragePath({ storagePath: explicit, userDataPath: stringArg(args, 'userDataPath') }) - if (explicit && /\.(?:json|db|sqlite)$/i.test(explicit) && !existsSync(paths.storagePath)) { - throw new Error('Project database not found. 请检查指定的书库路径。') - } - if (paths.storagePath.toLowerCase().endsWith('.json') && (explicit || existsSync(paths.storagePath))) { - const { data, revision } = await loadJsonSnapshotReadonly(paths.storagePath) - return { data, projectId: findProjectId(data, args), storage: { source: 'json' as const, ...paths, revision } } - } - const sqlitePath = resolveSqliteStoragePath(paths.storagePath) - if (existsSync(sqlitePath)) { - const snapshot = await loadSqliteCollectionsReadonly(sqlitePath, collections, selector) + const choice = await resolveAgentRuntimeStoragePath({ + storagePath: stringArg(args, 'storagePath'), userDataPath: stringArg(args, 'userDataPath') + }) + if (choice.source === 'sqlite') { + const snapshot = await loadSqliteCollectionsReadonly(choice.storagePath, collections, selector) const data = normalizeAppData(snapshot.data) - return { data, projectId: findProjectId(data, args), storage: { source: 'sqlite' as const, storagePath: sqlitePath, - userDataPath: paths.userDataPath, revision: snapshot.revision } } + return { data, projectId: findProjectId(data, args), storage: { ...choice, revision: snapshot.revision } } } - const { data, ...storage } = await loadAgentRuntimeData(paths) + const { data, ...storage } = await loadResolvedAgentRuntime(choice) return { data, projectId: findProjectId(data, args), storage } } diff --git a/src/agent/tools/agentChapterTitleTools.ts b/src/agent/tools/agentChapterTitleTools.ts index 1d4492f..9ce9fc3 100644 --- a/src/agent/tools/agentChapterTitleTools.ts +++ b/src/agent/tools/agentChapterTitleTools.ts @@ -1,9 +1,7 @@ -import { existsSync } from 'node:fs' import { AgentAuthorizationService } from '../../main/services/AgentAuthorizationService' import { chapterTitleRows, parseChapterTitleRename, previewChapterTitleRename } from '../../services/ChapterTitleRenameService' -import { JsonStorageService } from '../../storage/JsonStorageService' -import { loadChapterTitlesReadonly, SqliteStorageService } from '../../storage/SqliteStorageService' -import { resolveJsonStoragePath, resolveSqliteStoragePath, StorageRevisionConflictError } from '../../storage/StorageService' +import { createExistingLibraryStorage, loadChapterTitlesReadonly } from '../../storage/SqliteStorageService' +import { StorageRevisionConflictError } from '../../storage/StorageService' import { resolveAgentRuntimeStoragePath } from '../AgentRuntime' import { requiredString, stringArg } from './agentToolArguments' import type { AgentToolCallResult, AgentToolDefinition, JsonSchema } from './agentToolTypes' @@ -29,14 +27,14 @@ export function chapterTitleDefinitions(schema: (properties: JsonSchema, require } export async function runChapterTitleTool(name: string, args: Record): Promise { - const projectId = requiredString(args, 'projectId'), explicit = stringArg(args, 'storagePath') - const paths = await resolveAgentRuntimeStoragePath({ storagePath: explicit, userDataPath: stringArg(args, 'userDataPath') }) - const sqlite = resolveSqliteStoragePath(paths.storagePath) - // Never switch an explicitly named file to a sibling book or silently fall back at commit. - const json = paths.storagePath.toLowerCase().endsWith('.json') || (!explicit && !existsSync(sqlite)) - const storagePath = json ? resolveJsonStoragePath(paths.storagePath) : sqlite + const projectId = requiredString(args, 'projectId') + const choice = await resolveAgentRuntimeStoragePath({ + storagePath: stringArg(args, 'storagePath'), userDataPath: stringArg(args, 'userDataPath') + }) + const { storagePath, userDataPath } = choice + if (choice.source === 'empty') throw new Error(`找不到项目书库:${storagePath}`) const { state, revision } = await loadChapterTitlesReadonly(storagePath, projectId) - const storage = { source: json ? 'json' as const : 'sqlite' as const, storagePath, userDataPath: paths.userDataPath, revision } + const storage = { ...choice, revision } const { storagePath: _path, userDataPath: _profile, ...input } = args if (name === history) { if (Object.keys(input).some(k => !['projectId', 'chapterId', 'limit', 'offset'].includes(k))) throw new Error('不支持的改名历史查询参数。') @@ -52,7 +50,7 @@ export async function runChapterTitleTool(name: string, args: Record r.operationId === command.operationId) - const authority = new AgentAuthorizationService(paths.userDataPath) + const authority = new AgentAuthorizationService(userDataPath) const scope = () => ({ storagePath, projectId, actions: ['manage_chapters'] as const, chapterOrders: (previous?.rows ?? chapterTitleRows(state, command)).map(row => row.chapterOrder) }) if (name === preview) { @@ -64,10 +62,10 @@ export async function runChapterTitleTool(name: string, args: Record { - const store = json ? new JsonStorageService(storagePath) : new SqliteStorageService(storagePath) + const store = createExistingLibraryStorage(storagePath) try { return await store.renameChapterTitles({ command, expectedRevision, expectedPreviewHash }, { kind: 'agent', authorizationGrantId: grant.id }) - } finally { if ('close' in store) store.close() } + } finally { store.close?.() } }) return { tool: name, ok: true, storage: { ...storage, revision: saved.revision }, data: saved } } diff --git a/src/agent/tools/agentWorldTools.ts b/src/agent/tools/agentWorldTools.ts index bde6649..13e91cc 100644 --- a/src/agent/tools/agentWorldTools.ts +++ b/src/agent/tools/agentWorldTools.ts @@ -4,7 +4,7 @@ import { readAgentCharacterCardReview } from '../agentCharacterCardReview' import { AgentAuthorizationService } from '../../main/services/AgentAuthorizationService' import { redactSensitiveText } from '../../shared/errorUtils' import type { AgentRuntimeData } from '../AgentRuntime' -import { loadAgentRuntimeData, saveAgentRuntimeData } from '../AgentRuntime' +import { reloadAgentRuntimeData, saveAgentRuntimeData } from '../AgentRuntime' import { findProjectId, readOptions, requiredString, stringArg } from './agentToolArguments' import type { AgentToolHandlerResult } from './agentToolTypes' @@ -196,7 +196,7 @@ export async function applyAgentWorldAction( storagePath, projectId: worldCommand.projectId, actions: ['edit_world'] }, async (grant) => { // The authorization lock covers a fresh snapshot, its validation, and the short CAS save. - const current = await loadAgentRuntimeData({ storagePath, userDataPath: runtime.userDataPath }) + const current = await reloadAgentRuntimeData(runtime) const currentCommand = { ...worldCommand, projectId: findProjectId(current.data, { projectId: worldCommand.projectId }) diff --git a/src/main/AppConfigService.ts b/src/main/AppConfigService.ts index 14288b2..76932ae 100644 --- a/src/main/AppConfigService.ts +++ b/src/main/AppConfigService.ts @@ -1,9 +1,16 @@ -import { copyFile, mkdir, open, readFile, rename, stat, writeFile } from 'node:fs/promises' -import { dirname, join } from 'node:path' +import { constants } from 'node:fs' +import { copyFile, mkdir, open, readFile, rename, writeFile } from 'node:fs/promises' +import { dirname, isAbsolute, join } from 'node:path' import { SQLITE_DATA_FILE_NAME } from '../storage/StorageService' interface AppConfig { storagePath?: string + explicitJson?: boolean +} + +function libraryConfig(value: unknown): value is AppConfig & { storagePath: string } { + return !!value && typeof value === 'object' && 'storagePath' in value && + typeof value.storagePath === 'string' && isAbsolute(value.storagePath) && /\.(sqlite|db|json)$/i.test(value.storagePath) } export class AppConfigService { @@ -22,8 +29,28 @@ export class AppConfigService { return config.storagePath || this.getDefaultStoragePath() } - async setStoragePath(storagePath: string): Promise { - await this.writeConfig({ ...(await this.readConfig()), storagePath }) + // Starting a library is different from displaying a suggested default path. + // A broken config must not be interpreted as consent to create a new library. + async readLibrarySelection(): Promise<{ storagePath: string; explicitJson: boolean } | null> { + let raw: string + try { + raw = await readFile(this.configPath, 'utf-8') + } catch (error) { + if (errorCode(error) === 'ENOENT') return null + throw error + } + try { + const config: unknown = JSON.parse(raw) + if (libraryConfig(config)) { + return { storagePath: config.storagePath, explicitJson: config.explicitJson === true } + } + } catch { /* Never use a guessed path when the saved choice cannot be read. */ } + throw new Error('无法确定上次选择的书库。请选择已有书库,或明确新建。') + } + + async setStoragePath(storagePath: string, explicitJson = false): Promise { + const config = await this.readConfig() + await this.writeConfig({ ...config, storagePath, explicitJson: explicitJson || undefined }, libraryConfig(config)) } async resetStoragePath(): Promise { @@ -56,7 +83,7 @@ export class AppConfigService { } } - private async writeConfig(config: AppConfig): Promise { + private async writeConfig(config: AppConfig, previousIsValid = true): Promise { await mkdir(dirname(this.configPath), { recursive: true }) const tmpPath = `${this.configPath}.tmp` const backupPath = `${this.configPath}.bak` @@ -65,11 +92,11 @@ export class AppConfigService { await syncFile(tmpPath) try { - await stat(this.configPath) - await copyFile(this.configPath, backupPath) + // A successful recovery from a broken config must not overwrite the last good .bak. + await copyFile(this.configPath, backupPath, previousIsValid ? 0 : constants.COPYFILE_EXCL) } catch (error) { const code = errorCode(error) - if (code !== 'ENOENT') { + if (code !== 'ENOENT' && (previousIsValid || code !== 'EEXIST')) { throw error } } diff --git a/src/main/BackupService.ts b/src/main/BackupService.ts index a44e757..4923ede 100644 --- a/src/main/BackupService.ts +++ b/src/main/BackupService.ts @@ -2,6 +2,7 @@ import { randomUUID } from 'node:crypto' import { mkdir, readdir, readFile, rename, stat, unlink, writeFile } from 'node:fs/promises' import { isAbsolute, join, relative, resolve } from 'node:path' import { normalizeAppData, sanitizeAppDataForPersistence } from '../shared/defaults' +import { decodeLibraryText, normalizeLibraryData, parseLibraryJsonValue } from '../storage/libraryValidation' import type { AppData } from '../shared/types' export interface BackupInfo { @@ -107,8 +108,8 @@ export class BackupService { async loadBackup(backupPath: string): Promise { const safePath = this.assertManagedBackupPath(backupPath) - const raw = await readFile(safePath, 'utf-8') - return sanitizeAppDataForPersistence(normalizeAppData(JSON.parse(raw) as Partial)) + const raw = decodeLibraryText(await readFile(safePath)) + return normalizeLibraryData(parseLibraryJsonValue(raw), sanitizeAppDataForPersistence) } async deleteBackup(backupPath: string): Promise { diff --git a/src/main/LibrarySession.ts b/src/main/LibrarySession.ts new file mode 100644 index 0000000..cdd48e2 --- /dev/null +++ b/src/main/LibrarySession.ts @@ -0,0 +1,154 @@ +import { dirname, extname, join, relative } from 'node:path' +import { JSON_DATA_FILE_NAME, resolveSqliteStoragePath, type StorageService } from '../storage/StorageService' +import type { AppData } from '../shared/types' +import type { AppConfigService } from './AppConfigService' +import { backupLegacyJson, exists, hasLibraryTraces, libraryFile, libraryStores, + openExistingLibrary, writeNewLibrary, type LibraryStores, type OpenLibrary } from './libraryFiles' + +function failure(path: string, cause: unknown) { + const message = cause instanceof Error ? cause.message : '' + const reason = /NODE_MODULE_VERSION|bindings file|native module|better-sqlite3/i.test(message) + ? 'SQLite 组件与当前程序不匹配或无法加载。请修复安装后重试,或明确选择另一份书库。' + : /SQLITE_CORRUPT|SQLITE_NOTADB|malformed|not a database/i.test(message) + ? '文件损坏或不是有效的 SQLite 书库。请从备份恢复到新位置。' + : /EACCES|EPERM|EBUSY|SQLITE_BUSY/.test(message) ? '文件被占用或当前没有读写权限。请检查后重试。' + : message || '读取失败,请检查磁盘或选择备份。' + return Object.assign(new Error(`无法打开书库:${path}\n${reason}`), { code: 'LIBRARY_UNAVAILABLE' }) +} + +// The only handle exposed to desktop writers belongs to a successfully read library. +export class LibrarySession { + private active?: StorageService + private unavailable?: Error + private opening?: Promise + private requestedPath: string + + constructor(private readonly config: AppConfigService, private readonly stores: LibraryStores = libraryStores, + private readonly prepareLegacy: (data: AppData) => Promise = async data => data, + private readonly previousDesktopLaunch = false) { + this.requestedPath = config.getDefaultStoragePath() + } + + get path(): string { return this.active?.getStoragePath() ?? this.requestedPath } + getStorage(): StorageService { + if (!this.active) throw this.unavailable ?? failure(this.path, new Error('尚未成功读取书库,请稍后重试。')) + return this.active + } + adopt(storage: StorageService): void { + if (this.active !== storage) this.active?.close?.() + this.active = storage + this.requestedPath = storage.getStoragePath() + this.unavailable = undefined + } + close(): void { this.active?.close?.(); this.active = undefined } + + async load(): Promise { + if (this.opening) return this.opening + if (this.unavailable) throw this.unavailable + if (!this.active) return this.openConfigured() + const storage = this.active + try { + const snapshot = await storage.loadSnapshot() + if (storage !== this.active) throw new Error('书库已切换,请重新加载。') + return { storage, snapshot } + } catch (error) { + if (storage !== this.active) throw error + this.close() + throw (this.unavailable = failure(this.path, error)) + } + } + + assertRecoveryAvailable(action?: string): void { + if (action === 'retry' && this.active && !this.opening) return + if (this.active || !this.unavailable || this.opening) throw new Error('书库状态已改变,请刷新后重试。') + } + + async recover(action: 'retry' | 'choose' | 'restore' | 'create', path?: string, backup?: string): Promise { + this.assertRecoveryAvailable(action) + if (action === 'retry') { await (this.active ? this.load() : this.openConfigured()); return } + if (!path) throw new Error('尚未选择书库。') + const target = libraryFile(path) + const recover = async (): Promise => { + let opened: OpenLibrary + if (action === 'choose') { + opened = await openExistingLibrary(target, this.stores) + } else { + if (!relative(target, this.requestedPath)) throw new Error('请另选一个新文件名,原书库位置将保留。') + if (action === 'restore' && !backup) throw new Error('尚未选择备份。') + let data: AppData | undefined + if (backup) { + if (extname(backup).toLowerCase() !== '.json') throw new Error('请选择 JSON 书库备份。') + const source = await openExistingLibrary(backup, this.stores) + try { data = source.snapshot.data } finally { source.storage.close?.() } + } + opened = await writeNewLibrary(target, this.stores, data) + } + return this.finish(opened, true) + } + this.opening = recover() + try { await this.opening } finally { this.opening = undefined } + } + + private openConfigured(): Promise { + this.opening = this.readConfigured().catch(error => { + throw (this.unavailable = failure(this.requestedPath, error)) + }).finally(() => { this.opening = undefined }) + return this.opening + } + + private async readConfigured(): Promise { + this.requestedPath = await this.config.getConfigPath() + const selected = await this.config.readLibrarySelection() + const path = selected?.storagePath ?? this.config.getDefaultStoragePath() + this.requestedPath = path + const directory = dirname(path) + if (!selected && await hasLibraryTraces(directory, 'config')) { + throw new Error('发现旧书库配置记录,但无法确认上次使用的位置。请选择原书库。') + } + if (selected || await exists(path)) { + const opened = await openExistingLibrary(path, this.stores) + if (!selected || selected.explicitJson || extname(path).toLowerCase() !== '.json') { + return this.finish(opened, !selected) + } + // Old configs did not distinguish deliberate JSON use from an automatic fallback. + try { + const target = resolveSqliteStoragePath(path) + if (await hasLibraryTraces(directory, 'sqlite')) { + throw new Error('旧配置指向 JSON,但无法确定上次使用的是哪一份书库。请明确选择 SQLite 或独立 JSON。') + } + return await this.migrateLegacy(path, opened, target) + } finally { opened.storage.close?.() } + } + const legacy = join(directory, JSON_DATA_FILE_NAME) + const hasLegacy = await exists(legacy) + if ((!hasLegacy && this.previousDesktopLaunch) || await hasLibraryTraces(directory, hasLegacy ? 'sqlite' : 'any')) { + throw new Error('发现旧书库记录,但原书库不存在。请连接磁盘、选择书库或从备份恢复。') + } + if (hasLegacy) { + this.requestedPath = legacy + const opened = await openExistingLibrary(legacy, this.stores) + try { return await this.migrateLegacy(legacy, opened, path) } finally { opened.storage.close?.() } + } + return this.finish(await writeNewLibrary(path, this.stores), true) + } + + private async migrateLegacy(source: string, opened: OpenLibrary, target: string): Promise { + this.stores.checkSqlite() + const data = await this.prepareLegacy(opened.snapshot.data) + return this.finish(await writeNewLibrary(target, this.stores, data, () => backupLegacyJson(source)), true) + } + + private async finish(opened: OpenLibrary, remember: boolean): Promise { + try { + if (remember) { + const path = opened.storage.getStoragePath() + await this.config.setStoragePath(path, extname(path).toLowerCase() === '.json') + } + this.adopt(opened.storage) + return opened + } catch (error) { + opened.storage.close?.() + throw error + } + } +} diff --git a/src/main/dataMerge/dataFileStorage.ts b/src/main/dataMerge/dataFileStorage.ts index c032cd7..830ef9b 100644 --- a/src/main/dataMerge/dataFileStorage.ts +++ b/src/main/dataMerge/dataFileStorage.ts @@ -1,17 +1,8 @@ import { mkdir, stat } from 'node:fs/promises' import { dirname } from 'node:path' import type { AppData } from '../../shared/types' -import { JsonStorageService } from '../../storage/JsonStorageService' -import { SqliteStorageService } from '../../storage/SqliteStorageService' -import type { StorageService, StorageSnapshot } from '../../storage/StorageService' - -function isSqliteDataPath(path: string): boolean { - return /\.(sqlite|db)$/i.test(path) -} - -function createDataFileStorage(path: string): StorageService { - return isSqliteDataPath(path) ? new SqliteStorageService(path) : new JsonStorageService(path) -} +import { createExistingLibraryStorage as createDataFileStorage } from '../../storage/SqliteStorageService' +import type { StorageSnapshot } from '../../storage/StorageService' export async function loadDataFileSnapshot(path: string): Promise { const storage = createDataFileStorage(path) diff --git a/src/main/index.ts b/src/main/index.ts index 63d2d17..3a5e90f 100644 --- a/src/main/index.ts +++ b/src/main/index.ts @@ -9,12 +9,13 @@ import { TokenBucketRateLimiter } from './RateLimiter' import { AIService } from './services/AIService' import { AIHttpClient } from './services/AIHttpClient' import { appendProjectUsageAttempts } from '../storage/projectUsageJournal' -import type { StorageService } from '../storage/StorageService' -import { createStorageService } from '../storage/SqliteStorageService' +import { LibrarySession } from './LibrarySession' +import { registerLibraryRecoveryIpc } from './ipc/libraryRecoveryIpc' +import { secureAndSanitizeAppData } from './ipc/appDataCredentialSanitizer' import { registerIpcHandlers } from './ipc/registerIpcHandlers' import type { ApiProvider } from '../shared/types' -let storage: StorageService +let library: LibrarySession let appConfig: AppConfigService let credentialService: SecureCredentialService let backupService: BackupService @@ -261,34 +262,34 @@ if (!hasSingleInstanceLock) { }) app.whenReady().then(async () => { + // preview.20 did not always record the default path. Check before this launch writes its first log. + const previousDesktopLaunch = existsSync(join(app.getPath('userData'), 'logs', 'main.log')) LogService.initialize() app.setAppUserModelId('com.novel-director.mvp') registerContentSecurityPolicy() appConfig = new AppConfigService(app.getPath('userData')) credentialService = new SecureCredentialService(app.getPath('userData')) backupService = new BackupService(app.getPath('userData')) + library = new LibrarySession(appConfig, undefined, async data => (await secureAndSanitizeAppData({ credentialService }, data)).data, + previousDesktopLaunch) const aiService = new AIService(credentialService, aiRateLimiters, { httpClient: new AIHttpClient({ fetch: (url, options) => net.fetch(url, options), transport: 'electron-net' }), - usageJournal: { getStoragePath: () => storage.getStoragePath(), append: appendProjectUsageAttempts } + usageJournal: { getStoragePath: () => library.getStorage().getStoragePath(), append: appendProjectUsageAttempts } }) aiService.setLogger({ info: (message) => LogService.info(message), warn: (message) => LogService.warn(message) }) - const configuredStoragePath = await appConfig.getStoragePath() - storage = createStorageService(configuredStoragePath) - if (storage.getStoragePath() !== configuredStoragePath) { - await appConfig.setStoragePath(storage.getStoragePath()) - } + registerLibraryRecoveryIpc(library, app.getPath('userData'), backupService.getBackupDir()) + app.on('will-quit', () => library.close()) registerIpcHandlers({ appConfig, credentialService, backupService, - getStorage: () => storage, - setStorage: (nextStorage) => { - if (storage !== nextStorage) storage.close?.() - storage = nextStorage - }, + getStorage: () => library.getStorage(), + setStorage: (nextStorage) => library.adopt(nextStorage), + loadLibrary: () => library.load(), + getLibraryPath: () => library.path, aiService }) diff --git a/src/main/ipc/backupIpcHandlers.ts b/src/main/ipc/backupIpcHandlers.ts index 6a8d63f..a3a525d 100644 --- a/src/main/ipc/backupIpcHandlers.ts +++ b/src/main/ipc/backupIpcHandlers.ts @@ -44,13 +44,13 @@ export function registerBackupIpcHandlers(context: BackupIpcContext): void { safeIpcHandler(async (_event, input: string | BackupRestoreRequest): Promise => { const storage = context.getStorage() const request = typeof input === 'string' ? { backupPath: input } : input + const data = await context.backupService.loadBackup(validateFilePath(request.backupPath, 'Backup path')) let preRestoreBackupPath: string | undefined try { preRestoreBackupPath = await context.backupService.createBackup(await storage.load(), false) } catch (error) { LogService.warn('Pre-restore backup failed; continuing with selected backup restore.', error) } - const data = await context.backupService.loadBackup(validateFilePath(request.backupPath, 'Backup path')) const saved = await storage.saveIfCurrent(data, request.expectedRevision) LogService.info(`Backup restored: ${request.backupPath}`) return { diff --git a/src/main/ipc/dataIpcHandlers.ts b/src/main/ipc/dataIpcHandlers.ts index 0e040ea..6773381 100644 --- a/src/main/ipc/dataIpcHandlers.ts +++ b/src/main/ipc/dataIpcHandlers.ts @@ -1,13 +1,13 @@ import { dialog, ipcMain } from 'electron' import { readFile, writeFile } from 'node:fs/promises' -import { safeParseJson } from '../../services/AIJsonParser' +import { decodeLibraryText, normalizeImportedLibrary, parseLibraryJsonValue } from '../../storage/libraryValidation' import { observeFormalChapterWrites } from '../../services/ChapterWritebackFailureService' import { parseChapterTitleRename, previewChapterTitleRename } from '../../services/ChapterTitleRenameService' import { loadChapterTitlesReadonly } from '../../storage/SqliteStorageService' import { StorageRevisionConflictError } from '../../storage/StorageService' import type { ChapterTitleCommit, ChapterTitleRename } from '../../shared/types/chapterTitle' import { collectionName } from '../../shared/appDataCollections' -import { normalizeAppData, sanitizeAppDataForPersistence } from '../../shared/defaults' +import { sanitizeAppDataForPersistence } from '../../shared/defaults' import { IPC_CHANNELS } from '../../shared/ipc/ipcChannels' import { readProjectUsage } from '../../storage/readProjectUsage' import type { @@ -81,9 +81,10 @@ export function registerDataIpcHandlers(context: StorageDataOperationContext): v ipcMain.handle( IPC_CHANNELS.STORAGE_GET, safeIpcHandler(async (): Promise => { - const storage = context.getStorage() + const opened = await context.loadLibrary?.() + const storage = opened?.storage ?? context.getStorage() LogService.info(`Loading storage data from ${storage.getStoragePath()}`) - const snapshot = await storage.loadSnapshot() + const snapshot = opened?.snapshot ?? await storage.loadSnapshot() const loaded = snapshot.data const secured = await secureAndSanitizeAppData(context, loaded) let revision = snapshot.revision @@ -164,6 +165,7 @@ export function registerDataIpcHandlers(context: StorageDataOperationContext): v ipcMain.handle( IPC_CHANNELS.STORAGE_EXPORT, safeIpcHandler(async (_event, data: AppData): Promise => { + context.getStorage() const result = await dialog.showSaveDialog({ title: '导出 Novel Director 数据', defaultPath: 'novel-director-export.json', @@ -188,14 +190,11 @@ export function registerDataIpcHandlers(context: StorageDataOperationContext): v if (result.canceled || !result.filePaths[0]) return { canceled: true } const storage = context.getStorage() const importPath = result.filePaths[0] - const raw = await readFile(importPath, 'utf-8') - const parsed = safeParseJson>(raw, '导入数据文件') - if (!parsed.ok) throw new Error(parsed.parseError) - const imported = normalizeAppData(parsed.data) + const strategy = request?.strategy === 'merge' ? 'merge' : 'replace' + const imported = normalizeImportedLibrary(parseLibraryJsonValue(decodeLibraryText(await readFile(importPath))), strategy) const secured = await secureAndSanitizeAppData(context, imported, { migrateLegacyApiKey: false }) - const strategy = request?.strategy === 'merge' ? 'merge' : 'replace' const currentSnapshot = await storage.loadSnapshot() let importedProjectIds = secured.data.projects.map((project) => project.id) diff --git a/src/main/ipc/libraryRecoveryIpc.ts b/src/main/ipc/libraryRecoveryIpc.ts new file mode 100644 index 0000000..06fc108 --- /dev/null +++ b/src/main/ipc/libraryRecoveryIpc.ts @@ -0,0 +1,55 @@ +import { dialog, ipcMain } from 'electron' +import { join } from 'node:path' +import { IPC_CHANNELS } from '../../shared/ipc/ipcChannels' +import type { LibrarySession } from '../LibrarySession' +import { safeIpcHandler } from './safeIpcHandler' +import { assertSafeDataStoragePath, isPathInsideOrSame } from './storagePath' + +// File paths come exclusively from a native picker, never from the renderer. +export function registerLibraryRecoveryIpc(session: LibrarySession, userData: string, backups: string): void { + let busy = false + ipcMain.handle(IPC_CHANNELS.APP_RECOVER_LIBRARY, safeIpcHandler(async (_event, action: unknown) => { + if (action !== 'choose' && action !== 'restore' && action !== 'create' && action !== 'retry') { + throw new Error('未知的书库恢复操作。') + } + if (busy) throw new Error('已有书库操作正在进行。') + session.assertRecoveryAvailable(action) + busy = true + try { + let source: string | undefined + let target: string | undefined + if (action === 'choose' || action === 'restore') { + const chosen = await dialog.showOpenDialog({ + title: action === 'restore' ? '选择要恢复的 JSON 备份' : '选择已有书库', + defaultPath: action === 'restore' ? backups : undefined, properties: ['openFile'], + filters: [{ name: action === 'restore' ? 'JSON 书库备份' : '书库文件', extensions: action === 'restore' ? ['json'] : ['sqlite', 'db', 'json'] }] + }) + if (chosen.canceled || !chosen.filePaths[0]) return { ok: true, cancelled: true } + if (action === 'choose') target = chosen.filePaths[0] + else source = chosen.filePaths[0] + } + if (action === 'restore' || action === 'create') { + const chosen = await dialog.showSaveDialog({ + title: action === 'restore' ? '将备份恢复到新文件(不覆盖原书库)' : '新建独立书库', + defaultPath: join(userData, action === 'restore' ? '恢复的书库.sqlite' : '新书库.sqlite'), + filters: [{ name: 'SQLite 书库', extensions: ['sqlite'] }, { name: '独立 JSON 书库', extensions: ['json'] }] + }) + if (chosen.canceled || !chosen.filePath) return { ok: true, cancelled: true } + target = chosen.filePath + } + if (target) assertSafeDataStoragePath(target) + if (target && (isPathInsideOrSame(target, backups) || /\.before-sqlite-migrate\..*\.json$/i.test(target))) { + throw new Error('这是备份位置,不能用作当前书库。请用“从备份恢复到新文件”,另选保存位置。') + } + if (target && /\.json$/i.test(target)) { + const choice = await dialog.showMessageBox({ + type: 'question', title: '使用独立 JSON 书库', message: '后续修改只保存到这个文件,不会自动合回 SQLite。', + detail: target, buttons: ['取消', '使用这个 JSON 书库'], defaultId: 0, cancelId: 0 + }) + if (choice.response !== 1) return { ok: true, cancelled: true } + } + await session.recover(action, target, source) + return { ok: true } + } finally { busy = false } + })) +} diff --git a/src/main/ipc/storageDataOperations.ts b/src/main/ipc/storageDataOperations.ts index c705981..09fd933 100644 --- a/src/main/ipc/storageDataOperations.ts +++ b/src/main/ipc/storageDataOperations.ts @@ -4,9 +4,9 @@ import { getUserFriendlyError, redactSensitiveText } from '../../shared/errorUti import type { MigrateStoragePathResult } from '../../shared/ipc/ipcTypes' import type { AppData } from '../../shared/types' import type { StorageService } from '../../storage/StorageService' -import { createStorageService } from '../../storage/SqliteStorageService' import type { AppConfigService } from '../AppConfigService' import type { BackupService } from '../BackupService' +import { libraryStores, openExistingLibrary, writeNewLibrary, type OpenLibrary } from '../libraryFiles' import { backupFileForOverwrite, createMigrationMergePreview @@ -20,6 +20,8 @@ export interface StorageDataOperationContext { credentialService: SecureCredentialService backupService: BackupService getStorage: () => StorageService + loadLibrary?: () => Promise + getLibraryPath?: () => string setStorage: (storage: StorageService) => void } @@ -84,6 +86,7 @@ export async function migrateStoragePath( try { const secured = await secureAndSanitizeAppData(context, data) const currentSave = await storage.saveIfCurrent(secured.data, expectedRevision) + if (oldPath === targetPath) return { ok: true, storagePath: oldPath, revision: currentSave.revision } const targetAlreadyExists = oldPath !== targetPath && (await pathExists(targetPath)) if (targetAlreadyExists && !overwrite) { @@ -103,18 +106,26 @@ export async function migrateStoragePath( await mkdir(dirname(targetPath), { recursive: true }) const backupPath = await backupExistingSourceData(storage, oldPath) const targetBackupPath = targetAlreadyExists ? await backupFileForOverwrite(targetPath) : null - const nextStorage = createStorageService(targetPath) - const nextSave = await nextStorage.saveIfCurrent(secured.data) - const activeStoragePath = nextStorage.getStoragePath() - await context.appConfig.setStoragePath(activeStoragePath) - context.setStorage(nextStorage) + const next = targetAlreadyExists + ? await openExistingLibrary(targetPath, libraryStores) + : await writeNewLibrary(targetPath, libraryStores, secured.data) + let revision = next.snapshot.revision + const activeStoragePath = next.storage.getStoragePath() + try { + if (targetAlreadyExists) revision = (await next.storage.saveIfCurrent(secured.data, revision)).revision + await context.appConfig.setStoragePath(activeStoragePath, /\.json$/i.test(activeStoragePath)) + context.setStorage(next.storage) + } catch (error) { + next.storage.close?.() + throw error + } return { ok: true, storagePath: activeStoragePath, backupPath: backupPath ?? undefined, targetBackupPath: targetBackupPath ?? undefined, - revision: nextSave.revision + revision } } catch (error) { return { diff --git a/src/main/ipc/storageManagementIpcHandlers.ts b/src/main/ipc/storageManagementIpcHandlers.ts index c54ac93..8a6cf55 100644 --- a/src/main/ipc/storageManagementIpcHandlers.ts +++ b/src/main/ipc/storageManagementIpcHandlers.ts @@ -17,7 +17,7 @@ import { confirmMigrationMerge, createMigrationMergePreview } from '../DataMergeService' -import { createStorageService } from '../../storage/SqliteStorageService' +import { createExistingLibraryStorage } from '../../storage/SqliteStorageService' import { safeIpcHandler } from './safeIpcHandler' import { resolveDataStoragePath } from './storagePath' import type { StorageDataOperationContext } from './storageDataOperations' @@ -27,7 +27,7 @@ export function registerStorageManagementIpcHandlers(context: StorageDataOperati ipcMain.handle( IPC_CHANNELS.APP_GET_STORAGE_PATH, safeIpcHandler(async (): Promise => ({ - storagePath: context.getStorage().getStoragePath(), + storagePath: context.getLibraryPath?.() ?? context.getStorage().getStoragePath(), defaultStoragePath: context.appConfig.getDefaultStoragePath() })) ) @@ -66,6 +66,7 @@ export function registerStorageManagementIpcHandlers(context: StorageDataOperati IPC_CHANNELS.APP_CREATE_MIGRATION_MERGE_PREVIEW, safeIpcHandler( async (_event, request: MigrationMergePreviewRequest): Promise => { + context.getStorage() const sourcePath = await resolveDataStoragePath(request.sourcePath) const targetPath = await resolveDataStoragePath(request.targetPath) return { @@ -79,12 +80,13 @@ export function registerStorageManagementIpcHandlers(context: StorageDataOperati ipcMain.handle( IPC_CHANNELS.APP_CONFIRM_MIGRATION_MERGE, safeIpcHandler(async (_event, request: ConfirmMigrationMergeRequest): Promise => { + context.getStorage() const sourcePath = await resolveDataStoragePath(request.sourcePath) const targetPath = await resolveDataStoragePath(request.targetPath) const result = await confirmMigrationMerge(sourcePath, targetPath, request.expectedRevision) - const nextStorage = createStorageService(targetPath) + const nextStorage = createExistingLibraryStorage(targetPath) const activeStoragePath = nextStorage.getStoragePath() - await context.appConfig.setStoragePath(activeStoragePath) + await context.appConfig.setStoragePath(activeStoragePath, /\.json$/i.test(activeStoragePath)) context.setStorage(nextStorage) return { ok: true, diff --git a/src/main/libraryFiles.ts b/src/main/libraryFiles.ts new file mode 100644 index 0000000..1eeaf55 --- /dev/null +++ b/src/main/libraryFiles.ts @@ -0,0 +1,77 @@ +import { constants } from 'node:fs' +import { copyFile, lstat, open, readdir, unlink } from 'node:fs/promises' +import { extname, isAbsolute, join, resolve } from 'node:path' +import type { AppData } from '../shared/types' +import type { StorageService, StorageSnapshot } from '../storage/StorageService' +import { checkSqliteBindingAvailable, createExistingLibraryStorage, createNewLibraryStorage } from '../storage/SqliteStorageService' + +export const libraryStores = { existing: createExistingLibraryStorage, create: createNewLibraryStorage, checkSqlite: checkSqliteBindingAvailable } +export type LibraryStores = typeof libraryStores +export interface OpenLibrary { storage: StorageService; snapshot: StorageSnapshot } + +export function libraryFile(path: string): string { + if (!isAbsolute(path) || !['.json', '.db', '.sqlite'].includes(extname(path).toLowerCase())) { + throw new Error('请选择完整路径的 .sqlite、.db 或 .json 书库文件。') + } + return resolve(path) +} + +export async function exists(path: string): Promise { + try { await lstat(path); return true } catch (error) { + if ((error as NodeJS.ErrnoException).code === 'ENOENT') return false + throw error + } +} + +export async function hasLibraryTraces(directory: string, kind: 'config' | 'sqlite' | 'any' = 'any'): Promise { + const entries = await readdir(directory) + if (kind === 'config') return entries.some(name => /^app-config\.json\.(?:bak|tmp|corrupt\.)/i.test(name)) + if (entries.some(name => /\.sqlite(?:$|[.-])|^novel-director-data\.db(?:$|[.-])|\.before-sqlite-migrate\./i.test(name))) return true + if (kind === 'sqlite') return false + if (entries.some(name => /^(secure-credentials\.json|agent-authorizations\.json)$|^novel-director-data\.json\./i.test(name))) return true + const backups = join(directory, 'backups') + return await exists(backups) && (await readdir(backups)).some(name => /\.json(?:\.gz)?$/i.test(name)) +} + +export async function openExistingLibrary(path: string, stores: LibraryStores): Promise { + if (!await exists(path)) throw new Error('找不到原书库。请连接原磁盘,或选择已有书库。') + const storage = stores.existing(libraryFile(path)) + try { + return { storage, snapshot: await storage.loadSnapshot() } + } catch (error) { + storage.close?.() + throw error + } +} + +// Never pass an existing destination (or an orphaned WAL) to the initializing backend. +export async function writeNewLibrary(path: string, stores: LibraryStores, data?: AppData, + beforeInitialize?: () => Promise): Promise { + path = libraryFile(path) + for (const suffix of ['-wal', '-shm', '-journal', '.bak', '.tmp']) { + if (await exists(path + suffix)) throw new Error('目标位置留有旧书库文件,请另选一个新文件名。') + } + let reserved: Awaited> + try { reserved = await open(path, 'wx', 0o600) } catch (error) { + if ((error as NodeJS.ErrnoException).code === 'EEXIST') throw new Error('目标文件已存在。不会覆盖,请另选一个新文件名。') + throw error + } + const owned = await reserved.stat() + let storage: StorageService | undefined + try { + await reserved.close() + await beforeInitialize?.() + // Schema and initial data commit together: an interruption cannot publish an empty migrated book. + storage = stores.create(path, data) + return { storage, snapshot: await storage.loadSnapshot() } + } catch (error) { + storage?.close?.() + const current = await lstat(path).catch(() => null) + if (current && current.dev === owned.dev && current.ino === owned.ino) await unlink(path) + throw error + } +} + +export async function backupLegacyJson(path: string): Promise { + await copyFile(path, path + '.before-sqlite-migrate.' + Date.now() + '.json', constants.COPYFILE_EXCL) +} diff --git a/src/preload/index.ts b/src/preload/index.ts index 8d99f9d..df61dde 100644 --- a/src/preload/index.ts +++ b/src/preload/index.ts @@ -43,12 +43,14 @@ import type { ImportDataRequest, ImportDataStrategy, IpcFailure, + LibraryRecoveryAction, LogsGetPathResult, LogsOpenResult, ListAiCallProgressResult, MigrateStoragePathResult, MigrationMergePreviewResult, OpenStorageFolderResult, + RecoverLibraryResult, SaveFileResult, StorageWriteResult, SelectStoragePathResult, @@ -153,6 +155,10 @@ const novelDirector = { }, app: { getStoragePath: () => invokeOrThrow(IPC_CHANNELS.APP_GET_STORAGE_PATH), + recoverLibrary: async (action: LibraryRecoveryAction) => { + if (!['retry', 'choose', 'restore', 'create'].includes(action)) throw new Error('不支持的书库恢复操作。') + return invokeOrThrow(IPC_CHANNELS.APP_RECOVER_LIBRARY, action) + }, getRuntimeInfo: () => invokeOrThrow(IPC_CHANNELS.APP_GET_RUNTIME_INFO), selectStoragePath: () => invokeOrThrow(IPC_CHANNELS.APP_SELECT_STORAGE_PATH), migrateStoragePath: async (storagePath: string, data: AppData, overwrite = false) => { diff --git a/src/renderer/src/App.tsx b/src/renderer/src/App.tsx index a0204b7..360824d 100644 --- a/src/renderer/src/App.tsx +++ b/src/renderer/src/App.tsx @@ -1,9 +1,10 @@ -import { lazy, Suspense, useRef, useState } from 'react' +import { lazy, Suspense, useEffect, useRef, useState } from 'react' import type { AppData, ID, Project } from '../../shared/types' import { ErrorBoundary } from './components/ErrorBoundary' import { useConfirm } from './components/ConfirmDialog' import { Shell, type View } from './components/Layout' import { useAppData } from './hooks/useAppData' +import { LibraryRecoveryView } from './views/LibraryRecoveryView' import { QuickRewriteDraftProvider } from './components/QuickRewriteDraftProvider' import type { BookProgressLocation } from './views/bookProgress/navigation' @@ -55,6 +56,8 @@ function StorageConflictNotice({ onReload, writeback }: { onReload: () => Promis export default function App() { const { data, + recovery, + recoverLibrary, storagePath, setStoragePath, status, @@ -81,6 +84,17 @@ export default function App() { const [bookLocation, setBookLocation] = useState(null) const [readerInitialChapterId, setReaderInitialChapterId] = useState(null) + const needsRecovery = recovery !== null + useEffect(() => { + if (!needsRecovery) return + setCurrentProjectId(null) + setView('dashboard') + setRevisionPrefill(null) + setPipelineSnapshotId(null) + setBookLocation(null) + setReaderInitialChapterId(null) + }, [needsRecovery]) + async function reloadAfterStorageConflict(): Promise<{ ok: boolean; errorMessage?: string }> { const confirmed = await confirmAction({ title: '重新加载最新数据', @@ -93,8 +107,9 @@ export default function App() { } if (!data) { + if (recovery) return return ( -
+
Novel Director 正在读取本地数据...
diff --git a/src/renderer/src/hooks/useAppData.ts b/src/renderer/src/hooks/useAppData.ts index 47437a8..40992fe 100644 --- a/src/renderer/src/hooks/useAppData.ts +++ b/src/renderer/src/hooks/useAppData.ts @@ -14,6 +14,7 @@ import { type SaveDataOutcome } from '../utils/saveDataState' import { getNovelDirectorDataApi } from '../platform/novelDirectorBridge' +import { useLibraryOpen } from './useLibraryOpen' export type { SaveDataHandler, SaveDataInput, SaveDataOutcome } @@ -29,39 +30,24 @@ export function useAppData() { const saveQueueRef = useRef(null) const latestDataRef = useRef(EMPTY_APP_DATA) - useEffect(() => { - let dataApi: ReturnType - try { - dataApi = getNovelDirectorDataApi() - } catch (error) { - latestDataRef.current = EMPTY_APP_DATA - setData(EMPTY_APP_DATA) - setStatus(`读取数据失败:${getUserFriendlyError(error)}`) - return - } - - dataApi - .load() - .then((result) => { - latestDataRef.current = result.data - setData(result.data) - setStoragePath(result.storagePath) - setHasStorageConflict(false) - if (result.credentialWarning) setStatus(result.credentialWarning) - }) - .catch((error) => { - latestDataRef.current = EMPTY_APP_DATA - setData(EMPTY_APP_DATA) - setStatus(`读取数据失败:${getUserFriendlyError(error)}`) - }) - }, []) + const { recovery, recoverLibrary, blockLibrary, assertLibraryAccess } = useLibraryOpen((loaded) => { + commitPersistedData(loaded.data) + setStoragePath(loaded.storagePath) + setNeedsWritebackCheck(false) + setStatus(loaded.credentialWarning || '') + }, () => setData(null)) useEffect(() => { if (!data) return applyTheme(data.settings.theme) }, [data]) - function enqueueStorageWrite(operation: () => Promise): Promise { + async function enqueueStorageWrite(write: () => Promise): Promise { + assertLibraryAccess('write') + const operation = () => { + assertLibraryAccess('write') + return write() + } if (!saveQueueRef.current) { saveQueueRef.current = createOperationQueue() } @@ -78,6 +64,11 @@ export function useAppData() { setHasStorageConflict(false) } + function handleLoadFailure(error: unknown): never { + blockLibrary(error) + throw error + } + const saveData: SaveDataHandler = async (nextInput): Promise => { setStatus('保存中...') try { @@ -184,7 +175,7 @@ export function useAppData() { const runPersistedStorageOperation: RunPersistedStorageOperation = async (operation, options = {}) => { try { const result = await enqueueStorageWrite(async () => { - const current = latestDataRef.current + const current = getCurrentData() if (options.persistCurrentFirst) { await getNovelDirectorDataApi().save(current) } @@ -210,7 +201,7 @@ export function useAppData() { if (api.executeCandidateDecision) { const saved = await api.executeCandidateDecision(command) // A replay may follow an external write. Reload instead of applying an old response patch. - if (saved.replayed) commitPersistedData((await api.load()).data) + if (saved.replayed) commitPersistedData((await api.load().catch(handleLoadFailure)).data) else commitPersistedData(applyCandidateDecisionChanges(latestDataRef.current, saved.changes, saved.removedRecords ?? [])) return saved } @@ -256,7 +247,7 @@ export function useAppData() { setStatus('正在重新加载最新本地数据...') try { const result = await enqueueStorageWrite(async () => { - const loaded = await getNovelDirectorDataApi().load() + const loaded = await getNovelDirectorDataApi().load().catch(handleLoadFailure) commitPersistedData(loaded.data) return { ok: true as const, @@ -278,6 +269,8 @@ export function useAppData() { return { data, + recovery, + recoverLibrary, storagePath, setStoragePath, status, @@ -292,7 +285,10 @@ export function useAppData() { runPersistedStorageOperation, importData, reloadData, - getCurrentData + getCurrentData: () => { + assertLibraryAccess('read') + return getCurrentData() + } } } diff --git a/src/renderer/src/hooks/useLibraryOpen.ts b/src/renderer/src/hooks/useLibraryOpen.ts new file mode 100644 index 0000000..58a31b4 --- /dev/null +++ b/src/renderer/src/hooks/useLibraryOpen.ts @@ -0,0 +1,80 @@ +import { useEffect, useRef, useState } from 'react' +import { getUserFriendlyError } from '../../../shared/errorUtils' +import type { LibraryRecoveryAction, StorageGetResult } from '../../../shared/ipc/ipcTypes' +import { getNovelDirectorBridge, getNovelDirectorDataApi } from '../platform/novelDirectorBridge' + +export interface LibraryRecoveryState { + error: string + pending: LibraryRecoveryAction | 'loading' | null + feedback: string +} + +type OpenSession = { phase: 'initial' | 'unavailable' | 'recovering' | 'ready' } + +export function useLibraryOpen(onLoaded: (result: StorageGetResult) => void, onUnavailable: () => void) { + const [recovery, setRecovery] = useState(null) + const mounted = useRef(false) + const current = useRef({ phase: 'initial' }) + const session = current.current + const isCurrent = () => mounted.current && current.current === session + + function blockLibrary(error: unknown) { + if (!isCurrent()) return + // Invalidate callbacks and queued writes captured for the previous library. + current.current = { phase: 'unavailable' } + onUnavailable() + setRecovery({ error: getUserFriendlyError(error), pending: null, feedback: '' }) + } + + async function openLibrary() { + const loaded = await getNovelDirectorDataApi().load() + if (!isCurrent()) return + onLoaded(loaded) + session.phase = 'ready' + setRecovery(null) + } + + useEffect(() => { + mounted.current = true + let active = true + // StrictMode can restart this effect before the first IPC read should begin. + queueMicrotask(() => { + if (active) void openLibrary().catch(blockLibrary) + }) + return () => { + active = false + mounted.current = false + } + }, []) + + async function recoverLibrary(action: LibraryRecoveryAction) { + if (!recovery || !isCurrent() || session.phase !== 'unavailable') return + session.phase = 'recovering' + setRecovery({ ...recovery, pending: action, feedback: '' }) + let feedback = '已取消。' + try { + const app = getNovelDirectorBridge().app + if (typeof app?.recoverLibrary !== 'function') throw new Error('书库恢复功能未加载,请重新启动应用。') + const result = await app.recoverLibrary(action) + if (isCurrent() && !result.cancelled) { + setRecovery({ ...recovery, pending: 'loading', feedback: '' }) + await openLibrary() + } + } catch (error) { + feedback = `本次操作失败:${getUserFriendlyError(error)}` + } finally { + if (isCurrent() && session.phase === 'recovering') { + session.phase = 'unavailable' + setRecovery({ ...recovery, pending: null, feedback }) + } + } + } + + function assertLibraryAccess(mode: 'read' | 'write') { + // The synchronous getter can be polled during the initial load, but not after a failed open. + if (isCurrent() && (session.phase === 'ready' || (mode === 'read' && session.phase === 'initial'))) return + throw new Error('书库未打开。') + } + + return { recovery, recoverLibrary, blockLibrary, assertLibraryAccess } +} diff --git a/src/renderer/src/styles/app-shell.css b/src/renderer/src/styles/app-shell.css index 915db8e..53fa32f 100644 --- a/src/renderer/src/styles/app-shell.css +++ b/src/renderer/src/styles/app-shell.css @@ -39,6 +39,66 @@ code { animation: nd-rise-in var(--duration-slow) var(--ease-out) backwards; } +.library-recovery { + display: grid; + align-content: center; + gap: var(--space-4); + max-width: 720px; + min-height: 100dvh; + margin: 0 auto; + padding: var(--space-8) var(--space-5); +} + +.library-recovery h1, +.library-recovery p, +.library-recovery ul { + margin: 0; +} + +.library-recovery h1 { + font-size: var(--text-3xl); + letter-spacing: 0; +} + +.library-recovery-error, +.library-recovery-feedback { + overflow-wrap: anywhere; + white-space: pre-wrap; +} + +.library-recovery-error { + border-inline-start: 3px solid var(--color-warning); + padding: var(--space-3) var(--space-4); + background: var(--color-warning-soft); +} + +.library-recovery ul { + padding: 0; + list-style: none; +} + +.library-recovery li { + display: grid; + grid-template-columns: 180px minmax(0, 1fr); + align-items: center; + gap: var(--space-4); + padding: var(--space-3) 0; + border-bottom: 1px solid var(--line); + color: var(--muted); +} + +.library-recovery-feedback { + min-height: 1.5em; +} + +@media (max-width: 540px) { + .library-recovery li { + grid-template-columns: 1fr; + justify-items: start; + gap: var(--space-2); + } +} + .home { min-height: 100dvh; padding: 40px 48px 64px; diff --git a/src/renderer/src/views/LibraryRecoveryView.tsx b/src/renderer/src/views/LibraryRecoveryView.tsx new file mode 100644 index 0000000..f62a4dd --- /dev/null +++ b/src/renderer/src/views/LibraryRecoveryView.tsx @@ -0,0 +1,35 @@ +import { ArchiveRestore, FilePlus2, FolderOpen, RotateCw } from 'lucide-react' +import type { LibraryRecoveryAction } from '../../../shared/ipc/ipcTypes' +import type { LibraryRecoveryState } from '../hooks/useLibraryOpen' + +const actions = [ + { action: 'retry', label: '重试', icon: RotateCw, hint: '重新打开原书库。如果书库在外接磁盘上,请先连接磁盘。', pending: '正在重试...' }, + { action: 'choose', label: '选择已有书库', icon: FolderOpen, hint: '可选 SQLite 或 JSON。选择 JSON 后,更改将写入所选文件,不会合并到原书库。', pending: '等待选择书库文件...' }, + { action: 'restore', label: '从备份恢复到新文件', icon: ArchiveRestore, hint: '选择备份和新的保存位置,保留原书库。', pending: '正在处理备份恢复...' }, + { action: 'create', label: '新建书库', icon: FilePlus2, hint: '选择新文件的位置并确认后,才会创建空书库。', pending: '等待选择位置并确认新建...' } +] as const + +export function LibraryRecoveryView({ recovery, onRecover }: { + recovery: LibraryRecoveryState + onRecover: (action: LibraryRecoveryAction) => Promise +}) { + const pending = recovery.pending === 'loading' ? '正在读取书库...' : actions.find((item) => item.action === recovery.pending)?.pending + return ( +
+

书库未打开

+

{recovery.error}

+
    + {actions.map(({ action, label, icon: Icon, hint }) => ( +
  • + + {hint} +
  • + ))} +
+

{pending ?? recovery.feedback}

+
+ ) +} diff --git a/src/renderer/tests/library-recovery-window.mjs b/src/renderer/tests/library-recovery-window.mjs new file mode 100644 index 0000000..8429a88 --- /dev/null +++ b/src/renderer/tests/library-recovery-window.mjs @@ -0,0 +1,154 @@ +import assert from 'node:assert/strict' +import { join } from 'node:path' +import { setTimeout as delay } from 'node:timers/promises' +import { app, BrowserWindow, ipcMain } from 'electron' +import { normalizeAppData } from '../../shared/defaults' +import { IPC_CHANNELS as channels } from '../../shared/ipc/ipcChannels' +import { StorageRevisionConflictError } from '../../shared/storageRevisionConflict' + +const labels = { retry: '重试', choose: '选择已有书库', restore: '从备份恢复到新文件', create: '新建书库' } +const original = '无法打开书库:Z:\\离线磁盘\\' + '长路径'.repeat(20) + '\\原库.sqlite\n磁盘未连接。' +const failure = (error) => ({ ok: false, code: 'LIBRARY_UNAVAILABLE', error }) +const loads = [], actions = [], saves = [], unexpected = [] +for (const channel of Object.values(channels)) { + ipcMain.handle(channel, (_event, ...args) => { + const records = channel === channels.STORAGE_GET ? loads : channel === channels.APP_RECOVER_LIBRARY ? actions + : channel === channels.STORAGE_SAVE ? saves : null + if (records) return new Promise((resolve) => records.push({ args, resolve })) + unexpected.push(channel) + return failure('测试不允许此操作。') + }) +} + +async function until(read, description) { + const deadline = Date.now() + 5000 + while (Date.now() < deadline) { + const result = await read() + if (result) return result + await delay(15) + } + throw new Error('Timed out: ' + description) +} + +async function next(records, trigger) { + const before = records.length + await trigger() + return until(() => records[before], 'IPC request') +} + +async function verify() { + let expectedError = original + const win = new BrowserWindow({ show: false, width: 390, height: 800, useContentSize: true, + webPreferences: { preload: join(__dirname, 'preload.cjs'), sandbox: true, contextIsolation: true, backgroundThrottling: false } }) + const run = (code) => win.webContents.executeJavaScript(code) + const restart = () => next(loads, () => win.loadFile(join(__dirname, 'index.html'))) + const press = (...names) => run(`for (const name of ${JSON.stringify(names)}) { + [...document.querySelectorAll('button')].find(b => b.textContent.trim() === name).click() + }`) + const snapshot = () => run(`({ + error: document.querySelector('.library-recovery-error')?.textContent, + feedback: document.querySelector('.library-recovery-feedback')?.textContent, + buttons: [...document.querySelectorAll('button')].map(b => [b.textContent.trim(), b.disabled]), + editor: Boolean(document.querySelector('.home, .app-shell, form, input, textarea, [contenteditable="true"]')), + overflow: document.documentElement.scrollWidth > document.documentElement.clientWidth + })`) + async function recoveryPage(busy, feedback) { + const state = await until(async () => { + const s = await snapshot() + return s.error && s.feedback.includes(feedback) && s.buttons.every(([, disabled]) => disabled === busy) && s + }, 'recovery page: ' + feedback) + assert.equal(state.error, expectedError) + assert.deepEqual(state.buttons.map(([name]) => name), Object.values(labels)) + assert.equal(state.editor, false) + assert.equal(state.overflow, false) + } + async function attempt(action, ...duplicates) { + const request = await next(actions, () => press(labels[action], ...duplicates.map((key) => labels[key]))) + assert.deepEqual(request.args, [action]) + await recoveryPage(true, '') + return request + } + + const first = await restart() + assert.deepEqual(first.args, []) + assert.equal(await run("Boolean(document.querySelector('.loading-screen'))"), true) + assert.deepEqual((await snapshot()).buttons, []) + first.resolve(failure(original)) + await recoveryPage(false, '') + assert.equal(loads.length, 1, 'StrictMode starts just one read') + assert.equal(await run("window.novelDirector.app.recoverLibrary({ path: 'fixture.json' }).then(() => false, () => true)"), true) + + ;(await attempt('choose', 'choose', 'create')).resolve({ ok: true, cancelled: true }) + await recoveryPage(false, '已取消') + ;(await attempt('restore')).resolve(failure('备份无法读取。')) + await recoveryPage(false, '备份无法读取。') + ;(await attempt('create')).resolve({ ok: true, cancelled: true }) + await recoveryPage(false, '已取消') + assert.equal(loads.length, 1, 'cancelled and rejected actions never reload') + assert.deepEqual(actions.map((call) => call.args), [['choose'], ['restore'], ['create']]) + + const retry = await attempt('retry') + const reread = await next(loads, () => retry.resolve({ ok: true })) + await recoveryPage(true, '正在读取书库') + reread.resolve(failure('无法打开书库:Z:\\测试\\另一文件.sqlite\n文件无法读取。')) + await recoveryPage(false, '另一文件.sqlite') + assert.equal(saves.length, 0) + assert.deepEqual(unexpected, [], 'no second path lookup or other side effects') + + const choose = await attempt('choose') + const opened = await next(loads, () => choose.resolve({ ok: true })) + await recoveryPage(true, '正在读取书库') + const book = { data: normalizeAppData({ projects: [{ id: 'fixture', name: '测试书稿' }], + settings: { hasApiKey: true } }), storagePath: 'Z:\\测试\\选择的书库.json', revision: 'selected-revision' } + opened.resolve(book) + await until(() => run("document.querySelector('.home')?.textContent.includes('测试书稿')"), 'recovered project') + const save = await next(saves, () => press('进入')) + assert.equal(save.args[0].expectedRevision, book.revision) + assert.equal(save.args[0].data.settings.hasApiKey, true) + const conflict = new StorageRevisionConflictError() + save.resolve({ ok: false, code: conflict.code, error: conflict.message }) + await until(() => run("Boolean(document.querySelector('.storage-conflict-notice'))"), 'revision conflict') + await press('重新加载最新数据') + await until(() => run("Boolean(document.querySelector('[role=dialog]'))"), 'confirm reload') + const reload = await next(loads, () => press('重新加载')) + await until(() => run("!document.querySelector('[role=dialog]')"), 'confirmation closed') + await press('进入') + await until(() => run("document.querySelector('.home')?.textContent.includes('正在打开...')"), 'queued save') + expectedError = '无法打开书库:' + book.storagePath + '\n磁盘未连接。' + reload.resolve(failure(expectedError)) + await recoveryPage(false, '') + assert.equal(saves.length, 1, 'a write queued behind the failed reload never reaches IPC') + + // A new renderer must perform its own read, even if an earlier recovery dialog completes late. + const abandoned = await attempt('restore') + const afterRestart = await restart() + abandoned.resolve({ ok: true }) + afterRestart.resolve(failure(expectedError)) + await recoveryPage(false, '') + const create = await attempt('create') + const created = await next(loads, () => create.resolve({ ok: true })) + await recoveryPage(true, '正在读取书库') + created.resolve({ data: normalizeAppData({}), storagePath: 'Z:\\测试\\新书库.sqlite', revision: 'empty-revision' }) + await until(() => run("document.querySelector('.home')?.textContent.includes('还没有小说项目')"), 'genuinely empty library') + ;(await restart()).resolve(book) + await until(() => run("document.querySelector('.home')?.textContent.includes('测试书稿')"), 'ordinary startup') + assert.equal(loads.length, 7, 'one fresh snapshot per open, none from the abandoned dialog') + + for (const action of ['retry', 'restore']) { + ;(await restart()).resolve(failure(expectedError)) + await recoveryPage(false, '') + const request = await attempt(action) + const read = await next(loads, () => request.resolve({ ok: true })) + const title = action === 'retry' ? '原盘书稿' : '备份书稿' + read.resolve({ ...book, data: normalizeAppData({ projects: [{ id: 'again', name: title }] }), + storagePath: action === 'retry' ? book.storagePath : 'Z:\\测试\\备份副本.sqlite', revision: action + '-revision' }) + await until(() => run(`document.querySelector('.home')?.textContent.includes(${JSON.stringify(title)})`), title) + } + assert.equal(loads.length, 11) + assert.equal(saves.length, 1) + assert.deepEqual(unexpected, []) + console.log('PASS library recovery: loading, failure, four actions, cancellation, double click, reread, revision, credentials and restart') +} + +app.disableHardwareAcceleration() +app.whenReady().then(verify).then(() => app.exit(0), (error) => { console.error(error); app.exit(1) }) diff --git a/src/renderer/tests/validate-library-recovery.mjs b/src/renderer/tests/validate-library-recovery.mjs new file mode 100644 index 0000000..6305c1e --- /dev/null +++ b/src/renderer/tests/validate-library-recovery.mjs @@ -0,0 +1,34 @@ +import { execFile } from 'node:child_process' +import assert from 'node:assert/strict' +import { mkdir, mkdtemp, rm, writeFile } from 'node:fs/promises' +import { createRequire } from 'node:module' +import { dirname, join, resolve } from 'node:path' +import { promisify } from 'node:util' +import { build } from 'esbuild' +import { repoRoot } from '../../../scripts/utils/repo-root.mjs' + +const parent = resolve(repoRoot, 'tmp') +await mkdir(parent, { recursive: true }) +const dir = await mkdtemp(join(parent, 'library-recovery-ui-')) +try { + const common = { absWorkingDir: repoRoot, bundle: true, logLevel: 'silent' } + await build({ ...common, entryPoints: ['src/renderer/src/main.tsx'], outfile: join(dir, 'ui.js'), + jsx: 'automatic', loader: { '.png': 'dataurl' }, define: { 'process.env.NODE_ENV': '"development"' } }) + for (const [entry, name] of [['src/preload/index.ts', 'preload'], ['src/renderer/tests/library-recovery-window.mjs', 'verify']]) { + await build({ ...common, entryPoints: [entry], outfile: join(dir, name + '.cjs'), + platform: 'node', format: 'cjs', external: ['electron'] }) + } + await writeFile(join(dir, 'index.html'), ` + +
`) + const env = { ...process.env } + delete env.ELECTRON_RUN_AS_NODE + const { stdout } = await promisify(execFile)(createRequire(import.meta.url)('electron'), + [join(dir, 'verify.cjs'), '--user-data-dir=' + join(dir, 'profile'), '--disable-background-networking'], + { env, windowsHide: true, timeout: 60_000 }) + assert.match(stdout, /PASS library recovery/) + console.log(stdout.trim()) +} finally { + assert.equal(dirname(resolve(dir)), parent) + await rm(dir, { recursive: true, force: true, maxRetries: 4, retryDelay: 100 }) +} diff --git a/src/shared/ipc/ipcChannels.ts b/src/shared/ipc/ipcChannels.ts index 9eb6ffc..e6953b2 100644 --- a/src/shared/ipc/ipcChannels.ts +++ b/src/shared/ipc/ipcChannels.ts @@ -13,6 +13,7 @@ export const IPC_CHANNELS = { STORAGE_IMPORT: 'storage:import', APP_GET_STORAGE_PATH: 'app:get-storage-path', + APP_RECOVER_LIBRARY: 'app:recover-library', APP_GET_RUNTIME_INFO: 'app:get-runtime-info', APP_SELECT_STORAGE_PATH: 'app:select-storage-path', APP_MIGRATE_STORAGE_PATH: 'app:migrate-storage-path', diff --git a/src/shared/ipc/ipcTypes.ts b/src/shared/ipc/ipcTypes.ts index 35d0439..ed0fb59 100644 --- a/src/shared/ipc/ipcTypes.ts +++ b/src/shared/ipc/ipcTypes.ts @@ -144,6 +144,13 @@ export interface GetStoragePathResult { defaultStoragePath: string } +export type LibraryRecoveryAction = 'retry' | 'choose' | 'restore' | 'create' + +export interface RecoverLibraryResult { + ok: true + cancelled?: boolean +} + export type GetRuntimeInfoResult = RuntimeInfo export interface SelectStoragePathResult { diff --git a/src/storage/JsonStorageService.ts b/src/storage/JsonStorageService.ts index 62c84cf..b421a29 100644 --- a/src/storage/JsonStorageService.ts +++ b/src/storage/JsonStorageService.ts @@ -1,4 +1,5 @@ import { createHash } from 'node:crypto' +import { accessSync, closeSync, constants, openSync, readFileSync, writeFileSync } from 'node:fs' import { copyFile, mkdir, readFile, rename, stat, writeFile } from 'node:fs/promises' import { dirname } from 'node:path' import type { AppData, CandidateDecisionCommand, CandidateDecisionReceipt, CharacterCardReviewCommand, ChapterCommitBundle, ChapterWritebackFailure, GenerationRunBundle, RevisionCommitBundle } from '../shared/types' @@ -11,6 +12,8 @@ import { applyChapterCommitBundleToAppData } from '../services/ChapterCommitBund import { applyGenerationRunBundleToAppData } from '../services/GenerationRunBundleService' import { applyRevisionCommitBundleToAppData } from '../services/RevisionCommitBundleService' import { withJsonWriteLock } from './jsonWriteLock' +import { assertReservedLibraryFile } from './libraryFile' +import { decodeLibraryText, normalizeLibraryData, parseLibraryJsonValue } from './libraryValidation' import { jsonTitleVersionIds, readJsonChapterTitles } from './chapterTitleState' import { applyTitleReceiptToChapters } from '../shared/chapterTitleHistory' import { decideChapterTitleCommit } from '../services/ChapterTitleRenameService' @@ -23,10 +26,16 @@ import { type StorageSnapshot } from './StorageService' -function revisionForRaw(raw: string): string { +function revisionForRaw(raw: string | Buffer): string { return createHash('sha256').update(raw).digest('hex') } +function checkedJsonSnapshot(bytes: Buffer, initial?: StorageSnapshot): StorageSnapshot { + const revision = revisionForRaw(bytes) + if (initial?.revision === revision) return initial + return { data: normalizeLibraryData(parseLibraryJsonValue(decodeLibraryText(bytes))), revision } +} + function redactDecisionValue(value: unknown, secrets: string[], field = ''): unknown { if (typeof value === 'string') { const redacted = value && /^(?:api[_-]?key|x-api-key|authorization|access[_-]?token|token)$/i.test(field) @@ -89,7 +98,24 @@ export function applyCandidateDecisionForStorage(data: AppData, command: Candida } export class JsonStorageService implements StorageService { - constructor(private readonly storagePath: string) {} + private readonly strict: boolean + private initial?: StorageSnapshot + + constructor(private readonly storagePath: string, mode: 'legacy' | 'existing' | 'new' = 'legacy', initialData: AppData = EMPTY_APP_DATA) { + this.strict = mode !== 'legacy' + if (mode === 'existing') { + this.initial = checkedJsonSnapshot(readFileSync(storagePath)) + closeSync(openSync(storagePath, 'r+')) + accessSync(dirname(storagePath), constants.W_OK) + } else if (mode === 'new') { + const fd = openSync(storagePath, 'r+') + try { + assertReservedLibraryFile(storagePath, fd) + const data = normalizeLibraryData(initialData, sanitizeAppDataForPersistence) + writeFileSync(fd, JSON.stringify(data, null, 2), 'utf8') + } finally { closeSync(fd) } + } + } getStoragePath(): string { return this.storagePath @@ -100,6 +126,11 @@ export class JsonStorageService implements StorageService { } async loadSnapshot(): Promise { + if (this.strict) { + const bytes = await readFile(this.storagePath), initial = this.initial + this.initial = undefined + return checkedJsonSnapshot(bytes, initial) + } let raw = '' try { raw = await readFile(this.storagePath, 'utf-8') @@ -137,16 +168,29 @@ export class JsonStorageService implements StorageService { } async backupTo(targetPath: string): Promise { + if (this.strict) { + const raw = await readFile(this.storagePath) + checkedJsonSnapshot(raw, this.initial) + await mkdir(dirname(targetPath), { recursive: true }) + await writeFile(targetPath, raw) + return + } await mkdir(dirname(targetPath), { recursive: true }) await copyFile(this.storagePath, targetPath) } async saveIfCurrent(data: AppData, expectedRevision?: string, onWriteStarted?: () => void): Promise { - return withJsonWriteLock(this.storagePath, () => this.writeIfCurrent(data, expectedRevision, onWriteStarted)) + return this.withWriteLock(revision => this.writeIfCurrent(data, expectedRevision, onWriteStarted, revision)) + } + + private withWriteLock(write: (validatedRevision?: string) => Promise): Promise { + // A file replaced or removed after open must not be recreated, even by an unconditional save. + return withJsonWriteLock(this.storagePath, async () => + write(this.strict ? (await this.loadSnapshot()).revision : undefined), !this.strict) } - private async writeIfCurrent(data: AppData, expectedRevision?: string, onWriteStarted?: () => void): Promise { - const currentRevision = await this.readCurrentRevision() + private async writeIfCurrent(data: AppData, expectedRevision?: string, onWriteStarted?: () => void, validatedRevision?: string): Promise { + const currentRevision = validatedRevision ?? await this.readCurrentRevision() if (expectedRevision !== undefined && currentRevision !== expectedRevision) { throw new StorageRevisionConflictError() } @@ -167,19 +211,20 @@ export class JsonStorageService implements StorageService { return revisionForRaw(await readFile(this.storagePath, 'utf-8')) } catch (error) { const code = typeof error === 'object' && error && 'code' in error ? String(error.code) : '' - if (code === 'ENOENT') return 'missing' + if (code === 'ENOENT' && !this.strict) return 'missing' throw error } } private async writeData(data: AppData, onWriteStarted?: () => void): Promise { - await this.replaceDocument(sanitizeAppDataForPersistence(data), onWriteStarted) + await this.replaceDocument(this.strict ? normalizeLibraryData(data, sanitizeAppDataForPersistence) + : sanitizeAppDataForPersistence(data), onWriteStarted) } private async replaceDocument(data: Partial, onWriteStarted?: () => void): Promise { const raw = JSON.stringify(data, null, 2) onWriteStarted?.() - await mkdir(dirname(this.storagePath), { recursive: true }) + if (!this.strict) await mkdir(dirname(this.storagePath), { recursive: true }) const tmpPath = `${this.storagePath}.tmp` const backupPath = `${this.storagePath}.bak` @@ -190,7 +235,7 @@ export class JsonStorageService implements StorageService { await copyFile(this.storagePath, backupPath) } catch (error) { const code = typeof error === 'object' && error && 'code' in error ? String(error.code) : '' - if (code !== 'ENOENT') { + if (this.strict || code !== 'ENOENT') { throw error } } @@ -298,7 +343,7 @@ export class JsonStorageService implements StorageService { } async recordChapterWritebackFailureIfUncommitted(failure: ChapterWritebackFailure): Promise { - await withJsonWriteLock(this.storagePath, async () => { + await this.withWriteLock(async () => { // A missing/corrupt file is not evidence of absence. Do not initialize it or save a fallback snapshot. const data = JSON.parse(await readFile(this.storagePath, 'utf-8')) as Partial const safe = normalizeChapterWritebackFailure(failure, data.settings?.apiKey) @@ -329,7 +374,7 @@ export class JsonStorageService implements StorageService { } async reviewCharacterCard(command: CharacterCardReviewCommand, expectedRevision: string): Promise { - return withJsonWriteLock(this.storagePath, async () => { + return this.withWriteLock(async () => { const raw = await readFile(this.storagePath, 'utf-8') if (!expectedRevision || revisionForRaw(raw) !== expectedRevision) throw new StorageRevisionConflictError() const original = JSON.parse(raw) as AppData @@ -344,7 +389,7 @@ export class JsonStorageService implements StorageService { } async renameChapterTitles(commit: ChapterTitleCommit, actor: ChapterTitleActor): Promise { - return withJsonWriteLock(this.storagePath, async () => { + return this.withWriteLock(async () => { const { data, state, revision } = await readJsonChapterTitles(this.storagePath, commit.command.projectId) const { receipt, replayed } = decideChapterTitleCommit(state, commit, this.storagePath, revision, actor, () => jsonTitleVersionIds(data, state.projectId)) diff --git a/src/storage/SqliteStorageService.ts b/src/storage/SqliteStorageService.ts index b28fd02..1d9ba32 100644 --- a/src/storage/SqliteStorageService.ts +++ b/src/storage/SqliteStorageService.ts @@ -1,7 +1,7 @@ import { randomUUID } from 'node:crypto' import { mkdir } from 'node:fs/promises' import { createRequire } from 'node:module' -import { dirname } from 'node:path' +import { dirname, extname, resolve } from 'node:path' import type { AppData, CandidateDecisionCommand, Character, CharacterCardReviewCommand, ChapterCommitBundle, ChapterWritebackFailure, GenerationRunBundle, RevisionCommitBundle } from '../shared/types' import type { CandidateDecisionWriteResult, CharacterCardReviewWriteResult, StorageWriteResult } from '../shared/ipc/ipcTypes' import { EMPTY_APP_DATA, normalizeAppData, sanitizeAppDataForPersistence } from '../shared/defaults' @@ -11,6 +11,7 @@ import { applyChapterCommitBundleToAppData } from '../services/ChapterCommitBund import { validateGenerationRunBundle, type GenerationRunBundleValidationContext } from '../services/GenerationRunBundleService' import { applyRevisionCommitBundleToAppData } from '../services/RevisionCommitBundleService' import { JsonStorageService } from './JsonStorageService' +import { normalizeLibraryData } from './libraryValidation' import { readSqliteChapterTitles, readJsonChapterTitles, sqliteTitleVersionIds } from './chapterTitleState' import { decideChapterTitleCommit } from '../services/ChapterTitleRenameService' import type { ChapterTitleActor, ChapterTitleCommit } from '../shared/types/chapterTitle' @@ -31,7 +32,7 @@ import { arrayCollectionKeys, entityInsertValues } from './sqlite/sqliteEntityMapper' -import { currentRevisionFromDatabase, readAppDataFromDatabase, readAppDataCollectionsFromDatabase } from './sqlite/sqliteSnapshot' +import { currentRevisionFromDatabase, readAppDataFromDatabase, readAppDataCollectionsFromDatabase, readLibrarySnapshot } from './sqlite/sqliteSnapshot' import { writeSqliteSnapshotEntities } from './sqlite/sqliteSnapshotWriter' import { readGenerationBundleValidationContext } from './sqlite/sqliteGenerationContext' import { executeSqliteCandidateDecision } from './sqlite/sqliteCandidateDecisions' @@ -61,6 +62,8 @@ function loadBetterSqlite3(): SqliteDatabaseConstructor { function nowIso(): string { return new Date().toISOString() } +export function checkSqliteBindingAvailable(): void { loadBetterSqlite3() } + /** * Agent read tools must not initialize schemas, change WAL mode, or create files. * This path opens an existing SQLite database in native read-only mode. @@ -114,13 +117,15 @@ export async function loadSqliteCollectionsReadonly( export class SqliteStorageService implements StorageService { private readonly initialization: SqliteInitialization private readonly storagePath: string + private readonly strict: boolean - constructor(storagePath: string, options: { legacyJsonPath?: string } = {}) { + constructor(storagePath: string, options: { legacyJsonPath?: string; mode?: 'existing' | 'new'; initialData?: AppData } = {}) { const Database = loadBetterSqlite3() - this.storagePath = resolveSqliteStoragePath(storagePath) + this.strict = options.mode !== undefined + this.storagePath = this.strict ? resolve(storagePath) : resolveSqliteStoragePath(storagePath) this.initialization = new SqliteInitialization( Database, this.storagePath, options.legacyJsonPath ?? resolveJsonStoragePath(storagePath), - (db, data) => this.saveSnapshot(db, data) + (db, data) => this.saveSnapshot(db, data), options.mode, options.initialData ) } @@ -135,10 +140,11 @@ export class SqliteStorageService implements StorageService { async loadSnapshot(): Promise { const db = await this.open() - const readTransaction = db.transaction(() => ({ - data: this.readAppData(db), - revision: this.currentRevision(db) - })) + const readTransaction = db.transaction(() => { + const revision = this.currentRevision(db) + if (this.strict) return this.initialization.takeInitialSnapshot(db, revision) ?? readLibrarySnapshot(db) + return { data: this.readAppData(db), revision } + }) return readTransaction() } @@ -159,7 +165,7 @@ export class SqliteStorageService implements StorageService { } private saveSnapshot(db: SqliteDatabase, data: AppData, expectedRevision?: string): StorageWriteResult { - const normalized = sanitizeAppDataForPersistence(data) + const normalized = this.strict ? normalizeLibraryData(data, sanitizeAppDataForPersistence) : sanitizeAppDataForPersistence(data) const collections = arrayCollectionKeys() const updatedAt = nowIso() const revision = `${updatedAt}-${randomUUID()}` @@ -371,10 +377,30 @@ export class SqliteStorageService implements StorageService { } private readAppData(db: SqliteDatabase): AppData { - return readAppDataFromDatabase(db) + return this.strict ? readLibrarySnapshot(db).data : readAppDataFromDatabase(db) + } +} + +function libraryStorage(filePath: string, mode: 'existing' | 'new', initialData?: AppData): StorageService { + const path = resolve(filePath) + switch (extname(path).toLowerCase()) { + case '.json': return new JsonStorageService(path, mode, initialData) + case '.sqlite': + case '.db': return new SqliteStorageService(path, { mode, initialData }) + default: throw new Error('书库文件扩展名必须是 .sqlite、.db 或 .json。') } } +export function createExistingLibraryStorage(filePath: string): StorageService { + return libraryStorage(filePath, 'existing') +} + +/** The caller owns an exclusively reserved (wx), zero-byte file at this exact path. */ +export function createNewLibraryStorage(filePath: string, initialData?: AppData): StorageService { + return libraryStorage(filePath, 'new', initialData) +} + +/** Legacy Agent/migration compatibility. Desktop library selection must use the explicit factories above. */ export function createStorageService(storagePath: string): StorageService { try { return new SqliteStorageService(storagePath) diff --git a/src/storage/jsonReadonly.ts b/src/storage/jsonReadonly.ts index 419a12d..fb7ba9c 100644 --- a/src/storage/jsonReadonly.ts +++ b/src/storage/jsonReadonly.ts @@ -3,10 +3,11 @@ import { readFile } from 'node:fs/promises' import { normalizeAppData } from '../shared/defaults' import type { AppData } from '../shared/types' import type { StorageSnapshot } from './StorageService' +import { decodeLibraryText } from './libraryValidation' /** Queries must not initialize a missing book or run the legacy corruption-recovery writer. */ export async function loadJsonSnapshotReadonly(path: string): Promise { - const raw = await readFile(path, 'utf8') - return { data: normalizeAppData(JSON.parse(raw) as Partial), - revision: createHash('sha256').update(raw).digest('hex') } + const bytes = await readFile(path) + return { data: normalizeAppData(JSON.parse(decodeLibraryText(bytes)) as Partial), + revision: createHash('sha256').update(bytes).digest('hex') } } diff --git a/src/storage/jsonWriteLock.ts b/src/storage/jsonWriteLock.ts index 0c55628..9400e8d 100644 --- a/src/storage/jsonWriteLock.ts +++ b/src/storage/jsonWriteLock.ts @@ -3,8 +3,8 @@ import { dirname } from 'node:path' // Covers every JSON writer, including separate desktop and Agent processes. // We never steal an existing lock: a timed-out writer may still be committing. -export async function withJsonWriteLock(storagePath: string, operation: () => Promise): Promise { - await mkdir(dirname(storagePath), { recursive: true }) +export async function withJsonWriteLock(storagePath: string, operation: () => Promise, createParent = true): Promise { + if (createParent) await mkdir(dirname(storagePath), { recursive: true }) const lockPath = `${storagePath}.write-lock` const deadline = Date.now() + 5_000 let handle: Awaited> diff --git a/src/storage/libraryFile.ts b/src/storage/libraryFile.ts new file mode 100644 index 0000000..7c1b228 --- /dev/null +++ b/src/storage/libraryFile.ts @@ -0,0 +1,10 @@ +import { fstatSync, lstatSync, type Stats } from 'node:fs' + +export function assertReservedLibraryFile(path: string, fd?: number): void { + const reserved = (s: Stats) => s.isFile() && s.size === 0 && s.nlink === 1 + const pathStat = lstatSync(path) + const stat = fd === undefined ? pathStat : fstatSync(fd) + if (!reserved(pathStat) || !reserved(stat) || pathStat.dev !== stat.dev || pathStat.ino !== stat.ino) { + throw new Error('新建书库需要独占预留的零字节普通文件。') + } +} diff --git a/src/storage/libraryValidation.ts b/src/storage/libraryValidation.ts new file mode 100644 index 0000000..ff312d2 --- /dev/null +++ b/src/storage/libraryValidation.ts @@ -0,0 +1,75 @@ +import { ALL_ENTITY_COLLECTIONS, PROJECT_SCOPED_COLLECTIONS, type AppDataArrayKey } from '../shared/appDataCollections' +import { EMPTY_APP_DATA, normalizeAppData } from '../shared/defaults' +import type { AppData } from '../shared/types' + +function isRecord(value: unknown): value is Record { + return value !== null && typeof value === 'object' && !Array.isArray(value) +} + +function nonemptyString(value: unknown): value is string { + return typeof value === 'string' && !!value.trim() +} + +const projectScoped = new Set(PROJECT_SCOPED_COLLECTIONS) + +export function libraryEntityId(collection: AppDataArrayKey, entity: unknown): string | undefined { + if (isRecord(entity) && (!projectScoped.has(collection) || nonemptyString(entity.projectId))) { + if (nonemptyString(entity.id)) return entity.id + if (collection === 'storyBibles' && nonemptyString(entity.projectId)) return entity.projectId + // The old snapshot writer deliberately keyed id-less prompt versions by collection position. + if (collection === 'promptVersions' && entity.id === undefined) return undefined + } + throw new Error(`书库中的 ${collection} 记录缺少有效标识。`) +} + +export function parseLibraryJsonValue(text: string): unknown { + try { return JSON.parse(text) } catch { throw new Error('书库含有无法解析的 JSON。') } +} + +export function decodeLibraryText(bytes: Uint8Array): string { + // Accept a Windows UTF-8 BOM, but never replace corrupted prose bytes with U+FFFD. + try { return new TextDecoder('utf-8', { fatal: true }).decode(bytes) } + catch { throw new Error('书库 JSON 不是有效的 UTF-8 文本。') } +} + +export function normalizeImportedLibrary(input: unknown, strategy: 'merge' | 'replace'): AppData { + // Older project-only exports can be merged, but cannot stand in for an entire library. + if (strategy === 'merge' && isRecord(input) && + ALL_ENTITY_COLLECTIONS.some(key => Array.isArray(input[key]) && input[key].length > 0)) { + return normalizeLibraryData({ ...EMPTY_APP_DATA, ...input }) + } + return normalizeLibraryData(input) +} + +// Every released JSON format has this core. Later collections can legitimately be absent. +export function normalizeLibraryData( + input: unknown, normalize: (data: AppData) => AppData = normalizeAppData +): AppData { + if (!isRecord(input) || !isRecord(input.settings) || !Array.isArray(input.projects) || + !Array.isArray(input.chapters) || !Array.isArray(input.storyBibles)) { + throw new Error('文件不是完整的 Novel Director 书库。') + } + if (typeof input.schemaVersion !== 'number' || !Number.isInteger(input.schemaVersion) || + input.schemaVersion < 1 || input.schemaVersion > EMPTY_APP_DATA.schemaVersion) { + throw new Error('书库的数据版本不受支持。') + } + for (const key of ALL_ENTITY_COLLECTIONS) { + if (!Object.hasOwn(input, key)) continue + const records = input[key] + if (!Array.isArray(records)) throw new Error(`书库中的 ${key} 不是数组。`) + const ids = new Set() + for (const [index, record] of records.entries()) { + const id = libraryEntityId(key, record) ?? `${key}:${index}` + if (ids.has(id)) throw new Error(`书库中的 ${key} 含有重复标识。`) + ids.add(id) + } + } + const data = normalize(input as unknown as AppData) + for (const key of ALL_ENTITY_COLLECTIONS) { + const records = input[key] + if (Array.isArray(records) && data[key].length < records.length) { + throw new Error(`书库中的 ${key} 含有无法读取的记录。`) + } + } + return data +} diff --git a/src/storage/sqlite/sqliteInitialization.ts b/src/storage/sqlite/sqliteInitialization.ts index 977475a..896ccb9 100644 --- a/src/storage/sqlite/sqliteInitialization.ts +++ b/src/storage/sqlite/sqliteInitialization.ts @@ -3,6 +3,7 @@ import { dirname } from 'node:path' import type { AppData } from '../../shared/types' import { EMPTY_APP_DATA, normalizeAppData } from '../../shared/defaults' import { ensureSqliteSchema } from './sqliteSchema' +import { dataVersion, initializeReservedSqliteLibrary, openExistingSqliteLibrary, type ValidatedSnapshot } from './sqliteLibrary' import type { SqliteDatabase, SqliteDatabaseConstructor } from './sqliteTypes' function isSqliteOpenFailure(error: unknown): boolean { @@ -34,16 +35,27 @@ export class SqliteInitialization { private initializing: Promise | null = null private generation = 0 private recoveredData: AppData | null = null + private initial?: ValidatedSnapshot constructor( private readonly Database: SqliteDatabaseConstructor, private readonly storagePath: string, private readonly legacyJsonPath: string, - private readonly persist: (db: SqliteDatabase, data: AppData) => void - ) {} + private readonly persist: (db: SqliteDatabase, data: AppData) => void, + private readonly mode: 'legacy' | 'existing' | 'new' = 'legacy', + initialData?: AppData + ) { + if (mode === 'existing') { + const { db, initial } = openExistingSqliteLibrary(Database, storagePath) + this.db = db + this.initial = initial + } + if (mode === 'new') this.db = initializeReservedSqliteLibrary(Database, storagePath, persist, initialData) + } async open(): Promise { if (this.db) return this.db + if (this.mode !== 'legacy') throw new Error('SQLite 书库已关闭。') const opening = this.opening ??= this.openDatabase(this.generation) try { const db = await opening @@ -59,11 +71,19 @@ export class SqliteInitialization { this.initializing = null this.db?.close() this.db = null + this.initial = undefined // Keep a pending open registered until it exits, so recovery I/O cannot race a reopen. } + takeInitialSnapshot(db: SqliteDatabase, revision: string) { + const initial = this.initial + this.initial = undefined + if (initial && initial.snapshot.revision === revision && initial.dataVersion === dataVersion(db)) return initial.snapshot + } + async ensureSnapshot(db: SqliteDatabase): Promise { this.assertCurrent(db) + if (this.mode !== 'legacy') return if (!this.initializing && !isSqliteEmpty(db)) return const initializing = this.initializing ??= this.initializeSnapshot(db) try { diff --git a/src/storage/sqlite/sqliteLibrary.ts b/src/storage/sqlite/sqliteLibrary.ts new file mode 100644 index 0000000..46708be --- /dev/null +++ b/src/storage/sqlite/sqliteLibrary.ts @@ -0,0 +1,66 @@ +import { accessSync, constants, lstatSync, statSync } from 'node:fs' +import { dirname } from 'node:path' +import { EMPTY_APP_DATA } from '../../shared/defaults' +import type { AppData } from '../../shared/types' +import { assertReservedLibraryFile } from '../libraryFile' +import { ensureSqliteSchema } from './sqliteSchema' +import { readLibrarySnapshot } from './sqliteSnapshot' +import type { StorageSnapshot } from '../StorageService' +import type { SqliteDatabase, SqliteDatabaseConstructor } from './sqliteTypes' + +export interface ValidatedSnapshot { + snapshot: StorageSnapshot + dataVersion: number +} + +export function dataVersion(db: SqliteDatabase): number { + return Number(db.prepare('PRAGMA data_version').get()?.data_version) +} + +export function openExistingSqliteLibrary(Database: SqliteDatabaseConstructor, path: string) { + const before = statSync(path, { bigint: true }) + if (!before.isFile() || !before.size) throw new Error('文件不是 SQLite 书库。') + const reader = new Database(path, { readonly: true, fileMustExist: true }) + let db: SqliteDatabase | undefined + try { + reader.pragma('query_only = ON') + const beforeVersion = dataVersion(reader) + const snapshot = reader.transaction(() => readLibrarySnapshot(reader))() + accessSync(path, constants.W_OK) + accessSync(dirname(path), constants.W_OK) + db = new Database(path, { fileMustExist: true }) + db.pragma('foreign_keys = ON') + const version = dataVersion(db) + const after = statSync(path, { bigint: true }) + // Compare data_version only on the same handle. Pin the writer's token before + // releasing the reader so a concurrent commit cannot slip into the cached snapshot. + if (before.dev !== after.dev || before.ino !== after.ino || dataVersion(reader) !== beforeVersion) { + throw new Error('预检期间 SQLite 书库已更改,请重试。') + } + return { db, initial: { snapshot, dataVersion: version } } + } catch (error) { db?.close(); throw error } finally { reader.close() } +} + +export function initializeReservedSqliteLibrary( + Database: SqliteDatabaseConstructor, path: string, persist: (db: SqliteDatabase, data: AppData) => void, + initialData: AppData = EMPTY_APP_DATA +): SqliteDatabase { + assertReservedLibraryFile(path) + for (const suffix of ['-wal', '-shm', '-journal']) { + if (lstatSync(path + suffix, { throwIfNoEntry: false })) { + throw new Error('新书库位置已有 SQLite 日志文件,请选择其他位置。') + } + } + const db = new Database(path, { fileMustExist: true }) + try { + db.pragma('foreign_keys = ON') + db.transaction(() => { + assertReservedLibraryFile(path) + if (db.prepare('SELECT 1 FROM sqlite_schema LIMIT 1').get()) throw new Error('新建书库的位置已被使用。') + ensureSqliteSchema(db, new Date().toISOString()) + persist(db, initialData) + })() + db.pragma('journal_mode = WAL') + return db + } catch (error) { db.close(); throw error } +} diff --git a/src/storage/sqlite/sqliteSchema.ts b/src/storage/sqlite/sqliteSchema.ts index 98c5aaf..9b76579 100644 --- a/src/storage/sqlite/sqliteSchema.ts +++ b/src/storage/sqlite/sqliteSchema.ts @@ -61,3 +61,31 @@ export function ensureSqliteSchema(db: SqliteDatabase, appliedAt: string): void VALUES (?, ?)` ).run(SQLITE_SCHEMA_VERSION, appliedAt) } + +const coreTables = { + meta: { pk: ['key'], fields: ['value'] }, + app_settings: { pk: ['id'], fields: ['json', 'updated_at'] }, + entities: { pk: ['collection', 'id'], fields: [ + 'project_id', 'chapter_id', 'job_id', 'character_id', 'chapter_order', 'title', 'updated_at', 'json' + ] }, + schema_migrations: { pk: ['version'], fields: ['applied_at'] } +} + +export function assertLibrarySchema(db: SqliteDatabase): void { + const table = db.prepare("SELECT type FROM pragma_table_list WHERE schema = 'main' AND name = ? COLLATE NOCASE") + const columns = db.prepare('SELECT lower(name) AS name, pk, hidden, "notnull" AS required, dflt_value FROM pragma_table_xinfo(?)') + for (const [name, { pk, fields }] of Object.entries(coreTables)) { + const required = [...pk, ...fields] + const actual = columns.all(name) + const hasCore = required.every(key => actual.some(row => row.name === key && row.pk === pk.indexOf(key) + 1 && !row.hidden)) + const blocksInsert = actual.some(row => !required.includes(String(row.name)) && + (row.pk || (!row.hidden && row.required && row.dflt_value === null))) + if (table.get(name)?.type !== 'table' || !hasCore || blocksInsert) { + throw new Error(`SQLite 书库缺少兼容的表结构:${name}。`) + } + } + const versions = db.prepare('SELECT version FROM schema_migrations ORDER BY version LIMIT 2').all() + if (versions.length !== 1 || versions[0].version !== SQLITE_SCHEMA_VERSION) { + throw new Error('SQLite 书库的结构版本不受支持。') + } +} diff --git a/src/storage/sqlite/sqliteSnapshot.ts b/src/storage/sqlite/sqliteSnapshot.ts index afb3247..270f52d 100644 --- a/src/storage/sqlite/sqliteSnapshot.ts +++ b/src/storage/sqlite/sqliteSnapshot.ts @@ -1,7 +1,10 @@ import type { AppData } from '../../shared/types' import { EMPTY_APP_DATA, normalizeAppData } from '../../shared/defaults' +import { ALL_ENTITY_COLLECTIONS } from '../../shared/appDataCollections' +import type { StorageSnapshot } from '../StorageService' +import { libraryEntityId, normalizeLibraryData, parseLibraryJsonValue } from '../libraryValidation' import { arrayCollectionKeys, parseJsonObject } from './sqliteEntityMapper' -import { SETTINGS_ROW_ID } from './sqliteSchema' +import { assertLibrarySchema, SETTINGS_ROW_ID } from './sqliteSchema' import type { AppDataArrayKey, SqliteDatabase } from './sqliteTypes' // A read projection is not a full AppData snapshot and must never be saved as one. @@ -29,6 +32,32 @@ export function currentRevisionFromDatabase(db: SqliteDatabase): string { return typeof value === 'string' && value ? value : '0' } +/** Read under the caller's snapshot transaction; never substitute defaults for damaged rows. */ +export function readLibrarySnapshot(db: SqliteDatabase): StorageSnapshot { + assertLibrarySchema(db) + const version = db.prepare("SELECT value FROM meta WHERE key = 'schemaVersion'").get()?.value + const revision = db.prepare("SELECT value FROM meta WHERE key = 'revision'").get()?.value + const settings = db.prepare('SELECT id, json FROM app_settings LIMIT 2').all() + if (typeof version !== 'string' || !/^[1-9]\d*$/.test(version) || typeof revision !== 'string' || + !revision.trim() || settings.length !== 1 || settings[0].id !== SETTINGS_ROW_ID || typeof settings[0].json !== 'string') { + throw new Error('SQLite 书库缺少有效的数据版本、修订号或设置。') + } + const collections = new Map(ALL_ENTITY_COLLECTIONS.map(key => [key, []])) + const rows = db.prepare('SELECT collection, id, json FROM entities ORDER BY collection, id').iterate() + for (const row of rows) { + const target = typeof row.collection === 'string' && collections.get(row.collection) + if (!target || typeof row.json !== 'string') throw new Error('SQLite 书库包含无法识别的记录。') + const value = parseLibraryJsonValue(row.json) + const id = libraryEntityId(row.collection as AppDataArrayKey, value) + if (id === undefined ? typeof row.id !== 'string' || !/^promptVersions:(0|[1-9]\d*)$/.test(row.id) : row.id !== id) { + throw new Error(`SQLite 书库中的 ${row.collection} 记录标识不一致。`) + } + target.push(value) + } + return { revision, data: normalizeLibraryData({ ...Object.fromEntries(collections), schemaVersion: Number(version), + settings: parseLibraryJsonValue(settings[0].json) }) } +} + export function readAppDataFromDatabase(db: SqliteDatabase): AppData { const schemaVersionRow = db.prepare('SELECT value FROM meta WHERE key = ?').get('schemaVersion') const settingsRow = db.prepare('SELECT json FROM app_settings WHERE id = ?').get(SETTINGS_ROW_ID) diff --git a/src/storage/sqlite/sqliteTypes.ts b/src/storage/sqlite/sqliteTypes.ts index fb756f4..f304145 100644 --- a/src/storage/sqlite/sqliteTypes.ts +++ b/src/storage/sqlite/sqliteTypes.ts @@ -6,6 +6,7 @@ export interface SqliteStatement { run(...values: SqliteValue[]): unknown get(...values: SqliteValue[]): Record | undefined all(...values: SqliteValue[]): Array> + iterate(...values: SqliteValue[]): Iterable> } export interface SqliteDatabase { From 6563d1512486a4d871a1660ffb6a27ffb07f62d7 Mon Sep 17 00:00:00 2001 From: SHIN09-git Date: Fri, 13 Nov 2026 04:48:33 +0800 Subject: [PATCH 3/8] Qualify Windows setup and narrow library commits and backups --- .github/workflows/library-s2-installer.yml | 100 +++++++++ README.md | 8 +- ROADMAP.md | 14 +- docs/RELEASE_ARTIFACTS.md | 8 +- docs/S2_WINDOWS_QUALIFICATION.md | 44 ++++ docs/S4_AUTO_BACKUPS.md | 41 ++++ package-lock.json | 4 +- package.json | 2 +- scripts/benchmark-auto-backup.mjs | 51 +++++ scripts/checks/backup-formats.mjs | 208 ++++++++++++++++++ scripts/manage-sqlite-native.mjs | 12 +- scripts/qa-library-recovery.mjs | 26 ++- scripts/run-dist-win.mjs | 15 +- scripts/run-tests.mjs | 1 + scripts/s2-windows/installed.mjs | 182 +++++++++++++++ scripts/s2-windows/qualify.ps1 | 127 +++++++++++ scripts/s2-windows/readback.mjs | 104 +++++++++ scripts/s2-windows/readback.test.mjs | 72 ++++++ scripts/validate-agent-revision-tools.mjs | 4 +- scripts/validate-backend-backup-laziness.mjs | 48 +++- scripts/validate-book-progress-ui.mjs | 5 + scripts/validate-core-modularization.mjs | 2 +- scripts/validate-director-dashboard.mjs | 4 + ...alidate-project-list-recent-open-order.mjs | 3 +- scripts/validate-project-touch.mjs | 157 +++++++++++++ scripts/validate-release-p0-readiness.mjs | 2 +- .../validate-unreviewed-draft-acceptance.mjs | 3 +- scripts/verify-win-installer-payload.mjs | 6 +- src/main/BackupService.ts | 103 +++++---- src/main/LibrarySession.ts | 7 +- src/main/backupFileFormat.ts | 46 ++++ src/main/ipc/backupIpcHandlers.ts | 8 +- src/main/ipc/dataIpcHandlers.ts | 26 ++- src/main/ipc/libraryRecoveryIpc.ts | 3 +- src/main/libraryFiles.ts | 3 +- src/preload/index.ts | 9 +- src/renderer/src/App.tsx | 3 +- src/renderer/src/hooks/useAppData.ts | 10 + src/renderer/src/views/HomeView.tsx | 27 ++- .../src/views/settings/SettingsDataPanels.tsx | 4 +- .../tests/library-recovery-window.mjs | 54 ++++- .../chapterAcceptanceReviewValidation.ts | 2 + .../chapterDiagnosticBindings.ts | 26 +++ src/shared/ipc/ipcChannels.ts | 1 + src/shared/ipc/ipcTypes.ts | 1 + src/shared/normalizers/appData.ts | 2 +- src/storage/JsonStorageService.ts | 30 ++- src/storage/SqliteStorageService.ts | 44 +++- src/storage/StorageService.ts | 1 + src/storage/sqlite/sqliteCommitContext.ts | 163 ++++++++++++++ src/storage/sqlite/sqliteReceiptAliases.ts | 21 ++ src/storage/sqlite/sqliteSnapshot.ts | 16 +- 52 files changed, 1707 insertions(+), 156 deletions(-) create mode 100644 .github/workflows/library-s2-installer.yml create mode 100644 docs/S2_WINDOWS_QUALIFICATION.md create mode 100644 docs/S4_AUTO_BACKUPS.md create mode 100644 scripts/benchmark-auto-backup.mjs create mode 100644 scripts/checks/backup-formats.mjs create mode 100644 scripts/s2-windows/installed.mjs create mode 100644 scripts/s2-windows/qualify.ps1 create mode 100644 scripts/s2-windows/readback.mjs create mode 100644 scripts/s2-windows/readback.test.mjs create mode 100644 scripts/validate-project-touch.mjs create mode 100644 src/main/backupFileFormat.ts create mode 100644 src/services/commitBundles/chapterDiagnosticBindings.ts create mode 100644 src/storage/sqlite/sqliteCommitContext.ts create mode 100644 src/storage/sqlite/sqliteReceiptAliases.ts diff --git a/.github/workflows/library-s2-installer.yml b/.github/workflows/library-s2-installer.yml new file mode 100644 index 0000000..44c8783 --- /dev/null +++ b/.github/workflows/library-s2-installer.yml @@ -0,0 +1,100 @@ +name: S2 Windows Setup +on: + push: + branches: [codex/library-s2-installer-qa] + paths-ignore: + - 'docs/**' + - 'ROADMAP.md' + - 'scripts/benchmark-*.mjs' + +permissions: + contents: read + actions: read + +jobs: + qualify: + if: github.repository == 'SHIN09-git/novel' && !github.event.deleted + runs-on: windows-2025 + timeout-minutes: 60 + strategy: + fail-fast: false + matrix: + route: [clean, upgrade] + name: S2 ${{ matrix.route }} + env: + ROUTE: ${{ matrix.route }} + SETUP_ARTIFACT: s2-setup-${{ github.sha }}-${{ github.run_attempt }} + defaults: + run: + shell: pwsh + steps: + - name: Start evidence record + run: | + $dir = New-Item -ItemType Directory -Path (Join-Path $env:RUNNER_TEMP 's2-evidence') + @{ route=$env:ROUTE; commit=$env:GITHUB_SHA; run=$env:GITHUB_RUN_ID; attempt=$env:GITHUB_RUN_ATTEMPT } | + ConvertTo-Json | Set-Content (Join-Path $dir.FullName 'job.json') + - uses: actions/checkout@v4 + with: + persist-credentials: false + - uses: actions/setup-node@v4 + with: + node-version: 24 + - name: Install locked dependencies + run: npm.cmd ci + - name: Require a newer candidate version from this commit + run: node -e "const v=require('./package.json').version,l=require('./package-lock.json');if(!require('semver').gt(v,'0.1.6-preview.20')||v!==l.version||v!==l.packages[''].version)throw Error('Set and lock the candidate version before pushing')" + - name: Build the single candidate (no publishing) + if: matrix.route == 'clean' + run: npm.cmd run dist:win + - name: Retain candidate Setup and build records + if: matrix.route == 'clean' + uses: actions/upload-artifact@v4 + with: + name: ${{ env.SETUP_ARTIFACT }} + path: | + release/Novel Director Setup *.exe + release/SHA256SUMS.txt + release/BUILD_INFO.json + compression-level: 0 + if-no-files-found: error + retention-days: 14 + # Both VMs consume the uploaded artifact. No third Windows build VM. + - name: Wait for this run's candidate + if: matrix.route == 'upgrade' + timeout-minutes: 40 + env: + GH_TOKEN: ${{ github.token }} + run: | + $run = "repos/$env:GITHUB_REPOSITORY/actions/runs/$env:GITHUB_RUN_ID" + while ($true) { + $assets = gh api "$run/artifacts?name=$env:SETUP_ARTIFACT" | ConvertFrom-Json + if ($LASTEXITCODE) { throw 'Cannot read run artifacts' } + if (@($assets.artifacts | Where-Object { $_.name -ceq $env:SETUP_ARTIFACT -and !$_.expired }).Count -eq 1) { break } + $jobs = gh api "$run/attempts/$env:GITHUB_RUN_ATTEMPT/jobs" | ConvertFrom-Json + if ($LASTEXITCODE) { throw 'Cannot read packaging job status' } + if ($jobs.jobs | Where-Object { $_.name -eq 'S2 clean' -and $_.status -eq 'completed' -and $_.conclusion -ne 'success' }) { + throw 'Packaging job ended without handing over the Setup' + } + Start-Sleep -Seconds 12 + } + - name: Download the same immutable candidate + uses: actions/download-artifact@v4 + with: + name: ${{ env.SETUP_ARTIFACT }} + path: ${{ runner.temp }}/s2-input + - name: Exercise installed product + run: ./scripts/s2-windows/qualify.ps1 + - name: Record job outcome + if: always() + env: + RESULT: ${{ job.status }} + run: | + $env:RESULT | Set-Content (Join-Path $env:RUNNER_TEMP 's2-evidence/job-result.txt') + - name: Retain qualification evidence + if: always() + uses: actions/upload-artifact@v4 + with: + name: s2-${{ matrix.route }}-report-${{ github.sha }}-${{ github.run_attempt }} + path: ${{ runner.temp }}/s2-evidence/ + if-no-files-found: warn + retention-days: 14 diff --git a/README.md b/README.md index 31f4727..dcc5914 100644 --- a/README.md +++ b/README.md @@ -86,13 +86,13 @@ npm.cmd run dist:win ### Windows 打包、SQLite 与 native 依赖 -Novel Director 使用 `better-sqlite3` 作为 main process 的 SQLite 后端。它是 native 依赖,在 Windows 上可能需要 Visual Studio C++ Build Tools(含 MSVC、Windows SDK 和 C++ CMake tools)才能重新编译 Electron 绑定。 +开发主线使用 `better-sqlite3` 13。打包前先让 Electron 执行一次 SQLite 查询。Windows x64 随依赖提供的组件能用时无需编译;只有需要从源码重新编译时才需要 Visual Studio C++ Build Tools。 建议: - 开发机先运行 `npm.cmd install`,再运行 `npm.cmd run build`。 - 运行 Electron 预览并连接 SQLite 前,先运行 `npm.cmd run native:electron`,或直接使用 `npm.cmd run preview:sqlite`。 -- 跑 Agent CLI、验证脚本或 Node.js 侧 SQLite 测试前,运行 `npm.cmd run native:node`,把 `better-sqlite3` 恢复为本机 Node.js ABI。 +- 运行 `npm.cmd run native:node` 可验证 Agent 和测试使用的 Node 也能查询 SQLite;已有组件可用时不会重新编译。 - 打包使用 `npm.cmd run dist:win`,脚本会使用仓库内 `.electron-cache` 和 `.electron-builder-cache`,减少 C 盘缓存压力。 - 打包流程会在生成 `release/win-unpacked/Novel Director.exe` 后运行 `npm.cmd run smoke:packaged`,检查图标、preload API、SQLite 默认路径、导入导出 API 和无 API Key 状态。 - 如果 SQLite 组件无法加载,桌面版会显示“书库未打开”,不会自动切换到 JSON。修复安装后点“重试”。明确选择独立 JSON 时,之后的修改只写入该文件。 @@ -280,13 +280,13 @@ The default package is an unsigned trial build. Windows may show a SmartScreen w ### Windows Packaging, SQLite, and Native Dependencies -Novel Director uses `better-sqlite3` as the main-process SQLite backend. It is a native dependency, so Windows builds may require Visual Studio C++ Build Tools, including MSVC, Windows SDK, and C++ CMake tools, to rebuild Electron bindings. +The development branch uses `better-sqlite3` 13. Packaging tests the installed binding with an actual SQLite query in Electron. The bundled Windows x64 binary normally needs no compilation; Visual Studio C++ Build Tools are needed for a source rebuild. Recommendations: - Run `npm.cmd install`, then `npm.cmd run build` on the development machine. - Before running Electron preview against SQLite, run `npm.cmd run native:electron`, or use `npm.cmd run preview:sqlite`. -- Before running Agent CLI, validation scripts, or Node-side SQLite tests, run `npm.cmd run native:node` to restore the `better-sqlite3` binding for the local Node.js ABI. +- Use `npm.cmd run native:node` to verify the binding used by the Agent CLI and tests. It does not rebuild a working binding. - Use `npm.cmd run dist:win` for packaging. The script uses repository-local `.electron-cache` and `.electron-builder-cache` directories to reduce C drive pressure. - The packaging flow runs `npm.cmd run smoke:packaged` after creating `release/win-unpacked/Novel Director.exe`; the smoke test checks the icon, preload API, SQLite default path, import/export API, and no-key state. - If the SQLite component cannot load, the desktop app shows a recovery screen instead of switching to JSON. Repair the installation and retry. Changes made to an explicitly selected JSON library stay in that file. diff --git a/ROADMAP.md b/ROADMAP.md index 2f940f3..001476a 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -1,6 +1,6 @@ # 路线图 / Roadmap -更新日期:2026-11-12。这里只保留当前方向和下一步,不再重复各版本的施工日志、源码行数和已完成待办。历史变更见 [CHANGELOG](./CHANGELOG.md),包的实际状态见 [发行产物说明](./docs/RELEASE_ARTIFACTS.md)。 +更新日期:2026-11-13。这里只保留当前方向和下一步,不再重复各版本的施工日志、源码行数和已完成待办。历史变更见 [CHANGELOG](./CHANGELOG.md),包的实际状态见 [发行产物说明](./docs/RELEASE_ARTIFACTS.md)。 ## 当前方向 @@ -17,14 +17,14 @@ ## 下一步优先事项 -执行[书库可靠性与长篇性能施工计划](./docs/LIBRARY_STABILITY_AND_PERFORMANCE_PLAN.md)。**S0 已固定旧版样本,S1 已实现书库恢复。下一项是 S2:实际运行 Windows 安装器,验证干净安装和从 preview.20 升级。** 这些更改未包含在已发布的 preview.20 中。 +见[书库可靠性与长篇性能施工计划](./docs/LIBRARY_STABILITY_AND_PERFORMANCE_PLAN.md)。本轮候选为 **preview.21**;候选测试与公开发布分开,未替换已发布的 preview.20。 | 优先级 | 工作 | 完成条件 | | --- | --- | --- | | S0 / S1,本地完成 | 保留 preview.20 虚构库;区分书库未打开和空书库 | 从错误页重试、选择已有库、恢复至新文件或明确新建;不自动切换独立 JSON | -| S2,下一版 | 核对运行时依赖,用 Windows 安装器验证升级 | 干净安装、从 preview.20 升级后均能读写 SQLite;重启后作品和版本链仍在 | -| S3-A / S3-B | 先优化打开项目,再优化采纳与修订 | 更新时间不传整库;提交不读取无关项目历史。用虚构长篇对照耗时,保留并发、回滚和版本恢复能力 | -| S4,有依据再做 | 历史详情按需读取、减少新副本或压缩自动备份 | 不把局部数据当整库保存;不删除作者采纳来源;旧备份可恢复,不自动删手动备份 | +| S2 | 用两台独立 Windows 虚拟机测试同一安装包 | 全新安装、从 preview.20 原地升级、重启、卸载及回退;以对应提交的验收报告为准 | +| S3-A / S3-B | 打开项目只写最近打开时间;SQLite 按本次采纳或修订所需的记录校验 | 继续检查版本、项目归属、报告与正文;使用同一份虚构长篇书库对照 | +| S4 | 压缩新自动备份,保留可供旧版恢复的普通 JSON;手动备份不轮替 | 两种格式均可恢复;不删除历史步骤或改写现有提交回执。首屏历史按需加载继续暂缓 | 具体的修订质量、Novelty 误报和未保存输入仍按作者反馈跟进,不阻塞上述交付。本轮性能验收不消耗模型额度,也不替代成文质量评估。 @@ -56,8 +56,8 @@ npm.cmd run build ## English -Updated 2026-11-12. The [preview.20 prerelease](https://github.com/SHIN09-git/novel/releases/tag/v0.1.6-preview.20) includes the [long-novel fixes](./docs/LONG_NOVEL_COMPLETION_REPAIR_PLAN.md). Provider balances and unrecorded historical usage remain unknown. +Updated 2026-11-13. The [preview.20 prerelease](https://github.com/SHIN09-git/novel/releases/tag/v0.1.6-preview.20) includes the [long-novel fixes](./docs/LONG_NOVEL_COMPLETION_REPAIR_PLAN.md). Provider balances and unrecorded historical usage remain unknown. -The [library reliability and performance plan](./docs/LIBRARY_STABILITY_AND_PERFORMANCE_PLAN.md) now has a synthetic preview.20 library and an explicit desktop recovery flow. Neither changes the existing public release. Next, verify a clean Windows Setup installation and an in-place upgrade from preview.20. Then measure and narrow project opening and chapter commits. History compaction and automatic backup compression are conditional, not release blockers. +The [library plan](./docs/LIBRARY_STABILITY_AND_PERFORMANCE_PLAN.md) qualifies preview.21 separately from public releases. Project opening no longer sends a full snapshot; SQLite commits load their validation inputs by scope. New automatic backups use compression where the file can be restored safely. Existing receipts and manual backups are retained. Full-library startup reads are a separate, deferred optimization. Keep project isolation, version conflict checks, idempotent commits, report provenance, rollback, backups, authorization and credential redaction. Full CRUD rewrites, FTS, graphs, cloud sync and generic extension systems remain deferred. Historical plans and measurements are references, not additional mandatory work. diff --git a/docs/RELEASE_ARTIFACTS.md b/docs/RELEASE_ARTIFACTS.md index 54b6d13..f8793a6 100644 --- a/docs/RELEASE_ARTIFACTS.md +++ b/docs/RELEASE_ARTIFACTS.md @@ -13,13 +13,17 @@ Novel Director 的本地目录按用途分为三层: - `Novel Director Setup <版本号>.exe`:给普通用户安装的文件。 - `win-unpacked/Novel Director.exe`:无需安装的本地测试版本。 - `SHA256SUMS.txt`:安装包 SHA-256 校验值。 -- `BUILD_INFO.json`:版本、构建时间、Electron 版本和打包烟雾测试状态。 +- `BUILD_INFO.json`:版本、构建时间、Electron 与 SQLite 绑定版本、安装包及 app.asar 的 SHA-256、烟雾测试结果;干净检出还会记录源码提交号。 同一版本重新打包时,旧的同版本构建产物会被替换。升级版本时,上一版本会自动归档。因此请始终从 `release/` 上传当前 GitHub Release,不要从 `releases/archive/` 选择安装包。 小说项目和 SQLite 用户数据库不存放在这些目录中,整理发行产物不会移动或删除用户数据。 -### 当前本地发行(2026-11-11) +### Preview.21 验收候选 + +`0.1.6-preview.21` 是书库恢复、项目打开、正式提交与自动备份的 Windows 验收版本,使用 Electron 44.9.1 与 better-sqlite3 13.0.3。专用分支的 [Setup 验收工作流](../.github/workflows/library-s2-installer.yml)让两台临时 Windows 机器使用同一安装包和校验文件;验收附件不等于公开 Release。下载和核对时使用同一构建的 SHA256SUMS.txt,不混用不同机器生成的未签名安装包。 + +### 上一公开发行(2026-11-11) `0.1.6-preview.20` 合并了[长篇完稿修复 P0–P2](./LONG_NOVEL_COMPLETION_REPAIR_PLAN.md)。安装请用 `release/Novel Director Setup 0.1.6-preview.20.exe`;不安装可直接打开 `release/win-unpacked/Novel Director.exe`。打开项目,在**设置 → 运行信息**中核对应用版本、运行方式和程序位置。已安装的旧版不会因更新仓库目录而自动升级。 diff --git a/docs/S2_WINDOWS_QUALIFICATION.md b/docs/S2_WINDOWS_QUALIFICATION.md new file mode 100644 index 0000000..f147163 --- /dev/null +++ b/docs/S2_WINDOWS_QUALIFICATION.md @@ -0,0 +1,44 @@ +# S2:Windows Setup 验收 + +唯一的真实安装入口是 [S2 Windows Setup](../.github/workflows/library-s2-installer.yml)。它只响应 `SHIN09-git/novel` 的 `codex/library-s2-installer-qa` 分支推送。在 GitHub 托管的两台临时 Windows x64 上运行,不是发布流程。只改 `docs/**`、`ROADMAP.md` 或 `scripts/benchmark-*.mjs` 不会触发;根目录 `README.md` 不在排除范围。 + +**证据边界。** 本机只读检查未发现可确认的一次性 Windows 客户机:没有可用的 Hyper-V 管理命令或虚拟机 CIM 命名空间,未找到 Windows Sandbox 或常见桌面虚拟机管理程序。仅看到计算服务运行,不足以证明有可用客户机。未安装虚拟化组件,也未在作者账号运行 Setup。本页是复测说明,不替代某一次 CI 的验收记录。 + +## 如何运行 + +1. 由集成人确认候选提交,包含 S1、候选版本号及其锁文件、当前构建记录生成脚本和 S2 验收脚本。候选号必须高于 `0.1.6-preview.20`。只在获授权后推送到上述精确分支,不需要先修改默认分支、不需要创建 Release。 +2. 矩阵的 `S2 clean` 用现有 `npm.cmd run dist:win` 打一次 x64 Setup;该命令自带解包成品启动检查,不是安装验收。`S2 upgrade` 不打包,只等同一次运行、同一次尝试的 artifact。两台机器都下载这一份,核实 Setup 的 SHA-256、`SHA256SUMS.txt`、`BUILD_INFO.json`、提交 SHA 和包名才安装。 +3. 等两个任务结束。若需重跑,选择 **Re-run all jobs**;不要仅重跑失败的矩阵项。新尝试有独立的 artifact 名。要复核原来的安装包,保留原尝试的文件,不能用第二次构建的同版本包代替。 + +旧版取自[公开的 preview.20](https://github.com/SHIN09-git/novel/releases/tag/v0.1.6-preview.20)。`Novel.Director.Setup.0.1.6-preview.20.exe` 固定 SHA-256 为 `aa38b6dbcba629591113d928617f4ea82ee9c9dcfdfd199e67a7a9ce4c0d50ce`(与该发布的 [SHA256SUMS.txt](https://github.com/SHIN09-git/novel/releases/download/v0.1.6-preview.20/SHA256SUMS.txt) 一致)。每次执行 Setup 前重新算哈希,不下载“最新版”,不更新标签或附件。安装和卸载均为当前客户机用户的静默 NSIS 路径,不测试提升权限、全用户安装或交互向导。 + +## 验收判据 + +| 路径 | 实际动作及失败条件 | +| --- | --- | +| 全新安装 | 确认无既有安装或应用配置;运行候选 Setup;启动安装位置的应用,确认默认库为空。用桌面保存接口写入 S0 数据,在阅读页编辑并保存一章。关闭、重开,核对正文、三版版本链和独立 Agent 读数。运行真正的卸载器,检查程序和注册项消失;重装同一 Setup,默认路径和编辑仍在;再次卸载。 | +| 原地升级 | 在另一台机器安装固定的 preview.20;旧版通过桌面接口把固定 S0 备份写到非默认 SQLite 路径。关闭、重开旧版并备份已关闭的数据库。**不卸载旧版**,运行候选 Setup,不再传安装目录。候选必须接管旧位置,读到旧配置选择的原库;桌面编辑并重启后,随后运行的 Agent 必须读到同一路径和第三个版本。 | +| 回退 | 卸载候选、重装固定旧版。旧版先在单独配置目录打开升级后数据库的**副本**,验证编辑及三版链;另在新文件和新配置目录恢复升级前 SQLite 备份,验证原正文及两版链。两次均检查升级后原库的哈希不变。即使新版文件兼容性检查失败,也尝试备份恢复;前一失败不会因后一成功变绿,应记录为“仅备份回退已证实”,交由集成人判定。 | + +每次启动都核对运行中的 exe、成品模式和 renderer 来源。应用版本以已安装 `app.asar/package.json` 和运行中应用报告为准;记录注册表 `DisplayVersion`,但不用它决定预发布后缀。候选应用的 asar 哈希、Electron 和 better-sqlite3 版本须与构建记录一致;依赖版本也与锁文件比对,记录实际原生绑定文件的校验值。只有安装后的界面能在 SQLite 上读写并在冷启动后读回才通过。 + +两本书的项目 ID、章节 ID、正文哈希、已采纳草稿、提交回执、版本 ID 和父版本、另一项目待续跑任务的步骤与检查点,均对照 [S0 清单](../scripts/fixtures/library-preview20/manifest.json)。编辑必须产生一条引用原第二版的新手动修订,原记录保留。Agent 在独立 Node 进程中仅指定应用配置目录,**不传数据库路径**;它自己选中的后端必须是 SQLite,不接受 JSON 回退。样本模型地址限制为回环拒绝端口;不启动生成任务,不携带模型凭据。 + +## 取回证据 + +在对应提交的 Actions 运行页下载以下三个 artifact,保留 **run ID、attempt、commit SHA、Setup SHA-256**: + +- `s2-setup--`:实际分给两台机器的 Setup、`SHA256SUMS.txt`、`BUILD_INFO.json`。 +- `s2-clean-report--` 和 `s2-upgrade-report--`:每台的 `qualification.json` 记录 NSIS 的命令、退出码、注册项、安装位置和哈希;各阶段 JSON 记录桌面及 Agent 回读、版本链、绑定、模型请求数和失败原因。若在安装前就失败,可能只有 job 记录;详见对应步骤日志。 + +按原始字节自行核对下载的 Setup、两份报告中的候选 SHA-256、构建记录中的提交与 asar 哈希。两份报告都须为 `passed`,关键阶段不能缺失。artifact 保留 14 天,长期留存由集成人负责。解包成品启动、静态检查、合成测试或单台成功都不能证明两条路径通过。 + +## 本地复测 + +只运行以下命令。它使用仓库内虚构样本的临时副本,通过真实的 SQLite 修订事务和 Node Agent 验证回读,不启动 Electron、不安装程序、不接触作者书库: + +```powershell +node --test scripts/s2-windows/readback.test.mjs +``` + +该测试覆盖原始两版及修订后三版回读、损坏样本拒绝、升级前备份异地恢复、拒绝覆盖已有文件及拒绝复制尚有未写回日志的数据库。真实安装脚本有托管环境检查;不要在本机设置 CI 环境变量来绕过它。 diff --git a/docs/S4_AUTO_BACKUPS.md b/docs/S4_AUTO_BACKUPS.md new file mode 100644 index 0000000..a34b0be --- /dev/null +++ b/docs/S4_AUTO_BACKUPS.md @@ -0,0 +1,41 @@ +# S4 自动备份:兼容性与同机测量 + +本批次只改变新自动备份。手动备份、恢复或导入前的安全副本、作品导出仍是普通 JSON。备份只包含书库数据;不搬运本机凭据、授权文件或库外的 `.usage` 收据。 + +## 恢复与轮替 + +- 设置页列出并能恢复、删除旧 `.json` 和新 `.json.gz`。启动恢复及合并/覆盖导入的文件选择器也支持两种格式。压缩备份只作输入,不能选为可写书库。 +- 自动备份到期时,先写普通 JSON 临时文件,再从同一文件生成 gzip。两个文件写完后,先公布普通回退副本,最后公布 gzip;成功后才轮替。失败不提前删除旧备份。若进程意外退出,未公布的临时文件不参与列表和到期判断。 +- 新旧自动备份合计保留七份。七份都是 gzip 时,另有一份最新、完整、可用旧版 JSON 读取方式打开的 `novel-director-<时间>-rollback-.json`。它不依赖碰巧存在的手动备份,也不占七个名额。只轮替程序命名的自动备份和回退副本。 +- 压缩输入最多 385 MiB,解压后的 JSON 最多 384 MiB,按实际流过的字节数检查,包括多个 gzip 成员。超过 **384 MiB 的新库**自动改存旧格式普通 JSON,仍参与当天到期判断和七份轮替;此时新自动备份本身即可回退。手动和已有普通 JSON 不受 gzip 上限影响,仍受原有内存和 JSON 解析能力限制。 +- 恢复、导入均在数据校验后才写目标库。设置页覆盖恢复现在必须先成功保存恢复前副本;磁盘满等错误会停止覆盖。常规保存仍沿用已有的自动备份错误处理,不因自动备份失败禁止作者保存书库。 + +## 测量 + +Windows x64,Node 24.14.0。使用 `createLongNovelFixture(500)` 的两个虚构项目;规范化后紧凑 UTF-8 JSON 为 77,569,300 字节,原格式含缩进的备份为 81,004,739 字节。相同配方、同机、各热身一次,再测七次。每轮把时钟推进 25 小时以触发真实备份,接着检查未到期路径,再读取备份并逐项核对全库。时间为毫秒,不含测试断言。基线为工作树起点 `804f738` 的备份实现。 + +| 指标 | 原普通 JSON | 新自动备份 | +| --- | ---: | ---: | +| 单份自动备份(字节) | 81,004,739 | 2,280,072 | +| 八次到期写入后实际保留(字节) | 567,033,173,7 个文件 | 96,965,243,8 个文件,含普通回退副本 | +| 到期:写入、公布及清理(中位/最慢) | 228.56 / 253.13 | 368.41 / 390.60 | +| 未到期:只检查目录(中位/最慢) | 0.53 / 0.69 | 0.85 / 1.10 | +| 读取并校验备份(中位/最慢) | 115.71 / 137.40 | 290.21 / 308.32 | + +此样本长期占用减少 82.9%,但到期备份和读取都变慢。样本含重复的虚构正文、提示词和历史,不代表作者书库的压缩率。轮替期间还需要能同时容纳新普通副本和新 gzip 的额外空间。上表是文件字节数,不是文件系统分配的块数。 + +## 复测与交接 + +在被测工作树运行 `node scripts/benchmark-auto-backup.mjs`。同一脚本可分别搭配改前、改后的备份服务复测;临时库在本工作树的 `tmp` 内,结束后删除。不需要改包脚本。 + +```powershell +npm.cmd test -- validate-backend-backup-laziness validate-library-session validate-import-safety +$env:S4_TEST_LARGE = '1' +node --max-old-space-size=4096 scripts/validate-backend-backup-laziness.mjs +``` + +专项覆盖混合轮替、最后一份旧自动 JSON 退出后的普通回退副本、原手动文件不变、两处部分写入 ENOSPC、最终重命名失败、重试、损坏/截断、CRC、UTF-8、BOM、多成员超限、合并与覆盖的结构差异,以及真实 SQLite 恢复后重启。另用默认生产上限写出并恢复了 **402,674,364 字节**的普通自动备份,确认同日不重做。恢复样本取自仓库内 preview.20 虚构备份;没有运行旧发行程序或真实安装器。 + +生产入口:[写入与轮替](../src/main/BackupService.ts)、[格式识别与有界读取](../src/main/backupFileFormat.ts)、[书库痕迹](../src/main/libraryFiles.ts)、[启动恢复](../src/main/LibrarySession.ts)、[恢复选择器](../src/main/ipc/libraryRecoveryIpc.ts)、[设置恢复](../src/main/ipc/backupIpcHandlers.ts)、[数据导入](../src/main/ipc/dataIpcHandlers.ts)、[列表字段](../src/shared/ipc/ipcTypes.ts)、[设置列表](../src/renderer/src/views/settings/SettingsDataPanels.tsx)。 + +验证入口:[已有回归入口](../scripts/validate-backend-backup-laziness.mjs)、[S4 用例](../scripts/checks/backup-formats.mjs)、[计时脚本](../scripts/benchmark-auto-backup.mjs)。生产构建使用工作树临时输出目录,不生成安装包。 diff --git a/package-lock.json b/package-lock.json index 220957e..52d987a 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "novel-director", - "version": "0.1.6-preview.20", + "version": "0.1.6-preview.21", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "novel-director", - "version": "0.1.6-preview.20", + "version": "0.1.6-preview.21", "license": "MIT", "dependencies": { "@vitejs/plugin-react": "^5.1.1", diff --git a/package.json b/package.json index 4473432..903ca38 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "novel-director", - "version": "0.1.6-preview.20", + "version": "0.1.6-preview.21", "productName": "Novel Director", "description": "AI long-form novel creation management desktop MVP", "license": "MIT", diff --git a/scripts/benchmark-auto-backup.mjs b/scripts/benchmark-auto-backup.mjs new file mode 100644 index 0000000..1db13ce --- /dev/null +++ b/scripts/benchmark-auto-backup.mjs @@ -0,0 +1,51 @@ +import assert from 'node:assert/strict' +import { mkdir, mkdtemp, readdir, rm, stat } from 'node:fs/promises' +import { join } from 'node:path' +import { pathToFileURL } from 'node:url' +import { build } from 'esbuild' +import { createLongNovelFixture } from './fixtures/long-novel-performance.mjs' +import { repoRoot } from './utils/repo-root.mjs' + +// Run unchanged against each revision. Times include real publication/rotation and parsing. +const chapters = Number(process.argv[2] ?? 500) +const root = join(repoRoot, 'tmp') +await mkdir(root, { recursive: true }) +const dir = await mkdtemp(join(root, 's4-backup-bench-')) +try { + const entry = join(dir, 'backup.mjs') + await build({ stdin: { resolveDir: repoRoot, contents: [ + "export { BackupService } from './src/main/BackupService'", + "export { normalizeAppData } from './src/shared/defaults'" + ].join('\n') }, outfile: entry, bundle: true, format: 'esm', platform: 'node' }) + const { BackupService, normalizeAppData } = await import(pathToFileURL(entry).href) + const data = normalizeAppData(createLongNovelFixture(chapters)) + const service = new BackupService(dir), snapshotBytes = Buffer.byteLength(JSON.stringify(data)) + const samples = { due: [], notDue: [], read: [] } + async function time(key, fn, warmup) { + const before = performance.now(), value = await fn() + if (!warmup) samples[key].push(performance.now() - before) + return value + } + let path + for (let day = 0; day < 8; day++) { + const now = Date.now, simulated = now() + day * 25 * 3600000 + Date.now = () => simulated + try { + path = await time('due', () => service.maybeCreateAutomaticBackup(async () => data), day === 0) + assert.ok(path) + assert.equal(await time('notDue', () => service.maybeCreateAutomaticBackup(() => { + throw new Error('non-due backup read the library') + }), day === 0), null) + const restored = await time('read', () => service.loadBackup(path), day === 0) + assert.deepEqual(restored, data) + } finally { Date.now = now } + } + const folder = service.getBackupDir(), files = await readdir(folder) + const retained = await Promise.all(files.map(name => stat(join(folder, name)))) + console.log(JSON.stringify({ node: process.version, chapters, snapshotBytes, autoBytes: (await stat(path)).size, + retainedFiles: files.length, retainedBytes: retained.reduce((sum, s) => sum + s.size, 0), + timingsMs: Object.fromEntries(Object.entries(samples).map(([key, v]) => { + v.sort((a, b) => a - b) + return [key, { n: v.length, median: +v[3].toFixed(2), max: +v.at(-1).toFixed(2) }] + })) }, null, 2)) +} finally { await rm(dir, { recursive: true, force: true }) } diff --git a/scripts/checks/backup-formats.mjs b/scripts/checks/backup-formats.mjs new file mode 100644 index 0000000..d8ff6cf --- /dev/null +++ b/scripts/checks/backup-formats.mjs @@ -0,0 +1,208 @@ +import assert from 'node:assert/strict' +import { mkdir, readdir, readFile, writeFile } from 'node:fs/promises' +import { join } from 'node:path' +import { gunzipSync, gzipSync } from 'node:zlib' +import { repoRoot } from '../utils/repo-root.mjs' + +async function directoryBytes(dir) { + return Object.fromEntries(await Promise.all((await readdir(dir)).map(async name => + [name, await readFile(join(dir, name))]))) +} + +// SQLite and the JSON snapshot may enumerate the same entities in different orders. +function persisted(data) { + const snapshot = JSON.parse(JSON.stringify(data)) + for (const records of Object.values(snapshot).filter(Array.isArray)) { + records.sort((a, b) => String(a.id ?? a.projectId).localeCompare(String(b.id ?? b.projectId))) + } + return snapshot +} + +export async function checkBackupFormats({ api, work, handlers, test }) { + const old = await readFile(join(repoRoot, 'scripts/fixtures/library-preview20/backup.json')) + const data = api.normalizeAppData(JSON.parse(old)), expected = persisted(data) + const invoke = (channel, arg) => handlers.get(api.IPC_CHANNELS[channel])({}, arg) + const dialogs = globalThis.__backupDialogs + + await test('mixed rotations, ENOSPC on both formats and failed gzip publish preserve prior files', async () => { + const service = new api.BackupService(join(work, 'partial-backup-rotation')), dir = service.getBackupDir() + await service.ensureBackupDir() + const userFile = join(dir, 'my-auto-copy.json'), broken = join(dir, 'bad.json') + await writeFile(userFile, old) + await writeFile(broken, '{}') + const manual = await service.createBackup(data, false), manualBytes = await readFile(manual) + const now = Date.now, base = now(), day = 25 * 3600_000 + try { + for (let i = 0; i < 7; i++) await writeFile(join(dir, `novel-director-${base + day * i}-auto.json`), old) + Date.now = () => base + 6 * day + assert.equal(await service.maybeCreateAutomaticBackup(() => assert.fail('old JSON satisfies schedule')), null) + Date.now = () => base + 7 * day + const first = await service.maybeCreateAutomaticBackup(data) + assert.match(first, /\.json\.gz$/) + const before = await directoryBytes(dir) + assert.equal((await service.listBackups()).filter(b => b.isAutomatic).length, 7) + assert.equal(before[`novel-director-${base}-auto.json`], undefined) + Date.now = () => base + 8 * day + for (const [flag, code] of [['__backupPartialWriteFailure', 'ENOSPC'], ['__gzipDiskFull', 'ENOSPC'], + ['__backupRenameFailure', 'EACCES']]) { + globalThis[flag] = true + try { await assert.rejects(service.maybeCreateAutomaticBackup(data), { code }) } + finally { delete globalThis[flag] } + assert.deepEqual(await directoryBytes(dir), before) + } + let latest + for (let i = 8; i <= 15; i++) { + Date.now = () => base + i * day + latest = await service.maybeCreateAutomaticBackup({ ...data, + projects: data.projects.map(p => ({ ...p, name: 'snapshot ' + i })) }) + } + const entries = await service.listBackups(), autos = entries.filter(e => e.isAutomatic) + assert.equal(autos.length, 7) + assert(autos.every(e => e.path.endsWith('.json.gz'))) + assert.deepEqual(autos.map(b => b.timestamp), [15,14,13,12,11,10,9].map(i => base + i * day)) + const rolls = entries.filter(e => e.isRollback) + assert.equal(rolls.length, 1) + assert.deepEqual(await readFile(rolls[0].path), gunzipSync(await readFile(latest))) + assert.equal(JSON.parse(await readFile(rolls[0].path)).projects[0].name, 'snapshot 15') + assert.deepEqual(await readFile(manual), manualBytes) + assert.deepEqual(await readFile(userFile), old) + assert.equal(await readFile(broken, 'utf8'), '{}') + assert.deepEqual(persisted(await service.loadBackup(userFile)), expected) + await assert.rejects(service.deleteBackup(join(work, 'external.json.gz')), /outside/) + } finally { Date.now = now } + }) + + await test('gzip boundary is inclusive; oversized auto is a scheduled, recoverable plain backup', async () => { + const home = join(work, 'boundary') + const reference = new api.BackupService(home) + const manual = await reference.createBackup(data, false), bytes = await readFile(manual) + const inclusive = new api.BackupService(home, bytes.length) + const gzip = await inclusive.createBackup(data) + assert.deepEqual(gunzipSync(await readFile(gzip)), bytes) + const smaller = new api.BackupService(home, bytes.length - 1) + await assert.rejects(smaller.loadBackup(gzip), /上限/) + const now = Date.now + Date.now = () => now() + 25 * 3600_000 + try { + const plain = await smaller.maybeCreateAutomaticBackup(data) + assert.match(plain, /-auto-.*\.json$/) + assert.deepEqual(await readFile(plain), bytes) + assert.deepEqual(persisted(await smaller.loadBackup(plain)), expected) + assert.deepEqual(persisted(await smaller.loadBackup(manual)), expected) + assert.equal(await smaller.maybeCreateAutomaticBackup(() => assert.fail('plain satisfies schedule')), null) + } finally { Date.now = now } + }) + + await test('reject bad header, truncation, CRC, UTF-8, JSON, shape and expansion across gzip members', async () => { + const service = new api.BackupService(join(work, 'invalid')) + await service.ensureBackupDir() + const path = join(service.getBackupDir(), 'input.JSON.GZ'), gz = gzipSync(old), crc = Buffer.from(gz) + crc[crc.length - 8] ^= 1 + for (const [bytes, reason] of [[old, /Gzip/], [gz.subarray(0, -2), /Gzip/], [crc, /Gzip/], + [gzipSync(Buffer.from([0xff])), /UTF-8/], [gzipSync('{'), /JSON/], [gzipSync('{}'), /完整/]]) { + await writeFile(path, bytes) + await assert.rejects(service.loadBackup(path), reason) + } + await writeFile(path, Buffer.concat([gzipSync(' '.repeat(8192)), gzipSync(' '.repeat(8192) + '{}')])) + await assert.rejects(api.readBackupJson(path, 8193), /上限/) + await writeFile(path, Buffer.concat([gzipSync('{}'), Buffer.alloc(1024 * 1024 + 8192)])) + await assert.rejects(api.readBackupJson(path, 8192), /压缩备份文件超出/) + await writeFile(path, gzipSync(Buffer.concat([Buffer.from('\ufeff'), old]))) + assert.deepEqual(persisted(await service.loadBackup(path)), expected) + }) + + await test('gzip-only profile is not new; recovery picker restores the entire old fixture and reopens it', async () => { + const home = join(work, 'recover'), dir = join(home, 'backups'), target = join(home, 'restored.sqlite') + await mkdir(dir, { recursive: true }) + const path = join(dir, 'book.json.gz'), config = new api.AppConfigService(home) + const session = new api.LibrarySession(config), bytes = gzipSync(old) + await writeFile(path, bytes.subarray(0, -3)) + try { + await assert.rejects(session.load(), /旧书库记录/) + await assert.rejects(api.resolveAgentRuntimeStoragePath({ userDataPath: home }), /找不到原书库/) + api.registerLibraryRecoveryIpc(session, home, dir) + dialogs.showOpenDialog = async opts => { + assert.equal(opts.defaultPath, dir) + assert.deepEqual(opts.filters[0].extensions, ['json', 'gz']) + return { filePaths: [path] } + } + dialogs.showSaveDialog = async () => ({ filePath: target }) + assert.equal((await invoke('APP_RECOVER_LIBRARY', 'restore')).ok, false) + assert.deepEqual(await readdir(home), ['backups']) + await writeFile(path, bytes) + assert.equal((await invoke('APP_RECOVER_LIBRARY', 'restore')).ok, true) + assert.deepEqual(persisted((await session.load()).snapshot.data), expected) + assert.deepEqual(await readFile(path), bytes) + } finally { session.close() } + const restarted = new api.LibrarySession(config) + try { assert.deepEqual(persisted((await restarted.load()).snapshot.data), expected) } + finally { restarted.close() } + }) + + await test('settings restore/import validate before replacing; merge stays partial; export stays plain', async () => { + const service = new api.BackupService(join(work, 'partial-backup-ipc')) + const path = await service.createBackup(data), dir = service.getBackupDir() + const invalid = join(dir, 'invalid.json.gz'), partial = join(dir, 'partial.json.gz') + await writeFile(invalid, gzipSync(old).subarray(0, -2)) + await writeFile(partial, gzipSync(JSON.stringify({ projects: [{ ...data.projects[0], id: 'merge-only' }] }))) + const storage = new api.SqliteStorageService(join(work, 'import.sqlite')) + const context = { getStorage: () => storage, backupService: service, + credentialService: { hasApiKey: async () => false } } + let selected = invalid + dialogs.showOpenDialog = async opts => { + assert.deepEqual(opts.filters[0].extensions, ['json', 'gz']) + return { filePaths: [selected] } + } + try { + api.registerDataIpcHandlers(context) + api.registerBackupIpcHandlers(context) + const before = await storage.loadSnapshot(), files = await directoryBytes(dir) + assert((await invoke('BACKUP_LIST')).backups.some(b => b.path === path && b.isAutomatic)) + assert.equal((await invoke('BACKUP_RESTORE', invalid)).ok, false) + assert.equal((await invoke('STORAGE_IMPORT', { strategy: 'replace' })).ok, false) + selected = partial + assert.equal((await invoke('STORAGE_IMPORT', { strategy: 'replace' })).ok, false) + selected = path + globalThis.__backupPartialWriteFailure = true + try { + assert.equal((await invoke('BACKUP_RESTORE', path)).code, 'ENOSPC') + assert.equal((await invoke('STORAGE_IMPORT', { strategy: 'replace' })).code, 'ENOSPC') + } finally { delete globalThis.__backupPartialWriteFailure } + assert.deepEqual(await storage.loadSnapshot(), before) + assert.deepEqual(await directoryBytes(dir), files) + const restored = await invoke('BACKUP_RESTORE', path) + assert.equal(restored.ok, true, restored.error) + assert.deepEqual(persisted(JSON.parse(await readFile(restored.preRestoreBackupPath))), persisted(before.data)) + assert.deepEqual(persisted(await storage.load()), expected) + selected = partial + assert.equal((await invoke('STORAGE_IMPORT', { strategy: 'merge' })).mergeBlocked, false) + assert((await storage.load()).projects.some(p => p.id === 'merge-only')) + selected = path + assert.equal((await invoke('STORAGE_IMPORT', { strategy: 'replace' })).mergeBlocked, false) + assert.deepEqual(persisted(await storage.load()), expected) + selected = join(work, 'export.json') + dialogs.showSaveDialog = async opts => { + assert.deepEqual(opts.filters[0].extensions, ['json']) + return { filePath: selected } + } + assert.equal((await invoke('STORAGE_EXPORT', await storage.load())).filePath, selected) + assert.deepEqual(persisted(JSON.parse(await readFile(selected))), expected) + assert.equal((await invoke('STORAGE_IMPORT', { strategy: 'replace' })).mergeBlocked, false) + } finally { storage.close() } + }) + + if (process.env.S4_TEST_LARGE === '1') { + await test('real oversized UTF-8 library writes and restores using the default production limit', async () => { + const service = new api.BackupService(join(work, 'large-plain')) + const chapter = data.chapters[0], body = '书'.repeat(128 * 1024 * 1024 + 1) + const path = await service.maybeCreateAutomaticBackup({ ...data, chapters: [{ ...chapter, body }] }) + assert.match(path, /-auto-.*\.json$/) + const rows = await service.listBackups() + assert.equal(rows.length, 1) + assert(rows[0].isAutomatic && rows[0].size > 384 * 1024 * 1024) + assert.equal((await service.loadBackup(path)).chapters[0].body, body) + assert.equal(await service.maybeCreateAutomaticBackup(() => assert.fail('unexpected retry')), null) + console.log('Oversized synthetic auto JSON bytes:', rows[0].size) + }) + } +} diff --git a/scripts/manage-sqlite-native.mjs b/scripts/manage-sqlite-native.mjs index c45552b..1377985 100644 --- a/scripts/manage-sqlite-native.mjs +++ b/scripts/manage-sqlite-native.mjs @@ -9,7 +9,8 @@ const mode = process.argv[2] ?? 'help' const electronPackage = JSON.parse(readFileSync(join(root, 'node_modules', 'electron', 'package.json'), 'utf8')) const npmCli = process.env.npm_execpath || join(dirname(process.execPath), 'node_modules', 'npm', 'bin', 'npm-cli.js') const electronRebuildCli = join(root, 'node_modules', '@electron', 'rebuild', 'lib', 'cli.js') -const electronCli = join(root, 'node_modules', 'electron', 'cli.js') +const electronExe = join(root, 'node_modules', 'electron', 'dist', process.platform === 'win32' + ? 'electron.exe' : process.platform === 'darwin' ? 'Electron.app/Contents/MacOS/Electron' : 'electron') const nodeBinding = join(root, 'node_modules', 'better-sqlite3', 'build', 'Release', 'better_sqlite3.node') const packagedBinding = join( root, @@ -34,7 +35,8 @@ const env = { function run(command, args, options = {}) { const result = spawnSync(command, args, { cwd: root, - env, + env: options.env ?? env, + timeout: options.timeout, stdio: options.stdio ?? 'inherit', shell: false }) @@ -60,7 +62,7 @@ function probeScript(kind) { "db.prepare('select 1 as ok').get()", 'db.close()', "console.log('better-sqlite3 probe ok')", - "if (process.versions.electron) require('electron').app.quit()" + ...(kind === 'electron' ? ["if (!process.versions.electron) throw new Error('Expected Electron runtime')"] : []) ].join('\n'), 'utf8' ) @@ -72,7 +74,7 @@ function checkNode() { } function checkElectron() { - run(process.execPath, [electronCli, probeScript('electron')]) + run(electronExe, [probeScript('electron')], { env: { ...env, ELECTRON_RUN_AS_NODE: '1' }, timeout: 30_000 }) } function rebuildElectron() { @@ -92,6 +94,7 @@ function copyPackagedElectronBinding() { } function ensureElectron() { + try { checkElectron(); return } catch { /* Try rebuilding only if the installed binding cannot execute SQL. */ } try { rebuildElectron() } catch (error) { @@ -104,6 +107,7 @@ function ensureElectron() { } function ensureNode() { + try { checkNode(); return } catch { /* Restore a runtime-specific binding only when necessary. */ } console.log('Rebuilding better-sqlite3 for local Node.js scripts...') run(process.execPath, [npmCli, 'rebuild', 'better-sqlite3']) checkNode() diff --git a/scripts/qa-library-recovery.mjs b/scripts/qa-library-recovery.mjs index 9c5b674..5f4c90f 100644 --- a/scripts/qa-library-recovery.mjs +++ b/scripts/qa-library-recovery.mjs @@ -9,6 +9,7 @@ import { createRequire } from 'node:module' import { basename, dirname, join, relative, resolve } from 'node:path' import { pathToFileURL } from 'node:url' import { parseArgs, promisify } from 'node:util' +import { gzipSync } from 'node:zlib' import { repoRoot } from './utils/repo-root.mjs' import { button, click, closeElectron, fill, waitFor } from './utils/qa-agent-authorization-driver.mjs' import { connect, evaluate, launch, sha256, within } from './utils/long-novel-ui-driver.mjs' @@ -652,14 +653,17 @@ async function picker(s, p, title, action, file) { async function explicitRecovery(t) { const open = '选择已有书库', src = '选择要恢复的 JSON 备份', dest = '将备份恢复到新文件(不覆盖原书库)' const independent = '使用独立 JSON 书库' - for (const mode of ['choose', 'restore', 'explicit-json', 'create']) { + for (const mode of ['choose', 'restore', 'restore-gzip', 'explicit-json', 'create']) { const p = await fault('corrupt', mode) - const name = mode === 'restore' ? 'restored.sqlite' : mode === 'explicit-json' ? 'independent.json' : mode + '.sqlite' + const restore = mode === 'restore' || mode === 'restore-gzip' + const name = restore ? 'restored.sqlite' : mode === 'explicit-json' ? 'independent.json' : mode + '.sqlite' const alternative = join(p.dir, name), bad = join(p.dir, 'invalid.sqlite') - const manual = join(p.user, 'backups', 'novel-director-100-manual-s0.json') - if (mode === 'restore') { - await mkdir(dirname(manual)) - await copyNew(join(fixtureDir, 'backup.json'), manual) + const backup = join(p.user, 'backups', mode === 'restore-gzip' ? 'novel-director-100-auto.json.gz' : 'novel-director-100-manual-s0.json') + if (restore) { + await mkdir(dirname(backup)) + const fixture = join(fixtureDir, 'backup.json') + if (mode === 'restore-gzip') await writeFile(backup, gzipSync(await readFile(fixture)), { flag: 'wx' }) + else await copyNew(fixture, backup) } else if (mode !== 'create') await copyNew(join(fixtureDir, mode === 'choose' ? 'library.sqlite' : 'backup.json'), alternative) if (mode === 'choose') await writeFile(bad, 'Invalid synthetic library.\n', { flag: 'wx' }) const entry = { name: mode, before: await inventory(p) } @@ -676,7 +680,7 @@ async function explicitRecovery(t) { } await windowRun(t, p, 'explicit-' + mode, async (c, s) => { entry.blocked = await blocked(c, p) - const choice = mode === 'explicit-json' ? 'choose' : mode + const choice = restore ? 'restore' : mode === 'explicit-json' ? 'choose' : mode const operate = async cancel => { await recover(c, choice) if (mode === 'choose') await picker(s, p, open, cancel ? 'cancel' : 'select', cancel ? undefined : alternative) @@ -684,8 +688,8 @@ async function explicitRecovery(t) { await picker(s, p, open, 'select', alternative) await picker(s, p, independent, cancel ? 'cancel-json' : 'confirm-json') } - if (mode === 'restore') { - await picker(s, p, src, 'select', manual) + if (restore) { + await picker(s, p, src, 'select', backup) await picker(s, p, dest, cancel ? 'cancel' : 'save', cancel ? undefined : alternative) } if (mode === 'create') await picker(s, p, '新建独立书库', cancel ? 'cancel' : 'save', cancel ? undefined : alternative) @@ -696,7 +700,7 @@ async function explicitRecovery(t) { entry.afterCancel = await inventory(p) assert.deepEqual(entry.afterCancel, entry.before) assert(!await exists(p.defaultDb)) - if (mode === 'restore' || mode === 'create') assert(!await exists(alternative)) + if (restore || mode === 'create') assert(!await exists(alternative)) await screen(c, mode + '-cancelled') if (mode === 'choose') { const badHash = await sha256(bad) @@ -727,7 +731,7 @@ async function explicitRecovery(t) { assert.deepEqual(entry.afterRestart.volume, entry.before.volume) assert.deepEqual(entry.afterRestart.parked, entry.before.parked) assert(!await exists(p.defaultDb)) - if (mode === 'restore') assert.equal(await sha256(manual), manifest.backupSha256) + if (restore) assert.deepEqual(entry.afterRestart.backups, entry.before.backups) entry.status = 'passed' } } diff --git a/scripts/run-dist-win.mjs b/scripts/run-dist-win.mjs index a852afe..392ad7a 100644 --- a/scripts/run-dist-win.mjs +++ b/scripts/run-dist-win.mjs @@ -84,6 +84,9 @@ function writeReleaseMetadata() { } const installerHash = sha256(installerPath) + // Source archives may not include Git. Do not attribute a dirty build to an exact commit. + const git = args => spawnSync('git', args, { cwd: root, encoding: 'utf8', windowsHide: true }) + const head = git(['rev-parse', 'HEAD']), status = git(['status', '--porcelain']) writeFileSync(join(releaseDir, 'SHA256SUMS.txt'), `${installerHash} ${installerName}\n`, 'utf8') writeFileSync( join(releaseDir, 'BUILD_INFO.json'), @@ -91,6 +94,9 @@ function writeReleaseMetadata() { version: packageJson.version, builtAt: new Date().toISOString(), electronVersion: electronPackage.version, + sqliteBindingVersion: JSON.parse(readFileSync(join(root, 'node_modules', 'better-sqlite3', 'package.json'), 'utf8')).version, + sourceCommit: head.status === 0 && status.status === 0 && !status.stdout.trim() ? head.stdout.trim() : null, + appAsarSha256: sha256(join(releaseDir, 'win-unpacked', 'resources', 'app.asar')), installer: installerName, installerSha256: installerHash, unpackedExecutable: 'win-unpacked/Novel Director.exe', @@ -145,7 +151,7 @@ console.log(`Using ELECTRON_CACHE=${env.ELECTRON_CACHE}`) console.log(`Using ELECTRON_BUILDER_CACHE=${env.ELECTRON_BUILDER_CACHE}`) const npmCli = process.env.npm_execpath || join(dirname(process.execPath), 'node_modules', 'npm', 'bin', 'npm-cli.js') -const electronRebuildCli = join(root, 'node_modules', '@electron', 'rebuild', 'lib', 'cli.js') +const nativeManager = join(root, 'scripts', 'manage-sqlite-native.mjs') const electronBuilderCli = join(root, 'node_modules', 'electron-builder', 'cli.js') const packagedSmokeScript = join(root, 'scripts', 'smoke-packaged-app.mjs') @@ -154,9 +160,8 @@ let exitCode = 0 try { prepareReleaseOutput() run(process.execPath, [npmCli, 'run', 'build']) - console.log(`Rebuilding better-sqlite3 for Electron ${electronPackage.version} before packaging...`) - run(process.execPath, [electronRebuildCli, '-f', '-w', 'better-sqlite3', '--version', electronPackage.version]) - run(process.execPath, [electronBuilderCli, '--win', 'nsis']) + run(process.execPath, [nativeManager, 'electron']) + run(process.execPath, [electronBuilderCli, '--win', 'nsis', '--x64', '--publish', 'never']) run(process.execPath, [packagedSmokeScript, 'release']) writeReleaseMetadata() } catch (error) { @@ -165,7 +170,7 @@ try { } finally { console.log('Restoring Node.js better-sqlite3 binding after Electron packaging...') try { - runWithRetry(process.execPath, [npmCli, 'rebuild', 'better-sqlite3'], { retries: 4, delayMs: 2500 }) + runWithRetry(process.execPath, [nativeManager, 'node'], { retries: 4, delayMs: 2500 }) } catch (restoreError) { console.error(restoreError) if (exitCode === 0) exitCode = Number(restoreError?.exitCode ?? 1) diff --git a/scripts/run-tests.mjs b/scripts/run-tests.mjs index 277fc0a..6d9bd1b 100644 --- a/scripts/run-tests.mjs +++ b/scripts/run-tests.mjs @@ -55,6 +55,7 @@ const tests = [ ['validate-batch-writeback-order.mjs'], ['validate-chapter-writeback-failures.mjs'], ['validate-project-list-recent-open-order.mjs'], + ['validate-project-touch.mjs'], ['validate-appdata-project-boundaries.mjs'], ['validate-p0-stability.mjs'], ['validate-renderer-save-consistency.mjs'], diff --git a/scripts/s2-windows/installed.mjs b/scripts/s2-windows/installed.mjs new file mode 100644 index 0000000..09060a2 --- /dev/null +++ b/scripts/s2-windows/installed.mjs @@ -0,0 +1,182 @@ +import assert from 'node:assert/strict' +import { spawn } from 'node:child_process' +import { mkdir, writeFile } from 'node:fs/promises' +import { createServer } from 'node:http' +import { basename, dirname, join } from 'node:path' +import { pathToFileURL } from 'node:url' +import { extractFile } from '@electron/asar' +import { books } from '../fixtures/library-recovery.mjs' +import { cleanup, connect, evaluate, sha256 } from '../utils/long-novel-ui-driver.mjs' +import { button, click, fill, freePort, waitFor } from '../utils/qa-agent-authorization-driver.mjs' +import { repoRoot } from '../utils/repo-root.mjs' +import { agentRead, checkedState, closedHash, copyClosed, editedBody, fixture, readJson } from './readback.mjs' + +const env = process.env +assert(process.platform === 'win32' && process.arch === 'x64' && env.RUNNER_ENVIRONMENT === 'github-hosted' && + env.GITHUB_ACTIONS === 'true' && env.GITHUB_REPOSITORY === 'SHIN09-git/novel' && + env.GITHUB_REF === 'refs/heads/codex/library-s2-installer-qa' && env.GITHUB_EVENT_NAME === 'push', +'Installed-window probe is restricted to the dedicated disposable job') +const [stage, version] = process.argv.slice(2) +assert(['old-seed', 'old-reopen', 'clean-seed', 'candidate-edit', 'candidate-reopen', 'reinstall-reopen', + 'rollback-edited', 'rollback-backup'].includes(stage)) +const work = join(env.RUNNER_TEMP, 's2-work'), ev = join(env.RUNNER_TEMP, 's2-evidence') +const installed = join(work, 'Installed Novel Director'), executable = join(installed, 'Novel Director.exe') +const archive = join(installed, 'resources', 'app.asar') +const report = { stage, version, status: 'running', modelRequests: 0 } +const evidence = s => join(ev, s + '.json') +const seed = stage.endsWith('-seed'), rollback = stage.startsWith('rollback-') +const edited = ['candidate-reopen', 'reinstall-reopen', 'rollback-edited'].includes(stage) +const { manifest, backup } = await fixture() +const locked = (await readJson(join(repoRoot, 'package-lock.json'))).packages +const old = version === manifest.version +const meta = await readJson(join(env.RUNNER_TEMP, 's2-input', 'BUILD_INFO.json')) +const sink = createServer((_req, res) => { report.modelRequests++; res.writeHead(503).end() }) +const api = 'window.novelDirector', load = api + '.data.load()' +const q = selector => 'document.querySelector(' + JSON.stringify(selector) + ')' +const j = JSON.stringify +let session + +async function openWindow(profile) { + const port = await freePort() + const childEnv = {} + for (const key of ['SystemRoot','WINDIR','ComSpec','SYSTEMDRIVE','PATHEXT','NUMBER_OF_PROCESSORS', + 'PROCESSOR_ARCHITECTURE','USERPROFILE','APPDATA','LOCALAPPDATA','TEMP','TMP']) childEnv[key] = env[key] + childEnv.PATH = join(env.SystemRoot, 'System32') + childEnv.NOVEL_DIRECTOR_API_REGISTRY = join(work, 'no-registry.json') + // Normal install/upgrade/reinstall uses Electron's actual default guest userData. + // Only rollback is redirected: preview.20 must never open the upgraded original. + if (profile) childEnv.NOVEL_DIRECTOR_SMOKE_USER_DATA = profile + const args = ['--disable-gpu', '--no-first-run', '--disable-background-networking', + '--host-resolver-rules=MAP * ~NOTFOUND, EXCLUDE 127.0.0.1', '--remote-debugging-address=127.0.0.1', + '--remote-debugging-port=' + port, ...(profile ? ['--user-data-dir=' + profile] : [])] + const child = spawn(executable, args, { cwd: installed, env: childEnv, windowsHide: true, stdio: ['ignore','pipe','pipe'] }) + session = { child, port, logs: [], networkRequests: [], exceptions: [], processIds: [] } + child.on('error', e => { session.logs.push(e.message) }) + for (const stream of [child.stdout, child.stderr]) stream.on('data', chunk => session.logs.push(String(chunk))) + await connect(session) + assert(session.processIds.includes(child.pid)) + const c = session.client + await waitFor(c, api + '?.app?.getRuntimeInfo') + assert.equal(await evaluate(c, 'location.href'), pathToFileURL(join(archive, 'out', 'renderer', 'index.html')).href) + report.runtime = await evaluate(c, api + '.app.getRuntimeInfo()') + assert.equal(report.runtime.mode, 'packaged') + assert.equal(report.runtime.execPath.toLowerCase(), executable.toLowerCase()) + assert.equal(report.runtime.version, version) + assert.equal(report.runtime.electronVersion, old ? '39.8.10' : locked['node_modules/electron'].version) + if (!old) assert.equal(report.runtime.electronVersion, meta.electronVersion) + return c +} + +async function edit(c) { + const row = '[...document.querySelectorAll(".project-row")].find(r=>r.innerText.includes(' + j(books[0].name) + '))' + async function press(expr) { + await waitFor(c, '(' + expr + ')?.disabled === false') + await click(c, expr) + } + await press(button('进入', '(' + row + ')')) + await press('[...document.querySelectorAll(".sidebar button")].find(b=>b.textContent.trim()==="章节")') + await press(button('连贯阅读')) + await press(button('编辑本章')) + await waitFor(c, q('.reader-inline-editor textarea')) + await fill(c, q('.reader-inline-editor textarea'), editedBody) + await press(button('保存本章')) + await waitFor(c, q('.reader-message') + '?.innerText.includes("已保存本章正文")') +} + +let profile, database, protectedDb, protectedSha, seedReport +try { + await new Promise((ok, fail) => { sink.once('error', fail); sink.listen(57760, '127.0.0.1', ok) }) + report.asarSha256 = await sha256(archive) + assert.equal(report.asarSha256, old ? manifest.appAsarSha256 : meta.appAsarSha256.toLowerCase()) + report.installedVersion = JSON.parse(extractFile(archive, 'package.json')).version + assert.equal(report.installedVersion, version) + const sqlitePkg = join('node_modules', 'better-sqlite3') + report.sqliteVersion = JSON.parse(extractFile(archive, join(sqlitePkg, 'package.json'))).version + assert.equal(report.sqliteVersion, old ? '12.9.0' : locked['node_modules/better-sqlite3'].version) + if (!old) assert.equal(report.sqliteVersion, meta.sqliteBindingVersion) + const bindings = [] + for (const file of ['build/Release/better_sqlite3.node','prebuilds/win32-x64.node']) { + const path = join(archive + '.unpacked', sqlitePkg, file) + try { bindings.push({ file, sha256: await sha256(path) }) } + catch (e) { if (e.code !== 'ENOENT') throw e } + } + assert.equal(bindings.length, 1, 'Ambiguous/missing installed x64 SQLite binding') + report.binding = bindings[0] + if (!seed) { + seedReport = await readJson(evidence(env.ROUTE === 'clean' ? 'clean-seed' : 'old-seed')) + profile = seedReport.profile + database = seedReport.database + assert.equal((await readJson(join(profile, 'app-config.json'))).storagePath, database) + if (rollback) { + protectedDb = database + protectedSha = (await readJson(evidence('candidate-reopen'))).closedSha256 + assert.equal(await closedHash(protectedDb), protectedSha) + const snapshot = stage === 'rollback-backup' ? join(work, 'before-upgrade.sqlite') : protectedDb + const hash = stage === 'rollback-backup' ? (await readJson(evidence('old-reopen'))).closedSha256 : protectedSha + profile = join(work, stage + '-profile') + await mkdir(profile) + database = join(work, stage + '.sqlite') + assert.equal(await copyClosed(snapshot, database), hash) + await writeFile(join(profile, 'app-config.json'), j({ storagePath: database }), { flag: 'wx' }) + } + } + const c = await openWindow(rollback ? profile : undefined) + const paths = () => evaluate(c, api + '.app.getStoragePath()') + if (seed) { + await waitFor(c, q('.home-empty')) + const first = await paths() + profile = dirname(first.defaultStoragePath) + assert.equal(dirname(profile).toLowerCase(), env.APPDATA.toLowerCase()) + assert(/^(?:Novel Director|novel-director)$/i.test(basename(profile))) + assert.equal(first.storagePath, first.defaultStoragePath) + assert.equal((await evaluate(c, load)).data.projects.length, 0) + database = stage === 'old-seed' ? join(work, 'selected-s0.sqlite') : first.defaultStoragePath + const result = stage === 'old-seed' + ? await evaluate(c, api + '.app.migrateStoragePath(' + j(database) + ',' + j(backup) + ')') + : await evaluate(c, api + '.data.save(' + j(backup) + ')') + assert(result.ok && result.storagePath === database) + await c.send('Page.reload', { ignoreCache: true }) + } + await waitFor(c, 'document.querySelectorAll(".project-row").length === 2') + const actualPaths = await paths() + assert.equal(dirname(actualPaths.defaultStoragePath), profile) + assert.equal(actualPaths.storagePath, database) + Object.assign(report, { profile, database }) + async function read(editedNow) { + const loaded = await evaluate(c, load) + assert.equal(loaded.storagePath, database) + assert.equal(loaded.data.settings.baseUrl, manifest.baseUrl) + assert(!loaded.data.settings.apiKey && !loaded.data.settings.hasApiKey) + return checkedState(loaded.data, manifest.signature, editedNow) + } + report.state = await read(edited) + if (stage === 'candidate-edit') { await edit(c); report.state = await read(true) } + if (edited) assert.deepEqual(report.state, (await readJson(evidence('candidate-edit'))).state) + assert.equal(await evaluate(c, api + '.ai.getCallProgress({})'), null) + assert.deepEqual(session.networkRequests, []) + assert.deepEqual(session.exceptions, []) + await cleanup(session) + assert.equal(session.child.exitCode, 0, 'Installed application did not close cleanly') + report.closedSha256 = await closedHash(database) + if (stage === 'old-reopen') await copyClosed(database, join(work, 'before-upgrade.sqlite')) + if (stage.endsWith('reopen') || rollback) { + report.agent = agentRead(profile, database, report.state) + assert.equal(await closedHash(database), report.closedSha256) + } + assert.equal(report.modelRequests, 0) + report.status = 'passed' +} catch (e) { + report.error = e.stack + process.exitCode = 1 + console.error(e) +} finally { + try { + if (session) await cleanup(session) + if (protectedDb) assert.equal(await closedHash(protectedDb), protectedSha) + assert.equal(report.modelRequests, 0) + } catch (e) { report.cleanupError = e.message; process.exitCode = 1 } + if (process.exitCode) report.status = 'failed' + if (sink.listening) await new Promise(ok => sink.close(ok)) + if (session && report.status !== 'passed') await writeFile(join(ev, stage + '.log'), session.logs.join('')) + await writeFile(evidence(stage), j(report, null, 2) + '\n') +} diff --git a/scripts/s2-windows/qualify.ps1 b/scripts/s2-windows/qualify.ps1 new file mode 100644 index 0000000..4cded44 --- /dev/null +++ b/scripts/s2-windows/qualify.ps1 @@ -0,0 +1,127 @@ +# This entry point runs actual NSIS programs. Never use the developer's login. +$ErrorActionPreference = 'Stop' +Set-StrictMode -Version Latest +if (!$IsWindows -or $env:GITHUB_ACTIONS -cne 'true' -or $env:RUNNER_ENVIRONMENT -cne 'github-hosted' -or + $env:RUNNER_OS -cne 'Windows' -or $env:RUNNER_ARCH -cne 'X64' -or $env:GITHUB_EVENT_NAME -cne 'push' -or + $env:GITHUB_REPOSITORY -cne 'SHIN09-git/novel' -or $env:GITHUB_REF -cne 'refs/heads/codex/library-s2-installer-qa') { + throw 'Refusing Setup outside the dedicated GitHub-hosted Windows QA job' +} +$route = $env:ROUTE +if ($route -cnotin @('clean','upgrade')) { throw 'Unknown route' } +$old = '0.1.6-preview.20' +$oldHash = 'aa38b6dbcba629591113d928617f4ea82ee9c9dcfdfd199e67a7a9ce4c0d50ce' +$oldUrl = 'https://github.com/SHIN09-git/novel/releases/download/v0.1.6-preview.20/Novel.Director.Setup.0.1.6-preview.20.exe' +$candidate = (Get-Content -Raw ./package.json | ConvertFrom-Json).version +$inputDir = Join-Path $env:RUNNER_TEMP 's2-input' +$work = Join-Path $env:RUNNER_TEMP 's2-work' +$ev = Join-Path $env:RUNNER_TEMP 's2-evidence' +$install = Join-Path $work 'Installed Novel Director' +$exe = Join-Path $install 'Novel Director.exe' +$uninstaller = Join-Path $install 'Uninstall Novel Director.exe' +$evidence = [ordered]@{ route=$route; commit=$env:GITHUB_SHA; run=$env:GITHUB_RUN_ID; attempt=$env:GITHUB_RUN_ATTEMPT + artifact=$env:SETUP_ARTIFACT; candidate=$candidate; image=$env:ImageVersion; status='running'; steps=@() } +function Record($name, $details) { + $evidence.steps += [ordered]@{ at=[datetime]::UtcNow.ToString('o'); name=$name; details=$details } + $evidence | ConvertTo-Json -Depth 9 | Set-Content -LiteralPath (Join-Path $ev 'qualification.json') -Encoding utf8NoBOM +} +function Hash($path) { (Get-FileHash -LiteralPath $path -Algorithm SHA256).Hash.ToLowerInvariant() } +function Check-Hash($path, $expected) { + $actual = Hash $path + if ($expected -notmatch '^[0-9a-f]{64}$' -or $actual -ne $expected) { throw "SHA-256 mismatch: $path" } + return $actual +} +function Registration($hive) { + $branch = "${hive}:\Software\Microsoft\Windows\CurrentVersion\Uninstall" + if (!(Test-Path -LiteralPath $branch)) { return } + Get-ItemProperty "$branch\*" | + Where-Object { $_.PSObject.Properties['DisplayName'] -and $_.DisplayName -match '^Novel Director(?: |$)' } +} +function Assert-Absent { + if (@(Registration HKCU).Count -or @(Registration HKLM).Count -or (Test-Path -LiteralPath $exe) -or + (Test-Path -LiteralPath $uninstaller)) { throw 'Expected no installed program or uninstall registration' } +} +function Run-NSIS($file, $arguments) { + $p = Start-Process -FilePath $file -ArgumentList $arguments -PassThru -Wait -WindowStyle Hidden + Record 'NSIS exit' @{ file=[IO.Path]::GetFileName($file); arguments=$arguments; exitCode=$p.ExitCode } + if ($p.ExitCode -ne 0) { throw "NSIS exit $($p.ExitCode): $file" } +} +function Install-Setup($file, $version, $expected, [switch]$InPlace) { + $hash = Check-Hash $file $expected + $arguments = '/S /currentuser' + if (!$InPlace) { Assert-Absent; $arguments += " /D=$install" } + Run-NSIS $file $arguments + $entries = @(Registration HKCU) + if ($entries.Count -ne 1 -or @(Registration HKLM).Count -ne 0 -or + $entries[0].QuietUninstallString -ine ('"' + $uninstaller + '" /currentuser /S') -or + !(Test-Path -LiteralPath $exe -PathType Leaf) -or !(Test-Path -LiteralPath $uninstaller -PathType Leaf)) { + throw "Installed files or per-user uninstall entry did not match $version" + } + $archive = Join-Path $install 'resources/app.asar' + $asarHash = if ($version -eq $old) { Hash $archive } else { Check-Hash $archive $meta.appAsarSha256 } + Record 'installed' @{ expectedVersion=$version; displayVersion=$entries[0].DisplayVersion; setupSha256=$hash + inPlace=[bool]$InPlace; executable=$exe; registration=$entries[0].PSChildName + exeSha256=(Hash $exe); asarSha256=$asarHash } +} +function Uninstall-Setup { + Run-NSIS $uninstaller '/currentuser /S' + Assert-Absent + Record 'uninstalled' @{ executableAbsent=$true; uninstallerAbsent=$true; registrationAbsent=$true } +} +function Probe($stage, $version) { + Record 'starting probe' $stage + node ./scripts/s2-windows/installed.mjs $stage $version + if ($LASTEXITCODE -ne 0) { throw "Installed product probe failed: $stage" } + Record 'passed probe' $stage +} + +New-Item -ItemType Directory -Path $ev -Force | Out-Null +try { + Record 'start' $route + if ((git rev-parse HEAD) -cne $env:GITHUB_SHA -or $LASTEXITCODE) { throw 'Checkout does not match push' } + New-Item -ItemType Directory -Path $work -ErrorAction Stop | Out-Null + Assert-Absent + foreach ($name in @('Novel Director','novel-director')) { + if (Test-Path -LiteralPath (Join-Path $env:APPDATA $name)) { throw "Not a clean guest profile: $name" } + } + $name = "Novel Director Setup $candidate.exe" + $setup = Join-Path $inputDir $name + $meta = Get-Content -Raw -LiteralPath (Join-Path $inputDir 'BUILD_INFO.json') | ConvertFrom-Json + $sum = (Get-Content -Raw -LiteralPath (Join-Path $inputDir 'SHA256SUMS.txt')).TrimEnd() + $hash = Check-Hash $setup $meta.installerSha256 + if ($meta.installer -cne $name -or $meta.sourceCommit -cne $env:GITHUB_SHA -or + $meta.version -cne $candidate -or $meta.packagedSmokeTest -cne 'passed' -or + $sum -ine "$hash $name" -or $hash -eq $oldHash -or $candidate -eq $old) { throw 'Candidate build record mismatch' } + Record 'candidate input verified' @{ sha256=$hash; name=$name; buildInfo=$meta } + if ($route -eq 'clean') { + Install-Setup $setup $candidate $hash + Probe clean-seed $candidate + } else { + $previous = Join-Path $work 'published-preview20.exe' + Invoke-WebRequest -Uri $oldUrl -OutFile $previous + Install-Setup $previous $old $oldHash + Probe old-seed $old + Probe old-reopen $old + Install-Setup $setup $candidate $hash -InPlace + } + Probe candidate-edit $candidate + Probe candidate-reopen $candidate + Uninstall-Setup + if ($route -eq 'clean') { + Install-Setup $setup $candidate $hash + Probe reinstall-reopen $candidate + Uninstall-Setup + } else { + Install-Setup $previous $old $oldHash + # Even if edited-copy compatibility fails, still exercise backup recovery. + try { Probe rollback-edited $old } + finally { Probe rollback-backup $old } + Uninstall-Setup + } + $evidence.status = 'passed' +} catch { + $evidence.status = 'failed' + $evidence.error = $_.Exception.Message + throw +} finally { + Record 'finish' $evidence.status +} diff --git a/scripts/s2-windows/readback.mjs b/scripts/s2-windows/readback.mjs new file mode 100644 index 0000000..0a1990c --- /dev/null +++ b/scripts/s2-windows/readback.mjs @@ -0,0 +1,104 @@ +import assert from 'node:assert/strict' +import { execFileSync } from 'node:child_process' +import { constants } from 'node:fs' +import { copyFile, readFile, stat } from 'node:fs/promises' +import { join } from 'node:path' +import { books, digest, revisedBody, signature } from '../fixtures/library-recovery.mjs' +import { sha256 } from '../utils/long-novel-ui-driver.mjs' +import { repoRoot } from '../utils/repo-root.mjs' + +export const fixtureDir = join(repoRoot, 'scripts', 'fixtures', 'library-preview20') +export const editedBody = revisedBody + '\n\nS2: 06:15.' +export const readJson = async path => JSON.parse(await readFile(path, 'utf8')) +export async function fixture() { + const m = await readJson(join(fixtureDir, 'manifest.json')) + for (const [file, hash, pinned] of [ + ['library.sqlite', m.sqliteSha256, '6e1b99178daeb837fcbd20e31e981cb74fc5fc01fc72f1116aeae239a001bcc6'], + ['backup.json', m.backupSha256, 'c436210de250b5aff90f4e1bff209aabc7ca10329dc39ae9840f40ef7e5b6050'] + ]) { + assert.equal(hash, pinned) + assert.equal(await sha256(join(fixtureDir, file)), hash) + } + const backup = await readJson(join(fixtureDir, 'backup.json')) + assert.equal(m.baseUrl, 'http://127.0.0.1:57760/v1') + assert.equal(backup.settings.baseUrl, m.baseUrl) + assert.deepEqual(signature(backup), m.signature) + return { manifest: m, backup } +} + +// Check the new head and receipt separately, then run the original S0 assertions +// on the historical slice, including the other book's unfinished checkpoint. +export function checkedState(data, expected, edited) { + for (const book of books) assert.equal(data.projects.find(p => p.id === book.id)?.name, book.name) + if (!edited) { + assert.deepEqual(signature(data), expected) + return expected + } + const oldIds = expected.versions.map(v => v.id) + const newVersions = data.chapterVersions.filter(v => !oldIds.includes(v.id)) + const receipts = data.revisionCommitBundles.filter(r => r.revisionCommitId !== expected.revisionId) + assert.equal(newVersions.length, 1) + assert.equal(receipts.length, 1) + const [v] = newVersions, [r] = receipts + for (const item of [v, r]) { + assert.equal(item.projectId, books[0].id) + assert.equal(item.chapterId, expected.chapterId) + assert.equal(item.baseChapterVersionId, oldIds.at(-1)) + assert.equal(item.linkedChapterCommitId, expected.acceptanceId) + } + assert(v.id && r.revisionCommitId) + assert.equal(v.id, r.newChapterVersionId) + assert.equal(v.linkedRevisionCommitId, r.revisionCommitId) + assert.equal(v.source, 'manual_revision') + assert.equal(r.beforeText, revisedBody) + assert.equal(data.chapters.length, 1) + for (const text of [data.chapters[0].body, r.afterText, v.body, r.chapter.body, r.chapterVersion.body]) { + assert.equal(text, editedBody) + } + const original = signature({ ...data, chapters: [{ ...data.chapters[0], body: revisedBody }], + chapterVersions: data.chapterVersions.filter(x => x !== v), revisionCommitBundles: data.revisionCommitBundles.filter(x => x !== r) }) + assert.deepEqual(original, expected) + return { ...original, chapterSha256: digest(editedBody), latestRevisionId: r.revisionCommitId, + versions: [...original.versions, { id: v.id, base: v.baseChapterVersionId, source: v.source, bodySha256: digest(v.body) }] } +} + +export async function closedHash(path) { + // A plain DB copy is valid only after a clean close with no outstanding journal. + for (const suffix of ['-wal', '-journal']) { + const sidecar = await stat(path + suffix).catch(e => { if (e.code !== 'ENOENT') throw e }) + assert(!sidecar?.size, 'Uncheckpointed SQLite sidecar: ' + suffix) + } + return sha256(path) +} + +export async function copyClosed(source, destination) { + const hash = await closedHash(source) + await copyFile(source, destination, constants.COPYFILE_EXCL) + assert.equal(await sha256(destination), hash) + return hash +} + +export function agentRead(profile, database, state) { + const env = { SystemRoot: process.env.SystemRoot, WINDIR: process.env.WINDIR, PATH: process.env.PATH, + APPDATA: profile, LOCALAPPDATA: profile, HOME: profile, USERPROFILE: profile, TEMP: profile, TMP: profile, + NOVEL_DIRECTOR_API_REGISTRY: join(profile, 'no-registry.json') } + function call(...args) { + const result = JSON.parse(execFileSync(process.execPath, [join(repoRoot, 'scripts', 'run-agent.mjs'), + ...args, '--user-data', profile], { cwd: repoRoot, env, encoding: 'utf8', windowsHide: true, timeout: 60000 })) + assert.equal(result.ok, true) + assert.equal(result.storage.source, 'sqlite') + assert.equal(result.storage.storagePath, database) + return result.data + } + assert.deepEqual(call('list-projects').map(p => p.id).sort(), state.projectIds) + const chapter = ['--project-id', books[0].id, '--chapter', '1'] + const text = call('chapter-text', ...chapter, '--detail', 'full') + assert.equal(text.id, state.chapterId) + assert.equal(digest(text.text), state.chapterSha256) + const chain = call('version-chain', ...chapter) + assert.equal(chain.chapterId, state.chapterId) + assert.deepEqual(chain.versions.map(v => v.id), state.versions.map(v => v.id).reverse()) + assert.deepEqual(chain.versions.filter(v => v.isCurrent).map(v => v.id), [state.versions.at(-1).id]) + return { node: process.version, source: 'sqlite', storagePath: database, projectIds: state.projectIds, + chapterSha256: digest(text.text), versionIds: chain.versions.map(v => v.id) } +} diff --git a/scripts/s2-windows/readback.test.mjs b/scripts/s2-windows/readback.test.mjs new file mode 100644 index 0000000..a560db9 --- /dev/null +++ b/scripts/s2-windows/readback.test.mjs @@ -0,0 +1,72 @@ +import assert from 'node:assert/strict' +import { mkdir, mkdtemp, realpath, writeFile } from 'node:fs/promises' +import { join } from 'node:path' +import { test } from 'node:test' +import { pathToFileURL } from 'node:url' +import { build } from 'esbuild' +import { repoRoot } from '../utils/repo-root.mjs' +import { agentRead, checkedState, closedHash, copyClosed, editedBody, fixture, fixtureDir } from './readback.mjs' + +test('S2 synthetic readback only (no installer, desktop, real profile or model)', async t => { + const { manifest, backup } = await fixture() + const tmp = join(repoRoot, 'tmp') + await mkdir(tmp, { recursive: true }) + assert.equal(await realpath(tmp), tmp) + const work = await mkdtemp(join(tmp, 's2-readback-')) + const profile = join(work, 'profile'), database = join(work, 'selected.sqlite') + const config = join(profile, 'app-config.json'), saved = join(work, 'before-upgrade.sqlite') + await mkdir(profile) + await copyClosed(join(fixtureDir, 'library.sqlite'), database) + await copyClosed(database, saved) + await writeFile(config, JSON.stringify({ storagePath: database }), { flag: 'wx' }) + const base = manifest.signature + await t.test('fixed preview.20 database and independently selected Agent database agree', () => { + assert.deepEqual(checkedState(backup, base, false), base) + agentRead(profile, database, base) + }) + const bundle = join(work, 'revision-service.mjs') + await build({ stdin: { resolveDir: repoRoot, contents: [ + "export { buildRevisionCommitBundle } from './src/services/RevisionCommitBundleService'", + "export { SqliteStorageService } from './src/storage/SqliteStorageService'" + ].join('\n') }, bundle: true, format: 'esm', platform: 'node', external: ['electron','better-sqlite3'], outfile: bundle }) + const { SqliteStorageService, buildRevisionCommitBundle } = await import(pathToFileURL(bundle)) + const open = () => new SqliteStorageService(database, { mode: 'existing' }) + const db = open() + try { + const { data, revision } = await db.loadSnapshot() + const record = buildRevisionCommitBundle({ appData: data, projectId: 's0-quay', chapterId: base.chapterId, + newChapterVersionId: 's2-test-version', revisionCommitId: 's2-test-revision', revisedBy: 'user', + revisedAt: '2030-01-01T00:00:00.000Z', afterText: editedBody }) + await db.saveRevisionCommitBundle(record, revision) + } finally { db.close() } + const reopened = open() + let edited + try { edited = (await reopened.loadSnapshot()).data } finally { reopened.close() } + await t.test('persisted edit, third version, receipt, checkpoint and independent Node CLI read', () => { + const actual = checkedState(edited, base, true) + assert.equal(actual.latestRevisionId, 's2-test-revision') + assert.equal(actual.versions.at(-1).id, 's2-test-version') + agentRead(profile, database, actual) + }) + await t.test('wrong body, broken parent, changed draft, missing book and lost checkpoint are rejected', () => { + for (const corrupt of [d => { d.chapters[0].body += 'changed' }, + d => { d.chapterVersions.find(v => v.id === 's2-test-version').baseChapterVersionId = null }, + d => { d.chapterCommitBundles[0].generatedDraftId = 'wrong' }, d => d.projects.pop(), + d => { d.chapterGenerationSteps.find(s => s.id === base.unfinished.stepId).status = 'completed' }]) { + const copy = structuredClone(edited) + corrupt(copy) + assert.throws(() => checkedState(copy, base, true)) + } + }) + await t.test('restores elsewhere; preserves updated file; rejects stale WAL and overwrite', async () => { + const original = await closedHash(database), restored = join(work, 'restored.sqlite') + assert.notEqual(original, manifest.sqliteSha256) + assert.equal(await copyClosed(saved, restored), manifest.sqliteSha256) + await writeFile(config, JSON.stringify({ storagePath: restored })) + agentRead(profile, restored, base) + assert.equal(await closedHash(database), original) + await assert.rejects(copyClosed(saved, restored), { code: 'EEXIST' }) + await writeFile(saved + '-wal', 'unfinished synthetic WAL', { flag: 'wx' }) + await assert.rejects(copyClosed(saved, join(work, 'unsafe.sqlite')), /Uncheckpointed/) + }) +}) diff --git a/scripts/validate-agent-revision-tools.mjs b/scripts/validate-agent-revision-tools.mjs index 010880d..2a08ba5 100644 --- a/scripts/validate-agent-revision-tools.mjs +++ b/scripts/validate-agent-revision-tools.mjs @@ -162,8 +162,8 @@ try { } } const storage = Object.create(api.SqliteStorageService.prototype) Object.assign(storage, { - storagePath, open: async () => db, isEmpty: () => false, - readAppData: () => copy(state.data), currentRevision: () => state.revision, + storagePath, open: async () => db, + readRevisionCommitContext: () => copy(state.data), currentRevision: () => state.revision, writeEntityEntries: (_db, entries) => { assert.ok(inTransaction) for (const { collection, value } of entries) { diff --git a/scripts/validate-backend-backup-laziness.mjs b/scripts/validate-backend-backup-laziness.mjs index 912f6b6..b89eef1 100644 --- a/scripts/validate-backend-backup-laziness.mjs +++ b/scripts/validate-backend-backup-laziness.mjs @@ -3,6 +3,8 @@ import assert from 'node:assert/strict' import { mkdir, mkdtemp, readFile, readdir, rm } from 'node:fs/promises' import { isAbsolute, join, relative } from 'node:path' import { pathToFileURL } from 'node:url' +import { gunzipSync } from 'node:zlib' +import { checkBackupFormats } from './checks/backup-formats.mjs' import { build } from 'esbuild' import { repoRoot } from './utils/repo-root.mjs' import { createLongNovelFixture } from './fixtures/long-novel-performance.mjs' @@ -17,6 +19,12 @@ const test = async (name, run) => { await run(); checks++; console.log(`PASS ${n try { await build({ stdin: { resolveDir: repoRoot, loader: 'ts', contents: ` export { BackupService } from './src/main/BackupService'; + export { readBackupJson } from './src/main/backupFileFormat'; + export { registerBackupIpcHandlers } from './src/main/ipc/backupIpcHandlers'; + export { registerLibraryRecoveryIpc } from './src/main/ipc/libraryRecoveryIpc'; + export { AppConfigService } from './src/main/AppConfigService'; + export { resolveAgentRuntimeStoragePath } from './src/agent/agentLibraryChoice'; + export { LibrarySession } from './src/main/LibrarySession'; export { registerDataIpcHandlers } from './src/main/ipc/dataIpcHandlers'; export { SqliteStorageService } from './src/storage/SqliteStorageService'; export { normalizeAppData } from './src/shared/defaults'; @@ -30,7 +38,7 @@ try { export * from 'node:fs/promises'; import { writeFile as realWriteFile, rename as realRename } from 'node:fs/promises'; export async function rename(from, to) { - if (globalThis.__backupRenameFailure && String(from).includes('rename-backup')) { + if (globalThis.__backupRenameFailure && String(to).endsWith('.json.gz')) { throw Object.assign(new Error('isolated rename denied'), { code: 'EACCES' }); } return realRename(from, to); @@ -43,11 +51,24 @@ try { return realWriteFile(path, data, options); } `, loader: 'js' })) + builder.onResolve({ filter: /^node:fs$/ }, args => args.importer.endsWith('BackupService.ts') + ? { path: 'fs', namespace: 'gzip-output' } : undefined) + builder.onLoad({ filter: /.*/, namespace: 'gzip-output' }, () => ({ contents: ` + export * from 'node:fs'; + import { createWriteStream as create, open, close, write } from 'node:fs'; + export function createWriteStream(path, opts) { + if (!globalThis.__gzipDiskFull) return create(path, opts); + return create(path, { ...opts, fs: { open, close, write(fd, buffer, offset, len, pos, cb) { + write(fd, buffer, offset, Math.min(len, 2), pos, err => + cb(err ?? Object.assign(new Error('gzip disk full'), { code: 'ENOSPC' }))); + } } }); + } + `, loader: 'js' })) builder.onResolve({ filter: /^electron$/ }, () => ({ path: 'electron', namespace: 'isolated' })) builder.onResolve({ filter: /\/LogService$/ }, () => ({ path: 'logger', namespace: 'isolated' })) builder.onLoad({ filter: /.*/, namespace: 'isolated' }, ({ path }) => ({ contents: path === 'logger' ? 'export const LogService = {info(){},warn(){},error(){}};' - : 'export const app={}; export const dialog={}; export const ipcMain={handle(name,fn){globalThis.__backupTestHandlers.set(name,fn)}};', loader: 'js' })) + : 'export const app={}, shell={}; export const dialog=globalThis.__backupDialogs={}; export const ipcMain={handle(name,fn){globalThis.__backupTestHandlers.set(name,fn)}};', loader: 'js' })) } }] }) const api = await import(pathToFileURL(entry).href) const data = api.normalizeAppData(createLongNovelFixture(10)) @@ -58,7 +79,7 @@ try { const results = await Promise.all(Array.from({ length: 12 }, () => service.maybeCreateAutomaticBackup(load))) assert.equal(loads, 1) assert.equal(new Set(results).size, 1) - assert.equal((await service.listBackups()).length, 1) + assert.equal((await service.listBackups()).filter(b => b.isAutomatic).length, 1) assert.deepEqual((await service.loadBackup(results[0])).chapters, data.chapters) }) await test('different storage scopes publishing in the same millisecond keep both backups', async () => { @@ -75,7 +96,8 @@ try { }), {}))) } finally { Date.now = now } assert.notEqual(paths[0], paths[1]) - assert.equal((await list()).length, 2) + assert.equal((await list()).filter(b => b.isAutomatic).length, 2) + assert.equal((await list()).filter(b => b.isRollback).length, 1) assert.equal((await simultaneous.loadBackup(paths[0])).projects[0].name, 'A') assert.equal((await simultaneous.loadBackup(paths[1])).projects[0].name, 'B') }) @@ -84,7 +106,7 @@ try { assert.equal(loads, 1) }) await test('deleting a backup is observed without stale schedule caching', async () => { - await service.deleteBackup((await service.listBackups())[0].path) + await service.deleteBackup((await service.listBackups()).find(b => b.isAutomatic).path) assert.ok(await service.maybeCreateAutomaticBackup(load)) assert.equal(loads, 2) }) @@ -116,7 +138,7 @@ try { assert.deepEqual(await readdir(failed.getBackupDir()), []) const path = await failed.maybeCreateAutomaticBackup(load) assert.ok(path) - assert.equal(JSON.parse(await readFile(path, 'utf8')).chapters.length, 10) + assert.equal(JSON.parse(gunzipSync(await readFile(path))).chapters.length, 10) }) await test('failed atomic publish cleans the temporary file and allows retry', async () => { const failed = new api.BackupService(join(work, 'rename-backup')) @@ -130,7 +152,7 @@ try { const legacy = new api.BackupService(join(work, 'eager')) const path = await legacy.maybeCreateAutomaticBackup({ ...data, settings: { ...data.settings, apiKey: 'TEST_BACKUP_SECRET' } }) assert.ok(path) - const text = await readFile(path, 'utf8') + const text = gunzipSync(await readFile(path)).toString('utf8') assert.ok(!text.includes('TEST_BACKUP_SECRET')) assert.equal(JSON.parse(text).settings.apiKey, '') }) @@ -153,7 +175,14 @@ try { assert.equal(fullReads, 0) const byId = (items) => [...items].sort((a, b) => a.id.localeCompare(b.id)) assert.deepEqual(byId((await storage.loadSnapshot()).data.chapters), byId(data.chapters)) - assert.equal((await backupService.listBackups()).length, 1) + const before = await storage.loadSnapshot() + const { chapterGenerationSteps, ...projection } = before.data + assert.ok(chapterGenerationSteps.length) + const invalid = await handlers.get(api.IPC_CHANNELS.STORAGE_SAVE)({}, { data: projection, expectedRevision: before.revision }) + assert.equal(invalid.ok, false, 'a partial query must not replace the library') + assert.match(invalid.error, /完整书库/) + assert.deepEqual(await storage.loadSnapshot(), before) + assert.equal((await backupService.listBackups()).filter(b => b.isAutomatic).length, 1) }) await test('storage switch during backup prevents old reads/writes and does not coalesce the new database', async () => { const first = new api.SqliteStorageService(join(work, 'switch-a.sqlite')) @@ -243,12 +272,15 @@ try { assert.deepEqual(actualCommand.actor, { kind: 'user' }) assert.equal(actualRevision, 'known-revision') }) + await checkBackupFormats({ api, work, handlers, test }) console.log(`Backend backup validation: ${checks}/${checks} passed (native SQLite and real main handlers).`) } finally { storage?.close() delete globalThis.__backupTestHandlers delete globalThis.__backupPartialWriteFailure delete globalThis.__backupRenameFailure + delete globalThis.__gzipDiskFull + delete globalThis.__backupDialogs const rel = relative(tempRoot, work) assert(rel && !rel.startsWith('..') && !isAbsolute(rel)) await rm(work, { recursive: true, force: true }) diff --git a/scripts/validate-book-progress-ui.mjs b/scripts/validate-book-progress-ui.mjs index 5e2a6d1..97e3641 100644 --- a/scripts/validate-book-progress-ui.mjs +++ b/scripts/validate-book-progress-ui.mjs @@ -77,6 +77,11 @@ function installBridge(seed) { const qa = window.progressQA window.novelDirector = { data: { load: async () => ({ data: qa.data, storagePath: 'synthetic-progress' }), + touchProject: async (id) => { + const updatedAt = new Date().toISOString() + qa.data = { ...qa.data, projects: qa.data.projects.map(p => p.id === id ? { ...p, lastOpenedAt: updatedAt } : p) } + return { updatedAt, storagePath: 'synthetic-progress' } + }, save: async (data) => { qa.writes++ if (qa.fail) throw new Error('合成保存失败') diff --git a/scripts/validate-core-modularization.mjs b/scripts/validate-core-modularization.mjs index 5a62af5..0835d75 100644 --- a/scripts/validate-core-modularization.mjs +++ b/scripts/validate-core-modularization.mjs @@ -247,7 +247,7 @@ assertLineLimit('src/renderer/src/views/revision/revisionGenerationActions.ts', assertLineLimit('src/renderer/src/views/revision/revisionVersionActions.ts', 300) assertLineLimit('src/main/DataMergeService.ts', 380) assertLineLimit('src/main/ipc/appDataCredentialSanitizer.ts', 90) -assertLineLimit('src/storage/SqliteStorageService.ts', 420) +assertIncludes('src/storage/SqliteStorageService.ts', "from './sqlite/sqliteCommitContext'") assertLineLimit('src/agent/AgentReadableSummaryService.ts', 180) assertLineLimit('src/agent/tools/AgentToolService.ts', 80) assertLineLimit('src/services/NoveltyDetector.ts', 340) diff --git a/scripts/validate-director-dashboard.mjs b/scripts/validate-director-dashboard.mjs index a504190..f37c311 100644 --- a/scripts/validate-director-dashboard.mjs +++ b/scripts/validate-director-dashboard.mjs @@ -147,6 +147,10 @@ async function verifyBrowser() { data: { load: async () => ({ data: window.__demo, storagePath: 'isolated-demo' }), save: async (data) => { window.__writes++; await new Promise(r => setTimeout(r, 120)); if(window.__failSave) throw new Error('QA 模拟保存失败'); window.__demo = data; return { storagePath: 'isolated-demo' }; + }, touchProject: async (id) => { + const updatedAt = new Date().toISOString(); + window.__demo.projects = window.__demo.projects.map(p => p.id === id ? { ...p, lastOpenedAt: updatedAt } : p); + return { updatedAt, storagePath: 'isolated-demo' }; } }, ai: { chatCompletion: async () => { throw new Error('No provider calls in dashboard QA') }, cancelRun: async () => ({ ok: true }) } };` }) diff --git a/scripts/validate-project-list-recent-open-order.mjs b/scripts/validate-project-list-recent-open-order.mjs index bbc023a..ae837f2 100644 --- a/scripts/validate-project-list-recent-open-order.mjs +++ b/scripts/validate-project-list-recent-open-order.mjs @@ -28,7 +28,8 @@ const checks = [ }, { ok: - /markProjectOpened/.test(homeView) && + /await touchProject\(projectId\)/.test(homeView) && + /markProjectOpened\(current, projectId, saved.updatedAt\)/.test(read('src/renderer/src/hooks/useAppData.ts')) && /lastOpenedAt:\s*openedAt/.test(projectLifecycle), message: 'Opening a project records lastOpenedAt' }, diff --git a/scripts/validate-project-touch.mjs b/scripts/validate-project-touch.mjs new file mode 100644 index 0000000..4761722 --- /dev/null +++ b/scripts/validate-project-touch.mjs @@ -0,0 +1,157 @@ +import assert from 'node:assert/strict' +import { mkdir, mkdtemp, readFile, rm, writeFile } from 'node:fs/promises' +import { dirname, join } from 'node:path' +import { pathToFileURL } from 'node:url' +import Database from 'better-sqlite3' +import { build } from 'esbuild' +import { createLongNovelFixture } from './fixtures/long-novel-performance.mjs' +import { repoRoot } from './utils/repo-root.mjs' + +const parent = join(repoRoot, 'tmp') +await mkdir(parent, { recursive: true }) +const work = await mkdtemp(join(parent, 'project-touch-')) +const entry = join(work, 'touch.mjs') +let storage, inspect +try { + await build({ stdin: { resolveDir: repoRoot, contents: ` + export { normalizeAppData } from './src/shared/defaults' + export { createExistingLibraryStorage, createNewLibraryStorage } from './src/storage/SqliteStorageService' + export { loadAgentRuntimeData, saveAgentRuntimeData } from './src/agent/AgentRuntime' + export { listProjects } from './src/agent/agentProjectSummaries' + export { projectsByRecentOpen } from './src/renderer/src/views/home/homeProjectState' + export { registerDataIpcHandlers } from './src/main/ipc/dataIpcHandlers' + export { IPC_CHANNELS } from './src/shared/ipc/ipcChannels' + ` }, outfile: entry, bundle: true, platform: 'node', format: 'esm', packages: 'external', + plugins: [{ name: 'isolated-main', setup(b) { + b.onResolve({ filter: /^electron$|\/LogService$/ }, ({ path }) => ({ path, namespace: 'isolated' })) + b.onLoad({ filter: /.*/, namespace: 'isolated' }, ({ path }) => ({ contents: path === 'electron' + ? 'export const app={}, dialog={}, ipcMain={handle:(k,v)=>globalThis.handlers.set(k,v)}' + : 'export const LogService={info(){},warn(){},error(){}}' })) + } }] }) + const api = await import(pathToFileURL(entry)) + const seed = api.normalizeAppData(createLongNovelFixture(3)) + const id = seed.projects[1].id + seed.projects[1].updatedAt = '2000-01-01T00:00:00.000Z' + seed.projects[1].createdAt = seed.projects[1].updatedAt + const ordered = projects => projects.map(p => p.id) + const firstOrder = ordered(api.projectsByRecentOpen(seed.projects)) + assert.notEqual(firstOrder[0], id) + const conflict = { code: 'STORAGE_REVISION_CONFLICT' } + for (const ext of ['json', 'sqlite']) { + const path = join(work, 'library.' + ext), options = { storagePath: path, userDataPath: work } + await writeFile(path, '', { flag: 'wx' }) + storage = api.createNewLibraryStorage(path, seed) + const agent = await api.loadAgentRuntimeData(options) + const edited = { ...agent.data, chapters: agent.data.chapters.map(c => ({ ...c, title: c.title + ' edited' })) } + const stale = agent.revision + await api.saveAgentRuntimeData(edited, agent) + const before = await storage.loadSnapshot() + await assert.rejects(storage.touchProject(id, stale), conflict) + await assert.rejects(storage.touchProject(id, undefined), conflict) + await assert.rejects(storage.touchProject('missing', before.revision), /项目不存在/) + assert.deepEqual(await storage.loadSnapshot(), before) + + let rows, settings, raw + if (ext === 'sqlite') { + inspect = new Database(path, { fileMustExist: true }) + rows = inspect.prepare('SELECT * FROM entities ORDER BY collection,id').all() + settings = inspect.prepare('SELECT * FROM app_settings').all() + inspect.exec("CREATE TRIGGER fail_meta BEFORE INSERT ON meta WHEN NEW.key='revision' BEGIN SELECT RAISE(ABORT,'fixture rollback'); END") + await assert.rejects(storage.touchProject(id, before.revision), /fixture rollback/) + assert.deepEqual(await storage.loadSnapshot(), before) + inspect.exec('DROP TRIGGER fail_meta') + } else { + raw = JSON.parse(await readFile(path, 'utf8')) + raw.settings.apiKey = 'synthetic-legacy-secret' + raw.settings.futureSetting = 123 + raw.projects[1].futureMetadata = { keep: true } + raw.chapters[0].futureMetadata = [3, 2, 1] + await writeFile(path, '\uFEFF' + JSON.stringify(raw)) + } + const latest = await storage.loadSnapshot() + const sameRevisionAgent = await api.loadAgentRuntimeData(options) + const queries = [] + const prepare = Database.prototype.prepare + let saved + try { + if (inspect) Database.prototype.prepare = function(sql) { + if (/\bentities\b/i.test(sql)) queries.push(sql) + return prepare.call(this, sql) + } + saved = await storage.touchProject(id, latest.revision) + } finally { Database.prototype.prepare = prepare } + if (inspect) { + assert(queries.length) + for (const sql of queries) { + const plan = inspect.prepare('EXPLAIN QUERY PLAN ' + sql).all(...Array((sql.match(/\?/g) ?? []).length).fill(null)) + assert(plan.some(r => /SEARCH entities .*\(collection=\? AND id=\?\)/.test(r.detail)), JSON.stringify(plan)) + assert(!plan.some(r => /SCAN entities/.test(r.detail)), 'no whole-library entity scan') + } + } + assert.deepEqual(saved.savedCollections, ['projects']) + assert.notEqual(saved.revision, latest.revision) + assert.equal(Date.parse(saved.updatedAt) > 0, true) + await assert.rejects(api.saveAgentRuntimeData(edited, sameRevisionAgent), conflict) + storage.close?.() + storage = api.createExistingLibraryStorage(path) + const reopened = await storage.loadSnapshot() + assert.equal(reopened.revision, saved.revision) + assert.deepEqual(reopened.data.chapters, latest.data.chapters) + assert.deepEqual(ordered(api.projectsByRecentOpen(reopened.data.projects)), [id, firstOrder[0]]) + assert.deepEqual(ordered(api.listProjects(reopened.data)), ordered(api.listProjects(latest.data)), 'Agent edit order is unchanged') + if (ext === 'sqlite') { + const row = rows.find(r => r.collection === 'projects' && r.id === id) + row.json = JSON.stringify({ ...JSON.parse(row.json), lastOpenedAt: saved.updatedAt }) + assert.deepEqual(inspect.prepare('SELECT * FROM entities ORDER BY collection,id').all(), rows, + 'one project payload; all index columns, including updated_at, stay unchanged') + assert.deepEqual(inspect.prepare('SELECT * FROM app_settings').all(), settings) + inspect.close(); inspect = undefined + } else { + raw.settings.apiKey = '' + assert.deepEqual(JSON.parse(await readFile(path + '.bak', 'utf8')), raw, 'redacted pre-touch backup') + raw.projects.find(p => p.id === id).lastOpenedAt = saved.updatedAt + assert.deepEqual(JSON.parse(await readFile(path, 'utf8')), raw, 'raw fields and document order survive') + } + storage.close?.() + storage = undefined + console.log('PASS ' + ext + ': actual Agent conflicts in both directions, restart, index order, isolated update') + } + + // Copies can have identical project IDs and revisions. Bind the request to the displayed library as well. + globalThis.handlers = new Map() + let backupChecks = 0, due = false, switchDuringBackup = false, writes = 0, reads = 0 + const first = { getStoragePath: () => 'first', load: async () => { reads++ }, + touchProject: async (projectId, revision) => { writes++; return { projectId, revision } } } + const second = { ...first, getStoragePath: () => 'copy' } + let current = second + api.registerDataIpcHandlers({ getStorage: () => current, backupService: { + async maybeCreateAutomaticBackup(load) { + backupChecks++ + if (due) await load() + if (switchDuringBackup) current = second + } + } }) + const invoke = (request = {}) => globalThis.handlers.get(api.IPC_CHANNELS.DATA_TOUCH_PROJECT)({}, { + projectId: id, expectedRevision: 'same-revision', expectedStoragePath: 'first', ...request }) + assert.equal((await invoke()).code, conflict.code) + assert.equal(backupChecks, 0) + current = first + assert.equal((await invoke({ expectedRevision: undefined })).code, conflict.code) + assert.equal((await invoke({ projectId: ' ' })).ok, false) + assert.equal(backupChecks, 0) + assert.deepEqual(await invoke(), { projectId: id, revision: 'same-revision' }) + assert.equal(reads, 0) + due = true + await invoke() + assert.equal(reads, 1) + switchDuringBackup = true + assert.equal((await invoke()).ok, false) + assert.equal(writes, 2) + console.log('PASS IPC: library identity including equal revisions, due/non-due backup and switch during backup') +} finally { + inspect?.close() + storage?.close?.() + delete globalThis.handlers + assert.equal(dirname(work), parent) + await rm(work, { recursive: true, force: true }) +} diff --git a/scripts/validate-release-p0-readiness.mjs b/scripts/validate-release-p0-readiness.mjs index 4e3b038..936a81a 100644 --- a/scripts/validate-release-p0-readiness.mjs +++ b/scripts/validate-release-p0-readiness.mjs @@ -65,7 +65,7 @@ check('preview smoke script uses isolated userData path', /NOVEL_DIRECTOR_SMOKE_ check('preview smoke script switches better-sqlite3 to Electron and restores Node binding', /manage-sqlite-native\.mjs/.test(smokePreview) && /'electron'/.test(smokePreview) && /'node'/.test(smokePreview)) check('preview smoke script verifies SQLite backend creation', /novel-director-data\.sqlite/.test(smokePreview) && /smoke userData did not create novel-director-data\.sqlite/.test(smokePreview)) check('preview smoke failure path throws so native binding restore still runs', /function fail\(message\)[\s\S]*throw error/.test(smokePreview) && !/function fail\(message\)[\s\S]*process\.exit\(1\)/.test(smokePreview)) -check('Windows dist flow runs packaged smoke before rebuilding Node binding', /smoke-packaged-app\.mjs/.test(runDist) && /rebuild', 'better-sqlite3'/.test(runDist)) +check('Windows dist verifies the packaged app and checks the Node binding on exit', /smoke-packaged-app\.mjs/.test(runDist) && /finally\s*\{[\s\S]*nativeManager, 'node'/.test(runDist)) check('HomeView exposes a queued JSON import path', /导入 JSON/.test(homeView) && /importData\(strategy\)/.test(homeView) && !/window\.novelDirector/.test(homeView)) check('HomeView merges into non-empty workspaces instead of receiving destructive replaceData', /data\.projects\.length > 0 \? 'merge' : 'replace'/.test(homeView) && /]*importData=\{importStoredData\}/s.test(appView) && !/]*replaceData=/s.test(appView)) check('README documents better-sqlite3 native setup', /better-sqlite3/.test(readme) && /Visual Studio C\+\+ Build Tools/.test(readme) && /native:electron/.test(readme) && /native:node/.test(readme)) diff --git a/scripts/validate-unreviewed-draft-acceptance.mjs b/scripts/validate-unreviewed-draft-acceptance.mjs index eefe323..c931845 100644 --- a/scripts/validate-unreviewed-draft-acceptance.mjs +++ b/scripts/validate-unreviewed-draft-acceptance.mjs @@ -642,8 +642,7 @@ function sqliteBoundary(initial) { const storage = Object.create(modules.sqlite.SqliteStorageService.prototype) storage.storagePath = join(outDir, 'not-opened.sqlite') storage.open = async () => db - storage.isEmpty = () => false - storage.readAppData = () => copy(state.data) + storage.readChapterCommitContext = () => copy(state.data) storage.currentRevision = () => state.revision storage.writeEntityEntries = (_db, entries) => { assert.ok(inTransaction, 'Entry writes must stay inside the transaction boundary') diff --git a/scripts/verify-win-installer-payload.mjs b/scripts/verify-win-installer-payload.mjs index cc409d4..2beb1c0 100644 --- a/scripts/verify-win-installer-payload.mjs +++ b/scripts/verify-win-installer-payload.mjs @@ -57,8 +57,10 @@ try { const members = await files(unpacked) assert.deepEqual(await files(payload), members, 'Installer and unpacked program have different file lists') assert(members.includes('Novel Director.exe')) - const native = members.find(path => basename(path) === 'better_sqlite3.node') - assert(native, 'SQLite native binding missing') + const sqliteDir = join('resources', 'app.asar.unpacked', 'node_modules', 'better-sqlite3') + const native = [join(sqliteDir, 'prebuilds', 'win32-x64.node'), + join(sqliteDir, 'build', 'Release', 'better_sqlite3.node')].find(path => members.includes(path)) + assert(native, 'Windows x64 SQLite native binding missing') for (const member of members) { assert.equal(await hash(join(payload, member)), await hash(join(unpacked, member)), `File differs: ${member}`) } diff --git a/src/main/BackupService.ts b/src/main/BackupService.ts index 4923ede..568e393 100644 --- a/src/main/BackupService.ts +++ b/src/main/BackupService.ts @@ -1,26 +1,20 @@ import { randomUUID } from 'node:crypto' -import { mkdir, readdir, readFile, rename, stat, unlink, writeFile } from 'node:fs/promises' -import { isAbsolute, join, relative, resolve } from 'node:path' -import { normalizeAppData, sanitizeAppDataForPersistence } from '../shared/defaults' -import { decodeLibraryText, normalizeLibraryData, parseLibraryJsonValue } from '../storage/libraryValidation' +import { createReadStream, createWriteStream } from 'node:fs' +import { lstat, mkdir, readdir, rename, unlink, writeFile } from 'node:fs/promises' +import { dirname, join, resolve } from 'node:path' +import { pipeline } from 'node:stream/promises' +import { constants, createGzip } from 'node:zlib' +import { sanitizeAppDataForPersistence } from '../shared/defaults' +import type { BackupInfo } from '../shared/ipc/ipcTypes' +import { normalizeLibraryData } from '../storage/libraryValidation' +import { MAX_GZIP_JSON_BYTES, isBackupFile, readBackupJson } from './backupFileFormat' import type { AppData } from '../shared/types' -export interface BackupInfo { - path: string - timestamp: number - size: number - isAutomatic: boolean -} - const BACKUP_INTERVAL_MS = 24 * 60 * 60 * 1000 const MAX_AUTOMATIC_BACKUPS = 7 -function isInsideOrSame(childPath: string, parentPath: string): boolean { - const parent = resolve(parentPath) - const child = resolve(childPath) - const relationship = relative(parent, child) - return relationship === '' || (!!relationship && !relationship.startsWith('..') && !isAbsolute(relationship)) -} +const autoName = /^novel-director-\d+-auto(?:-[\da-f]{8}(?:-[\da-f]{4}){3}-[\da-f]{12})?\.json(?:\.gz)?$/i +const rollbackName = /^novel-director-\d+-rollback-[\da-f]{8}(?:-[\da-f]{4}){3}-[\da-f]{12}\.json$/i function backupTimestamp(fileName: string, fallback: number): number { const match = fileName.match(/novel-director-(\d+)/) @@ -30,8 +24,9 @@ function backupTimestamp(fileName: string, fallback: number): number { export class BackupService { private readonly backupDir: string private readonly automaticBackupsPending = new Map>() + private publication = Promise.resolve() - constructor(userDataPath: string) { + constructor(userDataPath: string, private readonly gzipLimit = MAX_GZIP_JSON_BYTES) { this.backupDir = join(userDataPath, 'backups') } @@ -44,23 +39,41 @@ export class BackupService { } async createBackup(data: AppData, isAutomatic = true): Promise { + if (!isAutomatic) return this.publishBackup(data, false) + // Concurrent saves of different libraries must not rotate a half-published pair. + const next = this.publication.then(() => this.publishBackup(data, true)) + this.publication = next.then(() => {}, () => {}) + return next + } + + private async publishBackup(data: AppData, isAutomatic: boolean): Promise { await this.ensureBackupDir() - const timestamp = Date.now() - const backupName = `novel-director-${timestamp}${isAutomatic ? '-auto' : '-manual'}-${randomUUID()}.json` - const backupPath = join(this.backupDir, backupName) - const safeData = sanitizeAppDataForPersistence(normalizeAppData(data)) - const temporaryPath = `${backupPath}.tmp` - // Publish only complete JSON; partial writes must not satisfy the daily schedule. + const prefix = join(this.backupDir, `novel-director-${Date.now()}`), id = randomUUID() + const safe = normalizeLibraryData(data, sanitizeAppDataForPersistence) + const raw = JSON.stringify(safe, null, 2) + const compress = isAutomatic && Buffer.byteLength(raw) <= this.gzipLimit + const plain = `${prefix}-${compress ? 'rollback' : isAutomatic ? 'auto' : 'manual'}-${id}.json` + const gzip = `${prefix}-auto-${id}.json.gz` + let publishedPlain = false try { - await writeFile(temporaryPath, JSON.stringify(safeData, null, 2), { encoding: 'utf-8', flag: 'wx' }) - await rename(temporaryPath, backupPath) + await writeFile(plain + '.tmp', raw, { encoding: 'utf8', flag: 'wx' }) + if (compress) { + await pipeline(createReadStream(plain + '.tmp'), createGzip({ level: constants.Z_BEST_SPEED }), + createWriteStream(gzip + '.tmp', { flags: 'wx' })) + } + await rename(plain + '.tmp', plain) + publishedPlain = true + if (compress) await rename(gzip + '.tmp', gzip) + } catch (error) { + if (publishedPlain) await unlink(plain) + throw error } finally { - await unlink(temporaryPath).catch(() => undefined) - } - if (isAutomatic) { - await this.cleanupOldAutomaticBackups() + await unlink(plain + '.tmp').catch(() => undefined) + if (compress) await unlink(gzip + '.tmp').catch(() => undefined) } - return backupPath + const path = compress ? gzip : plain + if (isAutomatic) await this.cleanupOldAutomaticBackups(path, plain) + return path } async maybeCreateAutomaticBackup(data: AppData | (() => Promise), source: object = this): Promise { @@ -88,18 +101,23 @@ export class BackupService { async listBackups(): Promise { await this.ensureBackupDir() - const files = await readdir(this.backupDir) + const files = await readdir(this.backupDir, { withFileTypes: true }) const backups: BackupInfo[] = [] - for (const file of files) { - if (!file.endsWith('.json')) continue + for (const entry of files) { + const file = entry.name + if (!entry.isFile() || !isBackupFile(file)) continue const path = join(this.backupDir, file) - const info = await stat(path) + const info = await lstat(path).catch(error => { + if (error.code !== 'ENOENT') throw error + }) + if (!info?.isFile()) continue backups.push({ path, timestamp: backupTimestamp(file, info.mtimeMs), size: info.size, - isAutomatic: file.includes('-auto') + isAutomatic: autoName.test(file), + isRollback: rollbackName.test(file) }) } @@ -108,8 +126,8 @@ export class BackupService { async loadBackup(backupPath: string): Promise { const safePath = this.assertManagedBackupPath(backupPath) - const raw = decodeLibraryText(await readFile(safePath)) - return normalizeLibraryData(parseLibraryJsonValue(raw), sanitizeAppDataForPersistence) + if (!(await lstat(safePath)).isFile()) throw new Error('请选择备份目录中的文件。') + return normalizeLibraryData(await readBackupJson(safePath, this.gzipLimit), sanitizeAppDataForPersistence) } async deleteBackup(backupPath: string): Promise { @@ -119,15 +137,16 @@ export class BackupService { private assertManagedBackupPath(backupPath: string): string { const resolved = resolve(backupPath) - if (!isInsideOrSame(resolved, this.backupDir) || !resolved.endsWith('.json')) { + if (dirname(resolved) !== resolve(this.backupDir) || !isBackupFile(resolved)) { throw new Error('Backup path is outside the managed backup folder.') } return resolved } - private async cleanupOldAutomaticBackups(): Promise { - const automaticBackups = (await this.listBackups()).filter((backup) => backup.isAutomatic) - const toDelete = automaticBackups.slice(MAX_AUTOMATIC_BACKUPS) + private async cleanupOldAutomaticBackups(current: string, plain: string): Promise { + const backups = await this.listBackups() + const toDelete = [...backups.filter(backup => backup.isAutomatic && backup.path !== current).slice(MAX_AUTOMATIC_BACKUPS - 1), + ...backups.filter(backup => backup.isRollback && backup.path !== plain)] for (const backup of toDelete) { try { await this.deleteBackup(backup.path) diff --git a/src/main/LibrarySession.ts b/src/main/LibrarySession.ts index cdd48e2..fe40f38 100644 --- a/src/main/LibrarySession.ts +++ b/src/main/LibrarySession.ts @@ -2,6 +2,9 @@ import { dirname, extname, join, relative } from 'node:path' import { JSON_DATA_FILE_NAME, resolveSqliteStoragePath, type StorageService } from '../storage/StorageService' import type { AppData } from '../shared/types' import type { AppConfigService } from './AppConfigService' +import { readBackupJson } from './backupFileFormat' +import { normalizeLibraryData } from '../storage/libraryValidation' +import { sanitizeAppDataForPersistence } from '../shared/defaults' import { backupLegacyJson, exists, hasLibraryTraces, libraryFile, libraryStores, openExistingLibrary, writeNewLibrary, type LibraryStores, type OpenLibrary } from './libraryFiles' @@ -77,9 +80,7 @@ export class LibrarySession { if (action === 'restore' && !backup) throw new Error('尚未选择备份。') let data: AppData | undefined if (backup) { - if (extname(backup).toLowerCase() !== '.json') throw new Error('请选择 JSON 书库备份。') - const source = await openExistingLibrary(backup, this.stores) - try { data = source.snapshot.data } finally { source.storage.close?.() } + data = normalizeLibraryData(await readBackupJson(backup), sanitizeAppDataForPersistence) } opened = await writeNewLibrary(target, this.stores, data) } diff --git a/src/main/backupFileFormat.ts b/src/main/backupFileFormat.ts new file mode 100644 index 0000000..c8b4e9d --- /dev/null +++ b/src/main/backupFileFormat.ts @@ -0,0 +1,46 @@ +import { createReadStream } from 'node:fs' +import { readFile } from 'node:fs/promises' +import { Transform, Writable } from 'node:stream' +import { pipeline } from 'node:stream/promises' +import { createGunzip } from 'node:zlib' +import { decodeLibraryText, parseLibraryJsonValue } from '../storage/libraryValidation' + +// Bound actual expanded bytes (not the untrusted, modulo-2^32 gzip footer). +// Above this ceiling the automatic writer uses ordinary JSON, with its legacy read path. +export const MAX_GZIP_JSON_BYTES = 384 * 1024 * 1024 +export const backupFileFilter = { name: '书库备份 (*.json, *.json.gz)', extensions: ['json', 'gz'] } +export const isBackupFile = (path: string): boolean => /\.json(?:\.gz)?$/i.test(path) + +export async function readBackupJson(path: string, limit = MAX_GZIP_JSON_BYTES): Promise { + if (!isBackupFile(path)) throw new Error('请选择 .json 或 .json.gz 书库备份。') + if (!/\.gz$/i.test(path)) return parseLibraryJsonValue(decodeLibraryText(await readFile(path))) + + const decoder = new TextDecoder('utf-8', { fatal: true }), chunks: string[] = [] + let bytes = 0, inputBytes = 0 + // Allow framing/deflate overhead even for incompressible input. Empty gzip members + // or padding should not allow unbounded disk reads with little or no JSON output. + const inputBudget = new Transform({ transform(chunk: Buffer, _encoding, done) { + inputBytes += chunk.length + if (inputBytes > limit + 1024 * 1024) done(new Error('压缩备份文件超出大小上限。请改用普通 JSON 备份。')) + else done(null, chunk) + } }) + try { + await pipeline(createReadStream(path), inputBudget, createGunzip(), new Writable({ + write(chunk: Buffer, _encoding, done) { + try { + bytes += chunk.length + if (bytes > limit) throw new Error('备份解压后的 JSON 超出大小上限。请改用普通 JSON 备份。') + chunks.push(decoder.decode(chunk, { stream: true })) + done() + } catch (error) { done(error as Error) } + } + })) + chunks.push(decoder.decode()) + } catch (error) { + const code = (error as NodeJS.ErrnoException).code + if (code === 'ERR_ENCODING_INVALID_ENCODED_DATA') throw new Error('书库 JSON 不是有效的 UTF-8 文本。') + if (code === 'Z_DATA_ERROR' || code === 'Z_BUF_ERROR') throw new Error('Gzip 备份不完整或已损坏。') + throw error + } + return parseLibraryJsonValue(chunks.join('')) +} diff --git a/src/main/ipc/backupIpcHandlers.ts b/src/main/ipc/backupIpcHandlers.ts index a3a525d..053108d 100644 --- a/src/main/ipc/backupIpcHandlers.ts +++ b/src/main/ipc/backupIpcHandlers.ts @@ -45,12 +45,8 @@ export function registerBackupIpcHandlers(context: BackupIpcContext): void { const storage = context.getStorage() const request = typeof input === 'string' ? { backupPath: input } : input const data = await context.backupService.loadBackup(validateFilePath(request.backupPath, 'Backup path')) - let preRestoreBackupPath: string | undefined - try { - preRestoreBackupPath = await context.backupService.createBackup(await storage.load(), false) - } catch (error) { - LogService.warn('Pre-restore backup failed; continuing with selected backup restore.', error) - } + // Overwriting the current library requires a usable rollback, including on a full disk. + const preRestoreBackupPath = await context.backupService.createBackup(await storage.load(), false) const saved = await storage.saveIfCurrent(data, request.expectedRevision) LogService.info(`Backup restored: ${request.backupPath}`) return { diff --git a/src/main/ipc/dataIpcHandlers.ts b/src/main/ipc/dataIpcHandlers.ts index 6773381..441f11a 100644 --- a/src/main/ipc/dataIpcHandlers.ts +++ b/src/main/ipc/dataIpcHandlers.ts @@ -1,12 +1,13 @@ import { dialog, ipcMain } from 'electron' -import { readFile, writeFile } from 'node:fs/promises' -import { decodeLibraryText, normalizeImportedLibrary, parseLibraryJsonValue } from '../../storage/libraryValidation' +import { writeFile } from 'node:fs/promises' +import { normalizeImportedLibrary } from '../../storage/libraryValidation' +import { backupFileFilter, readBackupJson } from '../backupFileFormat' import { observeFormalChapterWrites } from '../../services/ChapterWritebackFailureService' import { parseChapterTitleRename, previewChapterTitleRename } from '../../services/ChapterTitleRenameService' import { loadChapterTitlesReadonly } from '../../storage/SqliteStorageService' import { StorageRevisionConflictError } from '../../storage/StorageService' import type { ChapterTitleCommit, ChapterTitleRename } from '../../shared/types/chapterTitle' -import { collectionName } from '../../shared/appDataCollections' +import { ALL_ENTITY_COLLECTIONS, collectionName } from '../../shared/appDataCollections' import { sanitizeAppDataForPersistence } from '../../shared/defaults' import { IPC_CHANNELS } from '../../shared/ipc/ipcChannels' import { readProjectUsage } from '../../storage/readProjectUsage' @@ -35,6 +36,16 @@ import { secureAndSanitizeAppData } from './appDataCredentialSanitizer' import { assertCurrentStorage, maybeCreateAutomaticBackup } from './storageDataOperations' export function registerDataIpcHandlers(context: StorageDataOperationContext): void { + ipcMain.handle(IPC_CHANNELS.DATA_TOUCH_PROJECT, safeIpcHandler(async (_event, + request: { projectId: string; expectedRevision: string; expectedStoragePath: string }): Promise => { + if (typeof request?.projectId !== 'string' || !request.projectId.trim()) throw new Error('请选择要打开的项目。') + if (typeof request.expectedRevision !== 'string' || !request.expectedRevision) throw new StorageRevisionConflictError() + const storage = context.getStorage() + if (request.expectedStoragePath !== storage.getStoragePath()) throw new StorageRevisionConflictError() + await maybeCreateAutomaticBackup(context, 'project open', storage) + assertCurrentStorage(context, storage) + return storage.touchProject(request.projectId, request.expectedRevision) + })) ipcMain.handle(IPC_CHANNELS.DATA_PREVIEW_CHAPTER_TITLES, safeIpcHandler(async (_event, request: { command: ChapterTitleRename; expectedRevision: string }) => { const command = parseChapterTitleRename(request?.command), storage = context.getStorage() @@ -111,6 +122,11 @@ export function registerDataIpcHandlers(context: StorageDataOperationContext): v ? (input as StorageSaveRequest) : { data: input as AppData, expectedRevision: undefined } const data = request.data + // Only the desktop's complete snapshot can express deletions. Imports validate legacy formats separately. + if (!data?.settings || !Number.isInteger(data.schemaVersion) || + ALL_ENTITY_COLLECTIONS.some(key => !Object.hasOwn(data, key) || !Array.isArray(data[key]))) { + throw new Error('此操作需要完整书库,不能用局部查询结果覆盖已有数据。请重新加载。') + } const secured = await secureAndSanitizeAppData(context, data) await maybeCreateAutomaticBackup(context, 'full AppData save', storage) assertCurrentStorage(context, storage) @@ -184,14 +200,14 @@ export function registerDataIpcHandlers(context: StorageDataOperationContext): v const result = await dialog.showOpenDialog({ title: '导入 Novel Director 数据', properties: ['openFile'], - filters: [{ name: 'JSON', extensions: ['json'] }] + filters: [backupFileFilter] }) if (result.canceled || !result.filePaths[0]) return { canceled: true } const storage = context.getStorage() const importPath = result.filePaths[0] const strategy = request?.strategy === 'merge' ? 'merge' : 'replace' - const imported = normalizeImportedLibrary(parseLibraryJsonValue(decodeLibraryText(await readFile(importPath))), strategy) + const imported = normalizeImportedLibrary(await readBackupJson(importPath), strategy) const secured = await secureAndSanitizeAppData(context, imported, { migrateLegacyApiKey: false }) diff --git a/src/main/ipc/libraryRecoveryIpc.ts b/src/main/ipc/libraryRecoveryIpc.ts index 06fc108..d8ef7b3 100644 --- a/src/main/ipc/libraryRecoveryIpc.ts +++ b/src/main/ipc/libraryRecoveryIpc.ts @@ -2,6 +2,7 @@ import { dialog, ipcMain } from 'electron' import { join } from 'node:path' import { IPC_CHANNELS } from '../../shared/ipc/ipcChannels' import type { LibrarySession } from '../LibrarySession' +import { backupFileFilter } from '../backupFileFormat' import { safeIpcHandler } from './safeIpcHandler' import { assertSafeDataStoragePath, isPathInsideOrSame } from './storagePath' @@ -22,7 +23,7 @@ export function registerLibraryRecoveryIpc(session: LibrarySession, userData: st const chosen = await dialog.showOpenDialog({ title: action === 'restore' ? '选择要恢复的 JSON 备份' : '选择已有书库', defaultPath: action === 'restore' ? backups : undefined, properties: ['openFile'], - filters: [{ name: action === 'restore' ? 'JSON 书库备份' : '书库文件', extensions: action === 'restore' ? ['json'] : ['sqlite', 'db', 'json'] }] + filters: [action === 'restore' ? backupFileFilter : { name: '书库文件', extensions: ['sqlite', 'db', 'json'] }] }) if (chosen.canceled || !chosen.filePaths[0]) return { ok: true, cancelled: true } if (action === 'choose') target = chosen.filePaths[0] diff --git a/src/main/libraryFiles.ts b/src/main/libraryFiles.ts index 1eeaf55..c79f9af 100644 --- a/src/main/libraryFiles.ts +++ b/src/main/libraryFiles.ts @@ -1,4 +1,5 @@ import { constants } from 'node:fs' +import { isBackupFile } from './backupFileFormat' import { copyFile, lstat, open, readdir, unlink } from 'node:fs/promises' import { extname, isAbsolute, join, resolve } from 'node:path' import type { AppData } from '../shared/types' @@ -30,7 +31,7 @@ export async function hasLibraryTraces(directory: string, kind: 'config' | 'sqli if (kind === 'sqlite') return false if (entries.some(name => /^(secure-credentials\.json|agent-authorizations\.json)$|^novel-director-data\.json\./i.test(name))) return true const backups = join(directory, 'backups') - return await exists(backups) && (await readdir(backups)).some(name => /\.json(?:\.gz)?$/i.test(name)) + return await exists(backups) && (await readdir(backups)).some(isBackupFile) } export async function openExistingLibrary(path: string, stores: LibraryStores): Promise { diff --git a/src/preload/index.ts b/src/preload/index.ts index df61dde..89f121b 100644 --- a/src/preload/index.ts +++ b/src/preload/index.ts @@ -116,6 +116,12 @@ const novelDirector = { getProjectUsage: (projectId: string) => invokeOrThrow(IPC_CHANNELS.DATA_GET_PROJECT_USAGE, assertText(projectId, 'projectId')), load: loadStorageData, save: saveStorageData, + touchProject: async (projectId: string, expectedStoragePath: string) => { + const result = await invokeOrThrow(IPC_CHANNELS.DATA_TOUCH_PROJECT, + { projectId: assertText(projectId, 'projectId'), expectedStoragePath, expectedRevision: storageRevision }) + storageRevision = result.revision + return result + }, previewChapterTitleRename: (command: ChapterTitleRename) => invokeOrThrow(IPC_CHANNELS.DATA_PREVIEW_CHAPTER_TITLES, { command, expectedRevision: storageRevision }), applyChapterTitleRename: async (commit: ChapterTitleCommit) => { @@ -149,7 +155,8 @@ const novelDirector = { import: async (strategy: ImportDataStrategy = 'replace') => { const request: ImportDataRequest = { expectedRevision: storageRevision, strategy } const result = await invokeOrThrow(IPC_CHANNELS.STORAGE_IMPORT, request) - rememberStorageRevision(result) + // A blocked merge may report an external writer's revision without updating renderer data. + if (result.data) rememberStorageRevision(result) return result } }, diff --git a/src/renderer/src/App.tsx b/src/renderer/src/App.tsx index 360824d..95e9a41 100644 --- a/src/renderer/src/App.tsx +++ b/src/renderer/src/App.tsx @@ -65,6 +65,7 @@ export default function App() { needsWritebackCheck, setStatus, saveData, + touchProject, saveGenerationRunBundle, saveChapterCommitBundle, saveRevisionCommitBundle, @@ -275,7 +276,7 @@ export default function App() { 正在加载页面...
}>{renderCurrentView(data, currentProject)} : 正在加载项目列表...}> - + } {hasStorageConflict || needsWritebackCheck ? : null} diff --git a/src/renderer/src/hooks/useAppData.ts b/src/renderer/src/hooks/useAppData.ts index 40992fe..68d331b 100644 --- a/src/renderer/src/hooks/useAppData.ts +++ b/src/renderer/src/hooks/useAppData.ts @@ -1,5 +1,6 @@ import { useEffect, useRef, useState } from 'react' import { EMPTY_APP_DATA } from '../../../shared/defaults' +import { markProjectOpened } from '../../../services/ProjectLifecycleService' import type { AppData, CandidateDecisionCommand, ChapterCommitBundle, GenerationRunBundle, RevisionCommitBundle } from '../../../shared/types' import { getUserFriendlyError } from '../../../shared/errorUtils' import { isStorageRevisionConflict } from '../../../shared/storageRevisionConflict' @@ -192,6 +193,14 @@ export function useAppData() { } } + async function touchProject(projectId: string): Promise { + await runPersistedStorageOperation(async (current) => { + if (!current.projects.some(project => project.id === projectId)) throw new Error('项目不存在。') + const saved = await getNovelDirectorDataApi().touchProject(projectId, storagePath) + return { ...saved, data: markProjectOpened(current, projectId, saved.updatedAt) } + }) + } + async function executeCandidateDecision(command: CandidateDecisionCommand): Promise { setStatus('正在保存候选决定...') try { @@ -278,6 +287,7 @@ export function useAppData() { needsWritebackCheck, setStatus, saveData, + touchProject, saveGenerationRunBundle, saveChapterCommitBundle, saveRevisionCommitBundle, diff --git a/src/renderer/src/views/HomeView.tsx b/src/renderer/src/views/HomeView.tsx index 7c6ee24..1cff19f 100644 --- a/src/renderer/src/views/HomeView.tsx +++ b/src/renderer/src/views/HomeView.tsx @@ -5,7 +5,6 @@ import type { ID, Project } from '../../../shared/types' import { formatProjectDeletionImpact, getProjectDeletionSummary, - markProjectOpened, removeProjectFromAppData } from '../../../services/ProjectLifecycleService' import { useConfirm } from '../components/ConfirmDialog' @@ -30,11 +29,13 @@ type BusyAction = 'create' | 'import' | 'open' | 'edit' | 'delete' export function HomeView({ data, saveData, + touchProject, importData, setProjectId, setView }: PersistProps & { importData: ImportDataHandler + touchProject: (id: ID) => Promise setProjectId: (id: ID) => void setView: (view: View) => void }) { @@ -95,17 +96,17 @@ export function HomeView({ }) } + async function enterStoredProject(projectId: ID) { + await touchProject(projectId) + setProjectId(projectId) + setView('dashboard') + } + async function openProject(projectId: ID) { if (!beginAction('open')) return setOperationError(null) try { - const saved = await saveData((current) => markProjectOpened(current, projectId, now())) - if (!saved.ok) { - reportSaveFailure('打开项目', saved.errorMessage) - return - } - setProjectId(projectId) - setView('dashboard') + await enterStoredProject(projectId) } catch (error) { reportUnexpectedFailure('打开项目', error) } finally { @@ -163,13 +164,11 @@ export function HomeView({ if (!result.data) return const openedProjectId = result.importedProjectIds?.[0] ?? result.data.projects[0]?.id if (openedProjectId) { - const saved = await saveData((current) => markProjectOpened(current, openedProjectId, now())) - if (!saved.ok) { - reportSaveFailure('导入后的打开项目', saved.errorMessage) - return + try { + await enterStoredProject(openedProjectId) + } catch (error) { + reportUnexpectedFailure('导入后的打开项目', error) } - setProjectId(openedProjectId) - setView('dashboard') } } catch (error) { reportUnexpectedFailure('导入', error) diff --git a/src/renderer/src/views/settings/SettingsDataPanels.tsx b/src/renderer/src/views/settings/SettingsDataPanels.tsx index 7a51733..81a9a22 100644 --- a/src/renderer/src/views/settings/SettingsDataPanels.tsx +++ b/src/renderer/src/views/settings/SettingsDataPanels.tsx @@ -33,7 +33,7 @@ function BackupRow({ return (
- {backup.isAutomatic ? '自动备份' : '手动备份'} · {formatBackupTime(backup.timestamp)} + {backup.isRollback ? '旧版回退副本' : backup.isAutomatic ? '自动备份' : '手动备份'} · {formatBackupTime(backup.timestamp)} {formatBytes(backup.size)}
{backup.path} @@ -168,7 +168,7 @@ export function SettingsDataPanels({ storagePath, storage, backupAndLogs }: Sett {backupAndLogs.backups.length ? (
- {backupAndLogs.backups.slice(0, 8).map((backup) => ( + {backupAndLogs.backups.map((backup) => ( ({ ok: false, code: 'LIBRARY_UNAVAILABLE', error }) -const loads = [], actions = [], saves = [], unexpected = [] +const loads = [], actions = [], touches = [], imports = [], saves = [], unexpected = [] for (const channel of Object.values(channels)) { ipcMain.handle(channel, (_event, ...args) => { const records = channel === channels.STORAGE_GET ? loads : channel === channels.APP_RECOVER_LIBRARY ? actions + : channel === channels.DATA_TOUCH_PROJECT ? touches : channel === channels.STORAGE_IMPORT ? imports : channel === channels.STORAGE_SAVE ? saves : null if (records) return new Promise((resolve) => records.push({ args, resolve })) unexpected.push(channel) @@ -102,11 +103,10 @@ async function verify() { settings: { hasApiKey: true } }), storagePath: 'Z:\\测试\\选择的书库.json', revision: 'selected-revision' } opened.resolve(book) await until(() => run("document.querySelector('.home')?.textContent.includes('测试书稿')"), 'recovered project') - const save = await next(saves, () => press('进入')) - assert.equal(save.args[0].expectedRevision, book.revision) - assert.equal(save.args[0].data.settings.hasApiKey, true) + const touch = await next(touches, () => press('进入')) + assert.deepEqual(touch.args, [{ projectId: 'fixture', expectedStoragePath: book.storagePath, expectedRevision: book.revision }]) const conflict = new StorageRevisionConflictError() - save.resolve({ ok: false, code: conflict.code, error: conflict.message }) + touch.resolve({ ok: false, code: conflict.code, error: conflict.message }) await until(() => run("Boolean(document.querySelector('.storage-conflict-notice'))"), 'revision conflict') await press('重新加载最新数据') await until(() => run("Boolean(document.querySelector('[role=dialog]'))"), 'confirm reload') @@ -117,7 +117,7 @@ async function verify() { expectedError = '无法打开书库:' + book.storagePath + '\n磁盘未连接。' reload.resolve(failure(expectedError)) await recoveryPage(false, '') - assert.equal(saves.length, 1, 'a write queued behind the failed reload never reaches IPC') + assert.equal(touches.length, 1, 'a touch queued behind the failed reload never reaches IPC') // A new renderer must perform its own read, even if an earlier recovery dialog completes late. const abandoned = await attempt('restore') @@ -145,9 +145,47 @@ async function verify() { await until(() => run(`document.querySelector('.home')?.textContent.includes(${JSON.stringify(title)})`), title) } assert.equal(loads.length, 11) - assert.equal(saves.length, 1) + assert.equal(saves.length, 0) assert.deepEqual(unexpected, []) - console.log('PASS library recovery: loading, failure, four actions, cancellation, double click, reread, revision, credentials and restart') + + const blocked = await next(imports, () => press('导入 JSON')) + blocked.resolve({ canceled: false, mergeBlocked: true, revision: 'newer-agent-revision' }) + await until(() => run("document.querySelector('.home-operation-notice')?.textContent.includes('合并已停止')"), 'blocked import') + assert.equal(touches.length, 1) + const imported = await next(imports, () => press('导入 JSON')) + assert.equal(imported.args[0].expectedRevision, 'restore-revision', 'a blocked import cannot adopt an unseen revision') + const importPath = 'Z:\\测试\\备份副本.sqlite' + const importedData = { ...book.data, projects: [...book.data.projects, { ...book.data.projects[0], id: 'second', name: '导入样书' }] } + const afterImport = await next(touches, () => imported.resolve({ storagePath: importPath, canceled: false, revision: 'imported', + importedProjectIds: ['second'], data: importedData })) + assert.deepEqual(afterImport.args, [{ projectId: 'second', expectedStoragePath: importPath, expectedRevision: 'imported' }]) + assert.equal(await run("Boolean(document.querySelector('.home'))"), true, 'navigation waits for persistence') + afterImport.resolve({ ok: false, code: conflict.code, error: conflict.message }) + await until(() => run("document.querySelector('.home-operation-notice')?.textContent.includes('导入后的打开项目失败')"), 'import succeeded but open conflicted') + await press('重新加载最新数据') + await until(() => run("Boolean(document.querySelector('[role=dialog]'))"), 'confirm Agent reload') + const agentReload = await next(loads, () => press('重新加载')) + await until(() => run("!document.querySelector('[role=dialog]')"), 'reload pending') + await run("[...document.querySelectorAll('.project-row')].find(r => r.querySelector('h3').textContent === '导入样书').querySelector('.primary-button').click()") + await until(() => run("document.querySelector('.home')?.textContent.includes('正在打开...')"), 'touch queued behind Agent reload') + assert.equal(touches.length, 2) + const queued = await next(touches, () => agentReload.resolve({ storagePath: importPath, revision: 'agent-edit', + data: { ...importedData, projects: importedData.projects.map(p => ({ ...p, description: 'Agent 已更新' })) } })) + assert.deepEqual(queued.args, [{ projectId: 'second', expectedStoragePath: importPath, expectedRevision: 'agent-edit' }]) + const updatedAt = '2099-01-02T03:04:05.123Z' + queued.resolve({ ok: true, storagePath: importPath, revision: 'touched', updatedAt, savedCollections: ['projects'] }) + await until(() => run("document.querySelector('.project-badge strong')?.textContent === '导入样书'"), 'entered imported project') + await press('返回项目列表') + await until(() => run("document.querySelector('.project-row h3')?.textContent === '导入样书'"), 'recently opened project first') + await run("document.querySelector('.project-row [title=编辑项目资料]').click()") + assert.equal(saves.length, 0, 'neither project open transmitted AppData') + const save = await next(saves, () => press('保存资料')) + assert.equal(save.args[0].expectedRevision, 'touched') + assert.equal(save.args[0].data.projects.find(p => p.id === 'second').lastOpenedAt, updatedAt) + assert(save.args[0].data.projects.every(p => p.description === 'Agent 已更新')) + assert.equal(save.args[0].data.settings.hasApiKey, true) + save.resolve({ ok: true, storagePath: importPath, revision: 'edited' }) + console.log('PASS library recovery: four actions, restart, revision conflicts, queued project open, import and credentials') } app.disableHardwareAcceleration() diff --git a/src/services/commitBundles/chapterAcceptanceReviewValidation.ts b/src/services/commitBundles/chapterAcceptanceReviewValidation.ts index 30f8e16..157b088 100644 --- a/src/services/commitBundles/chapterAcceptanceReviewValidation.ts +++ b/src/services/commitBundles/chapterAcceptanceReviewValidation.ts @@ -12,6 +12,7 @@ import { hasCurrentChapterFragment } from '../../shared/chapterStepFragment' import { draftContentHash, noveltyAuditMatchesDraft } from '../DraftDiagnosticBindingService' import { isChapterArchived } from '../ChapterLifecycleService' import { assertImmutableCommit, requireCommitText } from './commitBundleUtils' +import { validateDiagnosticBindings } from './chapterDiagnosticBindings' export function validateAcceptanceReviewShape(bundle: ChapterCommitBundle): void { const review = bundle.acceptanceReview @@ -43,6 +44,7 @@ export function validateAcceptanceReviewShape(bundle: ChapterCommitBundle): void /** Re-evaluate new commits inside the storage transaction; historical replay is checked elsewhere. */ export function validateCurrentAcceptanceReview(bundle: ChapterCommitBundle, data: AppData): void { + validateDiagnosticBindings(bundle, data) const review = bundle.acceptanceReview const draft = data.generatedChapterDrafts.find((item) => item.id === bundle.generatedDraftId && item.projectId === bundle.projectId) const jobId = bundle.jobId ?? draft?.jobId ?? bundle.generatedDraft?.jobId diff --git a/src/services/commitBundles/chapterDiagnosticBindings.ts b/src/services/commitBundles/chapterDiagnosticBindings.ts new file mode 100644 index 0000000..ed9c226 --- /dev/null +++ b/src/services/commitBundles/chapterDiagnosticBindings.ts @@ -0,0 +1,26 @@ +import type { AppData, ChapterCommitBundle } from '../../shared/types' +import { draftContentHash } from '../DraftDiagnosticBindingService' + +// The primary quality result is re-elected below. Secondary diagnostics (including +// ID-only links) must not carry evidence from another draft or an older body either. +export function validateDiagnosticBindings(bundle: ChapterCommitBundle, data: AppData): void { + type Report = { id: string; draftId?: string | null; draftContentHash?: string | null } + let hash: string | undefined + function check(report: Report) { + if ((bundle.generatedDraftId && report.draftId && report.draftId !== bundle.generatedDraftId) || + (report.draftContentHash && report.draftContentHash !== (hash ??= draftContentHash(bundle.chapter.body))) || + (bundle.acceptanceReview && (!report.draftContentHash || !report.draftId))) { + throw new Error(`Report ${report.id} does not match the accepted draft text.`) + } + } + function checkLinks(incoming: readonly Report[] = [], stored: readonly Report[], primary?: string | null) { + const ids = new Set(incoming.map(report => report.id)) + if (primary) ids.add(primary) + if (!ids.size) return + for (const report of incoming) check(report) + for (const report of stored) if (ids.has(report.id)) check(report) + } + checkLinks(bundle.qualityGateReports, data.qualityGateReports, bundle.qualityGateReportId) + checkLinks(bundle.consistencyReviewReports, data.consistencyReviewReports, bundle.consistencyReviewReportId) + checkLinks(bundle.redundancyReports, data.redundancyReports) +} diff --git a/src/shared/ipc/ipcChannels.ts b/src/shared/ipc/ipcChannels.ts index e6953b2..f1bd666 100644 --- a/src/shared/ipc/ipcChannels.ts +++ b/src/shared/ipc/ipcChannels.ts @@ -1,4 +1,5 @@ export const IPC_CHANNELS = { + DATA_TOUCH_PROJECT: 'data:touch-project', DATA_GET_PROJECT_USAGE: 'data:get-project-usage', DATA_EXECUTE_CANDIDATE_DECISION: 'data:execute-candidate-decision', DATA_REVIEW_CHARACTER_CARD: 'data:review-character-card', diff --git a/src/shared/ipc/ipcTypes.ts b/src/shared/ipc/ipcTypes.ts index ed0fb59..888678b 100644 --- a/src/shared/ipc/ipcTypes.ts +++ b/src/shared/ipc/ipcTypes.ts @@ -200,6 +200,7 @@ export interface BackupInfo { timestamp: number size: number isAutomatic: boolean + isRollback?: boolean } export interface BackupCreateResult { diff --git a/src/shared/normalizers/appData.ts b/src/shared/normalizers/appData.ts index 3df712d..d394f48 100644 --- a/src/shared/normalizers/appData.ts +++ b/src/shared/normalizers/appData.ts @@ -242,7 +242,7 @@ function normalizeHardCanonItem(value: HardCanonItem | Record, } } -function normalizeHardCanonPack(value: HardCanonPack | Record, characters?: Character[]): HardCanonPack { +export function normalizeHardCanonPack(value: HardCanonPack | Record, characters?: Character[]): HardCanonPack { const pack = objectOrEmpty(value) const timestamp = new Date().toISOString() const projectId = stringValue(pack.projectId) diff --git a/src/storage/JsonStorageService.ts b/src/storage/JsonStorageService.ts index b421a29..76cef8d 100644 --- a/src/storage/JsonStorageService.ts +++ b/src/storage/JsonStorageService.ts @@ -167,6 +167,30 @@ export class JsonStorageService implements StorageService { await this.saveIfCurrent(data) } + async touchProject(projectId: string, expectedRevision: string): Promise { + if (!expectedRevision) throw new StorageRevisionConflictError() + return withJsonWriteLock(this.storagePath, async () => { + const bytes = await readFile(this.storagePath) + if (revisionForRaw(bytes) !== expectedRevision) throw new StorageRevisionConflictError() + const original = parseLibraryJsonValue(decodeLibraryText(bytes)) as AppData + // Its digest matches the loaded snapshot. Validate shape without normalizing unrelated rows. + if (this.strict) normalizeLibraryData(original, data => data) + const project = original.projects.find(row => row.id === projectId) + if (!project) throw new Error('项目不存在。') + // Write the original document, not its normalized view: keep unknown fields and old records. + let cleanBackup: string | undefined + if (original.settings?.apiKey) { + original.settings.apiKey = '' + cleanBackup = JSON.stringify(original, null, 2) + } + const updatedAt = new Date().toISOString() + project.lastOpenedAt = updatedAt + const written = await this.replaceDocument(original, undefined, cleanBackup) + return { ok: true, storagePath: this.storagePath, revision: revisionForRaw(written), + updatedAt, savedCollections: ['projects'] } + }, false) + } + async backupTo(targetPath: string): Promise { if (this.strict) { const raw = await readFile(this.storagePath) @@ -221,7 +245,7 @@ export class JsonStorageService implements StorageService { : sanitizeAppDataForPersistence(data), onWriteStarted) } - private async replaceDocument(data: Partial, onWriteStarted?: () => void): Promise { + private async replaceDocument(data: Partial, onWriteStarted?: () => void, cleanBackup?: string): Promise { const raw = JSON.stringify(data, null, 2) onWriteStarted?.() if (!this.strict) await mkdir(dirname(this.storagePath), { recursive: true }) @@ -232,7 +256,8 @@ export class JsonStorageService implements StorageService { try { await stat(this.storagePath) - await copyFile(this.storagePath, backupPath) + if (cleanBackup !== undefined) await writeFile(backupPath, cleanBackup, 'utf-8') + else await copyFile(this.storagePath, backupPath) } catch (error) { const code = typeof error === 'object' && error && 'code' in error ? String(error.code) : '' if (this.strict || code !== 'ENOENT') { @@ -241,6 +266,7 @@ export class JsonStorageService implements StorageService { } await rename(tmpPath, this.storagePath) + return raw } async saveGenerationRunBundle(bundle: GenerationRunBundle, expectedRevision?: string): Promise { diff --git a/src/storage/SqliteStorageService.ts b/src/storage/SqliteStorageService.ts index 1d9ba32..635e835 100644 --- a/src/storage/SqliteStorageService.ts +++ b/src/storage/SqliteStorageService.ts @@ -4,7 +4,7 @@ import { createRequire } from 'node:module' import { dirname, extname, resolve } from 'node:path' import type { AppData, CandidateDecisionCommand, Character, CharacterCardReviewCommand, ChapterCommitBundle, ChapterWritebackFailure, GenerationRunBundle, RevisionCommitBundle } from '../shared/types' import type { CandidateDecisionWriteResult, CharacterCardReviewWriteResult, StorageWriteResult } from '../shared/ipc/ipcTypes' -import { EMPTY_APP_DATA, normalizeAppData, sanitizeAppDataForPersistence } from '../shared/defaults' +import { normalizeAppData, sanitizeAppDataForPersistence } from '../shared/defaults' import { normalizeChapterWritebackFailure } from '../shared/normalizers/chapterWritebackFailures' import { applyCharacterCardReview, CHARACTER_CARD_REVIEW_COLLECTIONS } from '../services/characterState/cardReview' import { applyChapterCommitBundleToAppData } from '../services/ChapterCommitBundleService' @@ -35,10 +35,12 @@ import { import { currentRevisionFromDatabase, readAppDataFromDatabase, readAppDataCollectionsFromDatabase, readLibrarySnapshot } from './sqlite/sqliteSnapshot' import { writeSqliteSnapshotEntities } from './sqlite/sqliteSnapshotWriter' import { readGenerationBundleValidationContext } from './sqlite/sqliteGenerationContext' +import { readChapterCommitContext, readRevisionCommitContext } from './sqlite/sqliteCommitContext' +import { ensureReceiptIndexes } from './sqlite/sqliteReceiptAliases' import { executeSqliteCandidateDecision } from './sqlite/sqliteCandidateDecisions' import { SETTINGS_ROW_ID } from './sqlite/sqliteSchema' import { sqliteUsageHistory } from './sqlite/sqliteProjectUsage' -import { isSqliteEmpty, SqliteInitialization } from './sqlite/sqliteInitialization' +import { SqliteInitialization } from './sqlite/sqliteInitialization' import type { AppDataArrayKey, SqliteDatabase, @@ -143,7 +145,7 @@ export class SqliteStorageService implements StorageService { const readTransaction = db.transaction(() => { const revision = this.currentRevision(db) if (this.strict) return this.initialization.takeInitialSnapshot(db, revision) ?? readLibrarySnapshot(db) - return { data: this.readAppData(db), revision } + return { data: readAppDataFromDatabase(db), revision } }) return readTransaction() } @@ -152,6 +154,25 @@ export class SqliteStorageService implements StorageService { await this.saveIfCurrent(data) } + async touchProject(projectId: string, expectedRevision: string): Promise { + if (!expectedRevision) throw new StorageRevisionConflictError() + const db = await this.open() + return db.transaction((): StorageWriteResult => { + const updatedAt = nowIso() + // Compare and update in the first write statement, including across Agent connections. + const { changes } = db.prepare(`UPDATE entities SET json = json_set(json, '$.lastOpenedAt', ?) + WHERE collection = 'projects' AND id = ? AND (SELECT value FROM meta WHERE key = 'revision') = ?`) + .run(updatedAt, projectId, expectedRevision) as { changes: number } + if (!changes) { + this.assertExpectedRevision(db, expectedRevision) + throw new Error('项目不存在。') + } + const revision = `${updatedAt}-${randomUUID()}` + this.writeRevisionMeta(db, revision, updatedAt) + return { ok: true, storagePath: this.storagePath, revision, updatedAt, savedCollections: ['projects'] } + })() + } + async backupTo(targetPath: string): Promise { const db = await this.open() await mkdir(dirname(targetPath), { recursive: true }) @@ -191,6 +212,7 @@ export class SqliteStorageService implements StorageService { this.writeRevisionMeta(db, revision, updatedAt) // Keep full-save deletion/import semantics without rewriting unchanged history and prose. writeSqliteSnapshotEntities(db, entries, updatedAt) + ensureReceiptIndexes(db) }) writeTransaction(normalized) @@ -214,14 +236,14 @@ export class SqliteStorageService implements StorageService { return this.saveEntityBundle(expectedRevision, (existing) => { const nextData = applyChapterCommitBundleToAppData(existing, bundle) return nextData === existing ? [] : chapterCommitEntityEntries(bundle, nextData) - }, (db) => this.isEmpty(db) ? EMPTY_APP_DATA : this.readAppData(db), onWriteStarted) + }, (db) => this.readChapterCommitContext(db, bundle, this.strict), onWriteStarted) } async saveRevisionCommitBundle(bundle: RevisionCommitBundle, expectedRevision?: string, onWriteStarted?: () => void): Promise { return this.saveEntityBundle(expectedRevision, (existing) => { const nextData = applyRevisionCommitBundleToAppData(existing, bundle) return nextData === existing ? [] : revisionCommitEntityEntries(bundle, nextData) - }, (db) => this.isEmpty(db) ? EMPTY_APP_DATA : this.readAppData(db), onWriteStarted) + }, (db) => this.readRevisionCommitContext(db, bundle, this.strict), onWriteStarted) } async recordChapterWritebackFailureIfUncommitted(row: ChapterWritebackFailure): Promise { @@ -326,6 +348,9 @@ export class SqliteStorageService implements StorageService { updatedAt: string, replace: boolean ): void { + if (entries[0]?.collection === 'chapterCommitBundles' || entries[0]?.collection === 'revisionCommitBundles') { + ensureReceiptIndexes(db) + } const insertSql = replace ? `INSERT OR REPLACE INTO entities ( collection, id, project_id, chapter_id, job_id, character_id, @@ -372,13 +397,8 @@ export class SqliteStorageService implements StorageService { return db } - private isEmpty(db: SqliteDatabase): boolean { - return isSqliteEmpty(db) - } - - private readAppData(db: SqliteDatabase): AppData { - return this.strict ? readLibrarySnapshot(db).data : readAppDataFromDatabase(db) - } + private readonly readChapterCommitContext = readChapterCommitContext + private readonly readRevisionCommitContext = readRevisionCommitContext } function libraryStorage(filePath: string, mode: 'existing' | 'new', initialData?: AppData): StorageService { diff --git a/src/storage/StorageService.ts b/src/storage/StorageService.ts index cc015b6..2a841ce 100644 --- a/src/storage/StorageService.ts +++ b/src/storage/StorageService.ts @@ -23,6 +23,7 @@ export interface StorageService extends FormalChapterCommitWriter { load(): Promise loadSnapshot(): Promise save(data: AppData): Promise + touchProject(projectId: string, expectedRevision: string): Promise saveIfCurrent(data: AppData, expectedRevision?: string): Promise saveGenerationRunBundle(bundle: GenerationRunBundle, expectedRevision?: string): Promise saveChapterCommitBundle(bundle: ChapterCommitBundle, expectedRevision?: string, onWriteStarted?: () => void): Promise diff --git a/src/storage/sqlite/sqliteCommitContext.ts b/src/storage/sqlite/sqliteCommitContext.ts new file mode 100644 index 0000000..015dc89 --- /dev/null +++ b/src/storage/sqlite/sqliteCommitContext.ts @@ -0,0 +1,163 @@ +import type { AppData, ChapterCommitBundle, RevisionCommitBundle } from '../../shared/types' +import { normalizeAppData } from '../../shared/defaults' +import { normalizeHardCanonPack } from '../../shared/normalizers/appData' +import { applicableRules } from '../../services/hardCanonAudit/rules' +import { libraryEntityId, parseLibraryJsonValue } from '../libraryValidation' +import { asRecord } from './sqliteEntityMapper' +import { receiptKeys } from './sqliteReceiptAliases' +import { readLibraryHeader } from './sqliteSnapshot' +import type { AppDataArrayKey, SqliteDatabase, SqliteValue } from './sqliteTypes' + +type Key = AppDataArrayKey +type Id = string | null | undefined + +// A transaction-local input to the existing bundle validators and entry mappers. +// This projection must never be passed to a snapshot writer. +class CommitRows { + private readonly rows = new Map>>() + private readonly byId + constructor(private readonly db: SqliteDatabase) { + this.byId = db.prepare('SELECT id, json FROM entities WHERE collection = ? AND id = ?') + } + + private keep(key: Key, row: Record) { + let group = this.rows.get(key) + if (!group) this.rows.set(key, group = new Map()) + if (typeof row.id !== 'string' || typeof row.json !== 'string') throw new Error('Invalid SQLite commit record.') + if (!group.has(row.id)) { + const value = parseLibraryJsonValue(row.json) + if (libraryEntityId(key, value) !== row.id) throw new Error(`SQLite record ID mismatch: ${key}/${row.id}`) + group.set(row.id, asRecord(value)) + } + } + + ids(key: Key, ...ids: Id[]) { + for (const id of new Set(ids)) { + if (!id || this.rows.get(key)?.has(id)) continue + const row = this.byId.get(key, id) + if (row) this.keep(key, row) + } + } + + one(key: Key, id: Id) { + this.ids(key, id) + return id ? this.rows.get(key)?.get(id) : undefined + } + + receipt(key: 'chapterCommitBundles' | 'revisionCommitBundles', id: Id) { + if (id) this.ids(key, ...receiptKeys(this.db, key, id)) + return Boolean(this.rows.get(key)?.size) + } + + private select(key: Key, where: string, ...args: SqliteValue[]) { + for (const row of this.db.prepare(`SELECT id, json FROM entities WHERE collection = ? AND ${where}`) + .iterate(key, ...args)) this.keep(key, row) + } + + scoped(key: Key, column: keyof typeof legacyKeys, value: string | number, filter = '', ...args: SqliteValue[]) { + // Separate index probes keep SQLite from scanning the whole collection for an OR. + this.select(key, column + ' = ?' + filter, value, ...args) + this.select(key, column + ' IS NULL AND ' + legacyKeys[column] + ' = ?' + filter, value, ...args) + } + + private values(key: Key) { + const rows = this.rows.get(key) + if (!rows) return [] + // Both the audit and the validators see the same binary ID ordering as a full SQLite read. + const ids = [...rows.keys()].sort((a, b) => Buffer.compare(Buffer.from(a), Buffer.from(b))) + return ids.map(id => rows.get(id)!) + } + + hardCanon(projectId: string, order: number) { + this.scoped('hardCanonPacks', 'project_id', projectId) + const packs = this.values('hardCanonPacks') + if (!packs.length) return + // First normalize without a character directory. The final normalization below + // resolves character bindings against the same project as a full snapshot. + const rules = applicableRules(packs.map(pack => normalizeHardCanonPack(pack)), projectId, order) + if (!rules.length) return + this.scoped('characters', 'project_id', projectId) + const state = rules.some(r => r.check.kind === 'actor_action' && r.check.state) + if (state || rules.some(r => r.check.kind === 'literal')) this.scoped('chapters', 'project_id', projectId) + if (state) { + this.scoped('characterStateFacts', 'project_id', projectId) + this.scoped('characterStateTransactions', 'project_id', projectId) + } + } + + data(): AppData { + const partial: Partial = { characters: [] } + for (const key of this.rows.keys()) Object.assign(partial, { [key]: this.values(key) }) + return normalizeAppData(partial) + } +} + +// Older stores may lack a denormalized key; filter those rows in SQLite. +const legacyKeys = { + project_id: "json_extract(json, '$.projectId')", + job_id: "json_extract(json, '$.jobId')", + chapter_order: "CASE WHEN json_type(json, '$.order') IN ('integer', 'real') THEN json_extract(json, '$.order') ELSE 1 END" +} +const inProject = " AND coalesce(project_id, json_extract(json, '$.projectId')) = ?" + +export function readChapterCommitContext(db: SqliteDatabase, b: ChapterCommitBundle, strict = false): AppData { + if (strict) readLibraryHeader(db) + const r = new CommitRows(db) + const replay = r.receipt('chapterCommitBundles', b.commitId) + // Point reads deliberately have no project predicate: a foreign colliding ID + // must be visible to the same validator that handles JSON and in-memory data. + r.ids('chapters', b.chapterId) + r.ids('chapterVersions', b.chapterVersion?.id, b.previousChapterVersion?.id, b.chapterVersion?.baseChapterVersionId) + const draft = r.one('generatedChapterDrafts', b.generatedDraftId) + r.ids('generatedChapterDrafts', b.generatedDraft?.id) + r.ids('generationRunTraces', b.generationRunTraceId, b.generationRunTrace?.id) + r.ids('qualityGateReports', b.qualityGateReportId, ...(b.qualityGateReports ?? []).map(x => x.id)) + r.ids('consistencyReviewReports', b.consistencyReviewReportId, ...(b.consistencyReviewReports ?? []).map(x => x.id)) + r.ids('redundancyReports', ...(b.redundancyReports ?? []).map(x => x.id)) + r.ids('memoryUpdateCandidates', ...(b.acceptedMemoryUpdateCandidates ?? []).map(x => x.id)) + r.ids('characterStateChangeCandidates', ...(b.acceptedCharacterStateChangeCandidates ?? []).map(x => x.id)) + r.ids('characterStateFacts', ...(b.appliedCharacterStateFacts ?? []).map(x => x.id), + ...(b.appliedCharacterStateTransactions ?? []).map(x => x.factId)) + r.ids('characterStateTransactions', ...(b.appliedCharacterStateTransactions ?? []).map(x => x.id)) + r.ids('foreshadowings', ...(b.appliedForeshadowingUpdates ?? []).map(x => x.id)) + r.ids('timelineEvents', ...(b.appliedTimelineEvents ?? []).map(x => x.id)) + const jobId = b.jobId ?? (draft?.jobId as Id) ?? b.generatedDraft?.jobId + const job = r.one('chapterGenerationJobs', jobId) + if (!replay) { + if (jobId && job?.projectId === b.projectId) { + if (b.acceptanceReview) { + for (const key of ['qualityGateReports', 'editorialVerdicts'] as const) { + r.scoped(key, 'job_id', jobId, inProject + " AND json_extract(json, '$.draftId') = ?", b.projectId, b.generatedDraftId ?? null) + } + } else { + // Only legacy adoption checks the first prose step for an incomplete fragment. + r.scoped('chapterGenerationSteps', 'job_id', jobId, " AND json_extract(json, '$.type') = 'generate_chapter_draft'") + } + } + if (b.acceptanceReview) { + const order = b.chapter.order + r.scoped('chapters', 'chapter_order', order, inProject, b.projectId) + } + const order = b.acceptanceReview && typeof job?.targetChapterOrder === 'number' ? job.targetChapterOrder : b.chapter.order + r.hardCanon(b.projectId, order) + } + return r.data() +} + +export function readRevisionCommitContext(db: SqliteDatabase, b: RevisionCommitBundle, strict = false): AppData { + if (strict) readLibraryHeader(db) + const r = new CommitRows(db) + if (r.receipt('revisionCommitBundles', b.revisionCommitId)) return r.data() + r.ids('chapters', b.chapterId) + r.ids('chapterVersions', b.baseChapterVersionId, b.newChapterVersionId) + r.receipt('chapterCommitBundles', b.linkedChapterCommitId) + r.ids('generationRunTraces', b.linkedGenerationRunTraceId, b.generationRunTrace?.id) + r.ids('agentRuns', b.actor?.agentRunId) + r.ids('agentActionPreviews', b.actor?.actionPreviewId) + const version = r.one('revisionVersions', b.revisionVersionId ?? b.revisionVersion?.id) + const session = r.one('revisionSessions', (version?.sessionId as Id) ?? b.revisionSessionId ?? b.revisionSession?.id) + r.ids('revisionSessions', b.revisionSessionId, b.revisionSession?.id) + r.ids('generatedChapterDrafts', session?.sourceDraftId as Id, b.generatedDraft?.id) + r.hardCanon(b.projectId, b.chapter.order) + return r.data() +} diff --git a/src/storage/sqlite/sqliteReceiptAliases.ts b/src/storage/sqlite/sqliteReceiptAliases.ts new file mode 100644 index 0000000..922e7a4 --- /dev/null +++ b/src/storage/sqlite/sqliteReceiptAliases.ts @@ -0,0 +1,21 @@ +import type { SqliteDatabase } from './sqliteTypes' + +const aliases = { + chapterCommitBundles: "json_extract(json, '$.commitId')", + revisionCommitBundles: "json_extract(json, '$.revisionCommitId')" +} as const +type Collection = keyof typeof aliases + +// Normal writers require id === commitId. Only imported noncanonical keys need an index entry. +function predicate(key: Collection) { return `collection = '${key}' AND id <> ${aliases[key]}` } + +export function ensureReceiptIndexes(db: SqliteDatabase) { + for (const key of Object.keys(aliases) as Collection[]) { + db.exec(`CREATE INDEX IF NOT EXISTS idx_alias_${key} ON entities(${aliases[key]}) WHERE ${predicate(key)}`) + } +} + +export function receiptKeys(db: SqliteDatabase, key: Collection, id: string) { + const rows = db.prepare(`SELECT id FROM entities WHERE ${predicate(key)} AND ${aliases[key]} = ?`).all(id) + return [id, ...rows.map(row => String(row.id))] +} diff --git a/src/storage/sqlite/sqliteSnapshot.ts b/src/storage/sqlite/sqliteSnapshot.ts index 270f52d..4ed827d 100644 --- a/src/storage/sqlite/sqliteSnapshot.ts +++ b/src/storage/sqlite/sqliteSnapshot.ts @@ -32,8 +32,7 @@ export function currentRevisionFromDatabase(db: SqliteDatabase): string { return typeof value === 'string' && value ? value : '0' } -/** Read under the caller's snapshot transaction; never substitute defaults for damaged rows. */ -export function readLibrarySnapshot(db: SqliteDatabase): StorageSnapshot { +export function readLibraryHeader(db: SqliteDatabase) { assertLibrarySchema(db) const version = db.prepare("SELECT value FROM meta WHERE key = 'schemaVersion'").get()?.value const revision = db.prepare("SELECT value FROM meta WHERE key = 'revision'").get()?.value @@ -42,6 +41,16 @@ export function readLibrarySnapshot(db: SqliteDatabase): StorageSnapshot { !revision.trim() || settings.length !== 1 || settings[0].id !== SETTINGS_ROW_ID || typeof settings[0].json !== 'string') { throw new Error('SQLite 书库缺少有效的数据版本、修订号或设置。') } + const value = parseLibraryJsonValue(settings[0].json) + if (Number(version) > EMPTY_APP_DATA.schemaVersion || !value || typeof value !== 'object' || Array.isArray(value)) { + throw new Error('SQLite 书库的数据版本或设置无效。') + } + return { schemaVersion: Number(version), settings: value, revision } +} + +/** Read under the caller's snapshot transaction; never substitute defaults for damaged rows. */ +export function readLibrarySnapshot(db: SqliteDatabase): StorageSnapshot { + const { schemaVersion, settings, revision } = readLibraryHeader(db) const collections = new Map(ALL_ENTITY_COLLECTIONS.map(key => [key, []])) const rows = db.prepare('SELECT collection, id, json FROM entities ORDER BY collection, id').iterate() for (const row of rows) { @@ -54,8 +63,7 @@ export function readLibrarySnapshot(db: SqliteDatabase): StorageSnapshot { } target.push(value) } - return { revision, data: normalizeLibraryData({ ...Object.fromEntries(collections), schemaVersion: Number(version), - settings: parseLibraryJsonValue(settings[0].json) }) } + return { revision, data: normalizeLibraryData({ ...Object.fromEntries(collections), schemaVersion, settings }) } } export function readAppDataFromDatabase(db: SqliteDatabase): AppData { From 935f6cd8df22d443a3ec9a6724f251f2d3891bd0 Mon Sep 17 00:00:00 2001 From: SHIN09-git Date: Fri, 13 Nov 2026 04:52:00 +0800 Subject: [PATCH 4/8] Resolve the pinned Electron binary on a clean packaging host --- .../LIBRARY_STABILITY_AND_PERFORMANCE_PLAN.md | 11 +- scripts/manage-sqlite-native.mjs | 6 +- scripts/run-tests.mjs | 2 + .../validate-sqlite-commit-scoped-read.mjs | 208 ++++++++++++++++++ 4 files changed, 223 insertions(+), 4 deletions(-) create mode 100644 scripts/validate-sqlite-commit-scoped-read.mjs diff --git a/docs/LIBRARY_STABILITY_AND_PERFORMANCE_PLAN.md b/docs/LIBRARY_STABILITY_AND_PERFORMANCE_PLAN.md index 4ece2c9..ceceda7 100644 --- a/docs/LIBRARY_STABILITY_AND_PERFORMANCE_PLAN.md +++ b/docs/LIBRARY_STABILITY_AND_PERFORMANCE_PLAN.md @@ -1,6 +1,6 @@ # 书库可靠性与长篇性能施工计划 / Library Reliability and Performance -拟定于 2026-11-12。本计划承接 [ROADMAP](../ROADMAP.md),不重新排期已发布的长篇完稿修复。**S0 样本已固定,S1 已实现;Windows Setup 安装、升级及发布仍属 S2。** +拟定于 2026-11-12,更新于 2026-11-13。本计划承接 [ROADMAP](../ROADMAP.md)。S0 / S1 是旧版样本和打开失败恢复;S2 是独立 Windows 安装验收;S3 和 S4 分别优化正式写入和备份。候选验收不等于公开发布。 下一版先让作者确信“打开的是原来的书库”,并用真正的 Windows 安装器验证升级。之后再缩短打开一本书、采纳或修订一章的等待时间。不要求先重构全库,也不拿删除作者历史来换取性能。 @@ -83,7 +83,14 @@ preview.20 实包使用 Electron 39.8.10 / better-sqlite3 12.9.0;核查时主 每个修改程序的批次运行 `npm.cmd run typecheck`、`npm.cmd test`、`npm.cmd run build`。S3 额外保留 [采纳](../scripts/validate-chapter-commit-bundle.mjs)、[修订](../scripts/validate-p2c-revision-commit-bundle.mjs)、[版本恢复](../scripts/validate-version-chain-restore.mjs)的对照;S4 压缩若启动,还要验证旧 JSON 导入导出、备份列表和恢复。性能记录只提交样本配方、指标与复现方式,不含私人路径、正文或密钥。不调用付费模型。 -暂不安排流水线引擎搬目录、所有页面改为增删改查接口、清理作者文件或按文件行数重构。**下一批是 S2**:用真正的 Windows Setup 做干净安装和原地升级;本次解包窗口验收不是安装升级结论。 +### Preview.21 实施记录 + +- **S2:** [Windows 双机验收](./S2_WINDOWS_QUALIFICATION.md)。以同一构建的安装包、两份安装报告和各自的冷启动回读作为依据,不用解包 smoke 代替安装升级。公开 preview.20 保持原样。 +- **S3-A:** 界面携带项目 ID、所选书库路径与修订号,在原保存队列内更新时间。SQLite 只改目标项目及书库修订标记;JSON 后端仍需重写单文件。编辑时间不随打开项目而改变。 +- **S3-B:** 采纳和修订在原事务内读取关联章节、版本、报告、候选与回执,并保留冲突拒绝及失败回滚。涉及全书计数的硬规则仍可读取当前项目的章节;不拿部分书库数据走整库保存。 +- **S4:** 选择压缩新自动备份,不批量改写旧回执或裁剪生成历史。[备份实测](./S4_AUTO_BACKUPS.md)包含恢复、空间不足及超出解压上限的处理。虚构的 500 章样本保留占用从 541 MiB 降至 92 MiB,代价是到期写入中位数由 229 ms 增至 368 ms。启动读取整库仍是独立问题。 + +暂不安排流水线引擎搬目录、全库 CRUD 重构或清理作者历史。 ## English summary diff --git a/scripts/manage-sqlite-native.mjs b/scripts/manage-sqlite-native.mjs index 1377985..e381e72 100644 --- a/scripts/manage-sqlite-native.mjs +++ b/scripts/manage-sqlite-native.mjs @@ -1,6 +1,7 @@ #!/usr/bin/env node import { copyFileSync, existsSync, mkdirSync, readFileSync, writeFileSync } from 'node:fs' import { dirname, join } from 'node:path' +import { createRequire } from 'node:module' import { spawnSync } from 'node:child_process' import { fileURLToPath } from 'node:url' @@ -9,8 +10,7 @@ const mode = process.argv[2] ?? 'help' const electronPackage = JSON.parse(readFileSync(join(root, 'node_modules', 'electron', 'package.json'), 'utf8')) const npmCli = process.env.npm_execpath || join(dirname(process.execPath), 'node_modules', 'npm', 'bin', 'npm-cli.js') const electronRebuildCli = join(root, 'node_modules', '@electron', 'rebuild', 'lib', 'cli.js') -const electronExe = join(root, 'node_modules', 'electron', 'dist', process.platform === 'win32' - ? 'electron.exe' : process.platform === 'darwin' ? 'Electron.app/Contents/MacOS/Electron' : 'electron') +const require = createRequire(import.meta.url) const nodeBinding = join(root, 'node_modules', 'better-sqlite3', 'build', 'Release', 'better_sqlite3.node') const packagedBinding = join( root, @@ -74,6 +74,8 @@ function checkNode() { } function checkElectron() { + // Electron 44 downloads its locked binary on first use, not during npm ci. + const electronExe = require('electron') run(electronExe, [probeScript('electron')], { env: { ...env, ELECTRON_RUN_AS_NODE: '1' }, timeout: 30_000 }) } diff --git a/scripts/run-tests.mjs b/scripts/run-tests.mjs index 6d9bd1b..ea1228f 100644 --- a/scripts/run-tests.mjs +++ b/scripts/run-tests.mjs @@ -120,6 +120,8 @@ const tests = [ ['validate-storage-migration-merge.mjs'], ['validate-import-safety.mjs'], ['validate-sqlite-storage.mjs'], + ['validate-sqlite-commit-scoped-read.mjs'], + ['s2-windows/readback.test.mjs'], ['validate-backend-backup-laziness.mjs'], ['validate-sqlite-initialization.mjs'], ['validate-sqlite-differential-save.mjs'], diff --git a/scripts/validate-sqlite-commit-scoped-read.mjs b/scripts/validate-sqlite-commit-scoped-read.mjs new file mode 100644 index 0000000..c755b8f --- /dev/null +++ b/scripts/validate-sqlite-commit-scoped-read.mjs @@ -0,0 +1,208 @@ +import assert from 'node:assert/strict' +import { mkdirSync, mkdtempSync, rmSync } from 'node:fs' +import { createRequire } from 'node:module' +import { dirname, join } from 'node:path' +import { after, test } from 'node:test' +import Database from 'better-sqlite3' +import { build } from 'esbuild' +import { repoRoot } from './utils/repo-root.mjs' + +const base = join(repoRoot, 'tmp') +mkdirSync(base, { recursive: true }) +const root = mkdtempSync(join(base, 'sqlite-commit-context-')) +after(() => { assert.equal(dirname(root), base); rmSync(root, { recursive: true, force: true }) }) +const runtime = await build({ stdin: { resolveDir: repoRoot, contents: [ + 'shared/defaults', 'services/DraftDiagnosticBindingService', 'services/ChapterCommitBundleService', + 'services/RevisionCommitBundleService', 'storage/SqliteStorageService' +].map(path => `export * from './src/${path}'`).join('\n') }, write: false, + bundle: true, platform: 'node', format: 'cjs', packages: 'external', logLevel: 'silent' }) + +// Observe exactly what a real transaction reads, separately from preflight and verification loads. +let seen +const json = { ...JSON, stringify: JSON.stringify, parse(text, ...args) { + const row = JSON.parse(text, ...args) + if (seen && row?.unrelatedHistory) seen.unrelated.push(row.id) + return row +} } +function Binding(file, options) { + return new Database(file, { ...options, verbose(sql) { if (seen) seen.sql.push(sql) } }) +} +const require = createRequire(import.meta.url) +const localRequire = id => id === 'better-sqlite3' ? Binding + : id === 'node:module' ? { ...require(id), createRequire: () => localRequire } : require(id) +const mod = { exports: {} } +new Function('require', 'module', 'exports', 'JSON', runtime.outputFiles[0].text)(localRequire, mod, mod.exports, json) +const api = mod.exports +const t = '2026-01-01T00:00:00Z', t2 = '2026-01-02T00:00:00Z' +const stamp = { createdAt: t, updatedAt: t }, copy = structuredClone +const body = 'The source manuscript.' +const report = { id: 'quality', projectId: 'book', jobId: 'job', draftId: 'draft', + chapterId: 'ch', draftContentHash: api.draftContentHash(body), pass: true, ...stamp } + +function fixture() { + return api.normalizeAppData({ + projects: [{ id: 'book', ...stamp }, { id: 'other', ...stamp }], + chapters: [ + { id: 'ch', projectId: 'book', order: 2, title: 'Two', body: 'Old text', ...stamp }, + { id: 'one', projectId: 'book', order: 1, title: 'One', body: 'A lost key', unrelatedHistory: true, ...stamp }, + { id: 'other-ch', projectId: 'other', order: 2, title: 'Two', body: 'Foreign text', unrelatedHistory: true, ...stamp } + ], + chapterGenerationJobs: [{ id: 'job', projectId: 'book', targetChapterOrder: 2, status: 'completed', ...stamp }], + generatedChapterDrafts: [ + { id: 'draft', projectId: 'book', jobId: 'job', chapterId: 'ch', title: 'Target', body, status: 'draft', ...stamp }, + { id: 'other-draft', projectId: 'book', jobId: 'job', title: 'Later draft', body: 'Do not adopt', status: 'draft', unrelatedHistory: true, ...stamp } + ], + qualityGateReports: [report, { ...report, id: 'other-quality', draftId: 'other-draft', unrelatedHistory: true }], + consistencyReviewReports: [{ ...report, id: 'stale', draftContentHash: api.draftContentHash('Former source') }], + revisionSessions: [{ id: 'session', projectId: 'book', chapterId: 'ch', sourceDraftId: 'draft', status: 'active', ...stamp }], + revisionVersions: [{ id: 'rv', sessionId: 'session', body: 'Revised text', status: 'pending', + sourceContentHash: api.draftContentHash(body), ...stamp }], + chapterVersions: [{ id: 'foreign-v', projectId: 'other', chapterId: 'other-ch', title: 'Two', body: 'Old foreign', ...stamp }], + revisionCommitBundles: [{ id: 'old', revisionCommitId: 'old', projectId: 'other', chapterId: 'other-ch', unrelatedHistory: true }] + }) +} + +const adopt = data => api.buildAcceptedDraftCommitBundle({ appData: data, projectId: 'book', chapterId: 'ch', + draftId: 'draft', targetChapterOrder: 2, commitId: 'adopt', chapterVersionId: 'v2', acceptedAt: t2, acceptanceMode: 'reviewed' }) +const revise = data => api.buildRevisionCommitBundle({ appData: data, projectId: 'book', chapterId: 'ch', + revisionCommitId: 'revise', newChapterVersionId: 'v3', revisionVersionId: 'rv', revisedAt: t2 }) +async function library(t, data = fixture()) { + const file = join(mkdtempSync(join(root, 'book-')), 'fiction.sqlite') + const seed = new api.SqliteStorageService(file) + try { await seed.save(data) } finally { seed.close() } + const store = api.createExistingLibraryStorage(file) + const peer = new Database(file, { fileMustExist: true }) + t.after(() => { store.close(); peer.close() }) + return { store, peer, file } +} +async function narrow(action) { + seen = { unrelated: [], sql: [] } + try { + const result = await action() + assert.deepEqual(seen.unrelated, [], 'parsed an unrelated manuscript, report or historical receipt') + assert.ok(!seen.sql.some(s => /FROM entities(?: ORDER BY|$)/i.test(s)), 'read the entire library') + assert.ok(seen.sql.some(s => /^BEGIN/i.test(s)) && seen.sql.some(s => /^COMMIT/i.test(s))) + return { result, sql: seen.sql } + } finally { seen = undefined } +} +async function rejectLikeFull(store, bundle, pattern) { + const before = await store.loadSnapshot() + const revision = Boolean(bundle.revisionCommitId) + const apply = revision ? api.applyRevisionCommitBundleToAppData : api.applyChapterCommitBundleToAppData + const method = revision ? 'saveRevisionCommitBundle' : 'saveChapterCommitBundle' + let expected + assert.throws(() => apply(before.data, bundle), err => { expected = err.message; return pattern.test(expected) }) + await assert.rejects(store[method](bundle, before.revision), err => err.message === expected) + assert.deepEqual(await store.loadSnapshot(), before) +} + +test('cold writer: both transactions read scoped payloads; CAS and historical replay preserve the version chain', async t => { + const { store, peer, file } = await library(t) + const initial = await store.loadSnapshot() + const chapter = adopt(initial.data) + const { result: first, sql } = await narrow(() => store.saveChapterCommitBundle(chapter, initial.revision)) + const changed = await store.loadSnapshot(), revision = revise(changed.data) + // The validator must resolve persisted version -> session -> source draft even with only a version ID in the packet. + delete revision.generatedDraft + delete revision.revisionSession + delete revision.revisionSessionId + delete revision.revisionVersion + await assert.rejects(store.saveRevisionCommitBundle(revision, initial.revision), { code: 'STORAGE_REVISION_CONFLICT' }) + assert.deepEqual(await store.loadSnapshot(), changed) + const { result: second, sql: revSql } = await narrow(() => store.saveRevisionCommitBundle(revision, first.revision)) + for (const [key, queries] of [['chapterCommitBundles', sql], ['revisionCommitBundles', revSql]]) { + const lookup = queries.find(s => s.includes('SELECT id FROM entities') && s.includes(key)) + assert.ok(lookup) + assert.match(peer.prepare('EXPLAIN QUERY PLAN ' + lookup).all().map(r => r.detail).join(' '), + new RegExp('SEARCH entities USING (?:COVERING )?INDEX idx_alias_' + key)) + } + for (const [key, packet] of [['saveChapterCommitBundle', chapter], ['saveRevisionCommitBundle', revision]]) { + const { result } = await narrow(() => store[key](packet, second.revision)) + assert.deepEqual(result.savedCollections, []) + assert.equal(result.revision, second.revision) + } + const reopened = api.createExistingLibraryStorage(file) + try { + const final = await reopened.load() + assert.equal(final.chapters.find(c => c.id === 'ch').body, revision.afterText) + assert.equal(final.chapters.find(c => c.id === 'other-ch').body, 'Foreign text') + assert.equal(final.generatedChapterDrafts.find(d => d.id === 'other-draft').status, 'draft') + assert.deepEqual(final.chapterVersions.filter(v => v.projectId === 'book').map(v => v.id).sort(), ['v2', 'v2:before', 'v3']) + assert.equal(final.revisionVersions[0].status, 'accepted') + } finally { reopened.close() } +}) + +test('full and scoped validation reject mismatched secondary diagnostics, stored ID-only links and foreign ID collisions', async t => { + const { store } = await library(t) + const data = await store.load(), good = adopt(data) + const stale = data.consistencyReviewReports[0] + for (const explicit of [true, false]) { + for (const embedded of [true, false]) { + const b = copy(good) + if (!explicit) delete b.acceptanceReview + b.consistencyReviewReportId = stale.id + b.consistencyReviewReports = embedded ? [stale] : [] + await rejectLikeFull(store, b, /does not match the accepted draft text/) + } + } + const swapped = copy(good) + swapped.previousChapterVersion.id = swapped.chapterVersion.baseChapterVersionId = 'foreign-v' + await rejectLikeFull(store, swapped, /another project/) + const revision = revise(data) + revision.baseChapterVersionId = revision.chapterVersion.baseChapterVersionId = 'foreign-v' + await rejectLikeFull(store, revision, /does not match the target chapter/) + const old = revise(data) + delete old.revisionSession + delete old.revisionSessionId + delete old.generatedDraft + delete old.revisionVersion + data.generatedChapterDrafts.find(d => d.id === 'draft').body = 'Edited after preview' + await store.save(data) + await rejectLikeFull(store, old, /sourceContentHash/) +}) + +test('legacy receipt aliases remain visible across peer writes; failed old-library writes roll back index installation too', async t => { + const { store, peer } = await library(t) + const data = await store.load(), b = adopt(data) + peer.exec('DROP INDEX idx_alias_chapterCommitBundles; DROP INDEX idx_alias_revisionCommitBundles') + const indexes = () => peer.prepare("SELECT name FROM sqlite_schema WHERE name LIKE 'idx_alias_%' ORDER BY name").all() + const before = await store.loadSnapshot() + peer.exec(`CREATE TRIGGER fail_late BEFORE INSERT ON meta WHEN NEW.key = 'revision' + BEGIN SELECT RAISE(ABORT, 'late marker'); END`) + await assert.rejects(store.saveChapterCommitBundle(b, before.revision), /late marker/) + assert.deepEqual(await store.loadSnapshot(), before) + assert.deepEqual(indexes(), []) + peer.exec('DROP TRIGGER fail_late') + await store.saveChapterCommitBundle(b, before.revision) + assert.equal(indexes().length, 2) + const rename = peer.prepare("UPDATE entities SET id = ?, json = json_set(json, '$.id', ?) WHERE collection = ? AND id = ?") + rename.run('physical-adopt', 'physical-adopt', 'chapterCommitBundles', 'adopt') + await rejectLikeFull(store, b, /immutable/) + const revision = revise(await store.load()) + assert.equal(revision.linkedChapterCommitId, 'adopt') + await narrow(() => store.saveRevisionCommitBundle(revision)) + rename.run('physical-revise', 'physical-revise', 'revisionCommitBundles', 'revise') + await rejectLikeFull(store, revision, /immutable/) +}) + +test('the current project remains authoritative for book-wide rules, order collisions and latest job evidence', async t => { + const { store, peer } = await library(t) + const data = await store.load(), b = adopt(data), revision = revise(data) + data.hardCanonPacks.find(p => p.projectId === 'book').items.push({ + id: 'book-count', projectId: 'book', title: 'Limit', content: 'At most one of these per book.', + status: 'active', check: { kind: 'literal', mode: 'terms', terms: ['lost key', 'source manuscript', 'Revised text'], + maxPerBook: 1, enforcement: 'block' }, ...stamp + }) + await store.save(data) + await rejectLikeFull(store, b, /硬设定/) + await rejectLikeFull(store, revision, /硬设定/) + data.hardCanonPacks = [] + data.chapters.find(c => c.id === 'ch').id = 'different-chapter-same-order' + await store.save(data) + await rejectLikeFull(store, b, /目标章节已变化/) + data.chapters.find(c => c.id === 'different-chapter-same-order').id = 'ch' + data.qualityGateReports.push({ ...report, id: 'later-quality', pass: false, createdAt: t2 }) + await store.save(data) + peer.prepare("UPDATE entities SET job_id = NULL, project_id = NULL WHERE collection = 'qualityGateReports' AND id = 'later-quality'").run() + await rejectLikeFull(store, b, /immutable/) +}) From e021f1105cb984615ac5d36a9c72f213cf900cff Mon Sep 17 00:00:00 2001 From: SHIN09-git Date: Fri, 13 Nov 2026 04:55:53 +0800 Subject: [PATCH 5/8] Record repeatable long-library desktop performance measurements [skip ci] --- .../LIBRARY_STABILITY_AND_PERFORMANCE_PLAN.md | 1 + docs/RELEASE_ARTIFACTS.md | 6 +- docs/S3_LIBRARY_PERFORMANCE.md | 83 ++ scripts/benchmark-s3-library.mjs | 194 +++++ scripts/performance/s3-after.json | 713 ++++++++++++++++++ scripts/performance/s3-before.json | 692 +++++++++++++++++ scripts/performance/s3-evidence.mjs | 64 ++ scripts/performance/s3-probe.mjs | 53 ++ scripts/performance/s3-readback.mjs | 65 ++ scripts/performance/s3-seed.mjs | 121 +++ 10 files changed, 1991 insertions(+), 1 deletion(-) create mode 100644 docs/S3_LIBRARY_PERFORMANCE.md create mode 100644 scripts/benchmark-s3-library.mjs create mode 100644 scripts/performance/s3-after.json create mode 100644 scripts/performance/s3-before.json create mode 100644 scripts/performance/s3-evidence.mjs create mode 100644 scripts/performance/s3-probe.mjs create mode 100644 scripts/performance/s3-readback.mjs create mode 100644 scripts/performance/s3-seed.mjs diff --git a/docs/LIBRARY_STABILITY_AND_PERFORMANCE_PLAN.md b/docs/LIBRARY_STABILITY_AND_PERFORMANCE_PLAN.md index ceceda7..d6157c0 100644 --- a/docs/LIBRARY_STABILITY_AND_PERFORMANCE_PLAN.md +++ b/docs/LIBRARY_STABILITY_AND_PERFORMANCE_PLAN.md @@ -88,6 +88,7 @@ preview.20 实包使用 Electron 39.8.10 / better-sqlite3 12.9.0;核查时主 - **S2:** [Windows 双机验收](./S2_WINDOWS_QUALIFICATION.md)。以同一构建的安装包、两份安装报告和各自的冷启动回读作为依据,不用解包 smoke 代替安装升级。公开 preview.20 保持原样。 - **S3-A:** 界面携带项目 ID、所选书库路径与修订号,在原保存队列内更新时间。SQLite 只改目标项目及书库修订标记;JSON 后端仍需重写单文件。编辑时间不随打开项目而改变。 - **S3-B:** 采纳和修订在原事务内读取关联章节、版本、报告、候选与回执,并保留冲突拒绝及失败回滚。涉及全书计数的硬规则仍可读取当前项目的章节;不拿部分书库数据走整库保存。 +- **成品对照:** [720 章虚构库及原始数据](./S3_LIBRARY_PERFORMANCE.md)。未到备份期时,进入项目、采纳、修订的中位数分别从 1,389、863、860 ms 降至 382、220、17 ms。冷启动约 1.9 秒,仍读取完整书库。 - **S4:** 选择压缩新自动备份,不批量改写旧回执或裁剪生成历史。[备份实测](./S4_AUTO_BACKUPS.md)包含恢复、空间不足及超出解压上限的处理。虚构的 500 章样本保留占用从 541 MiB 降至 92 MiB,代价是到期写入中位数由 229 ms 增至 368 ms。启动读取整库仍是独立问题。 暂不安排流水线引擎搬目录、全库 CRUD 重构或清理作者历史。 diff --git a/docs/RELEASE_ARTIFACTS.md b/docs/RELEASE_ARTIFACTS.md index f8793a6..aa94141 100644 --- a/docs/RELEASE_ARTIFACTS.md +++ b/docs/RELEASE_ARTIFACTS.md @@ -101,7 +101,11 @@ Rebuilding the same version replaces generated output. Building a newer version Novel projects and the SQLite user database are stored elsewhere. Organizing release artifacts does not move or delete user data. -### Current Local Release (2026-11-11) +### Preview.21 Qualification Candidate + +This unsigned Windows x64 candidate is qualified separately from public releases. Both Windows jobs must use the same Setup from the same workflow run and attempt. See the [qualification procedure](./S2_WINDOWS_QUALIFICATION.md). Do not mix a locally built installer and the checksum from CI. + +### Previous Public Release (2026-11-11) Preview.20 ships the [P0–P2 long-manuscript repairs](./LONG_NOVEL_COMPLETION_REPAIR_PLAN.md). Install `release/Novel Director Setup 0.1.6-preview.20.exe` or start `release/win-unpacked/Novel Director.exe`. In an open project, Settings → Runtime information shows the application version, packaged mode and executable path. An installed older copy is not automatically updated by a new repository build. diff --git a/docs/S3_LIBRARY_PERFORMANCE.md b/docs/S3_LIBRARY_PERFORMANCE.md new file mode 100644 index 0000000..874a0b4 --- /dev/null +++ b/docs/S3_LIBRARY_PERFORMANCE.md @@ -0,0 +1,83 @@ +# S3 书库性能:同机桌面对照 + +这份记录只使用合成书库。不打开真实作品,不调用模型,不运行安装器,不评价安装升级。测试源为干净的 S1 提交 `804f738`。S1 计时使用已验收的 `tmp/s1-final/win-unpacked`,改后只认本工作树的 `release/win-unpacked`;版本号变化本身不是 S3 已打包的证据。 + +## 结果 + +**未到期时,打开项目、采纳和修订分别比 S1 少等 72.5%、74.5% 和 98.1%(比较中位数);启动没有提速。** 改后程序在本地从 `6563d15` 对应的生产代码打包,包含 S3-A/B 及 S4 自动备份。到期组测得的是这个组合成品,不是 S3 或 gzip 的单项效果。 + +| 操作 | 未到期 S1 | 未到期改后 | 到期 S1 | 到期改后 | +| --- | ---: | ---: | ---: | ---: | +| 启动至项目列表可用 | 1,882 / 1,989 | 1,957 / 1,990 | 1,924 / 2,382 | 1,921 / 1,975 | +| 进入已有项目 | 1,389 / 1,440 | 382 / 466 | 2,667 / 3,010 | 1,940 / 2,000 | +| 确认已审稿草稿至采纳成功 | 863 / 878 | 220 / 243 | 2,330 / 3,086 | 1,817 / 1,922 | +| 确认修订至正式提交成功 | 860 / 877 | 17 / 50 | 2,464 / 2,624 | 1,683 / 1,726 | + +单位毫秒,每格为 **中位数 / 最慢一次,n=5**,均另有一次不计入的热身。启动尚未触发写入,两种备份条件下启动的小幅差异不能解释为备份成本。测量依次完成,未锁定系统后台负载;只比较本机本样本。17 毫秒接近两帧就绪检查的分辨率,不代表 SQLite 事务本身耗时 17 毫秒。 + +**未到期的逐操作诊断**(各 1 次,未参与计时)。箭头左为 S1,右为改后: + +| 操作 | 整库正文读取次数 | SQLite 返回的实体 JSON 行 | IPC 请求字节 | IPC 应答字节 | 净变更实体 | +| --- | ---: | ---: | ---: | ---: | ---: | +| 启动 | 1 → 1 | 30,921 → 30,921 | 0 → 0 | 172,845,392 → 172,845,391 | 0 → 0 | +| 进入 | 0 → 0 | 0 → 0 | 172,845,273 → 246 | 235 → 282 | 1 → 1 | +| 采纳 | 1 → 0 | 30,921 → 11 | 31,944 → 31,944 | 419 → 418 | 5 → 5 | +| 修订 | 1 → 0 | 30,923 → 6 | 46,922 → 46,922 | 363 → 362 | 5 → 5 | + +S1 进入项目虽然未读取全部实体正文,但枚举了 **30,921 条集合/ID** 并接收完整书库;改后该枚举为 0,两版均只改变目标项目的实体 JSON。采纳和修订的 IPC 提交包大小没变,改善位于数据库校验读:改后仅返回 43,475 和 68,937 字节实体 JSON,带第二本书 ID 前缀的行从各 9,470 条降为 0。启动仍读完整书库并传给窗口;此处不能宣称已做历史按需加载。改后两项正式提交的结果、回执与正文均已回读;不是用减少写入冒充提速。 + +**到期时**,启动、进入、采纳、修订的整库正文读取次数由 `1/1/2/2` 变为 `1/1/1/1`;三个写操作都真实生成了备份。改后三项的 IPC 请求/应答及净变更实体数与上表一致,但备份各需读取一次完整库。仅看打开项目这一次,到期时 S1 自动 JSON 为 179,005,905 字节;改后自动 gzip 为 5,213,522 字节,**另有 179,005,905 字节的普通回退 JSON**。这次写出的两份文件合计 184,219,427 字节,不能把 gzip 单文件当成总磁盘占用。本档 720 章样本未测 7 份轮替后的长期占用;另见 [S4 的 500 章轮替实测](./S4_AUTO_BACKUPS.md#测量)。未到期的两版均已实证:单一兼容备份条目、大小、标识、修改时间及内容哈希未变。 + +**S4 取舍:** 若目标是减少打开列表的传输,值得先测只读项目概要:启动应答仍约 165 MiB,最大的生成步骤集合占 43.34%。若目标是减小新历史,两个回执集合合计占 24.54%,这是整个集合的体积,**不是可回收量**;早期采纳稿与后续修订链确有不同正文,不能用“留最后成功稿”代替历史。这些是下一次实验方向,不是已经测出的优化收益。 + +原始样本、42 个集合的字节数、SQL 形状及回读证据见 [S1 数值记录](../scripts/performance/s3-before.json) 和 [改后数值记录](../scripts/performance/s3-after.json)。包内依赖指纹一致;两版规范化依赖锁哈希均为 `785b1000108997831132713949eeadd955f0f5468c8dba4f8fd6809695392c2a`。S1 和改后 app.asar SHA-256 分别为 `4b235d3221f59c64f69a8dd7182a521f2e12e9eb296fb41053f8f65f0a9614ae`、`7727b338110c6e6cc25ca9f82304368e68cfe82154ebd6456661e99865a6572b`;同一 SQLite 种子为 `978c6f1da08db48ff3428c0e798e88f1896a1ce781494bb82f319183639a0946`。 + +## 样本与回读 + +种子 SQLite 文件为 190,050,304 字节;经 S1 存储服务保存并回读的 AppData,序列化为**无缩进 UTF-8 JSON** 后是 **172,845,159 字节(164.84 MiB)**。SQLite、JSON、自动备份是三种不同口径。配方使用已有长篇合成器,主项目 500 章,另一本 220 章;不是给小库追加无关填充文本。42 个数组集合合计 30,921 个实体。正式对照需要每轮复制同一份已关闭的 SQLite,而不是为两版分别造库。 + +| 集合 | 条数 | 紧凑 JSON 字节 | 整库占比 | +| --- | ---: | ---: | ---: | +| chapterGenerationSteps | 20,202 | 74,910,523 | 43.34% | +| chapterCommitBundles | 720 | 28,969,153 | 16.76% | +| chapterVersions | 1,800 | 16,162,133 | 9.35% | +| promptContextSnapshots | 720 | 13,739,653 | 7.95% | +| revisionCommitBundles | 360 | 13,442,541 | 7.78% | +| generatedChapterDrafts | 1,441 | 13,024,850 | 7.54% | +| chapters | 720 | 7,101,693 | 4.11% | +| generationRunTraces | 721 | 2,817,267 | 1.63% | + +每个集合按 `Buffer.byteLength(JSON.stringify(array))` **实测**,包括方括号和逗号,不含集合键名;空集合为 2 字节。整库还包含设置、键名和顶层结构。完整明细随原始数值记录给出。合成文本重复有规律,其压缩率不能外推为真实小说的节省比例。 + +两个项目每章都有较早的采纳回执和之后生成的不同草稿;每四章有两轮相连的正式修订,每本另有一次失败中断。每轮先核对桌面实际使用的是本轮 SQLite,再从这份库只读核对: +- 两本书各抽第 4 章:回执仍指向较早、状态为 accepted 的草稿;回执正文与当时版本相同。后来的草稿属于另一个已完成的生成任务,正文不同、状态仍为 draft。两份修订回执前后正文衔接、父版本衔接,最终正文与第二份修订一致。 +- 两个 unfinished 任务的失败步骤均为 generate_chapter_draft;之前有已完成且保留输出的步骤,之后有 pending 步骤。检查的是**保留续跑所需的记录**,没有实际调用模型续跑。 +- 另外核对两个**通过窗口提交**的目标:主项目第 2 章走已审稿采纳,回执关联匹配的质量报告与编辑裁定;第 500 章接受一份有来源正文哈希的修订。各新增一份正式回执;两处正式正文、修订版本及候选状态均须与提交一致。 + +## 测量口径 + +Windows x64,Ryzen 7 7800X3D,16 个逻辑处理器,33,397,133,312 字节内存。两边均核对 Electron 实际版本 44.9.1、better-sqlite3 13.0.3、原生模块哈希及包内依赖文件指纹。锁文件核对 `lock.packages`:**只删除** `packages[""].version`,递归排序对象键、保留数组顺序,对紧凑 UTF-8 JSON 求 SHA-256。这样根包名称、依赖声明、传递依赖和完整性字段仍参与比对;不要求整个文件字节相同。 + +- **计时**:实际解包程序、全新进程和临时用户目录。每种备份条件各 1 轮热身、5 轮正式样本,串行;不清操作系统文件缓存。复制种子、造备份和读回都在计时之外。启动从生成进程开始,到列表显示两个项目且目标“进入”可用;打开从原生鼠标点击到工作台显示正确项目且导航可用。采纳和修订分别从最后一次原生确认点击到界面显示该次提交成功。条件连续两个动画帧成立。选择项目、导航、阅读、首次点击与最终确认之间的等待不计入提交耗时。 +- **备份**:未到期时提供可读的合成整库 JSON,文件名为 `novel-director-