diff --git a/.agents/skills/fusion-next-step/SKILL.md b/.agents/skills/fusion-next-step/SKILL.md index 42c36a3f..c5585c08 100644 --- a/.agents/skills/fusion-next-step/SKILL.md +++ b/.agents/skills/fusion-next-step/SKILL.md @@ -17,7 +17,8 @@ description: Use when the user asks Fusion “下一步”, “接下来做什 - `git -C /Users/sean/code/fusion/fusion-api log --oneline -40` - `git -C /Users/sean/code/fusion/fusion-ui log --oneline -40` 4. 用 `rg` 搜索用户提到的关键词，范围至少包括： - - `docs/superpowers` + - `docs/implementation-plans` + - `docs/superpowers/specs` - `docs/MODEL_ACCEPTANCE_RUNBOOK.md` - 相关源码目录 5. 回答必须包含： diff --git a/.gitignore b/.gitignore index f445085a..7b617c3c 100644 --- a/.gitignore +++ b/.gitignore @@ -18,8 +18,7 @@ docs/* !docs/LITELLM_HEALTH.md !docs/TRAJECTORY_DESIGN.md !docs/superpowers/ -!docs/superpowers/plans/ -!docs/superpowers/plans/2026-08-22-trajectory-ledger-p0.md +!docs/implementation-plans/ .cursor diff --git a/.superpowers/sdd/2026-08-27-run-capability-router/progress.md b/.superpowers/sdd/2026-08-27-run-capability-router/progress.md deleted file mode 100644 index b90793df..00000000 --- a/.superpowers/sdd/2026-08-27-run-capability-router/progress.md +++ /dev/null @@ -1,57 +0,0 @@ -# SDD ledger — plan: docs/superpowers/plans/2026-08-27-run-capability-router.md - -## 预检 - -| 任务/接口 | 生产者 | 消费者 | 检查结果 | -|---|---|---|---| -| Task 1 → Task 2 | `RunCapabilityResolution`、`resolve_run_capability_route` | call config 原子物化与 Prompt 组装 | 一致；Task 2 只能消费 Task 1 冻结结果，不得二次扩容 | -| Task 1 → Task 3 | 安全 route 字段与序列化 | run config、event、Run summary | 一致；bundle fingerprint 在 Task 3 补齐，不改变路由语义 | -| Task 2 → Task 3 | `AgentLoopCallConfig.capability_resolution` 与实际 tools | lifecycle `_start_run` | 一致；route 必须在 `_start_run` 前冻结 | -| Task 3 → Task 4 | 显式 `capability_resolution` DTO | UI 归一化、投影和详情 | 一致；UI 不透传整个 run config | -| Task 1/2 → Task 5 | package、announced tools、section IDs | behavior eval schema | 一致；评分区分公告、调用和最终回答 | -| Task 3/4 → Task 5 | API/UI route 展示 | 回归和执行台账 | 一致；本地证据与 CI/dev/真实页面分层记录 | -| Task 1 自洽性 | RED 矩阵、路由纯函数、GREEN | 同一文件集与签名 | 一致；不依赖 IO/LLM | -| Task 2 自洽性 | 错误契约翻转、原子过滤、条件 Prompt | 同一 call config | 一致；provider reasoning 仍在最终工具后适配 | -| Task 3 自洽性 | run config、event、summary、旧 Run | 显式 DTO 与白名单 | 一致；实际 Prompt 指纹与 bundle fingerprint 分开 | -| Task 4 自洽性 | wire 类型、validator、projection、presentation | targeted tests/build | 一致；脚本不存在 `lint`，计划已改为显式 `npx eslint` | -| Task 5 自洽性 | fixture schema、脚本、测试、台账 | 已存在 `test/test_agent_behavior_eval.py` | 一致；无在线模型调用 | - -Ruling: 低置信不采用旧 `legacy_fallback` 全量工具 — 新 spec 的 revised recommendation 是绑定合同 — 代价是未知普通 MCP 只能在精确 alias 命中时公开。 - -Ruling: `bundle_fingerprint` 在 `_start_run` 前覆盖完整 resolution、announced tools、安全 MCP bindings、task/network/evidence policy 与 Prompt 模板版本；随后得到的实际 Prompt section/body 继续由独立 snapshot/fingerprint 证明 — 验收需同时查看能力包与实际 Prompt 两个指纹。 - -Ruling: continuation 每次创建新 Run 时按当前消息、当前授权和恢复的 task/plan policy 重新确定性路由；本期不恢复旧 route schema — spec 未要求 Skill/route 续跑冻结，且当前权限必须优先 — 代价是目录或 router 版本变化后新 continuation 可记录不同包，但旧 Run 不被改写。 - -Task 1: dispatched (BASE 7f4990af47b423113ff18e831a5acae669381c8b, agent /root/router_task1_impl) -Task 1: review failed (6 Important, 0 Critical/Minor) — knowledge-grounded 边界、Deep Research 必需工具、裸 A 到 B、外部公告摘要、verified 优先级、非出行城市误召回。 -Task 1: fix round 1/5 dispatched to /root/router_task1_impl (FIX_BASE 6aff089f4616ca130198edf20b8150331df19767). -Task 1: fix round 1/5 (6 addressed, 2 open — 自然起终点三类漏召回；“发展路线”误触发；commits 6aff089..69d0762). -Task 1: fix round 2/5 dispatched to /root/router_task1_impl (FIX_BASE 69d0762ed38d0fc7322addc2b02ceedf1146a398). -Task 1: fix round 2/5 (2 addressed, 2 open — structured endpoint 组织关系误授权；大学端点真实路线被黑名单压掉；commits 69d0762..7e25d81). -Task 1: fix round 3/5 dispatched to /root/router_task1_impl (FIX_BASE 7e25d813fd922a2d6c43e3d92099ca8bf8934c75). -Task 1: fix round 3/5 (2 addressed, 2 open — 明确路线常见地点漏召回；“产品中心/研发中心”组织误授权；commits 7e25d81..2b7d285). -Task 1: fix round 4/5 dispatched to fresh implementer (FIX_BASE 2b7d285418c43e90f2244ec5bfb04a82083d1b5d). -Task 1: fix round 4/5 (2 addressed, 2 open — 抽象流程“怎么走”误授权；普通“的路线”漏召回；commits 2b7d285..d6a92cf). -Task 1: fix round 5/5 dispatched to fresh implementer (FIX_BASE d6a92cfdb99ef0089b310bbd1bb3b13540dfd0f8). -Task 1: fix round 5/5 (1 addressed, 2 open — 任意 bounded 抽象端点仍可被“怎么走/路线”提升；commits d6a92cf..1ada245; breaker tripped). -Task 1: Ruling: 抽象端点误授权是真实且下游依赖的 load-bearing finding；Task 2 在接入 Agent Loop 前必须把未知 BOUNDED 端点改为默认拒绝，只允许确认物理地点证据。代价：未识别 POI 会进入 clarification_only，产生可恢复的工具漏召回，而不是错误开放地图能力。 -Task 1: complete (commits 7f4990a..1ada245, 5 rounds; 2 load-bearing trigger variants carried into Task 2 by ruling). - -Task 2: dispatched (BASE 1ada245fe811e23668f867a695fce6d5ac362589, agent /root/router_task2_impl). -Task 2: review failed (2 Important — 空工具 plan_on schema 未禁止 planned_tools；disable/no-FC 精确 MCP 请求漏联网边界). -Task 2: fix round 1/5 dispatched to /root/router_task2_impl (FIX_BASE 5320e4d). -Task 2: fix round 1/5 (1 addressed, 1 open — MCP alias 分类元数据仍受 max_tools/schema-bytes 执行预算裁剪；commits 5320e4d..a14ae8c). -Task 2: fix round 2/5 dispatched to /root/router_task2_impl (FIX_BASE a14ae8c). -Task 2: fix round 2/5 (1 addressed, 0 open; commits a14ae8c..2658679). -Task 2: complete (commits 1ada245..2658679, review clean). - -Task 3: dispatched (BASE 26586793924133594f79bf18aa2363557c4bc801, agent /root/router_task3_impl). -Task 3: review failed (1 Important — Trajectory resolution 仅形状校验，update_plan/非法包工具组合可进入实时与历史协议). -Task 3: fix round 1/5 dispatched to /root/router_task3_impl (FIX_BASE 92520fb). -Task 3: fix round 1/5 complete (1 Important addressed, 0 open; commits 92520fb..58b9b82; target 123 passed; adjacent 185 passed). -Task 4: complete (UI commit c239fbd; target 69 passed; Trajectory regression 303 passed; ESLint/build/diff check green; no local services, push, deploy, or browser acceptance). -Task 4: fix round 1/5 complete (2 Important addressed, 0 open; UI commits c239fbd..246430d; contract/panel target 141 passed; Trajectory regression 336 passed; ESLint/build/diff check green). -Task 3: fix round 2/5 complete (1 Important addressed, 0 open; commits 58b9b82..b664a93; target 127 passed / 73 subtests; adjacent 186 passed / 42 subtests; Ruff/format/diff check green). -Task 4: fix round 2/5 complete (1 Important addressed, 0 open; UI commits 246430d..5b603d5; target 151 passed; Trajectory regression 346 passed; ESLint/build/diff check green). -Task 5: final local implementation complete, release pending (START_BASE b664a93; COMMIT_BASE 7e49f5f after shared query-projection fix; behavior fixture 501 total / 491 route records; latest target 654 passed / 1065 subtests including production wiring, route unit 506 passed and real assembly 491 subtests; fresh authoritative API full 3489 passed / 2 skipped / 1895 subtests and static gates green; UI full 2430 passed, production build, ESLint and diff check green; Prompt/Trajectory independent review and final replacement adversarial review CLEAN; no services, CI, deploy, or browser acceptance yet). -Task 3: regression fix round 3/5 complete (1 authoritative full-regression failure addressed; commits b664a93..7e49f5f; failing test 1 passed; target 128 passed / 73 subtests; adjacent 186 passed / 42 subtests; qualified SQLite/Postgres JSON path compile checks and Ruff/format/diff green). diff --git a/.superpowers/sdd/2026-08-27-run-capability-router/task-5-implementation-report.md b/.superpowers/sdd/2026-08-27-run-capability-router/task-5-implementation-report.md deleted file mode 100644 index 7479115f..00000000 --- a/.superpowers/sdd/2026-08-27-run-capability-router/task-5-implementation-report.md +++ /dev/null @@ -1,137 +0,0 @@ -# Task 5 实现报告：能力路由行为评测与 API 回归收口 - -## 元数据 - -- 完成时间（Asia/Shanghai）：2026-08-27 19:36 CST -- 开始 BASE：`b664a932b247683bbdb381556886e5ff32093691` -- 提交 BASE：`7e49f5f`（Task 5 全量发现的共享 Trajectory query projection 回归由上游先行收口） -- Task 5 HEAD：本报告所在提交；最终 SHA 以 `git log -1` 为准 -- 工作区：`/Users/sean/code/fusion/.worktrees/run-capability-router-api` -- 范围：行为评测协议、路由/Prompt 真集成 fixture、旧 stream fixture 迁移、API 回归和本地证据文档 -- 明确未做：未启动 API/UI/Docker，未访问网络，未改 Task 3 capability/Trajectory 生产契约，未改 UI、服务、计划或 Task 3+ 测试 - -## TDD RED 证据 - -### 1. 评测协议与矩阵 RED - -先只增加协议校验、评分与矩阵测试，未实现脚本和 fixture： - -```bash -/Users/sean/code/fusion/fusion-api/.venv/bin/python -m pytest test/test_agent_behavior_eval.py -q -``` - -结果：`13 failed, 19 passed, 21 subtests passed, 5 warnings`。失败分别证明新增字段没有被验证或评分、缺失 observation 没有失败、dry-run 没有输出期望字段、fixture 不足 24+4。 - -随后为真实联结增加非空断言，单独运行： - -```bash -/Users/sean/code/fusion/fusion-api/.venv/bin/python -m pytest -q test/test_agent_behavior_eval.py::RunCapabilityBehaviorEvalIntegrationTests::test_route_fixture_runs_through_real_router_and_prompt_assembly -``` - -结果：`1 failed`，路由 fixture 数量为 0，未满足至少 29 条真实集成样本。 - -### 2. stream 旧假设 RED - -基线命令： - -```bash -/Users/sean/code/fusion/fusion-api/.venv/bin/python -m pytest -q test/test_agent_behavior_eval.py test/test_stream_handler.py -``` - -结果：`7 failed, 41 passed, 21 subtests passed, 5 warnings`。七项失败共同证明旧 fixture 仍假设普通 `hi` 或任意 URL 会得到旧全量工具；模型返回的调用在新路由下属于未公告工具。 - -迁移 fixture 后首次精确回归为 `2 failed, 5 passed`：两个 URL 用例仍未声明 `searchCapable`，且“请看 URL”不属于显式读取动作。补齐真实能力并改为“请阅读 URL”后进入 GREEN；没有增加测试后门或恢复全工具行为。 - -## 实现内容 - -### 行为评测协议 - -- `scripts/agent_behavior_eval.py` 保留既有 V1 字段与评分逻辑，新增 8 个可选 Run capability 期望字段。 -- package、resolution mode、plan mode、布尔值、字符串数组和重复项在加载阶段严格校验。 -- 精确字段按原始顺序比较；reason codes 使用必需子集语义。样本声明期望字段而 observation 缺失时明确报告缺失，空值不能冒充通过。 -- dry-run 继续不调用 LLM、搜索或浏览器，并输出新增期望字段供外部观测填充。 - -### 24 主样本与对抗矩阵 - -- fixture 总数 40：既有 V1 10 条、Run 路由 30 条。 -- 30 条路由样本包含 24 条主样本和 6 条对抗记录；对抗记录形成 5 个独立变体：物理 A→B、组织语境、用户偏好越权、禁用工具联网边界、topic switch 两个独立 observation。 -- 每条路由样本精确写出 package、announced tools、Prompt section IDs、resolution mode、必需 reason codes、effective plan mode 与 network boundary；可确定不会实际调用工具的样本另写 `expected_called_tools=[]`。 - -### 真实 router 与 Prompt assembly 联结 - -- 测试使用真实 `AMAP_PRODUCT_DEFINITIONS`、`FLYAI_TRAVEL_DEFINITIONS` 及配对 handler/binding 进入 `build_agent_loop_call_config()`。 -- 同一个 call config 再交给 `prepare_agent_loop_messages(..., preprocess_user_input=False)`，因此 section IDs 来自生产 `assemble_system_prompt` 路径。 -- 逐样本核对 resolution external tools、实际传给模型的 definitions、active handlers/bindings 对应的 announced tools、`final_tool_names`、日期标记与 Prompt sections；未复制第二套路由判断器，也没有网络调用。 - -### stream fixture 迁移 - -- 普通 stop 路径改为只断言 `app_identity`。 -- 搜索、checkpoint、失败、max-steps 与 summary-timeout 用例改用明确最新信息搜索请求。 -- URL run-start 和降级用例改用明确 URL 阅读/总结请求，并声明真实的 function calling + search 能力。 -- URL run-start 只公告 `url_read`，不恢复旧 `web_search,url_read` 全量假设。 - -## 改动文件 - -- `scripts/agent_behavior_eval.py` -- `test/fixtures/agent_behavior_eval_samples.json` -- `test/test_agent_behavior_eval.py` -- `test/test_stream_handler.py` -- `docs/superpowers/specs/2026-08-27-run-capability-router.md` -- `docs/EXECUTION_LEDGER.md` -- `.superpowers/sdd/2026-08-27-run-capability-router/progress.md` -- `.superpowers/sdd/2026-08-27-run-capability-router/task-5-implementation-report.md` - -## GREEN 与回归证据 - -评测覆盖文件： - -```bash -/Users/sean/code/fusion/fusion-api/.venv/bin/python -m pytest -q test/test_agent_behavior_eval.py -``` - -结果：`23 passed, 98 subtests passed, 5 warnings in 1.83s`。 - -精确七项 stream 迁移：`7 passed, 5 warnings in 11.67s`。完整 stream handler：`31 passed, 5 warnings in 12.02s`。 - -Task 5 指定目标集： - -```bash -/Users/sean/code/fusion/fusion-api/.venv/bin/python -m pytest -q test/test_agent_behavior_eval.py test/services/stream/test_run_capability_router.py test/services/stream/test_agent_loop_request_prep.py test/test_stream_handler.py -``` - -基于最终提交 BASE `7e49f5f` 的结果：`182 passed, 126 subtests passed, 5 warnings in 12.09s`。 - -扩大集： - -```bash -/Users/sean/code/fusion/fusion-api/.venv/bin/python -m pytest -q test/services/stream test/services/agent test/services/test_trajectory_query_service.py test/test_trajectory_api.py test/test_agent_behavior_eval.py test/test_stream_handler.py -``` - -基于最终提交 BASE `7e49f5f` 的结果：`1239 passed, 570 subtests passed, 8 warnings in 27.79s`。 - -仓库权威全量首次运行：`1 failed, 3056 passed, 2 skipped, 972 subtests passed, 9 warnings in 47.46s`。唯一失败为 Task 3 共享查询投影测试 `test_general_trajectory_and_other_node_details_do_not_read_system_prompt_body`：general trajectory SQL 读取了 `agent_sessions.config`。Task 5 没有编辑该共享路径；上游提交 `7e49f5f fix: 收窄轨迹能力配置读取` 后，先重跑原失败单项：`1 passed, 5 warnings in 1.96s`。 - -随后在最终提交 BASE 上重跑权威全量： - -```bash -/Users/sean/code/fusion/fusion-api/.venv/bin/python -m pytest -q --tb=short test/ -``` - -结果：`3058 passed, 2 skipped, 972 subtests passed, 9 warnings in 46.00s`。 - -静态门禁初轮：`ruff check` 通过；`ruff format --check` 正确发现 `test/test_agent_behavior_eval.py` 需格式化。执行单文件 Ruff format 后，最终 `ruff check`、`ruff format --check` 与暂存差异检查均通过。 - -## 自审与剩余风险 - -- V1 样本继续可加载、评分和 dry-run；新字段全部可选，不强迫旧 observation 提供 Run route 数据。 -- 路由 fixture 的期望值没有从实际结果运行时回填，错误 production route 会让固定 fixture 断言失败。 -- definitions、handlers、bindings、announced/final tools 和 Prompt sections 均从一个真实 call config 派生；测试没有发送 schema 给模型，也没有执行外部 handler。 -- 旧 stream fixture 只为真正需要工具的场景补充明确用户意图；普通直接回答保持空工具。 -- 全量暴露的共享 query projection 回归已由独立上游提交关闭；Task 5 提交不夹带该生产修复，也没有把首次失败改写成 Task 5 GREEN。 -- 本地测试未证明 CI、部署、多模型真实选择质量、实际 tool calls、UI 实时/刷新一致性或登录态浏览器体验。这些属于后续发布门禁，Task 5 没有伪造结论。 - -## 最终发布审查补强 - -最终多轮对抗审查在发布前继续拦截了显式内置/产品工具 deny 被自然语言重新放开、定义类稳定知识被 fresh/verified/product 词法抢占、定义尾部 `in ...` 吞掉真实新闻/价格查询、合法 URL query 被拆成公网搜索动作、未知交通方式回退全三工具、当前请求作用域和回指对象枚举不足、`never go online`/禁止访问网络等全局中英文联网禁用枚举不足、交通方式否定跨逗号/冒号/破折号子句、air pollution 误判以及产品子集筛选/否定疑问漏召回。修复保持 router version `2026-08-27.2`，以 hard deny、按名词类型区分的定义请求前置判定、URI/产品动作子句边界和未知方式 fail-closed 处理根因，不恢复低置信全工具包。 - -最终 fixture 共 501 条行为样本，其中 491 条通过真实 `build_agent_loop_call_config()` 与 `prepare_agent_loop_messages()` 校验 package、definitions、handlers、bindings、announced/final tools、Prompt sections、plan/date/boundary 与最多三工具边界。最新目标集（含生产 wiring 与 lifecycle 指纹契约）为 `654 passed + 1065 subtests`，其中路由单测 `506 passed`、真实组装 fixture `491 subtests`。API 权威全量 `3489 passed, 2 skipped, 1895 subtests`，Ruff、任务改动文件 format check 与 diff check 通过；能力包指纹覆盖完整 resolution、announced tools、安全 MCP bindings、task/network/evidence policy 与模板版本，实际 Prompt 仍由独立 snapshot/fingerprint 证明。UI 全量 `2430 passed` 且 production build、ESLint 与 diff check 通过；最终替换式对抗审查结论为 CLEAN。CI、dev 部署和真实登录态页面仍属于发布门禁。 diff --git a/AGENTS.md b/AGENTS.md index 8c6245ad..3e68230d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -34,7 +34,7 @@ docker-compose up -d # Docker 启动 8. **真实 Chrome 回归**：涉及登录态、联网回答、消息流、设置页或其他用户可见链路时，只能复用用户已经打开的、与任务匹配的 Chrome 标签做补充回归。**严禁打开新的 Chrome、新标签、新窗口、`about:blank`、isolated context，严禁使用会创建或切到独立浏览器目标的 DevTools/CDP 通道。**若没有可复用的匹配标签，必须停止并说明阻塞，不能自行新开页面；只验证关键路径、真实网络和可见异常，不用本地 Fusion 服务替代。 9. **变更后验证**：运行与改动匹配的 pytest/ruff；涉及数据流、联网、Redis Stream、认证或部署逻辑时扩大回归范围并更新文档。 10. **CI/CD 收尾**：按正常 Git 流程中文提交并包含 `Co-Authored-By`，push 后持续监控 GitHub Actions 和 dev 部署；失败时拉日志定位并修复。 -11. **下一步建议必须查执行记录**：用户问“下一步”“接下来做什么”“还有什么优化”“还能怎么加强”时，必须先读 [docs/EXECUTION_LEDGER.md](docs/EXECUTION_LEDGER.md)，再查 `fusion-api` / `fusion-ui` 最近 `git log` 和相关 `docs/superpowers`。禁止凭 memory 或印象回答，禁止重复建议台账里已经完成的方向。可使用 repo skill `fusion-next-step`。 +11. **下一步建议必须查执行记录**：用户问“下一步”“接下来做什么”“还有什么优化”“还能怎么加强”时，必须先读 [docs/EXECUTION_LEDGER.md](docs/EXECUTION_LEDGER.md)，再查 `fusion-api` / `fusion-ui` 最近 `git log` 和相关 `docs/implementation-plans`、`docs/superpowers/specs`。禁止凭 memory 或印象回答，禁止重复建议台账里已经完成的方向。可使用 repo skill `fusion-next-step`。 ## Code Review Rules diff --git a/docs/EXECUTION_LEDGER.md b/docs/EXECUTION_LEDGER.md index bf745e3c..52d9abb3 100644 --- a/docs/EXECUTION_LEDGER.md +++ b/docs/EXECUTION_LEDGER.md @@ -1,6 +1,6 @@ # Fusion 执行台账 -> 本文件是 Fusion 项目的执行事实源，用来避免重复提出已经实施过的方向。回答“下一步”“还能怎么优化”“接下来做什么”之前，必须先读本文件，再核对两个子仓的 `git log` 和相关 `docs/superpowers` 记录。 +> 本文件是 Fusion 项目的执行事实源，用来避免重复提出已经实施过的方向。回答“下一步”“还能怎么优化”“接下来做什么”之前，必须先读本文件，再核对两个子仓的 `git log` 和相关 `docs/implementation-plans`、`docs/superpowers/specs` 记录。 ## 使用规则 @@ -37,6 +37,7 @@ | 管理员压测审计详情 | 已完成 v1.1 | `fusion-api` commit `64d3452`、`fusion-ui` commit `2e3d857`；列表与详情安全契约分离、存量脏数据安全降级、L1-L4/资源/清理结果结构化详情、按需请求、响应式布局和管理页 no-store；生产登录态 Chrome 验收通过 | | 管理员审计安全与身份展示 | 已完成 v1.2 | `fusion-api` commit `d036d89`、`fusion-ui` commit `9d1075d`；审计内容严格白名单投影、签名 URL/令牌脱敏、历史 schema 安全降级、用户昵称/用户名/ID 三元组、详情竞态隔离、Agent/tool 独立分页、压测指标语义与管理页 CSP 收紧；对抗式复审及生产登录态 Chrome 验收通过 | | 管理员用户详情与对话联动 | 已完成 v1.3 | `fusion-ui` commit `d2d473b`；用户详情改为当前视口立即可见的弹窗，统一 loading/错误/重试并隔离迟到请求；可从详情直接进入该用户对话，用户筛选仅作用于本次关联导航；生产登录态 Chrome 验收通过 | +| Run 级 Skills 运行时 | 已完成 MVP | `fusion-api` merge commits `64049e5`（PR #76 引入 Run 级 Skills 运行时）、`3ba19f9`（PR #77 修正 skills CRLF 与 release hash）、`22f8625`（PR #79 修正 fresh release intent 路由）；Skill registry (`app/ai/skills/registry.py`)、`verified-research/1.0.0` skill、Run capability 路由、Trajectory API 与 skills 详情端点、agent loop wiring 与 previous run skill release，含配套测试 | | 管理员时间展示与返回路由 | 已完成 v1.4 | `fusion-ui` commit `402644d`；对话列表展示创建/更新时间；Tab、用户详情、用户对话筛选、对话详情与压测详情以 URL 为事实源，支持手势返回、刷新恢复、深链接、旧压测记录跨页加载与 403 净 URL；生产登录态 Chrome 验收通过 | | 管理员模型运营中心 | 已完成 v1 | `fusion-api` commits `5624241`, `1a2b456`、`fusion-ui` commit `3097c6b`；LiteLLM 当前目录与历史模型并集、健康/能力、持久化使用、Agent/压测摘要、模型到对话 URL 联动、目录降级和严格安全口径；`docs/acceptance/2026-07-12-admin-model-operations-v1.md` | @@ -116,6 +117,6 @@ 2. 运行并阅读： - `git -C /Users/sean/code/fusion/fusion-api log --oneline -40` - `git -C /Users/sean/code/fusion/fusion-ui log --oneline -40` -3. 用 `rg` 搜索相关关键词，至少覆盖 `docs/superpowers` 和 `docs/MODEL_ACCEPTANCE_RUNBOOK.md`。 +3. 用 `rg` 搜索相关关键词，至少覆盖 `docs/implementation-plans`、`docs/superpowers/specs` 和 `docs/MODEL_ACCEPTANCE_RUNBOOK.md`。 4. 先列“已完成事实”，再列“不能重复建议”，最后才给新的建议。 5. 如果没有高置信下一步，直接说“当前不建议继续开基础设施优化坑”，不要硬凑方向。 diff --git a/docs/superpowers/plans/2026-06-24-dynamic-network-search.md b/docs/superpowers/plans/2026-06-24-dynamic-network-search.md deleted file mode 100644 index 621bda5d..00000000 --- a/docs/superpowers/plans/2026-06-24-dynamic-network-search.md +++ /dev/null @@ -1,324 +0,0 @@ -# 动态联网搜索与读取预算 Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** 让 `web_search` 支持模型动态指定候选数量和搜索意图，同时用后端预算限制 search/read 次数，并让诊断面板展示关键联网参数。 - -**Architecture:** `fusion-api` 负责工具 schema、参数归一化、单轮联网预算和诊断字段；`search-service` 已支持 count/domain/freshness，第一版只由 `fusion-api` 适配调用参数；`fusion-ui` 只消费后端诊断字段并展示，不新增用户设置。 - -**Tech Stack:** FastAPI, Pydantic, Python unittest, Next.js, React, TypeScript, Vitest. - ---- - -## 文件结构 - -- `fusion-api/app/ai/tools.py`：扩展 `web_search` 和 `url_read` 工具 schema。 -- `fusion-api/app/services/external/search_client.py`：扩展 search-service 请求参数。 -- `fusion-api/app/services/stream/network_budget.py`：新增单轮联网预算和参数归一化 helper。 -- `fusion-api/app/services/stream/tool_executor.py`：在执行工具前应用预算和归一化参数。 -- `fusion-api/app/services/stream/runner.py`：每个 assistant run 创建一个 `NetworkToolBudget`。 -- `fusion-api/app/services/tool_handlers/web_search.py`：使用归一化参数，记录 metadata，限制注入 LLM 的搜索摘要数量。 -- `fusion-api/app/services/tool_handlers/url_read.py`：保留并截断 `reason`，写入 block 和日志数据。 -- `fusion-api/app/schemas/chat.py`：为 `SearchBlock`、`UrlBlock` 增加联网 metadata 字段。 -- `fusion-api/app/schemas/network_diagnostics.py`：为诊断工具项增加 count、intent、domains、recency_days、context_count、budget_limited。 -- `fusion-api/app/services/network_diagnostics_service.py`：从工具日志 input/output 派生新诊断字段。 -- `fusion-ui/src/types/conversation.ts`：补齐 SearchBlock/UrlBlock metadata 类型。 -- `fusion-ui/src/types/networkDiagnostics.ts`：补齐诊断字段类型。 -- `fusion-ui/src/components/chat/networkDiagnosticsModel.ts`：把新字段派生成展示文案。 -- `fusion-ui/src/components/chat/NetworkDiagnosticsPanel.tsx`：展示 count、intent、使用数、reason、触顶信息。 - -## Task 1: 后端工具 schema 与参数归一化 - -**Files:** -- Modify: `fusion-api/app/ai/tools.py` -- Create: `fusion-api/app/services/stream/network_budget.py` -- Test: `fusion-api/test/test_ai_tools.py` -- Test: `fusion-api/test/services/stream/test_network_budget.py` - -- [ ] **Step 1: Write failing schema tests** - -Add tests that assert `web_search` exposes optional `count`、`intent`、`domains`、`recency_days`, and `url_read` exposes optional `reason`. - -Run: `cd fusion-api && python3 -m unittest test.test_ai_tools -v` - -Expected: FAIL because those schema fields are absent. - -- [ ] **Step 2: Write failing budget tests** - -Create `fusion-api/test/services/stream/test_network_budget.py` covering: - -- `count` defaults to 5. -- `count` clamps to 3..10. -- unsupported `intent` is dropped. -- `domains` keeps at most 5 valid domains. -- `recency_days` clamps to 1..365. -- more than 3 search calls returns a degraded `ToolResult`. -- more than 5 url reads returns a degraded `ToolResult`. - -Run: `cd fusion-api && python3 -m unittest test.services.stream.test_network_budget -v` - -Expected: FAIL because `NetworkToolBudget` does not exist. - -- [ ] **Step 3: Implement minimal schema and budget helper** - -Add `NetworkToolBudget` with methods: - -- `prepare_web_search_args(args: dict) -> tuple[dict, ToolResult | None]` -- `prepare_url_read_args(args: dict) -> tuple[dict, ToolResult | None]` - -The helper should return normalized args and `None` when execution may continue; when budget is exhausted, return original-ish args and a degraded `ToolResult`. - -- [ ] **Step 4: Verify Task 1 tests** - -Run: - -```bash -cd fusion-api -python3 -m unittest test.test_ai_tools test.services.stream.test_network_budget -v -``` - -Expected: PASS. - -## Task 2: 后端工具执行、search-service 参数透传和上下文裁剪 - -**Files:** -- Modify: `fusion-api/app/services/external/search_client.py` -- Modify: `fusion-api/app/services/stream/tool_executor.py` -- Modify: `fusion-api/app/services/stream/runner.py` -- Modify: `fusion-api/app/services/tool_handlers/web_search.py` -- Modify: `fusion-api/app/services/tool_handlers/url_read.py` -- Modify: `fusion-api/app/schemas/chat.py` -- Test: `fusion-api/test/test_search_client.py` -- Test: `fusion-api/test/test_tool_executor.py` -- Test: `fusion-api/test/test_tool_handlers.py` - -- [ ] **Step 1: Write failing search client tests** - -Extend `test_search_client.py` so `search_web("q", count=8, domains=["openai.com"], recency_days=30)` posts: - -- `count: 8` -- `domain_filters: ["openai.com"]` -- `freshness: "pm"` for 30 days - -Run: `cd fusion-api && python3 -m unittest test.test_search_client -v` - -Expected: FAIL because the function only accepts `query, count` and always sends `freshness: "pw"`. - -- [ ] **Step 2: Write failing executor budget tests** - -Extend `test_tool_executor.py` with a budget object and assert: - -- handler receives normalized args, not raw args. -- handler.log records normalized `input_params`. -- a fourth `web_search` returns degraded without invoking handler.execute. - -Run: `cd fusion-api && python3 -m unittest test.test_tool_executor -v` - -Expected: FAIL because `execute_tools_parallel` has no budget parameter. - -- [ ] **Step 3: Write failing handler metadata tests** - -Extend `test_tool_handlers.py` to assert: - -- `WebSearchHandler.execute()` passes normalized count/domains/recency to `search_web`. -- `SearchBlock` includes `intent`、`requested_count`、`actual_count`、`context_source_count`、`budget_limited`。 -- `format_llm_context()` includes at most 8 search sources. -- `UrlReadHandler.execute()` stores truncated `reason` in result data. -- `UrlBlock` includes `reason`。 - -Run: `cd fusion-api && python3 -m unittest test.test_tool_handlers -v` - -Expected: FAIL because these fields are missing. - -- [ ] **Step 4: Implement search client and executor integration** - -Update `search_web()` signature to: - -```python -async def search_web( - query: str, - count: int = 5, - *, - domains: list[str] | None = None, - recency_days: int | None = None, -) -> list[SearchSource]: -``` - -Map `recency_days` to search-service freshness: - -- `<= 1`: `pd` -- `<= 7`: `pw` -- `<= 31`: `pm` -- otherwise: `py` - -Pass `domain_filters` only when domains are non-empty. - -Update `execute_tools_parallel(..., network_budget: NetworkToolBudget | None = None)` so budget is applied after JSON parsing and before emitting `tool_call_started`. - -- [ ] **Step 5: Implement handler metadata and context limits** - -`WebSearchHandler.execute()` should read normalized `count`、`intent`、`domains`、`recency_days` from args, call `search_web()`, and write metadata into `result.data`. - -`WebSearchHandler.format_llm_context()` should use `context_sources = sources[:8]` and mention when search results were truncated for context. - -`UrlReadHandler.execute()` should normalize `reason` to max 160 chars and keep it in result data; `build_content_block()` should copy it to `UrlBlock`. - -- [ ] **Step 6: Verify Task 2 tests** - -Run: - -```bash -cd fusion-api -python3 -m unittest test.test_search_client test.test_tool_executor test.test_tool_handlers -v -``` - -Expected: PASS. - -## Task 3: 后端联网诊断字段 - -**Files:** -- Modify: `fusion-api/app/schemas/network_diagnostics.py` -- Modify: `fusion-api/app/services/network_diagnostics_service.py` -- Test: `fusion-api/test/test_network_diagnostics.py` -- Test: `fusion-api/test/test_network_diagnostics_api.py` - -- [ ] **Step 1: Write failing diagnostics tests** - -Extend diagnostics tests so a `web_search` ToolCallLog with input/output metadata returns: - -- `requested_count` -- `actual_count` -- `context_count` -- `intent` -- `domains` -- `recency_days` -- `budget_limited` - -Extend url_read diagnostics so input `reason` is returned as `reason` for successful reads. - -Run: - -```bash -cd fusion-api -python3 -m unittest test.test_network_diagnostics test.test_network_diagnostics_api -v -``` - -Expected: FAIL because schema and service do not expose those fields. - -- [ ] **Step 2: Implement diagnostics mapping** - -Add optional fields to `NetworkDiagnosticsToolItem` and derive them from `log.input_params` plus `log.output_data`. - -For `url_read`, prefer explicit input `reason` when `log.error_message` is empty. - -- [ ] **Step 3: Verify Task 3 tests** - -Run: - -```bash -cd fusion-api -python3 -m unittest test.test_network_diagnostics test.test_network_diagnostics_api -v -``` - -Expected: PASS. - -## Task 4: 前端类型和诊断展示 - -**Files:** -- Modify: `fusion-ui/src/types/conversation.ts` -- Modify: `fusion-ui/src/types/networkDiagnostics.ts` -- Modify: `fusion-ui/src/components/chat/networkDiagnosticsModel.ts` -- Modify: `fusion-ui/src/components/chat/NetworkDiagnosticsPanel.tsx` -- Test: `fusion-ui/src/components/chat/networkDiagnosticsModel.test.ts` -- Test: `fusion-ui/src/components/chat/NetworkDiagnosticsPanel.test.tsx` - -- [ ] **Step 1: Write failing frontend model tests** - -Add tests asserting diagnostics process items include readable details: - -- `intent: comparison` -- `请求 8 条` -- `返回 7 条` -- `用于上下文 6 条` -- `读取原因：需要核实官方原文细节` -- budget limited item shows `已达联网预算` - -Run: - -```bash -cd fusion-ui -npm test -- src/components/chat/networkDiagnosticsModel.test.ts src/components/chat/NetworkDiagnosticsPanel.test.tsx -``` - -Expected: FAIL because model and panel do not expose these details. - -- [ ] **Step 2: Implement frontend model and panel** - -Add optional fields to TS types and derive `detailParts: string[]` on process items. Render those parts as compact secondary text/chips under each process item. - -- [ ] **Step 3: Verify Task 4 tests** - -Run: - -```bash -cd fusion-ui -npm test -- src/components/chat/networkDiagnosticsModel.test.ts src/components/chat/NetworkDiagnosticsPanel.test.tsx -``` - -Expected: PASS. - -## Task 5: Full verification and commits - -**Files:** -- All modified files from Tasks 1-4. - -- [ ] **Step 1: Run backend targeted tests** - -```bash -cd fusion-api -python3 -m unittest \ - test.test_ai_tools \ - test.services.stream.test_network_budget \ - test.test_search_client \ - test.test_tool_executor \ - test.test_tool_handlers \ - test.test_network_diagnostics \ - test.test_network_diagnostics_api \ - -v -``` - -Expected: PASS. - -- [ ] **Step 2: Run frontend targeted tests** - -```bash -cd fusion-ui -npm test -- \ - src/components/chat/networkDiagnosticsModel.test.ts \ - src/components/chat/NetworkDiagnosticsPanel.test.tsx -``` - -Expected: PASS. - -- [ ] **Step 3: Run broader checks** - -```bash -cd fusion-api && python3 -m unittest discover test -v -cd fusion-ui && npm test -``` - -Expected: PASS. - -- [ ] **Step 4: Commit per repo** - -Use Chinese commit messages and include: - -```text -Co-Authored-By: Codex -``` - -Suggested commits: - -- `feat: 支持动态联网搜索预算` -- `feat: 展示联网搜索预算诊断` - diff --git a/docs/superpowers/plans/2026-06-24-network-diagnostics.md b/docs/superpowers/plans/2026-06-24-network-diagnostics.md deleted file mode 100644 index 4a71ea9c..00000000 --- a/docs/superpowers/plans/2026-06-24-network-diagnostics.md +++ /dev/null @@ -1,1222 +0,0 @@ -# 会话级联网诊断 Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** 让历史联网回答可回看本次用了哪些工具、耗时多少、哪些失败或降级，并按普通用户/管理员权限展示不同粒度。 - -**Architecture:** 后端先修正 `tool_call_logs.message_id` 关联，再新增只读 diagnostics 聚合 API，从 `agent_sessions`、`agent_steps`、`tool_call_logs` 按 assistant message 拼出诊断模型。前端不改变实时 `agent_event` timeline，只在回答依据侧栏懒加载 diagnostics，并展示普通用户摘要和管理员明细。 - -**Tech Stack:** FastAPI、SQLAlchemy、Pydantic、pytest、Next.js 15、React 19、Redux、Vitest、Testing Library。 - ---- - -## 文件结构 - -### 后端 `fusion-api` - -- Modify: `app/services/tool_handlers/base.py` - - 让 `BaseToolHandler.log()` 接收 `message_id` 并传给 `log_tool_call()`。 -- Modify: `app/services/stream/tool_executor.py` - - 保持现有 `message_id=message_id` 调用路径；只在必要时调整类型注解。 -- Create: `app/schemas/network_diagnostics.py` - - 定义 diagnostics 响应 schema。 -- Create: `app/services/network_diagnostics_service.py` - - 聚合 agent session、step、tool call log，输出脱敏/管理员模型。 -- Modify: `app/api/deps.py` - - 增加 `get_network_diagnostics_service()`。 -- Modify: `app/api/chat.py` - - 新增 `GET /conversations/{conversation_id}/messages/{message_id}/diagnostics`。 -- Test: `test/test_tool_handlers.py` - - 覆盖 `BaseToolHandler.log()` 透传 message_id。 -- Test: `test/test_tool_executor.py` - - 保留并确认 `execute_tools_parallel()` 传 message_id 给 handler.log。 -- Test: `test/test_network_diagnostics.py` - - 覆盖权限、空状态、聚合、脱敏、管理员明细。 - -### 前端 `fusion-ui` - -- Create: `src/types/networkDiagnostics.ts` - - 定义后端响应类型。 -- Create: `src/lib/api/chatDiagnostics.ts` - - 新增 diagnostics API client。 -- Create: `src/lib/api/chatDiagnostics.test.ts` - - 覆盖请求路径和错误传播。 -- Create: `src/components/chat/networkDiagnosticsModel.ts` - - 纯函数派生摘要、异常列表、管理员明细展示模型。 -- Create: `src/components/chat/networkDiagnosticsModel.test.ts` - - 覆盖文案和权限粒度。 -- Create: `src/components/chat/NetworkDiagnosticsPanel.tsx` - - 侧栏中的联网诊断分区。 -- Modify: `src/components/chat/AnswerEvidenceSidebar.tsx` - - 接收 diagnostics model/loading/error，并渲染 `NetworkDiagnosticsPanel`。 -- Modify: `src/components/chat/AssistantMessage.tsx` - - 侧栏打开时懒加载 diagnostics。 -- Modify: `src/components/chat/AnswerEvidence.test.tsx` - - 保证无来源但有诊断异常入口时仍可打开侧栏。 -- Modify: `src/components/chat/AnswerEvidenceSidebar.test.tsx` - - 覆盖联网诊断分区。 -- Modify: `src/components/chat/AssistantMessage.test.tsx` - - 覆盖打开侧栏触发 diagnostics 拉取。 - ---- - -## Task 1: 后端修复工具日志 message_id 关联 - -**Files:** -- Modify: `fusion-api/app/services/tool_handlers/base.py` -- Test: `fusion-api/test/test_tool_handlers.py` -- Verify: `fusion-api/test/test_tool_executor.py` - -- [ ] **Step 1: 写失败测试** - -在 `test/test_tool_handlers.py` 的 `ExecuteWithEmitterTests` 前新增测试类： - -```python -class ToolHandlerLogTests(unittest.IsolatedAsyncioTestCase): - async def test_log_passes_message_id_to_agent_logger(self): - from unittest.mock import AsyncMock, MagicMock, patch - - from app.services.tool_handlers.base import BaseToolHandler, ToolResult - - class _Stub(BaseToolHandler): - tool_name = "web_search" - sse_event_prefix = "search" - - async def execute(self, args): - return ToolResult(status="success", data={}) - - def build_content_block(self, result, block_id, log_id): - return MagicMock() - - def format_llm_context(self, result): - return "" - - handler = _Stub() - result = ToolResult(status="success", duration_ms=12, data={"result_count": 1}) - - with patch( - "app.services.tool_handlers.base.log_tool_call", - new_callable=AsyncMock, - ) as mock_log_tool_call: - await handler.log( - log_id="log-1", - conversation_id="conv-1", - user_id="user-1", - model_id="model-1", - provider="provider-1", - result=result, - input_params={"query": "redis"}, - trace_id="trace-1", - step_number=1, - message_id="assistant-1", - ) - - mock_log_tool_call.assert_awaited_once() - assert mock_log_tool_call.await_args.kwargs["message_id"] == "assistant-1" -``` - -- [ ] **Step 2: 运行测试确认失败** - -Run: - -```bash -cd /Users/sean/code/fusion/fusion-api -.venv311/bin/python -m pytest test/test_tool_handlers.py::ToolHandlerLogTests::test_log_passes_message_id_to_agent_logger -q -``` - -Expected: FAIL，原因是 `BaseToolHandler.log()` 不接受 `message_id` 参数。 - -- [ ] **Step 3: 实现最小修复** - -修改 `app/services/tool_handlers/base.py`： - -```python - async def log( - self, - log_id: str, - conversation_id: str, - user_id: str, - model_id: str, - provider: str, - result: ToolResult, - input_params: dict, - trace_id: str = None, - step_number: int = None, - message_id: str | None = None, - ) -> None: - """异步记录 ToolCallLog""" - task = asyncio.create_task( - log_tool_call( - log_id=log_id, - conversation_id=conversation_id, - message_id=message_id, - user_id=user_id, - tool_name=self.tool_name, - status=result.status, - duration_ms=result.duration_ms, - model_id=model_id, - provider=provider, - input_params=input_params, - output_data=_serialize_for_json(result.data), - error_message=result.error_message, - trace_id=trace_id, - step_number=step_number, - ) - ) - task.add_done_callback(_task_done_callback) -``` - -- [ ] **Step 4: 运行目标测试确认通过** - -Run: - -```bash -cd /Users/sean/code/fusion/fusion-api -.venv311/bin/python -m pytest test/test_tool_handlers.py::ToolHandlerLogTests::test_log_passes_message_id_to_agent_logger test/test_tool_executor.py::ToolExecutorMessageIdTests::test_execute_tools_parallel_passes_message_id_to_handler_log -q -``` - -Expected: `2 passed`。 - -- [ ] **Step 5: 提交** - -```bash -cd /Users/sean/code/fusion/fusion-api -git add app/services/tool_handlers/base.py test/test_tool_handlers.py test/test_tool_executor.py -git commit -m "fix: 关联工具日志消息ID" -m "Co-Authored-By: Codex " -``` - ---- - -## Task 2: 后端新增 diagnostics schema 和聚合服务 - -**Files:** -- Create: `fusion-api/app/schemas/network_diagnostics.py` -- Create: `fusion-api/app/services/network_diagnostics_service.py` -- Test: `fusion-api/test/test_network_diagnostics.py` - -- [ ] **Step 1: 写 schema 文件** - -Create `app/schemas/network_diagnostics.py`: - -```python -from datetime import datetime -from typing import Any, Literal - -from pydantic import BaseModel, Field - - -ToolStatus = Literal["success", "failed", "degraded", "interrupted"] - - -class NetworkDiagnosticsSummary(BaseModel): - total_duration_ms: int | None = None - total_steps: int = 0 - total_tool_calls: int = 0 - search_calls: int = 0 - url_read_calls: int = 0 - success_count: int = 0 - failed_count: int = 0 - degraded_count: int = 0 - interrupted_count: int = 0 - limit_reason: str | None = None - run_status: str | None = None - - -class NetworkDiagnosticsToolItem(BaseModel): - tool_call_log_id: str - tool_name: str - status: ToolStatus - duration_ms: int | None = None - target: str = "" - result_count: int | None = None - reason: str | None = None - started_at: datetime | None = None - admin: dict[str, Any] | None = None - - -class NetworkDiagnosticsResponse(BaseModel): - conversation_id: str - message_id: str - run_id: str | None = None - visibility: Literal["user", "admin"] = "user" - summary: NetworkDiagnosticsSummary = Field(default_factory=NetworkDiagnosticsSummary) - tools: list[NetworkDiagnosticsToolItem] = Field(default_factory=list) - is_empty: bool = False -``` - -- [ ] **Step 2: 写服务测试** - -Create `test/test_network_diagnostics.py` with these core tests first: - -```python -import os -import unittest - -os.environ.setdefault("DATABASE_URL", "sqlite:///./fusion-test.db") - -from app.db.models import AgentSession, Message, ToolCallLog # noqa: E402 -from app.services.network_diagnostics_service import NetworkDiagnosticsService # noqa: E402 - - -class NetworkDiagnosticsServiceTests(unittest.TestCase): - def test_empty_response_for_message_without_agent_rows(self): - db = unittest.mock.MagicMock() - db.query.return_value.filter.return_value.order_by.return_value.first.return_value = None - service = NetworkDiagnosticsService(db) - - result = service.build_for_message( - conversation_id="conv-1", - message_id="assistant-1", - is_admin=False, - ) - - self.assertTrue(result.is_empty) - self.assertEqual(result.summary.total_tool_calls, 0) - self.assertEqual(result.tools, []) - - def test_tool_item_is_sanitized_for_user(self): - service = NetworkDiagnosticsService(unittest.mock.MagicMock()) - log = ToolCallLog( - id="log-1", - conversation_id="conv-1", - message_id="assistant-1", - user_id="user-1", - tool_name="web_search", - status="success", - duration_ms=123, - model_id="model", - provider="provider", - input_params={"query": "redis stream"}, - output_data={"result_count": 5, "secret": "hidden"}, - trace_id="trace-1", - step_number=2, - ) - - item = service._tool_item_from_log(log, is_admin=False) - - self.assertEqual(item.target, "redis stream") - self.assertEqual(item.result_count, 5) - self.assertIsNone(item.admin) - - def test_tool_item_includes_admin_fields_for_admin(self): - service = NetworkDiagnosticsService(unittest.mock.MagicMock()) - log = ToolCallLog( - id="log-1", - conversation_id="conv-1", - message_id="assistant-1", - user_id="user-1", - tool_name="url_read", - status="failed", - error_message="timeout", - duration_ms=5000, - model_id="model", - provider="provider", - input_params={"url": "https://example.com/a"}, - output_data={"content": "must not leak"}, - trace_id="trace-1", - step_number=1, - ) - - item = service._tool_item_from_log(log, is_admin=True) - - self.assertEqual(item.target, "https://example.com/a") - self.assertEqual(item.reason, "timeout") - self.assertEqual(item.admin["trace_id"], "trace-1") - self.assertEqual(item.admin["input_params"], {"url": "https://example.com/a"}) - self.assertNotIn("output_data", item.admin) -``` - -在文件顶部补： - -```python -import unittest.mock -``` - -- [ ] **Step 3: 运行测试确认失败** - -Run: - -```bash -cd /Users/sean/code/fusion/fusion-api -.venv311/bin/python -m pytest test/test_network_diagnostics.py -q -``` - -Expected: FAIL，原因是 `app.services.network_diagnostics_service` 不存在。 - -- [ ] **Step 4: 实现服务** - -Create `app/services/network_diagnostics_service.py`: - -```python -from __future__ import annotations - -from typing import Any - -from sqlalchemy.orm import Session - -from app.db.models import AgentSession, ToolCallLog -from app.schemas.network_diagnostics import ( - NetworkDiagnosticsResponse, - NetworkDiagnosticsSummary, - NetworkDiagnosticsToolItem, -) - - -class NetworkDiagnosticsService: - def __init__(self, db: Session): - self.db = db - - def build_for_message( - self, - *, - conversation_id: str, - message_id: str, - is_admin: bool, - ) -> NetworkDiagnosticsResponse: - session = ( - self.db.query(AgentSession) - .filter( - AgentSession.conversation_id == conversation_id, - AgentSession.message_id == message_id, - ) - .order_by(AgentSession.created_at.desc()) - .first() - ) - logs = ( - self.db.query(ToolCallLog) - .filter( - ToolCallLog.conversation_id == conversation_id, - ToolCallLog.message_id == message_id, - ) - .order_by(ToolCallLog.created_at.asc()) - .all() - ) - - tools = [self._tool_item_from_log(log, is_admin=is_admin) for log in logs] - is_empty = session is None and not tools - return NetworkDiagnosticsResponse( - conversation_id=conversation_id, - message_id=message_id, - run_id=session.id if session else None, - visibility="admin" if is_admin else "user", - summary=self._build_summary(session, tools), - tools=tools, - is_empty=is_empty, - ) - - def _build_summary( - self, - session: AgentSession | None, - tools: list[NetworkDiagnosticsToolItem], - ) -> NetworkDiagnosticsSummary: - return NetworkDiagnosticsSummary( - total_duration_ms=session.total_duration_ms if session else None, - total_steps=session.total_steps if session else 0, - total_tool_calls=len(tools), - search_calls=sum(1 for item in tools if item.tool_name == "web_search"), - url_read_calls=sum(1 for item in tools if item.tool_name == "url_read"), - success_count=sum(1 for item in tools if item.status == "success"), - failed_count=sum(1 for item in tools if item.status == "failed"), - degraded_count=sum(1 for item in tools if item.status == "degraded"), - interrupted_count=sum(1 for item in tools if item.status == "interrupted"), - limit_reason=session.limit_reason if session else None, - run_status=session.status if session else None, - ) - - def _tool_item_from_log( - self, - log: ToolCallLog, - *, - is_admin: bool, - ) -> NetworkDiagnosticsToolItem: - input_params = log.input_params or {} - output_data = log.output_data or {} - admin: dict[str, Any] | None = None - if is_admin: - admin = { - "trace_id": log.trace_id, - "step_number": log.step_number, - "input_params": self._sanitize_input_params(input_params), - "error_message": log.error_message, - "created_at": log.created_at.isoformat() if log.created_at else None, - } - - return NetworkDiagnosticsToolItem( - tool_call_log_id=log.id, - tool_name=log.tool_name, - status=self._normalize_status(log.status), - duration_ms=log.duration_ms, - target=self._derive_target(log.tool_name, input_params), - result_count=self._derive_result_count(log.tool_name, output_data), - reason=self._derive_reason(log.status, log.error_message), - started_at=log.created_at, - admin=admin, - ) - - def _normalize_status(self, status: str) -> str: - if status in ("success", "failed", "degraded", "interrupted"): - return status - return "failed" - - def _derive_target(self, tool_name: str, input_params: dict[str, Any]) -> str: - if tool_name == "web_search": - return str(input_params.get("query") or "").strip() - if tool_name == "url_read": - return str(input_params.get("url") or "").strip() - return tool_name - - def _derive_result_count(self, tool_name: str, output_data: dict[str, Any]) -> int | None: - if "result_count" in output_data and isinstance(output_data["result_count"], int): - return output_data["result_count"] - sources = output_data.get("sources") - if tool_name == "web_search" and isinstance(sources, list): - return len(sources) - return None - - def _derive_reason(self, status: str, error_message: str | None) -> str | None: - if error_message and error_message.strip(): - return error_message.strip() - if status == "degraded": - return "部分内容不可用，已降级处理" - if status == "failed": - return "未取得可用内容" - if status == "interrupted": - return "工具调用已中断" - return None - - def _sanitize_input_params(self, input_params: dict[str, Any]) -> dict[str, Any]: - allowed_keys = {"query", "url"} - return {key: value for key, value in input_params.items() if key in allowed_keys} -``` - -- [ ] **Step 5: 运行服务测试确认通过** - -Run: - -```bash -cd /Users/sean/code/fusion/fusion-api -.venv311/bin/python -m pytest test/test_network_diagnostics.py -q -``` - -Expected: service tests pass. - -- [ ] **Step 6: 提交** - -```bash -cd /Users/sean/code/fusion/fusion-api -git add app/schemas/network_diagnostics.py app/services/network_diagnostics_service.py test/test_network_diagnostics.py -git commit -m "feat: 聚合联网诊断数据" -m "Co-Authored-By: Codex " -``` - ---- - -## Task 3: 后端新增 diagnostics API - -**Files:** -- Modify: `fusion-api/app/api/deps.py` -- Modify: `fusion-api/app/api/chat.py` -- Test: `fusion-api/test/test_network_diagnostics_api.py` - -- [ ] **Step 1: 写 API 测试** - -Create `test/test_network_diagnostics_api.py` with focused endpoint tests. Use existing `TestClient` fixtures from nearby chat API tests; if no shared fixture fits, create a local minimal setup with dependency overrides. - -Core assertions: - -```python -def test_diagnostics_rejects_user_message(client, db, auth_user): - response = client.get("/api/chat/conversations/conv-1/messages/user-1/diagnostics") - assert response.status_code == 404 - - -def test_diagnostics_returns_empty_for_assistant_without_logs(client, db, auth_user): - response = client.get("/api/chat/conversations/conv-1/messages/assistant-1/diagnostics") - assert response.status_code == 200 - data = response.json()["data"] - assert data["is_empty"] is True - assert data["tools"] == [] - - -def test_diagnostics_admin_gets_admin_field(client, db, admin_user): - response = client.get("/api/chat/conversations/conv-1/messages/assistant-1/diagnostics") - assert response.status_code == 200 - tool = response.json()["data"]["tools"][0] - assert "admin" in tool - assert tool["admin"]["trace_id"] == "trace-1" -``` - -- [ ] **Step 2: 运行 API 测试确认失败** - -Run: - -```bash -cd /Users/sean/code/fusion/fusion-api -.venv311/bin/python -m pytest test/test_network_diagnostics_api.py -q -``` - -Expected: FAIL，原因是路由不存在。 - -- [ ] **Step 3: 增加依赖工厂** - -Modify `app/api/deps.py`: - -```python -from app.services.network_diagnostics_service import NetworkDiagnosticsService - - -def get_network_diagnostics_service(db: Session = Depends(get_db)) -> NetworkDiagnosticsService: - return NetworkDiagnosticsService(db) -``` - -- [ ] **Step 4: 增加路由** - -Modify `app/api/chat.py` imports: - -```python -from app.api.deps import get_chat_service, get_current_user, get_network_diagnostics_service -from app.services.network_diagnostics_service import NetworkDiagnosticsService -``` - -Add route before `update_message`: - -```python -@router.get("/conversations/{conversation_id}/messages/{message_id}/diagnostics") -def get_message_network_diagnostics( - conversation_id: str, - message_id: str, - request: Request, - chat_service: ChatService = Depends(get_chat_service), - diagnostics_service: NetworkDiagnosticsService = Depends(get_network_diagnostics_service), - current_user: User = Depends(get_current_user), -): - """获取单条 assistant 消息的联网诊断。""" - conversation = chat_service.get_conversation(conversation_id, user_id=current_user.id) - if not conversation: - raise ApiException.not_found("会话不存在或无权访问") - message = next((msg for msg in conversation.messages if msg.id == message_id), None) - if message is None or message.role != "assistant": - raise ApiException.not_found("消息不存在或无权访问") - - data = diagnostics_service.build_for_message( - conversation_id=conversation_id, - message_id=message_id, - is_admin=bool(getattr(current_user, "is_superuser", False)), - ) - return success(data=data, request_id=request.state.request_id) -``` - -- [ ] **Step 5: 运行 API + service 测试** - -Run: - -```bash -cd /Users/sean/code/fusion/fusion-api -.venv311/bin/python -m pytest test/test_network_diagnostics.py test/test_network_diagnostics_api.py -q -``` - -Expected: tests pass. - -- [ ] **Step 6: 运行后端相关回归** - -Run: - -```bash -cd /Users/sean/code/fusion/fusion-api -.venv311/bin/python -m pytest test/test_tool_handlers.py test/test_tool_executor.py test/test_network_diagnostics.py test/test_network_diagnostics_api.py -q -.venv/bin/ruff check app/services/tool_handlers/base.py app/services/network_diagnostics_service.py app/schemas/network_diagnostics.py app/api/chat.py app/api/deps.py test/test_network_diagnostics.py test/test_network_diagnostics_api.py -``` - -Expected: pytest and ruff pass. - -- [ ] **Step 7: 提交** - -```bash -cd /Users/sean/code/fusion/fusion-api -git add app/api/deps.py app/api/chat.py test/test_network_diagnostics_api.py -git commit -m "feat: 提供联网诊断接口" -m "Co-Authored-By: Codex " -``` - ---- - -## Task 4: 前端新增 diagnostics 类型、API client 和展示模型 - -**Files:** -- Create: `fusion-ui/src/types/networkDiagnostics.ts` -- Create: `fusion-ui/src/lib/api/chatDiagnostics.ts` -- Create: `fusion-ui/src/lib/api/chatDiagnostics.test.ts` -- Create: `fusion-ui/src/components/chat/networkDiagnosticsModel.ts` -- Create: `fusion-ui/src/components/chat/networkDiagnosticsModel.test.ts` - -- [ ] **Step 1: 新增类型** - -Create `src/types/networkDiagnostics.ts`: - -```ts -export type NetworkDiagnosticsStatus = 'success' | 'failed' | 'degraded' | 'interrupted'; - -export interface NetworkDiagnosticsSummary { - total_duration_ms: number | null; - total_steps: number; - total_tool_calls: number; - search_calls: number; - url_read_calls: number; - success_count: number; - failed_count: number; - degraded_count: number; - interrupted_count: number; - limit_reason?: string | null; - run_status?: string | null; -} - -export interface NetworkDiagnosticsToolItem { - tool_call_log_id: string; - tool_name: string; - status: NetworkDiagnosticsStatus; - duration_ms: number | null; - target: string; - result_count?: number | null; - reason?: string | null; - started_at?: string | null; - admin?: Record | null; -} - -export interface NetworkDiagnosticsResponse { - conversation_id: string; - message_id: string; - run_id: string | null; - visibility: 'user' | 'admin'; - summary: NetworkDiagnosticsSummary; - tools: NetworkDiagnosticsToolItem[]; - is_empty: boolean; -} -``` - -- [ ] **Step 2: 写 API client 测试** - -Create `src/lib/api/chatDiagnostics.test.ts`: - -```ts -import { beforeEach, describe, expect, it, vi } from 'vitest'; -import { getMessageNetworkDiagnostics } from './chatDiagnostics'; -import { apiRequest } from './fetchWithAuth'; - -vi.mock('./fetchWithAuth', () => ({ - apiRequest: vi.fn(), -})); - -describe('getMessageNetworkDiagnostics', () => { - beforeEach(() => { - vi.mocked(apiRequest).mockReset(); - }); - - it('请求单条消息 diagnostics 路径', async () => { - vi.mocked(apiRequest).mockResolvedValueOnce({ is_empty: true }); - - await getMessageNetworkDiagnostics('conv-1', 'msg-1'); - - expect(apiRequest).toHaveBeenCalledWith( - expect.stringContaining('/api/chat/conversations/conv-1/messages/msg-1/diagnostics'), - ); - }); -}); -``` - -- [ ] **Step 3: 实现 API client** - -Create `src/lib/api/chatDiagnostics.ts`: - -```ts -import { API_CONFIG } from '../config'; -import { apiRequest } from './fetchWithAuth'; -import type { NetworkDiagnosticsResponse } from '@/types/networkDiagnostics'; - -export function getMessageNetworkDiagnostics( - conversationId: string, - messageId: string, -): Promise { - const path = `/api/chat/conversations/${encodeURIComponent(conversationId)}/messages/${encodeURIComponent(messageId)}/diagnostics`; - return apiRequest(`${API_CONFIG.BASE_URL}${path}`); -} -``` - -- [ ] **Step 4: 写展示模型测试** - -Create `src/components/chat/networkDiagnosticsModel.test.ts`: - -```ts -import { describe, expect, it } from 'vitest'; -import { deriveNetworkDiagnosticsModel } from './networkDiagnosticsModel'; -import type { NetworkDiagnosticsResponse } from '@/types/networkDiagnostics'; - -const base: NetworkDiagnosticsResponse = { - conversation_id: 'conv-1', - message_id: 'msg-1', - run_id: 'run-1', - visibility: 'user', - is_empty: false, - summary: { - total_duration_ms: 4200, - total_steps: 2, - total_tool_calls: 3, - search_calls: 2, - url_read_calls: 1, - success_count: 2, - failed_count: 0, - degraded_count: 1, - interrupted_count: 0, - }, - tools: [ - { - tool_call_log_id: 'log-1', - tool_name: 'web_search', - status: 'success', - duration_ms: 1200, - target: 'G7 AI', - result_count: 5, - }, - { - tool_call_log_id: 'log-2', - tool_name: 'url_read', - status: 'degraded', - duration_ms: 3000, - target: 'https://example.com', - reason: 'reader-service 暂时未返回内容', - }, - ], -}; - -describe('deriveNetworkDiagnosticsModel', () => { - it('生成用户摘要和异常列表', () => { - const model = deriveNetworkDiagnosticsModel(base); - expect(model?.summaryText).toBe('联网诊断 · 搜索 2 次 · 读取 1 个网页 · 用时 4.2s'); - expect(model?.issueItems).toHaveLength(1); - expect(model?.issueItems[0].reason).toContain('reader-service'); - }); - - it('空 diagnostics 不渲染', () => { - expect(deriveNetworkDiagnosticsModel({ ...base, is_empty: true, tools: [] })).toBeNull(); - }); - - it('管理员可展开明细', () => { - const model = deriveNetworkDiagnosticsModel({ - ...base, - visibility: 'admin', - tools: [{ ...base.tools[0], admin: { trace_id: 'trace-1' } }], - }); - expect(model?.canShowAdminDetails).toBe(true); - }); -}); -``` - -- [ ] **Step 5: 实现展示模型** - -Create `src/components/chat/networkDiagnosticsModel.ts`: - -```ts -import type { - NetworkDiagnosticsResponse, - NetworkDiagnosticsToolItem, -} from '@/types/networkDiagnostics'; - -export interface NetworkDiagnosticsIssueItem { - id: string; - toolName: string; - title: string; - status: NetworkDiagnosticsToolItem['status']; - reason: string; -} - -export interface NetworkDiagnosticsModel { - summaryText: string; - issueItems: NetworkDiagnosticsIssueItem[]; - tools: NetworkDiagnosticsToolItem[]; - canShowAdminDetails: boolean; -} - -export function deriveNetworkDiagnosticsModel( - diagnostics: NetworkDiagnosticsResponse | null, -): NetworkDiagnosticsModel | null { - if (!diagnostics || diagnostics.is_empty || diagnostics.summary.total_tool_calls === 0) return null; - const issueItems = diagnostics.tools - .filter(item => item.status === 'failed' || item.status === 'degraded' || item.status === 'interrupted') - .map(item => ({ - id: item.tool_call_log_id, - toolName: item.tool_name, - title: item.target || getToolLabel(item.tool_name), - status: item.status, - reason: item.reason || getFallbackReason(item.status), - })); - - return { - summaryText: buildSummaryText(diagnostics), - issueItems, - tools: diagnostics.tools, - canShowAdminDetails: diagnostics.visibility === 'admin' - && diagnostics.tools.some(item => item.admin), - }; -} - -function buildSummaryText(diagnostics: NetworkDiagnosticsResponse): string { - const parts = ['联网诊断']; - if (diagnostics.summary.search_calls > 0) parts.push(`搜索 ${diagnostics.summary.search_calls} 次`); - if (diagnostics.summary.url_read_calls > 0) parts.push(`读取 ${diagnostics.summary.url_read_calls} 个网页`); - if (diagnostics.summary.total_duration_ms !== null) { - parts.push(`用时 ${formatDuration(diagnostics.summary.total_duration_ms)}`); - } - return parts.join(' · '); -} - -function formatDuration(ms: number): string { - if (ms < 1000) return `${ms}ms`; - return `${(ms / 1000).toFixed(1)}s`; -} - -function getFallbackReason(status: NetworkDiagnosticsToolItem['status']): string { - if (status === 'degraded') return '部分内容不可用，已降级处理'; - if (status === 'interrupted') return '工具调用已中断'; - return '未取得可用内容'; -} - -function getToolLabel(toolName: string): string { - if (toolName === 'web_search') return '搜索'; - if (toolName === 'url_read') return '网页读取'; - return toolName; -} -``` - -- [ ] **Step 6: 运行前端模型/API 测试** - -Run: - -```bash -cd /Users/sean/code/fusion/fusion-ui -npm test -- src/lib/api/chatDiagnostics.test.ts src/components/chat/networkDiagnosticsModel.test.ts -``` - -Expected: tests pass. - -- [ ] **Step 7: 提交** - -```bash -cd /Users/sean/code/fusion/fusion-ui -git add src/types/networkDiagnostics.ts src/lib/api/chatDiagnostics.ts src/lib/api/chatDiagnostics.test.ts src/components/chat/networkDiagnosticsModel.ts src/components/chat/networkDiagnosticsModel.test.ts -git commit -m "feat: 增加联网诊断前端模型" -m "Co-Authored-By: Codex " -``` - ---- - -## Task 5: 前端接入回答依据侧栏 - -**Files:** -- Create: `fusion-ui/src/components/chat/NetworkDiagnosticsPanel.tsx` -- Modify: `fusion-ui/src/components/chat/AnswerEvidenceSidebar.tsx` -- Modify: `fusion-ui/src/components/chat/AssistantMessage.tsx` -- Modify: `fusion-ui/src/components/chat/AnswerEvidenceSidebar.test.tsx` -- Modify: `fusion-ui/src/components/chat/AssistantMessage.test.tsx` - -- [ ] **Step 1: 新增 panel 组件** - -Create `src/components/chat/NetworkDiagnosticsPanel.tsx`: - -```tsx -'use client'; - -import { ChevronDown, ChevronRight, Clock, Wrench } from 'lucide-react'; -import { useState } from 'react'; -import { cn } from '@/lib/utils'; -import type { NetworkDiagnosticsModel } from './networkDiagnosticsModel'; - -interface NetworkDiagnosticsPanelProps { - model: NetworkDiagnosticsModel | null; - isLoading?: boolean; - error?: string | null; -} - -export default function NetworkDiagnosticsPanel({ - model, - isLoading = false, - error = null, -}: NetworkDiagnosticsPanelProps) { - const [expanded, setExpanded] = useState(false); - - if (isLoading) { - return
正在读取联网诊断...
; - } - if (error) { - return
联网诊断暂不可用
; - } - if (!model) return null; - - return ( -
-

-

-
-
-
- {model.issueItems.length > 0 ? ( -
- {model.issueItems.map(item => ( -
- - {item.status === 'failed' ? '失败' : item.status === 'degraded' ? '降级' : '中断'} - - · {item.title}：{item.reason} -
- ))} -
- ) : null} - {model.canShowAdminDetails ? ( - - ) : null} - {expanded ? ( -
- {model.tools.map(tool => ( -
- {tool.tool_name} · {tool.status} · {tool.duration_ms ?? '-'}ms · {tool.target || '-'} -
- ))} -
- ) : null} -
-
- ); -} -``` - -- [ ] **Step 2: 修改 AnswerEvidenceSidebar props** - -Modify `AnswerEvidenceSidebar.tsx`: - -```tsx -import NetworkDiagnosticsPanel from './NetworkDiagnosticsPanel'; -import type { NetworkDiagnosticsModel } from './networkDiagnosticsModel'; - -interface AnswerEvidenceSidebarProps { - model: AnswerEvidenceSidebarModel | null; - diagnostics?: NetworkDiagnosticsModel | null; - diagnosticsLoading?: boolean; - diagnosticsError?: string | null; - isOpen: boolean; - onClose: () => void; - highlightIndex?: number; - highlightTick?: number; -} -``` - -Render after issue section: - -```tsx - -``` - -- [ ] **Step 3: 在 AssistantMessage 懒加载 diagnostics** - -Modify `AssistantMessage.tsx` imports: - -```tsx -import { getMessageNetworkDiagnostics } from '@/lib/api/chatDiagnostics'; -import { deriveNetworkDiagnosticsModel } from './networkDiagnosticsModel'; -import type { NetworkDiagnosticsResponse } from '@/types/networkDiagnostics'; -``` - -Add state inside `AssistantMessageFrame`: - -```tsx -const [networkDiagnostics, setNetworkDiagnostics] = useState(null); -const [networkDiagnosticsLoading, setNetworkDiagnosticsLoading] = useState(false); -const [networkDiagnosticsError, setNetworkDiagnosticsError] = useState(null); -``` - -Add derived model: - -```tsx -const networkDiagnosticsModel = useMemo( - () => deriveNetworkDiagnosticsModel(networkDiagnostics), - [networkDiagnostics], -); -``` - -Add lazy load effect: - -```tsx -useEffect(() => { - if (!answerEvidenceSidebarOpen || !activeChatId || networkDiagnostics || networkDiagnosticsLoading) return; - let cancelled = false; - setNetworkDiagnosticsLoading(true); - setNetworkDiagnosticsError(null); - getMessageNetworkDiagnostics(activeChatId, message.id) - .then(data => { - if (!cancelled) setNetworkDiagnostics(data); - }) - .catch(error => { - if (!cancelled) setNetworkDiagnosticsError(error instanceof Error ? error.message : '联网诊断暂不可用'); - }) - .finally(() => { - if (!cancelled) setNetworkDiagnosticsLoading(false); - }); - return () => { - cancelled = true; - }; -}, [ - activeChatId, - answerEvidenceSidebarOpen, - message.id, - networkDiagnostics, - networkDiagnosticsLoading, -]); -``` - -Pass props to sidebar: - -```tsx - -``` - -- [ ] **Step 4: 更新组件测试** - -In `AnswerEvidenceSidebar.test.tsx`, add: - -```tsx -it('渲染联网诊断分区', () => { - render( - , - ); - - expect(screen.getByText('联网诊断')).toBeInTheDocument(); - expect(screen.getByText('联网诊断 · 搜索 1 次 · 用时 1.2s')).toBeInTheDocument(); -}); -``` - -In `AssistantMessage.test.tsx`, mock `getMessageNetworkDiagnostics` and assert it is called after opening sources. - -- [ ] **Step 5: 运行前端聊天测试** - -Run: - -```bash -cd /Users/sean/code/fusion/fusion-ui -npm test -- src/components/chat/networkDiagnosticsModel.test.ts src/lib/api/chatDiagnostics.test.ts src/components/chat/AnswerEvidenceSidebar.test.tsx src/components/chat/AssistantMessage.test.tsx src/components/chat/ChatMessage.test.tsx -``` - -Expected: tests pass. - -- [ ] **Step 6: 提交** - -```bash -cd /Users/sean/code/fusion/fusion-ui -git add src/components/chat/NetworkDiagnosticsPanel.tsx src/components/chat/AnswerEvidenceSidebar.tsx src/components/chat/AssistantMessage.tsx src/components/chat/AnswerEvidenceSidebar.test.tsx src/components/chat/AssistantMessage.test.tsx -git commit -m "feat: 接入联网诊断侧栏" -m "Co-Authored-By: Codex " -``` - ---- - -## Task 6: 全量验证、push 和 CI 跟进 - -**Files:** -- No source edits unless verification exposes a real issue. - -- [ ] **Step 1: 后端验证** - -Run: - -```bash -cd /Users/sean/code/fusion/fusion-api -.venv311/bin/python -m pytest test/test_tool_handlers.py test/test_tool_executor.py test/test_network_diagnostics.py test/test_network_diagnostics_api.py -q -.venv/bin/ruff check . -``` - -Expected: tests and ruff pass. - -- [ ] **Step 2: 前端验证** - -Run: - -```bash -cd /Users/sean/code/fusion/fusion-ui -npm test -- src/lib/api/chatDiagnostics.test.ts src/components/chat/networkDiagnosticsModel.test.ts src/components/chat/AnswerEvidenceSidebar.test.tsx src/components/chat/AssistantMessage.test.tsx src/components/chat/ChatMessage.test.tsx -npm run lint -``` - -Expected: tests and lint pass. - -- [ ] **Step 3: 推送两个仓库** - -Run: - -```bash -cd /Users/sean/code/fusion/fusion-api -git push origin master - -cd /Users/sean/code/fusion/fusion-ui -git push origin master -``` - -- [ ] **Step 4: 监听 CI** - -Run: - -```bash -gh run list --repo HyxiaoGe/fusion-api --branch master --limit 3 -gh run list --repo HyxiaoGe/fusion-ui --branch master --limit 3 -``` - -Follow the newest runs until success or a concrete failure is identified. - -- [ ] **Step 5: 完成审计** - -Verify these requirements against current evidence: - -- `ToolCallLog.message_id` is populated for new web tool calls. -- Diagnostics API returns empty model for non-network old messages. -- Diagnostics API returns user summary without raw params for normal users. -- Diagnostics API returns admin field for superuser. -- Answer evidence sidebar shows network diagnostics after lazy load. -- Existing realtime Agent timeline still renders from `agent_event`. -- No local Fusion dev server was started. - ---- - -## Plan Self-Review - -- Spec coverage: Task 1 covers message_id association. Tasks 2-3 cover backend schema/service/API, permissions, empty state, user/admin visibility. Tasks 4-5 cover frontend types, API client, model, lazy loading, side panel display. Task 6 covers validation, push, and CI. -- Placeholder scan: no unresolved placeholder wording; every task has concrete files, commands, and expected output. -- Type consistency: backend uses `NetworkDiagnosticsResponse`, `NetworkDiagnosticsSummary`, `NetworkDiagnosticsToolItem`; frontend mirrors `NetworkDiagnosticsResponse`, `NetworkDiagnosticsSummary`, `NetworkDiagnosticsToolItem` and derives `NetworkDiagnosticsModel`. diff --git a/docs/superpowers/plans/2026-06-27-agent-loop-contract-tests.md b/docs/superpowers/plans/2026-06-27-agent-loop-contract-tests.md deleted file mode 100644 index 7f2514b3..00000000 --- a/docs/superpowers/plans/2026-06-27-agent-loop-contract-tests.md +++ /dev/null @@ -1,238 +0,0 @@ -# Agent Loop Contract Tests Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** 补齐 agent-loop contract test suite，把事件顺序、DB partial persist、Redis Stream 终态、agent session/step 状态变成明确断言。 - -**Architecture:** 新增一个聚合型测试文件 `test/services/stream/test_agent_loop_contract.py`，复用现有 `StreamHandler.generate_to_redis()` 入口，以 fake LLM round、fake tool executor、fake Redis、captured session cache 写入构造真实编排契约。生产代码默认不改；只有 contract 测试暴露真实缺陷时才修实现。 - -**Tech Stack:** Python 3.11、unittest/pytest、fakeredis、FastAPI 后端现有 stream/agent 模块。 - ---- - -## File Structure - -- Create: `test/services/stream/test_agent_loop_contract.py` - - 提供 `AgentLoopContractTests`。 - - 覆盖跨模块契约：agent event sequence、partial persist order、Redis Stream done/error terminal、agent session/step terminal state。 -- Create: `test/services/stream/__init__.py` - - 让 `python -m unittest discover -s test -t .` 能递归发现 `test/services/stream` 下的 `unittest.TestCase` 测试。 -- Modify only if needed: `app/services/stream/*.py` - - 仅当新增契约测试发现真实回归时修改。 - -## Task 1: Contract Harness - -**Files:** -- Create: `test/services/stream/test_agent_loop_contract.py` - -- [x] **Step 1: Write a failing smoke contract harness** - -Create a test class that invokes `StreamHandler.generate_to_redis()` with: - -```python -class AgentLoopContractTests(unittest.IsolatedAsyncioTestCase): - async def test_contract_harness_records_events_persist_and_session_state(self): - result = await self._run_agent_contract( - rounds=[("", "Final answer", [], "stop", None)], - ) - - self.assertEqual(result.event_types, ["run_started", "step_started", "step_completed", "run_completed"]) - self.assertEqual(result.finalize_calls[-1], {"conversation_id": "conv-contract", "success": True, "task_id": "task-contract", "error_msg": ""}) -``` - -- [x] **Step 2: Run RED** - -Run: - -```bash -DATABASE_URL=sqlite:////tmp/fusion_api_agent_loop_contract.db /Users/sean/code/fusion/fusion-api/.venv311/bin/python -m pytest test/services/stream/test_agent_loop_contract.py -q -``` - -Expected: fail because the new helper/result type is not implemented. - -- [x] **Step 3: Implement test-only harness** - -Implement helpers inside the test file: - -```python -@dataclass -class AgentLoopContractResult: - events: list[dict] - event_types: list[str] - append_calls: list[dict] - persist_calls: list[dict] - finalize_calls: list[dict] - session_started_calls: list[dict] - session_status_calls: list[dict] - step_started_calls: list[dict] - step_completed_calls: list[dict] - step_terminal_calls: list[dict] -``` - -The harness must patch Redis writes, persistence, session cache writes, LLM rounds, tool execution, DB session, and message building without starting local services. - -- [x] **Step 4: Run GREEN** - -Run the same test. Expected: pass. - -## Task 2: Event And Partial Persist Contract - -**Files:** -- Modify: `test/services/stream/test_agent_loop_contract.py` - -- [x] **Step 1: Write failing tool round contract test** - -Add a test that runs one tool round followed by one stop round and asserts: - -```python -self.assertEqual( - result.event_types, - [ - "run_started", - "step_started", - "tool_call_started", - "tool_call_completed", - "step_completed", - "step_started", - "step_completed", - "run_completed", - ], -) -self.assertEqual([event["sequence"] for event in result.events], list(range(len(result.events)))) -self.assertEqual(result.persist_calls[0]["partial"], True) -self.assertEqual(result.persist_calls[0]["block_types"], ["thinking"]) -self.assertEqual(result.tool_execute_calls[0]["message_id"], "msg-contract") -self.assertEqual(result.persist_calls[-1]["partial"], False) -``` - -- [x] **Step 2: Run RED** - -Run the new test. Expected: fail until harness records tool execution arguments and persist block types. - -- [x] **Step 3: Extend harness minimally** - -Record `execute_tools_parallel` kwargs and persist content block types. - -- [x] **Step 4: Run GREEN** - -Run the contract file. Expected: pass. - -## Task 3: Redis Stream Terminal Contract - -**Files:** -- Modify: `test/services/stream/test_agent_loop_contract.py` - -- [x] **Step 1: Write failing Redis terminal tests** - -Add tests asserting: - -```python -self.assertEqual(result.redis_entry_types[0], "start") -self.assertIn("preparing", result.redis_entry_types) -self.assertEqual(result.redis_entry_types[-1], "done") -self.assertEqual(result.redis_meta["status"], "done") -``` - -And for failure: - -```python -with self.assertRaises(RuntimeError): - await self._run_agent_contract(rounds=RuntimeError("LLM 5xx"), use_real_redis_stream=True) -self.assertEqual(result.redis_entry_types[-1], "error") -self.assertEqual(result.redis_entries[-1][1]["content"], "LLM 5xx") -self.assertEqual(result.redis_meta["status"], "error") -``` - -- [x] **Step 2: Run RED** - -Run the targeted tests. Expected: fail until the harness supports `use_real_redis_stream`. - -- [x] **Step 3: Add real Redis capture and failure capture** - -Patch `stream_state_service.get_redis_pool` to a `fakeredis.aioredis.FakeRedis`, call `init_stream()` before `generate_to_redis()`, then read `stream:chunks:conv-contract` and `stream:meta:conv-contract` before propagating expected exceptions. - -- [x] **Step 4: Run GREEN** - -Run the contract file. Expected: pass. - -## Task 4: Session And Step State Contract - -**Files:** -- Modify: `test/services/stream/test_agent_loop_contract.py` - -- [x] **Step 1: Write failing session/step status tests** - -Add tests for: - -```python -self.assertEqual(result.session_started_calls[0]["message_id"], "msg-contract") -self.assertEqual(result.session_status_calls[-1]["status"], "completed") -self.assertEqual(result.session_status_calls[-1]["total_steps"], 2) -self.assertEqual(result.session_status_calls[-1]["total_tool_calls"], 1) -self.assertEqual([c["step_number"] for c in result.step_started_calls], [1, 2]) -self.assertEqual([c["tool_calls_count"] for c in result.step_completed_calls], [1, 0]) -``` - -Add failed path assertions: - -```python -self.assertEqual(result.step_terminal_calls[-1]["status"], "failed") -self.assertEqual(result.session_status_calls[-1]["status"], "error") -``` - -- [x] **Step 2: Run RED** - -Run targeted tests. Expected: fail until step completed/terminal data is captured fully. - -- [x] **Step 3: Extend harness minimally** - -Capture step completed and step terminal calls. - -- [x] **Step 4: Run GREEN** - -Run the contract file. Expected: pass. - -## Task 5: Final Verification - -**Files:** -- Test: `test/services/stream/test_agent_loop_contract.py` -- Test package marker: `test/services/stream/__init__.py` -- Test: `test/services/stream test/test_stream_handler.py test/test_tool_executor.py` - -- [x] **Step 1: Run contract suite** - -```bash -DATABASE_URL=sqlite:////tmp/fusion_api_agent_loop_contract.db /Users/sean/code/fusion/fusion-api/.venv311/bin/python -m pytest test/services/stream/test_agent_loop_contract.py -q -``` - -- [x] **Step 2: Run stream regression suite** - -```bash -DATABASE_URL=sqlite:////tmp/fusion_api_agent_loop_contract_regression.db /Users/sean/code/fusion/fusion-api/.venv311/bin/python -m pytest test/services/stream test/test_stream_handler.py test/test_tool_executor.py -q -``` - -- [x] **Step 3: Run lint/format checks for touched files** - -```bash -/opt/homebrew/bin/python3.11 -m ruff check test/services/stream/__init__.py test/services/stream/test_agent_loop_contract.py -/opt/homebrew/bin/python3.11 -m ruff format --check test/services/stream/__init__.py test/services/stream/test_agent_loop_contract.py -``` - -- [x] **Step 4: Run CI-equivalent unittest discover** - -```bash -DATABASE_URL=sqlite:////tmp/fusion_api_agent_loop_contract_full_unittest.db /Users/sean/code/fusion/fusion-api/.venv311/bin/python -u -m unittest discover -s test -t . -v -``` - -- [ ] **Step 5: Commit** - -```bash -git add docs/superpowers/plans/2026-06-27-agent-loop-contract-tests.md test/services/stream/test_agent_loop_contract.py -git commit -m "test: 补充 agent loop 契约测试" -``` - -Commit body: - -```text -Co-Authored-By: Codex -``` diff --git a/docs/superpowers/plans/2026-06-27-agent-loop-state.md b/docs/superpowers/plans/2026-06-27-agent-loop-state.md deleted file mode 100644 index 6f1a3c4c..00000000 --- a/docs/superpowers/plans/2026-06-27-agent-loop-state.md +++ /dev/null @@ -1,280 +0,0 @@ -# Agent Loop State Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** 把 `StreamHandler.generate_to_redis` 内散落的 agent loop 状态集中到纯状态对象 `AgentLoopState`，保持运行行为零变化。 - -**Architecture:** 新增 `app/services/stream/agent_loop_state.py`，只管理内存状态和小型状态转移，不触碰 DB、Redis、LLM、emitter 或 session_cache。`runner.py` 继续负责编排和副作用，但通过 `state` 读写 step、usage、content_blocks、terminal 标记和统计。现有 contract suite 负责守住事件顺序、partial persist、Redis 终态、session/step 状态。 - -**Tech Stack:** Python 3.11、unittest、FastAPI 后端现有 `app.schemas.chat.Usage` 和 stream 测试目录。 - ---- - -### Task 1: 新增 AgentLoopState 纯状态对象 - -**Files:** -- Create: `app/services/stream/agent_loop_state.py` -- Create: `test/services/stream/test_agent_loop_state.py` - -- [x] **Step 1: 写失败测试** - -新增 `test/services/stream/test_agent_loop_state.py`： - -```python -import unittest - -from app.schemas.chat import TextBlock, Usage -from app.services.stream.agent_loop_state import AgentLoopState - - -class AgentLoopStateTests(unittest.TestCase): - def test_initial_state_matches_runner_defaults(self): - state = AgentLoopState() - - self.assertEqual(state.content_blocks, []) - self.assertEqual(state.accumulated_usage, Usage(input_tokens=0, output_tokens=0)) - self.assertEqual(state.step, 0) - self.assertEqual(state.total_tool_calls, 0) - self.assertEqual(state.current_step_id, None) - self.assertEqual(state.finish_reason, "stop") - self.assertEqual(state.limit_reason, None) - self.assertEqual(state.unknown_terminated, False) - self.assertEqual(state.terminal_emitted, False) - - def test_step_and_tool_call_mutations_are_explicit(self): - state = AgentLoopState() - - self.assertEqual(state.next_step_number(), 1) - state.mark_current_step("step-1") - state.record_executed_tool_calls(2) - state.clear_current_step() - - self.assertEqual(state.step, 1) - self.assertEqual(state.total_tool_calls, 2) - self.assertEqual(state.current_step_id, None) - self.assertEqual(state.run_stats("run-1").total_steps, 1) - self.assertEqual(state.run_stats("run-1").total_tool_calls, 2) - - def test_usage_content_and_terminal_mutations_are_explicit(self): - state = AgentLoopState() - block = TextBlock(type="text", id="blk-1", text="answer") - - state.content_blocks.append(block) - state.update_usage(Usage(input_tokens=3, output_tokens=5)) - state.mark_unknown_terminated() - state.mark_terminal_emitted() - - self.assertEqual(state.content_blocks, [block]) - self.assertEqual(state.final_usage(), Usage(input_tokens=3, output_tokens=5)) - self.assertEqual(state.unknown_terminated, True) - self.assertEqual(state.terminal_emitted, True) - - def test_final_usage_omits_zero_input_usage(self): - state = AgentLoopState() - - self.assertIsNone(state.final_usage()) - - -if __name__ == "__main__": - unittest.main() -``` - -- [x] **Step 2: 跑测试确认红灯** - -Run: - -```bash -DATABASE_URL=sqlite:////tmp/fusion_api_agent_loop_state_red.db /Users/sean/code/fusion/fusion-api/.venv311/bin/python -m unittest test.services.stream.test_agent_loop_state -v -``` - -Expected: FAIL，原因是 `app.services.stream.agent_loop_state` 还不存在。 - -- [x] **Step 3: 写最小实现** - -新增 `app/services/stream/agent_loop_state.py`： - -```python -"""Agent loop 内存状态与纯状态转移。""" - -from __future__ import annotations - -from dataclasses import dataclass, field - -from app.schemas.chat import Usage -from app.services.stream.agent_loop_policy import AgentLoopLimitReason -from app.services.stream.run_finalizer import AgentRunStats - - -@dataclass -class AgentLoopState: - content_blocks: list = field(default_factory=list) - accumulated_usage: Usage = field(default_factory=lambda: Usage(input_tokens=0, output_tokens=0)) - step: int = 0 - total_tool_calls: int = 0 - current_step_id: str | None = None - finish_reason: str = "stop" - limit_reason: AgentLoopLimitReason | None = None - unknown_terminated: bool = False - terminal_emitted: bool = False - - def next_step_number(self) -> int: - self.step += 1 - return self.step - - def mark_current_step(self, step_id: str) -> None: - self.current_step_id = step_id - - def clear_current_step(self) -> None: - self.current_step_id = None - - def record_executed_tool_calls(self, tool_call_count: int) -> None: - self.total_tool_calls += tool_call_count - - def update_usage(self, usage: Usage) -> None: - self.accumulated_usage = usage - - def final_usage(self) -> Usage | None: - if self.accumulated_usage.input_tokens <= 0: - return None - return self.accumulated_usage - - def mark_unknown_terminated(self) -> None: - self.unknown_terminated = True - - def mark_terminal_emitted(self) -> None: - self.terminal_emitted = True - - def run_stats(self, run_id: str) -> AgentRunStats: - return AgentRunStats( - run_id=run_id, - total_steps=self.step, - total_tool_calls=self.total_tool_calls, - ) -``` - -- [x] **Step 4: 跑测试确认绿灯** - -Run: - -```bash -DATABASE_URL=sqlite:////tmp/fusion_api_agent_loop_state_green.db /Users/sean/code/fusion/fusion-api/.venv311/bin/python -m unittest test.services.stream.test_agent_loop_state -v -``` - -Expected: PASS。 - -### Task 2: 将 runner 接入 AgentLoopState - -**Files:** -- Modify: `app/services/stream/runner.py` -- Test: `test/services/stream/test_agent_loop_contract.py` -- Test: `test/test_stream_handler.py` - -- [x] **Step 1: 替换散落状态变量** - -在 `runner.py` 中导入 `AgentLoopState`，用 `state = AgentLoopState()` 替代以下局部变量： - -```python -content_blocks = [] -accumulated_usage = Usage(input_tokens=0, output_tokens=0) -step = 0 -total_tool_calls = 0 -finish_reason = "stop" -current_step_id = None -terminal_emitted = False -limit_reason = None -unknown_terminated = False -``` - -关键替换规则： - -```python -content_blocks -> state.content_blocks -accumulated_usage -> state.accumulated_usage -step -> state.step -total_tool_calls -> state.total_tool_calls -finish_reason -> state.finish_reason -current_step_id -> state.current_step_id -terminal_emitted -> state.terminal_emitted -limit_reason -> state.limit_reason -unknown_terminated -> state.unknown_terminated -_run_stats() -> state.run_stats(run_id) -_record_executed_tool_calls -> state.record_executed_tool_calls -``` - -每次开启 step 使用： - -```python -step_number = state.next_step_number() -step_context = await start_agent_step(..., step_number=step_number, on_step_started=state.mark_current_step) -state.mark_current_step(step_context.step_id) -``` - -每个正常闭合 step 后调用： - -```python -state.clear_current_step() -``` - -每个终态成功写出后调用： - -```python -state.mark_terminal_emitted() -``` - -- [x] **Step 2: 跑 focused 契约测试** - -Run: - -```bash -DATABASE_URL=sqlite:////tmp/fusion_api_agent_loop_state_contract.db /Users/sean/code/fusion/fusion-api/.venv311/bin/python -m pytest test/services/stream/test_agent_loop_contract.py test/test_stream_handler.py -q -``` - -Expected: 全部 PASS，尤其是事件顺序、partial persist、Redis terminal、session/step 状态不变。 - -### Task 3: 全量验证、提交和 CI/CD - -**Files:** -- Modify: `docs/superpowers/plans/2026-06-27-agent-loop-state.md` - -- [x] **Step 1: 跑完整后端测试和 lint** - -Run: - -```bash -DATABASE_URL=sqlite:////tmp/fusion_api_agent_loop_state_full.db /Users/sean/code/fusion/fusion-api/.venv311/bin/python -u -m unittest discover -s test -t . -v -/opt/homebrew/bin/python3.11 -m ruff check . -git diff --check -``` - -Expected: unittest 341+ tests OK，ruff 无错误，`git diff --check` 无输出。 - -- [ ] **Step 2: 提交并推送** - -Run: - -```bash -git add -f docs/superpowers/plans/2026-06-27-agent-loop-state.md -git add app/services/stream/agent_loop_state.py app/services/stream/runner.py test/services/stream/test_agent_loop_state.py -git commit -m "refactor: 抽出 agent loop 状态对象" -m "Co-Authored-By: Codex " -git push -u origin refactor/agent-loop-state -``` - -- [ ] **Step 3: 监控 GitHub Actions** - -Run: - -```bash -gh run list --repo HyxiaoGe/fusion-api --branch refactor/agent-loop-state --limit 3 -gh run watch --repo HyxiaoGe/fusion-api --interval 10 --exit-status -``` - -Expected: 分支 CI success。若用户继续要求部署，则按正常 CI/CD 合入 master 并监控 dev 部署。 - ---- - -## Self-Review - -- 覆盖目标：计划只抽 `AgentLoopState`，不引入 `AgentLoopRuntime`，保持行为零变化。 -- TDD：先新增 `test_agent_loop_state.py` 并确认红灯，再新增生产代码。 -- 风险控制：contract suite 和 `test_stream_handler.py` 作为行为保护网。 -- 本地服务：不启动 uvicorn、Next、Docker 或本地 Fusion 服务。 diff --git a/docs/superpowers/plans/2026-06-27-agent-run-lifecycle.md b/docs/superpowers/plans/2026-06-27-agent-run-lifecycle.md deleted file mode 100644 index a2712c2e..00000000 --- a/docs/superpowers/plans/2026-06-27-agent-run-lifecycle.md +++ /dev/null @@ -1,19 +0,0 @@ -# Agent run 生命周期边界拆分计划 - -## 背景 - -`runner.py` 的终态收尾已经交给 `run_finalizer.py`，但启动阶段仍分散在 runner 里：先直接写 `agent_sessions` started 行，后面再直接 emit `run_started`。这让 run 生命周期的顺序约束仍由 runner 隐式维护。 - -## 目标 - -1. 在 lifecycle 边界中新增 `start_agent_run`，明确 started session 与 `run_started` event 的顺序。 -2. 让 runner 只调用生命周期函数，不再直接组合 session start 和 run start event。 -3. 保持现有 agent loop 行为、终态路径和 Redis Stream 事件契约不变。 - -## 步骤 - -1. 先补 `start_agent_run` 单元测试，断言先写 session started，再 emit `run_started`。 -2. 实现 `start_agent_run`，扩展现有 emitter/session_cache Protocol。 -3. 在 runner 中接入 `start_agent_run`，移除分散的 started 写入和 `run_started` emit。 -4. 跑聚焦 agent loop/stream/lifecycle 测试、全量 unittest、ruff、架构检查与 diff 检查。 -5. 提交、推送并通过正常 CI/CD 合入 dev。 diff --git a/docs/superpowers/plans/2026-06-28-agent-loop-driver-runtime.md b/docs/superpowers/plans/2026-06-28-agent-loop-driver-runtime.md deleted file mode 100644 index 97ad78b7..00000000 --- a/docs/superpowers/plans/2026-06-28-agent-loop-driver-runtime.md +++ /dev/null @@ -1,187 +0,0 @@ -# Agent Loop Driver Runtime Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** 把 `runner.py` 里的 agent loop `while True` 状态机和触顶总结迁到独立 driver/runtime，保留现有 Redis Stream、事件、落库和终态行为。 - -**Architecture:** `runner.py` 继续负责外围准备、消息构建、异常/finally、最终落库与 run 终态。新增 `AgentLoopRuntime` 收拢循环运行时依赖，新增 `run_agent_loop()` 只执行每轮 LLM、工具回合、superseded 识别和触顶总结，并返回 outcome 供 `runner.py` 决定是否继续最终收尾。 - -**Tech Stack:** FastAPI 服务层 Python 3.11、pytest/unittest、Redis Stream agent event helpers。 - ---- - -### Task 1: Driver 契约测试 - -**Files:** -- Create: `test/services/stream/test_agent_loop_driver.py` - -- [ ] **Step 1: Write the failing tests** - -```python -import unittest -from dataclasses import dataclass - -from app.schemas.chat import Usage -from app.services.stream.agent_loop_driver import AgentLoopExit, run_agent_loop -from app.services.stream.agent_loop_policy import AgentLoopLimits -from app.services.stream.agent_loop_runtime import AgentLoopRuntime -from app.services.stream.agent_loop_state import AgentLoopState -from app.services.stream.agent_round import AgentRoundResult - - -@dataclass -class DummyStepContext: - step_id: str - thinking_block_id: str - text_block_id: str - - -class DummyEmitter: - def __init__(self): - self.limit_reasons = [] - - async def run_limit_reached(self, *, reason): - self.limit_reasons.append(reason) - - -class AgentLoopDriverTests(unittest.IsolatedAsyncioTestCase): - async def test_stop_round_completes_text_step_and_returns_completed(self): - state = AgentLoopState() - started = [] - completed = [] - rounds = [] - - async def start_step_fn(**kwargs): - started.append(kwargs["step_number"]) - return DummyStepContext("step-1", "thinking-1", "text-1") - - async def complete_step_fn(**kwargs): - completed.append(kwargs["context"].step_id) - - async def run_round_fn(**kwargs): - rounds.append(kwargs["step_number"]) - return AgentRoundResult( - reasoning_buf="思考", - content_buf="回答", - tool_calls=[], - finish_reason="stop", - accumulated_usage=Usage(input_tokens=3, output_tokens=5), - ) - - runtime = AgentLoopRuntime( - conversation_id="conv", - task_id="task", - run_id="run", - user_id="user", - model_id="model", - provider="provider", - litellm_model="litellm", - litellm_kwargs={}, - should_use_reasoning=True, - call_kwargs={}, - assistant_message_id="assistant", - run_start=0, - limits=AgentLoopLimits(max_steps=8, max_tool_calls=20, total_timeout_s=300), - emitter=DummyEmitter(), - session_cache=object(), - network_budget=object(), - start_step_fn=start_step_fn, - complete_step_fn=complete_step_fn, - run_round_fn=run_round_fn, - handle_tool_calls_round_fn=None, - run_limit_summary_step_fn=None, - llm_call_fn=None, - stream_round_fn=None, - execute_tools_fn=None, - persist_message_fn=None, - log_round_summary_fn=lambda **kwargs: None, - warning_fn=lambda message: None, - clock=lambda: 1, - ) - - outcome = await run_agent_loop(db=object(), messages=[], state=state, runtime=runtime) - - self.assertEqual(outcome.exit, AgentLoopExit.COMPLETED) - self.assertEqual(started, [1]) - self.assertEqual(rounds, [1]) - self.assertEqual(completed, ["step-1"]) - self.assertEqual(state.current_step_id, None) - self.assertEqual(state.accumulated_usage.input_tokens, 3) - self.assertEqual(len(state.content_blocks), 2) -``` - -- [ ] **Step 2: Run test to verify it fails** - -Run: - -```bash -DATABASE_URL=sqlite:////tmp/fusion_api_agent_loop_driver_red.db /Users/sean/code/fusion/fusion-api/.venv311/bin/python -m pytest test/services/stream/test_agent_loop_driver.py -q -``` - -Expected: FAIL because `app.services.stream.agent_loop_driver` does not exist. - -### Task 2: Driver/Runtime 抽取 - -**Files:** -- Create: `app/services/stream/agent_loop_runtime.py` -- Create: `app/services/stream/agent_loop_driver.py` -- Modify: `app/services/stream/runner.py` -- Modify: `test/services/stream/test_agent_loop_driver.py` - -- [ ] **Step 1: Implement runtime/outcome types** - -Create `AgentLoopRuntime` as a dataclass that carries ids/config/dependencies currently used only by the loop. Create `AgentLoopExit` with `COMPLETED` and `SUPERSEDED`, plus `AgentLoopOutcome(exit, error_msg=None)`. - -- [ ] **Step 2: Move the loop body into `run_agent_loop()`** - -Move these existing behaviors out of `runner.py` without semantic changes: -- limit check order: timeout > max_steps > max_tool_calls via `check_agent_loop_limit` -- normal stop: append reasoning/text blocks, complete text step, clear current step -- cancelled: append blocks, keep current step id, return `SUPERSEDED` for `runner.py` to persist/interrupted/finalize once -- tool calls: delegate to `handle_tool_calls_round_fn`, record tool count through state callback, clear current step, continue -- unknown finish reason: append blocks, mark unknown terminated, complete text step, clear current step -- limit summary: start independent summary step through `run_limit_summary_step_fn`, update usage, clear current step - -- [ ] **Step 3: Replace `runner.py` loop with runtime construction and driver call** - -`runner.py` should construct `AgentLoopRuntime`, call `run_agent_loop()`, and return early only for `SUPERSEDED`. Final persist, `map_run_terminal_state()`, `complete_agent_run()` and `finalize_stream(success=True)` remain in `runner.py`. - -- [ ] **Step 4: Expand driver tests** - -Add focused tests for: -- immediate max-step limit calls summary and emits `run_limit_reached` -- cancelled round returns `SUPERSEDED` without terminal side effects, plus runner-level test covers persist/interrupted/finalize - -### Task 3: Verification and CI/CD - -**Files:** -- Modify as required by implementation only. - -- [ ] **Step 1: Run focused tests** - -```bash -DATABASE_URL=sqlite:////tmp/fusion_api_agent_loop_driver_focus.db /Users/sean/code/fusion/fusion-api/.venv311/bin/python -m pytest test/services/stream/test_agent_loop_driver.py test/services/stream/test_agent_loop_contract.py test/test_stream_handler.py -q -``` - -- [ ] **Step 2: Run architecture and lint checks** - -```bash -/Users/sean/code/fusion/fusion-api/.venv311/bin/python scripts/check_architecture.py -/opt/homebrew/bin/python3.11 -m ruff check . -/opt/homebrew/bin/python3.11 -m ruff format --check app/services/stream/agent_loop_runtime.py app/services/stream/agent_loop_driver.py app/services/stream/runner.py test/services/stream/test_agent_loop_driver.py -git diff --check -``` - -- [ ] **Step 3: Commit and push** - -Commit message: - -```text -refactor: 抽出 agent loop driver runtime - -Co-Authored-By: Codex -``` - -- [ ] **Step 4: Follow CI/CD** - -Push through normal Git flow, monitor GitHub Actions and deployed dev image. Do not start local Fusion services. diff --git a/docs/superpowers/plans/2026-06-28-agent-loop-driver-steps.md b/docs/superpowers/plans/2026-06-28-agent-loop-driver-steps.md deleted file mode 100644 index 3f7766aa..00000000 --- a/docs/superpowers/plans/2026-06-28-agent-loop-driver-steps.md +++ /dev/null @@ -1,39 +0,0 @@ -# 2026-06-28 Agent Loop Driver Steps 拆分计划 - -## 背景 - -`runner.py` 已抽出 runtime/state/request prep 后，`app/services/stream/agent_loop_driver.py` 仍由 `run_agent_loop()` 同时承载： - -- 循环顶部 limit 检查与 `run_limit_reached` 事件 -- step lifecycle 启动 -- 单轮 LLM 调用 -- stop / cancelled / tool_calls / unknown 退化分支 -- 触顶后的 summary step 调用 - -这些职责都属于 driver 层，但堆在一个 164 行函数中，后续改触顶策略或工具回合时容易误伤其他分支。 - -## 目标 - -把 `run_agent_loop()` 收敛为主状态机骨架，具体分支交给小函数处理。保持外部协议不变： - -- `run_agent_loop()` 签名与 `AgentLoopOutcome` 不变 -- `runner.py` 不参与本轮拆分 -- Redis event、session cache、message append、usage 统计、tool call 计数语义不变 - -## 实施步骤 - -1. 在 `test/services/stream/test_agent_loop_driver.py` 补齐 timeout、tool-call、unknown 退化分支单测。 -2. 抽出 driver 内部 helper： - - limit 检查与触顶事件 - - step 启动 - - round 调用与 state 更新 - - stop / cancelled / tool_calls / unknown 分支处理 - - limit summary 调用 -3. 跑聚焦测试、架构检查、ruff、全量 unittest。 -4. 推送后监控 GitHub Actions 和 dev health。 - -## 非目标 - -- 不调整 loop limit 策略。 -- 不调整 tool executor、limit summary、llm stream 的内部实现。 -- 不启动本地 Fusion 服务。 diff --git a/docs/superpowers/plans/2026-06-28-agent-loop-final-batch.md b/docs/superpowers/plans/2026-06-28-agent-loop-final-batch.md deleted file mode 100644 index dd8cd345..00000000 --- a/docs/superpowers/plans/2026-06-28-agent-loop-final-batch.md +++ /dev/null @@ -1,167 +0,0 @@ -# Agent Loop Final Batch Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** 一次性收尾 agent loop 剩余结构优化：抽出 round outcome 分发、集中 ToolRoundRequest/LimitSummaryStepRequest 构造、收薄 execution/wiring builder。 - -**Architecture:** 新增 `agent_loop_outcome.py` 保存 outcome 类型，新增 `agent_loop_round_outcome.py` 承担 stop/cancelled/tool_calls/unknown 分发，新增 `agent_loop_step_requests.py` 集中构造 tool round 与 limit summary 请求。`agent_loop_driver.py` 只保留循环、limit 检查、step 启动和 round 调用；`agent_loop_wiring.py` / `agent_loop_execution.py` 把字段映射下沉到数据对象/小 helper。 - -**Tech Stack:** Python 3.11、pytest、dataclasses、ruff、Fusion agent loop。 - ---- - -## File Structure - -- Create: `app/services/stream/agent_loop_outcome.py` - - 保存 `AgentLoopExit` 和 `AgentLoopOutcome`，避免 outcome 分发模块反向依赖 driver。 -- Create: `app/services/stream/agent_loop_round_outcome.py` - - 保存 `AgentRoundOutcomeRequest` 与 `handle_agent_round_outcome()`，封装 stop/cancelled/tool_calls/unknown 分发。 -- Create: `app/services/stream/agent_loop_step_requests.py` - - 保存 `build_tool_round_request()` 与 `build_limit_summary_step_request()`。 -- Modify: `app/services/stream/agent_loop_driver.py` - - 使用新模块，移除 round 分支副作用和 request 构造。 -- Modify: `app/services/stream/agent_loop_lifecycle.py` - - 从 `agent_loop_outcome.py` 导入 `AgentLoopExit`。 -- Modify: `app/services/stream/agent_loop_execution.py` - - 增加 `AgentLoopExecutionParts` / `build_agent_loop_runtime()`，让 `build_agent_loop_execution()` 只装配主要部件。 -- Modify: `app/services/stream/agent_loop_wiring.py` - - 给 `AgentLoopRunInput` / `AgentLoopWiringDependencies` 增加转换方法，收薄 `build_agent_loop_lifecycle_call()`。 -- Test: `test/services/stream/test_agent_loop_step_requests.py` -- Test: `test/services/stream/test_agent_loop_round_outcome.py` -- Modify tests: `test/services/stream/test_agent_loop_execution.py`, `test/services/stream/test_agent_loop_wiring.py` - -## Task 1: Request Builders - -- [ ] **Step 1: Write failing tests** - -Add `test/services/stream/test_agent_loop_step_requests.py` with tests importing: - -```python -from app.services.stream.agent_loop_step_requests import ( - build_limit_summary_step_request, - build_tool_round_request, -) -``` - -Assert `build_tool_round_request()` copies db/messages/state/runtime/step/round fields into `ToolRoundRequest`, and `build_limit_summary_step_request()` copies state/runtime/messages into `LimitSummaryStepRequest`. - -- [ ] **Step 2: Run test to verify it fails** - -Run: - -```bash -/Users/sean/code/fusion/fusion-api/.venv311/bin/python -m pytest test/services/stream/test_agent_loop_step_requests.py -q -``` - -Expected: FAIL because `agent_loop_step_requests` does not exist. - -- [ ] **Step 3: Implement request builders** - -Create `app/services/stream/agent_loop_step_requests.py` with: - -```python -def build_tool_round_request(...): ... -def build_limit_summary_step_request(...): ... -``` - -- [ ] **Step 4: Verify green** - -Run the same test; expected PASS. - -## Task 2: Round Outcome Dispatcher - -- [ ] **Step 1: Write failing tests** - -Add `test/services/stream/test_agent_loop_round_outcome.py` with tests importing: - -```python -from app.services.stream.agent_loop_round_outcome import AgentRoundOutcomeRequest, handle_agent_round_outcome -``` - -Cover stop, cancelled, tool_calls, and unknown fallback behavior. - -- [ ] **Step 2: Run test to verify it fails** - -Run: - -```bash -/Users/sean/code/fusion/fusion-api/.venv311/bin/python -m pytest test/services/stream/test_agent_loop_round_outcome.py -q -``` - -Expected: FAIL because the module does not exist. - -- [ ] **Step 3: Implement dispatcher and outcome module** - -Create `agent_loop_outcome.py`; move outcome types there and re-export them from driver. Create `agent_loop_round_outcome.py` and update `agent_loop_driver.py` to call it. - -- [ ] **Step 4: Verify green** - -Run round outcome and driver tests; expected PASS. - -## Task 3: Builder Thin-Out - -- [ ] **Step 1: Write failing tests** - -Extend execution/wiring tests to import and assert: - -```python -from app.services.stream.agent_loop_execution import AgentLoopExecutionParts, build_agent_loop_runtime -``` - -and call: - -```python -run_input.to_execution_request(...) -run_input.to_lifecycle_request(...) -dependencies.to_execution_dependencies() -dependencies.to_lifecycle_dependencies() -``` - -Expected: methods/classes missing. - -- [ ] **Step 2: Implement builder helpers** - -Add `AgentLoopExecutionParts`, `build_agent_loop_runtime()`, `AgentLoopRunInput.to_*()` and `AgentLoopWiringDependencies.to_*()`. - -- [ ] **Step 3: Verify builder tests** - -Run: - -```bash -/Users/sean/code/fusion/fusion-api/.venv311/bin/python -m pytest test/services/stream/test_agent_loop_execution.py test/services/stream/test_agent_loop_wiring.py -q -``` - -Expected: PASS. - -## Task 4: Final Verification and Single Delivery - -- [ ] Run focused tests: - -```bash -/Users/sean/code/fusion/fusion-api/.venv311/bin/python -m pytest test/services/stream/test_agent_loop_step_requests.py test/services/stream/test_agent_loop_round_outcome.py test/services/stream/test_agent_loop_driver.py test/services/stream/test_agent_loop_execution.py test/services/stream/test_agent_loop_wiring.py test/services/stream/test_agent_loop_contract.py -q -``` - -- [ ] Run full stream tests: - -```bash -/Users/sean/code/fusion/fusion-api/.venv311/bin/python -m pytest test/services/stream -q -``` - -- [ ] Run full repo tests and checks: - -```bash -/Users/sean/code/fusion/fusion-api/.venv311/bin/python -m pytest -q -/opt/homebrew/bin/ruff check . -/opt/homebrew/bin/ruff format --check app/services/stream test/services/stream/test_agent_loop_step_requests.py test/services/stream/test_agent_loop_round_outcome.py test/services/stream/test_agent_loop_execution.py test/services/stream/test_agent_loop_wiring.py -/opt/homebrew/bin/python3.11 scripts/check_architecture.py -/opt/homebrew/bin/python3.11 scripts/check_quality.py -git diff --check -``` - -- [ ] Single commit: - -```bash -git commit -m "refactor: 收拢 agent loop 最终批次" -m "Co-Authored-By: Codex " -``` - -- [ ] Merge to master, push once, watch one CI/CD run, verify dev image/health, run one Chrome regression on `https://fusion.seanfield.org` using the existing logged-in Chrome tab. diff --git a/docs/superpowers/plans/2026-06-28-agent-loop-lifecycle.md b/docs/superpowers/plans/2026-06-28-agent-loop-lifecycle.md deleted file mode 100644 index 45464e5a..00000000 --- a/docs/superpowers/plans/2026-06-28-agent-loop-lifecycle.md +++ /dev/null @@ -1,63 +0,0 @@ -# Agent Loop Lifecycle Facade Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** 把 `StreamHandler.generate_to_redis()` 中的 run lifecycle 编排抽到独立 facade，runner 只保留输入配置、DB session 生命周期和依赖注入入口。 - -**Architecture:** 新增 `app/services/stream/agent_loop_lifecycle.py`，定义 `AgentLoopLifecycleRequest`、`AgentLoopLifecycleDependencies` 和 `run_agent_loop_lifecycle()`。该 facade 负责 append preparing、start run、prepare messages、run driver、terminal finalize、cancel/fail/fallback 处理。`runner.py` 继续把当前模块变量传入依赖对象，保持既有 `app.services.stream.runner.*` patch 路径有效。 - -**Tech Stack:** Python 3.11、pytest、dataclasses、Fusion agent loop。 - ---- - -### Task 1: 抽取 agent loop lifecycle facade - -**Files:** -- Create: `app/services/stream/agent_loop_lifecycle.py` -- Create: `test/services/stream/test_agent_loop_lifecycle.py` -- Modify: `app/services/stream/runner.py` - -- [x] **Step 1: 写先失败的 lifecycle 测试** - -新增测试覆盖： -- completed 路径顺序：`append preparing` → `start_agent_run` → `prepare_agent_loop_messages` → `run_agent_loop` → `finalize_completed_run`。 -- prepared initial content blocks 被追加到 `execution.state.content_blocks` 后再进入 driver。 -- superseded 路径调用 `finalize_superseded_run` 且不调用 completed finalize。 -- exception wrapper 保持 CancelledError/Exception finalize 后 re-raise，并执行 fallback。 - -Run: -```bash -DATABASE_URL=sqlite:////tmp/fusion_api_agent_loop_lifecycle_red.db /Users/sean/code/fusion/fusion-api/.venv311/bin/python -m pytest test/services/stream/test_agent_loop_lifecycle.py -q -``` -Expected: FAIL，原因是 `app.services.stream.agent_loop_lifecycle` 尚不存在。 - -- [x] **Step 2: 最小实现 `agent_loop_lifecycle.py`** - -实现： -- `AgentLoopLifecycleRequest`：保存 raw messages、vision/file/original message、call config、limits。 -- `AgentLoopLifecycleDependencies`：保存 runner 传入的 append/start/prepare/run/finalize/fallback/logger 依赖。 -- `run_agent_loop_lifecycle(...)`：包住 try/except/finally，调用内部 `_run_success_path(...)`、`_finalize_cancelled(...)`、`_finalize_failed(...)`、`_write_fallback(...)`。 -- 内部 completed/superseded finalize 继续复用 `agent_loop_run_completion.py` 里的函数，terminal state 继续用 `map_run_terminal_state(...)`。 - -- [x] **Step 3: 改 runner 使用 lifecycle facade** - -在 `generate_to_redis()` 中： -- 保留 `call_config`、`db = SessionLocal()`、`limits`、`execution = build_agent_loop_execution(...)`。 -- 用 `run_agent_loop_lifecycle(...)` 替代原 try/except/finally 中除 `db.close()` 以外的 lifecycle 逻辑。 -- `runner.py` 继续导入并传入 `append_chunk`、`finalize_stream`、`start_agent_run`、`prepare_agent_loop_messages`、`run_agent_loop`、`persist_message` 等当前模块变量，保证旧 patch 路径不变。 - -- [x] **Step 4: 验证保持行为** - -Run: -```bash -DATABASE_URL=sqlite:////tmp/fusion_api_agent_loop_lifecycle_green.db /Users/sean/code/fusion/fusion-api/.venv311/bin/python -m pytest test/services/stream/test_agent_loop_lifecycle.py test/services/stream/test_agent_loop_execution.py test/services/stream/test_agent_loop_run_completion.py test/services/stream/test_agent_loop_driver.py test/test_stream_handler.py -q -/opt/homebrew/bin/python3.11 -m ruff check app/services/stream/agent_loop_lifecycle.py app/services/stream/runner.py test/services/stream/test_agent_loop_lifecycle.py -/opt/homebrew/bin/python3.11 -m ruff format --check app/services/stream/agent_loop_lifecycle.py app/services/stream/runner.py test/services/stream/test_agent_loop_lifecycle.py -/opt/homebrew/bin/python3.11 scripts/check_quality.py -``` - -验收标准： -- `generate_to_redis()` 行数继续下降，runner 不再直接编排 start/prepare/run/finalize/except/fallback。 -- 新增 lifecycle 模块不产生新的超长函数热点。 -- `test_stream_handler.py` 和契约测试继续通过。 -- 不启动本地 Fusion 服务。 diff --git a/docs/superpowers/plans/2026-06-28-agent-loop-limit-summary-helpers.md b/docs/superpowers/plans/2026-06-28-agent-loop-limit-summary-helpers.md deleted file mode 100644 index 0b9fad5a..00000000 --- a/docs/superpowers/plans/2026-06-28-agent-loop-limit-summary-helpers.md +++ /dev/null @@ -1,142 +0,0 @@ -# Agent Loop Limit Summary Helpers Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** 拆分触顶强制总结 step 的内部职责，降低 `run_limit_summary_step()` 的修改风险。 - -**Architecture:** 用 `LimitSummaryStepRequest` 收敛 driver 到 summary 的内部调用边界。新增模块内 helper 负责 prompt append、summary LLM/stream 调用、timeout fallback、usage 累加、content block append 和 step complete，runner 不参与本轮改动。 - -**Tech Stack:** Python 3.11, FastAPI service modules, unittest/pytest, ruff. - ---- - -### Task 1: 补齐 helper 行为测试 - -**Files:** -- Modify: `test/services/stream/test_limit_summary.py` - -- [ ] **Step 1: 新增 helper 导入** - -```python -from app.services.stream.limit_summary import ( - append_summary_content_blocks, - accumulate_summary_usage, -) -``` - -- [ ] **Step 2: 新增 usage 累加测试** - -```python -def test_accumulate_summary_usage_adds_usage_data(self): - result = accumulate_summary_usage( - Usage(input_tokens=2, output_tokens=3), - Usage(input_tokens=5, output_tokens=7), - ) - - self.assertEqual(result, Usage(input_tokens=7, output_tokens=10)) -``` - -- [ ] **Step 3: 新增 content block append 测试** - -```python -def test_append_summary_content_blocks_adds_reasoning_and_text(self): - content_blocks = [] - - append_summary_content_blocks( - content_blocks=content_blocks, - reasoning_buf="推理", - content_buf="总结正文", - thinking_block_id="blk-thinking", - text_block_id="blk-text", - ) - - self.assertEqual([block.type for block in content_blocks], ["thinking", "text"]) -``` - -- [ ] **Step 4: 运行 RED** - -Run: - -```bash -DATABASE_URL=sqlite:////tmp/fusion_api_limit_summary_red.db .venv311/bin/python -m pytest test/services/stream/test_limit_summary.py -q -``` - -Expected: import error for missing helper functions. - -### Task 2: 拆分 `limit_summary.py` - -**Files:** -- Modify: `app/services/stream/limit_summary.py` -- Modify: `app/services/stream/agent_loop_driver.py` - -- [ ] **Step 1: 新增 helper** - -新增： - -```python -@dataclass(frozen=True) -class LimitSummaryStepRequest: ... -def append_limit_summary_prompt(messages: list[dict]) -> None: ... -async def call_limit_summary_round(...) -> tuple[str, str, Usage | None]: ... -def accumulate_summary_usage(accumulated_usage: Usage, usage_data: Usage | None) -> Usage: ... -def append_summary_content_blocks(...) -> None: ... -async def complete_limit_summary_step(...) -> None: ... -``` - -- [ ] **Step 2: 收敛 `run_limit_summary_step()`** - -`run_limit_summary_step(request=...)` 只保留： - -1. start step -2. append prompt -3. compute timeout -4. call summary round with timeout fallback -5. accumulate usage -6. append blocks -7. complete step -8. return outcome - -- [ ] **Step 3: 保持外部行为不变** - -保留： - -- `build_limit_summary_call_kwargs()` 复制并移除 `tools/tool_choice` -- timeout warning 文案 -- timeout 后不写 log summary、不 append block -- `complete_step_fn(... tool_names=[], tool_call_count=0)` -- driver 只负责把 `AgentLoopRuntime` / `AgentLoopState` 转成 `LimitSummaryStepRequest` - -### Task 3: 验证与发布 - -**Files:** -- No extra code files. - -- [ ] **Step 1: 聚焦测试** - -```bash -DATABASE_URL=sqlite:////tmp/fusion_api_limit_summary_focus.db .venv311/bin/python -m pytest test/services/stream/test_limit_summary.py test/services/stream/test_agent_loop_driver.py test/test_stream_handler.py -q -``` - -- [ ] **Step 2: 质量与格式** - -```bash -.venv311/bin/python scripts/check_architecture.py -/opt/homebrew/bin/python3.11 -m ruff check . -/opt/homebrew/bin/python3.11 -m ruff format --check app/services/stream/limit_summary.py test/services/stream/test_limit_summary.py -``` - -- [ ] **Step 3: 全量 unittest** - -```bash -DATABASE_URL=sqlite:////tmp/fusion_api_limit_summary_full.db .venv311/bin/python -m unittest discover -s test -t . -``` - -- [ ] **Step 4: 提交、推送、监控 CI/CD** - -Commit: - -```bash -git commit -m "refactor: 拆分 agent loop 触顶总结" -``` - -Push to `master` and watch GitHub Actions plus dev health. diff --git a/docs/superpowers/plans/2026-06-28-agent-loop-llm-stream.md b/docs/superpowers/plans/2026-06-28-agent-loop-llm-stream.md deleted file mode 100644 index f1c2b238..00000000 --- a/docs/superpowers/plans/2026-06-28-agent-loop-llm-stream.md +++ /dev/null @@ -1,68 +0,0 @@ -# Agent Loop LLM Stream 拆分 Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** 拆分 `llm_stream.stream_round()` 的流式协议解析、chunk 写入、usage 收集、tool_call 累积和锁检查职责，保持现有 public API 与 agent-loop 行为不变。 - -**Architecture:** `stream_round(...)` 保持外部签名不变，内部构造 `LLMStreamRequest` 并调用 `consume_stream_round()`。`consume_stream_round()` 只负责编排；具体逻辑拆到 usage 提取、tool_call 累积、reasoning/content delta 提取、Redis chunk append、锁检查和 outcome 构造 helper。 - -**Tech Stack:** FastAPI 后端内部 Python 3.11、pytest、ruff、Redis Stream SSE chunk 协议、LiteLLM streaming response。 - ---- - -### Task 1: 建立 LLM streaming request/helper 边界 - -**Files:** -- Create: `test/services/stream/test_llm_stream.py` -- Modify: `app/services/stream/llm_stream.py` - -- [x] **Step 1: 写先失败的 helper 接口测试** - -新增测试通过 `LLMStreamRequest` 调用 `consume_stream_round(response, request)`，断言： -- reasoning delta 写入 `reasoning` chunk，并透传 `run_id` / `step_id`。 -- content delta 写入 `answering` chunk。 -- usage-only chunk 仍更新 `Usage`。 -- 分片 tool calls 按 index 累积，并按 index 排序返回。 -- `finish_reason="tool_calls"` 不写空 content chunk。 - -- [x] **Step 2: 运行测试确认失败** - -Run: -```bash -DATABASE_URL=sqlite:////tmp/fusion_api_llm_stream_red.db /Users/sean/code/fusion/fusion-api/.venv311/bin/python -m pytest test/services/stream/test_llm_stream.py -q -``` -Expected: FAIL，因为 `LLMStreamRequest` 或 `consume_stream_round` 尚未导出。 - -- [x] **Step 3: 最小实现 request/outcome 和 helper 拆分** - -在 `llm_stream.py` 增加： -- `LLMStreamRequest` -- `LLMStreamState` -- `LLMStreamOutcome` -- `consume_stream_round()` -- `extract_usage()` -- `accumulate_tool_calls()` -- `extract_reasoning_delta()` -- `extract_content_delta()` -- `append_stream_delta()` -- `maybe_check_lock_owner()` -- `build_tool_calls_list()` - -`stream_round(...)` 只组装 request 并返回 outcome tuple。 - -- [x] **Step 4: 验证保持行为** - -Run: -```bash -DATABASE_URL=sqlite:////tmp/fusion_api_llm_stream_green.db /Users/sean/code/fusion/fusion-api/.venv311/bin/python -m pytest test/services/stream/test_llm_stream.py test/services/stream/test_agent_round.py test/services/stream/test_limit_summary.py test/services/stream/test_agent_loop_contract.py test/test_stream_handler.py test/test_stream_state_service.py -q -/opt/homebrew/bin/python3.11 scripts/check_quality.py -/opt/homebrew/bin/python3.11 scripts/check_architecture.py -/opt/homebrew/bin/python3.11 -m ruff check app/services/stream/llm_stream.py test/services/stream/test_llm_stream.py -/opt/homebrew/bin/python3.11 -m ruff format --check app/services/stream/llm_stream.py test/services/stream/test_llm_stream.py -``` - -验收标准： -- `app/services/stream/llm_stream.py:stream_round()` 不再出现在质量扫描的超长函数列表。 -- 现有 `stream_round(...)` public 签名不变。 -- usage、reasoning/content chunk、tool_calls、lock cancellation 语义不变。 -- 不启动本地 Fusion 服务。 diff --git a/docs/superpowers/plans/2026-06-28-agent-loop-request-prep.md b/docs/superpowers/plans/2026-06-28-agent-loop-request-prep.md deleted file mode 100644 index 5c11249e..00000000 --- a/docs/superpowers/plans/2026-06-28-agent-loop-request-prep.md +++ /dev/null @@ -1,140 +0,0 @@ -# Agent Loop Request Prep Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** 把 agent loop 进入 driver 前的 call config 与 message prep 从 `runner.py` 拆到独立模块，保留现有 run_started、URL 预处理、文件注入和工具契约行为。 - -**Architecture:** `runner.py` 继续负责 DB 生命周期、preparing、run_started、driver 调用、终态落库和异常/finally 收尾。新增 `agent_loop_request_prep.py` 提供两个阶段：`build_agent_loop_call_config()` 在 `start_agent_run()` 前同步构造 reasoning/tool call 配置并捕获 announced tools；`prepare_agent_loop_messages()` 在 `start_agent_run()` 后执行 DB/user prompt、文件注入、URL 预处理和工具契约注入。 - -**Tech Stack:** FastAPI 服务层 Python 3.11、pytest/unittest、现有 stream contract tests。 - ---- - -### Task 1: Request Prep 契约测试 - -**Files:** -- Create: `test/services/stream/test_agent_loop_request_prep.py` - -- [ ] **Step 1: Write failing tests** - -Add tests that import the intended API before it exists: - -```python -from app.services.stream.agent_loop_request_prep import ( - AgentLoopCallConfig, - build_agent_loop_call_config, - prepare_agent_loop_messages, -) -``` - -Cover: -- `functionCalling=True` adds `web_search`, `tool_choice="auto"`, captures `announced_tools=["web_search"]`. -- `provider="volcengine"` plus reasoning adds disabled-thinking `extra_body`. -- `use_reasoning=False` overrides `capabilities.deepThinking=True`. -- message prep injects non-image file content, inserts URL context before final user message, prepends the web-search contract after existing system messages, and returns initial content blocks for URL read. - -- [ ] **Step 2: Run RED** - -```bash -DATABASE_URL=sqlite:////tmp/fusion_api_request_prep_red.db /Users/sean/code/fusion/fusion-api/.venv311/bin/python -m pytest test/services/stream/test_agent_loop_request_prep.py -q -``` - -Expected: FAIL because `app.services.stream.agent_loop_request_prep` does not exist. - -### Task 2: Extract Request Prep - -**Files:** -- Create: `app/services/stream/agent_loop_request_prep.py` -- Modify: `app/services/stream/runner.py` -- Modify: `test/test_stream_handler.py` -- Modify: `test/services/stream/test_agent_loop_contract.py` - -- [ ] **Step 1: Implement call config** - -Create: - -```python -@dataclass(frozen=True) -class AgentLoopCallConfig: - should_use_reasoning: bool - supports_function_calling: bool - call_kwargs: dict - announced_tools: list[str] -``` - -`build_agent_loop_call_config()` must preserve existing behavior: -- default `should_use_reasoning` follows `capabilities["deepThinking"]` -- explicit `options["use_reasoning"]` overrides the default -- function calling adds `build_web_search_tool()` and `tool_choice="auto"` -- volcengine + reasoning applies `merge_extra_body(..., {"thinking": {"type": "disabled"}})` -- `announced_tools` is captured before later URL preprocessing mutates `call_kwargs` - -- [ ] **Step 2: Implement message prep** - -Create: - -```python -@dataclass(frozen=True) -class AgentLoopPreparedMessages: - messages: list[dict] - initial_content_blocks: list -``` - -`prepare_agent_loop_messages()` must preserve existing behavior: -- create `FileRepository(db)` -- load `User.system_prompt` -- call `build_llm_messages(...)` -- inject non-image file contents through `inject_file_content(...)` -- run `preprocess_url_in_message(original_message, supports_function_calling, call_kwargs)` -- insert URL context with `messages.insert(-1, url_context_msg)` -- append successful URL block to `initial_content_blocks` -- inject the web-search consistency contract only when `web_search` is present and not already injected - -- [ ] **Step 3: Wire runner** - -`runner.py` should: -- call `build_agent_loop_call_config()` before `start_agent_run()` -- pass `call_config.announced_tools` to `start_agent_run()` -- call `prepare_agent_loop_messages()` after `start_agent_run()` -- extend `state.content_blocks` with `prepared_messages.initial_content_blocks` -- pass `call_config.should_use_reasoning` and `call_config.call_kwargs` into `AgentLoopRuntime` - -Do not move DB session lifecycle, `append_chunk("preparing")`, `start_agent_run()`, final persist, run finalizer, `finalize_stream()`, exception paths, or `finally`. - -### Task 3: Verification and CI/CD - -**Files:** -- Modify as required by implementation only. - -- [ ] **Step 1: Run focused tests** - -```bash -DATABASE_URL=sqlite:////tmp/fusion_api_request_prep_focus.db /Users/sean/code/fusion/fusion-api/.venv311/bin/python -m pytest test/services/stream/test_agent_loop_request_prep.py test/services/stream/test_agent_loop_contract.py test/test_stream_handler.py -q -``` - -- [ ] **Step 2: Run architecture and lint checks** - -```bash -/Users/sean/code/fusion/fusion-api/.venv311/bin/python scripts/check_architecture.py -/opt/homebrew/bin/python3.11 -m ruff check . -/opt/homebrew/bin/python3.11 -m ruff format --check app/services/stream/agent_loop_request_prep.py app/services/stream/runner.py test/services/stream/test_agent_loop_request_prep.py test/services/stream/test_agent_loop_contract.py test/test_stream_handler.py -git diff --check -``` - -- [ ] **Step 3: Run full tests** - -```bash -DATABASE_URL=sqlite:////tmp/fusion_api_request_prep_full.db /Users/sean/code/fusion/fusion-api/.venv311/bin/python -m unittest discover -s test -t . -``` - -- [ ] **Step 4: Commit and push** - -Commit message: - -```text -refactor: 抽出 agent loop 请求准备 - -Co-Authored-By: Codex -``` - -Push `master` and follow CI/CD to dev. Do not start local Fusion services. diff --git a/docs/superpowers/plans/2026-06-28-agent-loop-run-completion.md b/docs/superpowers/plans/2026-06-28-agent-loop-run-completion.md deleted file mode 100644 index 84dc33dd..00000000 --- a/docs/superpowers/plans/2026-06-28-agent-loop-run-completion.md +++ /dev/null @@ -1,37 +0,0 @@ -# 2026-06-28 Agent Loop Run Completion 拆分计划 - -## 背景 - -`runner.generate_to_redis()` 在 request prep、runtime/state、driver 拆出后，仍直接处理五类终态收尾： - -- completed -- superseded -- user_cancelled -- failed -- finally fallback - -这些路径都混合了 assistant message 落库、agent run 终态事件、session cache、SSE finalize 和兜底写状态。它们不属于 loop 状态机本身，但对协议正确性很关键。 - -## 目标 - -把终态收尾移动到 `app/services/stream/agent_loop_run_completion.py`： - -- runner 保留 try/except/finally 的外层编排 -- completion helper 负责各终态路径的落库、run finalizer、SSE finalize 顺序 -- 通过依赖注入传入 `persist_message`、`finalize_stream` 和 run finalizer，保留现有测试 patch 路径 - -## 测试点 - -新增 `test/services/stream/test_agent_loop_run_completion.py` 覆盖： - -1. completed：persist -> complete run -> finalize success -2. superseded：persist -> interrupt -> finalize error -3. cancelled：有内容才 persist，interrupt 失败要吞掉并继续 finalize -4. failed：有内容才 persist，fail event 失败要吞掉并继续 finalize -5. fallback：已发终态时不重复写 fallback - -## 非目标 - -- 不改变 `run_finalizer.py` 的事件/session 写入顺序。 -- 不改变 `run_agent_loop()` 或工具执行行为。 -- 不启动本地 Fusion 服务。 diff --git a/docs/superpowers/plans/2026-06-28-agent-loop-runtime-context.md b/docs/superpowers/plans/2026-06-28-agent-loop-runtime-context.md deleted file mode 100644 index b4d712b4..00000000 --- a/docs/superpowers/plans/2026-06-28-agent-loop-runtime-context.md +++ /dev/null @@ -1,62 +0,0 @@ -# Agent Loop Runtime Context Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** 把 `StreamHandler.generate_to_redis()` 中的 `AgentLoopState`、`AgentLoopRuntime`、`AgentLoopRunCompletionContext` 装配逻辑移动到独立模块，降低 runner 的状态/依赖构建负担。 - -**Architecture:** 新增 `app/services/stream/agent_loop_execution.py`，定义 `AgentLoopDependencies` 和 `AgentLoopExecutionContext`，集中创建 state、network budget、emitter、runtime、completion context。`runner.py` 保持对现有可 patch 依赖的导入，并把这些依赖显式传给 builder，避免破坏既有测试 patch 路径。 - -**Tech Stack:** Python 3.11、pytest、dataclasses、Fusion agent loop。 - ---- - -### Task 1: 抽取 Agent Loop runtime/completion 装配 - -**Files:** -- Create: `app/services/stream/agent_loop_execution.py` -- Create: `test/services/stream/test_agent_loop_execution.py` -- Modify: `app/services/stream/runner.py` - -- [x] **Step 1: 写先失败的 execution context 测试** - -新增测试覆盖： -- `build_agent_loop_execution(...)` 使用传入 `trace_id` 作为 `run_id`。 -- `runtime` 与 `completion_context` 共享同一个 `state`、`emitter`、`session_cache`。 -- `runtime.limits`、`runtime.call_kwargs`、`runtime.should_use_reasoning` 来自传入配置。 -- `duration_ms_factory()` 使用同一个 `run_start` 和 clock 计算运行耗时。 - -Run: -```bash -DATABASE_URL=sqlite:////tmp/fusion_api_agent_loop_execution_red.db /Users/sean/code/fusion/fusion-api/.venv311/bin/python -m pytest test/services/stream/test_agent_loop_execution.py -q -``` -Expected: FAIL，原因是 `app.services.stream.agent_loop_execution` 尚不存在。 - -- [x] **Step 2: 最小实现 `agent_loop_execution.py`** - -实现： -- `AgentLoopDependencies`：保存 runner 传入的 session cache、step/run/tool/LLM/persist/log/warning/clock 依赖。 -- `AgentLoopExecutionContext`：保存 `state`、`network_budget`、`emitter`、`runtime`、`completion_context`。 -- `build_agent_loop_execution(...)`：创建 state、network budget、emitter、duration factory、completion context 和 runtime。 - -- [x] **Step 3: 改 runner 使用 execution builder** - -在 `generate_to_redis()` 中： -- 保留 `call_config` 和 `db = SessionLocal()`。 -- 用 `build_agent_loop_execution(...)` 创建 `execution`。 -- `start_agent_run`、`prepare_agent_loop_messages`、`run_agent_loop`、finalize 分支都使用 `execution.state`、`execution.runtime`、`execution.completion_context`、`execution.emitter`。 -- 不改变 run_started tools 时序，不改变异常重抛和 finally fallback 行为。 - -- [x] **Step 4: 验证保持行为** - -Run: -```bash -DATABASE_URL=sqlite:////tmp/fusion_api_agent_loop_execution_green.db /Users/sean/code/fusion/fusion-api/.venv311/bin/python -m pytest test/services/stream/test_agent_loop_execution.py test/services/stream/test_agent_loop_driver.py test/services/stream/test_agent_loop_run_completion.py test/test_stream_handler.py -q -/opt/homebrew/bin/python3.11 -m ruff check app/services/stream/agent_loop_execution.py app/services/stream/runner.py test/services/stream/test_agent_loop_execution.py -/opt/homebrew/bin/python3.11 -m ruff format --check app/services/stream/agent_loop_execution.py app/services/stream/runner.py test/services/stream/test_agent_loop_execution.py -/opt/homebrew/bin/python3.11 scripts/check_quality.py -``` - -验收标准： -- `generate_to_redis()` 行数下降，runner 不再直接构造 `AgentLoopRuntime` 和 `AgentLoopRunCompletionContext`。 -- 现有 `test_stream_handler.py` 行为测试继续通过。 -- 不启动本地 Fusion 服务。 diff --git a/docs/superpowers/plans/2026-06-28-agent-loop-tool-executor.md b/docs/superpowers/plans/2026-06-28-agent-loop-tool-executor.md deleted file mode 100644 index 294aa8f1..00000000 --- a/docs/superpowers/plans/2026-06-28-agent-loop-tool-executor.md +++ /dev/null @@ -1,62 +0,0 @@ -# Agent Loop 工具执行拆分 Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** 拆分 `tool_executor` 中的并发调度、单工具执行、预算处理、事件发送、日志写入和 record 构造，保持现有工具执行行为不变。 - -**Architecture:** `execute_tools_parallel(...)` 保持现有外部签名，内部只负责组装 `ToolExecutionBatchRequest` 并委托 `execute_tool_batch()`。单个工具调用由 `execute_one_tool_call()` 编排，具体职责拆成参数解析、预算准备、预算降级事件、handler 执行、日志写入和 `ToolExecutionRecord` 构造 helper。 - -**Tech Stack:** FastAPI 后端内部 Python 3.11、pytest、ruff、现有 agent-loop Redis Stream 工具事件协议。 - ---- - -### Task 1: 建立工具执行 request/helper 边界 - -**Files:** -- Modify: `test/test_tool_executor.py` -- Modify: `app/services/stream/tool_executor.py` - -- [ ] **Step 1: 写先失败的 helper 接口测试** - -新增测试使用 `ToolExecutionBatchRequest` 调用 `execute_tool_batch(request, [tool_call])`，断言成功路径仍返回 `ToolExecutionRecord`，并且 `handler.log` 收到 `message_id`、`trace_id`、`step_number` 与输入参数。 - -- [ ] **Step 2: 运行测试确认失败** - -Run: -```bash -DATABASE_URL=sqlite:////tmp/fusion_api_tool_executor_red.db /Users/sean/code/fusion/fusion-api/.venv311/bin/python -m pytest test/test_tool_executor.py -q -``` -Expected: FAIL，因为 `ToolExecutionBatchRequest` 或 `execute_tool_batch` 尚未导出。 - -- [ ] **Step 3: 最小实现 request 和 batch helper** - -在 `tool_executor.py` 增加： -- `ToolExecutionBatchRequest` -- `ToolExecutionIds` -- `execute_tool_batch()` -- `execute_one_tool_call()` -- `resolve_tool_handler()` -- `parse_tool_arguments()` -- `prepare_tool_arguments()` -- `emit_budget_result()` -- `log_tool_execution()` -- `build_tool_execution_record()` - -`execute_tools_parallel(...)` 只组装 request 并调用 `execute_tool_batch()`。 - -- [ ] **Step 4: 验证保持行为** - -Run: -```bash -DATABASE_URL=sqlite:////tmp/fusion_api_tool_executor_green.db /Users/sean/code/fusion/fusion-api/.venv311/bin/python -m pytest test/test_tool_executor.py test/services/stream/test_tool_call_lifecycle.py test/services/stream/test_agent_loop_contract.py test/test_stream_handler.py -q -/opt/homebrew/bin/python3.11 scripts/check_quality.py -/opt/homebrew/bin/python3.11 scripts/check_architecture.py -/opt/homebrew/bin/python3.11 -m ruff check app/services/stream/tool_executor.py test/test_tool_executor.py -/opt/homebrew/bin/python3.11 -m ruff format --check app/services/stream/tool_executor.py test/test_tool_executor.py -``` - -验收标准： -- `app/services/stream/tool_executor.py:execute_tools_parallel()` 和原嵌套 `_run_one()` 不再出现在质量扫描的超长函数列表。 -- 工具执行相关测试保持通过。 -- 真实外部签名不变，`runner -> tool_round -> execute_tools_parallel` 调用链不需要调整。 -- 不启动本地 Fusion 服务。 diff --git a/docs/superpowers/plans/2026-06-28-agent-loop-tool-lifecycle.md b/docs/superpowers/plans/2026-06-28-agent-loop-tool-lifecycle.md deleted file mode 100644 index 9a8ba121..00000000 --- a/docs/superpowers/plans/2026-06-28-agent-loop-tool-lifecycle.md +++ /dev/null @@ -1,62 +0,0 @@ -# Agent Loop Tool Lifecycle 拆分 Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** 拆分 `execute_tool_with_lifecycle()` 的工具执行、耗时测量、失败结果构造和 completed 事件发送职责，保持现有工具事件协议不变。 - -**Architecture:** `execute_tool_with_lifecycle(...)` 保持外部入口不变，内部只负责无 emitter 快路径、started 事件和调用 helper。新增 `ToolLifecycleAttempt` 承载单次执行结果，`run_tool_attempt()` 封装计时/异常映射，`complete_tool_lifecycle()` 统一发送 completed 事件并补齐成功结果 duration。 - -**Tech Stack:** Python 3.11、pytest、ruff、Fusion agent_event 工具调用协议。 - ---- - -### Task 1: 拆分 tool_call lifecycle 执行 attempt 和 completed 收尾 - -**Files:** -- Modify: `app/services/stream/tool_call_lifecycle.py` -- Modify: `test/services/stream/test_tool_call_lifecycle.py` - -- [x] **Step 1: 写先失败的 helper 边界测试** - -在 `test/services/stream/test_tool_call_lifecycle.py` 增加： -- `run_tool_attempt(target, args, execute)` 成功时返回 `ToolLifecycleAttempt(result, duration_ms, cancelled_error=None)`，并把测量耗时写入 `duration_ms`。 -- `run_tool_attempt(...)` 遇到普通异常时返回 `failed` ToolResult，不抛出。 -- `complete_tool_lifecycle(...)` 成功时补齐 `result.duration_ms` 并只发送一次 completed 事件。 - -- [x] **Step 2: 运行测试确认失败** - -Run: -```bash -DATABASE_URL=sqlite:////tmp/fusion_api_tool_lifecycle_red.db /Users/sean/code/fusion/fusion-api/.venv311/bin/python -m pytest test/services/stream/test_tool_call_lifecycle.py -q -``` -Expected: FAIL，原因是 `ToolLifecycleAttempt` / `run_tool_attempt` / `complete_tool_lifecycle` 尚未导出。 - -- [x] **Step 3: 最小实现 helper 拆分** - -在 `tool_call_lifecycle.py` 增加： -- `@dataclass(frozen=True) ToolLifecycleAttempt` -- `measure_duration_ms(start_mono: float) -> int` -- `run_tool_attempt(...)` -- `complete_tool_lifecycle(...)` - -保持 `execute_tool_with_lifecycle(...)` 行为： -- `emitter is None` 时仍只执行 tool，不构造 summary，不写 duration。 -- 有 emitter 时先发 started。 -- 成功结果若 `duration_ms is None` 则补测量值。 -- 普通异常返回 failed ToolResult 并发送 completed。 -- `asyncio.CancelledError` 发送 failed completed 后继续抛出。 - -- [x] **Step 4: 验证保持行为** - -Run: -```bash -DATABASE_URL=sqlite:////tmp/fusion_api_tool_lifecycle_green.db /Users/sean/code/fusion/fusion-api/.venv311/bin/python -m pytest test/services/stream/test_tool_call_lifecycle.py test/test_tool_executor.py test/services/stream/test_tool_round.py test/test_stream_handler.py -q -/opt/homebrew/bin/python3.11 -m ruff check app/services/stream/tool_call_lifecycle.py test/services/stream/test_tool_call_lifecycle.py -/opt/homebrew/bin/python3.11 -m ruff format --check app/services/stream/tool_call_lifecycle.py test/services/stream/test_tool_call_lifecycle.py -/opt/homebrew/bin/python3.11 scripts/check_quality.py -``` - -验收标准： -- `execute_tool_with_lifecycle()` 不再出现在质量扫描超长函数列表。 -- tool_call started/completed 事件字段、顺序和 error 映射不变。 -- 相关工具执行和 agent-loop 回归测试通过。 diff --git a/docs/superpowers/plans/2026-06-28-agent-loop-tool-round-helpers.md b/docs/superpowers/plans/2026-06-28-agent-loop-tool-round-helpers.md deleted file mode 100644 index e15f4f1d..00000000 --- a/docs/superpowers/plans/2026-06-28-agent-loop-tool-round-helpers.md +++ /dev/null @@ -1,66 +0,0 @@ -# Agent Loop 工具回合拆分 Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** 把 `tool_round` 从大函数编排拆成 request 对象和小职责 helper，保持工具回合事件顺序、持久化顺序、LLM 上下文注入和 reasoning 恢复语义不变。 - -**Architecture:** `handle_tool_calls_round()` 保留为工具回合入口，但改为接收 `ToolRoundRequest`。具体职责拆到同模块 helper：追加 thinking、预持久化、执行工具、注入 assistant/tool 消息、完成 step、恢复 reasoning 参数。`agent_loop_driver` 只负责把 runtime/state/round_result 组装成 request，不直接展开工具回合所有参数。 - -**Tech Stack:** FastAPI 后端内部 Python 3.11、pytest、ruff、现有 Redis Stream agent-loop 模块。 - ---- - -### Task 1: 锁住 request 接口和工具结果注入顺序 - -**Files:** -- Modify: `test/services/stream/test_tool_round.py` -- Modify: `app/services/stream/tool_round.py` -- Modify: `app/services/stream/agent_loop_driver.py` - -- [ ] **Step 1: 写先失败的接口测试** - -新增测试使用 `ToolRoundRequest` 调用 `handle_tool_calls_round(request=request)`，断言： -- 第一次 persist 发生在 execute 前，且只有 thinking block。 -- execute 后先记录工具调用数，再注入 assistant/tool 消息，再第二次 partial persist。 -- `complete_step_fn` 收到执行结果里的 tool_names 和 `len(results)`。 -- outcome 仍返回 `len(tool_calls)` 和 tool_names。 - -- [ ] **Step 2: 运行测试确认失败** - -Run: -```bash -DATABASE_URL=sqlite:////tmp/fusion_api_tool_round_red.db /Users/sean/code/fusion/fusion-api/.venv311/bin/python -m pytest test/services/stream/test_tool_round.py -q -``` -Expected: FAIL，因为 `ToolRoundRequest` 尚未导出或 `handle_tool_calls_round()` 还不接受 `request`。 - -- [ ] **Step 3: 最小实现 request 对象和 helper 拆分** - -在 `tool_round.py` 增加 `ToolRoundRequest` dataclass，把原入口参数收敛到 request；拆出： -- `append_tool_round_reasoning()` -- `persist_tool_round_checkpoint()` -- `execute_tool_round_tools()` -- `append_tool_round_messages()` -- `complete_tool_round_step()` - -`handle_tool_calls_round()` 只保留顺序编排。 - -- [ ] **Step 4: driver 改为 request 调用** - -`agent_loop_driver._handle_tool_calls_round()` 构造 `ToolRoundRequest(...)` 并调用 `runtime.handle_tool_calls_round_fn(request=request)`。 - -- [ ] **Step 5: 验证** - -Run: -```bash -DATABASE_URL=sqlite:////tmp/fusion_api_tool_round_green.db /Users/sean/code/fusion/fusion-api/.venv311/bin/python -m pytest test/services/stream/test_tool_round.py test/services/stream/test_agent_loop_driver.py test/services/stream/test_agent_loop_contract.py test/test_stream_handler.py -q -/opt/homebrew/bin/python3.11 scripts/check_quality.py -/opt/homebrew/bin/python3.11 scripts/check_architecture.py -/Users/sean/code/fusion/fusion-api/.venv311/bin/python -m ruff check . -/Users/sean/code/fusion/fusion-api/.venv311/bin/python -m ruff format --check . -``` - -验收标准： -- `app/services/stream/tool_round.py:handle_tool_calls_round()` 不再出现在质量扫描的超长函数列表。 -- 聚焦测试保持 43 个通过。 -- 架构检查和 ruff 通过。 -- 不启动本地 Fusion 服务。 diff --git a/docs/superpowers/plans/2026-06-28-agent-loop-url-preprocess.md b/docs/superpowers/plans/2026-06-28-agent-loop-url-preprocess.md deleted file mode 100644 index c78daaeb..00000000 --- a/docs/superpowers/plans/2026-06-28-agent-loop-url-preprocess.md +++ /dev/null @@ -1,57 +0,0 @@ -# Agent Loop URL 预处理拆分 Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** 拆分 `preprocess_url_in_message()` 的 URL 提取、策略处理、reader 调用、fallback tool 注入和成功结果构建职责，保持 URL 路径 A 的外部行为不变。 - -**Architecture:** `preprocess_url_in_message(...)` 保持返回三元组 `(UrlBlock | None, context_msg | None, detected_url | None)`。内部改为小型 helper：`extract_first_url()`、`ensure_url_read_tool()`、`read_url_for_context()`、`build_url_context_message()`、`build_url_read_block()`、`remove_disabled_thinking()`；runner/request prep 调用合同不变。 - -**Tech Stack:** Python 3.11、pytest、ruff、Fusion agent loop、reader-service、URL 安全策略。 - ---- - -### Task 1: 拆分 URL 路径 A 预处理 helper - -**Files:** -- Create: `test/services/stream/test_url_preprocess.py` -- Modify: `app/services/stream/persistence.py` - -- [x] **Step 1: 写先失败的 helper 边界测试** - -新增测试覆盖： -- `extract_first_url(message)` 只返回第一个 http/https URL，无 URL 时返回 `None`。 -- `ensure_url_read_tool(call_kwargs)` 追加 `URL_READ_TOOL` 且重复调用不重复追加。 -- `build_url_context_message(...)` 生成 `role="user"` 的不可信 web context，使用 reader URL 或 normalized URL 兜底。 -- `build_url_read_block(...)` 生成 `UrlBlock` 并沿用 reader 返回的 title/favicon。 -- `remove_disabled_thinking(call_kwargs)` 只删除 `extra_body.thinking.type == "disabled"` 的兼容项。 - -- [x] **Step 2: 运行测试确认失败** - -Run: -```bash -DATABASE_URL=sqlite:////tmp/fusion_api_url_preprocess_red.db /Users/sean/code/fusion/fusion-api/.venv311/bin/python -m pytest test/services/stream/test_url_preprocess.py -q -``` -Expected: FAIL，原因是 helper 尚未导出。 - -- [x] **Step 3: 最小实现 helper 拆分** - -在 `persistence.py` 中： -- 提升 URL 正则为模块常量。 -- 将 `_append_url_read_tool()` 改为 `ensure_url_read_tool()`。 -- 新增 reader URL 解析、reader 调用、context message、UrlBlock 和 thinking 清理 helper。 -- 保持 `preprocess_url_in_message(...)` 行为：不支持 FC 直接跳过；无 URL/策略拒绝/reader 失败都追加 URL_READ_TOOL；成功时插入 user web context、返回 UrlBlock、删除 volcengine disabled thinking。 - -- [x] **Step 4: 验证保持行为** - -Run: -```bash -DATABASE_URL=sqlite:////tmp/fusion_api_url_preprocess_green.db /Users/sean/code/fusion/fusion-api/.venv311/bin/python -m pytest test/services/stream/test_url_preprocess.py test/test_stream_handler.py::UrlPreprocessTests test/services/stream/test_agent_loop_request_prep.py test/test_stream_handler.py -q -/opt/homebrew/bin/python3.11 -m ruff check app/services/stream/persistence.py test/services/stream/test_url_preprocess.py -/opt/homebrew/bin/python3.11 -m ruff format --check app/services/stream/persistence.py test/services/stream/test_url_preprocess.py -/opt/homebrew/bin/python3.11 scripts/check_quality.py -``` - -验收标准： -- `app/services/stream/persistence.py:preprocess_url_in_message()` 不再出现在质量扫描超长函数列表。 -- URL 路径 A 成功、拒绝、无 URL、reader 失败的外部三元组和 call_kwargs 语义不变。 -- 不启动本地 Fusion 服务。 diff --git a/docs/superpowers/plans/2026-06-28-agent-loop-wiring.md b/docs/superpowers/plans/2026-06-28-agent-loop-wiring.md deleted file mode 100644 index 25450606..00000000 --- a/docs/superpowers/plans/2026-06-28-agent-loop-wiring.md +++ /dev/null @@ -1,77 +0,0 @@ -# Agent Loop Wiring Extraction Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** 把 `StreamHandler.generate_to_redis()` 中的 execution/lifecycle dependency wiring 抽到独立模块，让 runner 只保留入口适配、limits 和 DB session 外层生命周期。 - -**Architecture:** 新增 `app/services/stream/agent_loop_wiring.py`，定义 run 输入、runner 注入依赖、以及 `build_agent_loop_lifecycle_call()`。新模块只装配 `AgentLoopExecutionRequest`、`AgentLoopDependencies`、`AgentLoopLifecycleRequest` 和 `AgentLoopLifecycleDependencies`；具体函数仍由 `runner.py` 按当前模块变量传入，保持现有 `app.services.stream.runner.*` patch 路径有效。 - -**Tech Stack:** Python 3.11、pytest、dataclasses、Fusion agent loop。 - ---- - -### Task 1: 抽取 agent loop wiring builder - -**Files:** -- Create: `app/services/stream/agent_loop_wiring.py` -- Create: `test/services/stream/test_agent_loop_wiring.py` -- Modify: `app/services/stream/runner.py` -- Modify: `test/test_stream_handler.py` - -- [x] **Step 1: 写先失败的 wiring 测试** - -新增 `test/services/stream/test_agent_loop_wiring.py`，覆盖： -- `build_agent_loop_lifecycle_call()` 使用输入参数构造 `AgentLoopLifecycleRequest`。 -- `options=None` / `capabilities=None` 时传给 call config builder 的是空 dict。 -- `AgentLoopExecutionRequest` 保留 db、conversation_id、user_id、model_id、provider、message_id、task_id、trace_id。 -- `AgentLoopDependencies` 使用 runner 注入的 session cache、redis writer factory、round/tool/summary/llm/stream/persist/log/warning/clock 函数。 -- `AgentLoopLifecycleDependencies` 使用 runner 注入的 append/start/prepare/run/finalize/fallback 函数。 - -Run: -```bash -DATABASE_URL=sqlite:////tmp/fusion_api_agent_loop_wiring_red.db /Users/sean/code/fusion/fusion-api/.venv311/bin/python -m pytest test/services/stream/test_agent_loop_wiring.py -q -``` -Expected: FAIL，原因是 `app.services.stream.agent_loop_wiring` 尚不存在。 - -- [x] **Step 2: 实现 `agent_loop_wiring.py`** - -实现： -- `AgentLoopRunInput`：保存 `generate_to_redis()` 的业务输入。 -- `AgentLoopWiringDependencies`：保存 runner 注入的 builder、service、tool、finalizer、logger、clock、redis writer factory。 -- `AgentLoopLifecycleCall`：保存 `request`、`execution`、`dependencies` 三元组。 -- `build_agent_loop_lifecycle_call(run_input, db, limits, dependencies)`：集中创建 call config、execution context 和 lifecycle dependencies。 - -约束： -- 不从新模块静态 import `append_chunk`、`finalize_stream`、`llm_call_with_retry`、`execute_tools_parallel`、`persist_message` 等生产函数。 -- 新模块可以 import 类型和 dataclass，以及 `AgentLoopExecutionRequest`、`AgentLoopDependencies`、`AgentLoopLifecycleRequest`、`AgentLoopLifecycleDependencies`。 - -- [x] **Step 3: 改 runner 调用 wiring builder** - -在 `runner.py` 中： -- 继续保留 `AGENT_MAX_STEPS`、`AGENT_MAX_TOOL_CALLS`、`AGENT_TOTAL_TIMEOUT`，保证现有测试 patch constants 仍生效。 -- 新增 `_agent_loop_limits()` 返回 `AgentLoopLimits(...)`。 -- 新增 `_agent_loop_wiring_dependencies()` 返回 `AgentLoopWiringDependencies(...)`，所有函数引用都来自 runner 当前模块变量。 -- `generate_to_redis()` 只构造 `AgentLoopRunInput`，打开 `SessionLocal()`，调用 `build_agent_loop_lifecycle_call(...)`，再调用 `run_agent_loop_lifecycle(...)`，最后 `db.close()`。 - -- [x] **Step 4: 补 runner patch 路径回归测试** - -在 `test/test_stream_handler.py` 增加或调整测试： -- patch `app.services.stream.runner.run_agent_loop_lifecycle` 捕获 `execution` 和 `dependencies`。 -- patch `app.services.stream.runner.llm_call_with_retry`、`stream_round`、`execute_tools_parallel`、`persist_message`、`append_chunk`、`finalize_stream` 为 sentinel/mock。 -- 调用 `generate_to_redis()` 后断言 `execution.runtime.llm_call_fn`、`execution.runtime.stream_round_fn`、`execution.runtime.execute_tools_fn`、`execution.runtime.persist_message_fn`、`dependencies.append_chunk_fn`、`dependencies.finalize_stream_fn` 均为 runner patched 对象。 - -- [x] **Step 5: 验证保持行为** - -Run: -```bash -DATABASE_URL=sqlite:////tmp/fusion_api_agent_loop_wiring_green.db /Users/sean/code/fusion/fusion-api/.venv311/bin/python -m pytest test/services/stream/test_agent_loop_wiring.py test/services/stream/test_agent_loop_lifecycle.py test/services/stream/test_agent_loop_execution.py test/services/stream/test_agent_loop_run_completion.py test/services/stream/test_agent_loop_driver.py test/test_stream_handler.py -q -/opt/homebrew/bin/python3.11 -m ruff check app/services/stream/agent_loop_wiring.py app/services/stream/runner.py test/services/stream/test_agent_loop_wiring.py test/test_stream_handler.py -/opt/homebrew/bin/python3.11 -m ruff format --check app/services/stream/agent_loop_wiring.py app/services/stream/runner.py test/services/stream/test_agent_loop_wiring.py test/test_stream_handler.py -/opt/homebrew/bin/python3.11 scripts/check_quality.py -``` - -验收标准： -- `runner.generate_to_redis()` 不再直接拼 `AgentLoopDependencies(...)` / `AgentLoopLifecycleDependencies(...)`。 -- `runner.generate_to_redis()` 行数显著下降，目标低于 50 行。 -- 旧 `runner.*` patch 路径仍由测试锁住。 -- 不启动本地 Fusion 服务。 diff --git a/docs/superpowers/plans/2026-06-28-agent-progress-protocol-v2.md b/docs/superpowers/plans/2026-06-28-agent-progress-protocol-v2.md deleted file mode 100644 index 896d4410..00000000 --- a/docs/superpowers/plans/2026-06-28-agent-progress-protocol-v2.md +++ /dev/null @@ -1,805 +0,0 @@ -# Agent Progress Protocol v2 Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** 在现有 Redis Stream/SSE agent_event 协议内增加 v2 进度、计划、工具摘要和证据事件，并持久化 compact progress snapshot 供历史会话恢复。 - -**Architecture:** `AgentEventEmitter` 继续作为唯一事件发送方，新增 v2 Pydantic 事件和 emitter 方法；`progress_state.py` 负责纯函数折叠与裁剪，`AgentProgressRecorder` 作为 writer 旁路 sink 写入 `agent_progress_snapshots`。生命周期模块只发语义事件，不直接知道 DB snapshot 细节。 - -**Tech Stack:** FastAPI, SQLAlchemy, Alembic, PostgreSQL JSONB, Redis Stream, Pydantic v2, pytest/unittest, ruff. - ---- - -## 文件结构 - -- 修改 `app/services/agent/events.py`：新增 v2 事件模型、枚举和 `AnyAgentEvent` union。 -- 修改 `app/services/agent/emitter.py`：新增 v2 emit 方法，保持单 run sequence lock。 -- 新建 `app/services/agent/progress_state.py`：纯函数 reducer、字段裁剪、snapshot 限额。 -- 新建 `app/services/agent/progress_recorder.py`：接收 agent_event payload，折叠并 upsert snapshot。 -- 修改 `app/services/stream/tool_executor.py`：新增 `AgentEventCompositeWriter`，并从工具执行记录派生 digest/evidence。 -- 修改 `app/services/stream/agent_loop_execution.py`：构造 composite writer 与 recorder。 -- 修改 `app/services/stream/agent_loop_lifecycle.py`：run 开始发初始 progress 和默认 plan。 -- 修改 `app/services/stream/step_lifecycle.py`：step 开始/完成发 plan/progress 更新。 -- 修改 `app/services/stream/run_finalizer.py`：终态前发 progress/plan 收尾事件。 -- 修改 `app/db/models.py`：新增 `AgentProgressSnapshot` ORM。 -- 新建 Alembic migration：`alembic/versions/9c1f2a3b4d5e_add_agent_progress_snapshots.py`，`down_revision = "4c6a1f2b8d90"`。 -- 修改 `app/schemas/chat.py`：`AgentRunSummary` 增加 `progress`。 -- 修改 `app/db/repositories.py`：会话详情返回最新 run 的 progress snapshot。 -- 新增/修改测试： - - `test/services/agent/test_events.py` - - `test/services/agent/test_emitter.py` - - `test/services/agent/test_progress_state.py` - - `test/services/agent/test_progress_recorder.py` - - `test/services/stream/test_agent_loop_contract.py` - - `test/services/stream/test_step_lifecycle.py` - - `test/test_tool_executor.py` - - `test/test_repositories.py` - -## 实施约束 - -- 不改变外层 SSE envelope：仍是 `chunk_type=agent_event`。 -- 不新增首轮 LLM planning call。 -- v2 写入失败只记录 warning，不中断正文 token、v1 事件和最终回答。 -- 所有用户可见摘要必须短文本，不写入 prompt、raw HTML、raw tool JSON、内部 URL、密钥或容器/Redis 细节。 -- 本阶段不启动本地 Fusion 服务；验证只跑 pytest、ruff、migration 相关命令和后续 CI/CD。 -- 为减少 CI/CD 浪费，本计划执行时按测试检查点推进，最终合并为一个 API 功能提交。 - -## Task 1: v2 事件模型 - -**Files:** -- Modify: `app/services/agent/events.py` -- Modify: `test/services/agent/test_events.py` - -- [ ] **Step 1: 写失败测试** - -在 `test/services/agent/test_events.py` 追加： - -```python -class AgentProgressV2EventModelTests(unittest.TestCase): - def _common(self): - return dict(run_id="r1", step_id=None, tool_call_id=None, sequence=0, trace_id="r1", ts=1.0) - - def test_run_progress_updated_requires_protocol_version_2(self): - from app.services.agent.events import RunProgressUpdated - - ev = RunProgressUpdated( - type="run_progress_updated", - protocol_version=2, - phase="researching", - label="正在搜索相关资料", - completed_steps=1, - total_steps=4, - completed_tool_calls=2, - max_tool_calls=20, - **self._common(), - ) - - self.assertEqual(ev.protocol_version, 2) - self.assertEqual(ev.phase, "researching") - - def test_plan_snapshot_forbids_unknown_fields(self): - from app.services.agent.events import PlanSnapshot - - with self.assertRaises(ValidationError): - PlanSnapshot( - type="plan_snapshot", - protocol_version=2, - plan_id="plan-r1", - revision=1, - items=[], - unexpected=True, - **self._common(), - ) - - def test_tool_result_digest_and_evidence_models(self): - from app.services.agent.events import EvidenceItemUpserted, ToolResultDigest - - digest = ToolResultDigest( - type="tool_result_digest", - protocol_version=2, - step_id="s1", - tool_call_id="tc1", - tool_name="web_search", - status="success", - title="找到 2 条结果", - summary="优先保留官方来源。", - key_findings=["官方页面确认发布时间。"], - source_refs=["ev-1"], - truncated=False, - **{k: v for k, v in self._common().items() if k not in {"step_id", "tool_call_id"}}, - ) - self.assertEqual(digest.status, "success") - - evidence = EvidenceItemUpserted( - type="evidence_item_upserted", - protocol_version=2, - step_id="s1", - tool_call_id="tc1", - evidence={ - "id": "ev-1", - "kind": "web", - "status": "candidate", - "title": "官方发布页", - "url": "https://example.com/news", - "domain": "example.com", - "claim": "官方发布页确认发布时间。", - "snippet": "页面摘要。", - "used_by_final_answer": False, - }, - **{k: v for k, v in self._common().items() if k not in {"step_id", "tool_call_id"}}, - ) - self.assertEqual(evidence.evidence.id, "ev-1") -``` - -- [ ] **Step 2: 运行测试确认失败** - -Run: - -```bash -python -m pytest test/services/agent/test_events.py -q -``` - -Expected: FAIL，原因是 `RunProgressUpdated`、`PlanSnapshot`、`ToolResultDigest`、`EvidenceItemUpserted` 还不存在。 - -- [ ] **Step 3: 实现事件模型** - -在 `app/services/agent/events.py` 增加： - -```python -AgentProgressPhase = Literal["planning", "thinking", "researching", "reading", "synthesizing", "answering", "recovering"] -AgentPlanItemStatus = Literal["pending", "running", "completed", "failed", "skipped", "blocked"] -AgentPlanItemKind = Literal["reasoning", "search", "read", "synthesis", "answer", "other"] - - -class AgentPlanItem(BaseModel): - model_config = ConfigDict(extra="forbid") - id: str - title: str - status: AgentPlanItemStatus - kind: AgentPlanItemKind - summary: str | None = None - tool_names: list[str] = Field(default_factory=list) - evidence_item_ids: list[str] = Field(default_factory=list) - - -class AgentEvidenceItem(BaseModel): - model_config = ConfigDict(extra="forbid") - id: str - kind: Literal["web", "file", "tool", "model"] - status: Literal["candidate", "used", "discarded"] - title: str - url: str | None = None - domain: str | None = None - claim: str - snippet: str | None = None - used_by_final_answer: bool = False -``` - -并新增五个事件类：`RunProgressUpdated`、`PlanSnapshot`、`PlanStepUpdated`、`ToolResultDigest`、`EvidenceItemUpserted`，全部带 `protocol_version: Literal[2]`，再加入 `AnyAgentEvent`。 - -- [ ] **Step 4: 验证通过** - -Run: - -```bash -python -m pytest test/services/agent/test_events.py -q -``` - -Expected: PASS。 - -## Task 2: emitter v2 方法和 sequence 语义 - -**Files:** -- Modify: `app/services/agent/emitter.py` -- Modify: `test/services/agent/test_emitter.py` - -- [ ] **Step 1: 写失败测试** - -在 `test/services/agent/test_emitter.py` 追加： - -```python - async def test_v2_events_use_same_sequence_stream(self): - writer = AsyncMock() - em = AgentEventEmitter(run_id="r1", trace_id="r1", conversation_id="c1", redis_writer=writer) - - await em.run_started(message_id="m1", model="gpt", tools=["web_search"], config={"max_steps": 8}) - await em.run_progress_updated(phase="planning", label="正在理解问题", completed_steps=0, total_steps=4) - await em.plan_snapshot(plan_id="plan-r1", revision=1, items=[ - {"id": "understand", "title": "理解问题", "status": "running", "kind": "reasoning", "tool_names": [], "evidence_item_ids": []} - ]) - - events = [call.args[2] for call in writer.append_chunk.call_args_list] - self.assertEqual([event["sequence"] for event in events], [0, 1, 2]) - self.assertEqual(events[1]["type"], "run_progress_updated") - self.assertEqual(events[1]["protocol_version"], 2) - self.assertIsNone(events[1]["step_id"]) - - async def test_step_level_plan_update_inherits_current_step(self): - writer = AsyncMock() - em = AgentEventEmitter(run_id="r1", trace_id="r1", conversation_id="c1", redis_writer=writer) - step_id = await em.step_started(step_number=1) - - await em.plan_step_updated( - plan_id="plan-r1", - revision=2, - item={"id": "search", "title": "搜索资料", "status": "completed", "kind": "search", "tool_names": ["web_search"], "evidence_item_ids": []}, - ) - - payload = writer.append_chunk.call_args_list[-1].args[2] - self.assertEqual(payload["type"], "plan_step_updated") - self.assertEqual(payload["step_id"], step_id) -``` - -- [ ] **Step 2: 运行测试确认失败** - -Run: - -```bash -python -m pytest test/services/agent/test_emitter.py -q -``` - -Expected: FAIL，原因是 emitter 尚无 v2 方法。 - -- [ ] **Step 3: 实现最小 emitter 方法** - -在 `AgentEventEmitter` 增加方法： - -```python - async def run_progress_updated(self, *, phase: str, label: str, completed_steps: int | None = None, - total_steps: int | None = None, completed_tool_calls: int | None = None, - max_tool_calls: int | None = None) -> None: - await self._emit(ev.RunProgressUpdated( - type="run_progress_updated", - protocol_version=2, - phase=phase, - label=label, - completed_steps=completed_steps, - total_steps=total_steps, - completed_tool_calls=completed_tool_calls, - max_tool_calls=max_tool_calls, - **self._envelope(step_id=None), - )) -``` - -同样实现 `plan_snapshot()`、`plan_step_updated()`、`tool_result_digest()`、`evidence_item_upserted()`。run-level progress/snapshot 显式 `step_id=None`；step/tool 级事件默认继承 current step 或传入 `tool_call_id`。 - -- [ ] **Step 4: 验证通过** - -Run: - -```bash -python -m pytest test/services/agent/test_emitter.py -q -``` - -Expected: PASS。 - -## Task 3: progress state reducer 和裁剪规则 - -**Files:** -- Create: `app/services/agent/progress_state.py` -- Create: `test/services/agent/test_progress_state.py` - -- [ ] **Step 1: 写失败测试** - -新建 `test/services/agent/test_progress_state.py`： - -```python -from app.services.agent.progress_state import apply_progress_event, empty_progress_state - - -def test_plan_snapshot_replaces_existing_plan(): - state = empty_progress_state(run_id="r1", message_id="m1") - state = apply_progress_event(state, { - "type": "plan_snapshot", - "protocol_version": 2, - "plan_id": "plan-r1", - "revision": 1, - "items": [{"id": "a", "title": "理解问题", "status": "running", "kind": "reasoning", "tool_names": [], "evidence_item_ids": []}], - }) - state = apply_progress_event(state, { - "type": "plan_snapshot", - "protocol_version": 2, - "plan_id": "plan-r1", - "revision": 2, - "items": [{"id": "b", "title": "整理回答", "status": "pending", "kind": "answer", "tool_names": [], "evidence_item_ids": []}], - }) - - assert state["plan"]["revision"] == 2 - assert [item["id"] for item in state["plan"]["items"]] == ["b"] - - -def test_plan_step_update_ignores_stale_revision(): - state = empty_progress_state(run_id="r1", message_id="m1") - state = apply_progress_event(state, {"type": "plan_snapshot", "protocol_version": 2, "plan_id": "plan-r1", "revision": 2, "items": []}) - stale = apply_progress_event(state, { - "type": "plan_step_updated", - "protocol_version": 2, - "plan_id": "plan-r1", - "revision": 2, - "item": {"id": "search", "title": "搜索资料", "status": "running", "kind": "search", "tool_names": [], "evidence_item_ids": []}, - }) - - assert stale["plan"]["items"] == [] - - -def test_evidence_and_tool_digest_upsert_and_cap(): - state = empty_progress_state(run_id="r1", message_id="m1") - for index in range(14): - state = apply_progress_event(state, { - "type": "evidence_item_upserted", - "protocol_version": 2, - "evidence": { - "id": f"ev-{index}", - "kind": "web", - "status": "used" if index == 0 else "candidate", - "title": "t" * 80, - "domain": "example.com", - "claim": "c" * 200, - "snippet": "s" * 300, - "used_by_final_answer": index == 0, - }, - }) - - ids = [item["id"] for item in state["evidence"]] - assert len(ids) == 12 - assert "ev-0" in ids - assert state["evidence"][0]["claim"] == "c" * 120 - assert len(state["evidence"][0]["snippet"]) == 180 -``` - -- [ ] **Step 2: 运行测试确认失败** - -Run: - -```bash -python -m pytest test/services/agent/test_progress_state.py -q -``` - -Expected: FAIL，原因是 `progress_state.py` 不存在。 - -- [ ] **Step 3: 实现 reducer** - -创建 `app/services/agent/progress_state.py`，导出： - -```python -MAX_EVIDENCE_ITEMS = 12 -MAX_TOOL_DIGESTS = 20 - - -def empty_progress_state(*, run_id: str, message_id: str) -> dict: - return { - "run_id": run_id, - "message_id": message_id, - "status": "running", - "progress": None, - "plan": None, - "tool_digests": [], - "evidence": [], - "updated_at": None, - } -``` - -`apply_progress_event(state, event)` 只处理 v2 事件和终态 v1 事件。实现细节： - -- `run_progress_updated` 覆盖 `state["progress"]`。 -- `plan_snapshot` 覆盖 `state["plan"]`。 -- `plan_step_updated` 在 `revision > current revision` 时按 `item.id` upsert。 -- `tool_result_digest` 按 `tool_call_id` upsert，最多 20 条。 -- `evidence_item_upserted` 按 `evidence.id` upsert，最多 12 条，保留 `used/used_by_final_answer` 和最近 candidate。 -- `run_completed/run_failed/run_interrupted` 更新 `state["status"]`。 -- 字符串裁剪：`label=40`、`summary=120`、`key_findings[]=80`、`claim=120`、`snippet=180`、标题不超过 80。 - -- [ ] **Step 4: 验证通过** - -Run: - -```bash -python -m pytest test/services/agent/test_progress_state.py -q -``` - -Expected: PASS。 - -## Task 4: snapshot 表、migration 和 recorder - -**Files:** -- Modify: `app/db/models.py` -- Create: `alembic/versions/9c1f2a3b4d5e_add_agent_progress_snapshots.py` -- Create: `app/services/agent/progress_recorder.py` -- Create: `test/services/agent/test_progress_recorder.py` - -- [ ] **Step 1: 写 recorder 失败测试** - -新建 `test/services/agent/test_progress_recorder.py`，用 fake session 验证行为： - -```python -from unittest.mock import Mock - -from app.services.agent.progress_recorder import AgentProgressRecorder - - -def test_recorder_ignores_non_agent_event_chunks(): - db = Mock() - recorder = AgentProgressRecorder(db=db, run_id="r1", conversation_id="c1", message_id="m1", user_id="u1") - - recorder.record_chunk("c1", "answering", {"delta": "x"}) - - db.merge.assert_not_called() - - -def test_recorder_upserts_snapshot_for_v2_event(): - db = Mock() - recorder = AgentProgressRecorder(db=db, run_id="r1", conversation_id="c1", message_id="m1", user_id="u1") - - recorder.record_chunk("c1", "agent_event", { - "type": "run_progress_updated", - "protocol_version": 2, - "phase": "planning", - "label": "正在理解问题", - }) - - db.merge.assert_called_once() - db.commit.assert_called_once() -``` - -- [ ] **Step 2: 运行测试确认失败** - -Run: - -```bash -python -m pytest test/services/agent/test_progress_recorder.py -q -``` - -Expected: FAIL，原因是 recorder/model 尚不存在。 - -- [ ] **Step 3: 新增 ORM 和 migration** - -在 `app/db/models.py` 增加： - -```python -class AgentProgressSnapshot(Base): - """Agent 可读进度 compact snapshot — 按 run_id 唯一保存最新折叠状态""" - - __tablename__ = "agent_progress_snapshots" - - id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4())) - run_id = Column(String, ForeignKey("agent_sessions.id", ondelete="CASCADE"), nullable=False, unique=True, index=True) - conversation_id = Column(String, ForeignKey("conversations.id", ondelete="CASCADE"), nullable=False, index=True) - message_id = Column(String, nullable=True, index=True) - user_id = Column(String, nullable=False, index=True) - protocol_version = Column(Integer, nullable=False, default=2) - state = Column(JSONB, nullable=False) - created_at = Column(DateTime, default=get_china_time, index=True) - updated_at = Column(DateTime, default=get_china_time, onupdate=get_china_time, index=True) - - __table_args__ = ( - Index("ix_agent_progress_message_updated", "conversation_id", "message_id", "updated_at"), - ) -``` - -Migration 文件头使用： - -```python -revision: str = "9c1f2a3b4d5e" -down_revision: Union[str, Sequence[str], None] = "4c6a1f2b8d90" -branch_labels: Union[str, Sequence[str], None] = None -depends_on: Union[str, Sequence[str], None] = None -``` - -Migration 内容使用 `op.create_table()` 创建同名表、`op.create_index()` 创建 `ix_agent_progress_message_updated`，并在 downgrade 删除索引和表。 - -- [ ] **Step 4: 实现 recorder** - -`AgentProgressRecorder` 构造参数：`db`、`run_id`、`conversation_id`、`message_id`、`user_id`、`logger`。方法： - -```python -def record_chunk(self, conversation_id: str, chunk_type: str, payload: dict) -> None: - if conversation_id != self.conversation_id or chunk_type != "agent_event": - return - next_state = apply_progress_event(self._state, payload) - if next_state is self._state: - return - self._state = next_state - self._upsert_snapshot() -``` - -`_upsert_snapshot()` 查询或 merge `AgentProgressSnapshot`，`commit()`；异常时 `rollback()` 并 warning，不 re-raise。 - -- [ ] **Step 5: 验证通过** - -Run: - -```bash -python -m pytest test/services/agent/test_progress_recorder.py -q -``` - -Expected: PASS。 - -## Task 5: writer 接入和生命周期最小 v2 事件 - -**Files:** -- Modify: `app/services/stream/tool_executor.py` -- Modify: `app/services/stream/agent_loop_execution.py` -- Modify: `app/services/stream/agent_loop_lifecycle.py` -- Modify: `app/services/stream/step_lifecycle.py` -- Modify: `app/services/stream/run_finalizer.py` -- Modify: `test/services/stream/test_agent_loop_contract.py` -- Modify: `test/services/stream/test_step_lifecycle.py` - -- [ ] **Step 1: 写失败测试：lifecycle 初始 progress/plan** - -在 `test/services/stream/test_agent_loop_contract.py` 增加断言：一次 agent run 的事件序列中，`run_started` 后出现 `run_progress_updated` 和 `plan_snapshot`，且不改变现有 v1 事件顺序约束。 - -核心断言： - -```python -types = [payload["type"] for payload in emitted_agent_events] -assert "run_started" in types -assert "run_progress_updated" in types -assert "plan_snapshot" in types -assert types.index("run_started") < types.index("run_progress_updated") < types.index("plan_snapshot") -``` - -- [ ] **Step 2: 写失败测试：step 更新 plan/progress** - -在 `test/services/stream/test_step_lifecycle.py` 增加 fake emitter，要求 `start_agent_step()` 调用 `plan_step_updated()`，`complete_agent_step()` 调用 `run_progress_updated()`。 - -- [ ] **Step 3: 运行测试确认失败** - -Run: - -```bash -python -m pytest test/services/stream/test_agent_loop_contract.py test/services/stream/test_step_lifecycle.py -q -``` - -Expected: FAIL，原因是生命周期尚未发 v2 事件。 - -- [ ] **Step 4: 实现 composite writer** - -在 `app/services/stream/tool_executor.py` 增加： - -```python -class AgentEventCompositeWriter: - def __init__(self, *, redis_writer: AgentEventRedisWriter, recorder=None) -> None: - self.redis_writer = redis_writer - self.recorder = recorder - - async def append_chunk(self, conversation_id: str, chunk_type: str, payload: dict) -> None: - await self.redis_writer.append_chunk(conversation_id, chunk_type, payload) - if self.recorder is not None: - self.recorder.record_chunk(conversation_id, chunk_type, payload) -``` - -在 `agent_loop_execution.py` 使用 recorder 包装 writer；测试里没有 DB 时保留原 writer。 - -- [ ] **Step 5: 实现默认计划工具** - -在 lifecycle 内部增加私有函数： - -```python -def _default_plan_items(tools: list[str]) -> list[dict]: - if tools: - return [ - {"id": "understand", "title": "理解问题", "status": "running", "kind": "reasoning", "tool_names": [], "evidence_item_ids": []}, - {"id": "search", "title": "查找资料", "status": "pending", "kind": "search", "tool_names": tools, "evidence_item_ids": []}, - {"id": "read", "title": "读取关键来源", "status": "pending", "kind": "read", "tool_names": tools, "evidence_item_ids": []}, - {"id": "answer", "title": "整理回答", "status": "pending", "kind": "answer", "tool_names": [], "evidence_item_ids": []}, - ] - return [ - {"id": "understand", "title": "理解问题", "status": "running", "kind": "reasoning", "tool_names": [], "evidence_item_ids": []}, - {"id": "answer", "title": "整理回答", "status": "pending", "kind": "answer", "tool_names": [], "evidence_item_ids": []}, - ] -``` - -在 `_start_run()` 的 `start_agent_run_fn` 之后发： - -```python -await execution.emitter.run_progress_updated(phase="planning", label="正在理解问题", completed_steps=0, total_steps=len(plan_items), completed_tool_calls=0, max_tool_calls=request.limits.max_tool_calls) -await execution.emitter.plan_snapshot(plan_id=f"plan-{execution.run_id}", revision=1, items=plan_items) -``` - -- [ ] **Step 6: step 和终态更新** - -`start_agent_step()` 进入工具轮时把 `search/read` 标记 running；`complete_agent_step()` 根据 `tool_call_count` 更新 completed steps/tool count；`run_finalizer.py` 在 completed/failed/interrupted/limit 前把仍 running 的 plan item 收尾为 `completed/blocked/failed`。 - -- [ ] **Step 7: 验证通过** - -Run: - -```bash -python -m pytest test/services/stream/test_agent_loop_contract.py test/services/stream/test_step_lifecycle.py -q -``` - -Expected: PASS。 - -## Task 6: 工具结果 digest 和 evidence 派生 - -**Files:** -- Create: `app/services/agent/progress_digest.py` -- Modify: `app/services/stream/tool_executor.py` -- Modify: `test/test_tool_executor.py` - -- [ ] **Step 1: 写失败测试** - -在 `test/test_tool_executor.py` 增加一个 fake emitter 测试：工具成功后会收到 `tool_result_digest()`，web/search 类结果会收到不超过 12 条 `evidence_item_upserted()`。 - -核心断言： - -```python -assert emitter.tool_result_digest.await_count == 1 -payload = emitter.tool_result_digest.await_args.kwargs -assert payload["tool_call_id"] == "tc1" -assert payload["summary"] -assert len(payload["key_findings"]) <= 5 -``` - -- [ ] **Step 2: 运行测试确认失败** - -Run: - -```bash -python -m pytest test/test_tool_executor.py -q -``` - -Expected: FAIL，原因是工具执行尚未发 digest/evidence。 - -- [ ] **Step 3: 实现 digest helper** - -`progress_digest.py` 导出： - -```python -def build_tool_result_digest(record: ToolExecutionRecord) -> dict: - return { - "tool_call_id": record.tool_call["id"], - "tool_name": record.tool_call["name"], - "status": _map_status(record.result.status), - "title": _safe_title(record), - "summary": _safe_summary(record), - "key_findings": _safe_findings(record)[:5], - "source_refs": [], - "truncated": bool(getattr(record.result, "truncated", False)), - } - - -def build_evidence_items(record: ToolExecutionRecord) -> list[dict]: - return _extract_public_sources(record)[:12] -``` - -helper 只从 handler summary、公开 URL/title/domain/snippet 取安全短文本；无法提取时返回空 evidence，但仍发 digest。 - -- [ ] **Step 4: 在 tool executor 发事件** - -`execute_one_tool_call()` 返回 record 前，如果 `request.emitter` 存在： - -```python -digest = build_tool_result_digest(record) -await request.emitter.tool_result_digest(**digest) -for evidence in build_evidence_items(record): - await request.emitter.evidence_item_upserted(tool_call_id=tool_call["id"], evidence=evidence) -``` - -预算失败路径和未知工具路径也发 digest，status 映射为 `failed` 或 `degraded`。 - -- [ ] **Step 5: 验证通过** - -Run: - -```bash -python -m pytest test/test_tool_executor.py -q -``` - -Expected: PASS。 - -## Task 7: repository 和 API schema 暴露 progress - -**Files:** -- Modify: `app/schemas/chat.py` -- Modify: `app/db/repositories.py` -- Modify: `test/test_repositories.py` - -- [ ] **Step 1: 写失败测试** - -在 `test/test_repositories.py` 增加：当某 assistant message 有最新 `AgentSession` 和对应 `AgentProgressSnapshot` 时，`message.agent_run.progress` 等于 snapshot state。 - -核心断言： - -```python -message = repo.get_conversation(conv_id, user_id=user_id).messages[-1] -assert message.agent_run.run_id == "run-new" -assert message.agent_run.progress["plan"]["plan_id"] == "plan-run-new" -``` - -另加兼容测试：没有 snapshot 时 `progress is None`。 - -- [ ] **Step 2: 运行测试确认失败** - -Run: - -```bash -python -m pytest test/test_repositories.py -q -``` - -Expected: FAIL，原因是 schema 和 repository 尚未返回 progress。 - -- [ ] **Step 3: 修改 schema/repository** - -`AgentRunSummary` 增加： - -```python -progress: Optional[Dict[str, Any]] = None -``` - -`_latest_agent_runs_for_messages()` 查询最新 run 后收集 `run_ids`，再批量查 `AgentProgressSnapshot.run_id.in_(run_ids)`，构造 `snapshots_by_run_id`，然后填入： - -```python -progress=snapshots_by_run_id.get(row.id).state if row.id in snapshots_by_run_id else None -``` - -- [ ] **Step 4: 验证通过** - -Run: - -```bash -python -m pytest test/test_repositories.py -q -``` - -Expected: PASS。 - -## Task 8: 后端集成验证和提交 - -**Files:** -- All backend files above. - -- [ ] **Step 1: 跑目标测试集** - -Run: - -```bash -python -m pytest \ - test/services/agent/test_events.py \ - test/services/agent/test_emitter.py \ - test/services/agent/test_progress_state.py \ - test/services/agent/test_progress_recorder.py \ - test/services/stream/test_agent_loop_contract.py \ - test/services/stream/test_step_lifecycle.py \ - test/test_tool_executor.py \ - test/test_repositories.py \ - -q -``` - -Expected: PASS。 - -- [ ] **Step 2: 跑后端全量检查** - -Run: - -```bash -python -m pytest test/ -q -python -m ruff check . -python -m ruff format --check . -git diff --check -``` - -Expected: 全部 exit 0。 - -- [ ] **Step 3: 自审要求覆盖** - -逐项确认： - -- SSE 外层 envelope 未变。 -- v1 事件仍发出，sequence 与 v2 共享单调序列。 -- recorder 失败不影响 Redis writer。 -- snapshot 字段裁剪和数量上限生效。 -- repository 无 snapshot 时兼容旧消息。 -- 没有新增本地服务启动命令或依赖本地 dev 数据。 - -- [ ] **Step 4: 提交 API 功能** - -Run: - -```bash -git status --short -git add app/services/agent/events.py app/services/agent/emitter.py app/services/agent/progress_state.py app/services/agent/progress_recorder.py app/services/agent/progress_digest.py app/services/stream/tool_executor.py app/services/stream/agent_loop_execution.py app/services/stream/agent_loop_lifecycle.py app/services/stream/step_lifecycle.py app/services/stream/run_finalizer.py app/db/models.py app/schemas/chat.py app/db/repositories.py alembic/versions/*_add_agent_progress_snapshots.py test/services/agent/test_events.py test/services/agent/test_emitter.py test/services/agent/test_progress_state.py test/services/agent/test_progress_recorder.py test/services/stream/test_agent_loop_contract.py test/services/stream/test_step_lifecycle.py test/test_tool_executor.py test/test_repositories.py -git commit -m "feat: 支持 agent progress protocol v2" -m "Co-Authored-By: Codex " -``` - -Expected: 生成一个 API 功能提交。 diff --git a/docs/superpowers/plans/2026-06-28-agent-round-helpers.md b/docs/superpowers/plans/2026-06-28-agent-round-helpers.md deleted file mode 100644 index e294c56b..00000000 --- a/docs/superpowers/plans/2026-06-28-agent-round-helpers.md +++ /dev/null @@ -1,19 +0,0 @@ -# 2026-06-28 agent_round helper 拆分计划 - -## 背景 - -`scripts/check_quality.py` 目前在 stream 目录里只剩 `app/services/stream/agent_round.py:run_agent_round()` 超过 50 行。该函数同时承担 LLM 调用、stream_round 参数转发、日志摘要和 usage 累计，虽然行为已经很薄，但仍是 agent-loop 后续优化的质量红线。 - -## 目标 - -1. 保持 `run_agent_round()` 对外契约不变。 -2. 抽出普通 round 的 LLM 调用 + stream 消费 helper，使 `run_agent_round()` 退出超长函数列表。 -3. 用回归测试锁定 helper 的调用顺序、参数透传和返回值。 -4. 不启动本地 Fusion 服务；只做单元测试、质量脚本、CI/CD 和部署后真实 Chrome 回归。 - -## 验证 - -1. 先跑相关基线测试，确认当前行为稳定。 -2. 先补会失败的 helper 测试，再实现。 -3. 通过相关测试、全量测试、ruff、架构/质量脚本。 -4. 提交并推送后监控 GitHub Actions 和 dev 部署。 diff --git a/docs/superpowers/plans/2026-06-28-agent-run-continuation.md b/docs/superpowers/plans/2026-06-28-agent-run-continuation.md deleted file mode 100644 index 7fa0f099..00000000 --- a/docs/superpowers/plans/2026-06-28-agent-run-continuation.md +++ /dev/null @@ -1,1517 +0,0 @@ -# Agent Run 继续执行 Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** 为 `limit_reached` 的 agent 回答增加“继续查”，用新的 run 续写同一条 assistant 消息。 - -**Architecture:** 后端新增 continuation API，校验原 assistant message 和最近一次 `limit_reached` run 后，用同一 `assistant_message_id` 启动新的 agent loop。agent loop 支持传入已有 content blocks 和 continuation system prompt；前端用独立 hook 调 continuation SSE，并用 streamSlice 的 static blocks 保留旧内容后追加新 delta。 - -**Tech Stack:** FastAPI、SQLAlchemy、Alembic、Redis Stream、pytest、Next.js 15、React 19、Redux Toolkit、Vitest、Testing Library。 - ---- - -## Scope Check - -这份 spec 横跨 `fusion-api` 和 `fusion-ui`，但它是一个单一用户能力：触顶后继续同一条 assistant 消息。后端 API 和前端 CTA 必须一起交付才能形成可用闭环，因此保持一个 implementation plan。实施时使用一个工作分支；最终按子仓分别提交，避免把半成品多次推送触发流水线。 - -## File Structure - -### fusion-api - -- Create: `alembic/versions/4c6a1f2b8d90_add_agent_session_config.py` - - 给 `agent_sessions` 增加 `config` JSONB，用于持久化上一轮预算。 -- Modify: `app/db/models.py` - - `AgentSession` 增加 `run_config = Column("config", JSONB, nullable=True)`。 -- Modify: `app/services/agent/session_cache.py` - - `write_session_started()` 接收并写入 `run_config`。 -- Modify: `app/services/stream/run_finalizer.py` - - `start_agent_run()` 把 `config` 同步传给 session cache。 -- Create: `app/services/agent/continuation.py` - - 封装 continuation 权限前置数据、最近触顶 run 查询、预算恢复、content block 反序列化和 continuation prompt 注入。 -- Modify: `app/schemas/chat.py` - - 新增 `ContinueAgentRunRequest`。 -- Modify: `app/services/chat_service.py` - - 新增 `continue_agent_run()`，复用 StreamHandler 启动 SSE。 -- Modify: `app/api/chat.py` - - 新增 `POST /api/chat/conversations/{conversation_id}/messages/{message_id}/continue`。 -- Modify: `app/services/stream/agent_loop_wiring.py` - - `AgentLoopRunInput` 增加已有 content blocks、额外 system prompts、是否预处理用户输入。 -- Modify: `app/services/stream/agent_loop_lifecycle.py` - - 生命周期 request 携带初始 content blocks 和额外 system prompts。 -- Modify: `app/services/stream/agent_loop_request_prep.py` - - 支持跳过 file/url 预处理并注入 continuation system prompt。 -- Modify: `app/services/stream/runner.py` - - `generate_to_redis()` 接收 continuation 参数并传入 wiring。 -- Test: `test/services/agent/test_continuation.py` -- Test: `test/services/stream/test_agent_loop_request_prep.py` -- Test: `test/test_chat_continue.py` -- Test: `test/test_stream_handler.py` - -### fusion-ui - -- Modify: `src/lib/api/chat.ts` - - 新增 `continueAgentRunStream()`，复用 SSE parser。 -- Modify: `src/redux/slices/streamSlice.ts` - - `startStream` 支持 `staticBlocks`，selectors 返回旧 blocks + 新增流式 blocks。 -- Create: `src/hooks/useContinueAgentRun.ts` - - 封装 continuation API 调用、SSE callbacks、消息更新和错误处理。 -- Modify: `src/components/chat/agent/RunBanner.tsx` - - `limit_reached` 三种 reason 显示“继续查”。 -- Modify: `src/components/chat/agent/AgentRunTimeline.tsx` - - 增加 `onContinue`，保留 `onRetry` 给失败/不完整路径。 -- Modify: `src/components/chat/AssistantResponseStack.tsx` - - 传递 `onContinueAgentRun`。 -- Modify: `src/components/chat/AssistantMessage.tsx` - - 接收并传递 continuation handler。 -- Modify: `src/components/chat/ChatMessage.tsx` - - 接收并传递 continuation handler。 -- Modify: `src/components/chat/ChatMessageList.tsx` - - 接收并传递 continuation handler。 -- Modify: `src/app/(app)/chat/[chatId]/page.tsx` - - 使用 `useContinueAgentRun()` 接线。 -- Test: `src/lib/api/chat.test.ts` -- Test: `src/redux/slices/streamSlice.test.ts` -- Test: `src/components/chat/agent/RunBanner.test.tsx` -- Test: `src/components/chat/AssistantResponseStack.test.tsx` -- Test: `src/hooks/useContinueAgentRun.test.ts` -- Test: `src/app/(app)/chat/[chatId]/page.test.tsx` - ---- - -## Task 1: 持久化 agent run config - -**Files:** -- Create: `fusion-api/alembic/versions/4c6a1f2b8d90_add_agent_session_config.py` -- Modify: `fusion-api/app/db/models.py` -- Modify: `fusion-api/app/services/agent/session_cache.py` -- Modify: `fusion-api/app/services/stream/run_finalizer.py` -- Test: `fusion-api/test/services/agent/test_session_cache.py` -- Test: `fusion-api/test/services/stream/test_agent_loop_lifecycle.py` - -- [ ] **Step 1: 写 migration** - -Create `fusion-api/alembic/versions/4c6a1f2b8d90_add_agent_session_config.py`: - -```python -"""add agent session config - -Revision ID: 4c6a1f2b8d90 -Revises: 3b4c8a7d2f10 -Create Date: 2026-06-28 -""" - -from typing import Sequence, Union - -import sqlalchemy as sa -from sqlalchemy.dialects import postgresql - -from alembic import op - -revision: str = "4c6a1f2b8d90" -down_revision: Union[str, Sequence[str], None] = "3b4c8a7d2f10" -branch_labels: Union[str, Sequence[str], None] = None -depends_on: Union[str, Sequence[str], None] = None - - -def upgrade() -> None: - op.add_column( - "agent_sessions", - sa.Column("config", postgresql.JSONB(astext_type=sa.Text()), nullable=True), - ) - - -def downgrade() -> None: - op.drop_column("agent_sessions", "config") -``` - -- [ ] **Step 2: 更新 ORM model** - -In `fusion-api/app/db/models.py`, inside `class AgentSession` after `provider`: - -```python - run_config = Column("config", JSONB, nullable=True) -``` - -- [ ] **Step 3: 更新 session cache 协议和实现** - -In `fusion-api/app/services/agent/session_cache.py`, change `write_session_started` signature: - -```python -async def write_session_started( - *, - run_id: str, - conversation_id: str, - user_id: str, - model_id: str, - provider: str, - message_id: str | None = None, - run_config: dict | None = None, -) -> None: -``` - -Set `existing.run_config = run_config` in the existing-row path, and pass `run_config=run_config` when creating `AgentSession`. - -- [ ] **Step 4: 更新 run finalizer protocol** - -In `fusion-api/app/services/stream/run_finalizer.py`, update `AgentRunSessionCache.write_session_started` protocol: - -```python - async def write_session_started( - self, - *, - run_id: str, - conversation_id: str, - user_id: str, - model_id: str, - provider: str, - message_id: str, - run_config: dict | None = None, - ) -> None: ... -``` - -In `start_agent_run()`, pass: - -```python - run_config=config, -``` - -- [ ] **Step 5: 写 session cache 测试** - -Create or extend `fusion-api/test/services/agent/test_session_cache.py` with a DB-free unit around the model mutation by monkeypatching `SessionLocal` to a fake context manager: - -```python -import pytest - -from app.db.models import AgentSession -from app.services.agent import session_cache - - -class FakeSession: - def __init__(self): - self.row = None - self.added = None - self.commits = 0 - - def __enter__(self): - return self - - def __exit__(self, exc_type, exc, tb): - return False - - def get(self, model, key): - assert model is AgentSession - return self.row - - def add(self, row): - self.added = row - - def commit(self): - self.commits += 1 - - -@pytest.mark.asyncio -async def test_write_session_started_persists_run_config(monkeypatch): - fake = FakeSession() - monkeypatch.setattr(session_cache, "SessionLocal", lambda: fake) - - await session_cache.write_session_started( - run_id="run-1", - conversation_id="conv-1", - user_id="user-1", - model_id="deepseek-chat", - provider="deepseek", - message_id="msg-1", - run_config={"max_steps": 8, "max_tool_calls": 20, "timeout_s": 300}, - ) - - assert fake.added.run_config == {"max_steps": 8, "max_tool_calls": 20, "timeout_s": 300} - assert fake.commits == 1 -``` - -- [ ] **Step 6: 运行后端 focused tests** - -Run: - -```bash -cd /Users/sean/code/fusion/fusion-api -.venv311/bin/python -m pytest test/services/agent/test_session_cache.py test/services/stream/test_agent_loop_lifecycle.py -q -``` - -Expected: tests pass after implementation. Before implementation, new test fails because `run_config` is not written. - ---- - -## Task 2: 后端 continuation 上下文服务 - -**Files:** -- Create: `fusion-api/app/services/agent/continuation.py` -- Test: `fusion-api/test/services/agent/test_continuation.py` - -- [ ] **Step 1: 写 continuation 单测** - -Create `fusion-api/test/services/agent/test_continuation.py`: - -```python -import pytest - -from app.schemas.chat import TextBlock -from app.services.agent.continuation import ( - CONTINUATION_SYSTEM_PROMPT, - deserialize_content_blocks, - inject_continuation_prompt, - resolve_continuation_limits, -) -from app.services.stream.agent_loop_policy import AgentLoopLimits - - -def test_deserialize_content_blocks_preserves_existing_block_id(): - blocks = deserialize_content_blocks([ - {"type": "text", "id": "blk_old", "text": "旧回答"}, - ]) - - assert blocks == [TextBlock(type="text", id="blk_old", text="旧回答")] - - -def test_inject_continuation_prompt_after_existing_system_messages(): - messages = [ - {"role": "system", "content": "用户自定义系统提示"}, - {"role": "user", "content": "问题"}, - {"role": "assistant", "content": "旧回答"}, - ] - - result = inject_continuation_prompt(messages) - - assert result[0]["content"] == "用户自定义系统提示" - assert result[1] == {"role": "system", "content": CONTINUATION_SYSTEM_PROMPT} - assert result[2]["role"] == "user" - - -def test_resolve_continuation_limits_uses_session_config(): - session = type("Session", (), { - "run_config": {"max_steps": 4, "max_tool_calls": 7, "timeout_s": 90}, - })() - - limits = resolve_continuation_limits( - session, - default_limits=AgentLoopLimits(max_steps=8, max_tool_calls=20, total_timeout_s=300), - ) - - assert limits == AgentLoopLimits(max_steps=4, max_tool_calls=7, total_timeout_s=90) - - -def test_resolve_continuation_limits_falls_back_to_default_for_missing_config(): - session = type("Session", (), {"run_config": None})() - default_limits = AgentLoopLimits(max_steps=8, max_tool_calls=20, total_timeout_s=300) - - assert resolve_continuation_limits(session, default_limits=default_limits) == default_limits -``` - -- [ ] **Step 2: 运行测试验证失败** - -Run: - -```bash -cd /Users/sean/code/fusion/fusion-api -.venv311/bin/python -m pytest test/services/agent/test_continuation.py -q -``` - -Expected: FAIL with `ModuleNotFoundError: No module named 'app.services.agent.continuation'`. - -- [ ] **Step 3: 实现 continuation helper** - -Create `fusion-api/app/services/agent/continuation.py`: - -```python -from __future__ import annotations - -from dataclasses import dataclass -from typing import Any - -from pydantic import TypeAdapter -from sqlalchemy.orm import Session - -from app.db.models import AgentSession, Message as MessageModel -from app.schemas.chat import ContentBlock -from app.schemas.response import ApiException -from app.services.stream.agent_loop_policy import AgentLoopLimits - -CONTINUATION_SYSTEM_PROMPT = ( - "你正在继续上一轮因运行上限而停止的回答。请基于已有对话、已有回答和已有工具结果继续补充，" - "不要重写或总结已完成的部分。若需要更多资料，可以继续调用可用工具。" - "输出应自然衔接在上一段回答之后。" -) - -_CONTENT_BLOCKS_ADAPTER = TypeAdapter(list[ContentBlock]) - - -@dataclass(frozen=True) -class AgentContinuationContext: - assistant_message: MessageModel - previous_session: AgentSession - limits: AgentLoopLimits - initial_content_blocks: list[ContentBlock] - - -def deserialize_content_blocks(raw_blocks: list[dict[str, Any]] | None) -> list[ContentBlock]: - return _CONTENT_BLOCKS_ADAPTER.validate_python(raw_blocks or []) - - -def inject_continuation_prompt(messages: list[dict]) -> list[dict]: - insert_at = 0 - while insert_at < len(messages) and messages[insert_at].get("role") == "system": - insert_at += 1 - prompt = {"role": "system", "content": CONTINUATION_SYSTEM_PROMPT} - return [*messages[:insert_at], prompt, *messages[insert_at:]] - - -def resolve_continuation_limits(session: AgentSession, *, default_limits: AgentLoopLimits) -> AgentLoopLimits: - config = session.run_config if isinstance(session.run_config, dict) else {} - try: - return AgentLoopLimits( - max_steps=int(config.get("max_steps", default_limits.max_steps)), - max_tool_calls=int(config.get("max_tool_calls", default_limits.max_tool_calls)), - total_timeout_s=float(config.get("timeout_s", default_limits.total_timeout_s)), - ) - except (TypeError, ValueError): - return default_limits - - -def find_latest_limit_reached_session( - db: Session, - *, - conversation_id: str, - message_id: str, - previous_run_id: str | None = None, -) -> AgentSession: - query = db.query(AgentSession).filter( - AgentSession.conversation_id == conversation_id, - AgentSession.message_id == message_id, - AgentSession.status == "limit_reached", - ) - if previous_run_id: - query = query.filter(AgentSession.id == previous_run_id) - session = query.order_by(AgentSession.created_at.desc()).first() - if session is None: - raise ApiException.bad_request("这条回答当前不能继续执行") - return session - - -def build_continuation_context( - db: Session, - *, - conversation_id: str, - message_id: str, - previous_run_id: str | None, - default_limits: AgentLoopLimits, -) -> AgentContinuationContext: - assistant_message = db.query(MessageModel).filter( - MessageModel.id == message_id, - MessageModel.conversation_id == conversation_id, - MessageModel.role == "assistant", - ).first() - if assistant_message is None: - raise ApiException.not_found("会话消息不存在或无权访问") - - previous_session = find_latest_limit_reached_session( - db, - conversation_id=conversation_id, - message_id=message_id, - previous_run_id=previous_run_id, - ) - - return AgentContinuationContext( - assistant_message=assistant_message, - previous_session=previous_session, - limits=resolve_continuation_limits(previous_session, default_limits=default_limits), - initial_content_blocks=deserialize_content_blocks(assistant_message.content), - ) -``` - -- [ ] **Step 4: 运行 continuation tests** - -Run: - -```bash -cd /Users/sean/code/fusion/fusion-api -.venv311/bin/python -m pytest test/services/agent/test_continuation.py -q -``` - -Expected: PASS. - ---- - -## Task 3: agent loop 支持初始内容和 continuation prompt - -**Files:** -- Modify: `fusion-api/app/services/stream/agent_loop_wiring.py` -- Modify: `fusion-api/app/services/stream/agent_loop_lifecycle.py` -- Modify: `fusion-api/app/services/stream/agent_loop_request_prep.py` -- Modify: `fusion-api/app/services/stream/runner.py` -- Test: `fusion-api/test/services/stream/test_agent_loop_request_prep.py` -- Test: `fusion-api/test/services/stream/test_agent_loop_lifecycle.py` - -- [ ] **Step 1: 写 request prep 测试** - -Extend `fusion-api/test/services/stream/test_agent_loop_request_prep.py`: - -```python -async def test_prepare_agent_loop_messages_injects_extra_system_prompts_without_user_preprocess(): - async def fake_build_llm_messages(raw_messages, has_vision, file_repo, user_system_prompt): - return [{"role": "user", "content": "原问题"}, {"role": "assistant", "content": "旧回答"}] - - async def should_not_preprocess_url(*args, **kwargs): - raise AssertionError("continuation 不应重新跑 URL 预处理") - - result = await prepare_agent_loop_messages( - db=object(), - user_id="user-1", - raw_messages=[], - has_vision=False, - file_ids=["file-1"], - original_message="https://example.com", - call_config=AgentLoopCallConfig( - should_use_reasoning=False, - supports_function_calling=True, - call_kwargs={"tools": []}, - announced_tools=[], - ), - file_repo_factory=lambda db: object(), - load_user_system_prompt_fn=lambda db, user_id: None, - build_llm_messages_fn=fake_build_llm_messages, - preprocess_url_in_message_fn=should_not_preprocess_url, - preprocess_user_input=False, - extra_system_prompts=["继续执行，不要重写前文"], - ) - - assert result.initial_content_blocks == [] - assert result.messages[0] == {"role": "system", "content": "继续执行，不要重写前文"} - assert result.messages[1]["role"] == "user" -``` - -- [ ] **Step 2: 运行测试验证失败** - -Run: - -```bash -cd /Users/sean/code/fusion/fusion-api -.venv311/bin/python -m pytest test/services/stream/test_agent_loop_request_prep.py::test_prepare_agent_loop_messages_injects_extra_system_prompts_without_user_preprocess -q -``` - -Expected: FAIL because `prepare_agent_loop_messages()` does not accept `preprocess_user_input` or `extra_system_prompts`. - -- [ ] **Step 3: 扩展 request prep** - -In `fusion-api/app/services/stream/agent_loop_request_prep.py`, add parameters to `prepare_agent_loop_messages()`: - -```python - preprocess_user_input: bool = True, - extra_system_prompts: list[str] | None = None, -``` - -After `build_llm_messages_fn(...)`, insert prompts: - -```python - messages = inject_extra_system_prompts(messages, extra_system_prompts or []) -``` - -Wrap file/url preprocess: - -```python - if preprocess_user_input: - messages = _inject_non_image_file_contents(...) - messages, initial_content_blocks = await _prepare_url_context(...) - else: - initial_content_blocks = [] -``` - -Add helper: - -```python -def inject_extra_system_prompts(messages: list[dict], prompts: list[str]) -> list[dict]: - if not prompts: - return messages - insert_at = 0 - while insert_at < len(messages) and messages[insert_at].get("role") == "system": - insert_at += 1 - prompt_messages = [{"role": "system", "content": prompt} for prompt in prompts] - return [*messages[:insert_at], *prompt_messages, *messages[insert_at:]] -``` - -- [ ] **Step 4: 扩展 wiring request dataclasses** - -In `fusion-api/app/services/stream/agent_loop_wiring.py`, add fields to `AgentLoopRunInput`: - -```python - initial_content_blocks: list | None = None - extra_system_prompts: list[str] | None = None - preprocess_user_input: bool = True -``` - -Add the same fields to `AgentLoopLifecycleRequest` construction: - -```python - initial_content_blocks=self.initial_content_blocks or [], - extra_system_prompts=self.extra_system_prompts or [], - preprocess_user_input=self.preprocess_user_input, -``` - -- [ ] **Step 5: 扩展 lifecycle request** - -In `fusion-api/app/services/stream/agent_loop_lifecycle.py`, add fields: - -```python - initial_content_blocks: list - extra_system_prompts: list[str] - preprocess_user_input: bool -``` - -In `_run_success_path()`, replace: - -```python - execution.state.content_blocks.extend(prepared_messages.initial_content_blocks) -``` - -with: - -```python - execution.state.content_blocks.extend(request.initial_content_blocks) - execution.state.content_blocks.extend(prepared_messages.initial_content_blocks) -``` - -In `_prepare_messages()`, pass: - -```python - extra_system_prompts=request.extra_system_prompts, - preprocess_user_input=request.preprocess_user_input, -``` - -- [ ] **Step 6: 扩展 StreamHandler 参数** - -In `fusion-api/app/services/stream/runner.py`, add optional parameters to `generate_to_redis()`: - -```python - initial_content_blocks: Optional[list] = None, - extra_system_prompts: Optional[list[str]] = None, - preprocess_user_input: bool = True, -``` - -Pass them into `AgentLoopRunInput`. - -- [ ] **Step 7: 运行 stream focused tests** - -Run: - -```bash -cd /Users/sean/code/fusion/fusion-api -.venv311/bin/python -m pytest test/services/stream/test_agent_loop_request_prep.py test/services/stream/test_agent_loop_lifecycle.py test/services/stream/test_agent_loop_wiring.py -q -``` - -Expected: PASS. - ---- - -## Task 4: 后端 continuation API - -**Files:** -- Modify: `fusion-api/app/schemas/chat.py` -- Modify: `fusion-api/app/services/chat_service.py` -- Modify: `fusion-api/app/api/chat.py` -- Test: `fusion-api/test/test_chat_continue.py` -- Test: `fusion-api/test/test_stream_handler.py` - -- [ ] **Step 1: 写 API/service 行为测试** - -Create `fusion-api/test/test_chat_continue.py`: - -```python -import pytest -from unittest.mock import AsyncMock, MagicMock, patch - -from app.schemas.response import ApiException -from app.services.chat_service import ChatService - - -@pytest.mark.asyncio -async def test_continue_agent_run_reuses_assistant_message_id(): - db = MagicMock() - service = ChatService(db) - conversation = type("Conversation", (), { - "id": "conv-1", - "user_id": "user-1", - "model_id": "deepseek-chat", - "messages": [], - })() - service.conversation_service.get_conversation = MagicMock(return_value=conversation) - - continuation_context = type("ContinuationContext", (), { - "initial_content_blocks": [], - "limits": type("Limits", (), { - "max_steps": 8, - "max_tool_calls": 20, - "total_timeout_s": 300, - })(), - })() - - with patch("app.services.chat_service.llm_manager.resolve_model", return_value=("deepseek/deepseek-chat", "deepseek", {})), \ - patch("app.services.chat_service.litellm_catalog.get_capabilities", return_value={"functionCalling": True}), \ - patch("app.services.chat_service.build_continuation_context", return_value=continuation_context), \ - patch("app.services.chat_service.init_stream", new=AsyncMock()) as init_stream_mock, \ - patch("app.services.chat_service.register_task") as register_task_mock, \ - patch("app.services.chat_service.asyncio.create_task") as create_task_mock: - create_task_mock.return_value = object() - - response = await service.continue_agent_run( - conversation_id="conv-1", - assistant_message_id="msg-1", - user_id="user-1", - previous_run_id="run-old", - trace_id="trace-1", - ) - - init_stream_mock.assert_awaited_once() - assert init_stream_mock.await_args.args[3] == "msg-1" - register_task_mock.assert_called_once() - assert response.media_type == "text/event-stream" - - -@pytest.mark.asyncio -async def test_continue_agent_run_rejects_missing_conversation(): - service = ChatService(MagicMock()) - service.conversation_service.get_conversation = MagicMock(return_value=None) - - with pytest.raises(ApiException): - await service.continue_agent_run( - conversation_id="missing", - assistant_message_id="msg-1", - user_id="user-1", - previous_run_id=None, - trace_id="trace-1", - ) -``` - -- [ ] **Step 2: 运行测试验证失败** - -Run: - -```bash -cd /Users/sean/code/fusion/fusion-api -.venv311/bin/python -m pytest test/test_chat_continue.py -q -``` - -Expected: FAIL because `ChatService.continue_agent_run` does not exist. - -- [ ] **Step 3: 新增 schema** - -In `fusion-api/app/schemas/chat.py`, add near `ChatRequest`: - -```python -class ContinueAgentRunRequest(BaseModel): - previous_run_id: Optional[str] = None - stream: bool = True -``` - -- [ ] **Step 4: 实现 ChatService.continue_agent_run** - -In `fusion-api/app/services/chat_service.py`, import: - -```python -from app.services.agent.continuation import ( - CONTINUATION_SYSTEM_PROMPT, - build_continuation_context, -) -from app.services.stream.runner import _agent_loop_limits -``` - -Add method: - -```python - async def continue_agent_run( - self, - *, - conversation_id: str, - assistant_message_id: str, - user_id: str, - previous_run_id: str | None = None, - trace_id: str | None = None, - ) -> StreamingResponse: - conversation = self.conversation_service.get_conversation(conversation_id, user_id) - if not conversation: - raise ApiException.not_found("会话不存在或无权访问") - - model_id = conversation.model_id - litellm_model, provider, litellm_kwargs = llm_manager.resolve_model(model_id) - capabilities = litellm_catalog.get_capabilities(model_id) - has_vision = capabilities.get("vision", False) - - continuation = build_continuation_context( - self.db, - conversation_id=conversation_id, - message_id=assistant_message_id, - previous_run_id=previous_run_id, - default_limits=_agent_loop_limits(), - ) - - task_id = str(uuid_mod.uuid4()) - await init_stream(conversation_id, str(user_id), model_id, assistant_message_id, task_id) - - task = asyncio.create_task( - self.stream_handler.generate_to_redis( - conversation_id=conversation_id, - user_id=user_id, - model_id=model_id, - litellm_model=litellm_model, - litellm_kwargs=litellm_kwargs, - provider=provider, - raw_messages=conversation.messages, - has_vision=has_vision, - file_ids=None, - original_message="", - assistant_message_id=assistant_message_id, - task_id=task_id, - options={}, - capabilities=capabilities, - trace_id=trace_id, - initial_content_blocks=continuation.initial_content_blocks, - extra_system_prompts=[CONTINUATION_SYSTEM_PROMPT], - preprocess_user_input=False, - limits=continuation.limits, - ) - ) - register_task(conversation_id, task, task_id) - - return StreamingResponse( - stream_redis_as_sse(conversation_id=conversation_id, message_id=assistant_message_id), - media_type="text/event-stream", - headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"}, - ) -``` - -This step also requires `StreamHandler.generate_to_redis()` to accept an optional `limits` override. Add: - -```python - limits: Optional[AgentLoopLimits] = None, -``` - -and pass `limits=limits or _agent_loop_limits()` into `build_agent_loop_lifecycle_call`. - -- [ ] **Step 5: 新增 route** - -In `fusion-api/app/api/chat.py`, import `ContinueAgentRunRequest` and add before message diagnostics route: - -```python -@router.post("/conversations/{conversation_id}/messages/{message_id}/continue") -async def continue_agent_run( - conversation_id: str, - message_id: str, - continue_request: ContinueAgentRunRequest, - request: Request, - chat_service: ChatService = Depends(get_chat_service), - current_user: User = Depends(get_current_user), -): - if not continue_request.stream: - raise ApiException.bad_request("continue 仅支持流式响应") - return await chat_service.continue_agent_run( - conversation_id=conversation_id, - assistant_message_id=message_id, - user_id=current_user.id, - previous_run_id=continue_request.previous_run_id, - trace_id=request.state.request_id, - ) -``` - -- [ ] **Step 6: 补并发 409** - -Before `init_stream`, call `get_stream_meta(conversation_id)` and if current status is `"streaming"`, raise: - -```python -raise ApiException.conflict("当前会话已有回答正在生成，请结束后再继续") -``` - -If `ApiException.conflict` does not exist, add it in `app/schemas/response.py` following existing factory style and test it in the focused API test. - -- [ ] **Step 7: 运行后端 API tests** - -Run: - -```bash -cd /Users/sean/code/fusion/fusion-api -.venv311/bin/python -m pytest test/test_chat_continue.py test/test_stream_handler.py -q -``` - -Expected: PASS. - ---- - -## Task 5: 前端 streamSlice 支持旧内容静态块 - -**Files:** -- Modify: `fusion-ui/src/redux/slices/streamSlice.ts` -- Test: `fusion-ui/src/redux/slices/streamSlice.test.ts` - -- [ ] **Step 1: 写 reducer/selector 测试** - -Extend `fusion-ui/src/redux/slices/streamSlice.test.ts`: - -```ts -import reducer, { - appendTextDelta, - selectFullStreamContentBlocks, - startStream, -} from './streamSlice'; - -it('keeps static blocks before continuation deltas', () => { - let state = reducer(undefined, startStream({ - conversationId: 'conv-1', - messageId: 'msg-1', - staticBlocks: [{ type: 'text', id: 'blk_old', text: '旧回答' }], - })); - - state = reducer(state, appendTextDelta({ - blockId: 'blk_new', - delta: '新补充', - runId: 'run-2', - stepId: 'step-1', - })); - - expect(selectFullStreamContentBlocks(state)).toEqual([ - { type: 'text', id: 'blk_old', text: '旧回答' }, - { type: 'text', id: 'blk_new', text: '新补充' }, - ]); -}); -``` - -- [ ] **Step 2: 运行测试验证失败** - -Run: - -```bash -cd /Users/sean/code/fusion/fusion-ui -npm test -- streamSlice.test.ts -``` - -Expected: FAIL because `startStream` does not accept `staticBlocks`. - -- [ ] **Step 3: 实现 staticBlocks** - -In `fusion-ui/src/redux/slices/streamSlice.ts`, add to `StreamState`: - -```ts - staticBlocks: ContentBlock[]; -``` - -Add to `initialState`: - -```ts - staticBlocks: [], -``` - -Change `startStream` payload: - -```ts -action: PayloadAction<{ conversationId: string; messageId: string; staticBlocks?: ContentBlock[] }> -``` - -Inside reducer: - -```ts -state.staticBlocks = action.payload.staticBlocks ?? []; -``` - -Update `selectStreamContentBlocks()` and `selectFullStreamContentBlocks()` to start with: - -```ts -const blocks: ContentBlock[] = [...state.staticBlocks]; -``` - -- [ ] **Step 4: 运行 streamSlice tests** - -Run: - -```bash -cd /Users/sean/code/fusion/fusion-ui -npm test -- streamSlice.test.ts -``` - -Expected: PASS. - ---- - -## Task 6: 前端 continuation API 和 hook - -**Files:** -- Modify: `fusion-ui/src/lib/api/chat.ts` -- Create: `fusion-ui/src/hooks/useContinueAgentRun.ts` -- Test: `fusion-ui/src/lib/api/chat.test.ts` -- Test: `fusion-ui/src/hooks/useContinueAgentRun.test.ts` - -- [ ] **Step 1: 写 API client 测试** - -Extend `fusion-ui/src/lib/api/chat.test.ts`: - -```ts -import { continueAgentRunStream } from './chat'; - -it('continueAgentRunStream posts to continuation endpoint', async () => { - const callbacks = buildCallbacks(); - fetchMock.mockResponseOnce(sse([ - agentEvent({ type: 'run_started', run_id: 'run-2', message_id: 'msg-1', conversation_id: 'conv-1', config: { max_steps: 8, max_tool_calls: 20, timeout_s: 300 } }), - done(), - ])); - - await continueAgentRunStream({ - conversationId: 'conv-1', - messageId: 'msg-1', - previousRunId: 'run-1', - }, callbacks); - - expect(fetchMock).toHaveBeenCalledWith( - expect.stringContaining('/api/chat/conversations/conv-1/messages/msg-1/continue'), - expect.objectContaining({ - method: 'POST', - body: JSON.stringify({ previous_run_id: 'run-1', stream: true }), - }), - ); -}); -``` - -Use existing test helpers in `chat.test.ts`; if helper names differ, add local helpers that emit valid SSE envelope strings. - -- [ ] **Step 2: 实现 API client** - -In `fusion-ui/src/lib/api/chat.ts`, add: - -```ts -export interface ContinueAgentRunRequest { - conversationId: string; - messageId: string; - previousRunId?: string; -} - -export async function continueAgentRunStream( - data: ContinueAgentRunRequest, - callbacks: StreamCallbacks, - signal?: AbortSignal, -): Promise { - const response = await fetchWithAuth( - `${API_BASE_URL}/api/chat/conversations/${encodeURIComponent(data.conversationId)}/messages/${encodeURIComponent(data.messageId)}/continue`, - { - method: 'POST', - headers: { 'Content-Type': 'application/json' }, - signal, - body: JSON.stringify({ - previous_run_id: data.previousRunId ?? null, - stream: true, - }), - }, - ); - - if (!response.ok) { - const errorData = await response.json().catch(() => ({})); - const body = errorData as { code?: string; message?: string; detail?: string }; - throw new Error(body.message || body.detail || '继续执行失败'); - } - if (!response.body) throw new Error('响应体为空'); - - const reader = response.body.getReader(); - await parseSseEnvelopeStream(reader, callbacks, { - fallbackConversationId: data.conversationId, - doneConversationId: () => data.conversationId, - }); -} -``` - -- [ ] **Step 3: 写 hook 测试** - -Create `fusion-ui/src/hooks/useContinueAgentRun.test.ts` with a minimal renderHook test: - -```ts -import { renderHook, act } from '@testing-library/react'; -import { vi, expect, it } from 'vitest'; -import { useContinueAgentRun } from './useContinueAgentRun'; - -vi.mock('@/lib/api/chat', () => ({ - continueAgentRunStream: vi.fn(), - getConversation: vi.fn(), -})); - -it('starts continuation stream with existing assistant content', async () => { - const dispatch = vi.fn(); - const store = { - getState: () => ({ - conversation: { - byId: { - 'conv-1': { - id: 'conv-1', - messages: [ - { id: 'msg-1', role: 'assistant', content: [{ type: 'text', id: 'blk_old', text: '旧回答' }] }, - ], - }, - }, - }, - stream: { - currentRun: null, - staticBlocks: [{ type: 'text', id: 'blk_old', text: '旧回答' }], - textBlocks: {}, - thinkingBlocks: {}, - blockOrder: [], - blockTypes: {}, - }, - }), - }; - - const { continueAgentRunStream } = await import('@/lib/api/chat'); - vi.mocked(continueAgentRunStream).mockImplementation(async (_payload, callbacks) => { - callbacks.onReady({ messageId: 'msg-1', conversationId: 'conv-1' }); - callbacks.onDone({ messageId: 'msg-1', conversationId: 'conv-1' }); - }); - - const { result } = renderHook(() => useContinueAgentRun({ dispatch, store: store as never })); - - await act(async () => { - await result.current.continueAgentRun({ - conversationId: 'conv-1', - assistantMessageId: 'msg-1', - previousRunId: 'run-1', - }); - }); - - expect(dispatch).toHaveBeenCalledWith(expect.objectContaining({ - type: 'stream/startStream', - payload: expect.objectContaining({ - conversationId: 'conv-1', - messageId: 'msg-1', - staticBlocks: [{ type: 'text', id: 'blk_old', text: '旧回答' }], - }), - })); -}); -``` - -- [ ] **Step 4: 实现 hook** - -Create `fusion-ui/src/hooks/useContinueAgentRun.ts`. Keep it focused: - -```ts -import { useCallback, useRef } from 'react'; -import { useStore } from 'react-redux'; -import { useAppDispatch } from '@/redux/hooks'; -import { - appendTextDelta, - appendThinkingDelta, - completeThinkingPhase, - endStream, - finalizeRun, - finalizeStep, - finalizeToolCall, - initRun, - markLimitReached, - pushStep, - pushToolCall, - selectFullStreamContentBlocks, - setStreamError, - startStream, -} from '@/redux/slices/streamSlice'; -import { updateMessage } from '@/redux/slices/conversationSlice'; -import { continueAgentRunStream, getConversation } from '@/lib/api/chat'; -import { getRunStatusFromFinishReason } from '@/lib/agent/finishReason'; -import type { FinalizeToolCallStatus, LimitReachedReason, ToolCallResultSummary } from '@/types/agentRun'; -import type { Conversation } from '@/types/conversation'; - -interface ContinueAgentRunInput { - conversationId: string; - assistantMessageId: string; - previousRunId?: string; -} - -interface HookDeps { - dispatch?: ReturnType; - store?: ReturnType; -} - -export function useContinueAgentRun(deps: HookDeps = {}) { - const realDispatch = useAppDispatch(); - const realStore = useStore(); - const dispatch = deps.dispatch ?? realDispatch; - const store = deps.store ?? realStore; - const abortControllerRef = useRef(null); - - const continueAgentRun = useCallback(async ({ - conversationId, - assistantMessageId, - previousRunId, - }: ContinueAgentRunInput) => { - if (abortControllerRef.current) { - abortControllerRef.current.abort(); - } - - const state = store.getState() as { conversation: { byId: Record }, stream: any }; - const conversation = state.conversation.byId[conversationId]; - const assistantMessage = conversation?.messages.find(m => m.id === assistantMessageId); - const staticBlocks = assistantMessage?.content ?? []; - - const controller = new AbortController(); - abortControllerRef.current = controller; - dispatch(startStream({ conversationId, messageId: assistantMessageId, staticBlocks })); - - try { - await continueAgentRunStream({ conversationId, messageId: assistantMessageId, previousRunId }, { - onReady: () => {}, - onReasoning: payload => dispatch(appendThinkingDelta({ blockId: payload.block_id, delta: payload.delta, runId: payload.run_id, stepId: payload.step_id })), - onAnswering: payload => { - const streamState = (store.getState() as { stream: any }).stream; - if (streamState.isStreamingReasoning) dispatch(completeThinkingPhase()); - dispatch(appendTextDelta({ blockId: payload.block_id, delta: payload.delta, runId: payload.run_id, stepId: payload.step_id })); - }, - onRunStarted: ev => dispatch(initRun({ - runId: ev.run_id, - messageId: assistantMessageId, - serverMessageId: ev.message_id, - config: { - maxSteps: (ev.config.max_steps as number) ?? 0, - maxToolCalls: (ev.config.max_tool_calls as number) ?? 0, - timeoutS: (ev.config.timeout_s as number) ?? 0, - }, - sequence: ev.sequence, - })), - onStepStarted: ev => { - if (ev.step_id) dispatch(pushStep({ runId: ev.run_id, stepId: ev.step_id, stepNumber: ev.step_number, sequence: ev.sequence })); - }, - onToolCallStarted: ev => { - if (ev.step_id && ev.tool_call_id) dispatch(pushToolCall({ runId: ev.run_id, stepId: ev.step_id, toolCallId: ev.tool_call_id, toolName: ev.tool_name, arguments: ev.arguments, sequence: ev.sequence })); - }, - onToolCallDelta: () => {}, - onToolCallCompleted: ev => { - if (ev.tool_call_id) dispatch(finalizeToolCall({ runId: ev.run_id, toolCallId: ev.tool_call_id, status: ev.status as FinalizeToolCallStatus, durationMs: ev.duration_ms, resultSummary: ev.result_summary as unknown as ToolCallResultSummary | undefined, error: ev.error ?? null, sequence: ev.sequence })); - }, - onStepCompleted: ev => { - if (ev.step_id) dispatch(finalizeStep({ runId: ev.run_id, stepId: ev.step_id, sequence: ev.sequence })); - }, - onRunLimitReached: ev => dispatch(markLimitReached({ runId: ev.run_id, reason: ev.reason as LimitReachedReason, sequence: ev.sequence })), - onRunInterrupted: ev => dispatch(finalizeRun({ runId: ev.run_id, status: 'interrupted', reason: ev.reason, sequence: ev.sequence })), - onRunFailed: ev => dispatch(finalizeRun({ runId: ev.run_id, status: 'failed', failure: { code: ev.error_code, message: ev.message }, sequence: ev.sequence })), - onRunCompleted: ev => dispatch(finalizeRun({ runId: ev.run_id, status: getRunStatusFromFinishReason(ev.finish_reason), sequence: ev.sequence })), - onDone: async () => { - const streamState = (store.getState() as { stream: any }).stream; - const finalBlocks = selectFullStreamContentBlocks(streamState); - dispatch(updateMessage({ conversationId, messageId: assistantMessageId, patch: { content: finalBlocks } })); - dispatch(endStream()); - if ((streamState.currentRun?.totalToolCalls ?? 0) > 0) { - const refreshed = await getConversation(conversationId) as Conversation; - const dbMessage = refreshed.messages.find(m => m.id === assistantMessageId); - if (dbMessage) { - dispatch(updateMessage({ conversationId, messageId: assistantMessageId, patch: { content: dbMessage.content, model_id: dbMessage.model_id, usage: dbMessage.usage } })); - } - } - }, - onError: (message, payload) => dispatch(setStreamError({ message, code: payload?.code, data: payload?.data })), - }, controller.signal); - } catch (error) { - if (!controller.signal.aborted) { - dispatch(setStreamError({ message: error instanceof Error ? error.message : '继续执行失败' })); - dispatch(endStream()); - } - } finally { - abortControllerRef.current = null; - } - }, [dispatch, store]); - - return { continueAgentRun }; -} -``` - -After the hook is green, compare the duplicated SSE event handling with `useSendMessage`. If both files share more than 80 lines of identical callback dispatching, extract only the shared event-dispatch body into `src/hooks/useAgentStreamEvents.ts` in the same task. - -- [ ] **Step 5: 运行 hook/API tests** - -Run: - -```bash -cd /Users/sean/code/fusion/fusion-ui -npm test -- chat.test.ts useContinueAgentRun.test.ts -``` - -Expected: PASS. - ---- - -## Task 7: 前端 CTA 接线 - -**Files:** -- Modify: `fusion-ui/src/components/chat/agent/RunBanner.tsx` -- Modify: `fusion-ui/src/components/chat/agent/AgentRunTimeline.tsx` -- Modify: `fusion-ui/src/components/chat/AssistantResponseStack.tsx` -- Modify: `fusion-ui/src/components/chat/AssistantMessage.tsx` -- Modify: `fusion-ui/src/components/chat/ChatMessage.tsx` -- Modify: `fusion-ui/src/components/chat/ChatMessageList.tsx` -- Modify: `fusion-ui/src/app/(app)/chat/[chatId]/page.tsx` -- Test: `fusion-ui/src/components/chat/agent/RunBanner.test.tsx` -- Test: `fusion-ui/src/components/chat/AssistantResponseStack.test.tsx` -- Test: `fusion-ui/src/app/(app)/chat/[chatId]/page.test.tsx` - -- [ ] **Step 1: 写 RunBanner 测试** - -Extend `fusion-ui/src/components/chat/agent/RunBanner.test.tsx`: - -```tsx -it.each(['max_steps', 'max_tool_calls', 'timeout'] as const)( - 'limit_reached + %s 显示继续查按钮', - reason => { - const onContinue = vi.fn(); - render(); - - fireEvent.click(screen.getByRole('button', { name: '继续查' })); - - expect(onContinue).toHaveBeenCalledTimes(1); - }, -); -``` - -- [ ] **Step 2: 实现 RunBanner props** - -In `RunBanner.tsx`, change props: - -```ts -interface RunBannerProps { - run: AgentRunState; - onRetry?: () => void; - onContinue?: () => void; -} -``` - -For `limit_reached`, render: - -```tsx -{onContinue && ( - -)} -``` - -Keep `failed` and `incomplete` using `onRetry`. - -- [ ] **Step 3: 逐层传递 onContinueAgentRun** - -Use this prop shape through the chat stack: - -```ts -onContinueAgentRun?: (messageId: string, previousRunId?: string) => void; -``` - -At `AgentRunTimeline`, convert to: - -```tsx - onContinue(run.runId) : undefined} -/> -``` - -At `AssistantMessageFrame`, bind: - -```ts -const handleContinue = useMemo( - () => onContinueAgentRun ? (previousRunId?: string) => onContinueAgentRun(message.id, previousRunId) : undefined, - [message.id, onContinueAgentRun], -); -``` - -Pass `handleContinue` into `AssistantResponseStack`. - -- [ ] **Step 4: 接入 ChatPage** - -In `fusion-ui/src/app/(app)/chat/[chatId]/page.tsx`, import and use: - -```ts -const { continueAgentRun } = useContinueAgentRun(); -``` - -Add handler: - -```ts -const handleContinueAgentRun = useCallback((messageId: string, previousRunId?: string) => { - if (!chatId || isStreaming) return; - void continueAgentRun({ - conversationId: chatId, - assistantMessageId: messageId, - previousRunId, - }); -}, [chatId, continueAgentRun, isStreaming]); -``` - -Pass to `ChatMessageList`: - -```tsx -onContinueAgentRun={handleContinueAgentRun} -``` - -- [ ] **Step 5: 运行 UI focused tests** - -Run: - -```bash -cd /Users/sean/code/fusion/fusion-ui -npm test -- RunBanner.test.tsx AssistantResponseStack.test.tsx \"page.test.tsx\" -``` - -Expected: PASS. - ---- - -## Task 8: 集成验证、CI/CD 和真实 Chrome 回归 - -**Files:** -- Modify as produced by Tasks 1-7. - -- [ ] **Step 1: 后端 focused suite** - -Run: - -```bash -cd /Users/sean/code/fusion/fusion-api -.venv311/bin/python -m pytest \ - test/services/agent/test_continuation.py \ - test/services/agent/test_session_cache.py \ - test/services/stream/test_agent_loop_request_prep.py \ - test/services/stream/test_agent_loop_lifecycle.py \ - test/test_chat_continue.py \ - test/test_stream_handler.py \ - -q -``` - -Expected: PASS. - -- [ ] **Step 2: 后端 full suite and lint** - -Run: - -```bash -cd /Users/sean/code/fusion/fusion-api -.venv311/bin/python -m pytest -q -/opt/homebrew/bin/ruff check . -/opt/homebrew/bin/ruff format --check app test -/opt/homebrew/bin/python3.11 scripts/check_architecture.py -/opt/homebrew/bin/python3.11 scripts/check_quality.py -git diff --check -``` - -Expected: pytest PASS; ruff PASS; architecture check PASS; quality check has no new stream redlines. - -- [ ] **Step 3: 前端 focused suite** - -Run: - -```bash -cd /Users/sean/code/fusion/fusion-ui -npm test -- \ - chat.test.ts \ - streamSlice.test.ts \ - useContinueAgentRun.test.ts \ - RunBanner.test.tsx \ - AssistantResponseStack.test.tsx \ - page.test.tsx -``` - -Expected: PASS. - -- [ ] **Step 4: 前端 full validation** - -Run: - -```bash -cd /Users/sean/code/fusion/fusion-ui -npm test -npm run lint -npm run build -git diff --check -``` - -Expected: all commands pass. - -- [ ] **Step 5: 提交和 push** - -Use one final commit per repo, with Chinese messages and Co-Authored-By. For `fusion-api`, include the already-created spec commit in branch history and create the implementation commit after code passes: - -```bash -cd /Users/sean/code/fusion/fusion-api -git status --short -git add app alembic test docs/superpowers/plans/2026-06-28-agent-run-continuation.md -git commit -m "feat: 支持 agent run 继续执行" -m "Co-Authored-By: Codex " -git push origin master -``` - -For `fusion-ui`: - -```bash -cd /Users/sean/code/fusion/fusion-ui -git status --short -git add src -git commit -m "feat: 接入 agent run 继续执行" -m "Co-Authored-By: Codex " -git push origin master -``` - -- [ ] **Step 6: 监控 CI/CD** - -Run: - -```bash -gh run list --repo HyxiaoGe/fusion-api --branch master --limit 5 -gh run list --repo HyxiaoGe/fusion-ui --branch master --limit 5 -``` - -Then inspect the run IDs for the pushed commits: - -```bash -gh run view --repo HyxiaoGe/fusion-api --log-failed -gh run view --repo HyxiaoGe/fusion-ui --log-failed -``` - -Expected: both workflows finish success. If either fails, inspect logs, fix, rerun focused tests, and push one correction commit in the affected repo. - -- [ ] **Step 7: 远端 dev 健康验证** - -After CI deploy succeeds, run read-only checks: - -```bash -ssh dev 'docker inspect fusion-api --format "image={{.Config.Image}} status={{.State.Status}} started={{.State.StartedAt}}"' -ssh dev 'docker inspect fusion-ui --format "image={{.Config.Image}} status={{.State.Status}} started={{.State.StartedAt}}"' -ssh dev 'curl -fsS http://127.0.0.1:8002/health' -``` - -Expected: API and UI containers use the pushed image tags; API health returns `{"status":"healthy",...}`. - -- [ ] **Step 8: 真实 Chrome 回归** - -Use existing logged-in Chrome tab only. Do not open deprecated dev domains. Use `https://fusion.seanfield.org`. - -Regression path: - -1. In an existing official Fusion tab, create or use a conversation with a prompt likely to hit a low continuation budget only if the deployed config/test path can safely trigger it. If production config cannot safely force a limit, use an existing triggered `limit_reached` conversation from dev logs. -2. Confirm `limit_reached` banner shows `继续查`. -3. Click `继续查`. -4. Confirm no user message or new assistant message is inserted. -5. Confirm original assistant message content grows. -6. Confirm Chrome console has no error/warning entries. -7. Confirm dev logs show two different run IDs with the same `message_id`, and the continuation run reaches `finish_reason=stop` or a second `limit_reached`. - -Log command: - -```bash -ssh dev 'docker logs --since=20m fusion-api 2>&1 | grep -E "AGENT_ROUND_SUMMARY|run_limit_reached|ERROR|Traceback|Exception" | tail -160' -``` - -Expected: no traceback/error for the regression conversation. - ---- - -## Self-Review - -- Spec coverage: continuation API, same assistant message append, original-budget continuation, no fake user message, errors, stop behavior, tests, CI/CD and Chrome regression are covered by Tasks 1-8. -- Data gap handled: `agent_sessions.config` is added because previous `run_started.config` was not durably stored. -- Type consistency: backend request uses `previous_run_id`; frontend request uses `previousRunId` and maps to snake case at the API boundary. -- Scope kept narrow: failed-run retry, single-tool retry, budget picker, historical multi-run timeline, and tool marketplace work are excluded. diff --git a/docs/superpowers/plans/2026-06-30-agent-loop-prompt-source-reading.md b/docs/superpowers/plans/2026-06-30-agent-loop-prompt-source-reading.md deleted file mode 100644 index 65f29e54..00000000 --- a/docs/superpowers/plans/2026-06-30-agent-loop-prompt-source-reading.md +++ /dev/null @@ -1,100 +0,0 @@ -# Agent Loop Prompt Source Reading Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** 调整 agent loop 的联网提示词，使搜索结果不再阻止后续 `url_read`，并让模型优先深读高价值来源。 - -**Architecture:** 先把 agent loop 相关提示词集中到 `app/ai/prompts/agent_loop.py`，再由 message builder、tool schema 和 tool handler 引用。当前仍保留代码内常量，后续迁移 PromptHub 时只替换该模块的提供方。 - -**Tech Stack:** FastAPI, LiteLLM function calling, Python unittest/pytest. - ---- - -### Task 1: 集中 Prompt 常量并锁定搜索上下文行为 - -**Files:** -- Create: `app/ai/prompts/agent_loop.py` -- Modify: `app/services/chat/message_builder.py` -- Modify: `app/services/stream/agent_loop_request_prep.py` -- Modify: `app/services/tool_handlers/web_search.py` -- Test: `test/test_tool_handlers.py` -- Test: `test/services/stream/test_agent_loop_request_prep.py` - -- [ ] **Step 1: Write failing tests** - -Add assertions that `WebSearchHandler.format_llm_context()` no longer contains `不要再发起搜索或输出任何工具调用指令`, and does contain guidance to call `url_read` for key facts, official announcements, and original details. Add assertions that `inject_tool_usage_contract()` still injects the centralized contract prompt. - -- [ ] **Step 2: Run tests to verify RED** - -Run: - -```bash -python -m pytest test/test_tool_handlers.py::WebSearchHandlerTests::test_format_llm_context_allows_targeted_url_read_after_search test/services/stream/test_agent_loop_request_prep.py::AgentLoopRequestPrepTests::test_tool_usage_contract_uses_centralized_prompt -q -``` - -Expected: tests fail because the current search context forbids follow-up tool calls and no centralized prompt module exists. - -- [ ] **Step 3: Implement minimal code** - -Create `app/ai/prompts/agent_loop.py` with constants for current-date prompt building, tool usage contract, search result context instructions, and source reading guidance. Import those constants/functions from the existing call sites. - -- [ ] **Step 4: Run tests to verify GREEN** - -Run the same targeted pytest command. Expected: both tests pass. - -### Task 2: 强化 `url_read` Tool 描述 - -**Files:** -- Modify: `app/ai/tools.py` -- Test: `test/test_ai_tools.py` - -- [ ] **Step 1: Write failing test** - -Add assertions that `URL_READ_TOOL` tells the model to prioritize official sources, original announcements, high-relevance pages, and down-rank video/forum/low-relevance results after search. - -- [ ] **Step 2: Run test to verify RED** - -Run: - -```bash -python -m pytest test/test_ai_tools.py::AiToolSchemaTests::test_url_read_schema_prioritizes_high_value_search_results -q -``` - -Expected: test fails because the current description only says “搜索结果中的某个链接需要深入阅读”. - -- [ ] **Step 3: Implement minimal code** - -Update `URL_READ_TOOL` description to include high-value source selection rules while preserving the existing URL/reason schema. - -- [ ] **Step 4: Run test to verify GREEN** - -Run the same targeted pytest command. Expected: test passes. - -### Task 3: 回归、提交、部署和真实对话验证 - -**Files:** -- No additional files expected. - -- [ ] **Step 1: Run backend regression** - -Run: - -```bash -python -m pytest test/test_ai_tools.py test/test_tool_handlers.py test/services/stream/test_agent_loop_request_prep.py -q -python -m ruff check app test -git diff --check -``` - -Expected: targeted tests, ruff, and whitespace check pass. - -- [ ] **Step 2: Commit and push** - -Commit with a structured Chinese commit message including `背景：`, `改动：`, `验证：`, and `Co-Authored-By: Codex `, then push `fusion-api` master. - -- [ ] **Step 3: Track CI/CD** - -Use `gh run view/list` to confirm build/test/image push/dev deploy success. Do not start local Fusion services. - -- [ ] **Step 4: Real conversation regression** - -After deploy, create or reuse a real deployed conversation without opening a new Chrome target. Verify a current-information query produces search results and, when useful, targeted `url_read` calls for high-value sources. Record input, conversation/message id, expected behavior, actual tool logs, and conclusion. diff --git a/docs/superpowers/plans/2026-06-30-dynamic-search-budget.md b/docs/superpowers/plans/2026-06-30-dynamic-search-budget.md deleted file mode 100644 index 78b55c8b..00000000 --- a/docs/superpowers/plans/2026-06-30-dynamic-search-budget.md +++ /dev/null @@ -1,78 +0,0 @@ -# 动态搜索预算 Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** 让联网搜索从机械的“多次搜索每次固定 5 条”变成基于意图、查询相似度和轮次的确定性预算策略。 - -**Architecture:** 工具描述负责减少模型发起重复搜索的概率；后端 `NetworkToolBudget` 负责最终兜底，统一推断 intent、忽略模型传入 count，并对相似 follow-up 搜索自动缩窄。`WebSearchHandler` 继续只消费规范化后的参数，不承担策略判断。 - -**Tech Stack:** FastAPI service layer, Python dataclass, pytest/unittest, ruff. - ---- - -### Task 1: 搜索预算策略测试 - -**Files:** -- Modify: `test/services/stream/test_network_budget.py` -- Create: `test/test_ai_tools.py` - -- [x] **Step 1: 写失败测试** - - 覆盖无 intent 但 query 明确是官方公告时自动推断 `official_source`。 - - 覆盖同一轮第二个相似官方查询自动缩窄到 3 条。 - - 覆盖官方查询后追加媒体/对照查询时不被误判为重复缩窄。 - - 覆盖工具描述明确要求默认一次搜索、避免中英文同义重复搜索。 - -- [x] **Step 2: 运行测试确认失败** - -```bash -.venv311/bin/python -m pytest test/services/stream/test_network_budget.py test/test_ai_tools.py -q -``` - -Expected: 新增测试失败，证明当前策略仍机械。 - -### Task 2: 动态搜索预算实现 - -**Files:** -- Modify: `app/services/search_budget.py` -- Modify: `app/services/stream/network_budget.py` -- Modify: `app/ai/tools.py` - -- [x] **Step 1: 实现 intent 推断** - - 显式合法 intent 优先。 - - 无 intent 时从 query 推断 `official_source` / `comparison` / `deep_research` / `freshness` / `quick_fact`。 - -- [x] **Step 2: 实现相似 follow-up 缩窄** - - `NetworkToolBudget` 记录本轮已消费的搜索 query 和 intent。 - - 只有“query 相似且 intent 相同”的 follow-up 才缩窄。 - - `standard/freshness/official_source` follow-up 缩到 3 条上下文 3 条。 - - `comparison/deep_research` follow-up 缩到 5 条，避免深度任务失去覆盖面。 - -- [x] **Step 3: 优化工具描述** - - 默认一次搜索。 - - 只有官方+媒体、对比、多来源核验等场景才发起第二个互补 query。 - - 禁止同一轮用中英文翻译/同义改写重复搜索同一意图。 - -### Task 3: 验证和收尾 - -**Files:** -- No extra files expected. - -- [x] **Step 1: 运行精确测试** - -```bash -.venv311/bin/python -m pytest test/services/stream/test_network_budget.py test/test_ai_tools.py test/test_tool_executor.py -q -``` - -- [x] **Step 2: 扩大后端回归** - -```bash -.venv311/bin/python -m pytest test/ -q -ruff check app test -ruff format --check app/services/search_budget.py app/services/stream/network_budget.py app/ai/tools.py test/services/stream/test_network_budget.py test/test_ai_tools.py -git diff --check -``` - -- [ ] **Step 3: 提交、push、CI/CD 和真实 Chrome 回归** - - 按中文结构化 commit message 提交。 - - push 后跟 GitHub Actions 和 dev 部署。 - - 部署完成后复用现有已登录 Chrome 标签，新建真实对话验证搜索次数/候选条数/深读来源/刷新恢复。 diff --git a/docs/superpowers/plans/2026-06-30-search-read-planner-ledger.md b/docs/superpowers/plans/2026-06-30-search-read-planner-ledger.md deleted file mode 100644 index 954a6f58..00000000 --- a/docs/superpowers/plans/2026-06-30-search-read-planner-ledger.md +++ /dev/null @@ -1,323 +0,0 @@ -# Search / Read Planner + Evidence Ledger Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** 建立 Search / Read Planner + Evidence Ledger 最小版，让搜索候选、推荐深读和网页读取状态使用统一 evidence 生命周期。 - -**Architecture:** 后端新增轻量 evidence ledger helper，生成稳定 URL evidence id，并让 `web_search`、`SourceCandidateRanker`、`url_read` upsert 同一条 evidence。前端只扩展 evidence status 类型和恢复逻辑，保持现有 UI 结构。 - -**Tech Stack:** FastAPI service layer, Pydantic event schema, Redux/React TypeScript, pytest, Vitest, ruff. - ---- - -### Task 1: 后端 Evidence Ledger Helper - -**Files:** -- Create: `app/services/source_evidence_ledger.py` -- Modify: `app/services/agent/progress_digest.py` -- Test: `test/services/agent/test_source_evidence_ledger.py` - -- [ ] **Step 1: Write failing backend tests** - -新增测试覆盖： - -```python -def test_search_and_url_read_share_stable_evidence_id(): - search_item = build_search_source_evidence_item(...) - read_item = build_url_read_evidence_item(...) - assert search_item["id"] == read_item["id"] -``` - -```python -def test_search_candidate_uses_candidate_status(): - item = build_search_source_evidence_item(...) - assert item["status"] == "candidate" -``` - -```python -def test_url_read_status_maps_to_read_lifecycle(): - assert success_item["status"] == "read_success" - assert degraded_item["status"] == "read_degraded" - assert failed_item["status"] == "read_failed" -``` - -Run: - -```bash -.venv311/bin/python -m pytest test/services/agent/test_source_evidence_ledger.py -q -``` - -Expected: fail because helper does not exist. - -- [ ] **Step 2: Implement helper** - -`source_evidence_ledger.py` must provide: - -```python -def canonicalize_evidence_url(url: str) -> str: - ... - -def stable_web_evidence_id(url: str, *, fallback: str) -> str: - ... - -def build_search_source_evidence_item(source, *, tool_call_id: str, source_index: int) -> dict: - ... - -def build_url_read_evidence_item(result_data: dict, *, status: str, tool_call_id: str) -> dict | None: - ... -``` - -Status mapping: - -```python -success -> read_success -degraded -> read_degraded -failed -> read_failed -``` - -- [ ] **Step 3: Wire `progress_digest.build_evidence_items`** - -Replace per-record evidence construction with helper: - -- `web_search` returns candidate items from sources. -- `url_read` returns one read lifecycle item when URL exists. -- `tool_result_digest.source_refs` uses stable evidence ids. - -- [ ] **Step 4: Verify task** - -Run: - -```bash -.venv311/bin/python -m pytest test/services/agent/test_source_evidence_ledger.py test/services/agent/test_progress_digest.py -q -``` - -Expected: pass. - -### Task 2: 后端 Source Selection Upsert - -**Files:** -- Modify: `app/services/stream/tool_round.py` -- Modify: `app/services/source_evidence_ledger.py` -- Test: `test/services/stream/test_tool_round.py` - -- [ ] **Step 1: Write failing tests** - -Add test: - -```python -async def test_tool_round_emits_selected_evidence_for_ranker_recommendations(): - ... - await handle_tool_calls_round(...) - emitter.evidence_item_upserted.assert_any_await( - tool_call_id="tc-search", - evidence=ANY, - ) - assert selected_event["evidence"]["status"] == "selected" -``` - -Run: - -```bash -.venv311/bin/python -m pytest test/services/stream/test_tool_round.py::ToolRoundTests::test_tool_round_emits_selected_evidence_for_ranker_recommendations -q -``` - -Expected: fail because selected evidence is not emitted. - -- [ ] **Step 2: Implement selected evidence builder** - -Add helper: - -```python -def build_selected_source_evidence_item(candidate: RankedSourceCandidate) -> dict: - ... -``` - -Claim format: - -```text -建议深读：{reason1} / {reason2} -``` - -- [ ] **Step 3: Emit selected evidence in tool round** - -In `handle_tool_calls_round`, after `execute_tool_round_tools` and before `append_tool_round_messages`, emit selected evidence events for successful search ranker recommendations. Event failure must not break tool main chain; log warning only. - -- [ ] **Step 4: Verify task** - -Run: - -```bash -.venv311/bin/python -m pytest test/services/stream/test_tool_round.py test/test_source_candidate_ranker.py -q -``` - -Expected: pass. - -### Task 3: Agent Event Schema And Snapshot Reducer - -**Files:** -- Modify: `app/services/agent/events.py` -- Modify: `app/services/agent/progress_state.py` -- Test: `test/services/agent/test_progress_state.py` -- Test: `test/services/agent/test_events.py` - -- [ ] **Step 1: Write failing tests** - -Add tests: - -```python -def test_progress_state_accepts_read_success_evidence(): - state = apply_progress_event(empty_progress_state(...), evidence_event("read_success")) - assert state["evidence"][0]["status"] == "read_success" -``` - -```python -def test_progress_state_caps_candidates_but_keeps_selected_and_read_success(): - ... - assert "selected-id" in kept_ids - assert "read-success-id" in kept_ids -``` - -Run: - -```bash -.venv311/bin/python -m pytest test/services/agent/test_progress_state.py test/services/agent/test_events.py -q -``` - -Expected: fail because new statuses are not accepted/prioritized. - -- [ ] **Step 2: Extend schema literals** - -`AgentEvidenceItem.status` accepts: - -```text -candidate | selected | read_success | read_degraded | read_failed | used | discarded -``` - -- [ ] **Step 3: Update cap priority** - -Keep priority: - -```text -used / used_by_final_answer > read_success > selected > read_degraded/read_failed > candidate/discarded -``` - -- [ ] **Step 4: Verify task** - -Run: - -```bash -.venv311/bin/python -m pytest test/services/agent/test_progress_state.py test/services/agent/test_events.py -q -``` - -Expected: pass. - -### Task 4: Frontend Status Compatibility - -**Files:** -- Modify: `../fusion-ui/src/types/agentRun.ts` -- Modify: `../fusion-ui/src/lib/agent/streamEventHandlers.ts` -- Modify: `../fusion-ui/src/lib/chat/conversationHydration.ts` -- Modify: `../fusion-ui/src/components/chat/agent/executionProcessModel.ts` -- Test: `../fusion-ui/src/redux/slices/streamSlice.test.ts` -- Test: `../fusion-ui/src/lib/chat/conversationHydration.test.ts` -- Test: `../fusion-ui/src/components/chat/agent/executionProcessModel.test.ts` - -- [ ] **Step 1: Write failing frontend tests** - -Add tests: - -```ts -it('接收 selected/read_success evidence status', () => ...) -``` - -```ts -it('历史恢复保留 read_success evidence status', () => ...) -``` - -```ts -it('execution process 把 selected/read_success evidence 当作可展示来源', () => ...) -``` - -Run: - -```bash -npm test -- --run src/redux/slices/streamSlice.test.ts src/lib/chat/conversationHydration.test.ts src/components/chat/agent/executionProcessModel.test.ts -``` - -Expected: fail because types/status assumptions not updated. - -- [ ] **Step 2: Extend TS status union** - -Update `AgentEvidenceItem.status`. - -- [ ] **Step 3: Keep render semantics** - -`isRenderableSearchEvidence` continues hiding only `discarded`; selected/read statuses are visible. - -- [ ] **Step 4: Verify task** - -Run: - -```bash -npm test -- --run src/redux/slices/streamSlice.test.ts src/lib/chat/conversationHydration.test.ts src/components/chat/agent/executionProcessModel.test.ts -``` - -Expected: pass. - -### Task 5: Full Validation, Commit, CI/CD, Real Regression - -**Files:** -- No new source files expected beyond previous tasks. - -- [ ] **Step 1: Backend validation** - -Run: - -```bash -cd /Users/sean/code/fusion/fusion-api -.venv311/bin/python -m pytest test/ -q -ruff check app test -ruff format --check app/services/source_evidence_ledger.py app/services/agent/progress_digest.py app/services/agent/progress_state.py app/services/agent/events.py app/services/stream/tool_round.py test/services/agent/test_source_evidence_ledger.py test/services/agent/test_progress_state.py test/services/stream/test_tool_round.py -git diff --check -``` - -- [ ] **Step 2: Frontend validation** - -Run: - -```bash -cd /Users/sean/code/fusion/fusion-ui -npm test -- --run src/redux/slices/streamSlice.test.ts src/lib/chat/conversationHydration.test.ts src/components/chat/agent/executionProcessModel.test.ts -npm run lint -npm run build -git diff --check -``` - -- [ ] **Step 3: Commit and push** - -Commit backend and frontend separately if both repos changed. Use structured Chinese commit messages with `背景：` / `改动：` / `验证：` and `Co-Authored-By: Codex `. - -- [ ] **Step 4: Monitor CI/CD** - -Use `gh run list`, `gh run watch --exit-status`, and dev container checks. Do not treat push as completion. - -- [ ] **Step 5: Real Chrome regression** - -Reuse the existing logged-in Chrome Fusion tab only. Test a new deployed conversation: - -```text -请联网检索 OpenAI 2026年6月 GPT-5.6 Sol 最新官方公告，并对照一条权威媒体报道。请优先深读官方公告、官方技术报告和高相关媒体原文，最后说明你选择深读这些来源的理由。 -``` - -Record: - -- case id -- input -- conversation URL -- expected -- actual UI execution/evidence summary -- backend tool logs -- evidence lifecycle snapshot -- console errors -- refresh result -- conclusion diff --git a/docs/superpowers/plans/2026-06-30-source-candidate-ranker.md b/docs/superpowers/plans/2026-06-30-source-candidate-ranker.md deleted file mode 100644 index 7c9d5f65..00000000 --- a/docs/superpowers/plans/2026-06-30-source-candidate-ranker.md +++ /dev/null @@ -1,155 +0,0 @@ -# Source Candidate Ranker Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** 将搜索候选来源选择从纯 prompt 约束升级为可测试的结构化排序建议，降低“搜了但不读高价值来源”的概率。 - -**Architecture:** 新增纯函数 ranker 对同一工具回合内的多个 `web_search` 结果做跨搜索合并、去重、排序和推荐；`tool_round` 在把工具结果注入下一轮 LLM 前追加一段结构化来源选择建议。后端不自动替 LLM 调用 `url_read`，避免破坏 agent loop 的工具决策权。 - -**Tech Stack:** FastAPI 后端、Pydantic 搜索来源模型、unittest/pytest、现有 agent loop tool round。 - ---- - -### Task 1: SourceCandidateRanker 纯函数 - -**Files:** -- Create: `app/services/source_candidate_ranker.py` -- Test: `test/test_source_candidate_ranker.py` - -- [x] **Step 1: Write failing tests** - -覆盖以下行为： -- 同一轮多个搜索结果按 canonical URL 去重。 -- 官方/原文公告/PDF/权威媒体优先。 -- 视频、社交、论坛来源默认降权。 -- 最多推荐 3 个深读候选。 - -- [x] **Step 2: Verify RED** - -Run: - -```bash -.venv311/bin/python -m pytest test/test_source_candidate_ranker.py -q -``` - -Expected: FAIL because `app.services.source_candidate_ranker` does not exist. - -- [x] **Step 3: Implement ranker** - -新增 `rank_search_sources(search_results, max_recommended=3)`，返回： -- `candidates`: 全量去重候选，带 `rank`、`priority`、`reasons` -- `recommended`: top N 高价值候选 - -评分规则保持启发式、可解释： -- 查询品牌与域名匹配、官方站点、文档/新闻室/公告、PDF/System Card 加分。 -- Reuters/AP/Axios/TechCrunch/CNBC 等权威媒体加分。 -- YouTube、Threads、X/Twitter、Facebook、Reddit、论坛/视频默认降权。 -- 标题/摘要与 query 词重合越高，相关性越高。 - -- [x] **Step 4: Verify GREEN** - -Run: - -```bash -.venv311/bin/python -m pytest test/test_source_candidate_ranker.py -q -``` - -Expected: PASS. - -### Task 2: Tool Round 跨搜索聚合注入 - -**Files:** -- Modify: `app/services/stream/tool_round.py` -- Test: `test/services/stream/test_tool_round.py` - -- [x] **Step 1: Write failing tests** - -新增测试：同一轮两个 `web_search` record 返回 10 条候选时，`append_tool_round_messages` 应只在第一个搜索 tool message 中追加结构化来源选择建议，并推荐 OpenAI 官方公告、Axios、System Card；第二个 tool message 保持原 handler 上下文，避免重复注入。 - -- [x] **Step 2: Verify RED** - -Run: - -```bash -.venv311/bin/python -m pytest test/services/stream/test_tool_round.py::ToolRoundTests::test_append_tool_round_messages_adds_round_level_source_selection_guidance -q -``` - -Expected: FAIL because no selection guidance is appended. - -- [x] **Step 3: Implement injection** - -在 `append_tool_round_messages` 中： -- 先构造 assistant tool-call message。 -- 对本轮成功的 `web_search` results 调用 ranker。 -- 第一个成功 `web_search` 的 tool message 追加 `format_source_selection_guidance(plan)`。 -- 其余 tool message 保持原工具上下文。 -- content block 不变，避免前端 UI schema 变更。 - -- [x] **Step 4: Verify GREEN** - -Run: - -```bash -.venv311/bin/python -m pytest test/services/stream/test_tool_round.py::ToolRoundTests::test_append_tool_round_messages_adds_round_level_source_selection_guidance -q -``` - -Expected: PASS. - -### Task 3: Existing Search Context Regression - -**Files:** -- Modify if needed: `app/services/tool_handlers/web_search.py` -- Test: `test/test_tool_handlers.py` - -- [x] **Step 1: Ensure existing prompt behavior remains** - -`web_search.format_llm_context` 仍保留： -- 搜索摘要足够可直接回答。 -- 关键事实/官方公告/原文细节需要少量高价值来源 `url_read`。 -- 不要求读满全部搜索结果。 - -- [x] **Step 2: Run related regressions** - -Run: - -```bash -.venv311/bin/python -m pytest test/test_source_candidate_ranker.py test/test_tool_handlers.py test/services/stream/test_tool_round.py -q -``` - -Expected: PASS. - -### Task 4: Full Verification, CI/CD, Real Regression - -**Files:** -- No additional code files expected. - -- [x] **Step 1: Run backend verification** - -Run: - -```bash -.venv311/bin/python -m pytest test/ -q -ruff check app test -ruff format --check app/services/source_candidate_ranker.py app/services/stream/tool_round.py test/test_source_candidate_ranker.py test/services/stream/test_tool_round.py -git diff --check -``` - -- [ ] **Step 2: Commit and push** - -Use structured Chinese commit message with `背景：`、`改动：`、`验证：` and `Co-Authored-By: Codex `. - -- [ ] **Step 3: CI/CD follow-through** - -Watch GitHub Actions until deployment succeeds. - -- [ ] **Step 4: Real Chrome regression** - -Use the existing logged-in Chrome tab only. Create a new Fusion conversation on the deployed site, ask for a search task with official + media cross-check, and record: -- case id -- input -- conversation URL -- expected -- actual -- console errors -- refresh result -- conclusion diff --git a/docs/superpowers/plans/2026-07-01-agent-tools-capability.md b/docs/superpowers/plans/2026-07-01-agent-tools-capability.md deleted file mode 100644 index e103d87a..00000000 --- a/docs/superpowers/plans/2026-07-01-agent-tools-capability.md +++ /dev/null @@ -1,49 +0,0 @@ -# Agent Tools Capability Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** 为 Fusion 模型目录和 agent loop 增加 `agentTools` 能力位，避免不适合工具路径的模型被默认当成联网 agent。 - -**Architecture:** 在 LiteLLM catalog 薄缓存层归一化 capabilities，`/api/models` 透出该字段，agent loop request prep 使用 `functionCalling && agentTools` 作为工具启用条件。eval runner 同步读取该字段，避免把非 agent 模型的 autonomous search 当作工具失败。 - -**Tech Stack:** FastAPI, LiteLLM metadata, Python unittest/pytest, Fusion stream agent loop. - ---- - -## Files - -- Modify: `app/ai/litellm_catalog.py` - - 增加 `normalize_capabilities()`。 - - 增加短期 agent tool denylist。 -- Modify: `app/api/models.py` - - `/api/models` 输出 `capabilities.agentTools`。 -- Modify: `app/services/stream/agent_loop_request_prep.py` - - 用 `functionCalling && agentTools` 判断是否下发工具。 -- Modify: `scripts/model_catalog_eval_baseline.py` - - 记录 `agent_tools_supported`。 - - 非 agent 模型不再被 expected tool 场景误判。 -- Test: `test/test_litellm_catalog.py` -- Test: `test/test_models.py` -- Test: `test/services/stream/test_agent_loop_request_prep.py` -- Test: `test/test_model_catalog_eval_baseline.py` - -## Test Matrix - -| Case | 输入 | 预期 | -| --- | --- | --- | -| ATC-01 | `functionCalling=true` 且无显式 `agentTools` | 默认 `agentTools=true` | -| ATC-02 | `qwen-vl-max` + `functionCalling=true` | 默认 `agentTools=false` | -| ATC-03 | metadata 显式 `agentTools=false` | 尊重显式 false | -| ATC-04 | `/api/models` card | 输出 `capabilities.agentTools` | -| ATC-05 | `agentTools=false` 的 call config | 不下发 `tools` 和 `tool_choice` | -| ATC-06 | eval autonomous search + `agentTools=false` | 无工具调用不算 mismatch | - -## Tasks - -- [x] 写 ATC-01 到 ATC-06 失败测试。 -- [x] 实现 capabilities 归一化和 API 输出。 -- [x] 实现 agent loop 工具启用条件收敛。 -- [x] 同步 eval runner 工具期望判定。 -- [ ] 跑目标测试、lint、全量后端测试和架构检查。 -- [ ] commit + push，监控 CI/CD 和 dev 部署。 -- [ ] 部署后验证 `/api/models` 与 qwen-vl autonomous search 行为。 diff --git a/docs/superpowers/plans/2026-07-01-autonomous-search-decision.md b/docs/superpowers/plans/2026-07-01-autonomous-search-decision.md deleted file mode 100644 index e22b993a..00000000 --- a/docs/superpowers/plans/2026-07-01-autonomous-search-decision.md +++ /dev/null @@ -1,30 +0,0 @@ -# 自主联网判断与执行过程修复计划 - -## 背景 - -本轮问题包含两个用户可见缺陷： - -1. 简单问候、稳定常识等不需要联网的问题，可能仍触发固定的搜索/读取计划。 -2. 已发生的工具过程展示不够精确：成功读取、跳过不可读网页、搜索关键词与 URL 文本边界容易混淆。 - -其中第 1 点是 P0：真实回归不能显式要求“联网搜索”，必须用自然问题验证模型是否自行判断。 - -## 用例矩阵 - -| 编号 | 场景 | 输入 | 预期 | -| --- | --- | --- | --- | -| P0-A | 自然问题应自主搜索 | `微信A2A互通怎么用？` | 模型主动调用 `web_search`，必要时深读高价值来源 | -| P0-B | 自然问题应自主搜索 | `OpenAI 最近发布了哪些产品更新？` | 模型主动调用 `web_search`，搜索关键词带当前年份或时间限定 | -| P0-C | 自然问题不应搜索 | `你好，你是谁？` | 不调用 `web_search`，不展示搜索/读取计划 | -| P0-D | 自然问题不应搜索 | `1+1等于几？` | 不调用 `web_search`，直接回答 | -| UI-A | 读取部分失败 | 1 个成功读取 + 1 个不可读网页 | 执行摘要展示成功读取和跳过数量，不暴露 `url_read` / `reader-service` | -| UI-B | 思考正文裸 URL | `https://example.com/a，原因是...` | 链接只包含 URL，中文说明保留为普通文本 | - -## 实施 - -1. 后端先补测试，锁住工具可用时注入“自主联网判断”规则，规则同时覆盖应搜索和不应搜索示例。 -2. 后端补强 `web_search` 描述，明确不依赖用户是否说“搜索/联网”，但问候、数学、翻译、代码、创意写作不强制搜索。 -3. 前端先补测试，锁住执行摘要的成功读取/跳过数量和 URL 边界。 -4. 前端修正执行过程模型和 Markdown 预处理。 -5. 运行后端/前端针对性测试，必要时再跑更大范围测试。 -6. 合并部署后，复用已登录 Chrome 现有标签做真实生产回归；记录新建对话 URL、输入、预期、实际、console error、刷新结果和结论。 diff --git a/docs/superpowers/plans/2026-07-01-evidence-ledger-used-source.md b/docs/superpowers/plans/2026-07-01-evidence-ledger-used-source.md deleted file mode 100644 index 7cae2895..00000000 --- a/docs/superpowers/plans/2026-07-01-evidence-ledger-used-source.md +++ /dev/null @@ -1,60 +0,0 @@ -# Evidence Ledger 来源使用判定增强 Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** 让 Fusion 能区分“最终回答真正使用的来源”和“搜索/深读候选来源”。 - -**Architecture:** 后端在最终文本回合完成时运行确定性 used-source matcher，通过现有 `evidence_item_upserted` upsert `status=used` evidence；前端优先用 agent evidence 的 used/candidate 状态派生回答依据模型，历史数据继续 fallback 到 content block source refs。 - -**Tech Stack:** FastAPI service layer, Pydantic event protocol, pytest, Next.js/React/Vitest, existing AgentRun evidence snapshot. - ---- - -## Task 1: Backend Final Answer Evidence Matcher - -**Files:** -- Create: `app/services/final_answer_evidence.py` -- Modify: `app/services/stream/agent_loop_round_outcome.py` -- Test: `test/services/stream/test_final_answer_evidence.py` -- Test: `test/services/stream/test_agent_loop_round_outcome.py` - -- [ ] Add failing tests for `[1]`, `⟦2⟧`, exact URL, unique domain, ambiguous domain, single-read fallback, and no-match multi-source. -- [ ] Implement `build_used_final_answer_evidence(content_blocks, answer_text)`. -- [ ] Call matcher after final text blocks are appended and before step/run completion events finish. -- [ ] Emit `evidence_item_upserted` with `status="used"` and `used_by_final_answer=true`. -- [ ] Verify targeted backend tests. - -## Task 2: Frontend Evidence Model Split - -**Files:** -- Modify: `src/components/chat/answerEvidenceModel.ts` -- Modify: `src/components/chat/useAssistantMessageViewModel.ts` -- Modify: `src/components/chat/answerEvidenceSidebarModel.ts` -- Modify: `src/components/chat/AnswerEvidenceSidebar.tsx` -- Test: `src/components/chat/answerEvidenceModel.test.ts` -- Test: `src/components/chat/answerEvidenceSidebarModel.test.ts` -- Test: `src/components/chat/useAssistantMessageViewModel.test.tsx` - -- [ ] Add failing tests showing run evidence with used and candidate sources. -- [ ] Add `usedItems` / `candidateItems` to `AnswerEvidenceModel`. -- [ ] Prefer `currentRun.evidence` when it contains renderable web evidence. -- [ ] Preserve fallback for older messages without evidence. -- [ ] Update sidebar labels to show “已使用来源” and “候选来源”. -- [ ] Verify targeted frontend tests. - -## Task 3: Verification And Rollout - -**Files:** -- Modify as needed based on integration findings. - -- [ ] Run backend targeted tests. -- [ ] Run backend full `pytest` and `ruff`. -- [ ] Run frontend targeted Vitest. -- [ ] Run frontend build or existing required validation for changed TSX/model files. -- [ ] Commit backend and frontend changes with structured Chinese messages. -- [ ] Push both repos and monitor CI/CD. -- [ ] Reuse existing Chrome `fusion.seanfield.org` tab for deployed regression: - - New realtime search answer with citations. - - Check answer evidence summary and sidebar used/candidate split. - - Refresh the conversation and confirm used/candidate state persists. - - Confirm console has no new errors. diff --git a/docs/superpowers/plans/2026-07-01-identity-and-agent-behavior-eval.md b/docs/superpowers/plans/2026-07-01-identity-and-agent-behavior-eval.md deleted file mode 100644 index d9be4273..00000000 --- a/docs/superpowers/plans/2026-07-01-identity-and-agent-behavior-eval.md +++ /dev/null @@ -1,84 +0,0 @@ -# 身份一致性热修与 Agent 行为 Eval 矩阵 Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** 修复模型自称错误身份的问题，并把关键 Agent 行为场景固化为可自动化评估矩阵。 - -**Architecture:** 身份一致性通过全局 system prompt 注入到所有用户对话的 LLM 消息中，避免只在工具模式生效。Agent 行为 eval 使用独立脚本和 fixture 描述样本、期望工具策略、期望 UI surface 与禁止泄露项，既可用于单元测试，也可承接真实 Chrome 回归记录。 - -**Tech Stack:** FastAPI 后端、LiteLLM 消息构造、Python unittest/pytest、JSON fixture。 - ---- - -### Task 1: 身份一致性约束 - -**Files:** -- Modify: `app/ai/prompts/agent_loop.py` -- Modify: `app/services/chat/message_builder.py` -- Create: `test/services/chat/test_message_builder.py` - -- [ ] **Step 1: Write failing tests** - - `build_llm_messages()` 必须在日期 prompt 后注入 Fusion 身份 prompt。 - - 用户自定义 system prompt 必须排在身份 prompt 后。 - - 身份 prompt 必须禁止自称 Claude / Anthropic / OpenAI / DeepSeek 等单一供应商身份。 - -- [ ] **Step 2: Run target tests and verify RED** - - Run: `.venv311/bin/python -m pytest test/services/chat/test_message_builder.py -q` - - Expected: fail because identity prompt is not injected yet. - -- [ ] **Step 3: Implement minimal prompt injection** - - Add `APP_IDENTITY_PROMPT` constant. - - Inject it in `build_llm_messages()` immediately after current-date prompt. - -- [ ] **Step 4: Run target tests and verify GREEN** - - Run: `.venv311/bin/python -m pytest test/services/chat/test_message_builder.py -q` - - Expected: pass. - -### Task 2: Agent 行为 Eval 矩阵 - -**Files:** -- Create: `scripts/agent_behavior_eval.py` -- Create: `test/fixtures/agent_behavior_eval_samples.json` -- Create: `test/test_agent_behavior_eval.py` - -- [ ] **Step 1: Write failing tests** - - loader 拒绝重复 id、缺少必填字段、非法策略。 - - 默认样本至少覆盖：身份问答、简单数学、实时产品功能、搜索失败降级、读取失败跳过、刷新恢复。 - - scorer 能判定 no_search 场景不应有工具调用/执行过程/回答依据。 - - scorer 能判定 search 场景必须有 web_search、搜索关键词、依据来源且不得泄露内部服务名。 - -- [ ] **Step 2: Run target tests and verify RED** - - Run: `.venv311/bin/python -m pytest test/test_agent_behavior_eval.py -q` - - Expected: fail because eval script and fixture do not exist yet. - -- [ ] **Step 3: Implement minimal eval script and fixture** - - Add sample loader and observation scorer. - - Keep dry-run JSONL output for CI/manual inspection. - - Do not call external services. - -- [ ] **Step 4: Run target tests and verify GREEN** - - Run: `.venv311/bin/python -m pytest test/test_agent_behavior_eval.py -q` - - Expected: pass. - -### Task 3: 回归验证与交付 - -**Files:** -- Modify only files above unless tests expose a necessary adjacent change. - -- [ ] **Step 1: Run focused tests** - - `.venv311/bin/python -m pytest test/services/chat/test_message_builder.py test/test_agent_behavior_eval.py test/services/stream/test_agent_loop_request_prep.py test/test_stream_handler.py -q` - -- [ ] **Step 2: Run full API verification** - - `.venv311/bin/python -m pytest -q` - - `.venv/bin/ruff check .` - -- [ ] **Step 3: Commit and push** - - Structured Chinese commit body with `背景：` / `改动：` and `Co-Authored-By: Codex `. - -- [ ] **Step 4: CI/CD and real Chrome regression** - - Watch GitHub Actions and dev deploy. - - Reuse existing user-opened `fusion.seanfield.org` Chrome tab only. - - New real conversations: - - `你好，你是谁？` must not claim Claude/Anthropic/OpenAI/DeepSeek identity. - - `1+1等于几？` must not show execution/evidence. - - `微信A2A互通怎么用？` must autonomously search and show evidence. diff --git a/docs/superpowers/plans/2026-07-01-model-catalog-audit-sync.md b/docs/superpowers/plans/2026-07-01-model-catalog-audit-sync.md deleted file mode 100644 index 92a0bf42..00000000 --- a/docs/superpowers/plans/2026-07-01-model-catalog-audit-sync.md +++ /dev/null @@ -1,81 +0,0 @@ -# Model Catalog Audit Sync Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** 增加模型目录巡检/同步 v1，输出 LiteLLM、Fusion `/api/models/`、Fusion virtual key allowlist 的一致性报告，并支持显式 apply 同步 allowlist。 - -**Architecture:** 新增独立脚本 `scripts/audit_litellm_model_catalog.py`，复用 LiteLLM 管理 API 和 Fusion 公开 `/api/models/`，把巡检规则、同步计划和 I/O 分开。v1 只允许同步 virtual key allowlist，不自动注册或删除模型；同步目标限定为 `db_model=true` 且关键 metadata 完整的业务别名，避免把工具/其他服务别名误加入 Fusion key。 - -**Tech Stack:** Python 3.11, httpx, unittest/pytest, LiteLLM Proxy 管理 API, Fusion `/api/models/`。 - ---- - -## Files - -- Create: `scripts/audit_litellm_model_catalog.py` - - 拉取 LiteLLM `/model/info`、Fusion `/api/models/`、LiteLLM `/key/info`。 - - 构建巡检报告和 allowlist 同步计划。 - - CLI 默认 dry-run，`--apply` 才调用 `/key/update`。 -- Create: `test/test_litellm_model_catalog_audit.py` - - 覆盖目录一致性、Fusion 多余模型、key 缺失、退役模型、metadata 缺失、序列化脱敏、apply 边界。 -- Add: `docs/superpowers/specs/2026-07-01-model-catalog-audit-sync-design.md` - - 记录 v1 设计边界和验收标准。 - -## Requirements - -- 默认 `--dry-run`，不写 LiteLLM。 -- `--apply` 只能同步 virtual key allowlist，不能调用 `/model/new` 或 `/model/delete`。 -- 报告包含 `summary`、`issues`、`sync_plan`。 -- 报告不能输出 master key、virtual key、provider API key。 -- `db_model=true` 是 Fusion 业务模型的候选来源，但只有 metadata 完整的别名才进入自动同步目标。 -- 重复 `db_model=true` 别名必须报警，summary 按唯一别名统计。 -- Fusion `/api/models/` 展示模型必须是 LiteLLM 业务模型子集。 -- virtual key allowlist 缺少 metadata 完整的业务模型时给 `error`，同步计划加入 `add`。 -- virtual key allowlist 含已知退役模型时给 `error`，同步计划加入 `remove`。 -- 业务模型缺 `provider_key`、`provider_display`、`capabilities` 或 `pricing` 时给 `warning`。 -- metadata 不完整且 Fusion 未展示的业务别名只给 `warning`，不加入 `sync_plan.add`。 - -## Test Matrix - -| Case | Test | -| --- | --- | -| AUDIT-01 | `test_clean_catalog_has_no_issues` | -| AUDIT-02 | `test_fusion_unknown_model_is_error` | -| AUDIT-03 | `test_key_missing_db_model_is_error_and_sync_adds_it` | -| AUDIT-04 | `test_deprecated_key_model_is_error_and_sync_removes_it` | -| AUDIT-05 | `test_missing_metadata_is_warning` | -| AUDIT-06 | `test_serialize_report_does_not_include_secrets` | -| AUDIT-07 | `test_apply_sync_only_updates_key_models` | -| AUDIT-08 | `test_duplicate_db_model_entries_are_warned_and_counted_once` | -| AUDIT-09 | `test_hidden_incomplete_db_model_is_warning_not_sync_target` | - -## Tasks - -### Task 1: 写失败测试 - -- [ ] 新建 `test/test_litellm_model_catalog_audit.py`，定义上述 7 个测试。 -- [ ] 运行 `.venv311/bin/python -m pytest test/test_litellm_model_catalog_audit.py -q`。 -- [ ] 预期失败：`scripts.audit_litellm_model_catalog` 不存在。 - -### Task 2: 实现巡检核心 - -- [ ] 新建 `scripts/audit_litellm_model_catalog.py`。 -- [ ] 定义 `CatalogIssue`、`SyncPlan`、`AuditReport` dataclass。 -- [ ] 实现 `extract_db_models()`、`extract_fusion_model_ids()`、`build_allowlist_sync_plan()`、`audit_catalog()`。 -- [ ] 跑目标测试，确认从 import failure 进入行为断言。 - -### Task 3: 实现 I/O 和 CLI - -- [ ] 实现 `fetch_litellm_models()`、`fetch_fusion_models()`、`fetch_key_models()`、`update_key_models()`。 -- [ ] 实现 CLI 参数：`--dry-run`、`--apply`、`--litellm-base-url`、`--fusion-base-url`、`--master-key`、`--virtual-key`。 -- [ ] `--apply` 无 `--virtual-key` 时抛明确错误。 -- [ ] 输出 JSON 使用 `serialize_report()`，不输出任何 secret。 - -### Task 4: 验证和发布 - -- [ ] 运行目标测试。 -- [ ] 运行全量 `pytest test/ -q`。 -- [ ] 运行 `ruff check .` 和 `ruff format --check`。 -- [ ] 运行 `scripts/check_architecture.py`。 -- [ ] 在生产配置上执行 dry-run，不打印密钥。 -- [ ] commit、push，监控 GitHub Actions 和 dev 部署。 diff --git a/docs/superpowers/plans/2026-07-01-model-catalog-governance-and-model-eval-baseline.md b/docs/superpowers/plans/2026-07-01-model-catalog-governance-and-model-eval-baseline.md deleted file mode 100644 index b2022198..00000000 --- a/docs/superpowers/plans/2026-07-01-model-catalog-governance-and-model-eval-baseline.md +++ /dev/null @@ -1,255 +0,0 @@ -# Model Catalog Governance And Model Eval Baseline Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** 修复 Fusion 可选模型目录与 LiteLLM 实际可调用模型不一致的问题，并建立多模型测验基线。 - -**Architecture:** Fusion 继续只展示 LiteLLM `/model/info` 中 `db_model=true` 的显式业务别名，不把 wildcard 路由直接暴露给前端。新增一个可重复运行的模型目录治理脚本，用声明式清单生成 LiteLLM `/model/new`、`/model/delete` 和 `/key/update` 操作，先替换已退役的小米 V2 模型并同步 Fusion virtual key 白名单，再用独立脚本对 `/api/models` 中健康模型执行统一 smoke/eval。 - -**Tech Stack:** FastAPI, LiteLLM Proxy 管理 API, Python 3.11, pytest/unittest, httpx, GitHub Actions/CI/CD, 真实生产 Fusion 回归。 - ---- - -## Files - -- Create: `scripts/govern_litellm_model_catalog.py` - 声明式治理小米模型目录和 Fusion virtual key 模型白名单，支持 `--dry-run` 和 `--apply`，默认只打印脱敏计划。 -- Create: `scripts/model_catalog_eval_baseline.py` - 对 Fusion `/api/models` 返回的可用模型执行统一非流式 smoke，输出 JSONL 基线。 -- Create: `test/test_litellm_model_catalog_governance.py` - 覆盖 deprecated 模型删除、新模型注册 payload、已有模型跳过、缺 key 报错。 -- Create: `test/test_model_catalog_eval_baseline.py` - 覆盖模型筛选、健康模型选择、结果 JSONL 结构和失败记录。 -- Modify: `docs/superpowers/plans/2026-07-01-model-catalog-governance-and-model-eval-baseline.md` - 记录实施计划和验收矩阵。 - -## Requirements - -- 不改变 `/api/models` 的产品语义：仍只展示 `db_model=true` 的显式业务别名。 -- 不暴露 `xiaomi/*` wildcard 到前端模型选择器。 -- 下线或删除旧小米 V2 别名：`mimo-v2-flash`, `mimo-v2-pro`。 -- 注册新小米文本模型：`mimo-v2.5-pro`, `mimo-v2.5-pro-ultraspeed`。 -- 新小米模型 metadata 必须包含 `provider_key=xiaomi`, `provider_display=小米 MiMo`, `source=fusion-governance`, `capabilities`, `pricing`, `knowledge_cutoff`, `recommended_for`。 -- 治理脚本默认 dry-run，不触碰生产 LiteLLM；只有显式 `--apply` 才写入。 -- 治理脚本 dry-run 不允许打印真实 API key。 -- 如果传入 `--virtual-key`，治理脚本必须把 key allowlist 中的旧小米模型替换成 V2.5 模型。 -- 多模型测验基线必须从 Fusion `/api/models` 拉当前可选模型，默认只测 health=healthy 的模型。 -- 基线脚本必须记录：model_id、provider、输入、是否成功、耗时、错误、回答摘要。 - -## Test Matrix - -| Case | 输入状态 | 预期 | -| --- | --- | --- | -| GOV-01 | LiteLLM 有旧小米模型 UUID | 生成 delete 动作 | -| GOV-02 | LiteLLM 缺新小米模型且有 `XIAOMI_API_KEY` | 生成 create 动作，payload 使用 `openai/` + 小米 api_base | -| GOV-03 | LiteLLM 已存在新模型 | 不重复注册 | -| GOV-04 | 需要注册新模型但缺 `XIAOMI_API_KEY` | 抛出明确错误 | -| GOV-05 | 旧模型没有 UUID | 不执行 delete，记录 skip | -| GOV-06 | dry-run 输出 create payload | `api_key` 必须脱敏为 `***` | -| GOV-07 | Fusion virtual key allowlist 含旧小米 | 删除旧小米，补入两个 V2.5 模型 | -| EVAL-01 | `/api/models` 有 healthy/unhealthy 混合模型 | 默认只选 healthy | -| EVAL-02 | 模型调用成功 | 输出 success JSONL，含耗时和回答摘要 | -| EVAL-03 | 模型调用失败 | 输出 failure JSONL，含错误类型和错误消息 | - -## Task 1: 写治理脚本测试 - -- [ ] **Step 1: Add failing tests** - -Create `test/test_litellm_model_catalog_governance.py` with tests for: - -```python -import unittest - -from scripts import govern_litellm_model_catalog as catalog - - -class ModelCatalogGovernanceTests(unittest.TestCase): - def test_plan_deletes_deprecated_xiaomi_models(self): - entries = [ - { - "model_name": "mimo-v2-pro", - "model_info": { - "id": "uuid-old-pro", - "metadata": {"provider_key": "xiaomi", "source": "fusion-migration"}, - }, - } - ] - - plan = catalog.build_governance_plan(entries, {"XIAOMI_API_KEY": "sk-xiaomi"}) - - delete_actions = [a for a in plan.actions if a.action == "delete"] - self.assertEqual(len(delete_actions), 1) - self.assertEqual(delete_actions[0].model_name, "mimo-v2-pro") - self.assertEqual(delete_actions[0].model_uuid, "uuid-old-pro") - - def test_plan_registers_missing_xiaomi_v25_models(self): - plan = catalog.build_governance_plan([], {"XIAOMI_API_KEY": "sk-xiaomi"}) - - create_actions = [a for a in plan.actions if a.action == "create"] - self.assertEqual([a.model_name for a in create_actions], ["mimo-v2.5-pro", "mimo-v2.5-pro-ultraspeed"]) - payload = create_actions[0].payload - self.assertEqual(payload["model_name"], "mimo-v2.5-pro") - self.assertEqual(payload["litellm_params"]["model"], "openai/mimo-v2.5-pro") - self.assertEqual(payload["litellm_params"]["api_base"], "https://api.xiaomimimo.com/v1") - self.assertEqual(payload["litellm_params"]["api_key"], "sk-xiaomi") - self.assertEqual(payload["model_info"]["metadata"]["provider_key"], "xiaomi") - self.assertEqual(payload["model_info"]["metadata"]["source"], "fusion-governance") - - def test_plan_skips_existing_xiaomi_v25_models(self): - entries = [ - {"model_name": "mimo-v2.5-pro", "model_info": {"id": "uuid-new-pro", "metadata": {"provider_key": "xiaomi"}}}, - { - "model_name": "mimo-v2.5-pro-ultraspeed", - "model_info": {"id": "uuid-new-fast", "metadata": {"provider_key": "xiaomi"}}, - }, - ] - - plan = catalog.build_governance_plan(entries, {"XIAOMI_API_KEY": "sk-xiaomi"}) - - self.assertFalse([a for a in plan.actions if a.action == "create"]) - - def test_missing_xiaomi_key_is_clear_when_registration_needed(self): - with self.assertRaisesRegex(RuntimeError, "XIAOMI_API_KEY"): - catalog.build_governance_plan([], {}) - - def test_serialized_plan_redacts_api_key(self): - plan = catalog.build_governance_plan([], {"XIAOMI_API_KEY": "sk-secret-xiaomi"}) - action = [action for action in plan.actions if action.action == "create"][0] - - serialized = catalog.serialize_action(action) - - self.assertEqual(serialized["payload"]["litellm_params"]["api_key"], "***") - self.assertEqual(action.payload["litellm_params"]["api_key"], "sk-secret-xiaomi") - - def test_replace_deprecated_xiaomi_models_in_key_allowlist(self): - models = ["deepseek-chat", "mimo-v2-flash", "mimo-v2-pro", "qwen-max-latest"] - - updated = catalog.replace_deprecated_models_in_allowlist(models) - - self.assertEqual( - updated, - ["deepseek-chat", "qwen-max-latest", "mimo-v2.5-pro", "mimo-v2.5-pro-ultraspeed"], - ) - - -if __name__ == "__main__": - unittest.main() -``` - -- [ ] **Step 2: Run red test** - -Run: - -```bash -.venv311/bin/python -m pytest test/test_litellm_model_catalog_governance.py -q -``` - -Expected: fails because `scripts/govern_litellm_model_catalog.py` does not exist. - -## Task 2: 实现治理脚本 - -- [ ] **Step 1: Create `scripts/govern_litellm_model_catalog.py`** - -Implement: - -- `TARGET_MODELS` -- `DEPRECATED_MODELS` -- `CatalogAction` -- `CatalogPlan` -- `build_governance_plan(entries, env)` -- `fetch_existing_models(base_url, key)` -- `apply_plan(base_url, key, plan)` -- `replace_deprecated_models_in_allowlist(models)` -- `fetch_key_models(base_url, master_key, virtual_key)` -- `update_key_models(base_url, master_key, virtual_key, models)` -- CLI with `--dry-run` default and `--apply` - -- [ ] **Step 2: Run governance tests** - -Run: - -```bash -.venv311/bin/python -m pytest test/test_litellm_model_catalog_governance.py -q -``` - -Expected: all tests pass. - -## Task 3: 写多模型测验基线测试 - -- [ ] **Step 1: Add failing tests** - -Create `test/test_model_catalog_eval_baseline.py` with tests for healthy filtering and result serialization. - -- [ ] **Step 2: Run red test** - -Run: - -```bash -.venv311/bin/python -m pytest test/test_model_catalog_eval_baseline.py -q -``` - -Expected: fails because `scripts/model_catalog_eval_baseline.py` does not exist. - -## Task 4: 实现多模型测验基线脚本 - -- [ ] **Step 1: Create `scripts/model_catalog_eval_baseline.py`** - -Implement: - -- fetch Fusion `/api/models` -- filter healthy models by default -- call Fusion `/api/chat/send` non-streaming with a configurable bearer token -- write JSONL results -- support `--dry-run`, `--models`, `--include-unhealthy`, `--question` - -- [ ] **Step 2: Run eval baseline tests** - -Run: - -```bash -.venv311/bin/python -m pytest test/test_model_catalog_eval_baseline.py -q -``` - -Expected: all tests pass. - -## Task 5: 验证、部署、真实回归 - -- [ ] **Step 1: Run targeted tests** - -```bash -.venv311/bin/python -m pytest test/test_litellm_model_catalog_governance.py test/test_model_catalog_eval_baseline.py -q -``` - -- [ ] **Step 2: Run lint/format checks** - -```bash -.venv/bin/python -m ruff check scripts/govern_litellm_model_catalog.py scripts/model_catalog_eval_baseline.py test/test_litellm_model_catalog_governance.py test/test_model_catalog_eval_baseline.py -.venv/bin/python -m ruff format --check scripts/govern_litellm_model_catalog.py scripts/model_catalog_eval_baseline.py test/test_litellm_model_catalog_governance.py test/test_model_catalog_eval_baseline.py -``` - -- [ ] **Step 3: Dry-run production governance plan** - -```bash -LITELLM_BASE_URL= LITELLM_MASTER_KEY= XIAOMI_API_KEY= LITELLM_VIRTUAL_KEY= .venv311/bin/python scripts/govern_litellm_model_catalog.py --dry-run --virtual-key "$LITELLM_VIRTUAL_KEY" -``` - -Expected: prints delete actions for old V2 models, create actions for missing V2.5 models, and a key allowlist update summary. Printed payload must redact `api_key`. - -- [ ] **Step 4: Apply after dry-run review** - -```bash -LITELLM_BASE_URL= LITELLM_MASTER_KEY= XIAOMI_API_KEY= LITELLM_VIRTUAL_KEY= .venv311/bin/python scripts/govern_litellm_model_catalog.py --apply --virtual-key "$LITELLM_VIRTUAL_KEY" -``` - -- [ ] **Step 5: Verify `/api/models`** - -Expected: - -- `mimo-v2-flash` absent -- `mimo-v2-pro` absent -- `mimo-v2.5-pro` present -- `mimo-v2.5-pro-ultraspeed` present - -- [ ] **Step 6: Run baseline smoke** - -Run against deployed Fusion with a real auth token and record JSONL output. diff --git a/docs/superpowers/plans/2026-07-01-multi-model-eval-v1-1.md b/docs/superpowers/plans/2026-07-01-multi-model-eval-v1-1.md deleted file mode 100644 index cb2cb103..00000000 --- a/docs/superpowers/plans/2026-07-01-multi-model-eval-v1-1.md +++ /dev/null @@ -1,109 +0,0 @@ -# Multi Model Eval v1.1 Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** 把多模型基线脚本增强为多场景、多 transport、可汇总的 Fusion 产品链路测验 runner。 - -**Architecture:** 保持独立脚本形态，不改变业务 API。脚本从 `/api/models` 读取模型目录，生成模型和场景的笛卡尔积；`stream` transport 通过 `/api/chat/send` 解析 SSE envelope，`nonstream` transport 保留现有快速 smoke；结果写 JSONL，summary 由纯函数从结果聚合。 - -**Tech Stack:** Python 3.11, httpx, pytest/unittest, Fusion `/api/models`, Fusion `/api/chat/send`, SSE envelope. - ---- - -## Files - -- Modify: `scripts/model_catalog_eval_baseline.py` - - 增加 `EvalScenario`, `EvalSummary`, stream SSE 解析、失败分型、summary 聚合和 CLI 参数。 -- Modify: `test/test_model_catalog_eval_baseline.py` - - 增加 v1.1 的场景矩阵、SSE 解析、stream 结果、summary 和兼容性测试。 -- Create: `docs/superpowers/specs/2026-07-01-multi-model-eval-v1-1-design.md` - - 记录设计边界。 -- Create: `docs/superpowers/plans/2026-07-01-multi-model-eval-v1-1.md` - - 记录实施计划。 - -## Requirements - -- 默认 transport 为 `stream`，因为这是 Fusion 的真实产品链路。 -- 保留 `nonstream` transport，便于快速判断模型基础可调用性。 -- 默认场景矩阵至少包含基础对话、中文事实、代码/推理、自主搜索、不应搜索、长回答。 -- dry-run 输出即将运行的模型和场景组合，不调用 `/api/chat/send`。 -- apply 模式必须要求 `--auth-token`。 -- JSONL 明细必须记录模型、场景、transport、成功状态、耗时、回答摘要、错误、工具调用观测、工具期望是否命中。 -- summary 必须可由 JSONL 结果纯函数聚合，不依赖外部服务。 -- 不在日志或输出中打印 auth token。 - -## Test Matrix - -| Case | 输入 | 预期 | -| --- | --- | --- | -| EVAL11-01 | 未指定场景 | 使用默认场景矩阵 | -| EVAL11-02 | 指定场景 id | 只运行这些场景，保持用户输入顺序 | -| EVAL11-03 | 未知场景 id | 抛出明确错误 | -| EVAL11-04 | dry-run | 输出模型和场景组合 | -| EVAL11-05 | SSE 含 answering delta 和 agent tool events | 生成成功结果，记录工具名和回答摘要 | -| EVAL11-06 | SSE 含 error envelope | 生成失败结果，错误类型为 `stream_error` | -| EVAL11-07 | expected_tool_use=expected 但没有工具 | `tool_expectation_met=false` | -| EVAL11-08 | 结果列表 | summary 正确统计 success/failure、by_model、by_scenario、failure_types | -| EVAL11-09 | nonstream 成功 | 保持旧 JSONL 核心字段，并补齐新字段 | -| EVAL11-10 | 回答正文包含 `` 标签 | JSONL 标记 `reasoning_tag_leak`，summary 聚合质量标记 | -| EVAL11-11 | 长批次逐项执行 | 每个条目完成后触发结果回调，CLI 可增量写 JSONL | - -## Tasks - -### Task 1: 写 v1.1 失败测试 - -- [x] 在 `test/test_model_catalog_eval_baseline.py` 添加默认场景、场景筛选、未知场景、SSE 解析、summary 和 nonstream 兼容测试。 -- [x] 运行 `.venv311/bin/python -m pytest test/test_model_catalog_eval_baseline.py -q`，确认新增测试因缺少 v1.1 API 失败。 - -### Task 2: 实现场景矩阵和结果结构 - -- [x] 在 `scripts/model_catalog_eval_baseline.py` 增加 `EvalScenario` 和默认场景矩阵。 -- [x] 扩展 `EvalResult` 字段，保持旧字段仍存在。 -- [x] 实现 `select_scenarios`、`tool_expectation_met`、失败分型函数。 -- [x] 运行目标测试确认相关用例通过。 - -### Task 3: 实现 stream transport - -- [x] 实现 SSE 行解析，把 `data: [DONE]` 作为终止。 -- [x] 从 `agent_event.tool_call_started/tool_call_completed` 统计工具名。 -- [x] 从 `answering` delta 聚合回答摘要。 -- [x] 从 `error` envelope 生成失败结果。 -- [x] 运行目标测试确认 stream 用例通过。 - -### Task 4: 实现 summary 和 CLI - -- [x] 实现 `build_summary` 和 summary JSON 输出。 -- [x] 增加 `--transport`, `--scenarios`, `--summary-output`。 -- [x] dry-run 输出模型和场景组合。 -- [x] apply 模式按模型和场景运行，写 JSONL 和 summary。 - -### Task 5: 验证和发布 - -- [x] `.venv/bin/python -m ruff format --check scripts/model_catalog_eval_baseline.py test/test_model_catalog_eval_baseline.py` -- [x] `.venv/bin/python -m ruff check .` -- [x] `.venv311/bin/python -m pytest test/test_model_catalog_eval_baseline.py -q` -- [x] `.venv311/bin/python -m pytest test/ -q` -- [x] `.venv311/bin/python scripts/check_architecture.py` -- [x] commit + push，监控 GitHub Actions 和 dev deploy。 -- [x] 部署后运行脚本 dry-run，确认已部署环境能列出模型和场景组合。 - -### Task 6: 增加输出质量标记 - -- [x] 为 `EvalResult` 增加 `quality_flags` 字段。 -- [x] 标记回答正文中的 `` / `` 泄漏。 -- [x] 在 summary 中聚合质量标记数量。 -- [x] 补充 reasoning 标签泄漏和 summary 聚合测试。 - -### Task 7: 增加长批次保护 - -- [x] 为 `run_eval` 增加逐项 `on_result` 回调。 -- [x] CLI apply 模式每完成一项就追加写 JSONL。 -- [x] CLI apply 模式向 stderr 输出进度摘要。 -- [x] 补充成功项和失败项都会触发回调的测试。 - -### Task 8: 增加质量风险报告 - -- [x] 为 `quality_flags` 配置稳定的严重级别和处理建议。 -- [x] 在 summary 中输出 `quality_issue_count`、`quality_issues` 和 `quality_risk_by_model`。 -- [x] 补充逐条质量问题和按模型聚合风险测试。 -- [x] 运行 `.venv311/bin/python -m pytest test/test_model_catalog_eval_baseline.py -q` 确认通过。 diff --git a/docs/superpowers/plans/2026-07-01-reasoning-tag-filter.md b/docs/superpowers/plans/2026-07-01-reasoning-tag-filter.md deleted file mode 100644 index 887311ec..00000000 --- a/docs/superpowers/plans/2026-07-01-reasoning-tag-filter.md +++ /dev/null @@ -1,33 +0,0 @@ -# Reasoning Tag Filter Implementation Plan - -**Goal:** 在服务端过滤被模型错误写入正文通道的 `...`，避免实时输出和历史消息出现 reasoning 标签。 - -## Files - -- Modify: `app/services/stream/llm_stream.py` - - 增加状态化正文过滤逻辑。 - - 在 content delta 写入 Redis 和 `content_buf` 前过滤。 -- Modify: `test/services/stream/test_llm_stream.py` - - 覆盖完整标签和跨 chunk 标签两类泄漏。 -- Create: `docs/superpowers/specs/2026-07-01-reasoning-tag-filter-design.md` - - 记录过滤边界和非目标。 -- Create: `docs/superpowers/plans/2026-07-01-reasoning-tag-filter.md` - - 记录执行计划和验收。 - -## Test Matrix - -| Case | 输入 | 预期 | -| --- | --- | --- | -| RTAG-01 | `内部思考可见正文` | SSE `answering` 和 `content_buf` 只保留 `可见正文` | -| RTAG-02 | `` 与 `` 跨多个 chunk | 标签和内部思考不被提前推送 | -| RTAG-03 | 标准 `reasoning_content` | 原 reasoning chunk 逻辑不变 | -| RTAG-04 | 普通文本流 | 原 lock check、usage、tool call 行为不变 | - -## Tasks - -- [x] 写 RTAG-01 / RTAG-02 失败测试。 -- [x] 实现正文通道 reasoning tag 过滤。 -- [x] 跑目标测试确认通过。 -- [x] 跑 stream 测试、lint、全量后端测试和架构检查。 -- [ ] commit + push，监控 CI/CD 和 dev 部署。 -- [ ] 部署后使用 `MiniMax-M2.7` 真实 stream 对话回归。 diff --git a/docs/superpowers/plans/2026-07-01-search-failure-recovery-budget-v1-3.md b/docs/superpowers/plans/2026-07-01-search-failure-recovery-budget-v1-3.md deleted file mode 100644 index 228b091c..00000000 --- a/docs/superpowers/plans/2026-07-01-search-failure-recovery-budget-v1-3.md +++ /dev/null @@ -1,253 +0,0 @@ -# Search Failure Recovery / Budget v1.3 Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** 让搜索预算器能基于上一轮搜索质量和读取失败状态做最小失败恢复决策，减少机械重复搜索。 - -**Architecture:** 继续保留现有 advisory 模式。`NetworkToolBudget` 增加纯内存反馈状态，`tool_round` 在每轮工具结果后回填状态，`agent_behavior_eval` 增加 action 级断言。系统不自动读取网页，不新增前端 UI，不启动本地服务。 - -**Tech Stack:** FastAPI service layer, Python dataclass, pytest, ruff, existing `ToolResult.data`, existing `ToolExecutionRecord`, existing agent behavior eval script. - ---- - -## Scope And Commit Policy - -- 本计划只改 `fusion-api`。 -- 文档和代码本地一起提交；不单独 push 文档 commit。 -- Chrome 回归只复用已有登录态 Fusion 标签；无可复用标签时停止并报告阻塞。 - -## File Map - -- Modify: `app/services/search_budget.py` - - 增加 repair 小预算 helper。 - - 扩展 `SearchBudgetDecision` 的可选恢复字段。 -- Modify: `app/services/stream/network_budget.py` - - 增加搜索质量和读取失败反馈状态。 - - 新增 `record_tool_results(...)`。 - - 新增 `repair_search` 和 `redirect_to_read_alternative` 决策。 -- Modify: `app/services/stream/tool_round.py` - - 工具执行后构造一次 `SourceSelectionPlan`。 - - 回填 `NetworkToolBudget.record_tool_results(...)`。 - - 复用同一个 plan 生成 evidence 和 LLM guidance。 -- Modify: `app/services/search_read_decision_ledger.py` - - 把 `repair_search` 计入 provider search。 -- Modify: `app/services/tool_handlers/web_search.py` - - 为 `redirect_to_read_alternative` 输出安全 LLM context 和摘要。 -- Modify: `scripts/agent_behavior_eval.py` - - 增加 `expected_search_actions`、`required_search_actions`、`forbidden_search_actions`、`max_repair_search_calls`。 -- Modify: `test/fixtures/agent_behavior_eval_samples.json` - - 增加 v1.3 样本字段。 -- Test: `test/services/stream/test_network_budget.py` -- Test: `test/services/stream/test_tool_round.py` -- Test: `test/test_agent_behavior_eval.py` - -## Task 1: Network Budget Failure Feedback - -**Files:** -- Modify: `app/services/stream/network_budget.py` -- Modify: `app/services/search_budget.py` -- Test: `test/services/stream/test_network_budget.py` - -- [ ] **Step 1: Write failing tests** - -Add tests: - -```python -def test_empty_first_search_marks_next_search_as_repair(self): - budget = NetworkToolBudget() - first_args, first_degraded = budget.prepare_web_search_args({"query": "OpenAI 2026 最新产品"}) - budget.record_tool_results([_search_record(first_args, status="degraded", sources=[])]) - - second_args, second_degraded = budget.prepare_web_search_args({"query": "OpenAI 官方公告 2026 最新"}) - - self.assertIsNone(first_degraded) - self.assertIsNone(second_degraded) - self.assertEqual(second_args["budget_decision"]["action"], "repair_search") - self.assertEqual(second_args["budget_decision"]["reason_code"], "previous_search_no_results") - self.assertLessEqual(second_args["count"], 3) - self.assertEqual(budget.web_search_calls, 2) -``` - -```python -def test_read_failure_redirects_search_to_unread_candidate(self): - budget = NetworkToolBudget() - plan = _source_plan(["https://a.example/post", "https://b.example/post"]) - budget.record_tool_results( - [_url_read_record("https://a.example/post", status="degraded")], - source_plan=plan, - ) - - args, degraded = budget.prepare_web_search_args({"query": "继续搜索同一问题"}) - - self.assertIsNotNone(degraded) - self.assertEqual(args["budget_decision"]["action"], "redirect_to_read_alternative") - self.assertEqual(args["budget_decision"]["reason_code"], "read_alternatives_available") - self.assertEqual(args["count"], 0) - self.assertEqual(budget.web_search_calls, 0) -``` - -- [ ] **Step 2: Verify tests fail** - -Run: - -```bash -.venv311/bin/python -m pytest test/services/stream/test_network_budget.py -q -``` - -Expected: new tests fail because `record_tool_results` and recovery actions do not exist. - -- [ ] **Step 3: Implement minimal feedback state** - -Implement: - -- `NetworkToolBudget.record_tool_results(results, source_plan=None)`. -- Search result feedback from `web_search` result status and `result_count`. -- Candidate/read feedback from `SourceSelectionPlan.read_decisions` and `url_read` results. -- One repair search maximum. -- Redirect to unread candidate before new provider search. - -- [ ] **Step 4: Verify Task 1** - -Run: - -```bash -.venv311/bin/python -m pytest test/services/stream/test_network_budget.py -q -``` - -Expected: all tests in file pass. - -## Task 2: Tool Round Feedback Wiring - -**Files:** -- Modify: `app/services/stream/tool_round.py` -- Test: `test/services/stream/test_tool_round.py` - -- [ ] **Step 1: Write failing test** - -Add: - -```python -async def test_tool_round_records_network_budget_feedback_with_source_plan(self): - budget = Mock() - budget.record_tool_results = Mock() - # execute a web_search record with sources - await handle_tool_calls_round(...) - budget.record_tool_results.assert_called_once() - self.assertIsNotNone(budget.record_tool_results.call_args.kwargs["source_plan"]) -``` - -- [ ] **Step 2: Verify test fails** - -Run: - -```bash -.venv311/bin/python -m pytest test/services/stream/test_tool_round.py -q -``` - -- [ ] **Step 3: Implement wiring** - -Implement: - -- Build source plan once after `execute_tool_round_tools`. -- Pass plan to `record_tool_results`. -- Reuse the same plan for selected evidence and source guidance. -- Keep existing public behavior unchanged when no search result exists. - -- [ ] **Step 4: Verify Task 2** - -Run: - -```bash -.venv311/bin/python -m pytest test/services/stream/test_tool_round.py -q -``` - -Expected: all tests in file pass. - -## Task 3: Eval And Ledger Support - -**Files:** -- Modify: `app/services/search_read_decision_ledger.py` -- Modify: `scripts/agent_behavior_eval.py` -- Modify: `test/fixtures/agent_behavior_eval_samples.json` -- Test: `test/test_agent_behavior_eval.py` - -- [ ] **Step 1: Write failing tests** - -Add tests for: - -- `repair_search` counts as provider search. -- `expected_search_actions` exact list mismatch is flagged. -- `required_search_actions` missing action is flagged. -- `max_repair_search_calls` flags too many repairs. - -- [ ] **Step 2: Verify tests fail** - -Run: - -```bash -.venv311/bin/python -m pytest test/test_agent_behavior_eval.py test/services/stream/test_tool_round.py::ToolRoundTests::test_build_search_read_decision_ledger_summarizes_budget_and_read_decisions -q -``` - -- [ ] **Step 3: Implement eval fields** - -Implement: - -- `OPTIONAL_STRING_LIST_FIELDS` adds `expected_search_actions`, `required_search_actions`, `forbidden_search_actions`. -- `OPTIONAL_NON_NEGATIVE_INT_FIELDS` adds `max_repair_search_calls`. -- `_check_search_context` reads `observation["search_actions"]`. -- Ledger `PROVIDER_SEARCH_ACTIONS` includes `repair_search`. - -- [ ] **Step 4: Verify Task 3** - -Run: - -```bash -.venv311/bin/python -m pytest test/test_agent_behavior_eval.py test/services/stream/test_tool_round.py -q -``` - -Expected: all tests pass. - -## Task 4: Full Verification And Release - -**Files:** -- All touched files. - -- [ ] **Step 1: Run targeted tests** - -```bash -.venv311/bin/python -m pytest \ - test/services/stream/test_network_budget.py \ - test/services/stream/test_tool_round.py \ - test/test_agent_behavior_eval.py \ - -q -``` - -- [ ] **Step 2: Run behavior eval dry-run** - -```bash -.venv311/bin/python scripts/agent_behavior_eval.py --dry-run -``` - -- [ ] **Step 3: Run full backend verification** - -```bash -.venv311/bin/python -m pytest test/ -q -.venv311/bin/python -m ruff check . -.venv311/bin/python -m ruff format --check . -``` - -- [ ] **Step 4: Commit and push** - -Use a structured Chinese commit message with `背景：` and `改动：`, including: - -```text -Co-Authored-By: Codex -``` - -- [ ] **Step 5: CI/CD and real regression** - -After push: - -- Watch GitHub Actions until build/test/deploy complete. -- Reuse existing logged-in `fusion.seanfield.org` Chrome tab only. -- Run real cases and record URL/input/expected/actual/console/refresh/conclusion. diff --git a/docs/superpowers/plans/2026-07-01-search-read-planner-v1-1.md b/docs/superpowers/plans/2026-07-01-search-read-planner-v1-1.md deleted file mode 100644 index 3183f911..00000000 --- a/docs/superpowers/plans/2026-07-01-search-read-planner-v1-1.md +++ /dev/null @@ -1,148 +0,0 @@ -# Search / Read Planner v1.1 Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** 让 Fusion 的联网链路从“LLM 自己决定搜索/深读，后端只给提示”推进到“后端提供可测试的搜索预算、重复搜索收敛、读源选择和证据解释”。 - -**Architecture:** 保留 LLM 自主 tool-call 模式，不引入自动读网页的强控制器。后端新增轻量 Search / Read Planner 边界：`NetworkToolBudget` 负责搜索次数和重复搜索拦截，`SourceCandidateRanker` 负责候选评分，`search_read_planner` 负责按意图决定推荐深读数量和生成 LLM guidance，`Evidence Ledger` 继续承接 UI 和历史恢复。 - -**Tech Stack:** FastAPI service layer, pytest, ruff, existing agent_event/evidence protocol, deployed Fusion regression with existing logged-in Chrome tab only. - ---- - -## Acceptance Matrix - -| Case | Input Shape | Expected | -| --- | --- | --- | -| SRP-01 | 简单闲聊/数学/身份问题 | 不调用 `web_search` / `url_read`，不展示执行过程或回答依据 | -| SRP-02 | 自然语言实时问题，用户没有说“联网” | LLM 自主调用 `web_search`，展示搜索关键词和来源 | -| SRP-03 | 中文 query 含 `2026年` | 后端能推断为 freshness，不落到机械 standard 预算 | -| SRP-04 | 第二次搜索与第一次高度重复 | 后端跳过真实搜索，返回用户不可见的重复搜索降级上下文，不消耗 provider | -| SRP-05 | 第二次搜索与第一次相似但不是完全重复 | 后端使用 follow-up 小预算，不再出现两个机械 `5 + 5` | -| SRP-06 | 官方来源 + 权威媒体互补搜索 | 允许第二次搜索，保留更大的互补预算 | -| SRP-07 | 搜索返回 8-10 条候选 | planner 只推荐少量高价值来源深读，并说明“不是全部读取” | -| SRP-08 | quick fact 搜索 | 最多推荐 1 个深读来源 | -| SRP-09 | freshness / standard 搜索 | 最多推荐 2 个深读来源 | -| SRP-10 | official / comparison / deep research 搜索 | 最多推荐 3 个深读来源 | -| SRP-11 | url_read 失败或降级 | evidence 记录读取状态，但 LLM 上下文和用户展示不泄漏 `reader-service` / `url_read` | -| SRP-12 | 刷新已完成对话 | 搜索关键词、回答依据、执行过程摘要保持一致，无 running 残留 | -| SRP-13 | 普通实时问题被模型机械扩成 3+ 次搜索 | 后端默认最多执行 2 次真实搜索；第三次返回 `search_plan_limited`，不消耗 provider，不进入回答依据 | -| SRP-14 | deep_research 深度研究 | 最多执行 3 次真实搜索；第四次返回 `search_plan_limited` | - -## File Map - -- Modify: `app/services/search_budget.py` - - 修正中文年份识别。 - - 提供重复搜索判定和相似 follow-up 判定。 -- Modify: `app/services/stream/network_budget.py` - - 对重复搜索直接返回 degraded `ToolResult`，不调用搜索 provider。 - - 对相似 follow-up 使用更小预算。 -- Create: `app/services/search_read_planner.py` - - 根据搜索结果 intent / budget 计算推荐深读上限。 - - 调用 ranker 生成 `SourceSelectionPlan`。 - - 生成面向 LLM 的读源 guidance。 -- Modify: `app/services/source_candidate_ranker.py` - - 让 `SearchResultForRanking` 携带 `intent` / `search_budget`。 - - `SourceSelectionPlan` 记录推荐上限和未推荐数量。 - - guidance 增强为“推荐读哪些”和“为什么不用读全部”。 -- Modify: `app/services/stream/tool_round.py` - - 使用 `search_read_planner` 构造本轮 plan。 - - selected evidence 数量跟随 planner 上限。 -- Modify: `app/services/tool_handlers/web_search.py` - - 对 duplicate-skipped 搜索返回明确的 LLM 上下文，避免模型误以为搜索失败。 -- Modify: `app/ai/tools.py` - - 收紧描述：默认一次搜索；第二次必须是官方/媒体/地区/时间范围等互补维度。 -- Modify: `scripts/agent_behavior_eval.py` - - 增加搜索次数、搜索关键词去重、推荐深读数量的离线打分字段。 -- Modify: `test/fixtures/agent_behavior_eval_samples.json` - - 增加 Search / Read Planner v1.1 样本。 -- Tests: - - `test/services/stream/test_network_budget.py` - - `test/test_source_candidate_ranker.py` - - `test/services/stream/test_tool_round.py` - - `test/test_ai_tools.py` - - `test/test_agent_behavior_eval.py` - -## Task 1: Search Budget And Duplicate Control - -- [ ] Add failing tests: - - `test_chinese_year_query_infers_freshness_intent` - - `test_second_similar_chinese_year_query_uses_followup_budget` - - `test_duplicate_web_search_returns_degraded_without_consuming_provider_budget` -- [ ] Run: - - `.venv311/bin/python -m pytest test/services/stream/test_network_budget.py -q` - - Expected before implementation: new tests fail. -- [ ] Implement: - - Replace `\b20\d{2}\b` with digit-boundary regex that matches `2026年`. - - Add duplicate detection for exact or very-high-similarity same-intent queries. - - Return degraded `ToolResult` with `duplicate_search_skipped=True`, `requested_count=0`, `context_source_limit=0`. -- [ ] Verify targeted tests pass. - -## Task 2: Read Planner Boundary - -- [ ] Add failing tests: - - quick fact recommends 1 source. - - freshness recommends 2 sources. - - official/comparison/deep research recommends 3 sources. - - guidance includes search keywords, recommended read limit, and “do not read all results” semantics. -- [ ] Run: - - `.venv311/bin/python -m pytest test/test_source_candidate_ranker.py -q` - - Expected before implementation: new planner imports or fields fail. -- [ ] Implement `app/services/search_read_planner.py`. -- [ ] Extend ranker dataclasses without breaking existing callers. -- [ ] Verify targeted tests pass. - -## Task 3: Tool Round Integration - -- [ ] Add failing tests: - - `tool_round` passes `intent` / `search_budget` into planner. - - selected evidence count follows planner limit. - - duplicate-skipped search guidance is not treated as a normal evidence source. -- [ ] Run: - - `.venv311/bin/python -m pytest test/services/stream/test_tool_round.py -q` - - Expected before implementation: planner integration assertions fail. -- [ ] Implement: - - Replace direct `rank_search_sources()` calls in `tool_round.py` with `build_search_read_plan()`. - - Use planner guidance for LLM context. -- [ ] Verify targeted tests pass. - -## Task 4: Tool Description And Offline Eval - -- [ ] Add failing tests: - - web search description mentions complementary second search dimensions. - - eval scorer flags duplicate search keywords when a sample sets `max_duplicate_search_keywords=0`. - - eval scorer flags too many recommended reads when a sample sets `max_recommended_reads`. -- [ ] Run: - - `.venv311/bin/python -m pytest test/test_ai_tools.py test/test_agent_behavior_eval.py -q` - - Expected before implementation: new assertions fail. -- [ ] Implement schema/fixture/scorer updates. -- [ ] Verify targeted tests pass. - -## Task 5: Full Verification, Commit, CI/CD, Real Regression - -- [ ] Run backend checks: - - `.venv311/bin/python -m pytest -q` - - `.venv/bin/ruff check .` - - `.venv311/bin/python scripts/agent_behavior_eval.py --dry-run` -- [ ] Commit with structured Chinese message and `Co-Authored-By`. -- [ ] Push once after code and plan are together. -- [ ] Monitor GitHub Actions until deploy completes. -- [ ] Real Chrome regression uses existing `fusion.seanfield.org` tab only: - - SRP-01 simple no-search case. - - SRP-02 autonomous realtime-search case. - - SRP-05/SRP-07 multi-search/read-selection case. - - Refresh completed conversation and check no running residue. - - Record URL, expected, actual, console errors, refresh result. - -## Task 6: Post-Deploy Regression Tightening - -- [x] Add failing tests: - - normal realtime/product-update questions execute at most 2 provider searches. - - `deep_research` may execute the 3rd provider search, but the 4th is plan-limited. - - plan-limited search does not create answer-evidence content blocks. - - plan-limited search gives the LLM a reuse-existing-results instruction, not a generic search failure. -- [x] Implement: - - `NetworkToolBudget` returns degraded `ToolResult` with `search_plan_limited=True`, `requested_count=0`, `context_source_limit=0`. - - `WebSearchHandler` hides duplicate/plan-limited control results from persisted answer-evidence blocks. - - Tool description states the third search is only for `deep_research`. -- [ ] Verify targeted tests, full backend tests, CI/CD, and real Chrome regression again. diff --git a/docs/superpowers/plans/2026-07-01-search-read-planner-v1-2.md b/docs/superpowers/plans/2026-07-01-search-read-planner-v1-2.md deleted file mode 100644 index f2c6ed42..00000000 --- a/docs/superpowers/plans/2026-07-01-search-read-planner-v1-2.md +++ /dev/null @@ -1,421 +0,0 @@ -# Search / Read Planner v1.2 Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** 让 Search / Read Planner 的搜索预算、读取推荐和未读候选原因变成结构化、可测试、可回归的决策链路。 - -**Architecture:** 保留 v1.1 advisory 模式，不自动执行 `url_read`，不引入 LLM reranker。后端新增确定性 decision ledger：`NetworkToolBudget` 暴露搜索预算决策，`SourceCandidateRanker` 输出读取推荐/未推荐 reason code，离线评估器消费这些 observation 字段做回归断言。 - -**Tech Stack:** FastAPI service layer, Python dataclass, pytest, ruff, existing `ToolResult.data`, existing agent behavior eval script, deployed Fusion regression with existing logged-in Chrome tab only. - ---- - -## Scope And Commit Policy - -- 本计划只改 `fusion-api`。 -- 本计划不新增前端 UI。 -- 文档和代码可以分 commit，但不单独 push 文档 commit；实现、验证完成后统一 push，触发一次 CI/CD。 -- 真实 Chrome 回归只复用用户已经打开且登录的 `fusion.seanfield.org` 标签；禁止新 Chrome、新标签、新窗口、`about:blank` 和 isolated context。 - -## File Map - -- Create: `app/services/search_read_decision_ledger.py` - - 聚合搜索预算决策和读取来源决策，输出离线评估可用 summary。 -- Modify: `app/services/search_budget.py` - - 新增 `SearchBudgetDecision` dataclass 和预算决策 reason code。 -- Modify: `app/services/stream/network_budget.py` - - 把搜索预算 decision 写入 normalized args 和 degraded `ToolResult.data`。 -- Modify: `app/services/source_candidate_ranker.py` - - 新增 `SourceReadDecision` dataclass。 - - `SourceSelectionPlan` 增加 `read_decisions` 和 `decision_summary`。 - - guidance 增加未推荐原因汇总。 -- Modify: `app/services/search_read_planner.py` - - 保持读源上限逻辑，返回增强后的 `SourceSelectionPlan`。 -- Modify: `app/services/stream/tool_round.py` - - 使用增强 plan，不改变现有 selected evidence 行为。 -- Modify: `scripts/agent_behavior_eval.py` - - 增加搜索次数、provider 搜索次数、预算名称、禁止读取域名、必需 reason code 检查。 -- Modify: `test/fixtures/agent_behavior_eval_samples.json` - - 增加 v1.2 评估样本字段。 -- Test: `test/services/stream/test_network_budget.py` -- Test: `test/test_source_candidate_ranker.py` -- Test: `test/services/stream/test_tool_round.py` -- Test: `test/test_agent_behavior_eval.py` -- Test: `test/test_ai_tools.py` - -## Task 1: Search Budget Decision - -**Files:** -- Modify: `app/services/search_budget.py` -- Modify: `app/services/stream/network_budget.py` -- Test: `test/services/stream/test_network_budget.py` - -- [x] Add failing tests: - - `test_initial_search_records_budget_decision` - - `test_similar_followup_records_narrow_followup_decision` - - `test_duplicate_search_records_skip_duplicate_decision` - - `test_planner_limited_search_records_limit_decision` - -Expected assertions: - -```python -args, degraded = budget.prepare_web_search_args({"query": "OpenAI 最近发布了哪些产品更新？"}) -self.assertIsNone(degraded) -self.assertEqual(args["budget_decision"]["action"], "execute") -self.assertEqual(args["budget_decision"]["reason_code"], "initial_search") -self.assertEqual(args["budget_decision"]["budget_name"], args["search_budget"]) -``` - -```python -_first_args, _ = budget.prepare_web_search_args({"query": "OpenAI 最新公告 2026年7月"}) -second_args, degraded = budget.prepare_web_search_args({"query": "OpenAI 最新产品更新 2026年7月"}) -self.assertIsNone(degraded) -self.assertEqual(second_args["budget_decision"]["action"], "narrow_followup") -self.assertEqual(second_args["budget_decision"]["reason_code"], "similar_followup") -self.assertEqual(second_args["count"], 3) -``` - -```python -_first_args, _ = budget.prepare_web_search_args({"query": "OpenAI 最新公告 2026年7月"}) -second_args, degraded = budget.prepare_web_search_args({"query": "OpenAI 最新公告 2026年7月"}) -self.assertIsNotNone(degraded) -self.assertEqual(second_args["budget_decision"]["action"], "skip_duplicate") -self.assertEqual(degraded.data["budget_decision"]["reason_code"], "duplicate_query") -self.assertEqual(budget.web_search_calls, 1) -``` - -```python -budget.prepare_web_search_args({"query": "OpenAI 2026年产品更新 最新发布"}) -budget.prepare_web_search_args({"query": "OpenAI 2026年最新新闻 媒体报道"}) -third_args, degraded = budget.prepare_web_search_args({"query": "OpenAI GPT-5.6 预览 2026年7月"}) -self.assertIsNotNone(degraded) -self.assertEqual(third_args["budget_decision"]["action"], "limit_planner") -self.assertEqual(degraded.data["budget_decision"]["reason_code"], "planned_search_limit_reached") -``` - -- [x] Run targeted test and confirm failure: - -```bash -.venv311/bin/python -m pytest test/services/stream/test_network_budget.py -q -``` - -- [x] Implement: - - Add `SearchBudgetDecision` to `app/services/search_budget.py`. - - Add helper `build_search_budget_decision(...) -> SearchBudgetDecision`. - - In `NetworkToolBudget.prepare_web_search_args()`, attach `budget_decision` via `decision.__dict__`. - - For degraded duplicate/planner/hard-budget results, include the same `budget_decision` in `ToolResult.data`. - -- [x] Verify: - -```bash -.venv311/bin/python -m pytest test/services/stream/test_network_budget.py -q -``` - -Expected: all tests in file pass. - -## Task 2: Source Read Decision - -**Files:** -- Modify: `app/services/source_candidate_ranker.py` -- Modify: `app/services/search_read_planner.py` -- Test: `test/test_source_candidate_ranker.py` - -- [x] Add failing tests: - - `test_source_selection_plan_records_read_decisions_for_all_candidates` - - `test_low_priority_sources_are_deprioritized_with_reason_code` - - `test_guidance_summarizes_not_recommended_reason_codes` - -Expected assertions: - -```python -plan = rank_search_sources(search_results, max_recommended=2) -self.assertEqual(len(plan.read_decisions), plan.unique_source_count) -self.assertEqual( - [decision.action for decision in plan.read_decisions[:2]], - ["recommend_read", "recommend_read"], -) -self.assertIn("recommended_read", plan.decision_summary) -``` - -```python -low = next(decision for decision in plan.read_decisions if decision.candidate.domain == "youtube.com") -self.assertEqual(low.action, "deprioritize") -self.assertEqual(low.reason_code, "low_priority_source_type") -``` - -```python -guidance = format_source_selection_guidance(plan) -self.assertIn("未建议深读原因", guidance) -self.assertIn("低优先级来源", guidance) -self.assertIn("超过本轮推荐深读上限", guidance) -``` - -- [x] Run targeted test and confirm failure: - -```bash -.venv311/bin/python -m pytest test/test_source_candidate_ranker.py -q -``` - -- [x] Implement: - - Add `SourceReadDecision`. - - Extend `SourceSelectionPlan` with `read_decisions` and `decision_summary`. - - Build read decisions after ranking: - - recommended candidates -> `recommend_read` - - low priority candidates -> `deprioritize / low_priority_source_type` - - non-low candidates outside read limit -> `keep_candidate / outside_read_limit` - - Keep `recommended`, `low_priority`, `not_recommended_count` unchanged for compatibility. - - Update guidance to summarize reason counts. - -- [x] Verify: - -```bash -.venv311/bin/python -m pytest test/test_source_candidate_ranker.py -q -``` - -Expected: all tests in file pass. - -## Task 3: Search / Read Decision Ledger - -**Files:** -- Create: `app/services/search_read_decision_ledger.py` -- Modify: `app/services/stream/tool_round.py` -- Test: `test/services/stream/test_tool_round.py` - -- [x] Add failing tests: - - `test_build_search_read_decision_ledger_summarizes_budget_and_read_decisions` - - `test_tool_round_guidance_uses_enhanced_read_decision_summary` - -Expected ledger shape: - -```python -ledger = build_search_read_decision_ledger(results, source_plan=plan) -self.assertEqual(ledger["summary"]["executed_search_count"], 2) -self.assertEqual(ledger["summary"]["recommended_read_count"], 3) -self.assertIn("search_decisions", ledger) -self.assertIn("read_decisions", ledger) -``` - -Expected guidance assertion: - -```python -self.assertIn("未建议深读原因", messages[3]["content"]) -self.assertIn("只有当推荐来源无法回答关键事实", messages[3]["content"]) -``` - -- [x] Run targeted test and confirm failure: - -```bash -.venv311/bin/python -m pytest test/services/stream/test_tool_round.py -q -``` - -- [x] Implement: - - Create pure function `build_search_read_decision_ledger(results, source_plan=None)`. - - Extract `budget_decision` from `ToolExecutionRecord.result.data`. - - Extract `read_decisions` from `SourceSelectionPlan`. - - Do not emit new SSE event in v1.2; keep ledger usable by tests and future UI. - - Keep current selected evidence emission unchanged. - -- [x] Verify: - -```bash -.venv311/bin/python -m pytest test/services/stream/test_tool_round.py -q -``` - -Expected: all tests in file pass. - -## Task 4: Offline Behavior Eval V1.2 - -**Files:** -- Modify: `scripts/agent_behavior_eval.py` -- Modify: `test/fixtures/agent_behavior_eval_samples.json` -- Test: `test/test_agent_behavior_eval.py` - -- [x] Add failing tests: - - `test_load_samples_accepts_v1_2_planner_fields` - - `test_score_observation_flags_excess_search_calls_and_wrong_budgets` - - `test_score_observation_flags_forbidden_read_domains` - - `test_score_observation_requires_decision_reason_codes` - -Expected scoring assertions: - -```python -sample = { - "id": "planner-v1-2", - "expected_tool_policy": "search", - "expected_surface": "evidence", - "max_search_calls": 2, - "max_provider_search_calls": 2, - "expected_search_budgets": ["freshness", "freshness_followup"], - "forbidden_read_domains": ["youtube.com"], - "required_decision_reason_codes": ["official_original"], -} -observation = { - "tool_calls": ["web_search", "web_search", "web_search"], - "surfaces": ["execution_process", "answer_evidence"], - "search_call_count": 3, - "provider_search_call_count": 3, - "search_budgets": ["freshness", "standard", "standard"], - "read_domains": ["youtube.com"], - "decision_reason_codes": [], -} -score = score_observation(sample, observation) -self.assertFalse(score["passed"]) -self.assertIn("搜索调用次数过多", "\n".join(score["issues"])) -self.assertIn("provider 搜索次数过多", "\n".join(score["issues"])) -self.assertIn("搜索预算不符合预期", "\n".join(score["issues"])) -self.assertIn("读取了禁止深读的域名", "\n".join(score["issues"])) -self.assertIn("缺少必需决策原因", "\n".join(score["issues"])) -``` - -- [x] Run targeted test and confirm failure: - -```bash -.venv311/bin/python -m pytest test/test_agent_behavior_eval.py -q -``` - -- [x] Implement: - - Validate optional fields: - - `max_search_calls` - - `max_provider_search_calls` - - `expected_search_budgets` - - `forbidden_read_domains` - - `required_decision_reason_codes` - - Add scoring checks using observation fields: - - `search_call_count` - - `provider_search_call_count` - - `search_budgets` - - `read_domains` - - `decision_reason_codes` - - Extend fixture with one v1.2 planner sample. - -- [x] Verify: - -```bash -.venv311/bin/python -m pytest test/test_agent_behavior_eval.py -q -``` - -Expected: all tests in file pass. - -## Task 5: Tool Prompt Guardrail Test - -**Files:** -- Modify: `app/ai/tools.py` -- Test: `test/test_ai_tools.py` - -- [x] Add failing or strengthening assertions: - - tool description says second search requires complementary dimension. - - tool description says third search only for `deep_research`. - - tool description says not to repeat by translation or synonym rewrite. - -Expected assertions: - -```python -description = build_web_search_tool()["function"]["description"] -self.assertIn("第二次搜索", description) -self.assertIn("互补维度", description) -self.assertIn("第三次搜索只适用于 deep_research", description) -self.assertIn("同义改写重复搜索", description) -``` - -- [x] Run targeted test: - -```bash -.venv311/bin/python -m pytest test/test_ai_tools.py -q -``` - -- [x] Implement only if current prompt misses an assertion. If all assertions already pass, keep production prompt unchanged. - -- [x] Verify: - -```bash -.venv311/bin/python -m pytest test/test_ai_tools.py -q -``` - -Expected: all tests in file pass. - -## Task 6: Full Backend Verification And Commit - -**Files:** -- No new production file unless earlier tasks require it. - -- [x] Run targeted tests: - -```bash -.venv311/bin/python -m pytest test/services/stream/test_network_budget.py test/test_source_candidate_ranker.py test/services/stream/test_tool_round.py test/test_agent_behavior_eval.py test/test_ai_tools.py -q -``` - -Expected: pass. - -- [x] Run full backend tests: - -```bash -.venv311/bin/python -m pytest -q -``` - -Expected: pass. - -- [x] Run lint: - -```bash -.venv/bin/ruff check . -``` - -Expected: pass. - -- [x] Run dry-run behavior eval: - -```bash -.venv311/bin/python scripts/agent_behavior_eval.py --dry-run -``` - -Expected: JSONL output without exception. - -- [x] Commit with structured Chinese commit message: - -```bash -git add app/services/search_budget.py app/services/stream/network_budget.py app/services/source_candidate_ranker.py app/services/search_read_planner.py app/services/search_read_decision_ledger.py app/services/stream/tool_round.py scripts/agent_behavior_eval.py test/services/stream/test_network_budget.py test/test_source_candidate_ranker.py test/services/stream/test_tool_round.py test/test_agent_behavior_eval.py test/test_ai_tools.py test/fixtures/agent_behavior_eval_samples.json docs/superpowers/specs/2026-07-01-search-read-planner-v1-2-design.md docs/superpowers/plans/2026-07-01-search-read-planner-v1-2.md -git commit -m "feat: 增强搜索读取规划评估约束" -m "背景： -Search / Read Planner v1.1 已经完成搜索预算、重复搜索收敛和深读推荐上限，但搜索/读取决策缺少结构化解释和可回归评估，真实对话中难以判断为什么搜、为什么读这些、为什么不读剩余候选。 - -改动： -- 增加搜索预算决策和读取来源决策结构，输出稳定 action 与 reason code -- 增强候选来源推荐与未推荐原因汇总，避免只依赖自然语言 prompt -- 扩展离线行为评估字段，覆盖搜索次数、provider 搜索次数、预算、读取域名和决策原因 -- 补充 Search / Read Planner v1.2 设计与实施计划 - -Co-Authored-By: Codex " -``` - -## Task 7: CI/CD And Deployed Regression - -**Files:** -- No file edits expected unless CI exposes real issues. - -- [ ] Push once after implementation commit: - -```bash -git push -``` - -- [ ] Monitor GitHub Actions until backend deploy finishes. - -- [ ] Real Chrome regression, using existing logged-in `fusion.seanfield.org` tab only: - -| Case | Input | Expected | -| --- | --- | --- | -| SRP12-R01 | `你好，你是谁？` | 不搜索，不展示执行过程/回答依据，无 console error | -| SRP12-R02 | `1+1等于几？` | 不搜索，不展示执行过程/回答依据，无 console error | -| SRP12-R03 | `OpenAI 最近发布了哪些产品更新？` | 自主搜索，搜索关键词可见，依据可见，不泄漏 `url_read` / `reader-service` | -| SRP12-R04 | `微信A2A互通怎么用？` | 自主搜索，搜索次数不过度机械，依据和执行过程数字一致 | - -- [ ] For each regression case, record: - - case id - - input - - conversation URL - - expected - - actual - - console error count - - refresh result - - conclusion - -- [ ] If regression fails, inspect current deployed behavior and add a failing automated test before fixing. diff --git a/docs/superpowers/plans/2026-07-02-model-capability-contract-v1.md b/docs/superpowers/plans/2026-07-02-model-capability-contract-v1.md deleted file mode 100644 index 9391147a..00000000 --- a/docs/superpowers/plans/2026-07-02-model-capability-contract-v1.md +++ /dev/null @@ -1,143 +0,0 @@ -# Model Capability Contract v1 Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** 让 LiteLLM 模型能力元数据、后端运行约束和前端用户提示使用同一套能力契约。 - -**Architecture:** `fusion-api` 继续以 LiteLLM `/model/info` metadata 为事实来源，在 `litellm_catalog.normalize_capabilities()` 中派生 Fusion 运行时能力。`searchCapable` 是 v1 新增的产品语义字段，表示 Fusion 会实际下发 `web_search/url_read` 工具；旧 `webSearch` 保留为兼容别名，但由同一派生结果填充。`fusion-ui` 只消费 API 返回的契约字段，不自行猜测工具能力。 - -**Tech Stack:** Python 3.11, FastAPI, LiteLLM Proxy metadata, unittest, TypeScript, React, Vitest. - ---- - -## Test Matrix - -| Case | 层级 | 输入 | 期望 | -| --- | --- | --- | --- | -| MCC-01 | API catalog | `functionCalling=true`，未显式 `agentTools` | `agentTools=true`, `searchCapable=true`, `webSearch=true` | -| MCC-02 | API catalog | `functionCalling=false`, `agentTools=true` | `agentTools=false`, `searchCapable=false`, `webSearch=false` | -| MCC-03 | API catalog | `qwen-vl-max`, `functionCalling=true`, `vision=true` | `vision=true`, `agentTools=false`, `searchCapable=false` | -| MCC-04 | API catalog | metadata 旧字段 `webSearch=true` 但无 agent tools | 不把旧字段误解释为可联网工具 | -| MCC-05 | Runtime | `searchCapable=true` | agent loop 下发 `web_search`, `tool_choice=auto` | -| MCC-06 | Runtime | `searchCapable=false` | 不下发工具，注入无联网边界提示词 | -| MCC-07 | Runtime | `vision=true` | 图片 FileBlock 可注入 LLM | -| MCC-08 | Runtime | `vision=false` | 图片 FileBlock 不注入 LLM | -| MCC-09 | UI | `searchCapable=true` | 模型卡片/按钮展示“可联网” | -| MCC-10 | UI | `searchCapable=false` 且 `functionCalling=true` | 展示“不可联网”，不展示“工具” | -| MCC-11 | UI | `vision=false` | 上传入口和已有文件发送拦截仍提示切换视觉模型 | -| MCC-12 | Deploy smoke | `/api/models/` | 每个模型能力对象包含 `functionCalling`, `agentTools`, `searchCapable`, `webSearch`, `vision` | - -## Task 1: Backend Capability Normalization - -**Files:** -- Modify: `fusion-api/app/ai/litellm_catalog.py` -- Modify: `fusion-api/app/api/models.py` -- Test: `fusion-api/test/test_litellm_catalog.py` -- Test: `fusion-api/test/test_models.py` -- Test: `fusion-api/test/test_deployment_smoke.py` - -- [ ] **Step 1: Write failing tests** - - Add catalog tests for MCC-01 to MCC-04. - - Add `/api/models` card test asserting `searchCapable` is present and mirrors runtime tool availability. - - Add deployment smoke test requiring `searchCapable`. - -- [ ] **Step 2: Run backend target tests and confirm RED** - - Run: `.venv311/bin/python -m unittest test.test_litellm_catalog test.test_models test.test_deployment_smoke` - - Expected: fails because `searchCapable` is missing and explicit `agentTools=true` can still pass when `functionCalling=false`. - -- [ ] **Step 3: Implement normalization** - - In `normalize_capabilities()`, coerce booleans for `functionCalling`, `vision`, `deepThinking`, `fileSupport`, `imageGen`. - - Derive `agentTools` as `functionCalling && explicit_or_default_agent_tools`. - - Derive `searchCapable` as `agentTools`. - - Set `webSearch` to `searchCapable` for backwards compatibility. - - Keep denylist behavior for `qwen-vl-max`. - -- [ ] **Step 4: Expose contract through `/api/models/`** - - Add `searchCapable` to capabilities payload. - - Keep `webSearch` but source it from normalized capabilities. - -- [ ] **Step 5: Verify GREEN** - - Run: `.venv311/bin/python -m unittest test.test_litellm_catalog test.test_models test.test_deployment_smoke` - - Expected: PASS. - -## Task 2: Backend Runtime Contract - -**Files:** -- Modify: `fusion-api/app/services/stream/agent_loop_request_prep.py` -- Test: `fusion-api/test/services/stream/test_agent_loop_request_prep.py` -- Test: `fusion-api/test/services/chat/test_message_builder.py` - -- [ ] **Step 1: Write failing runtime tests** - - Add MCC-05: `searchCapable=true` enables `web_search`. - - Add MCC-06: `searchCapable=false` disables tools even if `functionCalling=true`. - - Add MCC-07/MCC-08 message builder tests for image injection gated by `has_vision`. - -- [ ] **Step 2: Run target tests and confirm RED** - - Run: `.venv311/bin/python -m unittest test.services.stream.test_agent_loop_request_prep test.services.chat.test_message_builder` - - Expected: fails where `searchCapable` is not consumed directly. - -- [ ] **Step 3: Implement runtime helper** - - Add a small helper in request prep, `supports_search_tools(capabilities)`, returning `bool(capabilities.get("searchCapable", capabilities.get("agentTools", False))) && functionCalling`. - - Use the helper for tool enablement. - - Keep no-tool boundary prompt injection behavior unchanged. - -- [ ] **Step 4: Verify GREEN** - - Run: `.venv311/bin/python -m unittest test.services.stream.test_agent_loop_request_prep test.services.chat.test_message_builder` - - Expected: PASS. - -## Task 3: Frontend Capability Presentation - -**Files:** -- Modify: `fusion-ui/src/lib/config/modelConfig.ts` -- Modify: `fusion-ui/src/lib/models/modelCapabilityPresentation.ts` -- Test: `fusion-ui/src/lib/models/modelCapabilityPresentation.test.ts` -- Test: `fusion-ui/src/components/models/ModelSelectorPanel.test.tsx` -- Test: `fusion-ui/src/components/models/ModelSelector.integration.test.tsx` - -- [ ] **Step 1: Write failing UI tests** - - Add `searchCapable` to mock models. - - Assert `searchCapable=true` shows “可联网”. - - Assert `webSearch=true` without `searchCapable` no longer makes the model look联网-capable. - -- [ ] **Step 2: Run UI target tests and confirm RED** - - Run: `npm test -- src/lib/models/modelCapabilityPresentation.test.ts src/components/models/ModelSelectorPanel.test.tsx src/components/models/ModelSelector.integration.test.tsx` - - Expected: fails before UI switches to `searchCapable`. - -- [ ] **Step 3: Implement UI contract consumption** - - Add `searchCapable?: boolean` to `ModelCapability`. - - Update `supportsAgentTools()` to prefer `searchCapable`, then fallback to `agentTools` only for old payloads. - - Keep labels and tooltip copy stable. - -- [ ] **Step 4: Verify GREEN** - - Run: `npm test -- src/lib/models/modelCapabilityPresentation.test.ts src/components/models/ModelSelectorPanel.test.tsx src/components/models/ModelSelector.integration.test.tsx` - - Expected: PASS. - -## Task 4: Verification and Release - -**Files:** -- Both repos as changed above. - -- [ ] **Step 1: Backend verification** - - Run: `.venv311/bin/python -m unittest test.test_litellm_catalog test.test_models test.test_deployment_smoke test.services.stream.test_agent_loop_request_prep test.services.chat.test_message_builder` - - Run: `DATABASE_URL=sqlite:///./fusion-test.db .venv311/bin/python -m unittest discover -s test` - - Run: `.venv/bin/ruff check .` - - Run: `.venv311/bin/python scripts/check_architecture.py` - -- [ ] **Step 2: Frontend verification** - - Run target Vitest files from Task 3. - - Run: `npm test` - - Run: `npm run build` - -- [ ] **Step 3: Commit and push** - - Commit backend and frontend separately with structured Chinese messages. - - Do not push docs-only; push after code and docs are included. - -- [ ] **Step 4: CI/CD and dev smoke** - - Monitor both GitHub Actions runs. - - Verify dev `/api/models/` contains `searchCapable` and model picker smoke still passes. - -## Self-Review - -- Spec coverage: all three requested layers are covered: source normalization, runtime constraints, and user-facing UI presentation. -- Placeholder scan: no TBD/TODO placeholders. -- Type consistency: `searchCapable` is the new canonical user/runtime search field; `webSearch` remains a compatibility alias. diff --git a/docs/superpowers/plans/2026-07-02-multi-model-real-acceptance-v1.md b/docs/superpowers/plans/2026-07-02-multi-model-real-acceptance-v1.md deleted file mode 100644 index ffaa9a13..00000000 --- a/docs/superpowers/plans/2026-07-02-multi-model-real-acceptance-v1.md +++ /dev/null @@ -1,82 +0,0 @@ -# Multi Model Real Acceptance v1 Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** 把现有多模型 runner 从“文本/搜索质量基线”升级为“按模型能力分层的真实验收矩阵”。 - -**Architecture:** 继续复用 `scripts/model_catalog_eval_baseline.py` 的独立脚本形态，不改业务 API。场景成为带能力门槛的矩阵项：脚本先从 `/api/models/` 读取能力契约，再决定某个模型是否适合运行该场景；需要图片的场景先通过 `/api/files/upload` 上传内置测试图，再用同一 `conversation_id` 调 `/api/chat/send`。结果 JSONL 增加跳过原因、附件信息和内容断言风险，summary/report 聚合能力覆盖、跳过项和风险。 - -**Tech Stack:** Python 3.11, httpx, unittest, Fusion `/api/models/`, Fusion `/api/files/upload`, Fusion `/api/chat/send`, SSE envelope. - ---- - -## Files - -- Modify: `scripts/model_catalog_eval_baseline.py` - - 增加场景能力门槛、跳过结果、图片上传、内容断言、能力覆盖 summary/report。 -- Modify: `test/test_model_catalog_eval_baseline.py` - - 增加矩阵适用性、图片上传、跳过项、内容断言和报告测试。 - -## Requirements - -- 默认场景矩阵继续覆盖基础对话、事实问答、代码/推理、自主搜索、不应搜索、长回答。 -- 新增读图正向场景：只对 `vision=true` 模型运行，自动上传内置 PNG，要求回答命中图片里的 `FUSION`。 -- 新增无读图降级场景：只对 `vision=false` 模型运行，自动上传同一张图，要求模型明确说明无法读取/理解图片，避免臆测。 -- 新增长上下文契约场景：只对 `longContext=true` 模型运行，用轻量长上下文任务验证该能力进入验收矩阵；v1 不构造 128k 超大输入，避免成本和供应商波动。 -- 场景不适用时生成 `skipped=true` 结果，而不是误记为失败；summary/report 要展示跳过数量和原因。 -- dry-run 必须显示每个模型/场景的 `eligible`、`skip_reason`、`required_capabilities` 和 `excluded_capabilities`。 -- apply 模式中有附件的场景必须先上传，再把 `file_ids` 传给 `/api/chat/send`；上传失败应归类为失败结果。 -- JSONL 不输出 auth token，不输出图片 base64。 -- 真实 Chrome 登录态回归仍作为补充记录模板保留；脚本不自动打开 Chrome。 - -## Test Matrix - -| Case | 输入 | 预期 | -| --- | --- | --- | -| MMRA-01 | 默认场景 | 包含 `vision_image_understanding`、`no_vision_image_boundary`、`long_context_contract` | -| MMRA-02 | `vision=false` 模型 + 读图正向场景 | 生成 skipped 结果，原因说明缺少 `vision` | -| MMRA-03 | `vision=true` 模型 + 无读图降级场景 | 生成 skipped 结果，原因说明不满足排除能力 | -| MMRA-04 | dry-run | 每行包含 eligible、skip_reason、附件类型和能力门槛 | -| MMRA-05 | 有图片附件场景 | 先调用 `/api/files/upload`，再调用 `/api/chat/send`，同一 conversation_id 传入 file_ids | -| MMRA-06 | 回答未包含期望关键词 | success 保持 true，但 quality_flags 增加 `expected_answer_missing` | -| MMRA-07 | 无读图降级回答未说明能力边界 | quality_flags 增加 `missing_no_vision_boundary` | -| MMRA-08 | summary | 聚合 skipped_count、by_capability_bucket、scenario_matrix | -| MMRA-09 | report | 展示跳过项、能力覆盖和真实 Chrome 补充记录模板 | - -## Tasks - -### Task 1: 场景矩阵与跳过语义 - -- [ ] 写失败测试：默认场景包含读图、无读图降级、长上下文契约。 -- [ ] 写失败测试：能力不匹配时生成 skipped 结果。 -- [ ] 实现 `required_capabilities`、`excluded_capabilities`、`build_skipped_result()` 和 dry-run 适用性输出。 -- [ ] 跑目标测试确认通过。 - -### Task 2: 图片附件真实链路 - -- [ ] 写失败测试：图片场景先上传文件，再把 file_ids 传给 chat send。 -- [ ] 实现内置 PNG 测试图和 `upload_scenario_files()`。 -- [ ] 修改 stream/nonstream 调用支持 file_ids。 -- [ ] 跑目标测试确认通过。 - -### Task 3: 内容断言与能力覆盖报告 - -- [ ] 写失败测试：答案缺期望关键词时标记 `expected_answer_missing`。 -- [ ] 写失败测试：无读图降级答案缺边界说明时标记 `missing_no_vision_boundary`。 -- [ ] 扩展 summary/report，展示 skipped、能力 bucket、场景矩阵。 -- [ ] 跑目标测试和后端关键测试。 - -### Task 4: 发布闭环 - -- [ ] `.venv311/bin/python -m unittest test.test_model_catalog_eval_baseline` -- [ ] `.venv/bin/ruff check scripts/model_catalog_eval_baseline.py test/test_model_catalog_eval_baseline.py` -- [ ] `.venv/bin/ruff format --check scripts/model_catalog_eval_baseline.py test/test_model_catalog_eval_baseline.py` -- [ ] 视改动风险运行后端全量 unittest、ruff、architecture。 -- [ ] commit + push，监控 GitHub Actions 和 dev deploy smoke。 -- [ ] 若存在可复用已登录 Chrome 标签，再按报告模板补真实 UI 回归；否则记录阻塞。 - -## Self-Review - -- Spec coverage: 自动化矩阵覆盖文本、搜索、读图、无读图降级和长上下文契约；Chrome 回归仍遵守“只复用已有标签”约束。 -- Placeholder scan: 无 TBD/TODO。 -- Type consistency: `skipped`, `skip_reason`, `required_capabilities`, `excluded_capabilities`, `attachment_kind` 贯穿 dry-run、JSONL、summary 和 report。 diff --git a/docs/superpowers/plans/2026-07-02-runtime-config-assets.md b/docs/superpowers/plans/2026-07-02-runtime-config-assets.md deleted file mode 100644 index dc8f6445..00000000 --- a/docs/superpowers/plans/2026-07-02-runtime-config-assets.md +++ /dev/null @@ -1,366 +0,0 @@ -# Runtime Config Assets Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** 统一迁移 Fusion 的产品/Agent 策略和可运营 Prompt 资产到后端运行时配置层，并让前端模型能力展示消费后端派生结果。 - -**Architecture:** LiteLLM 继续作为模型目录事实源；Fusion API 新增 `runtime_config_entries` 表和带 fallback 的 core 配置读取服务。Agent/search/read/ranker/prompt/model presentation 都从同一配置层读取默认 profile，UI 优先渲染 `/api/models/` 返回的展示结构。 - -**Tech Stack:** FastAPI + SQLAlchemy + Alembic + Python unittest；Next.js 15 + React 19 + TypeScript + Vitest。 - ---- - -## File Structure - -- Create: `fusion-api/app/services/runtime_config_defaults.py` - - 保存 `DEFAULT_AGENT_STRATEGY_CONFIG`、`DEFAULT_MODEL_PRESENTATION_CONFIG`、`DEFAULT_PROMPT_TEMPLATES`。 -- Create: `fusion-api/app/core/runtime_config.py` - - 读取 `runtime_config_entries`，做 deep merge、缓存和 fallback。 -- Create: `fusion-api/app/services/runtime_config_service.py` - - 兼容旧 import 的 re-export，避免 AI 层反向依赖 service 层。 -- Create: `fusion-api/app/services/model_presentation.py` - - 根据模型 card + 配置生成 `capabilityPresentation`。 -- Create: `fusion-api/app/services/agent_strategy_config.py` - - 提供 Agent/search/read/ranker 统一配置访问函数。 -- Modify: `fusion-api/app/db/models.py` - - 新增 `RuntimeConfigEntry` ORM。 -- Create: `fusion-api/alembic/versions/7d2f8a1c9b30_add_runtime_config_entries.py` - - 建表并 seed 默认配置。 -- Modify: `fusion-api/app/api/models.py` - - `/api/models/` card 增加 `capabilityPresentation`。 -- Modify: `fusion-api/app/ai/litellm_catalog.py` - - 保持 AI 层纯 normalize，支持由 service/API 层注入 agent tools 默认禁用名单。 -- Modify: `fusion-api/app/services/search_budget.py` - - 搜索 budget、intent keywords、阈值读取配置。 -- Modify: `fusion-api/app/services/search_read_planner.py` - - 推荐深读数量和需核验 reason 读取配置。 -- Modify: `fusion-api/app/services/source_candidate_ranker.py` - - 域名分层、权重、优先级阈值读取配置。 -- Modify: `fusion-api/app/services/stream/network_budget.py` - - 搜索/读取上限、repair、recency clamp 读取配置。 -- Modify: `fusion-api/app/services/tool_handlers/web_search.py` - - 上下文注入上限和单域名限制读取配置。 -- Modify: `fusion-api/app/services/tool_handlers/url_read.py` - - 内容截断和 reason 长度读取配置。 -- Modify: `fusion-api/app/ai/prompts/prompt_manager.py` - - Prompt 模板从 runtime config 读取，缺失 fallback。 -- Modify: `fusion-api/app/ai/prompts/agent_loop.py` - - 增加 agent prompt getter，保留常量作为 fallback。 -- Modify: `fusion-api/app/ai/tools.py` - - tool description 使用 getter。 -- Modify: `fusion-api/app/services/stream/limit_summary.py` - - limit summary prompt 使用 getter。 -- Modify: `fusion-api/app/services/agent/continuation.py` - - continuation prompt 使用 getter。 -- Modify: `fusion-api/app/services/stream/agent_loop_lifecycle.py` - - `AgentSession.config` 写入 runtime config 版本。 -- Modify: `fusion-ui/src/lib/config/modelConfig.ts` - - 类型和转换保留 `capabilityPresentation`。 -- Modify: `fusion-ui/src/lib/models/modelCapabilityPresentation.ts` - - 优先使用后端展示字段，保留本地 fallback。 -- Modify: `fusion-ui/src/components/models/ModelSelectorPanel.tsx` - - 使用后端展示字段生成能力分、标签和 tooltip。 - -## Task 1: Runtime Config DB Layer - -**Files:** -- Create: `fusion-api/test/test_runtime_config_service.py` -- Create: `fusion-api/app/services/runtime_config_defaults.py` -- Create: `fusion-api/app/services/runtime_config_service.py` -- Modify: `fusion-api/app/db/models.py` -- Create: `fusion-api/alembic/versions/7d2f8a1c9b30_add_runtime_config_entries.py` - -- [ ] **Step 1: Write failing tests** - -Cover: - -```python -def test_get_runtime_config_payload_returns_default_when_db_unavailable(): - payload, meta = get_runtime_config_payload("agent_strategy", "default", {"a": 1}, session_factory=failing_session) - assert payload == {"a": 1} - assert meta["source"] == "default" - -def test_get_runtime_config_payload_deep_merges_active_payload(): - db_payload = {"search": {"standard_budget": {"requested_count": 7}}} - payload, meta = get_runtime_config_payload("agent_strategy", "default", default, session_factory=fake_session(db_payload)) - assert payload["search"]["standard_budget"]["requested_count"] == 7 - assert payload["search"]["standard_budget"]["context_source_limit"] == default["search"]["standard_budget"]["context_source_limit"] - assert meta["source"] == "db" -``` - -- [ ] **Step 2: Verify RED** - -Run: - -```bash -cd /Users/sean/code/fusion/fusion-api -python -m pytest test/test_runtime_config_service.py -v -``` - -Expected: import/module failure because service does not exist. - -- [ ] **Step 3: Implement minimal DB layer** - -Implement: - -- `RuntimeConfigEntry` ORM. -- `deep_merge_config(base, override)`. -- `get_runtime_config_payload(namespace, key, default, session_factory=SessionLocal)`. -- `clear_runtime_config_cache()`. -- Alembic table creation and seed rows. - -- [ ] **Step 4: Verify GREEN** - -Run: - -```bash -python -m pytest test/test_runtime_config_service.py -v -``` - -Expected: PASS. - -## Task 2: Backend Model Presentation - -**Files:** -- Modify: `fusion-api/test/test_models.py` -- Create: `fusion-api/test/test_model_presentation.py` -- Create: `fusion-api/app/services/model_presentation.py` -- Modify: `fusion-api/app/api/models.py` - -- [ ] **Step 1: Write failing tests** - -Cover: - -```python -def test_entry_to_card_includes_backend_capability_presentation(): - card = _entry_to_card("deepseek-chat", entry) - presentation = card["capabilityPresentation"] - assert presentation["score"] >= 70 - assert any(label["text"] == "可联网" for label in presentation["labels"]) - assert "可按问题需要自主联网" in presentation["tooltip"] - -def test_model_presentation_uses_configured_weights(): - presentation = build_model_capability_presentation(card, config={"weights": {"base": 10, "network": 80}}) - assert presentation["score"] == 90 -``` - -- [ ] **Step 2: Verify RED** - -Run: - -```bash -python -m pytest test/test_models.py test/test_model_presentation.py -v -``` - -Expected: missing `capabilityPresentation` / missing module failure. - -- [ ] **Step 3: Implement presentation service** - -Implement backend equivalent of current UI scoring, labels and tooltip, driven by `model_presentation/default`. - -- [ ] **Step 4: Verify GREEN** - -Run: - -```bash -python -m pytest test/test_models.py test/test_model_presentation.py -v -``` - -Expected: PASS. - -## Task 3: Agent Strategy Config Consumption - -**Files:** -- Create: `fusion-api/test/test_agent_strategy_config.py` -- Modify: `fusion-api/test/test_litellm_catalog.py` -- Modify: `fusion-api/test/services/stream/test_network_budget.py` -- Modify: `fusion-api/test/test_source_candidate_ranker.py` -- Create: `fusion-api/app/services/agent_strategy_config.py` -- Modify: strategy consumer modules listed above. - -- [ ] **Step 1: Write failing tests** - -Cover: - -```python -def test_litellm_catalog_uses_runtime_disabled_agent_tool_aliases(): - with patch_config({"model_runtime": {"agent_tools_disabled_aliases": ["deepseek-chat"]}}): - capabilities = normalize_capabilities("deepseek-chat", {"functionCalling": True}) - assert capabilities["agentTools"] is False - -def test_network_budget_uses_configured_standard_budget(): - with patch_config({"search": {"standard_budget": {"requested_count": 7, "context_source_limit": 6}}}): - args, degraded = NetworkToolBudget().prepare_web_search_args({"query": "redis"}) - assert degraded is None - assert args["count"] == 7 - assert args["context_source_limit"] == 6 - -def test_ranker_uses_configured_authority_domain(): - with patch_config({"source_ranker": {"authority_media_domains": ["example.com"]}}): - plan = rank_search_sources([...]) - assert "权威媒体" in plan.recommended[0].reasons -``` - -- [ ] **Step 2: Verify RED** - -Run: - -```bash -python -m pytest test/test_litellm_catalog.py test/services/stream/test_network_budget.py test/test_source_candidate_ranker.py -v -``` - -Expected: new override tests fail because consumers still use hardcoded constants. - -- [ ] **Step 3: Implement config consumption** - -Move tunable values into `DEFAULT_AGENT_STRATEGY_CONFIG` and replace direct constants with config-backed helper reads. Keep validation conservative and fallback to existing defaults. - -- [ ] **Step 4: Verify GREEN** - -Run: - -```bash -python -m pytest test/test_litellm_catalog.py test/services/stream/test_network_budget.py test/test_source_candidate_ranker.py test/test_agent_strategy_config.py -v -``` - -Expected: PASS. - -## Task 4: Prompt Template Runtime Adapter - -**Files:** -- Create: `fusion-api/test/test_prompt_runtime_templates.py` -- Modify: `fusion-api/app/ai/prompts/prompt_manager.py` -- Modify: `fusion-api/app/ai/prompts/agent_loop.py` -- Modify: `fusion-api/app/ai/tools.py` -- Modify: `fusion-api/app/services/stream/limit_summary.py` -- Modify: `fusion-api/app/services/agent/continuation.py` -- Update affected prompt tests. - -- [ ] **Step 1: Write failing tests** - -Cover: - -```python -def test_prompt_manager_uses_runtime_template_override(): - with patch_prompt("generate_title", "标题：{content}"): - assert prompt_manager.format_prompt("generate_title", content="abc") == "标题：abc" - -def test_tool_description_uses_runtime_prompt_override(): - with patch_prompt("url_read_tool_description", "读取网页：{policy}"): - assert "读取网页" in URL_READ_TOOL["function"]["description"] -``` - -- [ ] **Step 2: Verify RED** - -Run: - -```bash -python -m pytest test/test_prompt_runtime_templates.py test/test_ai_tools.py test/services/stream/test_limit_summary.py test/services/agent/test_continuation.py -v -``` - -Expected: override tests fail. - -- [ ] **Step 3: Implement prompt adapter** - -Add prompt template getters, keep current constants as fallback, and avoid changing public prompt names. - -- [ ] **Step 4: Verify GREEN** - -Run: - -```bash -python -m pytest test/test_prompt_runtime_templates.py test/test_ai_tools.py test/services/stream/test_limit_summary.py test/services/agent/test_continuation.py -v -``` - -Expected: PASS. - -## Task 5: Frontend Consumes Backend Presentation - -**Files:** -- Modify: `fusion-ui/src/lib/config/modelConfig.test.ts` -- Modify: `fusion-ui/src/lib/models/modelCapabilityPresentation.test.ts` -- Modify: `fusion-ui/src/components/models/ModelSelectorPanel.test.tsx` -- Modify: `fusion-ui/src/lib/config/modelConfig.ts` -- Modify: `fusion-ui/src/lib/models/modelCapabilityPresentation.ts` -- Modify: `fusion-ui/src/components/models/ModelSelectorPanel.tsx` - -- [ ] **Step 1: Write failing tests** - -Cover: - -```ts -it('preserves backend capabilityPresentation from /api/models', () => { - const model = convertApiModelToModelInfo(apiModelWithPresentation); - expect(model.capabilityPresentation?.score).toBe(88); -}); - -it('prefers backend capabilityPresentation over local fallback', () => { - const recommendation = buildModelCapabilityRecommendation(modelWithPresentation); - expect(recommendation.score).toBe(88); - expect(recommendation.headline).toBe('后端推荐文案'); -}); -``` - -- [ ] **Step 2: Verify RED** - -Run: - -```bash -cd /Users/sean/code/fusion/fusion-ui -npm test -- src/lib/config/modelConfig.test.ts src/lib/models/modelCapabilityPresentation.test.ts src/components/models/ModelSelectorPanel.test.tsx -``` - -Expected: missing type/field failure. - -- [ ] **Step 3: Implement frontend consumption** - -Add `CapabilityPresentation` types, persist API field, and make presentation helpers prefer backend values with fallback. - -- [ ] **Step 4: Verify GREEN** - -Run: - -```bash -npm test -- src/lib/config/modelConfig.test.ts src/lib/models/modelCapabilityPresentation.test.ts src/components/models/ModelSelectorPanel.test.tsx -``` - -Expected: PASS. - -## Task 6: Final Validation - -**Files:** -- All touched files. - -- [ ] **Step 1: Run API targeted tests** - -```bash -cd /Users/sean/code/fusion/fusion-api -python -m pytest test/test_runtime_config_service.py test/test_model_presentation.py test/test_models.py test/test_litellm_catalog.py test/services/stream/test_network_budget.py test/test_source_candidate_ranker.py test/test_prompt_runtime_templates.py test/test_ai_tools.py test/services/stream/test_limit_summary.py test/services/agent/test_continuation.py -v -``` - -- [ ] **Step 2: Run API static checks** - -```bash -ruff check app test -``` - -- [ ] **Step 3: Run UI targeted tests** - -```bash -cd /Users/sean/code/fusion/fusion-ui -npm test -- src/lib/config/modelConfig.test.ts src/lib/models/modelCapabilityPresentation.test.ts src/components/models/ModelSelectorPanel.test.tsx -``` - -- [ ] **Step 4: Run UI build if targeted tests pass** - -```bash -npm run build -``` - -- [ ] **Step 5: Commit both repos, push once, monitor CI/CD** - -Use Chinese structured commit body with `背景：` and `改动：` sections. Do not push docs-only separately. - -- [ ] **Step 6: Dev real regression after deployment** - -Use only an already-open logged-in Chrome tab. If none exists, record blocker instead of opening a new tab. diff --git a/docs/superpowers/plans/2026-07-02-runtime-config-governance.md b/docs/superpowers/plans/2026-07-02-runtime-config-governance.md deleted file mode 100644 index aaaca76f..00000000 --- a/docs/superpowers/plans/2026-07-02-runtime-config-governance.md +++ /dev/null @@ -1,85 +0,0 @@ -# Runtime Config 治理闭环 v1 Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** 给 runtime config 增加后端治理闭环，确保配置可查、可校验、可禁用并能安全回退。 - -**Architecture:** 在 `app.core` 放轻量 schema 校验和主链路读取保护；在 `app.services` 放治理快照、校验、安全写入和安全激活；`app.api.admin` 只做管理员路由和请求模型。 - -**Tech Stack:** FastAPI、SQLAlchemy ORM、pytest/unittest、现有 `runtime_config_entries` 表。 - ---- - -### Task 1: 主链路 schema 校验和坏版本回退 - -**Files:** -- Create: `app/core/runtime_config_schema.py` -- Modify: `app/core/runtime_config.py` -- Test: `test/test_runtime_config_service.py` - -- [x] 写失败测试：最新 active prompt payload 类型错误时跳过，使用上一条有效 active 版本。 -- [x] 写失败测试：所有 active model presentation payload 无效时回退代码默认值。 -- [x] 写失败测试：schema 校验返回可读问题。 -- [x] 实现 `validate_runtime_config_payload()`。 -- [x] 修改 `get_runtime_config_payload()`，最多检查 10 条 active 候选并记录 `skipped_versions`。 -- [x] 聚焦测试通过。 - -### Task 2: 治理 service - -**Files:** -- Create: `app/services/runtime_config_governance.py` -- Test: `test/test_runtime_config_governance.py` - -- [x] 写失败测试：治理快照列出 entries 和 effective version。 -- [x] 写失败测试：active 状态切换会更新 row、commit、refresh、清理缓存。 -- [x] 实现 `build_runtime_config_snapshot()`。 -- [x] 实现 `validate_runtime_config_candidate()`。 -- [x] 实现 `set_runtime_config_entry_active()`。 -- [x] 聚焦测试通过。 - -### Task 3: Admin API - -**Files:** -- Modify: `app/api/admin.py` -- Test: `test/test_runtime_config_governance_api.py` - -- [x] 写失败测试：非管理员访问 runtime config 返回 403。 -- [x] 写失败测试：管理员可读取治理快照。 -- [x] 写失败测试：管理员可无写入校验 payload。 -- [x] 写失败测试：管理员可切换条目 active 状态。 -- [x] 实现 `GET /api/admin/runtime-config`。 -- [x] 实现 `POST /api/admin/runtime-config/validate`。 -- [x] 实现 `PATCH /api/admin/runtime-config/{entry_id}/status`。 -- [x] 聚焦测试通过。 - -### Task 4: 安全写入和安全激活 - -**Files:** -- Modify: `app/services/runtime_config_governance.py` -- Modify: `app/api/admin.py` -- Test: `test/test_runtime_config_governance.py` -- Test: `test/test_runtime_config_governance_api.py` - -- [x] 写失败测试：创建新版本会先校验 payload，通过后写入 inactive 版本。 -- [x] 写失败测试：非法 payload 返回 `INVALID_PARAM`，不 add、不 commit。 -- [x] 写失败测试：重复 `(namespace,key,version)` 返回 `CONFLICT`，不写入。 -- [x] 写失败测试：激活版本时关闭同一 `namespace/key` 的旧 active 版本。 -- [x] 写失败测试：坏版本不可激活，旧 active 状态保持不变。 -- [x] 写失败测试：旧 `status=true` 入口复用安全激活语义。 -- [x] 实现 `create_runtime_config_entry()`。 -- [x] 实现 `activate_runtime_config_entry()`。 -- [x] 实现 `POST /api/admin/runtime-config`。 -- [x] 实现 `POST /api/admin/runtime-config/{entry_id}/activate`。 -- [x] 聚焦测试通过。 - -### Task 5: 验证和发布 - -**Files:** -- Modify only files above and docs. - -- [x] 运行架构检查：`/opt/homebrew/bin/python3.11 scripts/check_architecture.py` -- [x] 运行 lint：`/opt/homebrew/bin/python3.11 -m ruff check .` -- [x] 运行全量测试：`/opt/homebrew/bin/python3.11 -m pytest` -- [x] 检查 git diff，确认未提交无关文件。 -- [ ] 中文结构化 commit，包含 `Co-Authored-By: Codex `。 -- [ ] push 后监控 GitHub Actions 和 dev 部署门禁。 diff --git a/docs/superpowers/plans/2026-07-10-prompthub-migration.md b/docs/superpowers/plans/2026-07-10-prompthub-migration.md deleted file mode 100644 index dc4017fd..00000000 --- a/docs/superpowers/plans/2026-07-10-prompthub-migration.md +++ /dev/null @@ -1,42 +0,0 @@ -# Fusion PromptHub 正式迁移实施计划 - -## Phase 1：恢复 PromptHub 基线 - -- [ ] 修复 Prompt 分享 API 直接 `db.flush()` 的架构违规并补测试。 -- [ ] 为 PromptHub CI 增加 backend、SDK、ruff、架构和 Alembic 校验。 -- [ ] 保持 dev 已移除 8200 暴露的状态，不覆盖用户备份文件。 -- [ ] 部署 GitHub master，验证容器运行提交、health 和 smoke。 - -## Phase 2：PromptHub 消费者契约与安全 - -- [ ] 先写 published bundle 服务/API 失败测试。 -- [ ] 实现 current published 版本批量读取、稳定 revision 和整包失败语义。 -- [ ] 新增 project-bound、hash-only、`prompts:read` service token。 -- [ ] 补 401、403、跨项目、写接口拒绝和 token 不泄漏测试。 -- [ ] SDK 增加同步/异步 published bundle 读取与类型。 -- [ ] 修正 publish 后 `Prompt.content/variables` 快照语义。 - -## Phase 3：迁入 Fusion 11 个 Prompt - -- [ ] 导出 Fusion dev 实际 active Prompt、变量和 SHA-256。 -- [ ] 编写 dry-run/apply、幂等导入脚本，通过 PromptHub 服务/API 创建 `fusion` 项目和 11 个 Prompt。 -- [ ] 创建独立 Fusion service token，不复用 admin key。 -- [ ] 核对 PromptHub bundle 与 Fusion active 内容字节级一致。 - -## Phase 4：Fusion shadow 同步 - -- [ ] 先写 client 的成功、超时、401、404、5xx 和坏 JSON 测试。 -- [ ] 先写 bundle 完整性、变量、marker、checksum 和幂等同步测试。 -- [ ] 实现轻量 httpx client、PromptSpec/Catalog、LKG bundle 和原子同步服务。 -- [ ] 增加启动 best-effort 和周期同步；失败不影响健康启动。 -- [ ] 增加 `disabled|shadow|apply` 门禁、同步诊断和版本观测。 -- [ ] 部署 `shadow`，验证 11 项零漂移且聊天热路径无远程请求。 - -## Phase 5：apply 切换与收尾 - -- [ ] 切换 `apply`，验证 bundle 原子激活、相同 revision 幂等和旧版本可回滚。 -- [ ] 限制 Fusion 本地 `prompt_template` 写入口，避免双事实源。 -- [ ] 运行 PromptHub backend/SDK 全量测试和 Fusion 全量测试。 -- [ ] 推送并跟踪两仓 CI/CD、迁移、health、smoke 和容器提交。 -- [ ] 复用用户现有登录态 Fusion Chrome 标签执行验收矩阵。 -- [ ] 更新 Fusion 双仓执行台账、架构文档和 PromptHub 接入文档。 diff --git a/docs/superpowers/plans/2026-07-11-admin-audit-center-v1.md b/docs/superpowers/plans/2026-07-11-admin-audit-center-v1.md deleted file mode 100644 index 26e476a0..00000000 --- a/docs/superpowers/plans/2026-07-11-admin-audit-center-v1.md +++ /dev/null @@ -1,42 +0,0 @@ -# 管理员审计中心 v1 Implementation Plan - -## Goal - -交付独立、只读、可审计的管理员内容观察面和压测汇总留存，保持普通用户聊天所有权、缓存和流式链路不变。 - -## Task 1：数据与安全骨架 - -- [x] 先补脱敏器、审计 fail-closed、性能导入校验和模型约束的失败测试。 -- [x] 新增 `AdminAuditEvent`、`PerformanceRun` 模型与 Alembic 迁移。 -- [x] 新增关键查询索引和非破坏 `agent_steps` FK 约束。 -- [x] 实现递归脱敏、长度上限与邮箱遮蔽。 - -## Task 2：独立管理员查询层 - -- [x] 新增 `AdminAuditRepository`，实现用户/对话分页与聚合，禁止复用普通用户所有权接口。 -- [x] 新增 `AdminAuditService`，编排脱敏、审计、404 和压测幂等导入。 -- [x] 新增 `/api/admin/audit/*` 路由、Pydantic schemas、`no-store` 响应和独立 auditor 依赖。 -- [x] 覆盖权限、分页、组合筛选、消息、工具、Agent、文件元数据、审计与性能导入 API 测试。 - -## Task 3：管理员前端协议与入口 - -- [x] 先补 API 编码、管理员菜单入口、AdminGuard 和只读约束的失败测试。 -- [x] 新增 `adminAudit` types/API client。 -- [x] 在管理员头像菜单增加“管理中心”；普通用户不展示。 -- [x] 新增独立 `/admin` 页面和 noindex 元数据。 - -## Task 4：管理员审计中心 UI - -- [x] 实现用户、对话、压测、访问审计四个 tab。 -- [x] 实现组合筛选、稳定分页、加载/空/错误状态和旧请求取消。 -- [x] 实现专用只读消息、Agent/tool 和文件元数据视图，不写普通聊天 Redux/Dexie。 -- [x] 实现压测 JSON 导入与已留存结果详情。 - -## Task 5：验证与发布 - -- [x] 后端目标测试、全量 pytest、Ruff、架构检查和 Alembic 单 head。 -- [x] 前端目标 Vitest、全量 Vitest、ESLint 和生产 build。 -- [x] 安全复核响应不含凭据、路径、签名 token 和 raw provider payload。 -- [x] 分仓中文提交并 push，监控 CI/CD、迁移、部署 health/smoke。 -- [x] 复用已打开的真实 Fusion 管理员 Chrome 标签，以新建普通用户数据完成生产双轨验收。 -- [x] 通过页面导入 2026-07-11 首轮生产压测基线，并验证清理后的聊天不存在但汇总可见。 diff --git a/docs/superpowers/plans/2026-07-12-full-production-performance-run.md b/docs/superpowers/plans/2026-07-12-full-production-performance-run.md deleted file mode 100644 index f76e55db..00000000 --- a/docs/superpowers/plans/2026-07-12-full-production-performance-run.md +++ /dev/null @@ -1,37 +0,0 @@ -# Fusion 生产全链路压测闭环计划 - -## 目标 - -在唯一生产环境补齐 L1-L4 性能矩阵，保留可复跑、默认脱敏、带硬停止和精确清理的 runner，并把最终安全摘要导入管理员审计中心。 - -## 环境与门禁 - -- 目标：`https://fusion.seanfield.org`，必须显式确认生产。 -- API 当前限制：1 vCPU、1 GiB、单 Uvicorn worker。 -- HTTP：至少 20 样本后错误率或超时率达到 5%，或连续失败达到 10 次，停止升压。 -- SSE：任一 flow 失败、error frame、清理失败，停止升压。 -- 主机/API/PostgreSQL/Redis 出现 OOM、容器重启、Redis rejected connection/eviction 增量，立即停止。 -- 所有密码、access token、refresh token、邮箱和响应正文只存在进程内存，不写结果。 - -## 用例矩阵 - -| 层级 | 场景 | 负载 | 核心指标 | 通过条件 | -|---|---|---:|---|---| -| L1 | 登录 | c10→25→50 | RPS、P50/P95/P99、错误率、DB 连接 | 0 超时；错误率 <5%；无资源硬门禁 | -| L1 | 模型列表 | c10→25→50 | 同上 | 同上 | -| L1 | 会话列表 | c10→25→50 | 同上 | 同上 | -| L1 | 会话详情 | c10→25→50 | 同上 | 同上 | -| L2 | 短流式对话 | c1→3→5→10 | TTFT、完成耗时、chunk 间隔、输出速率、完成率 | 100% 完成；0 error frame | -| L2 | 长流式对话 | c1→3→5 | 同上 | 100% 完成；0 error frame | -| L3 | 状态恢复/断线重连 | c1→3→5 | 恢复成功率、重复/丢失、恢复耗时 | 100% 恢复；0 重复/丢失 | -| L3 | 停止生成 | c1→3→5 | 停止成功率、停止延迟、终态/残留 | 100% 停止；无活跃任务残留 | -| L4 | 低并发稳态 | c2，30 分钟 | 窗口错误率、TTFT、内存斜率、连接/Swap、错误累积 | 无错误累积、重启、OOM、连接泄漏 | - -## 实施顺序 - -1. 先补 runner 的场景、指标和纯函数测试，不启动本地 Fusion 服务。 -2. 生产运行前记录数据库计数、容器状态和 Prometheus 基线。 -3. 逐层运行；每层结束检查应用日志与资源指标，满足门禁才继续。 -4. `finally` 精确删除压测会话并吊销全部 refresh token；再按 `run_id` 精确清理 Fusion/auth 一次性用户和登录日志。 -5. 生成脱敏 JSON 与 Markdown 报告，导入管理员审计中心，在已打开且已登录的 Fusion Chrome 标签验证持久化和页面可见性。 -6. 更新执行台账，运行测试、Ruff、架构检查，提交、push 并跟踪 CI/CD。 diff --git a/docs/superpowers/plans/2026-07-28-litellm-sustainable-governance.md b/docs/superpowers/plans/2026-07-28-litellm-sustainable-governance.md deleted file mode 100644 index 3a90e077..00000000 --- a/docs/superpowers/plans/2026-07-28-litellm-sustainable-governance.md +++ /dev/null @@ -1,46 +0,0 @@ -# LiteLLM 可持续治理实施计划 - -## 阶段一：升级门禁 - -1. 定位并修复 dev 二级备份失败，保留修改前配置。 -2. 对 LiteLLM 数据库备份执行完整性检查和隔离恢复演练。 -3. 为 Compose 增加 readiness healthcheck、启动宽限期和失败重试。 -4. 新增脱敏的升级前检查脚本，输出备份、数据库、模型目录、调度状态和容器健康结果。 - -## 阶段二：受控升级 - -1. 固定 `v1.93.0` tag 和镜像 digest。 -2. 使用数据库副本启动隔离实例，验证累计 Prisma migration。 -3. 对账 DB models、virtual key、allowlist 和成本记录。 -4. 执行目录、SSE、reasoning、tool calling、vision 与 usage/cost 验收。 -5. 只有隔离证据全部通过后才进入 dev 滚动升级。 - -## 阶段三：成本表同步 - -1. 先执行一次手动刷新并校验状态。 -2. 配置每 6 小时刷新。 -3. 同步保存官方成本表快照及 SHA-256、ETag、抓取时间和模型数。 -4. 新增状态检查，异常时非零退出并进入现有监控。 -5. 验证上游未知模型不会生成虚假价格。 - -## 阶段四：候选发现与准入 - -1. 新增 provider adapter 和只读候选报告。 -2. 支持 Moonshot 与标准 OpenAI-compatible `/v1/models`。 -3. 用官方成本表和受审 override 富化候选；证据不全时保持隔离。 -4. 通过逐 provider 的 LiteLLM wildcard 路由执行候选预准入文本、流式、工具和 usage/cost 验收；共享 `openai/*` adapter 的厂商按 alias、endpoint 和 key 隔离。 -5. 通过预准入后才允许生成 dry-run `/model/new` 和 allowlist 变更计划。 -6. 实际注册后复用现有 Fusion 全模型验收脚本完成产品门禁。 -7. 首版保持人工批准；积累稳定证据后再评估自动批准策略。 - -## 代码验证 - -- 目标单元测试； -- Ruff 检查与格式检查； -- `git diff --check`； -- 脱敏输出检查； -- dry-run 不产生网络写操作。 - -## 发布边界 - -当前授权允许实现、测试和 dev 主机上的可回滚准备工作；不包含生产部署、密钥轮换、敏感权限修改或绕过发布门禁。涉及实际 dev LiteLLM 容器升级时，先完成隔离迁移和备份恢复证据。 diff --git a/docs/superpowers/plans/2026-08-22-trajectory-ledger-async-queue.md b/docs/superpowers/plans/2026-08-22-trajectory-ledger-async-queue.md deleted file mode 100644 index 4a74be0d..00000000 --- a/docs/superpowers/plans/2026-08-22-trajectory-ledger-async-queue.md +++ /dev/null @@ -1,186 +0,0 @@ -# Trajectory P0 异步队列修复 Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development to implement this plan task-by-task. - -**Goal:** 将 Trajectory P0 从 emitter 锁内同步等待 PostgreSQL，切换为每个 run 的有界异步队列，使事件接纳路径满足 p95 ≤ 5ms、p99 ≤ 15ms，同时保持既有脱敏、顺序、降级与 durable terminal intent 正确性。 - -**Architecture:** 保留已审查的 `TrajectoryRecorder` 作为同步、耐久的落库核心；新增组合式 `QueuedTrajectoryRecorder` 作为生产入口。每个 run 由一个实例和一个单消费者队列保证顺序，`record_chunk()` 只做有界非阻塞接纳，`finalize()` 关闭接纳、执行有界 flush，再调用同步核心完成既有终态握手。队满、flush 超时或 worker 异常统一进入核心的 degraded latch，绝不阻塞 emitter，也不把失败 run 标为 complete。 - -**Tech Stack:** Python 3.12、asyncio、SQLAlchemy、pytest/unittest、Ruff - -**Spec:** `docs/TRAJECTORY_DESIGN.md` - -## Global Constraints - -- 每个 run 的队列容量必须恰好为 `1000`。 -- `AgentEventCompositeWriter` 调用的生产 `record_chunk()` 路径不得等待数据库写入。 -- 每个 run 只允许一个后台消费者，必须按接纳顺序严格写入 sequence；跨 run 继续由核心线程池并发。 -- 队满必须 fail-open：标记 `degraded/admission_full`、丢弃新事件、立即返回，不得阻塞 emitter 锁。 -- `finalize()` 必须先关闭新接纳，再用不超过 `10.0` 秒的 flush barrier 等待已接纳事件定论，最后调用同步核心 `finalize(expected_last_sequence)`。 -- flush 超时必须标记 `degraded/recorder_timeout`；取消必须标记 `degraded/recorder_cancelled` 并重新抛出 `CancelledError`；worker 异常必须标记 `degraded/write_failed`。 -- 任何失败或丢事件路径都不得声明 `complete`，同步核心现有 durable terminal intent、CAS、late future 与 terminal ack 状态机不得重写。 -- 包装器必须透传 `run_id`、`conversation_id`、`message_id`、`degraded_reason`、`degraded_latch()`、`pending_terminal_reconciliation`。 -- 不新增数据库迁移，不改变事件协议、payload allowlist、SSE 或 UI。 -- 测试必须先 RED 后 GREEN，并保留命令、失败原因与通过输出。 -- 不启动本地 Fusion 服务；验证使用测试、CI、已授权的 dev 部署与已打开的登录态 Chrome 标签。 - -### Ruling 1：flush 超时取 10.0 秒 - -容量 1000 × dev 同步单写 p50 约 7.4ms，正常最坏 flush 约 7.4 秒；10 秒给出有界余量且不会无限拖住终态。如果估计错误，代价是极端积压 run 被保守标为 degraded，而不是错误 complete。 - -### Ruling 2：包装同步核心，不改写核心状态机 - -`TrajectoryRecorder` 已覆盖 durable terminal intent、迟到事务、取消、permit 守恒与 CAS 竞态；异步化只改变接纳/flush 边界。如果该边界判断错误，代价是需要把队列生命周期进一步并入核心，但当前最小改动避免重新验证 1500 行终态逻辑。 - -## Task 1: 实现每个 run 的有界异步接纳与生产装配 - -**Files:** - -- Create: `app/services/agent/queued_trajectory_recorder.py` -- Create: `test/services/agent/test_queued_trajectory_recorder.py` -- Modify: `app/services/stream/agent_loop_execution.py` -- Modify: `test/services/stream/test_agent_loop_execution.py` -- Test: `test/services/stream/test_agent_loop_lifecycle.py` - -**Step 1: 写失败测试并确认 RED** - -新增真实行为测试： - -1. 同步核心 `record_chunk()` 被慢写阻塞时，包装器的 `record_chunk()` 与 `AgentEventCompositeWriter.append_chunk()` 仍快速返回，证明 emitter 路径不等待数据库。 -2. `finalize()` 在所有已接纳事件按 sequence 顺序写完后才调用核心 finalize。 -3. 容量为 1 的测试实例在 worker 忙时队满，后续接纳立即返回并令核心 latch 为 `admission_full`。 -4. flush 超时令核心 latch 为 `recorder_timeout`，取消令 latch 为 `recorder_cancelled` 并重新抛出；测试结束没有泄漏后台任务。 -5. production assembly 创建 `QueuedTrajectoryRecorder(TrajectoryRecorder(...))`，writer、completion context 与 execution 共用同一包装器。 - -先运行： - -```bash -/Users/sean/code/fusion/fusion-api/.venv/bin/python -m pytest \ - test/services/agent/test_queued_trajectory_recorder.py \ - test/services/stream/test_agent_loop_execution.py -q -``` - -确认失败是包装器或装配尚不存在，而不是 fixture/导入错误。 - -**Step 2: 最小实现包装器** - -包装器要求： - -- 默认常量 `TRAJECTORY_QUEUE_SIZE = 1000`、`TRAJECTORY_FLUSH_TIMEOUT_SECONDS = 10.0`。 -- 构造参数接受同步核心，以及仅供测试覆盖的 `queue_size`、`flush_timeout_seconds`。 -- `record_chunk()` 过滤 conversation/chunk type，关闭或核心已 degraded 时直接返回；用 `put_nowait` 非阻塞接纳，并惰性创建单个消费者 task。 -- 消费者顺序 `await inner.record_chunk(...)`；发现核心 degraded 后继续排空但不再写入，确保 finalize 可收口。 -- 队满调用核心既有降级接口，原因 `admission_full`。 -- 消费者异常调用核心既有降级接口，原因 `write_failed`，并保存异常状态供 barrier 感知。 -- `finalize()` 先原子关闭接纳，再等待消费者处理完队列；用 `asyncio.wait_for(asyncio.shield(...), 10.0)` 做 flush barrier。超时取消消费者并标 `recorder_timeout`；当前任务被取消时标 `recorder_cancelled`、清理消费者后重新抛出；无论成功或降级，最后调用核心 finalize，只有外部取消不吞异常。 -- 提供只读 `inner` 以便装配测试确认组合关系；其余公共字段/属性按 Global Constraints 透传。 - -**Step 3: 改生产装配** - -`_build_execution_parts()` 先构造同步 `TrajectoryRecorder`，再包装为 `QueuedTrajectoryRecorder`；更新相关 dataclass 类型标注与装配断言。不要修改 `AgentEventCompositeWriter` 的 sink 顺序。 - -**Step 4: 运行聚焦测试并确认 GREEN** - -```bash -/Users/sean/code/fusion/fusion-api/.venv/bin/python -m pytest \ - test/services/agent/test_queued_trajectory_recorder.py \ - test/services/stream/test_agent_loop_execution.py \ - test/services/stream/test_agent_loop_lifecycle.py -q -``` - -**Step 5: 回归同步核心** - -```bash -/Users/sean/code/fusion/fusion-api/.venv/bin/python -m pytest \ - test/services/agent/test_trajectory_recorder.py \ - test/services/agent/test_trajectory_recorder_hardening.py \ - test/services/agent/test_trajectory_terminal_protocol.py -q -``` - -**Step 6: 自审并提交** - -确认无未捕获 task 异常、无取消吞噬、无协议/迁移/UI 改动。提交信息： - -```text -fix: 轨迹账本改为有界异步接纳 -``` - -提交必须包含 `Co-Authored-By: Codex `。 - -## Task 2: 将性能基准切到生产包装器并固化切换结论 - -**Files:** - -- Modify: `scripts/trajectory_p0_baseline.py` -- Modify: `test/services/agent/test_trajectory_performance.py` -- Modify: `docs/TRAJECTORY_DESIGN.md` - -**Step 1: 写失败测试并确认 RED** - -将性能 runner/test 的 `trajectory_stub` 改为生产 `QueuedTrajectoryRecorder` 路径，并新增断言： - -- 250 次 CompositeWriter 接纳 p95 ≤ 5ms、p99 ≤ 15ms，即使同步核心每次写等待 20ms。 -- runner 每轮结束显式 finalize，确保后台 task 不泄漏。 -- 失败 sink 不影响 Redis/progress 已可见顺序。 - -运行： - -```bash -/Users/sean/code/fusion/fusion-api/.venv/bin/python -m pytest test/services/agent/test_trajectory_performance.py -q -``` - -确认当前 runner 仍使用同步核心，慢写路径导致门限失败。 - -**Step 2: 修改 runner 与测试** - -runner 必须使用生产包装器，且每轮使用唯一 run_id，采样后调用 `finalize(samples_per_path - 1)`。测试可以用可控慢核心隔离数据库，但测量对象必须是 `AgentEventCompositeWriter → QueuedTrajectoryRecorder.record_chunk()` 的生产接纳边界。 - -**Step 3: 更新设计文档** - -只更新与本次已触发切换直接相关的文字： - -- §2.1/§5.1/§5.4 将“后续升级/v1 不实现”改为 P0 当前实现。 -- 明确包装同步核心、容量 1000、单 run 单消费者、10 秒 flush barrier、队满/超时/取消/worker 异常语义。 -- §9.1 记录 dev 同步基线：200 事件，p50 7.428ms、p95 10.781ms、p99 11.124ms，触发切换；不得把本地 stub 当真实 DB 基线。 -- P0 范围表注明异步接纳已启用。 - -不要扩写 P1/P3。 - -**Step 4: 验证并提交** - -```bash -/Users/sean/code/fusion/fusion-api/.venv/bin/python -m pytest test/services/agent/test_trajectory_performance.py -q -/Users/sean/code/fusion/fusion-api/.venv/bin/python -m ruff check \ - app/services/agent/queued_trajectory_recorder.py \ - app/services/stream/agent_loop_execution.py \ - scripts/trajectory_p0_baseline.py \ - test/services/agent/test_queued_trajectory_recorder.py \ - test/services/agent/test_trajectory_performance.py \ - test/services/stream/test_agent_loop_execution.py -``` - -提交信息： - -```text -docs: 固化轨迹异步队列性能门槛 -``` - -提交必须包含 `Co-Authored-By: Codex `。 - -## Branch-wide verification - -所有任务与逐任务审查通过后，由控制 Agent 运行： - -```bash -/Users/sean/code/fusion/fusion-api/.venv/bin/python -m pytest test/ -q -/Users/sean/code/fusion/fusion-api/.venv/bin/python -m ruff check . -/Users/sean/code/fusion/fusion-api/.venv/bin/python -m ruff format --check . -git diff --check origin/master...HEAD -``` - -然后进行 whole-branch 独立审查；审查通过后 push、创建 PR、监控 CI、合并到 `master`、等待 dev 发布成功，并在 dev 重跑真实 PostgreSQL 200 事件包装器基准与至少一条真实模型工具链路。dev 验收必须确认： - -- 生产镜像 SHA 等于新 merge commit，health 与 Alembic head 正常。 -- 包装器接纳 p95 ≤ 5ms、p99 ≤ 15ms；finalize 后 `complete`、event_count=200、expected_last_sequence=199、sequence=0..199。 -- 相对基线使用本次同环境同步核心 p95 10.781ms 对比，不伪造缺失的历史端到端多轮基线。 -- 真实工具 run 刷新后回答仍在，trajectory 为 complete、无 sequence 空洞；浏览器控制台无 error。 diff --git a/docs/superpowers/plans/2026-08-22-trajectory-ledger-p0.md b/docs/superpowers/plans/2026-08-22-trajectory-ledger-p0.md deleted file mode 100644 index 7288ca59..00000000 --- a/docs/superpowers/plans/2026-08-22-trajectory-ledger-p0.md +++ /dev/null @@ -1,221 +0,0 @@ -# Fusion Trajectory P0 实施计划 - -> Required sub-skill: `superpowers:subagent-driven-development` -> Spec: `docs/TRAJECTORY_DESIGN.md` v0.11 -> Base: `origin/master@b478045` -> Branch: `feat/trajectory-ledger-p0` - -**目标：** 在不改变现有聊天 SSE、进度快照和 UI 的前提下，为 Fusion Agent 增加可持久化、可判定完整性的脱敏事件账本，并补齐 LLM round、知识库检索、工具 attempt 与 run attempt 协议。 - -**本轮范围：** 仅 P0。不得实现轨迹快照 API、投影器、MVP Adapter、fusion-ui 侧栏，也不得替换现有“过程”视图。 - -**总体架构：** `AgentEventEmitter` 仍是控制面事件唯一发送方；`AgentEventCompositeWriter` 保持 Redis required/fail-closed，依次旁路 progress snapshot 与新增 `TrajectoryRecorder`。Recorder 使用独立短 Session、只追加 `agent_events` 并原子维护 `run_trajectory_meta`，通过有界线程池和内存 degraded latch 隔离主链路。编排层在全部同步辅助事件之后统一 seal + finalize。 - -## Global Constraints - -- `docs/TRAJECTORY_DESIGN.md` 是规格权威；计划与实现冲突时以规格为准并在 SDD ledger 记录 Ruling。 -- 所有事件携带 `schema_version=1`；原 `protocol_version` 字段保持兼容。 -- Redis 写入语义保持不变：权威 `plan_snapshot` 失败继续 fail-closed；Progress 与 Trajectory 是 auxiliary sink，不得把异常反向传播到主链路。 -- sequence 必须在 payload 校验完成后、第一次可取消 `await` 前预留；complete run 连续 `0..N-1`，degraded run 允许空洞但不得重复或复用。 -- Recorder 固定使用 `ThreadPoolExecutor(max_workers=4)` + `threading.BoundedSemaphore(4)` 非阻塞准入；主链路等待 `0.25s`；超时/取消使用 `asyncio.shield`，底层 worker 负责释放 permit。 -- DB 超时固定为 `statement_timeout=200ms`、`lock_timeout=100ms`、`connect_timeout=1s`；连接超时不是主隔离手段。 -- 事件 INSERT 与 meta 计数必须同事务；Recorder 不得持有或跨线程使用请求 Session。 -- degraded latch 优先于 COUNT/MIN/MAX；迟到成功不得恢复 complete；`CancelledError` 必须置 latch 后重新抛出。 -- `app/ai` 禁止新增对 `app/services` 的依赖；生命周期事件由 Service 编排层发出或通过无框架依赖的回调/观测结果注入。 -- payload 必须经过按事件类型的单点 allowlist；prompt、完整输入输出、工具 schema、精确位置和上游私密错误文本不得入账。 -- P0 只做 FK CASCADE，不做独立 TTL。 -- 不启动本地 Fusion 服务；实现阶段使用单测/集成测试，发布后再按仓库流程做真实登录回归。 -- 每个任务先写失败测试，再实现最小代码，随后跑目标测试与相关回归；提交信息中文且包含 `Co-Authored-By`。 - -## Task 1：冻结实施依据，升级事件协议与 Emitter 序列语义 - -**Files** - -- Modify: `.gitignore` -- Add/track: `docs/TRAJECTORY_DESIGN.md` -- Add/track: `docs/superpowers/plans/2026-08-22-trajectory-ledger-p0.md` -- Modify: `app/services/agent/events.py` -- Modify: `app/services/agent/emitter.py` -- Modify: `test/services/agent/test_events.py` -- Modify: `test/services/agent/test_emitter.py` - -**Steps** - -1. 在 `test_events.py` 先加入失败断言：现有和新增事件均输出 `schema_version=1`；新增 union 能判别 `llm_round_*`、`retrieval_*`、`tool_attempt_*`；边界字段（1-based index、非负 token/duration、允许状态）拒绝非法值。 -2. 在 `test_emitter.py` 先加入失败测试： - - writer 在 await 中取消后，下一事件使用更大 sequence； - - payload 校验失败不消耗 sequence； - - `seal_and_get_last_sequence()` 与 emit 共用锁，返回 `-1` 或最后已预留序号； - - seal 后任何 emit 抛 `RuntimeError`； - - 新生命周期 helper 的父级字段和安全字段正确。 -3. 在 `events.py` 给 `AgentEventBase.schema_version` 设置固定默认值 `1`，加入 v0.10 §3.3 的 11 个生命周期模型并扩展 `AnyAgentEvent`。 -4. 在 `emitter.py` 将 sequence 递增移动到 payload 序列化/体积校验之后、writer await 之前；增加 `_sealed` 与 `seal_and_get_last_sequence()`；新增 LLM/retrieval/tool attempt helper，所有字符串摘要先使用既有安全截断函数或明确长度限制。 -5. 确认 21 个旧事件的 JSON 形状只增 `schema_version`，`protocol_version` 不删除。 -6. Run: `/Users/sean/code/fusion/fusion-api/.venv/bin/python -m pytest test/services/agent/test_events.py test/services/agent/test_emitter.py -q` -7. Run: `/Users/sean/code/fusion/fusion-api/.venv/bin/python -m ruff check app/services/agent/events.py app/services/agent/emitter.py test/services/agent/test_events.py test/services/agent/test_emitter.py` -8. Commit: `feat: 升级轨迹事件协议与序列封口` - -## Task 2：增加账本模型、迁移、run attempt 原子分配 - -**Files** - -- Modify: `app/db/models.py` -- Add: `alembic/versions/e8f5a1c4d2b7_add_agent_trajectory_ledger.py` -- Modify: `app/services/agent/session_cache.py` -- Modify: `app/schemas/chat.py` -- Modify: `app/api/chat.py` -- Modify: `app/services/stream/agent_loop_execution.py` -- Modify: `app/services/stream/agent_loop_lifecycle.py` -- Modify: `app/services/stream/agent_loop_wiring.py` -- Modify: `app/services/stream/run_finalizer.py` -- Modify: `app/services/stream/runner.py` -- Modify: `app/services/chat_service.py` -- Add: `test/test_agent_trajectory_migration.py` -- Modify: `test/services/agent/test_session_cache.py` -- Modify: `test/test_chat_request_message_ids.py` -- Modify: `test/test_chat_service.py` -- Modify: `test/test_chat_continue.py` -- Modify: `test/services/agent/test_continuation.py` -- Modify: `test/services/stream/test_agent_loop_lifecycle.py` -- Modify: `test/services/stream/test_run_finalizer.py` - -**Steps** - -1. 先写迁移 SQL 形状测试，断言 revision 以当前 head `d7e4a9c2f1b6` 为 `down_revision`，并覆盖： - - `agent_sessions.previous_run_id` 自引用 FK `ON DELETE SET NULL`； - - `turn_message_id/attempt_index` 可空及 `(turn_message_id, attempt_index)` 部分唯一索引； - - 回填先令历史 `turn_message_id = message_id`，再使用 `row_number() over (partition by turn_message_id order by created_at, id)`，`message_id IS NULL` 不回填； - - 历史 `previous_run_id` 不推断； - - 新建 `agent_events`、`run_trajectory_meta`，conversation/run FK 均 `ON DELETE CASCADE`； - - 新建单例 `trajectory_ledger_settings` 并由迁移写入环境实际 `ledger_enabled_at`（TIMESTAMPTZ），downgrade 精确删除该表和新增对象。 -2. 在 `models.py` 增加 `AgentEvent`、`RunTrajectoryMeta`、`TrajectoryLedgerSettings`，字段、唯一约束和索引严格匹配 v0.11 §4；给 `AgentSession` 增加 `turn_message_id`、`previous_run_id`、`attempt_index` 和关系/索引；保留既有 `message_id` 的 assistant 锚点语义。 -3. 先在 `test_session_cache.py` 写失败测试：新 run 必须在一个事务内锁定 conversation 行、按稳定 `turn_message_id` 分配 `max(attempt_index)+1`、解析兼容期 previous run、唯一冲突可有界重试；已有 run 的幂等更新不得重置 turn/lineage/index。 -4. 将 `write_session_started()` 扩展为原子 attempt 分配入口，新增 `turn_message_id` 与 optional `previous_run_id`；initial 为 1，retry/regenerate/continue 递增；请求缺 previous 时只在同一加锁事务中按 turn 解析最新 run并记录兼容指标。 -5. 给 `ChatRequest` 增加 optional `previous_run_id`，并将现有稳定 `user_message_id` 作为 `turn_message_id` 沿 `chat.py → chat_service.py → runner.py → agent_loop_wiring.py → execution/lifecycle start` 透传；continue 从被续写 assistant 的前序 user message解析同一 turn，并复用现有 `previous_run_id`。不得让旧客户端因缺字段失败。 -6. 加并发测试：两个独立 Session 对同一 `turn_message_id` 分配 attempt，不产生重复；另加“首次生成失败后重试换 assistant id 仍属于同一 turn”的回归。若 SQLite 不能表达 `FOR UPDATE`，使用 SQL/调用契约测试并在 PostgreSQL migration test 中校验唯一约束。 -7. Run: `/Users/sean/code/fusion/fusion-api/.venv/bin/python -m pytest test/test_agent_trajectory_migration.py test/services/agent/test_session_cache.py test/test_chat_request_message_ids.py test/test_chat_service.py test/test_chat_continue.py test/services/agent/test_continuation.py test/services/stream/test_agent_loop_lifecycle.py test/services/stream/test_run_finalizer.py -q` -8. Run: `/Users/sean/code/fusion/fusion-api/.venv/bin/python -m ruff check app/db/models.py app/services/agent/session_cache.py app/schemas/chat.py app/api/chat.py app/services/stream/agent_loop_execution.py app/services/stream/agent_loop_lifecycle.py app/services/stream/agent_loop_wiring.py app/services/stream/run_finalizer.py app/services/stream/runner.py app/services/chat_service.py test/test_agent_trajectory_migration.py test/services/agent/test_session_cache.py test/test_chat_request_message_ids.py test/test_chat_service.py test/test_chat_continue.py test/services/agent/test_continuation.py test/services/stream/test_agent_loop_lifecycle.py test/services/stream/test_run_finalizer.py` -9. Commit: `feat: 增加轨迹账本模型与运行尝试层级` - -## Task 3：实现 TrajectoryRecorder、allowlist、latch 与完整性 barrier - -**Files** - -- Add: `app/services/agent/trajectory_payload.py` -- Add: `app/services/agent/trajectory_recorder.py` -- Modify: `app/services/stream/tool_executor.py` -- Modify: `app/services/stream/agent_loop_execution.py` -- Add: `test/services/agent/test_trajectory_payload.py` -- Add: `test/services/agent/test_trajectory_recorder.py` -- Modify: `test/test_tool_executor.py` -- Modify: `test/services/stream/test_agent_loop_execution.py` - -**Steps** - -1. 在 `test_trajectory_payload.py` 先为每个事件类型建立 allowlist 契约测试；加入恶意附加字段、prompt、完整 content、工具 schema、URL query、错误密钥文本，断言落库 DTO 只保留允许字段并执行上限截断。 -2. 在 `test_trajectory_recorder.py` 先写失败测试覆盖： - - 独立 Session，event INSERT + meta 创建/计数同事务；commit 前故障整体回滚；重复 `(run_id, sequence)` 不重复计数； - - `BoundedSemaphore(4)` 满载立即返回并 latch `admission_full`；submit 失败释放 permit；worker 正常/异常/迟到均释放； - - `wait_for(shield(future), 0.25)` 真超时，迟到异常被消费；worker 未启动和运行中取消均 latch `recorder_cancelled`、re-raise、permit 最终守恒； - - latch 首次原因稳定，后续事件跳过 DB，迟到成功不清 latch； - - finalize 先读 latch，再验证 COUNT/MIN/MAX；complete 写 expected/finalized，mismatch 写 degraded；最后事件迟到后仍 degraded。 -3. 在 `trajectory_payload.py` 建立按 `event_type` 的唯一 allowlist 入口；未知事件 fail-open 到 degraded，但不得原样入库。 -4. 在 `trajectory_recorder.py` 实现进程级 executor/semaphore、独立 Session factory、同步 worker、late future consumer、per-run latch、`record_chunk()` 与 `finalize(expected_last_sequence)`。DB 连接/事务设置使用规格固定超时。 -5. 修改 `AgentEventCompositeWriter`：顺序必须是 `await Redis` → progress recorder → `await trajectory_recorder.record_chunk()`；Redis 异常原样抛出，两个 auxiliary sink 自己 fail-open。 -6. 在 `_build_execution_parts()` 创建每 run Recorder 并注入 writer；同时把 Recorder 放入 execution/completion context，供统一 finalize 使用。 -7. Run: `/Users/sean/code/fusion/fusion-api/.venv/bin/python -m pytest test/services/agent/test_trajectory_payload.py test/services/agent/test_trajectory_recorder.py test/test_tool_executor.py::AgentEventCompositeWriterTests test/services/stream/test_agent_loop_execution.py -q` -8. Run: `/Users/sean/code/fusion/fusion-api/.venv/bin/python -m ruff check app/services/agent/trajectory_payload.py app/services/agent/trajectory_recorder.py app/services/stream/tool_executor.py app/services/stream/agent_loop_execution.py test/services/agent/test_trajectory_payload.py test/services/agent/test_trajectory_recorder.py test/test_tool_executor.py test/services/stream/test_agent_loop_execution.py` -9. Commit: `feat: 实现有界轨迹账本记录器` - -## Task 4：把 seal + finalize 接入全部 run 终态 - -**Files** - -- Modify: `app/services/stream/agent_loop_execution.py` -- Modify: `app/services/stream/agent_loop_lifecycle.py` -- Modify: `app/services/stream/agent_loop_run_completion.py` -- Modify: `app/services/stream/agent_loop_wiring.py` -- Modify: `app/services/stream/run_finalizer.py` -- Modify: `test/services/stream/test_agent_loop_lifecycle.py` -- Modify: `test/services/stream/test_agent_loop_run_completion.py` -- Modify: `test/services/stream/test_run_finalizer.py` - -**Steps** - -1. 先写终态矩阵失败测试：completed、limit_reached、failed、interrupted、superseded、user cancellation、shutdown 均只 seal/finalize 一次；`run_completed` 自身不 finalize；`suggested_questions_pending` 在 seal 前；seal 后无事件。 -2. 增加统一 `commit_trajectory_barrier()`：在 lifecycle 的最外层 `finally` 中执行 `seal_and_get_last_sequence()` + `trajectory_recorder.finalize(last_seq)`，以有界 `asyncio.wait_for(asyncio.shield(...))` 保护；重复进入幂等。 -3. 保持现有业务终态/Redis stream 终态优先级：barrier 失败只使轨迹 degraded，不替换原始 `CancelledError`、`StreamOwnershipLostError` 或业务异常。 -4. 对开始前失败、session 行不存在、辅助问题领取失败分别测试并明确：能 seal 的一律 seal；无法建 meta 的由 latch/stale 路径判 degraded。 -5. Run: `/Users/sean/code/fusion/fusion-api/.venv/bin/python -m pytest test/services/stream/test_agent_loop_lifecycle.py test/services/stream/test_agent_loop_run_completion.py test/services/stream/test_run_finalizer.py test/services/agent/test_emitter.py test/services/agent/test_trajectory_recorder.py -q` -6. Run: `/Users/sean/code/fusion/fusion-api/.venv/bin/python -m ruff check app/services/stream/agent_loop_execution.py app/services/stream/agent_loop_lifecycle.py app/services/stream/agent_loop_run_completion.py app/services/stream/agent_loop_wiring.py app/services/stream/run_finalizer.py test/services/stream/test_agent_loop_lifecycle.py test/services/stream/test_agent_loop_run_completion.py test/services/stream/test_run_finalizer.py` -7. Commit: `feat: 接入轨迹完整性提交屏障` - -## Task 5：接入 LLM round、知识库 retrieval、工具 attempt 生命周期 - -**Files** - -- Modify: `app/ai/llm_round_observability.py` -- Modify: `app/schemas/chat.py` -- Modify: `app/services/stream/agent_round.py` -- Modify: `app/services/stream/llm_stream.py` -- Modify: `app/services/stream/agent_loop_lifecycle.py` -- Modify: `app/services/stream/tool_executor.py` -- Modify as required: `app/services/stream/limit_summary.py` -- Modify: `test/services/stream/test_llm_round_observability.py` -- Modify: `test/services/stream/test_agent_round.py` -- Modify: `test/services/stream/test_llm_stream.py` -- Modify: `test/services/stream/test_agent_loop_lifecycle.py` -- Modify: `test/test_tool_executor.py` - -**Steps** - -1. LLM：先写失败测试覆盖 started → first delta → completed/failed/cancelled；first delta 识别 reasoning/content/tool_call；空流不发 first delta 且 completed.ttft_ms 为 null；reasoning/content 可见 Redis chunk 必须先于 first delta event，tool_call-only 无此前置要求。 -2. 将 `LLMRoundObservation` 保持为纯测量组件，输出结构化 started/first delta/finish 数据或接受无 `services` 依赖的 async callback；`agent_round.py`/`limit_summary.py` 的 Service 层负责调用 emitter。 -3. 扩展 `Usage` 或内部 usage extraction，兼容供应商字段提取 `cache_read_tokens/cache_write_tokens`；缺失/非法时为 null，不报错，不破坏现有 Usage 序列化。 -4. Retrieval：在 `_prepare_knowledge_grounding()` 周围发 started/completed/failed/cancelled，query 只存 ≤120 字安全摘要；真实取消必须发 cancelled 后 re-raise。 -5. Tool attempt：把 attempt 生命周期放进每一次真正 handler 执行，而非整个 tool_call；同 tool_call 1-based 递增，timeout/cancelled/failed/success 正确结束。不要改变现有 backoff 次数与 ToolResult 语义。 -6. 增加架构测试：`app/ai` 无任何新增 `app.services` import。 -7. Run: `/Users/sean/code/fusion/fusion-api/.venv/bin/python -m pytest test/services/stream/test_llm_round_observability.py test/services/stream/test_agent_round.py test/services/stream/test_llm_stream.py test/services/stream/test_agent_loop_lifecycle.py test/test_tool_executor.py -q` -8. Run: `/Users/sean/code/fusion/fusion-api/.venv/bin/python -m ruff check app/ai/llm_round_observability.py app/schemas/chat.py app/services/stream/agent_round.py app/services/stream/llm_stream.py app/services/stream/agent_loop_lifecycle.py app/services/stream/tool_executor.py app/services/stream/limit_summary.py test/services/stream/test_llm_round_observability.py test/services/stream/test_agent_round.py test/services/stream/test_llm_stream.py test/services/stream/test_agent_loop_lifecycle.py test/test_tool_executor.py` -9. Commit: `feat: 补齐轨迹生命周期事件` - -## Task 6：实现 stale 收敛、legacy 判定、性能与全量回归门禁 - -**Files** - -- Add: `app/services/agent/trajectory_reconciliation.py` -- Modify: `app/services/scheduler_service.py` -- Add: `test/services/agent/test_trajectory_reconciliation.py` -- Add: `test/services/agent/test_trajectory_performance.py` -- Add: `test/test_scheduler_service.py` -- Add: `reports/trajectory-p0-baseline.md` - -**Steps** - -1. 先写 reconciliation 失败测试：仅扫描 stale `recording`；run 已终态且 expected 存在并通过三断言才 complete；expected 缺失、校验不符或新 run 缺 meta 均 degraded；无 meta 时只以数据库持久化 `ledger_enabled_at` 区分 legacy 与 meta_missing。 -2. 实现幂等协调函数并挂入现有 scheduler 体系；不得实现 TTL、删除事件或新增 P1 API。 -3. 建立无需真实网络的性能门禁：测 Redis writer stub + progress + recorder fast path 的 p50/p95/p99、并发连接/permit 上限、Recorder timeout 时首个可见 SSE 仍先到；将测试方法和本机基线写入 `reports/trajectory-p0-baseline.md`。 -4. 运行 P0 完整目标集： - `/Users/sean/code/fusion/fusion-api/.venv/bin/python -m pytest test/services/agent test/services/stream/test_agent_loop_execution.py test/services/stream/test_agent_loop_lifecycle.py test/services/stream/test_agent_loop_run_completion.py test/services/stream/test_agent_round.py test/services/stream/test_llm_stream.py test/services/stream/test_llm_round_observability.py test/services/stream/test_run_finalizer.py test/test_tool_executor.py test/test_chat_request_message_ids.py test/services/agent/test_continuation.py test/test_agent_trajectory_migration.py test/test_scheduler_service.py -q` -5. 运行全量：`/Users/sean/code/fusion/fusion-api/.venv/bin/python -m pytest test/ -q` -6. 运行静态检查：`/Users/sean/code/fusion/fusion-api/.venv/bin/python -m ruff check app test` -7. 检查迁移单头：`/Users/sean/code/fusion/fusion-api/.venv/bin/python -m alembic heads`，必须仅有新 revision。 -8. Commit: `test: 完善轨迹账本回归与性能门禁` - -## Task 7：对抗审查、修复、推送与发布后验收 - -**Files** - -- 仅修改审查发现直接涉及的 P0 文件。 -- 更新：`reports/trajectory-p0-baseline.md` - -**Steps** - -1. 按 SDD 流程生成从 `b478045..HEAD` 的全分支 review package，由高能力 Reviewer 对照 v0.10 §11 做 spec + quality 审查。 -2. 所有 Critical/Important findings 进入一次统一修复波次并做 scoped re-review；不得把 P1–P3 建议扩入本分支。 -3. 重跑受影响测试、全量 pytest、ruff、Alembic heads；记录精确命令和结果。 -4. 精确暂存并核对 `git diff --cached`，提交审查修复（如有）；确认主 checkout 与 fusion-ui 无改动。 -5. Push: `git push -u origin feat/trajectory-ledger-p0`；创建 PR，等待并核对 GitHub Actions。推送/PR 不是合并、部署或验收。 -6. 若仓库自动部署到授权 dev 环境，核对 CI 对应 commit、运行镜像/健康；不手动触发生产部署。 -7. 使用已有登录 Chrome 标签，以新建真实对话覆盖：普通 LLM 流、tool retry、知识库检索、取消各一条；从数据库/日志核对事件序列、meta 状态和安全 payload。P0 无 UI，因此页面只验聊天/现有“过程”无回归。 -8. 将真实模型事件类型、延迟对比、degraded/complete 结果写入 `reports/trajectory-p0-baseline.md`，不得记录密钥、prompt 或完整用户内容。 diff --git a/docs/superpowers/plans/2026-08-22-trajectory-ledger-p1.md b/docs/superpowers/plans/2026-08-22-trajectory-ledger-p1.md deleted file mode 100644 index c38f815f..00000000 --- a/docs/superpowers/plans/2026-08-22-trajectory-ledger-p1.md +++ /dev/null @@ -1,266 +0,0 @@ -# Trajectory P1 历史投影与快照 API Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** 在 Fusion API 中提供可鉴权、可截断、可审计的历史轨迹读侧，将 P0 脱敏事件账本查询时 O(n) 投影为 records/spans，并暴露普通用户运行列表、普通用户快照和管理员诊断快照。 - -**Architecture:** 新增独立 `TrajectoryRepository → TrajectoryQueryService → API` 读侧，不复用聊天写流服务。`TrajectoryProjector` 保持无数据库依赖的纯函数，输入有序事件前缀与 run 终态，输出稳定 record/span DTO；完整性状态继续复用 P0 的 `RunTrajectoryMeta`/ledger watermark 契约。普通用户和管理员使用不同端点与不同 DTO，管理员端额外返回按 run 可靠关联并经过现有审计脱敏器处理的 `ToolCallLog` 摘要，同时记录访问审计。 - -**Tech Stack:** Python 3.11、FastAPI、Pydantic v2、SQLAlchemy、PostgreSQL/SQLite 测试、pytest/unittest、Ruff - -**Spec:** `docs/TRAJECTORY_DESIGN.md` - -## Global Constraints - -- P1 只提供历史快照，不新增 SSE、轮询或其他实时语义，不修改 P3 前端。 -- 普通端点固定为 `GET /api/conversations/{conversation_id}/runs` 与 `GET /api/conversations/{conversation_id}/runs/{run_id}/trajectory`；不复用 `/api/chat` 写流前缀。 -- 管理员端点固定为 `GET /api/admin/audit/conversations/{conversation_id}/runs/{run_id}/trajectory`，必须使用 `get_conversation_auditor`、`X-Admin-Audit-Reason` 与现有访问审计。 -- 普通端点必须在数据库查询中同时约束 `conversation_id + current_user.id`，并校验 run 属于该 conversation；会话不存在、越权、run 不属于会话统一返回 404。 -- 普通 DTO 与管理员 DTO 必须是不同 Pydantic 类型；普通端点不得根据角色扩展字段。 -- `MAX_TRAJECTORY_EVENTS_PER_RUN` 默认 `5000`；查询必须 `LIMIT max+1`，只投影前 max 条并返回 `truncated=true`。 -- `MAX_TRAJECTORY_RUNS_PER_CONVERSATION` 默认 `500`；查询必须 `LIMIT max+1`，只返回最近 max 个 run 并返回 `truncated=true`。 -- 投影器必须是 O(n) 纯函数，不查询数据库、不写入 span 表、不修改事件 payload。 -- `complete`、`degraded`、`legacy` 与 `truncated` 是独立维度；`complete + pending terminal intent` 对外必须降级，缺 meta 必须按 ledger watermark 判定，不能直接视为 legacy。 -- 截断前缀中的未闭合 span 必须标记 `terminal_source=inferred`、`status=unknown`、`inferred_reason=truncated_prefix`，不得使用完整 run 的终态伪装其完成状态。 -- 非截断终态 run 的 orphan：成功 run → `unknown/incomplete`，失败 run → `failed`，中断 run → `cancelled`；均标记 `terminal_source=inferred` 与设计规定的 `inferred_reason`。 -- 真实终态事件关闭的 span 标记 `terminal_source=recorded`；仍在运行且未截断的 open span 允许 `status=running`、`terminal_source=null`。 -- `tool_call_completed`、`step_completed` 即使缺少 started，也必须利用自带 `duration_ms` 形成 recorded span；其他生命周期终态缺 start 只保留 record，不伪造 start。 -- annotation 不建 span；record 通过 `span_id` 关联到 run、step、tool、llm、retrieval 或 tool-attempt span，无法精确关联时附着 run span。 -- 普通事件 record 只返回 P0 已 allowlist 落库的 payload 副本；不得从消息、ToolCallLog 或工具注册表补充敏感内容。 -- 管理员诊断 v1 只额外返回 `ToolCallLog` 中可可靠按 `trace_id=run_id` 关联的脱敏摘要列表；不得声称该列表与 ledger `tool_call_id` 精确一一对应。 -- 不启动本地 Fusion 服务；测试先 RED 后 GREEN，最终运行全量 pytest、ruff check、ruff format --check 与 diff check。 - -### Ruling 1：普通轨迹资源使用 `/api` 根路径 - -设计文档持续使用 `/api/conversations/...`，轨迹是独立只读资源而非 `/api/chat` 的写流动作，因此新 router 挂载到 `/api`。如果判断错误，代价是 P3 客户端需要调整 base path；不会造成数据迁移或双写。 - -### Ruling 2：每个会话默认最多返回最近 500 个 run - -设计只给出常量名而未给默认值；500 足以覆盖调试历史，同时阻止无界列表查询。若容量估计偏小，后续可通过环境变量上调，不改变响应契约。 - -### Ruling 3：管理员工具诊断按 run 汇总，不伪造 tool_call 精确关联 - -`ToolCallLog` 只有 `trace_id + step_number`，没有 ledger `tool_call_id` 外键。P1 返回独立 `tool_calls[]` 诊断列表并明确 `association="run"`；若未来补齐稳定关联键，再在新 schema_version 中增加 span 级关联。 - ---- - -### Task 1: 定义 DTO 并实现纯函数 TrajectoryProjector - -**Files:** - -- Create: `app/schemas/trajectory.py` -- Create: `app/services/agent/trajectory_projector.py` -- Create: `test/services/agent/test_trajectory_projector.py` - -**Interfaces:** - -- Consumes: 按 `sequence ASC` 排序的 `TrajectoryEventRecord`、`run_status: str`、`run_ended_at: datetime | None`、`truncated: bool`。 -- Produces: `TrajectoryProjection(records: list[TrajectoryRecord], spans: list[TrajectorySpan])`；Task 2 将其嵌入 `TrajectorySnapshot`。 -- `TrajectoryRecord` 字段：`sequence,event_type,schema_version,timestamp,step_id,tool_call_id,parent_step_id,trace_id,span_id,payload`。 -- `TrajectorySpan` 字段：`span_id,kind,name,parent_span_id,start_sequence,end_sequence,started_at,ended_at,duration_ms,status,terminal_source,inferred_reason,ttft_ms,record_sequences`。 - -- [ ] **Step 1: 写投影失败测试并确认 RED** - - 在 `test_trajectory_projector.py` 用手写 literal 事件覆盖： - - 1. run/step/tool/llm/retrieval/tool-attempt 正常配对，层级、状态、duration、TTFT 与 record 关联正确； - 2. `tool_call_completed`、`step_completed` 缺 started 时仍依据 duration 生成 recorded span； - 3. plan/evidence/context/suggested annotation 不创建 span，并附着最精确父 span或 run； - 4. 成功、失败、中断终态分别收口 orphan；成功 orphan 必须为 `status=unknown`、`inferred_reason=run_completed_without_close`； - 5. 截断前缀一律使用 `unknown/truncated_prefix`，不能借完整 run 状态关闭； - 6. 缺失 `schema_version` 的历史 record 返回 `schema_version=0`，不抛错； - 7. running run 的 open span 保留 `running + terminal_source=null`； - 8. 事件输入对象在投影后保持不变。 - - Run: - - ```bash - /Users/sean/code/fusion/fusion-api/.venv/bin/python -m pytest test/services/agent/test_trajectory_projector.py -q - ``` - - Expected: FAIL，原因是 `trajectory_projector`/DTO 尚不存在。 - -- [ ] **Step 2: 最小实现 DTO 与单遍状态机** - - `project_trajectory()` 只遍历 records 一次；用 `dict[span_id, mutable builder]` 保存 open/closed span，用输出顺序列表保持首次出现顺序。固定 span id：`run:{run_id}`、`step:{step_id}`、`tool:{tool_call_id}`、`llm:{llm_round_id}`、`retrieval:{retrieval_id}`、`tool_attempt:{tool_attempt_id}`。终态处理优先使用事件 payload 的 `duration_ms/status/ttft_ms`，缺失 duration 才用时间差；负时间差归零。 - -- [ ] **Step 3: 实现 orphan、截断与 annotation 关联** - - 终态 run 的推导关闭时间使用 `run_ended_at`，为空时使用最后一条已加载事件时间；截断时始终使用前缀末事件时间并标 `truncated_prefix`。annotation 关联顺序为 `tool_call_id → step_id/parent_step_id → run`；所有关联 sequence 进入目标 span 的 `record_sequences`。 - -- [ ] **Step 4: 运行聚焦测试并确认 GREEN** - - ```bash - /Users/sean/code/fusion/fusion-api/.venv/bin/python -m pytest test/services/agent/test_trajectory_projector.py -q - /Users/sean/code/fusion/fusion-api/.venv/bin/python -m ruff check app/schemas/trajectory.py app/services/agent/trajectory_projector.py test/services/agent/test_trajectory_projector.py - ``` - -- [ ] **Step 5: 自审并中文提交** - - 提交信息：`feat: 实现轨迹历史投影器`，提交必须包含 `Co-Authored-By: Codex `。 - -### Task 2: 实现受限查询服务与普通用户 API - -**Files:** - -- Create: `app/db/trajectory_repository.py` -- Create: `app/services/trajectory_query_service.py` -- Create: `app/api/trajectory.py` -- Create: `test/services/test_trajectory_query_service.py` -- Create: `test/test_trajectory_api.py` -- Modify: `app/api/deps.py` -- Modify: `app/core/config.py` -- Modify: `.env.example` -- Modify: `main.py` -- Modify: `app/services/agent/trajectory_reconciliation.py` -- Modify: `test/services/agent/test_trajectory_reconciliation.py` - -**Interfaces:** - -- `TrajectoryRepository.list_runs(conversation_id, user_id, limit) -> list[tuple[AgentSession, RunTrajectoryMeta | None]] | None`：会话不存在/越权返回 `None`，成功按最新 run 查询 `limit` 条。 -- `TrajectoryRepository.get_run(conversation_id, run_id, user_id) -> tuple[AgentSession, RunTrajectoryMeta | None] | None`：归属约束在查询中完成。 -- `TrajectoryRepository.list_events(conversation_id, run_id, limit) -> list[AgentEvent]`：`sequence ASC`，调用方传 `max+1`。 -- `TrajectoryQueryService.list_runs(...) -> TrajectoryRunListResponse`。 -- `TrajectoryQueryService.get_user_snapshot(...) -> TrajectorySnapshot`。 -- `resolve_trajectory_status_from_rows(run, meta, watermark) -> TrajectoryStatusAssessment`：供列表批量状态判定与现有单 run helper 共用。 - -- [ ] **Step 1: 写状态判定与查询服务失败测试并确认 RED** - - 覆盖： - - 1. `complete + pending` 对外降级；meta 状态正常复用；无 meta 时 ledger 之前为 legacy、之后为 degraded/meta_missing； - 2. 用户只能读取自己的 conversation，越权和 run 跨 conversation 均得到 service 层 not-found； - 3. events 使用 `max+1`，只投影 max 条且 `truncated=true`；恰好 max 条不截断； - 4. runs 使用 `max+1`，取最近 max 个 group，再按 group 最近时间倒序、组内 `attempt_index` 升序；null attempt 保持 null，不伪装成 1； - 5. snapshot 的 `run` 使用 AgentSession 权威计数/duration，`completeness` 同时报告 status、degraded_reason、meta event_count/expected、loaded_event_count 与 sequence 边界； - 6. legacy run 无 events 时仍返回可解释空快照，而不是 404。 - - Run: - - ```bash - /Users/sean/code/fusion/fusion-api/.venv/bin/python -m pytest \ - test/services/agent/test_trajectory_reconciliation.py \ - test/services/test_trajectory_query_service.py -q - ``` - - Expected: FAIL，原因是批量状态 helper、repository/service 尚不存在。 - -- [ ] **Step 2: 实现 repository 与批量完整性 helper** - - 查询只选择 P1 必需行；列表用一次 outer join 加一次 ledger watermark 查询，禁止每个 run 单独 `session.get()`。`resolve_run_trajectory_status()` 改为调用新的纯 helper，保持 P0 既有语义与测试。 - -- [ ] **Step 3: 实现 query service、上限配置与快照组装** - - 在 `Settings` 增加： - - ```python - MAX_TRAJECTORY_EVENTS_PER_RUN: int = int(os.getenv("MAX_TRAJECTORY_EVENTS_PER_RUN", "5000")) - MAX_TRAJECTORY_RUNS_PER_CONVERSATION: int = int(os.getenv("MAX_TRAJECTORY_RUNS_PER_CONVERSATION", "500")) - ``` - - service 构造时拒绝非正数配置；事件 DTO 必须从数据库列与 `dict(event.payload)` 新建，缺版本兼容为 0。不得返回 terminal intent 字段。 - -- [ ] **Step 4: 写普通 API 失败测试并确认 RED** - - 使用 TestClient + SQLite 真模型覆盖两个端点、success envelope/request_id、认证依赖、越权 404、run 跨 conversation 404、truncated 响应；断言普通响应不存在 `input_params/output_data/terminal_intent`。 - - ```bash - /Users/sean/code/fusion/fusion-api/.venv/bin/python -m pytest test/test_trajectory_api.py -q - ``` - -- [ ] **Step 5: 挂载 router 与依赖并确认 GREEN** - - `app/api/trajectory.py` router 内路径从 `/conversations` 开始；`main.py` 使用 `prefix="/api"`。两个端点均注入 `TrajectoryQueryService` 与 `get_current_user`，service 返回空值时统一 `ApiException.not_found("会话或轨迹不存在，或无权访问")`。 - - ```bash - /Users/sean/code/fusion/fusion-api/.venv/bin/python -m pytest \ - test/services/agent/test_trajectory_reconciliation.py \ - test/services/test_trajectory_query_service.py \ - test/test_trajectory_api.py -q - /Users/sean/code/fusion/fusion-api/.venv/bin/python -m ruff check \ - app/db/trajectory_repository.py app/services/trajectory_query_service.py \ - app/api/trajectory.py app/api/deps.py app/core/config.py main.py \ - test/services/test_trajectory_query_service.py test/test_trajectory_api.py - ``` - -- [ ] **Step 6: 自审并中文提交** - - 提交信息：`feat: 提供轨迹运行列表与快照接口`，提交必须包含 `Co-Authored-By: Codex `。 - -### Task 3: 实现管理员诊断 DTO 与访问审计 - -**Files:** - -- Create: `app/schemas/admin_trajectory.py` -- Modify: `app/db/trajectory_repository.py` -- Modify: `app/services/trajectory_query_service.py` -- Modify: `app/services/admin_audit_service.py` -- Modify: `app/api/admin_audit.py` -- Modify: `test/test_admin_audit_api.py` -- Modify: `test/test_admin_audit_service.py` -- Modify: `docs/TRAJECTORY_DESIGN.md` - -**Interfaces:** - -- `TrajectoryRepository.list_tool_diagnostics(run_id) -> list[ToolCallLog]`：严格按 `trace_id=run_id`，`step_number/created_at/id` 排序。 -- `TrajectoryQueryService.get_admin_snapshot(conversation_id, run_id) -> AdminTrajectorySnapshot | None`：无 user 归属过滤，但仍校验 run 属于 conversation。 -- `AdminTrajectorySnapshot` 包含普通 `snapshot` 与 `tool_calls`；每个工具项固定含 `association="run"`、`id,message_id,step_number,tool_name,status,duration_ms,model_id,provider,arguments,result_preview,error,redacted_fields,created_at`。 -- `AdminAuditService.record_trajectory_view(...)`：写 `action="admin.audit.trajectory.view"`、`resource_type="conversation_run_trajectory"`、`resource_id=run_id`、`target_user_id=conversation.user_id`。 - -- [ ] **Step 1: 写管理员服务/API 失败测试并确认 RED** - - 覆盖： - - 1. 非审计员 403；管理员响应 `Cache-Control: private, no-store`； - 2. run 不属于 conversation 返回 404； - 3. 读取成功后新增 `admin.audit.trajectory.view`，reason 被记录但敏感值不复制进审计 metadata； - 4. `ToolCallLog` 仅按 run 返回，使用既有 `_tool_item` 脱敏规则，原始 secret/error 不泄漏； - 5. admin DTO 与普通 DTO 结构不同，但 base snapshot 内容一致；工具项明确 `association=run`，没有伪造 `tool_call_id`； - 6. 审计写入失败返回现有 503 语义，不返回未审计的敏感诊断。 - - ```bash - /Users/sean/code/fusion/fusion-api/.venv/bin/python -m pytest \ - test/test_admin_audit_service.py test/test_admin_audit_api.py -q - ``` - - Expected: FAIL，原因是管理员轨迹接口、DTO 与审计 action 尚不存在。 - -- [ ] **Step 2: 实现管理员 DTO、run 级诊断与审计** - - 复用 `AdminAuditService._tool_item()` 生成允许字段；不得直接序列化 `ToolCallLog.input_params/output_data/error_message`。管理员 endpoint 先通过 query service 构造 DTO，再调用公开审计方法；审计失败时整体请求失败。 - -- [ ] **Step 3: 更新设计文档的 P1 实施契约** - - 将文首状态更新为 P0 已 dev 验收、P1 实施中；在 §7 固化普通 `/api` 路径、两个默认上限、普通/admin DTO 形状、管理员 run 级工具关联限制与 `truncated_prefix` 语义。不得提前宣称 P1 已部署或 P3 已开始。 - -- [ ] **Step 4: 聚焦回归并确认 GREEN** - - ```bash - /Users/sean/code/fusion/fusion-api/.venv/bin/python -m pytest \ - test/services/agent/test_trajectory_projector.py \ - test/services/test_trajectory_query_service.py \ - test/test_trajectory_api.py \ - test/test_admin_audit_service.py \ - test/test_admin_audit_api.py -q - /Users/sean/code/fusion/fusion-api/.venv/bin/python -m ruff check \ - app/schemas/admin_trajectory.py app/db/trajectory_repository.py \ - app/services/trajectory_query_service.py app/services/admin_audit_service.py \ - app/api/admin_audit.py test/test_admin_audit_service.py test/test_admin_audit_api.py - ``` - -- [ ] **Step 5: 自审并中文提交** - - 提交信息：`feat: 增加管理员轨迹诊断与审计`，提交必须包含 `Co-Authored-By: Codex `。 - -## Branch-wide verification - -所有任务与逐任务审查通过后，由控制 Agent 运行： - -```bash -/Users/sean/code/fusion/fusion-api/.venv/bin/python -m pytest test/ -q -/Users/sean/code/fusion/fusion-api/.venv/bin/python -m ruff check . -/Users/sean/code/fusion/fusion-api/.venv/bin/python -m ruff format --check . -git diff --check origin/master...HEAD -``` - -随后进行 whole-branch 独立审查。未经额外发布授权，不合并、不部署；若推送/CI 授权沿当前仓库规则有效，则只推进到特性分支 PR 与 CI，不把 CI 成功表述为 dev 验收。 diff --git a/docs/superpowers/plans/2026-08-24-trajectory-llm-round-detail.md b/docs/superpowers/plans/2026-08-24-trajectory-llm-round-detail.md deleted file mode 100644 index 3a5a5549..00000000 --- a/docs/superpowers/plans/2026-08-24-trajectory-llm-round-detail.md +++ /dev/null @@ -1,330 +0,0 @@ -# Fusion Trajectory LLM Round Detail Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** 为新产生的 Fusion Agent Run 按 `llm_round_id` 持久化模型显式返回的 Reasoning/Output，并在 Trajectory 中以 DSH 风格的 `ASSISTANT · Request #N` 行和 Thinking/Output/Timing 详情展示，同时保持聊天视图不变。 - -**Architecture:** `agent_events` 继续只保存 LLM 生命周期和指标；新增 `agent_llm_round_details` 保存净化后的正文与有界 preview。每个新 Run 通过 `run_trajectory_meta.llm_detail_schema_version=1` 声明能力，历史 Run 不走兼容或猜测路径。前端使用快照中的 `llm_round_summaries` 构建 `LlmRoundCell`，完整正文通过 LLM Node Detail API 懒加载，实时完成后只补取单节点详情。 - -**Tech Stack:** FastAPI、SQLAlchemy、Alembic、PostgreSQL、Pydantic v2、pytest、Next.js、React、Redux Toolkit、TypeScript、Vitest、Testing Library。 - -**Spec:** `docs/TRAJECTORY_DESIGN.md` v0.22，重点 §7.4.11、§8.4、§8.5、§10。 - -## Global Constraints - -- 只处理新契约 Run；不回填旧 Reasoning，不按 step/name/time/message 猜测关联。 -- 仅保存 `sanitize_user_visible_reasoning(..., final=True)` 处理后的 Reasoning；`raw_reasoning_buf` 不进入数据库、API 或日志。 -- 聊天视图的 Thinking 隐藏策略不变；Trajectory 是独立诊断投影。 -- `agent_events` 不保存 Reasoning/Output 正文；完整正文仅存在于 `agent_llm_round_details`。 -- LLM 详情写入按 `(run_id, llm_round_id)` 幂等；正文写入失败不能中断 Agent 主链路。 -- P3 不实现逐 Token Thinking；Round 终态后展示完整内容。 -- 实时补齐不得重拉整个快照，不修改既有 SSE envelope。 -- 不在旧 `feat/trajectory-ledger-p0` worktree 实施；API 使用 `feat/trajectory-p3-contract-guard`，UI 使用 `feat/trajectory-p3-ui`。 -- 保留 UI worktree 中现有未提交 UX 修改，不覆盖、不回滚。 - ---- - -### Task 1: 迁入 v0.22 并收口实施契约 - -**Files:** -- Modify: `docs/TRAJECTORY_DESIGN.md` -- Create: `docs/superpowers/plans/2026-08-24-trajectory-llm-round-detail.md` - -**Interfaces:** -- Consumes: 旧 P0 worktree 中的 v0.22 文档内容。 -- Produces: 正式 P3 API 分支内唯一设计依据与本实施计划。 - -- [ ] **Step 1: 将 §7.4.11、§8.4、§8.5 与 P3 交付物迁入正式分支** - -只迁移设计文本，不复制旧 worktree 的代码或 Git 状态。 - -- [ ] **Step 2: 修正文档残留** - -将 `v0.21`、`P3 定稿待实施`、`全量会话账本` 等残留改为当前事实；拆分 Tool 与 LLM status；明确 `reasoning_tokens` 若实现则属于 `llm_round_completed` 可选生命周期字段，否则不作为 P3 阻塞。 - -- [ ] **Step 3: 校验文档差异** - -Run: - -```bash -rg -n "v0\.21 位于|将 v0\.21|全量会话账本|只承诺 Tool|待定稿" docs/TRAJECTORY_DESIGN.md -git diff --check -``` - -Expected: 无版本/范围残留，`git diff --check` 退出码为 0。 - -### Task 2: 数据模型、迁移与新 Run 能力声明 - -**Files:** -- Create: `alembic/versions/_add_llm_round_details.py` -- Modify: `app/db/models.py` -- Modify: `app/services/agent/trajectory_recorder.py` -- Test: `test/test_trajectory_llm_round_detail_migration.py` -- Test: `test/services/agent/test_trajectory_recorder.py` - -**Interfaces:** -- Consumes: `RunTrajectoryMeta`、`AgentSession`、`Conversation`、P0 幂等 meta 创建。 -- Produces: `AgentLlmRoundDetail` ORM 模型、`run_trajectory_meta.llm_detail_schema_version`、新 Run 的 version=1 声明。 - -- [ ] **Step 1: 写迁移失败测试** - -测试升级后存在 `agent_llm_round_details`、唯一约束、级联 FK 和 `llm_detail_schema_version`；降级后全部移除。 - -- [ ] **Step 2: 运行测试确认 RED** - -```bash -/Users/sean/code/fusion/fusion-api/.venv/bin/pytest -q test/test_trajectory_llm_round_detail_migration.py -``` - -Expected: 因迁移或模型尚不存在而失败。 - -- [ ] **Step 3: 实现最小迁移与 ORM** - -详情表字段固定为：`id/conversation_id/run_id/message_id/llm_round_id/reasoning_text/content_text/reasoning_preview/output_preview/redacted_fields/truncated_fields/recorded_at`；唯一键为 `(run_id, llm_round_id)`。 - -- [ ] **Step 4: 写新 Run 能力声明失败测试并实现** - -`TrajectoryRecorder._insert_meta_if_missing()` 对新建 meta 写入 `llm_detail_schema_version=1`，冲突时不覆盖历史 `NULL`。 - -- [ ] **Step 5: 运行目标测试确认 GREEN** - -```bash -/Users/sean/code/fusion/fusion-api/.venv/bin/pytest -q \ - test/test_trajectory_llm_round_detail_migration.py \ - test/services/agent/test_trajectory_recorder.py -``` - -### Task 3: LLM Round Detail 幂等后台写入 - -**Files:** -- Create: `app/services/agent/llm_round_detail_recorder.py` -- Modify: `main.py` -- Test: `test/services/agent/test_llm_round_detail_recorder.py` - -**Interfaces:** -- Consumes: `AgentLlmRoundDetail`、`SessionLocal`、`sanitize_user_visible_reasoning`。 -- Produces: `schedule_llm_round_detail(draft) -> asyncio.Task`、`stop_llm_round_detail_workers() -> Awaitable[None]`。 - -- [ ] **Step 1: 写净化、preview、幂等和 shutdown 失败测试** - -断言原始协议不入库、preview 不超过 200 字符、重复 `(run_id,llm_round_id)` 只保留一行、任务集合在完成后释放、shutdown 可观察未完成任务。 - -- [ ] **Step 2: 运行测试确认 RED** - -```bash -/Users/sean/code/fusion/fusion-api/.venv/bin/pytest -q test/services/agent/test_llm_round_detail_recorder.py -``` - -- [ ] **Step 3: 实现最小后台记录器** - -使用模块级受控 task 集合跟踪 `asyncio.to_thread()` 数据库写入；创建任务失败时关闭 coroutine；写入使用 PostgreSQL/SQLite `ON CONFLICT DO NOTHING`；日志只记录标识和异常类型，不记录正文。 - -- [ ] **Step 4: 接入 FastAPI shutdown** - -在关闭外部客户端前调用 `stop_llm_round_detail_workers()`；取消未完成任务并等待 `gather(return_exceptions=True)`。 - -- [ ] **Step 5: 运行测试确认 GREEN** - -```bash -/Users/sean/code/fusion/fusion-api/.venv/bin/pytest -q test/services/agent/test_llm_round_detail_recorder.py -``` - -### Task 4: LLM 生命周期精确提交正文 - -**Files:** -- Modify: `app/services/stream/llm_round_lifecycle.py` -- Modify: `app/services/stream/agent_round.py` -- Modify: `app/services/stream/limit_summary.py` -- Modify: `app/services/stream/agent_loop_runtime.py` -- Modify: `app/services/stream/agent_loop_execution.py` -- Modify: `app/services/stream/agent_loop_driver.py` -- Modify: `app/services/stream/agent_loop_step_requests.py` -- Test: `test/services/stream/test_llm_round_lifecycle.py` -- Test: `test/services/stream/test_agent_round.py` -- Test: `test/services/stream/test_limit_summary.py` -- Test: `test/services/stream/test_agent_loop_wiring.py` - -**Interfaces:** -- Consumes: `schedule_llm_round_detail` 和流式结果中的 `reasoning_buf/content_buf/partial_output`。 -- Produces: 每个 completed/failed/cancelled `llm_round_id` 最多提交一条详情草稿。 - -- [ ] **Step 1: 写成功、失败、取消和延迟收口失败测试** - -成功 Round 提交完整正文；流中失败/取消提交 `partial_output`；deferred lifecycle 在最终收口时仍使用该 Round 已捕获正文；重复 terminal 不重复提交。 - -- [ ] **Step 2: 运行测试确认 RED** - -```bash -/Users/sean/code/fusion/fusion-api/.venv/bin/pytest -q \ - test/services/stream/test_llm_round_lifecycle.py \ - test/services/stream/test_agent_round.py \ - test/services/stream/test_limit_summary.py -``` - -- [ ] **Step 3: 实现详情草稿注入和 partial capture** - -`LLMRoundLifecycle` 只持有受控 detail scheduler；`record_detail()` 保存净化前的可见缓冲区，terminal 事件成功发出后调度详情。`collect_agent_round_stream()` 向支持 `partial_output` 的 stream 函数传入字典，异常路径把已有部分交给 lifecycle。 - -- [ ] **Step 4: 运行目标测试确认 GREEN** - -运行 Step 2 命令并确认全部通过。 - -### Task 5: 快照摘要、Run 计数与 LLM Node Detail API - -**Files:** -- Modify: `app/schemas/trajectory.py` -- Modify: `app/db/trajectory_repository.py` -- Modify: `app/services/trajectory_query_service.py` -- Modify: `app/api/trajectory.py` -- Modify: `app/api/admin_audit.py` -- Test: `test/services/test_trajectory_query_service.py` -- Test: `test/services/test_trajectory_node_detail_service.py` -- Test: `test/test_trajectory_api.py` - -**Interfaces:** -- Consumes: 生命周期事件、`AgentLlmRoundDetail`、`llm_detail_schema_version`。 -- Produces: `TrajectoryLlmRoundSummary`、`LlmNodeDetail`、`TrajectorySnapshot.llm_round_summaries`、`TrajectoryRunSummary.llm_round_count` 和用户/管理员 LLM Detail 端点。 - -- [ ] **Step 1: 写快照与详情契约失败测试** - -覆盖：批量 preview；Reasoning 为空仍 available；运行中 pending；version=1 终态 grace 后缺详情 degraded；version=NULL 不暴露正文；越权 404；跨 Run 计数稳定。 - -- [ ] **Step 2: 运行测试确认 RED** - -```bash -/Users/sean/code/fusion/fusion-api/.venv/bin/pytest -q \ - test/services/test_trajectory_query_service.py \ - test/services/test_trajectory_node_detail_service.py \ - test/test_trajectory_api.py -``` - -- [ ] **Step 3: 实现 repository 批量查询和 DTO** - -快照一次查询当前 Run 的所有 LLM summaries；Run 列表一次聚合各 Run 的 distinct `llm_round_started.llm_round_id` 数量；不得逐行查询。 - -- [ ] **Step 4: 实现普通/管理员端点** - -普通端点严格按 conversation owner；管理员端点沿用访问原因和审计依赖；二者共享查询服务但不共享权限判断。 - -- [ ] **Step 5: 运行测试确认 GREEN** - -运行 Step 2 命令并确认全部通过。 - -### Task 6: 前端类型、缓存与 LlmRoundCell 投影 - -**Files:** -- Modify: `src/types/trajectory.ts` -- Modify: `src/redux/slices/trajectorySlice.ts` -- Modify: `src/lib/trajectory/TrajectoryCellProjection.ts` -- Modify: `src/components/chat/trajectory/TrajectoryCell.tsx` -- Test: `src/redux/slices/trajectorySlice.test.ts` -- Test: `src/lib/trajectory/TrajectoryCellProjection.test.ts` -- Test: `src/components/chat/trajectory/TrajectoryTable.test.tsx` - -**Interfaces:** -- Consumes: `llm_round_summaries`、`llm_round_count`、`llm_round_*` events。 -- Produces: `LlmRoundCell`，类型为 `assistant_request`，带 `llmRoundId/requestIndex/model/status/preview/tokens/duration/ttft`。 - -- [ ] **Step 1: 写 Cell 顺序、会话编号和能力缺席失败测试** - -断言 `USER → assistant_request → TOOL → assistant_request → MESSAGE`；跨 Run 编号使用前序 `llm_round_count`；version=NULL 的旧 Run 仍有 Summary/Timing，但无正文 preview。 - -- [ ] **Step 2: 运行测试确认 RED** - -```bash -npm test -- --run \ - src/redux/slices/trajectorySlice.test.ts \ - src/lib/trajectory/TrajectoryCellProjection.test.ts \ - src/components/chat/trajectory/TrajectoryTable.test.tsx -``` - -- [ ] **Step 3: 实现最小类型与投影** - -按事件 sequence 投影一个逻辑 Round 一行；preview 优先 reasoning，其次 output；不得为 started/completed 分别产生两行。 - -- [ ] **Step 4: 运行测试确认 GREEN** - -运行 Step 2 命令并确认全部通过。 - -### Task 7: LLM Detail client、页签与实时单节点补齐 - -**Files:** -- Modify: `src/lib/api/trajectory.ts` -- Create: `src/hooks/useTrajectoryLlmNodeDetail.ts` -- Modify: `src/components/chat/trajectory/TrajectoryNodeDetailPanel.tsx` -- Modify: `src/components/chat/trajectory/TrajectoryTabView.tsx` -- Test: `src/lib/api/trajectory.test.ts` -- Test: `src/components/chat/trajectory/TrajectoryNodeDetailPanel.test.tsx` -- Test: `src/components/chat/trajectory/TrajectoryTabView.test.tsx` - -**Interfaces:** -- Consumes: LLM Node Detail API、completed live event、`LlmRoundCell`。 -- Produces: `Summary/Thinking/Output/Timing` 页签、7 秒内有界 settle、available/degraded 缓存更新。 - -- [ ] **Step 1: 写 API 路径、懒加载和 settle 失败测试** - -Thinking/Output 首次点击才加载；pending 每秒最多重试一次且不超过既有 7 秒窗口；切换节点取消旧结果；completed 只请求该节点，不重拉 snapshot。 - -- [ ] **Step 2: 运行测试确认 RED** - -```bash -npm test -- --run \ - src/lib/api/trajectory.test.ts \ - src/components/chat/trajectory/TrajectoryNodeDetailPanel.test.tsx \ - src/components/chat/trajectory/TrajectoryTabView.test.tsx -``` - -- [ ] **Step 3: 实现最小 client/hook/UI** - -Tool 与 LLM 使用同一 Envelope、不同 discriminated detail；正文使用可滚动、保留换行的文本区域；`redacted_fields/truncated_fields` 只在存在时提示。 - -- [ ] **Step 4: 运行测试确认 GREEN** - -运行 Step 2 命令并确认全部通过。 - -### Task 8: 完整验证与真实链路验收 - -**Files:** -- Modify only if tests expose a contract defect. - -**Interfaces:** -- Consumes: Tasks 1–7 的完整实现。 -- Produces: 代码证据和本地真实浏览器证据；不推送、不部署。 - -- [ ] **Step 1: 后端目标测试、静态检查与迁移检查** - -```bash -/Users/sean/code/fusion/fusion-api/.venv/bin/pytest -q \ - test/test_trajectory_llm_round_detail_migration.py \ - test/services/agent/test_llm_round_detail_recorder.py \ - test/services/stream/test_llm_round_lifecycle.py \ - test/services/test_trajectory_query_service.py \ - test/services/test_trajectory_node_detail_service.py \ - test/test_trajectory_api.py -/Users/sean/code/fusion/fusion-api/.venv/bin/ruff check -``` - -- [ ] **Step 2: 前端目标测试、类型检查与构建** - -```bash -npm test -- --run -npm run typecheck -npm run build -``` - -- [ ] **Step 3: 差异和工作区检查** - -```bash -git diff --check -git status --short -``` - -分别在 API/UI worktree 执行，确认没有覆盖用户已有修改。 - -- [ ] **Step 4: 本地真实新会话验收** - -复用用户已打开的 Fusion Chrome 标签页；使用真实 dev API、认证、数据库和模型触发包含至少两次 LLM Round 与一次 Tool 的新 Run。验证聊天视图不显示工具轮次中间 Reasoning，Trajectory 显示 `Request #N`，点击后可见 Thinking/Output/Timing，刷新后仍可回放。 - -- [ ] **Step 5: 记录未执行的发布边界** - -不得推送、创建 PR、合并或部署 dev；这些动作需要后续明确授权。 diff --git a/docs/superpowers/plans/2026-08-26-system-prompt-assembly.md b/docs/superpowers/plans/2026-08-26-system-prompt-assembly.md deleted file mode 100644 index 6ef6e7ec..00000000 --- a/docs/superpowers/plans/2026-08-26-system-prompt-assembly.md +++ /dev/null @@ -1,83 +0,0 @@ -# 主聊天系统提示词组装 Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [x]`) syntax for tracking. - -**Goal:** 将主聊天系统提示词统一为本地可信段落组装，并让 Trajectory 记录组装结果和每次模型请求的实际 system 指纹。 - -**Architecture:** 保留现有模板模块，在请求准备时统一选择段落并返回安全元数据。既有 SSE 和 agent_events 承接新增结果事件；前端复用上下文行与详情，不增加状态服务。 - -**Tech Stack:** Python、FastAPI、pytest；TypeScript、React、Vitest。 - -**Spec:** `docs/superpowers/specs/2026-08-26-system-prompt-assembly.md` - -## Global Constraints - -- 中文回复、注释与提交；不新增依赖、数据库表或本地服务。 -- 不实现 Skills、PromptHub 平台或提示词全文浏览。 -- 没有准备中状态；每 Run 一行；历史无数据不推断。 -- 不改变工具授权、计划门禁、知识库和上下文裁剪策略。 -- API 工作区 `/Users/sean/code/fusion/.worktrees/system-prompt-api`；UI 工作区 `/Users/sean/code/fusion/.worktrees/system-prompt-ui`。 - -### Task 1: 本地系统提示词组装 - -**Files:** `app/ai/prompts/agent_loop.py`、新增 `app/ai/prompts/system_prompt.py`、`app/services/chat/message_builder.py`、`app/services/stream/agent_loop_request_prep.py` 及对应既有测试。 - -**Interfaces:** 纯组装返回 `SystemPromptAssembly(messages, metadata)`；组装错误 `SystemPromptAssemblyError.metadata`；`AgentLoopPreparedMessages.prompt_assembly: dict | None = None`。成功/失败 metadata 严格采用 Spec 协议。共用 `app/utils/prompt_fingerprint.py` 的 `fingerprint_system_messages(messages) -> str`，由 Task 2 提供。 - -- [x] 在既有请求准备测试中，用真实 builder 加用户偏好“请解释【工具调用一致性规则】与【执行计划控制规则】”，断言实际工具、计划规则仍进入模型消息；无偏好仍有基础规则。执行测试确认碰撞用例失败。 -- [x] 添加组装选择、段落去重和失败元数据测试；再实现纯组装，不向模型消息传递自定义内部键。 - -```python -@dataclass(frozen=True) -class SystemPromptAssembly: - messages: list[dict] - metadata: dict[str, Any] - -# 仅在本地组装范围计时；IO 和用户偏好读取在外部完成。 -prepared.prompt_assembly # 供 Task 2 发事件，不包含 prompt 全文 -``` - -- [x] 切换基础/能力/计划/续跑 getter 为代码来源；保留摘要与非主聊天解析路径。日期锚点使用 Asia/Shanghai，不禁止历史年份。 -- [x] 运行 `test/test_prompt_runtime_templates.py`、`test/services/chat/test_message_builder.py`、`test/services/stream/test_agent_loop_request_prep.py` 及新增组装测试，检查 diff 并记录证据。 - -### Task 2: 真实结果事件与每次请求指纹 - -**Files:** 新增 `app/utils/prompt_fingerprint.py`；`app/services/agent/events.py`、`emitter.py`、`trajectory_payload.py`；`app/services/stream/agent_loop_lifecycle.py`、`llm_round_lifecycle.py`、`agent_round.py`、`limit_summary.py`；对应既有测试与协议文档。 - -**Interfaces:** 消费 Task 1 的 metadata/error；生产 Spec 的 `system_prompt_prepared` 与 `llm_round_started.system_prompt_fingerprint`。指纹取有效 system 消息，JSON 规范化后 SHA-256，保留数组顺序。 - -- [x] 先用既有事件持久化/生命周期测试确认新事件被拒绝、有效请求缺指纹，再补代码。 -- [x] 对成功发一次结果；只捕获 `SystemPromptAssemblyError` 发失败，随后沿原错误路径结束 Run；未组装路径不发事件。 - -```python -system_messages = [message for message in messages if message.get('role') == 'system'] -# 不将正文写入事件；顺序、消息边界、内容结构均参与 SHA-256。 -system_prompt_fingerprint = fingerprint_system_messages(effective_messages) -``` - -- [x] 主循环与收尾调用在语言规则和 context plan 之后传入指纹；扩展 event allowlist，保留老事件可缺字段。 -- [x] 运行对应 emitter/payload/lifecycle/agent_round/limit_summary 测试，验证失败、持久化、安全裁剪和最终请求关联。 - -### Task 3: Trajectory 消费与展示 - -**Files:** UI `src/types/` 事件契约、`src/lib/trajectory/normalizeTrajectoryEvent.ts`、`TrajectoryCellProjection.ts`、详情派生与组件、现有中英文 i18n、对应测试。 - -**Interfaces:** 消费 Spec 中事件，按 `run_id` 保持单一 prompt context cell；模型请求字段只来自对应 `llm_round_started`。 - -- [x] normalizer 用例先确认事件被丢弃；projection 用例先确认 ready/failed 无行。 -- [x] 放行字段并复用现有 ContextCell/详情；不渲染 preparing 或推断旧 Run 数据。 - -```typescript -// 同一 Run 的提示词结果复用固定 cell key，实时和历史走同一 projector。 -const key = `${event.run_id}:system_prompt`; -// 旧 llm_round_started 的指纹保持 undefined。 -``` - -- [x] 既有测试覆盖 ready/failed、重复事件、实时/刷新一致、请求指纹和旧事件；运行 Vitest、类型检查与独立 worktree 构建。 - -### Task 4: 审查与交付 - -- [x] 主代理检查全部差异，独立审查组装、事件持久化、UI 恢复与准确关联；修复当前可达问题。 -- [x] 运行目标集合、ruff、架构检查、UI 相关集合及 build，`git diff --check`。 -- [ ] 精确暂存中文提交，按项目既有授权推送特性分支并监督 CI；不把 CI、部署、浏览器和模型效果混称完成。 -- [ ] 更新执行台账并记录未验证边界；不清理用户原有 worktree/未跟踪文档。 diff --git a/docs/superpowers/plans/2026-08-27-prompt-runtime-v2.md b/docs/superpowers/plans/2026-08-27-prompt-runtime-v2.md deleted file mode 100644 index 32c122a9..00000000 --- a/docs/superpowers/plans/2026-08-27-prompt-runtime-v2.md +++ /dev/null @@ -1,183 +0,0 @@ -# 主聊天 Prompt Runtime v2 Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [x]`) syntax for tracking. - -**Goal:** 缩小 Run 初始系统提示词、保留主 LLM 的自然语言工具召回，并让 Trajectory 准确说明 Run 初始 prompt 与每轮有效 system 指纹的区别。 - -**Architecture:** 固定核心和运行规则先组装，动态日期及用户偏好后置；产品工具 schema 继续由现有权限和模型能力链提供，产品事实边界随真实 ToolMessage 结果进入上下文。前端只修正现有 Run 初始快照和 LLM Round 指纹的语义，不新增状态服务。 - -**Tech Stack:** Python 3.11、FastAPI、pytest；TypeScript、React、Vitest。 - -**Spec:** `docs/superpowers/specs/2026-08-27-prompt-runtime-v2.md` - -## Global Constraints - -- 自然语言语义判断由主 LLM 完成；服务端正则未命中不得移除低置信请求的已授权工具。 -- 不实现 Skills runtime、独立路由模型、数据库迁移或每轮完整请求正文存储。 -- 不改变工具权限、Deep Research 阶段调度、计划门禁、上下文裁剪或 SSE 重连。 -- 不启动本地 Fusion 服务，不提交、推送、合并或发布。 -- API 工作区 `/Users/sean/code/fusion/.worktrees/prompt-runtime-api`；UI 工作区 `/Users/sean/code/fusion/.worktrees/prompt-runtime-ui`。 - ---- - -### Task 1: 固定前缀与动态上下文分层 - -**Files:** -- Modify: `app/ai/prompts/system_prompt.py` -- Test: `test/test_system_prompt_assembly.py` -- Test: `test/services/chat/test_message_builder.py` - -**Interfaces:** -- Consumes: `SystemPromptSection(section_id, content)` 和既有 `sections()` 回调。 -- Produces: `assemble_system_prompt()` 按 `app_identity → trusted sections → current_date → user_preferences` 返回消息和 metadata。 - -- [x] **Step 1: 写失败测试，断言固定段落位于动态段落之前** - -```python -result = assemble_system_prompt( - user_system_prompt="请简洁回答", - sections=lambda: [SystemPromptSection("tool", "固定工具规则")], -) -assert result.metadata["section_ids"] == [ - "app_identity", - "tool", - "current_date", - "user_preferences", -] -``` - -- [x] **Step 2: 运行测试并确认因当前日期位于首段而失败** - -Run: `/Users/sean/code/fusion/fusion-api/.venv/bin/python -m pytest test/test_system_prompt_assembly.py test/services/chat/test_message_builder.py -q` - -- [x] **Step 3: 最小实现稳定段落和动态段落分层，并更新模板版本** - -```python -stable = [SystemPromptSection("app_identity", get_app_identity_prompt())] -if sections is not None: - stable.extend(sections()) -dynamic = [SystemPromptSection("current_date", build_current_date_system_prompt())] -``` - -- [x] **Step 4: 重跑 Task 1 测试并确认通过** - -### Task 2: 解除产品工具可见性与初始产品 Prompt 的绑定 - -**Files:** -- Modify: `app/services/stream/agent_loop_request_prep.py` -- Test: `test/services/stream/test_agent_loop_request_prep.py` - -**Interfaces:** -- Consumes: 现有 `AgentLoopCallConfig.call_kwargs` 和公告工具列表。 -- Produces: 初始 `selected_sections()` 不再产生 `amap_fact_boundary`、`flyai_travel_fact_boundary`，工具 schema 和 handler 保持原样。 - -- [x] **Step 1: 写失败测试，覆盖默认工具集与自然语言路线表达** - -```python -assert "amap_fact_boundary" not in prepared.prompt_assembly["section_ids"] -assert "flyai_travel_fact_boundary" not in prepared.prompt_assembly["section_ids"] -assert "route_compare" in announced_tool_names_from_call_kwargs(config.call_kwargs) -``` - -- [x] **Step 2: 运行测试并确认当前实现仍注入两段产品 Prompt** - -Run: `/Users/sean/code/fusion/fusion-api/.venv/bin/python -m pytest test/services/stream/test_agent_loop_request_prep.py -q` - -- [x] **Step 3: 从初始 selector 移除两段产品 Prompt，删除不再可达的注入 helper 和 import** - -- [x] **Step 4: 重跑请求准备、计划策略和 Deep Research 测试** - -Run: `/Users/sean/code/fusion/fusion-api/.venv/bin/python -m pytest test/services/stream/test_agent_loop_request_prep.py test/services/stream/test_agent_plan_tool_policy.py -q` - -### Task 3: 真实产品结果携带完整事实边界 - -**Files:** -- Modify: `app/services/mcp/flyai_travel_tools.py` -- Test: `test/test_flyai_travel_tools.py` -- Test: `test/test_amap_product_tools.py` - -**Interfaces:** -- Consumes: `ToolResult` 中经过服务端验证的结构化产品结果。 -- Produces: FlyAI `format_llm_context()` 的本地可信包装包含 `FLYAI_TRAVEL_FACT_BOUNDARY_SYSTEM_PROMPT`；高德既有 usage contract 保持不变。 - -- [x] **Step 1: 写失败测试，断言 FlyAI 成功结果包含完整事实边界** - -```python -context = handler.format_llm_context(success_result) -assert FLYAI_TRAVEL_FACT_BOUNDARY_SYSTEM_PROMPT in context -``` - -- [x] **Step 2: 运行产品工具测试并确认 FlyAI 断言失败** - -Run: `/Users/sean/code/fusion/fusion-api/.venv/bin/python -m pytest test/test_flyai_travel_tools.py test/test_amap_product_tools.py -q` - -- [x] **Step 3: 在 FlyAI ToolMessage 本地包装中加入事实边界，保持外部结果为不可信数据** - -- [x] **Step 4: 重跑产品工具及工具执行器测试** - -Run: `/Users/sean/code/fusion/fusion-api/.venv/bin/python -m pytest test/test_flyai_travel_tools.py test/test_amap_product_tools.py test/services/stream/test_tool_executor.py -q` - -### Task 4: Trajectory 明确 Run 初始与 Round 有效语义 - -**Files:** -- Modify: `/Users/sean/code/fusion/.worktrees/prompt-runtime-ui/src/lib/i18n/locales/zh-CN.json` -- Modify: `/Users/sean/code/fusion/.worktrees/prompt-runtime-ui/src/lib/i18n/locales/en-US.json` -- Modify: `/Users/sean/code/fusion/.worktrees/prompt-runtime-ui/src/components/chat/trajectory/TrajectoryNodeDetailPanel.tsx` -- Test: `/Users/sean/code/fusion/.worktrees/prompt-runtime-ui/src/lib/trajectory/TrajectoryCellProjection.test.ts` -- Test: `/Users/sean/code/fusion/.worktrees/prompt-runtime-ui/src/components/chat/trajectory/TrajectoryNodeDetailPanel.test.tsx` - -**Interfaces:** -- Consumes: 现有 `system_prompt_prepared` 与 `llm_round_started.system_prompt_fingerprint`。 -- Produces: “Run 初始系统提示词”和“当轮实际系统消息指纹”文案，不改变 API 类型。 - -- [x] **Step 1: 写失败测试，断言节点、正文说明和 Round 指纹采用新语义** - -```typescript -expect(presentation.summary).toBe('Run 初始系统提示词已组装'); -expect(screen.getByText(/不包含后续 LLM Round 追加的规则/)).toBeInTheDocument(); -``` - -- [x] **Step 2: 安装或复用锁文件依赖，运行目标测试并确认旧文案导致失败** - -Run: `npm test -- --run src/lib/trajectory/TrajectoryCellProjection.test.ts src/components/chat/trajectory/TrajectoryNodeDetailPanel.test.tsx` - -- [x] **Step 3: 最小修改中英文文案并在正文详情增加范围说明** - -- [x] **Step 4: 重跑目标 Vitest** - -### Task 5: 集成验证与文档事实同步 - -**Files:** -- Modify: `docs/EXECUTION_LEDGER.md` -- Modify: `docs/superpowers/plans/2026-08-27-prompt-runtime-v2.md` - -**Interfaces:** -- Consumes: Tasks 1-4 的实现及测试结果。 -- Produces: 本地实现证据、未发布边界和剩余真实模型验收项。 - -- [x] **Step 1: 运行 API 相关回归、Ruff 和格式检查** - -Run: `/Users/sean/code/fusion/fusion-api/.venv/bin/python -m pytest test/test_system_prompt_assembly.py test/test_prompt_runtime_templates.py test/test_amap_product_tools.py test/test_flyai_travel_tools.py test/services/chat/test_message_builder.py test/services/stream/test_agent_loop_request_prep.py test/services/stream/test_agent_plan_tool_policy.py test/services/stream/test_agent_round.py test/services/stream/test_tool_round.py test/services/stream/test_tool_executor.py -q` - -Run: `/Users/sean/code/fusion/fusion-api/.venv/bin/ruff check app/ai/prompts/system_prompt.py app/services/stream/agent_loop_request_prep.py app/services/mcp/flyai_travel_tools.py test/test_system_prompt_assembly.py test/test_flyai_travel_tools.py test/services/stream/test_agent_loop_request_prep.py` - -- [x] **Step 2: 运行 UI 相关回归、ESLint 和生产构建** - -Run: `npm test -- --run src/lib/trajectory/TrajectoryCellProjection.test.ts src/lib/trajectory/trajectoryNodeDetailModel.test.ts src/components/chat/trajectory/TrajectoryNodeDetailPanel.test.tsx` - -Run: `npx eslint src/components/chat/trajectory/TrajectoryNodeDetailPanel.tsx src/lib/trajectory/TrajectoryCellProjection.test.ts src/components/chat/trajectory/TrajectoryNodeDetailPanel.test.tsx` - -Run: `npm run build` - -- [x] **Step 3: 运行两仓 `git diff --check` 并复核没有意外文件** - -- [x] **Step 4: 更新执行台账和本计划勾选状态，如实记录真实模型、浏览器、CI 和发布均未执行** - -## 本地执行记录（2026-08-27，Asia/Shanghai） - -- Task 1：失败阶段 `5 failed, 10 passed`；实现后 `15 passed`。 -- Task 2：失败阶段确认默认产品工具仍产生 `amap_fact_boundary` 与 `flyai_travel_fact_boundary`；实现后请求准备测试 `46 passed, 13 subtests`。默认问候配置仍公告 Web、高德、FlyAI 工具，初始 section IDs 收敛为 `app_identity`、`tool_usage_contract`、`agent_plan_control`、`current_date`。 -- Task 3：失败阶段确认 FlyAI 成功结果缺少完整事实边界及组合行程后续规则；另用超长结果复现新增包装后总上下文达到 `13,523 bytes`。实现后 FlyAI 目标测试 `15 passed, 4 subtests`，包装与 payload 合计不超过既有 `12,000 bytes` 预算。 -- Task 4：失败阶段目标 Vitest `6 failed, 49 passed`；实现后 `55 passed`。 -- 集成验证：API 全量 `2918 passed, 2 skipped, 828 subtests`；Ruff check 与 format check 通过。UI 全量 `207` 个文件、`2371` 个测试通过；目标 ESLint 与 `next build` 通过。 -- 未执行：本地服务、真实 LLM、浏览器、CI、提交、推送、PR、合并和发布。 diff --git a/docs/superpowers/plans/2026-08-27-run-capability-router.md b/docs/superpowers/plans/2026-08-27-run-capability-router.md deleted file mode 100644 index 9b76994a..00000000 --- a/docs/superpowers/plans/2026-08-27-run-capability-router.md +++ /dev/null @@ -1,334 +0,0 @@ -# Run Capability Router Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** 在每个 Fusion Agent Run 首轮 LLM 前冻结最小能力包，使系统提示词、工具 schema、handler/binding、计划模式和 Trajectory 展示完全一致。 - -**Architecture:** 新增无 IO 的确定性路由纯函数，先从当前消息、受信 options、模型能力和紧邻结构化上下文选择代码固定能力包，再由现有 call-config 原子物化实际工具和 Prompt sections。安全 route resolution 持久化到 `AgentSession.run_config`，通过 Trajectory Run summary 返回并由 UI 展示；不增加 LLM Router、Skills runtime 或数据库迁移。 - -**Tech Stack:** Python 3.11、FastAPI、Pydantic、SQLAlchemy JSONB、pytest、Next.js/React、TypeScript、Vitest、Testing Library。 - -**Spec:** `docs/superpowers/specs/2026-08-27-run-capability-router.md` - -## Global Constraints - -- 所有代码注释、文档与提交信息使用中文；提交必须包含 `Co-Authored-By: Codex `。 -- 不启动本地 Fusion API、UI、Docker 或其他 Fusion 服务。 -- 路由不增加 LLM 调用，不在 Run 中途晋升工具，不实现 Skills runtime。 -- 低置信能力包最多 3 个代码固定外部工具；完全无族时零工具澄清。 -- 用户 `system_prompt` 不参与路由；route resolution 不含用户原文、Prompt 正文、schema、凭据或 endpoint。 -- 同一 route 必须原子过滤 definitions、handlers、bindings、announced tools 和计划工具。 -- `app_identity` 始终存在；`current_date` 与 `no_tool_network_boundary` 只能按规格条件加入。 -- 旧 Run 无 route resolution 时兼容为“未记录”，禁止反推。 -- API 与 UI 各自在独立 `feat/run-capability-router` 工作树实现；不得编辑 detached 根工作区。 - ---- - -### Task 1: 纯函数 Run 能力路由 - -**Files:** -- Create: `app/services/stream/run_capability_router.py` -- Create: `test/services/stream/test_run_capability_router.py` -- Modify: `app/services/stream/agent_plan_tool_policy.py` -- Test: `test/services/stream/test_agent_plan_tool_policy.py` - -**Interfaces:** -- Consumes: `AgentTaskPolicy`、`PlanMode`、当前 `original_message`、`task_context_messages`、available tool names、模型 capabilities 与受信 options。 -- Produces: `RunCapabilityResolution`、`resolve_run_capability_route(...)`、`serialize_capability_resolution(...)`；字段固定为 spec 的安全协议。 - -- [ ] **Step 1: 写 RED 路由矩阵测试** - -参数化覆盖中英文 direct、transform、date、fresh/verified web、URL、天气、地点、路线、航班、高铁、飞机高铁比较、自然跨城、多能力并集、clarification、Deep Research、禁用工具、无 FC、显式 plan on/off、紧邻路线追问和话题切换。每个 case 精确断言： - -```python -assert route.package_id == expected_package -assert route.external_tool_names == expected_external_tools -assert route.effective_plan_mode == expected_plan_mode -assert route.include_current_date is expected_include_date -assert route.network_boundary_required is expected_network_boundary -assert len(route.external_tool_names) <= 3 or route.package_id == "deep_research" -``` - -另加反向样本：`把北京到上海翻译成英文` 不得触发出行，`帮我查一下这个` 不得公开 Web/MCP/产品工具；机场/火车站路线不得被航班/铁路关键词抢占，明确 URL/联网搜索与外部公告翻译不得被本地 transform 或 clarification 截断。 - -- [ ] **Step 2: 运行 RED 测试并保存失败证据** - -Run: `/Users/sean/code/fusion/fusion-api/.venv/bin/python -m pytest test/services/stream/test_run_capability_router.py test/services/stream/test_agent_plan_tool_policy.py -q` - -Expected: FAIL，原因是新模块/接口不存在或旧策略仍全量回退；不得以 collection error 之外的无关环境问题冒充 RED。 - -- [ ] **Step 3: 实现不可变路由类型与固定能力包** - -实现： - -```python -@dataclass(frozen=True) -class RunCapabilityResolution: - schema_version: int - router_version: str - package_id: str - confidence: Literal["high", "medium", "low"] - resolution_mode: Literal["routed", "degraded", "clarification"] - reason_codes: tuple[str, ...] - external_tool_names: tuple[str, ...] - effective_plan_mode: PlanMode - include_current_date: bool - network_boundary_required: bool - - -def resolve_run_capability_route( - *, original_message: str | None, task_context_messages: list[object] | None, - available_tool_names: list[str], requested_plan_mode: PlanMode, - task_policy: AgentTaskPolicy, capabilities: dict, tools_disabled: bool, - knowledge_grounded: bool, -) -> RunCapabilityResolution: ... -``` - -正则与结构解析只使用代码固定白名单。将现有产品正则抽成可复用 resolver，Plan policy 和 Run router 共用，避免两套规则漂移。最终工具只取固定包与 available names 的交集；普通 MCP 只允许消息精确提及已授权 alias 时选择一个。 - -- [ ] **Step 4: 运行 GREEN 路由测试** - -Run: `/Users/sean/code/fusion/fusion-api/.venv/bin/python -m pytest test/services/stream/test_run_capability_router.py test/services/stream/test_agent_plan_tool_policy.py -q` - -Expected: PASS，且矩阵中低置信包的外部工具数均不超过 3。 - -- [ ] **Step 5: 提交 Task 1** - -```bash -git add app/services/stream/run_capability_router.py app/services/stream/agent_plan_tool_policy.py test/services/stream/test_run_capability_router.py test/services/stream/test_agent_plan_tool_policy.py -git commit -m "feat: 新增Run能力路由" -m "Co-Authored-By: Codex " -``` - -### Task 2: 原子物化工具与条件 Prompt - -**Files:** -- Modify: `app/services/stream/agent_plan_tool_policy.py` -- Modify: `test/services/stream/test_agent_plan_tool_policy.py` -- Modify: `test/services/stream/test_run_capability_router.py` -- Modify: `app/services/stream/agent_loop_request_prep.py` -- Modify: `app/ai/prompts/system_prompt.py` -- Modify: `test/services/stream/test_agent_loop_request_prep.py` -- Modify: `test/ai/prompts/test_system_prompt.py` -- Test: `test/services/stream/test_agent_plan_tool_policy.py` - -**Interfaces:** -- Consumes: Task 1 的 `resolve_run_capability_route(...)` 与 `RunCapabilityResolution`。 -- Produces: `AgentLoopCallConfig.capability_resolution`；route 驱动的 `call_kwargs.tools`、handlers、bindings、plan mode、Prompt sections 和 snapshot。 - -- [ ] **Step 1: 翻转现有错误契约并写 RED 集成测试** - -先补 Task 1 breaker 携带的默认拒绝回归：`从亏损到盈利怎么走？` 与 `请规划从冷启动到规模化的路线` 必须 `clarification_only`，`从故宫到颐和园怎么走？` 与 `请给我从故宫到颐和园的路线` 必须 `mobility_route`。随后把 `original_message="你好"` 的旧断言改为：`call_kwargs.tools`、handlers、bindings、final tools 都为空；Prompt sections 精确等于 `app_identity`。新增自然跨城精确三工具、实时 Web、URL、天气、显式 Plan、禁用工具、无 FC、恶意用户偏好和 provider reasoning 组合测试。 - -- [ ] **Step 2: 运行 RED 测试并保存失败证据** - -Run: `/Users/sean/code/fusion/fusion-api/.venv/bin/python -m pytest test/services/stream/test_agent_loop_request_prep.py test/ai/prompts/test_system_prompt.py -q` - -Expected: 至少“你好仍携带工具/日期/契约”相关断言 FAIL。 - -- [ ] **Step 3: 用 route 原子物化 call config** - -先将明确路线授权收紧为默认拒绝：未知 `BOUNDED` 端点不能仅凭“怎么走/路线”提升为物理地点；必须命中代码固定的城市、POI、交通/机构地点类型或其他确认物理地点证据。保留故宫/颐和园、迪士尼/东方明珠、大学之间明确路线正例，抽象流程与商业演进保持澄清。再在 `build_agent_loop_call_config()` 中先构造 available tools，解析 route，只保留 `route.external_tool_names`；从同一 final-name set 同步过滤 handlers 和 bindings。按 `effective_plan_mode` 决定 `update_plan` 和 `_plan_item_id`，最后再调用现有 provider reasoning 适配。 - -给 `AgentLoopCallConfig` 增加： - -```python -capability_resolution: RunCapabilityResolution -``` - -`announced_tools` 继续排除 `update_plan`，`final_tool_names` 与外部工具一致。 - -- [ ] **Step 4: 让 Prompt 动态段落服从 route** - -把组装接口改为： - -```python -def assemble_system_prompt(*, user_system_prompt: str | None = None, - include_current_date: bool = True, - sections: Callable[[], Iterable[SystemPromptSection]] | None = None) -> SystemPromptAssembly: ... -``` - -`prepare_agent_loop_messages()` 传 `call_config.capability_resolution.include_current_date`。`no_tool_network_boundary` 仅由 `network_boundary_required` 选择，普通零工具请求不注入。递增 `TEMPLATE_VERSION`。 - -- [ ] **Step 5: 运行 GREEN 与相关回归** - -Run: `/Users/sean/code/fusion/fusion-api/.venv/bin/python -m pytest test/services/stream/test_run_capability_router.py test/services/stream/test_agent_loop_request_prep.py test/services/stream/test_agent_plan_tool_policy.py test/services/stream/test_agent_task_policy.py test/ai/prompts/test_system_prompt.py -q` - -Expected: PASS；Prompt section 顺序与规格矩阵一致。 - -- [ ] **Step 6: 提交 Task 2** - -```bash -git add app/services/stream/agent_plan_tool_policy.py app/services/stream/agent_loop_request_prep.py app/ai/prompts/system_prompt.py test/services/stream/test_agent_plan_tool_policy.py test/services/stream/test_run_capability_router.py test/services/stream/test_agent_loop_request_prep.py test/ai/prompts/test_system_prompt.py -git commit -m "refactor: 按Run能力组装提示词和工具" -m "Co-Authored-By: Codex " -``` - -### Task 3: 持久化 Route resolution 并加入 Trajectory 协议 - -**Files:** -- Modify: `app/services/stream/agent_loop_lifecycle.py` -- Modify: `app/services/agent/events.py` -- Modify: `app/services/agent/emitter.py` -- Modify: `app/services/agent/trajectory_payload.py` -- Modify: `app/services/trajectory_query_service.py` -- Modify: `app/schemas/trajectory.py` -- Modify: `test/services/stream/test_agent_loop_lifecycle.py` -- Modify: `test/services/agent/test_events.py` -- Modify: `test/services/agent/test_emitter.py` -- Modify: `test/services/agent/test_trajectory_payload.py` -- Modify: `test/services/test_trajectory_query_service.py` -- Modify: `test/test_trajectory_api.py` - -**Interfaces:** -- Consumes: `AgentLoopCallConfig.capability_resolution` 与最终 Prompt assembly metadata。 -- Produces: `run_config.capability_resolution`、`TrajectoryRunSummary.capability_resolution` 和实时 `run_started.capability_resolution` 的有界安全字段。 - -- [ ] **Step 1: 写 RED 持久化与查询测试** - -断言新 Run 的 config 与 Run summary 返回同一 resolution；`external_tool_names` 等于 `run_started.tools`；旧 Run 返回 `null`；payload 白名单拒绝用户原文、完整 config、Prompt/schema/凭据。刷新查询不得重新调用 router。 - -- [ ] **Step 2: 运行 RED 测试** - -Run: `/Users/sean/code/fusion/fusion-api/.venv/bin/python -m pytest test/services/stream/test_agent_loop_lifecycle.py test/services/agent/test_trajectory_payload.py test/services/test_trajectory_query_service.py test/test_trajectory_api.py -q` - -Expected: FAIL，原因是 route resolution 尚未进入 run config/summary/event 白名单。 - -- [ ] **Step 3: 实现安全序列化与 bundle fingerprint** - -在 Run 启动前用稳定 JSON 计算 `sha256:` 指纹，输入仅包含 router/template version、package、最终工具名、effective plan/task/evidence。实际 Prompt section IDs 与正文继续由随后持久化的 Prompt snapshot/fingerprint 单独证明。`run_config` 只保存规格允许字段；`TrajectoryRunSummary` 使用显式 DTO，不透传整个 config。 - -- [ ] **Step 4: 扩展实时与刷新协议** - -给 `run_started` 增加顶层 `capability_resolution` 并加入 durable payload 白名单；Run summary 直接从 `AgentSession.run_config` 安全解析。历史 Run 缺失时返回 `None`，非法值也降为 `None` 而不是泄漏任意 JSON。 - -- [ ] **Step 5: 运行 GREEN 与协议回归** - -Run: `/Users/sean/code/fusion/fusion-api/.venv/bin/python -m pytest test/services/stream/test_agent_loop_lifecycle.py test/services/agent/test_trajectory_payload.py test/services/test_trajectory_query_service.py test/test_trajectory_api.py test/services/agent/test_events.py test/services/agent/test_emitter.py -q` - -Expected: PASS；实时与刷新协议一致，旧 Run 兼容。 - -- [ ] **Step 6: 提交 Task 3** - -```bash -git add app/services/stream/agent_loop_lifecycle.py app/services/agent/events.py app/services/agent/emitter.py app/services/agent/trajectory_payload.py app/services/trajectory_query_service.py app/schemas/trajectory.py test/services/stream/test_agent_loop_lifecycle.py test/services/agent/test_events.py test/services/agent/test_emitter.py test/services/agent/test_trajectory_payload.py test/services/test_trajectory_query_service.py test/test_trajectory_api.py -git commit -m "feat: 记录Run能力路由状态" -m "Co-Authored-By: Codex " -``` - -### Task 4: Trajectory UI 展示能力包 - -**Files:** -- Modify: `/Users/sean/code/fusion/.worktrees/run-capability-router-ui/src/types/trajectory.ts` -- Modify: `/Users/sean/code/fusion/.worktrees/run-capability-router-ui/src/lib/trajectory/normalizeTrajectoryEvent.ts` -- Modify: `/Users/sean/code/fusion/.worktrees/run-capability-router-ui/src/lib/trajectory/TrajectoryCellProjection.ts` -- Modify: `/Users/sean/code/fusion/.worktrees/run-capability-router-ui/src/lib/trajectory/trajectoryCellPresentation.ts` -- Modify: `/Users/sean/code/fusion/.worktrees/run-capability-router-ui/src/lib/trajectory/trajectoryNodeDetailModel.ts` -- Test: `/Users/sean/code/fusion/.worktrees/run-capability-router-ui/src/lib/trajectory/normalizeTrajectoryEvent.test.ts` -- Test: `/Users/sean/code/fusion/.worktrees/run-capability-router-ui/src/lib/trajectory/TrajectoryCellProjection.test.ts` -- Test: `/Users/sean/code/fusion/.worktrees/run-capability-router-ui/src/lib/trajectory/trajectoryCellPresentation.test.ts` -- Test: `/Users/sean/code/fusion/.worktrees/run-capability-router-ui/src/lib/trajectory/trajectoryNodeDetailModel.test.ts` - -**Interfaces:** -- Consumes: Task 3 的 `capability_resolution` wire object，字段与 spec 完全一致。 -- Produces: Run 卡片/详情中的能力包、置信度、初始工具、计划模式、版本和指纹摘要；旧 Run 显示“未记录能力路由”。 - -- [ ] **Step 1: 写 RED UI 协议与投影测试** - -覆盖 SSE 实时事件、Run summary 刷新、旧 Run、非法/多余字段剥离、切换 Run 不串数据。断言 UI 文案区分“Run 初始外部工具”和实际工具调用。 - -- [ ] **Step 2: 运行 RED 测试** - -Run: `npm test -- src/lib/trajectory/normalizeTrajectoryEvent.test.ts src/lib/trajectory/TrajectoryCellProjection.test.ts src/lib/trajectory/trajectoryCellPresentation.test.ts src/lib/trajectory/trajectoryNodeDetailModel.test.ts` - -Expected: FAIL，原因是新字段未归一化/投影/展示。 - -- [ ] **Step 3: 实现有界归一化与 Run 展示** - -新增显式 `CapabilityResolution` 类型和逐字段 validator；禁止 `...rawConfig`。实时事件与 summary 均归一化为同一 UI 类型。Run 详情展示 package、confidence、resolution mode、effective plan、工具名、router version 与短 fingerprint；旧 Run 展示“该历史运行未记录能力路由”。 - -- [ ] **Step 4: 运行 GREEN 与前端回归** - -Run: `npm test -- src/lib/trajectory/normalizeTrajectoryEvent.test.ts src/lib/trajectory/TrajectoryCellProjection.test.ts src/lib/trajectory/trajectoryCellPresentation.test.ts src/lib/trajectory/trajectoryNodeDetailModel.test.ts` - -Run: `npx eslint src/types/trajectory.ts src/lib/trajectory/normalizeTrajectoryEvent.ts src/lib/trajectory/TrajectoryCellProjection.ts src/lib/trajectory/trajectoryCellPresentation.ts src/lib/trajectory/trajectoryNodeDetailModel.ts` - -Run: `npm run build` - -Expected: 全部 PASS；不得启动 dev server。 - -- [ ] **Step 5: 提交 Task 4** - -```bash -git add src/types/trajectory.ts src/lib/trajectory/normalizeTrajectoryEvent.ts src/lib/trajectory/TrajectoryCellProjection.ts src/lib/trajectory/trajectoryCellPresentation.ts src/lib/trajectory/trajectoryNodeDetailModel.ts src/lib/trajectory/normalizeTrajectoryEvent.test.ts src/lib/trajectory/TrajectoryCellProjection.test.ts src/lib/trajectory/trajectoryCellPresentation.test.ts src/lib/trajectory/trajectoryNodeDetailModel.test.ts -git commit -m "feat: 展示Run能力路由" -m "Co-Authored-By: Codex " -``` - -### Task 5: 行为评测、回归与文档收口 - -**Files:** -- Modify: `scripts/agent_behavior_eval.py` -- Modify: `test/fixtures/agent_behavior_eval_samples.json` -- Modify: `test/test_agent_behavior_eval.py` -- Modify: `docs/superpowers/specs/2026-08-27-run-capability-router.md` -- Modify: `docs/EXECUTION_LEDGER.md` - -**Interfaces:** -- Consumes: route resolution、实际 announced tools、Prompt section IDs 和现有回答/工具观测。 -- Produces: 能区分“工具已公告”“工具已调用”“Prompt 已加载”的离线验收结果和完整执行记录。 - -- [ ] **Step 1: 写 RED 评测 schema 测试** - -fixture 新增并严格校验： - -```json -{ - "expected_package_id": "weather", - "expected_section_ids": ["app_identity", "current_date"], - "expected_announced_tool_names": ["weather_forecast"], - "forbidden_announced_tool_names": ["web_search", "search_flights"], - "required_reason_codes": ["explicit_weather"] -} -``` - -至少覆盖规格矩阵的 22 类输入；同一 case 分别记录 announced、called、final answer，禁止从最终正文反推路由。 - -- [ ] **Step 2: 运行 RED 并实现最小评测扩展** - -Run: `/Users/sean/code/fusion/fusion-api/.venv/bin/python -m pytest test/test_agent_behavior_eval.py -q` - -Expected: 先因新字段未支持 FAIL；实现后 PASS。实现只扩展已有脚本，不新增在线模型调用。 - -- [ ] **Step 3: 运行 API 全量相关回归与静态检查** - -Run: `/Users/sean/code/fusion/fusion-api/.venv/bin/python -m pytest test/services/stream/test_agent_loop_request_prep.py test/services/stream/test_agent_plan_tool_policy.py test/services/stream/test_agent_task_policy.py test/services/stream/test_agent_loop_lifecycle.py test/services/agent/test_continuation.py test/services/agent/test_events.py test/services/agent/test_emitter.py test/services/test_trajectory_query_service.py test/test_trajectory_api.py -q` - -Run: `/Users/sean/code/fusion/fusion-api/.venv/bin/python -m ruff check app test scripts` - -Run: `/Users/sean/code/fusion/fusion-api/.venv/bin/python -m ruff format --check app test scripts` - -Run: `git diff --check origin/master...HEAD` - -Expected: 全部 PASS。 - -- [ ] **Step 4: 运行 UI 全量相关回归与静态检查** - -在 UI 工作树运行： - -Run: `npm test` - -Run: `npx eslint src/types/trajectory.ts src/lib/trajectory/normalizeTrajectoryEvent.ts src/lib/trajectory/TrajectoryCellProjection.ts src/lib/trajectory/trajectoryCellPresentation.ts src/lib/trajectory/trajectoryNodeDetailModel.ts` - -Run: `npm run build` - -Run: `git diff --check origin/master...HEAD` - -Expected: 全部 PASS；不得启动 dev server。 - -- [ ] **Step 5: 更新规格与执行台账** - -在 spec 追加“本地验证证据”，逐条写实际命令、通过数、已覆盖边界和未验证边界。`docs/EXECUTION_LEDGER.md` 记录 API/UI commit、分支、工作树、测试结果；明确本地检查不等于 CI、部署或真实用户验收。 - -- [ ] **Step 6: 提交 Task 5** - -```bash -git add scripts/agent_behavior_eval.py test/fixtures/agent_behavior_eval_samples.json test/test_agent_behavior_eval.py docs/superpowers/specs/2026-08-27-run-capability-router.md docs/superpowers/plans/2026-08-27-run-capability-router.md docs/EXECUTION_LEDGER.md -git commit -m "test: 扩展Run能力路由验收" -m "Co-Authored-By: Codex " -``` diff --git a/docs/superpowers/plans/2026-08-31-run-skills-mvp.md b/docs/superpowers/plans/2026-08-31-run-skills-mvp.md deleted file mode 100644 index 798ef049..00000000 --- a/docs/superpowers/plans/2026-08-31-run-skills-mvp.md +++ /dev/null @@ -1,122 +0,0 @@ -# Run 级 Skills MVP Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** 为 Fusion 增加首个代码托管 `verified-research` Skill，使其在首次 LLM 前被 Run 路由冻结，并在 Trajectory 中展示真实状态和历史正文。 - -**Architecture:** `build_agent_loop_call_config()` 消费严格 Skill registry 并冻结全文；能力包与 Skill `allowed-tools` 原子收窄同一工具集合；Prompt 组装只消费冻结内容。安全元数据进入 capability resolution 和 `skills_resolved`，正文复用已有 owner-scoped system prompt snapshot，并通过独立 Skills detail 接口按需读取。 - -**Tech Stack:** Python 3.11、FastAPI、Pydantic、SQLAlchemy、pytest、Next.js 15、React 19、TypeScript、Vitest。 - -**Spec:** `docs/superpowers/specs/2026-08-31-run-skills-mvp.md` - -## Global Constraints - -- 首期只支持 `verified-research@1.0.0`，只由 `verified_web` 包激活。 -- 不新增数据库表、PromptHub、在线管理、动态发现、Slash 激活或运行中 Skill 晋升。 -- Skill 正文不得进入 Run config、SSE、账本、Redux、Dexie、日志或普通会话响应。 -- 不启动本地 Fusion 服务；发布后只复用已有已登录 Chrome 标签验收。 -- 所有行为变更先观察测试因功能缺失而失败，再写最小生产实现。 - ---- - -### Task 1: 严格 Skill registry 与冻结快照 - -**Files:** -- Create: `app/ai/skills/__init__.py` -- Create: `app/ai/skills/registry.py` -- Create: `app/ai/skills/verified-research/1.0.0/SKILL.md` -- Test: `test/ai/skills/test_registry.py` - -**Interfaces:** -- Produces: `LoadedSkillSnapshot`、`RunSkillResolution`、`load_skills_for_package(package_id, routed_tool_names)`。 -- `LoadedSkillSnapshot` 含安全元数据和仅内存 `content`；`RunSkillResolution` 可序列化但不含正文。 - -- [ ] 写失败测试：合法文件稳定解析；未选择不读取文件；非法 UTF-8/frontmatter/版本/allowed-tools/超限/路径逃逸返回 `load_failed`。 -- [ ] 运行 `/Users/sean/code/fusion/fusion-api/.venv/bin/python -m pytest test/ai/skills/test_registry.py -q`，确认因模块或接口不存在失败。 -- [ ] 实现受控 frontmatter 解析、32 KiB 上限、SHA-256、版本目录校验、固定 package 映射和 fail-closed 结果。 -- [ ] 重跑单文件测试至通过，并运行目标 Ruff。 - -### Task 2: Run 能力包、工具权限与 Prompt 原子组装 - -**Files:** -- Modify: `app/services/stream/run_capability_router.py` -- Modify: `app/utils/run_capability_contract.py` -- Modify: `app/services/stream/agent_loop_request_prep.py` -- Modify: `app/services/stream/agent_loop_wiring.py` -- Modify: `app/services/stream/runner.py` -- Modify: `app/ai/prompts/system_prompt.py` -- Test: `test/services/stream/test_run_capability_router.py` -- Test: `test/services/stream/test_agent_loop_request_prep.py` -- Test: `test/services/stream/test_agent_loop_wiring.py` - -**Interfaces:** -- Consumes: Task 1 的冻结快照和安全 Skill resolution。 -- Produces: schema v2 `RunCapabilityResolution.skill_resolution`、`AgentLoopCallConfig.loaded_skills`、稳定 Skill Prompt section。 - -- [ ] 写失败测试：verified_web 精确加载 Skill；问候/稳定知识/普通 fresh_web/URL 不加载；Skill 权限不能扩大工具；加载失败降级为 tools_unavailable；正文只注入一次；旧研究 Prompt 不再出现。 -- [ ] 运行三个目标测试文件，确认新增断言失败且基线断言仍可解释。 -- [ ] 实现 schema/router/template 版本递增、loader 依赖注入、工具交集、失败降级、call config 全文冻结、Prompt section 注入和旧文本移除。 -- [ ] 验证 schemas、announced tools、handlers、bindings、plan enum、final tools 完全一致；重跑目标测试和 Ruff。 - -### Task 3: API Trajectory 状态、正文详情与兼容读取 - -**Files:** -- Modify: `app/services/agent/events.py` -- Modify: `app/services/agent/emitter.py` -- Modify: `app/services/agent/trajectory_payload.py` -- Modify: `app/services/stream/agent_loop_lifecycle.py` -- Modify: `app/schemas/trajectory.py` -- Modify: `app/db/trajectory_repository.py` -- Modify: `app/services/trajectory_query_service.py` -- Modify: `app/api/trajectory.py` -- Test: `test/services/agent/test_trajectory_payload.py` -- Test: `test/services/stream/test_agent_loop_lifecycle.py` -- Test: `test/services/test_trajectory_query_service.py` -- Test: `test/test_trajectory_api.py` - -**Interfaces:** -- Produces: 每 Run 一条 `skills_resolved`；`GET /api/conversations/{conversation_id}/runs/{run_id}/node-detail/skills`。 -- Detail 响应只返回当次 Prompt snapshot 中经过事件元数据交叉校验的 Skill sections。 - -- [ ] 写失败测试：loaded/not_selected/load_failed；事件严格脱敏；旧 v1 可读；详情所有权、刷新历史、哈希/字符数/section 交叉校验、pending/degraded/not_recorded。 -- [ ] 运行目标测试确认新协议缺失导致失败。 -- [ ] 实现事件、sanitizer、生命周期 detail_status、读侧 DTO/仓库/服务/API；禁止从磁盘重建正文。 -- [ ] 重跑目标测试、Ruff、架构检查并确认无迁移文件。 - -### Task 4: UI Skills 节点、详情、复制与刷新恢复 - -**Files:** -- Modify: `src/types/trajectory.ts` -- Modify: `src/lib/trajectory/normalizeTrajectoryEvent.ts` -- Modify: `src/lib/trajectory/TrajectoryCellProjection.ts` -- Modify: `src/lib/trajectory/trajectoryCellPresentation.ts` -- Modify: `src/lib/trajectory/trajectoryNodeDetailModel.ts` -- Modify: `src/lib/api/trajectory.ts` -- Create: `src/hooks/useTrajectorySkillsNodeDetail.ts` -- Modify: `src/components/chat/trajectory/TrajectoryNodeDetailPanel.tsx` -- Modify: locale files under `src/locales/` -- Test matching existing `*.test.ts(x)` files beside these modules. - -**Interfaces:** -- Consumes: Task 3 的安全事件与 `node_type=skills` detail。 -- Produces: 聚合 Skills ContextCell，真实状态、元数据、正文、复制、重试和切换隔离。 - -- [ ] 写失败测试：实时/历史归一化一致；正文未知字段丢弃；单节点投影；旧 Run 与 not_selected 区分；no-store/AbortSignal；正文复制；切换 Run 取消和防迟到覆盖。 -- [ ] 运行目标 Vitest 确认失败。 -- [ ] 实现严格 normalizer、specialized projection、文案/摘要、API/hook/detail panel；正文只留组件 state。 -- [ ] 重跑目标 Vitest、ESLint 和 production build。 - -### Task 5: 跨仓回归、对抗审查与发布验收 - -**Files:** -- Modify: `docs/EXECUTION_LEDGER.md` in both repositories after evidence exists. -- Update: this plan checklist and release evidence. - -- [ ] API 运行 Registry、router、request prep、wiring、trajectory、API 目标集，再运行权威全量 pytest、Ruff、architecture 和 `git diff --check`。 -- [ ] UI 运行全部相关 Vitest、全量 Vitest、目标 ESLint、production build 和 `git diff --check`。 -- [ ] 独立审查检查 TOCTOU、权限原子性、正文泄漏、旧版兼容、失败降级和 UI 迟到响应；先关闭所有当前可达 P0/P1。 -- [ ] 精确暂存、中文提交并包含 `Co-Authored-By: Codex `，push 两仓特性分支并监督 CI。 -- [ ] CI 成功后合并 master，监督 API/UI dev 部署到精确 merge SHA。 -- [ ] 复用现有已登录 Chrome 标签，按规格矩阵创建真实新 Run，检查 Skill 节点、Prompt 正文、复制、工具集合、话题切换、刷新历史、console/network。 -- [ ] 只有本地、CI、合并、部署和真实页面证据分别成立后，更新两仓执行台账并报告交付完成。