diff --git a/CLAUDE.md b/CLAUDE.md index 540ef20..3b93fd0 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -41,86 +41,93 @@ - 重写参考： - Python 版本 CLI：`../awiki-agent-id-message/` - 飞书 CLI：`../cli/` - - ANP Go SDK（远端模块依赖）：`github.com/agent-network-protocol/anp/golang@v0.8.5` + - ANP Go SDK：依赖基线为远端发布版 `github.com/agent-network-protocol/anp/golang@v0.8.7`，P5 secure direct / OPK API 已随该版本发布，主线不得提交同级工作区 `replace` ## 成员清单 -**README.md**: 仓库入口说明文件。 +**README.md**: 仓库入口说明文件。 **config.template.yaml**: 标准用户主配置模板，展示当前 canonical `config.yaml` 字段与默认值；`service_base_url` 是平台服务入口，`did_domain` 可独立指定租户 DID provider domain。 -**go.mod / go.sum**: Go 模块定义与依赖锁定；当前 Go 版本基线固定为 `1.22`，直接依赖 `cobra`、`gojq`、`modernc.org/sqlite` 与远端模块 `github.com/agent-network-protocol/anp/golang@v0.8.5`，要求 pure Go。上游 / 间接依赖树中可能仍出现 secp256k1 相关库，但 `awiki-cli` 当前本地 DID 主路径已统一为 `e1` / Ed25519。 -**cmd/awiki-cli/main.go**: `awiki-cli` 主程序入口。 -**internal/buildinfo/buildinfo.go**: 版本、构建时间、CGO 状态等构建信息。 -**internal/cmdmeta/catalog.go**: 静态命令元数据目录，作为 schema/命令骨架的事实来源。 +**go.mod / go.sum**: Go 模块定义与依赖锁定；当前 Go 版本基线固定为 `1.22`，直接依赖 `cobra`、`gojq`、`modernc.org/sqlite` 与 ANP Go SDK `github.com/agent-network-protocol/anp/golang@v0.8.7`，要求 pure Go。P5 secure direct 通过远端发布版 SDK 消费 `OneTimePrekey`、`NewFileOneTimePrekeyStore` 与 top-level OPK publish/get API；主线不得提交 `replace => ../anp/anp/golang` 这类工作区本地依赖。上游 / 间接依赖树中可能仍出现 secp256k1 相关库，但 `awiki-cli` 当前本地 DID 主路径已统一为 `e1` / Ed25519。 +**cmd/awiki-cli/main.go**: `awiki-cli` 主程序入口。 +**internal/buildinfo/buildinfo.go**: 版本、构建时间、CGO 状态等构建信息。 +**internal/cmdmeta/catalog.go**: 静态命令元数据目录，作为 schema/命令骨架的事实来源。 **internal/config/config.go**: 单根目录工作区路径解析（默认 `~/.awiki-cli/`）、仅支持 `AWIKI_CLI_WORKSPACE_HOME_DIR` 作为工作区环境变量，并统一解析 `config.yaml`；旧 `config.json` 由 workspace upgrade 在首次访问时自动迁移到 `config.yaml`，其余历史业务环境变量不再驱动 awiki-cli 行为；默认 `ANPMessageService` 从 `service_base_url` 推导而不是从 `did_domain` 推导。 -**internal/output/output.go**: 统一 success/error JSON envelope、`--jq`、table/ndjson 渲染。 -**internal/doctor/doctor.go**: 诊断实现，检查构建、配置、env、identity store、SQLite、legacy 路径与 legacy DB；SQLite 检查会额外暴露 `contact_handle_bindings` 历史映射表状态与行数。 -**internal/docs/topics.go**: CLI 内建 docs 主题索引，`skills` 主题引用当前 single-entry `skills/SKILL.md` 与懒加载 `skills/references/*.md` 拓扑。 -**internal/anpsdk/registry.go**: ANP Go SDK 的远端模块依赖入口，统一暴露 DID WBA、HTTP Signatures、direct_e2ee 等后续 Phase 要用到的基础能力。 -**internal/authsdk/session.go**: 基于 ANP SDK `DIDWbaAuthHeader` 的身份鉴权封装，负责 HTTP/WSS hop auth、401 重试、JWT token 捕获与持久化。 -**internal/cli/app.go**: CLI 应用装配、配置解析与统一错误输出入口。 -**internal/cli/root.go**: Cobra 根命令、顶级命令树、status/docs/schema/doctor/version/init/config show 的实现。 -**internal/cli/init.go**: `init` 命令处理器，负责初始化工作区目录、upgrade 目录和最小 `config.yaml`。 +**internal/output/output.go**: 统一 success/error JSON envelope、`--jq`、table/ndjson 渲染。 +**internal/doctor/doctor.go**: 诊断实现，检查构建、配置、env、identity store、SQLite、legacy 路径、legacy DB 与 `anp-mls` binary/版本/状态目录；SQLite 检查会额外暴露 `contact_handle_bindings` 历史映射表状态与行数，MLS 检查会同时扫描 root 与 agent/device-scoped `state.db`/`state.lock`。 +**internal/docs/topics.go**: CLI 内建 docs 主题索引，`skills` 主题引用当前 single-entry `skills/SKILL.md` 与懒加载 `skills/references/*.md` 拓扑。 +**internal/anpsdk/registry.go**: ANP Go SDK 的远端模块依赖入口，统一暴露 DID WBA、HTTP Signatures、direct_e2ee 等后续 Phase 要用到的基础能力。 +**internal/authsdk/session.go**: 基于 ANP SDK `DIDWbaAuthHeader` 的身份鉴权封装，负责 HTTP/WSS hop auth、401 重试、JWT token 捕获与持久化。 +**internal/cli/app.go**: CLI 应用装配、配置解析与统一错误输出入口。 +**internal/cli/root.go**: Cobra 根命令、顶级命令树、status/docs/schema/doctor/version/init/config show 的实现。 +**internal/cli/init.go**: `init` 命令处理器，负责初始化工作区目录、upgrade 目录和最小 `config.yaml`。 **internal/cli/id.go**: `id` 域命令处理器，包含 create/list/current/use/register/bind/resolve/recover/profile/import-v1，以及公开但危险的维护命令 `replace-did`。 -**internal/cli/debug.go**: `debug db query`、`debug db handle-history` 与 `debug db import-v1` 的 CLI 处理器。 -**internal/cli/msg.go**: `msg send/inbox/history/mark-read` 的 CLI 处理器，现已支持 direct + group plain messaging。 -**internal/cli/group.go**: `group create/get/join/add/remove/leave/update/members/messages` 的 CLI 处理器。 -**internal/identity/types.go**: identity store、legacy scan、command result 等核心类型。 -**internal/identity/layout.go**: identity 根目录、index.json、路径与安全写入辅助。 -**internal/identity/store.go**: 当前 v2 identity store 的读写、默认 identity 管理。 -**internal/identity/legacy.go**: v1 indexed/flat credential layout 扫描与导入。 -**internal/identity/did.go**: 本地 DID 文档与 proof 生成，当前默认生成 `e1` profile DID（`key-1` 为 Ed25519）。 -**internal/identity/key_compat.go**: legacy ANP 私有 PEM 标签 / SEC1 私钥到标准 PKCS#8 PEM 的兼容迁移，确保旧身份在 ANP Go SDK 0.8.5+ 下仍可完成 DID WBA 签名。 -**internal/identity/client.go**: user-service RPC/REST 客户端。 +**internal/cli/debug.go**: `debug db query`、`debug db handle-history` 与 `debug db import-v1` 的 CLI 处理器。 +**internal/cli/msg.go**: `msg send/inbox/history/mark-read` 的 CLI 处理器，现已支持 direct + group plain messaging。 +**internal/cli/group.go**: `group create/get/join/add/remove/leave/update/members/messages` 的 CLI 处理器；E2EE 群 remove/leave 会路由到 hidden `group.e2ee.remove/leave` 组合编排而不是公开 P4-only 方法。 +**internal/cli/group_e2ee.go**: P6 group E2EE 诊断/维护命令处理器；支持本地 exec provider/status/KeyPackage 发布路径、`publish-key-package --recovery --group` recovery KeyPackage 发布、hidden/test-only `group.e2ee.head` local/service epoch 对比、`group.e2ee.notice` pending/repair 拉取与 welcome/commit 重放、PR-B2 accepted pending commit 安全 finalize / unrecoverable gap fail-closed 诊断、PR-B3 `recover-member` owner/admin same-device crypto recovery 编排（prepare -> hidden `group.e2ee.recover_member` -> finalize/abort，禁止 P4 `group.add`），以及 PR-B1 `process-leave-request` owner/admin epoch-advancing remove 编排；`contract-test` 仅在显式 flag 下启用。 +**internal/identity/types.go**: identity store、legacy scan、command result 等核心类型。 +**internal/identity/layout.go**: identity 根目录、index.json、路径与安全写入辅助。 +**internal/identity/store.go**: 当前 v2 identity store 的读写、默认 identity 管理。 +**internal/identity/legacy.go**: v1 indexed/flat credential layout 扫描与导入。 +**internal/identity/did.go**: 本地 DID 文档与 proof 生成，当前默认生成 `e1` profile DID（`key-1` 为 Ed25519）。 +**internal/identity/key_compat.go**: legacy ANP 私有 PEM 标签 / SEC1 私钥到标准 PKCS#8 PEM 的兼容迁移，确保旧身份在 ANP Go SDK 0.8.6+ 下仍可完成 DID WBA 签名。 +**internal/identity/client.go**: user-service RPC/REST 客户端。 **internal/identity/service.go**: Phase 2/3 高层 identity + user 业务流，封装本地 store、handle lifecycle、`replace_did` DID 换绑能力，以及远端 API。 -**internal/identity/did_test.go**: DID 文档和 proof 生成测试。 -**internal/identity/store_test.go**: identity store 与 legacy import 测试。 -**internal/store/types.go**: SQLite store 的核心类型、记录结构与导入报告类型。 -**internal/store/open.go**: pure Go SQLite 打开、WAL / foreign_keys / busy_timeout 配置。 -**internal/store/helpers.go**: thread id、row map、schema version、表/视图存在性等辅助函数。 -**internal/store/schema.go**: v12 schema、indexes、views 与 `EnsureSchema()`；新增 `contact_handle_bindings` 历史映射表，用于 Handle↔DID 历史绑定。 -**internal/store/dao.go**: messages / contacts / contact_handle_bindings / groups / outbox / relationship / rebind / execute_sql 的 DAO。 -**internal/store/rebind.go**: 基于工作区 SQLite 打开器的 owner DID 重绑与旧 E2EE 状态清理编排。 -**internal/store/import.go**: legacy SQLite 扫描与从 v1 DB 导入 v2 DB。 -**internal/store/schema_test.go**: schema 初始化和 version 测试。 -**internal/store/dao_test.go**: DAO、thread view、owner rebinding、E2EE 清理测试。 -**internal/store/import_test.go**: legacy SQLite 导入测试。 -**internal/message/types.go**: direct/group message 与 group lifecycle 的命令输入/输出模型和 transport 错误定义。 -**internal/message/auth.go**: direct message 的 hop-level auth 与本地 key / did document 读取。 -**internal/message/proof.go**: 基于 ANP Go SDK 0.8.5 的 RFC 9421 origin proof 薄封装。 -**internal/message/attachment.go**: 附件文件读取、manifest 组装、控制面/数据面 HTTP 交互与下载解析辅助。 -**internal/message/attachment_wire.go**: 附件 control-plane、download ticket 与 direct/group attachment manifest 的 RPC 参数构造器。 -**internal/message/attachment_service.go**: direct/group attachment send 与 `msg attachment download` 的业务编排层。 -**internal/message/group_wire.go**: group 标准面和 local-only RPC 参数构造器。 -**internal/message/http_client.go**: direct/group message 与 group lifecycle 的 HTTP JSON-RPC adapter。 -**internal/message/ws_proxy_client.go**: websocket 模式下通过本地 bridge 调用 listener/daemon 的 direct/group adapter。 -**internal/message/service.go**: direct inbox/send/history/mark-read 的业务编排层，融合 transport、identity、store；支持收件后自动 DID→Handle 补全，以及按 handle 聚合历史 DID 消息。 -**internal/message/contact_sync.go**: direct inbox/history 的联系人补全与 Handle 历史 DID 聚合辅助。 -**internal/message/group_service.go**: group lifecycle、group message、本地群缓存同步与群 inbox 聚合逻辑。 -**internal/message/helpers.go**: message 域常用值转换和解码辅助。 -**internal/message/proof_test.go**: origin_proof round-trip 测试。 -**internal/message/group_wire_test.go**: group RPC 参数构造与签名测试。 -**internal/runtime/config.go**: runtime mode（默认 websocket）、listener 默认策略、host notify 默认开启（默认 sink 为 `log`）与本地 bridge 配置解析。 -**internal/runtime/listener/types.go**: listener 状态与 session 状态结构。 -**internal/runtime/listener/files.go**: listener 的 pid/status/log/socket 路径与状态文件读写。 -**internal/runtime/listener/wsclient.go**: 远端 message-service WebSocket client。 -**internal/runtime/listener/server.go**: 本地 daemon server、session supervisor、notification 消费与 SQLite 落库；首条陌生来信会按 DID 反查 Handle 并更新通讯录。 -**internal/runtime/listener/contact_sync.go**: websocket 收件路径的 DID→Handle 自动补全与联系人重绑定辅助。 -**internal/runtime/listener/host_notify.go**: websocket 下行通知到宿主事件的标准化、字段裁剪与 host notify sink 注册入口；direct/group 事件可带 `sender_handle` / `recipient_handle`。 -**internal/runtime/listener/openclaw_host_notify.go**: OpenClaw 适配器，负责从本地 route registry 读取已注册 routes，并通过 `/hooks/agent` 执行 webhook fan-out；事件文本仍保留 sender/recipient handle 等可读字段。 -**internal/runtime/listener/service.go**: listener 系统服务编排，基于 `kardianos/service` 提供 install/start/stop/uninstall 与 service-run；`start` 在服务缺失时会自动 install，并等待 bridge ready 后再返回。 -**internal/runtime/listener/manager.go**: listener 的 start/stop/restart/status/run 管理逻辑与系统服务状态聚合。 -**internal/runtime/bridge_unix.go / bridge_windows.go**: 本地 bridge 跨平台 IPC 实现；Unix 平台使用 Unix Domain Socket，Windows 使用 Named Pipe。 -**docs/architecture/awiki-v2-architecture.md**: awiki CLI V2 的整体架构设计文档。 -**docs/architecture/awiki-command-v2.md**: awiki CLI 命令模型与命令层设计文档。 +**internal/identity/did_test.go**: DID 文档和 proof 生成测试。 +**internal/identity/store_test.go**: identity store 与 legacy import 测试。 +**internal/store/types.go**: SQLite store 的核心类型、记录结构与导入报告类型。 +**internal/store/open.go**: pure Go SQLite 打开、WAL / foreign_keys / busy_timeout 配置。 +**internal/store/helpers.go**: thread id、row map、schema version、表/视图存在性等辅助函数。 +**internal/store/schema.go**: v12 schema、indexes、views 与 `EnsureSchema()`；新增 `contact_handle_bindings` 历史映射表，用于 Handle↔DID 历史绑定。 +**internal/store/dao.go**: messages / contacts / contact_handle_bindings / groups / outbox / relationship / rebind / execute_sql 的 DAO。 +**internal/store/rebind.go**: 基于工作区 SQLite 打开器的 owner DID 重绑与旧 E2EE 状态清理编排。 +**internal/store/import.go**: legacy SQLite 扫描与从 v1 DB 导入 v2 DB。 +**internal/store/schema_test.go**: schema 初始化和 version 测试。 +**internal/store/dao_test.go**: DAO、thread view、owner rebinding、E2EE 清理测试。 +**internal/store/import_test.go**: legacy SQLite 导入测试。 +**internal/message/types.go**: direct/group message 与 group lifecycle 的命令输入/输出模型和 transport 错误定义。 +**internal/message/auth.go**: direct message 的 hop-level auth 与本地 key / did document 读取。 +**internal/message/proof.go**: 基于 ANP Go SDK 0.8.7 的 RFC 9421 origin proof 薄封装。 +**internal/message/attachment.go**: 附件文件读取、manifest 组装、控制面/数据面 HTTP 交互与下载解析辅助。 +**internal/message/attachment_wire.go**: 附件 control-plane、download ticket 与 direct/group attachment manifest 的 RPC 参数构造器。 +**internal/message/attachment_service.go**: direct/group attachment send 与 `msg attachment download` 的业务编排层。 +**internal/message/secure.go**: P5 direct E2EE secure send 的首版编排层，使用 ANP Go SDK direct_e2ee、本地文件会话/预密钥存储和 HTTP JSON-RPC；key-service 请求绑定当前 DID 文档里 `ANPMessageService.serviceDid`，并在有可用 sidecar OPK 时优先用 OPK 建链（本地保存 `p5-one-time-prekeys/`）；HTTP inbox/history 现已接入入站密文解密与会话推进，并会顺带补发本地 prekey bundle；轮询路径解密 direct-init 成功后会自动发送 encrypted ACK 并尝试 flush 该 peer 的 `e2ee_outbox`；当 initiator 仍处于 `pending-confirmation` 时，新的 secure 发送会进入 `e2ee_outbox` 排队。 +**internal/message/secure_control.go**: secure 控制面与恢复辅助，负责 secure ack/init payload、pending 阶段的 `e2ee_outbox` 排队、secure outbox flush，以及 `msg secure status/init/repair/failed/retry/drop` 需要的本地会话/发件箱读取与重试逻辑。 +**internal/message/group_e2ee_provider.go**: `anp-mls` exec provider 抽象；按 `AWIKI_ANP_MLS_BINARY`、测试/运行时注入路径、`PATH` 顺序发现二进制；JSON request 走 stdin、response 走 stdout、日志/错误走 stderr，默认 MLS 根目录为 `/mls`，实际 OpenMLS 私有状态按 agent/device 分到子目录，并可扫描同一 agent 下的本地 device state 供收件解密恢复，保持 Go 主工程 pure Go / no CGO。 +**internal/message/group_e2ee_service.go**: group E2EE 业务编排层；负责 KeyPackage 发布前用当前 DID `key-1` 生成 strict Appendix-B `did_wba_binding.proof`、owner create/add/remove、PR-B1 leave_request 创建与 owner/admin process-leave-request epoch-advancing remove、PR-B3 recovery KeyPackage tagging 与 owner/admin `recover-member` same-device crypto recovery 编排（anp-mls `recover-member-prepare`、hidden `group.e2ee.recover_member`、service accept 后 finalize、deterministic rejection 后 abort，且不调用/伪装 P4 `group.add`）、stale send epoch mismatch 时从 anp-mls pending status finalize/repair 后重试、pending commit finalize/abort、send encrypt、messages decrypt、P6 notice pending/repair、ratchet tree welcome/commit notice process，以及 MLS AAD 元数据传入 `anp-mls`；PR-B2/PR-B3 status/repair 会读取 hidden `group.e2ee.head`，仅在 service head 已接受对应 target epoch 时 finalize 本地 pending commit，重复 commit notice 以 local epoch 判定幂等 delivered，缺失 notice gap 输出 `needs_snapshot_or_readd` recovery artifact 并 fail closed；在同一工作区存在目标成员身份时也会本地处理 add/recovery 返回的 welcome notice，使 one-shot `anp-mls` agent/device 状态可恢复。 +**internal/message/group_wire.go**: group 标准面和 local-only RPC 参数构造器；P6 publish/get/notice 使用 `transport-protected` service/agent target，create 使用 service target，add/remove/leave/send 使用 group target；group E2EE send/remove/leave 会在签名/发送前裁剪 provider-local MLS/private 字段，只把 P6 service 允许的 opaque cipher/commit 字段送到 message-service。 +**internal/message/http_client.go**: direct/group message、group lifecycle 与 hidden/test-only P6 notice pull/mark-delivered 的 HTTP JSON-RPC adapter。 +**internal/message/ws_proxy_client.go**: websocket 模式下通过本地 bridge 调用 listener/daemon 的 direct/group adapter。 +**internal/message/service.go**: direct inbox/send/history/mark-read 的业务编排层，融合 transport、identity、store；支持收件后自动 DID→Handle 补全，以及按 handle 聚合历史 DID 消息。 +**internal/message/contact_sync.go**: direct inbox/history 的联系人补全与 Handle 历史 DID 聚合辅助。 +**internal/message/group_service.go**: group lifecycle、group message、本地群缓存同步与群 inbox 聚合逻辑；E2EE 群消息在落本地 message view 前先尝试解密，避免返回缓存时覆盖已解密展示内容。 +**internal/message/helpers.go**: message 域常用值转换和解码辅助。 +**internal/message/proof_test.go**: origin_proof round-trip 测试。 +**internal/message/group_wire_test.go**: group RPC 参数构造与签名测试。 +**internal/runtime/config.go**: runtime mode（默认 websocket）、listener 默认策略、host notify 默认开启（默认 sink 为 `log`）与本地 bridge 配置解析。 +**internal/runtime/listener/types.go**: listener 状态与 session 状态结构。 +**internal/runtime/listener/files.go**: listener 的 pid/status/log/socket 路径与状态文件读写。 +**internal/runtime/listener/wsclient.go**: 远端 message-service WebSocket client。 +**internal/runtime/listener/server.go**: 本地 daemon server、session supervisor、notification 消费与 SQLite 落库；首条陌生来信会按 DID 反查 Handle 并更新通讯录。 +**internal/runtime/listener/contact_sync.go**: websocket 收件路径的 DID→Handle 自动补全与联系人重绑定辅助。 +**internal/runtime/listener/host_notify.go**: websocket 下行通知到宿主事件的标准化、字段裁剪与 host notify sink 注册入口；direct/group 事件可带 `sender_handle` / `recipient_handle`。 +**internal/runtime/listener/openclaw_host_notify.go**: OpenClaw 适配器，负责从本地 route registry 读取已注册 routes，并通过 `/hooks/agent` 执行 webhook fan-out；事件文本仍保留 sender/recipient handle 等可读字段。 +**internal/runtime/listener/service.go**: listener 系统服务编排，基于 `kardianos/service` 提供 install/start/stop/uninstall 与 service-run；`start` 在服务缺失时会自动 install，并等待 bridge ready 后再返回。 +**internal/runtime/listener/manager.go**: listener 的 start/stop/restart/status/run 管理逻辑与系统服务状态聚合。 +**internal/runtime/bridge_unix.go / bridge_windows.go**: 本地 bridge 跨平台 IPC 实现；Unix 平台使用 Unix Domain Socket，Windows 使用 Named Pipe。 +**docs/architecture/awiki-v2-architecture.md**: awiki CLI V2 的整体架构设计文档。 +**docs/architecture/awiki-command-v2.md**: awiki CLI 命令模型与命令层设计文档。 **docs/architecture/anp-service-discovery.md**: awiki-cli 生成 DID 文档时的 `ANPMessageService` 填写规则、配置约束与实施记录。 -**docs/architecture/websocket-host-notification-v1.md**: websocket listener 向宿主 Agent 暴露统一通知事件的 v1 设计文档。 -**docs/architecture/openclaw-host-adapter-v1.md**: websocket host notification 到 OpenClaw `/hooks/agent` 的 v1 适配设计文档。 -**docs/architecture/output-format.md**: CLI 输出格式约束与展示设计文档。 -**docs/plan/awiki-v2-implementation-plan.md**: v2 的总体落地实施规划。 -**docs/plan/phase-0/implementation-constraints.md**: Phase 0 冻结后的实现约束表。 -**docs/plan/phase-0/capability-mapping.md**: v2 命令、v1 脚本、服务 API 的能力映射。 -**docs/plan/phase-0/audit-findings.md**: Phase 0 审计冲突与裁决。 -**docs/plan/phase-0/adr-index.md**: Phase 0 ADR 索引。 +**docs/architecture/websocket-host-notification-v1.md**: websocket listener 向宿主 Agent 暴露统一通知事件的 v1 设计文档。 +**docs/architecture/openclaw-host-adapter-v1.md**: websocket host notification 到 OpenClaw `/hooks/agent` 的 v1 适配设计文档。 +**docs/architecture/output-format.md**: CLI 输出格式约束与展示设计文档。 +**docs/plan/awiki-v2-implementation-plan.md**: v2 的总体落地实施规划。 +**docs/plan/phase-0/implementation-constraints.md**: Phase 0 冻结后的实现约束表。 +**docs/plan/phase-0/capability-mapping.md**: v2 命令、v1 脚本、服务 API 的能力映射。 +**docs/plan/phase-0/audit-findings.md**: Phase 0 审计冲突与裁决。 +**docs/plan/phase-0/adr-index.md**: Phase 0 ADR 索引。 + +**scripts/release/build-anp-mls.sh**: 本地 release hardening 辅助脚本，从同级 `../anp/anp/rust` 构建 `anp-mls` 并 stage 到 `dist/anp-mls/-/`，用于 awiki-cli group E2EE helper 的打包或 `AWIKI_ANP_MLS_BINARY` 注入验证。 ## 当前实现边界 @@ -157,7 +164,7 @@ - `debug db query` - `debug db import-v1` - `doctor` / `config show` 的数据库诊断增强 -- Phase 5（当前首版已落地 direct plain）： +- Phase 5（当前首版已落地 direct plain，P5 secure direct send 首版接入）： - `msg send --to` - `msg send --group` - `msg send --file` @@ -170,6 +177,8 @@ - attachment control-plane / data-plane HTTP upload, ticket, download - websocket runtime bridge / local daemon direct+group client - direct/group origin_proof 生成与本地消息落库 + - `msg send --secure on` 首版通过 ANP Go SDK P5 direct_e2ee 生成 `direct_init` / `direct_cipher` 并走 HTTP JSON-RPC；prekey key-service 请求使用本地 DID 文档的 `ANPMessageService.serviceDid` 做 `meta.target` 绑定，并在远端返回 top-level `one_time_prekey` 时优先走 OPK 建链；HTTP inbox/history 已支持入站密文自动解密与会话推进，并在轮询 direct-init 时自动 encrypted ACK + flush `e2ee_outbox`；listener 现已支持 secure incoming 自动解密、自动 first-reply/ack、以及在收到 secure ack 后 flush 已排队的 `e2ee_outbox` + - `msg secure status` / `init` / `repair` / `failed` / `retry` / `drop` 已有首版命令面实现；其中 `init` 当前通过发送 product-local secure control init 预热会话，`repair` 会重置本地会话并重排该 peer 的 failed outbox 后重新 init - attachment control 不再发送独立业务 proof - Phase 8（当前首版已落地 content page）： - `page create/list/get/update/rename/delete` @@ -208,9 +217,9 @@ ### 尚未实现 -- `msg` 域中 direct plain 已实现，listener 服务端首版也已实现，但 websocket 远端真实联调与 secure E2EE 仍未完成 +- `msg` 域中 direct plain 已实现，P5 secure direct 出站首版已接入；HTTP inbox/history 的 P5 入站自动解密、轮询 direct-init 自动 ACK 与 outbox flush 已接入；websocket listener 已能解密 secure incoming、自动 first-reply/ack，并在 secure ack 后尝试 flush `e2ee_outbox`；`msg secure status/init/repair/failed/retry/drop` 有首版，但完整 runtime 联调与更强系统测试仍未完成 - `group` 域的 plain lifecycle / local view / group messaging 已接入，`people` 仍大多为 stub；`page` 已完成 content pages 首版 -- secure E2EE 业务流、group plain、发布链路属于后续阶段 +- secure E2EE 业务流目前只完成 P5 出站首版；group E2EE 已接入 CLI 侧 `anp-mls` exec 编排、KeyPackage 发布（含 PR-B3 recovery KeyPackage）、group-e2ee create/add/remove/send、安全 self-leave request、owner/admin process-leave-request、owner/admin recover-member same-DID/device crypto recovery、pending commit finalize/abort、commit-delivery repair 与轮询消息本地解密分支，并已有隐藏 focused target 覆盖真实 OpenMLS Alice/Bob 最小闭环与 PR-B3 recovery；对外 discovery 仍保持隐藏，External Commit、多设备、cloud snapshot 与完整 MLS 群管理能力仍未实现；安全 leave-request 已处于 hidden/test-only PR-B1 路径。 ## 开发与验证约定 diff --git a/docs/architecture/awiki-command-v2.md b/docs/architecture/awiki-command-v2.md index 5d1f92d..8f7ab5b 100644 --- a/docs/architecture/awiki-command-v2.md +++ b/docs/architecture/awiki-command-v2.md @@ -117,15 +117,16 @@ awiki-cli msg mark-read MSG_ID... `msg attachment download` 会按 `message_id` 分页扫描 direct history 或 group messages，直到命中目标附件消息，而不是只检查最新一页结果。 -awiki-cli group create --name "Agent War Room" [--description "..."] [--discoverability private|listed|public] [--admission-mode admin-add|open-join] [--slug agent-war-room] [--goal "..."] [--rules "..."] [--message-prompt "..."] [--doc-url "https://..."] [--attachments-allowed] [--max-members 500] [--member-max-messages 10] [--member-max-total-chars 2000] [--identity alice] +awiki-cli group create --name "Agent War Room" [--description "..."] [--discoverability private|listed|public] [--admission-mode admin-add|open-join] [--message-security-profile transport-protected|group-e2ee] [--e2ee] [--slug agent-war-room] [--goal "..."] [--rules "..."] [--message-prompt "..."] [--doc-url "https://..."] [--attachments-allowed] [--max-members 500] [--member-max-messages 10] [--member-max-total-chars 2000] [--identity alice] awiki-cli group get --group GROUP_DID [--identity alice] awiki-cli group join --group GROUP_DID [--reason "..."] [--identity alice] -awiki-cli group add --group GROUP_DID --member did:wba:... [--role member|admin] [--reason "..."] [--identity alice] -awiki-cli group remove --group GROUP_DID --member did:wba:... [--reason "..."] [--identity alice] +awiki-cli group add --group GROUP_DID --member did:wba:... [--role member|admin] [--reason "..."] [--e2ee] [--identity alice] +awiki-cli group remove --group GROUP_DID --member did:wba:... [--reason "..."] [--e2ee] [--identity alice] awiki-cli group members --group GROUP_DID [--limit 100] [--identity alice] awiki-cli group messages --group GROUP_DID [--limit 50] [--cursor CURSOR] [--identity alice] awiki-cli group update --group GROUP_DID [--name "..."] [--description "..."] [--discoverability private|listed|public] [--admission-mode admin-add|open-join] [--slug "..."] [--goal "..."] [--rules "..."] [--message-prompt "..."] [--doc-url "https://..."] [--attachments-allowed=true|false] [--max-members 500] [--member-max-messages 10] [--member-max-total-chars 2000] [--identity alice] -awiki-cli group leave --group GROUP_DID [--identity alice] +awiki-cli group leave --group GROUP_DID [--reason "..."] [--e2ee] [--identity alice] +awiki-cli group e2ee process-leave-request --group GROUP_DID --member did:wba:... [--leave-request-id LR_ID] [--reason "..."] [--identity alice] 测试与示例约定： diff --git a/docs/architecture/awiki-skill-architecture.md b/docs/architecture/awiki-skill-architecture.md index 312eb51..6c2e64f 100644 --- a/docs/architecture/awiki-skill-architecture.md +++ b/docs/architecture/awiki-skill-architecture.md @@ -93,7 +93,7 @@ | product surface | implemented | `status / docs / schema / doctor / config show / version / completion` | | id | implemented | 含 register / bind / recover / profile / import-v1 | | msg plain | implemented | direct/group plain send + inbox/history/mark-read | -| msg secure | planned | `--secure` contract 存在，但 secure 业务流尚未落地 | +| msg secure | partial | `--secure` direct flow、listener auto-ack/queued flush、以及 `msg secure status/init/repair/failed/retry/drop` 首版已落地；仍缺更强系统测试与部分恢复细节 | | group | implemented | create/get/join/add/remove/leave/update/members/messages | | runtime mode | implemented | `runtime status/setup/mode get/set` | | runtime listener | partial | status/install/start/stop/restart/uninstall 已可用；heartbeat 仍未落地 | diff --git a/docs/architecture/go-cli+rust-mls.md b/docs/architecture/go-cli+rust-mls.md new file mode 100644 index 0000000..f87d111 --- /dev/null +++ b/docs/architecture/go-cli+rust-mls.md @@ -0,0 +1,2195 @@ +> **阶段口径（2026-05 P6 contract-first slice）** +> +> 本文是早期 Rust/OpenMLS 工程方案草稿，不代表当前已落地能力。 +> 当前 CLI 实现已接入 `anp-mls` one-shot exec provider、P6 wire/storage/API、KeyPackage 发布、 +> `group create --message-security-profile group-e2ee` / `--e2ee`、group-e2ee add/send/decrypt 编排。 +> `contract-test` 非加密 artifact 只能通过显式 flag 启用；真实 OpenMLS 可用性仍以 `anp-mls` 后端和系统测试验收为准。 +> HTTP server / daemon、OpenMLS `StorageProvider`、真实 MLS group state 持久化、 +> snapshot、多设备同步等内容均是后续设计方向，不属于本阶段验收范围。 +> 对外 discovery 必须继续隐藏 `anp.group.e2ee.v1` / `group-e2ee`，并由 feature flag +> 显式开启测试面。 + +下面是按你新约束改过的完整方案：**API 同时提供 HTTP JSON API 和 CLI API，但系统不能依赖一个常驻后台进程。** + +核心设计思想是： + +> Rust OpenMLS 组件不是“必须一直运行的 daemon”，而是一个独立二进制 `awiki-mls`。 +> 它既可以用 `awiki-mls serve` 提供 HTTP JSON API，也可以被 Go CLI 每次按需调用，执行一次命令后退出。 +> 所有 MLS 状态必须落盘，不能依赖进程内存。 + +OpenMLS 本身是 Rust 实现的 MLS 协议库，目标是作为端到端加密应用的构建块；MLS 的标准规范是 IETF RFC 9420。([GitHub][1]) OpenMLS 的 `MlsGroup` 状态会持续写入 `StorageProvider`，后续可以通过 `GroupId` 从 provider 重新 load，这正好适合“不依赖常驻进程”的设计。([OpenMLS Book][2]) + +--- + +# 1. 总体结论 + +你的新方案应该从： + +```text +Go CLI + Rust OpenMLS sidecar +``` + +调整为： + +```text +Go CLI + Rust OpenMLS command/server binary +``` + +也就是： + +```text +awiki-cli / ANP Go SDK + | + | 方式 A：exec 调用 Rust CLI，一次一进程 + | 方式 B：HTTP JSON 调用 Rust server，可选 + v +awiki-mls + | + v +OpenMLS + 本地持久化 StorageProvider +``` + +其中默认推荐： + +```text +默认模式：CLI exec mode +可选模式：HTTP server mode +``` + +也就是说： + +* **不要求 `awiki-mls` 常驻运行** +* **Go CLI 默认每次调用 Rust CLI** +* **HTTP JSON API 只是加速、调试、服务化、长期运行场景用** +* **两种 API 调用同一套 Rust service core** +* **状态全部保存在本地数据库中** + +--- + +# 2. 最终架构 + +```text +┌──────────────────────────────────────────────────────┐ +│ awiki-cli / ANP Go SDK │ +│ │ +│ - ANP/P6 消息封装 │ +│ - DID / did:wba 身份 │ +│ - 群业务逻辑 │ +│ - 消息收发 / relay / 跨域路由 │ +│ - MLSProvider 抽象接口 │ +└───────────────────────┬──────────────────────────────┘ + │ + ┌─────────────┴─────────────┐ + │ │ + ▼ ▼ +┌──────────────────────┐ ┌──────────────────────────┐ +│ ExecProvider │ │ HTTPProvider │ +│ │ │ │ +│ os/exec 调用 │ │ 调用 127.0.0.1 / UDS │ +│ awiki-mls command │ │ awiki-mls serve │ +└──────────┬───────────┘ └───────────┬──────────────┘ + │ │ + └─────────────┬──────────────┘ + ▼ +┌──────────────────────────────────────────────────────┐ +│ awiki-mls │ +│ Rust binary │ +│ │ +│ CLI mode: │ +│ awiki-mls group create --json-in - │ +│ awiki-mls message encrypt --json-in - │ +│ │ +│ Server mode: │ +│ awiki-mls serve --listen 127.0.0.1:8742 │ +│ │ +│ Shared core: │ +│ OpenMlsEngine │ +│ StorageRepository │ +│ OperationLog │ +│ LockManager │ +└───────────────────────┬──────────────────────────────┘ + ▼ +┌──────────────────────────────────────────────────────┐ +│ Local MLS Storage │ +│ │ +│ ~/.awiki/mls/sidecar.db │ +│ ~/.awiki/mls/locks/ │ +│ ~/.awiki/mls/snapshots/ │ +└──────────────────────────────────────────────────────┘ +``` + +--- + +# 3. 两种 API 的定位 + +## 3.1 CLI API：默认路径 + +这是你当前 Go CLI 最适合用的方式。 + +Go 层通过 `os/exec` 调用： + +```bash +awiki-mls group create --json-in - +awiki-mls key-package generate --json-in - +awiki-mls group add-member --json-in - +awiki-mls welcome process --json-in - +awiki-mls message encrypt --json-in - +awiki-mls message decrypt --json-in - +awiki-mls group restore --json-in - +awiki-mls snapshot export --json-in - +awiki-mls snapshot import --json-in - +``` + +特点： + +```text +1. 不需要后台进程 +2. 每次命令启动一次 Rust 进程 +3. 命令执行完立即退出 +4. 状态从本地 DB load，执行后写回 DB +5. Go CLI 仍然是纯 Go 主工程 +``` + +这是第一阶段的主路径。 + +--- + +## 3.2 HTTP JSON API：可选路径 + +用于： + +```text +1. 本地开发调试 +2. 长时间运行的 agent runtime +3. awiki desktop / daemon 模式 +4. 性能测试 +5. 集成测试 +``` + +启动方式： + +```bash +awiki-mls serve \ + --listen 127.0.0.1:8742 \ + --data-dir ~/.awiki/mls +``` + +HTTP API 不能作为唯一依赖。Go CLI 不能假设它一定存在。 + +--- + +# 4. Go 侧 Provider 策略 + +Go 层定义统一接口： + +```go +type MLSProvider interface { + GenerateKeyPackage(ctx context.Context, req GenerateKeyPackageRequest) (*GenerateKeyPackageResponse, error) + CreateGroup(ctx context.Context, req CreateGroupRequest) (*CreateGroupResponse, error) + AddMember(ctx context.Context, req AddMemberRequest) (*AddMemberResponse, error) + ProcessWelcome(ctx context.Context, req ProcessWelcomeRequest) (*ProcessWelcomeResponse, error) + Encrypt(ctx context.Context, req EncryptRequest) (*EncryptResponse, error) + Decrypt(ctx context.Context, req DecryptRequest) (*DecryptResponse, error) + RestoreGroup(ctx context.Context, req RestoreGroupRequest) (*RestoreGroupResponse, error) + ExportSnapshot(ctx context.Context, req ExportSnapshotRequest) (*ExportSnapshotResponse, error) + ImportSnapshot(ctx context.Context, req ImportSnapshotRequest) (*ImportSnapshotResponse, error) +} +``` + +实现三个 provider： + +```go +type ExecProvider struct { + BinaryPath string + DataDir string + Timeout time.Duration +} + +type HTTPProvider struct { + Endpoint string + Token string + Client *http.Client +} + +type AutoProvider struct { + HTTP *HTTPProvider + Exec *ExecProvider +} +``` + +推荐默认策略： + +```text +AWIKI_MLS_MODE=auto +``` + +`auto` 模式逻辑： + +```text +1. 检查 AWIKI_MLS_ENDPOINT 是否存在 +2. 尝试 GET /healthz +3. 如果 HTTP server 可用，走 HTTPProvider +4. 如果 HTTP server 不可用，走 ExecProvider +5. 永远不要求用户先手动启动后台进程 +``` + +配置项： + +```bash +AWIKI_MLS_MODE=auto # auto | exec | http +AWIKI_MLS_BIN=awiki-mls +AWIKI_MLS_ENDPOINT=http://127.0.0.1:8742 +AWIKI_MLS_DATA_DIR=~/.awiki/mls +AWIKI_MLS_TIMEOUT=15s +``` + +--- + +# 5. Rust 侧二进制设计 + +Rust 项目叫： + +```text +awiki-mls +``` + +目录建议： + +```text +awiki-mls/ + Cargo.toml + src/ + main.rs + cli.rs + http.rs + api/ + mod.rs + types.rs + error.rs + engine/ + mod.rs + key_package.rs + group.rs + welcome.rs + message.rs + snapshot.rs + storage/ + mod.rs + sqlite.rs + lock.rs + operations.rs + security/ + aad.rs + canonical_json.rs + redaction.rs + tests/ + cli_flow_test.rs + http_flow_test.rs + parity_test.rs +``` + +关键原则： + +```text +CLI 和 HTTP 不各写一套逻辑。 +它们必须调用同一个 OpenMlsEngine。 +``` + +也就是： + +```rust +struct OpenMlsEngine { + storage: StorageRepository, + lock_manager: LockManager, + operation_log: OperationLog, +} +``` + +CLI 调用： + +```rust +engine.create_group(req) +``` + +HTTP 调用： + +```rust +engine.create_group(req) +``` + +这样可以保证： + +```text +HTTP JSON API 和 CLI API 行为完全一致。 +``` + +--- + +# 6. 统一 API Envelope + +为了让 HTTP 和 CLI 共享同一个协议，建议所有 API 都用统一 JSON envelope。 + +## 6.1 请求结构 + +```json +{ + "api_version": "awiki-mls/v1", + "request_id": "req_01HT...", + "agent_did": "did:wba:example.com:agent:alice", + "device_id": "default", + "params": {} +} +``` + +## 6.2 成功响应 + +```json +{ + "ok": true, + "api_version": "awiki-mls/v1", + "request_id": "req_01HT...", + "result": {} +} +``` + +## 6.3 失败响应 + +```json +{ + "ok": false, + "api_version": "awiki-mls/v1", + "request_id": "req_01HT...", + "error": { + "code": "EPOCH_MISMATCH", + "message": "Group epoch mismatch", + "details": { + "expected_epoch": 3, + "actual_epoch": 2 + } + } +} +``` + +CLI 也输出同样结构。 + +例如： + +```bash +awiki-mls group create --json-in create_group.json +``` + +stdout： + +```json +{ + "ok": true, + "api_version": "awiki-mls/v1", + "request_id": "req_01HT...", + "result": { + "anp_group_id": "group1", + "mls_group_id": "base64url...", + "epoch": 0 + } +} +``` + +stderr 只打印日志，stdout 只打印机器可读 JSON。 + +--- + +# 7. HTTP API 与 CLI API 一一映射 + +| 能力 | HTTP JSON API | CLI API | +| ----------------- | ------------------------------------------ | -------------------------------------------- | +| 健康检查 | `GET /healthz` | `awiki-mls health --json` | +| 版本检查 | `GET /v1/version` | `awiki-mls version --json` | +| 生成 KeyPackage | `POST /v1/key-packages/generate` | `awiki-mls key-package generate --json-in -` | +| 创建群 | `POST /v1/groups/create` | `awiki-mls group create --json-in -` | +| 添加成员 | `POST /v1/groups/{group_id}/members/add` | `awiki-mls group add-member --json-in -` | +| 合并 pending commit | `POST /v1/groups/{group_id}/commits/merge` | `awiki-mls group commit-merge --json-in -` | +| 处理 Welcome | `POST /v1/welcomes/process` | `awiki-mls welcome process --json-in -` | +| 加密消息 | `POST /v1/groups/{group_id}/encrypt` | `awiki-mls message encrypt --json-in -` | +| 解密消息 | `POST /v1/groups/{group_id}/decrypt` | `awiki-mls message decrypt --json-in -` | +| 恢复群状态 | `POST /v1/groups/{group_id}/restore` | `awiki-mls group restore --json-in -` | +| 导出 snapshot | 默认关闭或本地-only | `awiki-mls snapshot export --json-in -` | +| 导入 snapshot | 默认关闭或本地-only | `awiki-mls snapshot import --json-in -` | + +注意：**snapshot API 不建议默认暴露 HTTP**。它涉及敏感 key material，第一阶段建议只允许 CLI 调用，HTTP 版本必须通过 `--enable-snapshot-api` 显式开启。 + +--- + +# 8. 为什么不依赖常驻进程也能工作 + +每一次 CLI 调用都执行这个流程： + +```text +1. Go CLI 启动 awiki-mls 子进程 +2. awiki-mls 读取 JSON 请求 +3. 打开 ~/.awiki/mls/sidecar.db +4. 获取对应 agent/group 的文件锁 +5. 根据 anp_group_id 查 mls_group_id +6. 从 OpenMLS StorageProvider load MlsGroup +7. 执行 create/add/encrypt/decrypt 等操作 +8. OpenMLS 写回 StorageProvider +9. awiki-mls 写 metadata / operation log +10. 输出 JSON response +11. 进程退出 +``` + +这个模型的关键是： + +```text +进程不是状态载体。 +本地 storage 才是状态载体。 +``` + +OpenMLS 官方文档说明，`MlsGroup` 会持续写入配置的 `StorageProvider`，之后可以通过 `GroupId` 重新 load。([OpenMLS Book][2]) 所以 `awiki-mls` 完全可以按需启动、按需退出。 + +--- + +# 9. 本地存储设计 + +## 9.1 路径 + +默认路径： + +```text +~/.awiki/mls/ + sidecar.db + sidecar.lock + locks/ + snapshots/ + runtime/ +``` + +建议权限： + +```text +~/.awiki/mls/ 0700 +~/.awiki/mls/sidecar.db 0600 +~/.awiki/mls/snapshots/ 0700 +``` + +## 9.2 数据库分层 + +本地 DB 分两层： + +```text +1. OpenMLS provider storage +2. ANP/P6 metadata storage +``` + +OpenMLS provider storage 保存： + +```text +- group state +- key material +- signature keys +- key package private material +``` + +ANP/P6 metadata storage 保存： + +```text +- agent_did / device_id +- anp_group_id -> mls_group_id 映射 +- member_did -> leaf_index 映射 +- epoch 记录 +- request_id 幂等记录 +- pending commit 状态 +``` + +OpenMLS README 中也列出了 sqlite provider 相关 feature，因此第一阶段可以优先使用 SQLite 存储。([GitHub][1]) + +--- + +# 10. 数据表建议 + +## 10.1 group bindings + +```sql +CREATE TABLE group_bindings ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + agent_did TEXT NOT NULL, + device_id TEXT NOT NULL, + anp_group_id TEXT NOT NULL, + mls_group_id BLOB NOT NULL, + epoch INTEGER NOT NULL, + self_leaf_index INTEGER, + status TEXT NOT NULL, + created_at TEXT NOT NULL, + updated_at TEXT NOT NULL, + UNIQUE(agent_did, device_id, anp_group_id) +); +``` + +`status` 可选值： + +```text +active +pending_commit +stale +corrupted +deleted +``` + +## 10.2 members + +```sql +CREATE TABLE group_members ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + agent_did TEXT NOT NULL, + device_id TEXT NOT NULL, + anp_group_id TEXT NOT NULL, + member_did TEXT NOT NULL, + member_device_id TEXT, + leaf_index INTEGER, + credential_hash TEXT, + created_at TEXT NOT NULL, + updated_at TEXT NOT NULL, + UNIQUE(agent_did, device_id, anp_group_id, member_did, member_device_id) +); +``` + +## 10.3 key packages + +```sql +CREATE TABLE key_packages ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + agent_did TEXT NOT NULL, + device_id TEXT NOT NULL, + key_package_id TEXT NOT NULL, + key_package BLOB NOT NULL, + used INTEGER DEFAULT 0, + expires_at TEXT, + created_at TEXT NOT NULL, + UNIQUE(agent_did, device_id, key_package_id) +); +``` + +## 10.4 operation log + +```sql +CREATE TABLE operations ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + request_id TEXT NOT NULL, + operation_type TEXT NOT NULL, + input_hash TEXT NOT NULL, + status TEXT NOT NULL, + response_json TEXT, + error_json TEXT, + created_at TEXT NOT NULL, + updated_at TEXT NOT NULL, + UNIQUE(request_id) +); +``` + +作用： + +```text +1. 防止 CLI 重试导致重复执行 +2. 支持崩溃恢复 +3. 支持 pending commit 查询 +4. 支持 Go CLI 重新拿上次结果 +``` + +## 10.5 pending commits + +```sql +CREATE TABLE pending_commits ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + request_id TEXT NOT NULL, + agent_did TEXT NOT NULL, + device_id TEXT NOT NULL, + anp_group_id TEXT NOT NULL, + old_epoch INTEGER NOT NULL, + new_epoch INTEGER NOT NULL, + commit BLOB NOT NULL, + welcome BLOB, + status TEXT NOT NULL, + created_at TEXT NOT NULL, + updated_at TEXT NOT NULL, + UNIQUE(request_id) +); +``` + +`status`： + +```text +created +delivered +merged +aborted +expired +``` + +--- + +# 11. 文件锁设计 + +因为你不能保证只有一个 CLI 进程在跑，所以必须做锁。 + +否则可能出现： + +```text +两个 awiki-cli 同时 encrypt +两个 awiki-cli 同时 add_member +一个 HTTP server 和一个 CLI command 同时写同一个 group +``` + +建议锁粒度： + +```text +identity lock: + ~/.awiki/mls/locks/identity_.lock + +group lock: + ~/.awiki/mls/locks/group_.lock +``` + +规则： + +| 操作 | 锁 | +| -------------------- | -------------------------- | +| generate_key_package | identity lock | +| create_group | identity lock + group lock | +| add_member | group lock | +| commit_merge | group lock | +| process_welcome | identity lock + group lock | +| encrypt | group lock | +| decrypt | group lock | +| restore | group lock | +| snapshot export | identity lock 或全局 lock | +| snapshot import | 全局 lock | + +注意：`decrypt` 也建议加写锁。MLS 在处理消息时可能更新本地状态、删除旧密钥或推进 ratchet，所以不能把 decrypt 当成纯读操作。 + +OpenMLS 文档特别提醒，StorageProvider 中包含敏感 key material，并且 OpenMLS 会为了 forward secrecy 删除旧 key material；storage 实现必须确保删除不可恢复。([OpenMLS Book][2]) 因此锁和存储一致性非常重要。 + +--- + +# 12. 核心 API 详细设计 + +## 12.1 generate_key_package + +### HTTP + +```http +POST /v1/key-packages/generate +``` + +### CLI + +```bash +awiki-mls key-package generate --json-in - +``` + +### Request + +```json +{ + "api_version": "awiki-mls/v1", + "request_id": "req_kp_001", + "agent_did": "did:wba:example.com:agent:bob", + "device_id": "default", + "params": { + "credential_identity": "did:wba:example.com:agent:bob", + "ciphersuite": "MLS_128_DHKEMX25519_AES128GCM_SHA256_Ed25519", + "expires_at": "2026-05-01T00:00:00Z" + } +} +``` + +### Response + +```json +{ + "ok": true, + "api_version": "awiki-mls/v1", + "request_id": "req_kp_001", + "result": { + "key_package_id": "kp_01HT...", + "agent_did": "did:wba:example.com:agent:bob", + "device_id": "default", + "ciphersuite": "MLS_128_DHKEMX25519_AES128GCM_SHA256_Ed25519", + "key_package": "base64url...", + "created_at": "2026-04-26T12:00:00Z", + "expires_at": "2026-05-01T00:00:00Z" + } +} +``` + +### 行为 + +```text +1. 生成 Bob 的 MLS KeyPackage +2. 保存对应 private init key material +3. 返回 public KeyPackage +4. 标记 used=false +``` + +--- + +## 12.2 create_group + +### HTTP + +```http +POST /v1/groups/create +``` + +### CLI + +```bash +awiki-mls group create --json-in - +``` + +### Request + +```json +{ + "api_version": "awiki-mls/v1", + "request_id": "req_group_create_001", + "agent_did": "did:wba:example.com:agent:alice", + "device_id": "default", + "params": { + "anp_group_id": "anp-group-001", + "credential_identity": "did:wba:example.com:agent:alice", + "use_ratchet_tree_extension": true + } +} +``` + +### Response + +```json +{ + "ok": true, + "api_version": "awiki-mls/v1", + "request_id": "req_group_create_001", + "result": { + "anp_group_id": "anp-group-001", + "mls_group_id": "base64url...", + "epoch": 0, + "self_leaf_index": 0, + "members": [ + { + "agent_did": "did:wba:example.com:agent:alice", + "device_id": "default", + "leaf_index": 0 + } + ] + } +} +``` + +### 行为 + +```text +1. 创建 MLS group +2. 写入 OpenMLS storage +3. 写入 group_bindings +4. 初始 epoch = 0 +5. Alice 是第一个成员 +``` + +--- + +## 12.3 add_member + +这里要特别设计好，因为没有后台进程时，`add_member` 之后可能 Go CLI 还没来得及发送 commit/welcome 就崩溃。 + +所以推荐支持两种策略： + +```text +1. immediate_merge：PoC / 本地测试用 +2. staged：真实 ANP/P6 网络发送用 +``` + +第一阶段可以默认 `staged`。 + +### HTTP + +```http +POST /v1/groups/{anp_group_id}/members/add +``` + +### CLI + +```bash +awiki-mls group add-member --json-in - +``` + +### Request + +```json +{ + "api_version": "awiki-mls/v1", + "request_id": "req_add_bob_001", + "agent_did": "did:wba:example.com:agent:alice", + "device_id": "default", + "params": { + "anp_group_id": "anp-group-001", + "target_agent_did": "did:wba:example.com:agent:bob", + "target_device_id": "default", + "key_package": "base64url...", + "merge_strategy": "staged", + "aad": { + "anp_version": "1.0", + "profile": "group-e2ee", + "operation": "group.add", + "anp_group_id": "anp-group-001", + "sender_did": "did:wba:example.com:agent:alice", + "target_did": "did:wba:example.com:agent:bob", + "message_id": "msg_add_bob_001", + "created_at": "2026-04-26T12:00:00Z" + } + } +} +``` + +### Response + +```json +{ + "ok": true, + "api_version": "awiki-mls/v1", + "request_id": "req_add_bob_001", + "result": { + "anp_group_id": "anp-group-001", + "mls_group_id": "base64url...", + "old_epoch": 0, + "new_epoch": 1, + "merge_strategy": "staged", + "pending_commit_id": "pc_req_add_bob_001", + "commit": "base64url...", + "welcome": "base64url...", + "ratchet_tree": "base64url-optional", + "added_member": { + "agent_did": "did:wba:example.com:agent:bob", + "device_id": "default" + } + } +} +``` + +### staged 模式流程 + +```text +1. Alice 本地生成 Add Commit 和 Welcome +2. sidecar 保存 pending_commit +3. sidecar 暂不 merge 到新 epoch +4. Go CLI 把 commit/welcome 发送到 ANP 网络 +5. 发送成功后，Go CLI 调用 commit_merge +6. sidecar merge pending commit，Alice 进入 epoch 1 +``` + +### 为什么需要 staged + +否则会出现： + +```text +Alice 本地已经 epoch=1 +但 commit/welcome 没发出去 +Bob 还不知道自己被加入 +群状态不一致 +``` + +为了第一阶段快速 PoC，也可以允许： + +```json +"merge_strategy": "immediate" +``` + +但真实网络场景建议用： + +```json +"merge_strategy": "staged" +``` + +--- + +## 12.4 commit_merge + +虽然你最初列出的最小闭环没有单独写 `commit_merge`，但真实发送流程里最好加这个支持命令。 + +### HTTP + +```http +POST /v1/groups/{anp_group_id}/commits/merge +``` + +### CLI + +```bash +awiki-mls group commit-merge --json-in - +``` + +### Request + +```json +{ + "api_version": "awiki-mls/v1", + "request_id": "req_merge_001", + "agent_did": "did:wba:example.com:agent:alice", + "device_id": "default", + "params": { + "anp_group_id": "anp-group-001", + "pending_commit_id": "pc_req_add_bob_001", + "delivery_result": "accepted" + } +} +``` + +### Response + +```json +{ + "ok": true, + "api_version": "awiki-mls/v1", + "request_id": "req_merge_001", + "result": { + "anp_group_id": "anp-group-001", + "mls_group_id": "base64url...", + "old_epoch": 0, + "new_epoch": 1, + "status": "merged" + } +} +``` + +--- + +## 12.5 process_welcome + +### HTTP + +```http +POST /v1/welcomes/process +``` + +### CLI + +```bash +awiki-mls welcome process --json-in - +``` + +### Request + +```json +{ + "api_version": "awiki-mls/v1", + "request_id": "req_process_welcome_001", + "agent_did": "did:wba:example.com:agent:bob", + "device_id": "default", + "params": { + "anp_group_id": "anp-group-001", + "welcome": "base64url...", + "ratchet_tree": "base64url-optional" + } +} +``` + +### Response + +```json +{ + "ok": true, + "api_version": "awiki-mls/v1", + "request_id": "req_process_welcome_001", + "result": { + "anp_group_id": "anp-group-001", + "mls_group_id": "base64url...", + "epoch": 1, + "self_leaf_index": 1, + "members": [ + { + "agent_did": "did:wba:example.com:agent:alice", + "leaf_index": 0 + }, + { + "agent_did": "did:wba:example.com:agent:bob", + "leaf_index": 1 + } + ] + } +} +``` + +### 行为 + +```text +1. Bob 收到 Welcome +2. 调用 process_welcome +3. sidecar 创建 Bob 本地 group state +4. 写入 group_bindings +5. Bob 之后可以 decrypt/encrypt 群消息 +``` + +--- + +## 12.6 encrypt + +OpenMLS 的 AAD 是 authenticated 但不 encrypted 的数据，适合放 ANP 元数据，例如 group_id、sender_did、message_id、epoch 等。AAD 在传输中可以被查看，但不能被篡改。([OpenMLS Book][3]) + +### HTTP + +```http +POST /v1/groups/{anp_group_id}/encrypt +``` + +### CLI + +```bash +awiki-mls message encrypt --json-in - +``` + +### Request + +```json +{ + "api_version": "awiki-mls/v1", + "request_id": "req_encrypt_001", + "agent_did": "did:wba:example.com:agent:alice", + "device_id": "default", + "params": { + "anp_group_id": "anp-group-001", + "plaintext": "base64url...", + "aad": { + "anp_version": "1.0", + "profile": "group-e2ee", + "security_profile": "mls-rfc9420", + "operation": "group.send", + "anp_group_id": "anp-group-001", + "mls_group_id": "base64url...", + "epoch": 1, + "sender_did": "did:wba:example.com:agent:alice", + "message_id": "msg_001", + "content_type": "text/plain", + "created_at": "2026-04-26T12:00:00Z" + } + } +} +``` + +### Response + +```json +{ + "ok": true, + "api_version": "awiki-mls/v1", + "request_id": "req_encrypt_001", + "result": { + "anp_group_id": "anp-group-001", + "mls_group_id": "base64url...", + "epoch": 1, + "ciphertext": "base64url...", + "aad_hash": "base64url...", + "sender_leaf_index": 0 + } +} +``` + +### AAD 序列化 + +建议： + +```text +JSON Canonicalization Scheme / JCS +UTF-8 bytes +SHA-256 hash for aad_hash +``` + +ANP 外层 envelope 和 MLS AAD 必须绑定。 + +--- + +## 12.7 decrypt + +### HTTP + +```http +POST /v1/groups/{anp_group_id}/decrypt +``` + +### CLI + +```bash +awiki-mls message decrypt --json-in - +``` + +### Request + +```json +{ + "api_version": "awiki-mls/v1", + "request_id": "req_decrypt_001", + "agent_did": "did:wba:example.com:agent:bob", + "device_id": "default", + "params": { + "anp_group_id": "anp-group-001", + "ciphertext": "base64url...", + "expected_aad": { + "anp_version": "1.0", + "profile": "group-e2ee", + "security_profile": "mls-rfc9420", + "operation": "group.send", + "anp_group_id": "anp-group-001", + "mls_group_id": "base64url...", + "epoch": 1, + "sender_did": "did:wba:example.com:agent:alice", + "message_id": "msg_001", + "content_type": "text/plain", + "created_at": "2026-04-26T12:00:00Z" + } + } +} +``` + +### Response + +```json +{ + "ok": true, + "api_version": "awiki-mls/v1", + "request_id": "req_decrypt_001", + "result": { + "anp_group_id": "anp-group-001", + "mls_group_id": "base64url...", + "epoch": 1, + "sender": { + "leaf_index": 0, + "agent_did": "did:wba:example.com:agent:alice" + }, + "plaintext": "base64url...", + "aad": { + "anp_version": "1.0", + "profile": "group-e2ee", + "security_profile": "mls-rfc9420", + "operation": "group.send", + "anp_group_id": "anp-group-001", + "mls_group_id": "base64url...", + "epoch": 1, + "sender_did": "did:wba:example.com:agent:alice", + "message_id": "msg_001", + "content_type": "text/plain", + "created_at": "2026-04-26T12:00:00Z" + } + } +} +``` + +### Go 层必须校验 + +```text +1. 外层 group_id == AAD anp_group_id +2. 外层 sender_did == AAD sender_did +3. 外层 message_id == AAD message_id +4. 外层 created_at == AAD created_at +5. AAD epoch == MLS 解密返回 epoch +6. sender leaf_index 能映射到 sender_did +7. content_type 符合预期 +``` + +--- + +# 13. restore 与 snapshot 重新定义 + +因为不依赖后台进程，所以这里要区分两个概念： + +```text +1. group restore / state load +2. snapshot export/import +``` + +--- + +## 13.1 group restore：每次操作自动发生 + +在这个设计里，`restore` 不是“把状态恢复到后台进程内存”。 + +因为没有常驻进程。 + +真正的 restore 是： + +```text +从本地 StorageProvider load group state。 +``` + +所以每次命令都会自动做： + +```text +load group -> execute operation -> persist group +``` + +显式 restore API 主要用于： + +```text +1. 检查 group state 是否存在 +2. 检查当前 epoch +3. 检查成员映射 +4. 检查状态是否损坏 +5. 给 Go CLI 做诊断 +``` + +### HTTP + +```http +POST /v1/groups/{anp_group_id}/restore +``` + +### CLI + +```bash +awiki-mls group restore --json-in - +``` + +### Response + +```json +{ + "ok": true, + "api_version": "awiki-mls/v1", + "request_id": "req_restore_001", + "result": { + "anp_group_id": "anp-group-001", + "mls_group_id": "base64url...", + "epoch": 3, + "self_leaf_index": 0, + "status": "active", + "restored": true + } +} +``` + +--- + +## 13.2 snapshot export/import：备份与迁移 + +`snapshot` 不是常规消息流程的一部分。 + +它只用于： + +```text +1. 开发调试 +2. 设备迁移 +3. 本地灾备 +4. 测试复现 +``` + +不建议默认频繁导出 snapshot，因为 snapshot 可能保留旧 key material，影响 forward secrecy。OpenMLS 文档明确提到，为了 forward secrecy，旧 key material 会被删除，StorageProvider 实现必须确保删除不可恢复且没有副本。([OpenMLS Book][2]) + +### snapshot export CLI + +```bash +awiki-mls snapshot export --json-in - +``` + +Request： + +```json +{ + "api_version": "awiki-mls/v1", + "request_id": "req_snapshot_export_001", + "agent_did": "did:wba:example.com:agent:alice", + "device_id": "default", + "params": { + "anp_group_id": "anp-group-001", + "output_file": "~/.awiki/mls/snapshots/group1.snapshot.enc", + "encryption": { + "type": "passphrase", + "kdf": "argon2id" + } + } +} +``` + +Response： + +```json +{ + "ok": true, + "api_version": "awiki-mls/v1", + "request_id": "req_snapshot_export_001", + "result": { + "snapshot_file": "~/.awiki/mls/snapshots/group1.snapshot.enc", + "snapshot_version": "awiki-mls-snapshot/v1", + "anp_group_id": "anp-group-001", + "epoch": 3, + "created_at": "2026-04-26T12:00:00Z" + } +} +``` + +### snapshot import CLI + +```bash +awiki-mls snapshot import --json-in - +``` + +Request： + +```json +{ + "api_version": "awiki-mls/v1", + "request_id": "req_snapshot_import_001", + "agent_did": "did:wba:example.com:agent:alice", + "device_id": "default", + "params": { + "snapshot_file": "~/.awiki/mls/snapshots/group1.snapshot.enc", + "mode": "create_if_missing", + "encryption": { + "type": "passphrase" + } + } +} +``` + +安全策略： + +```text +1. snapshot 必须加密 +2. 默认不允许覆盖本地更新的 epoch +3. 默认不上传云端 +4. 默认不保留多版本历史 snapshot +5. HTTP snapshot API 默认关闭 +``` + +--- + +# 14. Go CLI 调用 Rust CLI 的方式 + +不要把敏感内容放在命令行参数里，因为命令行参数可能被系统进程列表看到。 + +错误示例： + +```bash +awiki-mls message encrypt --plaintext "secret text" +``` + +推荐方式： + +```bash +awiki-mls message encrypt --json-in - +``` + +Go 代码逻辑： + +```go +func (p *ExecProvider) call(ctx context.Context, command []string, req any, resp any) error { + body, err := json.Marshal(req) + if err != nil { + return err + } + + args := append(command, "--json-in", "-") + cmd := exec.CommandContext(ctx, p.BinaryPath, args...) + cmd.Stdin = bytes.NewReader(body) + + var stdout bytes.Buffer + var stderr bytes.Buffer + cmd.Stdout = &stdout + cmd.Stderr = &stderr + + if err := cmd.Run(); err != nil { + return parseExecError(err, stderr.String()) + } + + return json.Unmarshal(stdout.Bytes(), resp) +} +``` + +原则： + +```text +1. JSON request 走 stdin +2. JSON response 走 stdout +3. 日志走 stderr +4. Go 不解析 stderr 作为正常结果 +5. 所有二进制字段 base64url 编码 +6. plaintext 不进入 argv +``` + +--- + +# 15. Go CLI 对用户暴露的命令 + +用户不一定直接调用 `awiki-mls`。 + +你可以在 `awiki-cli` 里包一层： + +```bash +awiki-cli p6 key-package create +awiki-cli p6 group create +awiki-cli p6 group add-member +awiki-cli p6 welcome process +awiki-cli p6 message encrypt +awiki-cli p6 message decrypt +awiki-cli p6 group restore +awiki-cli p6 snapshot export +awiki-cli p6 snapshot import +``` + +Go CLI 内部调用： + +```text +awiki-cli p6 message encrypt + | + v +ExecProvider or HTTPProvider + | + v +awiki-mls message encrypt +``` + +--- + +# 16. 完整最小流程 + +## 16.1 Bob 生成 KeyPackage + +```bash +awiki-cli p6 key-package create \ + --agent-did did:wba:example.com:agent:bob \ + --out bob_key_package.json +``` + +内部调用： + +```bash +awiki-mls key-package generate --json-in - +``` + +输出 Bob 的 public KeyPackage。 + +--- + +## 16.2 Alice 创建群 + +```bash +awiki-cli p6 group create \ + --agent-did did:wba:example.com:agent:alice \ + --group-id anp-group-001 +``` + +内部调用： + +```bash +awiki-mls group create --json-in - +``` + +Alice 本地创建 group state，epoch = 0。 + +--- + +## 16.3 Alice 添加 Bob + +```bash +awiki-cli p6 group add-member \ + --agent-did did:wba:example.com:agent:alice \ + --group-id anp-group-001 \ + --key-package bob_key_package.json \ + --merge-strategy staged \ + --out add_bob_result.json +``` + +内部调用： + +```bash +awiki-mls group add-member --json-in - +``` + +返回： + +```text +commit +welcome +pending_commit_id +new_epoch +``` + +--- + +## 16.4 Go CLI 发送 ANP 消息 + +Alice 的 Go CLI 发送两条 ANP 消息： + +```text +1. anp-group-mls-commit+json +2. anp-group-mls-welcome+json +``` + +Bob 收到 welcome。 + +--- + +## 16.5 Alice merge pending commit + +发送成功后： + +```bash +awiki-cli p6 group commit-merge \ + --agent-did did:wba:example.com:agent:alice \ + --group-id anp-group-001 \ + --pending-commit-id pc_req_add_bob_001 +``` + +内部调用： + +```bash +awiki-mls group commit-merge --json-in - +``` + +Alice 本地进入 epoch = 1。 + +--- + +## 16.6 Bob process welcome + +```bash +awiki-cli p6 welcome process \ + --agent-did did:wba:example.com:agent:bob \ + --group-id anp-group-001 \ + --welcome welcome.json +``` + +内部调用： + +```bash +awiki-mls welcome process --json-in - +``` + +Bob 本地生成 group state，epoch = 1。 + +--- + +## 16.7 Alice 发送加密消息 + +```bash +awiki-cli p6 message encrypt \ + --agent-did did:wba:example.com:agent:alice \ + --group-id anp-group-001 \ + --text "hello bob" +``` + +内部调用： + +```bash +awiki-mls message encrypt --json-in - +``` + +Go CLI 把返回的 ciphertext 封装成： + +```json +{ + "type": "application/anp-group-mls-cipher+json", + "group_id": "anp-group-001", + "sender_did": "did:wba:example.com:agent:alice", + "message_id": "msg_001", + "created_at": "2026-04-26T12:00:00Z", + "mls": { + "mls_group_id": "base64url...", + "epoch": 1, + "aad": {}, + "ciphertext": "base64url..." + } +} +``` + +--- + +## 16.8 Bob 解密消息 + +```bash +awiki-cli p6 message decrypt \ + --agent-did did:wba:example.com:agent:bob \ + --group-id anp-group-001 \ + --message cipher_message.json +``` + +内部调用： + +```bash +awiki-mls message decrypt --json-in - +``` + +Bob 拿到 plaintext。 + +--- + +# 17. HTTP server 模式 + +HTTP server 模式只是一种加速路径。 + +启动： + +```bash +awiki-mls serve \ + --listen 127.0.0.1:8742 \ + --data-dir ~/.awiki/mls \ + --auth-token-file ~/.awiki/mls/runtime/token +``` + +Go CLI 使用： + +```bash +AWIKI_MLS_MODE=http awiki-cli p6 message encrypt ... +``` + +或者自动： + +```bash +AWIKI_MLS_MODE=auto awiki-cli p6 message encrypt ... +``` + +`auto` 会先尝试 HTTP： + +```http +GET /healthz +``` + +失败则 fallback 到 exec。 + +HTTP server 也不应该把 group state 只保存在内存里。第一阶段建议 HTTP 每次请求也走： + +```text +load group from storage +execute +persist +``` + +这样即使 HTTP server 被 kill，也不会丢状态。 + +--- + +# 18. HTTP 安全设计 + +HTTP server 只绑定本地： + +```text +127.0.0.1 +``` + +或者 Unix Domain Socket： + +```text +~/.awiki/mls/runtime/awiki-mls.sock +``` + +不允许默认监听： + +```text +0.0.0.0 +``` + +认证： + +```http +Authorization: Bearer +``` + +token 文件： + +```text +~/.awiki/mls/runtime/token +``` + +权限： + +```text +0600 +``` + +HTTP snapshot API 默认关闭： + +```bash +awiki-mls serve --enable-snapshot-api=false +``` + +--- + +# 19. 错误码设计 + +| 错误码 | 含义 | +| ------------------------------- | ------------------- | +| `BAD_REQUEST` | 请求 JSON 错误 | +| `UNSUPPORTED_API_VERSION` | API 版本不兼容 | +| `GROUP_NOT_FOUND` | 本地没有 group state | +| `KEY_PACKAGE_NOT_FOUND` | 找不到 KeyPackage 私有材料 | +| `KEY_PACKAGE_ALREADY_USED` | KeyPackage 已使用 | +| `EPOCH_MISMATCH` | epoch 不一致 | +| `AAD_MISMATCH` | AAD 与预期不一致 | +| `DECRYPT_FAILED` | 解密失败 | +| `STORAGE_LOCKED` | 本地锁被占用 | +| `PENDING_COMMIT_NOT_FOUND` | 找不到 pending commit | +| `PENDING_COMMIT_ALREADY_MERGED` | commit 已经 merge | +| `SNAPSHOT_REJECTED` | snapshot 导入被拒绝 | +| `INTERNAL_ERROR` | 内部错误 | + +CLI exit code 建议： + +| Exit Code | 含义 | +| --------: | --------------------- | +| 0 | 成功 | +| 10 | bad request | +| 11 | not found | +| 20 | state conflict | +| 21 | epoch mismatch | +| 22 | AAD mismatch | +| 30 | crypto/decrypt failed | +| 40 | storage locked | +| 50 | internal error | + +--- + +# 20. 幂等性与崩溃恢复 + +所有写操作都必须带 `request_id`： + +```text +generate_key_package +create_group +add_member +commit_merge +process_welcome +encrypt +decrypt +snapshot import +``` + +处理逻辑： + +```text +1. 收到 request_id +2. 查询 operations 表 +3. 如果已成功且 input_hash 相同，直接返回上次 response +4. 如果已成功但 input_hash 不同，返回 REQUEST_ID_CONFLICT +5. 如果之前执行中崩溃，进入 recovery 逻辑 +``` + +典型恢复场景： + +## 场景一：add_member 成功，但 Go CLI 崩溃 + +```text +1. pending_commit 已写入 DB +2. commit/welcome response 可能没被 Go 拿到 +3. Go CLI 使用同一个 request_id 重试 +4. awiki-mls 返回同一个 commit/welcome +``` + +## 场景二：commit/welcome 已发送，但 commit_merge 前崩溃 + +```text +1. operations 表里 pending_commit 状态是 created +2. Go CLI 下次启动可以 list pending commits +3. 用户或自动流程继续 commit_merge +``` + +增加命令： + +```bash +awiki-mls group pending-commits --json-in - +``` + +Go 包一层： + +```bash +awiki-cli p6 group pending-commits --group-id anp-group-001 +``` + +--- + +# 21. ANP/P6 消息类型 + +第一阶段建议定义 3 个 wire message： + +```text +application/anp-group-mls-commit+json +application/anp-group-mls-welcome+json +application/anp-group-mls-cipher+json +``` + +## 21.1 Commit + +```json +{ + "type": "application/anp-group-mls-commit+json", + "anp_version": "1.0", + "group_id": "anp-group-001", + "sender_did": "did:wba:example.com:agent:alice", + "message_id": "msg_commit_001", + "created_at": "2026-04-26T12:00:00Z", + "mls": { + "mls_group_id": "base64url...", + "old_epoch": 0, + "new_epoch": 1, + "commit": "base64url..." + } +} +``` + +## 21.2 Welcome + +```json +{ + "type": "application/anp-group-mls-welcome+json", + "anp_version": "1.0", + "group_id": "anp-group-001", + "sender_did": "did:wba:example.com:agent:alice", + "target_did": "did:wba:example.com:agent:bob", + "message_id": "msg_welcome_001", + "created_at": "2026-04-26T12:00:00Z", + "mls": { + "mls_group_id": "base64url...", + "epoch": 1, + "welcome": "base64url...", + "ratchet_tree": "base64url-optional" + } +} +``` + +## 21.3 Cipher + +```json +{ + "type": "application/anp-group-mls-cipher+json", + "anp_version": "1.0", + "group_id": "anp-group-001", + "sender_did": "did:wba:example.com:agent:alice", + "message_id": "msg_001", + "created_at": "2026-04-26T12:00:00Z", + "content_type": "text/plain", + "mls": { + "mls_group_id": "base64url...", + "epoch": 1, + "aad": { + "anp_version": "1.0", + "profile": "group-e2ee", + "security_profile": "mls-rfc9420", + "operation": "group.send", + "anp_group_id": "anp-group-001", + "mls_group_id": "base64url...", + "epoch": 1, + "sender_did": "did:wba:example.com:agent:alice", + "message_id": "msg_001", + "content_type": "text/plain", + "created_at": "2026-04-26T12:00:00Z" + }, + "ciphertext": "base64url..." + } +} +``` + +--- + +# 22. CLI 与 HTTP 一致性测试 + +必须加一个 parity test： + +```text +同一个输入： + CLI 输出 + HTTP 输出 + +除了时间戳、随机值、request_id 外，语义必须一致。 +``` + +测试矩阵： + +```text +1. generate_key_package CLI vs HTTP +2. create_group CLI vs HTTP +3. add_member CLI vs HTTP +4. process_welcome CLI vs HTTP +5. encrypt CLI vs HTTP +6. decrypt CLI vs HTTP +7. restore CLI vs HTTP +``` + +--- + +# 23. 第一阶段验收标准 + +## 23.1 无后台进程测试 + +这个是你新约束下最重要的验收。 + +```bash +awiki-cli p6 group create ... +# awiki-mls 进程退出 + +awiki-cli p6 key-package create ... +# awiki-mls 进程退出 + +awiki-cli p6 group add-member ... +# awiki-mls 进程退出 + +awiki-cli p6 welcome process ... +# awiki-mls 进程退出 + +awiki-cli p6 message encrypt ... +# awiki-mls 进程退出 + +awiki-cli p6 message decrypt ... +# awiki-mls 进程退出 +``` + +通过标准： + +```text +1. 每一步都不依赖常驻进程 +2. 每一步都能从 storage restore 状态 +3. Alice/Bob 能完成加密解密 +4. kill 所有 awiki-mls 进程后，下一次命令仍能继续 +``` + +--- + +## 23.2 HTTP server 可选测试 + +```bash +awiki-mls serve --listen 127.0.0.1:8742 +AWIKI_MLS_MODE=http awiki-cli p6 message encrypt ... +``` + +通过标准： + +```text +1. HTTP 模式结果与 CLI 模式一致 +2. kill HTTP server 后，AWIKI_MLS_MODE=auto 能 fallback 到 exec +3. HTTP server 重启后能恢复 group state +``` + +--- + +## 23.3 snapshot 测试 + +```bash +awiki-cli p6 snapshot export --group-id anp-group-001 +rm -rf ~/.awiki/mls/test-restore +awiki-cli p6 snapshot import --snapshot group1.snapshot.enc +awiki-cli p6 group restore --group-id anp-group-001 +``` + +通过标准： + +```text +1. snapshot 是加密文件 +2. import 后能 restore group +3. epoch 不倒退 +4. 不允许默认覆盖更新状态 +``` + +--- + +# 24. 里程碑拆分 + +## M0：Rust 单体 CLI PoC + +目标： + +```text +先不做 HTTP。 +只做 awiki-mls CLI。 +``` + +完成： + +```text +awiki-mls key-package generate +awiki-mls group create +awiki-mls group add-member +awiki-mls welcome process +awiki-mls message encrypt +awiki-mls message decrypt +``` + +验收： + +```text +Alice/Bob 在没有后台进程的情况下完成加密解密。 +``` + +--- + +## M1：持久化与 restore + +完成： + +```text +1. SQLite storage +2. group_bindings +3. operation_log +4. file lock +5. group restore +``` + +验收： + +```text +每个命令都是独立进程。 +重启后仍可继续 encrypt/decrypt。 +``` + +--- + +## M2：Go ExecProvider 接入 + +完成： + +```text +1. Go MLSProvider interface +2. ExecProvider +3. awiki-cli p6 子命令 +4. JSON stdin/stdout 调用 +``` + +验收： + +```text +awiki-cli 不需要常驻 sidecar。 +Go CLI 可以完整跑通 P6 最小闭环。 +``` + +--- + +## M3：HTTP JSON API + +完成： + +```text +1. awiki-mls serve +2. /healthz +3. /v1/version +4. 所有核心 HTTP endpoints +5. token auth +``` + +验收： + +```text +AWIKI_MLS_MODE=http 可用。 +AWIKI_MLS_MODE=auto 可 fallback。 +``` + +--- + +## M4：staged add_member 与崩溃恢复 + +完成： + +```text +1. pending_commits +2. commit_merge +3. request_id 幂等 +4. pending commit list +``` + +验收： + +```text +add_member 之后 Go CLI 崩溃，重试可恢复。 +``` + +--- + +## M5：snapshot export/import + +完成： + +```text +1. 加密 snapshot +2. import 冲突检查 +3. epoch 防倒退 +4. 默认不开放 HTTP snapshot API +``` + +--- + +# 25. 第一阶段不做的事情 + +为了避免范围失控，第一阶段不做： + +```text +1. 不做 OpenMLS Go 重写 +2. 不做 Rust FFI +3. 不做常驻 daemon 依赖 +4. 不做多设备同步 +5. 不做完整 remove member +6. 不做 external commit +7. 不做云端 snapshot 自动备份 +8. 不做大型附件直接 MLS 加密 +9. 不做跨实现互操作承诺 +``` + +大型附件后续应该采用： + +```text +MLS 加密内容密钥 +内容本身用 AEAD 流式加密 +``` + +不要直接把大文件塞进 MLS application message。 + +--- + +# 26. 推荐默认策略 + +最终建议： + +```text +默认： + AWIKI_MLS_MODE=exec + +开发： + AWIKI_MLS_MODE=auto + +服务化： + AWIKI_MLS_MODE=http +``` + +也就是说： + +```text +awiki-cli 默认不依赖任何后台进程。 +``` + +HTTP server 只是优化： + +```text +1. 减少进程启动开销 +2. 方便本地 agent runtime 长期运行 +3. 方便调试 +4. 方便后续桌面端 / daemon 集成 +``` + +--- + +# 27. 最终一句话方案 + +你应该把第一阶段设计成： + +> **`awiki-mls` 是一个 Rust OpenMLS 能力二进制，支持 HTTP server mode 和 one-shot CLI mode；`awiki-cli` 默认通过 exec 调用 CLI mode，不依赖常驻后台进程；所有 MLS group state、key package、pending commit、operation log 都持久化到本地 storage，HTTP 和 CLI 共用同一个 Rust engine。** + +这样既满足： + +```text +Go CLI 当前架构 +``` + +也满足： + +```text +不能保证后台进程一直运行 +``` + +还保留了未来升级路径： + +```text +exec mode -> HTTP mode -> daemon mode -> Go 原生 MLS provider +``` + +[1]: https://github.com/openmls/openmls "GitHub - openmls/openmls: Rust implementation of the Messaging Layer Security (MLS) protocol · GitHub" +[2]: https://book.openmls.tech/user_manual/persistence.html "Persistence of group state - OpenMLS Book" +[3]: https://book.openmls.tech/user_manual/aad.html "Using Additional Authenticated Data (AAD) - OpenMLS Book" diff --git a/docs/installation.md b/docs/installation.md index 94cbc6f..fbe29b1 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -26,15 +26,15 @@ go version # go version go1.22.x darwin/arm64 ``` -### 1.2 ANP Go SDK（远端模块依赖） +### 1.2 ANP Go SDK（P5 secure direct） -awiki-cli 直接使用远端 ANP Go SDK 模块，版本固定为 `v0.8.5`： +awiki-cli 的依赖基线是 ANP Go SDK `v0.8.7`： ```bash -go get github.com/agent-network-protocol/anp/golang@v0.8.5 +go get github.com/agent-network-protocol/anp/golang@v0.8.7 ``` -首次拉取依赖时请确保本机可以访问公开 Go module proxy 或对应源码仓库。 +P5 secure direct / OPK 客户端能力已随 `v0.8.7` 发布；主线 `go.mod` 应直接依赖远端模块，不再提交同级工作区 `replace`。首次拉取依赖时请确保本机可以访问公开 Go module proxy 或对应源码仓库。 ### 1.3 Docker 备选（无本地 Go 时） @@ -116,6 +116,7 @@ awiki-cli 默认采用单根目录工作区模型，默认路径如下： | runtime 目录 | `~/.awiki-cli/runtime/` | 无 | | 缓存目录 | `~/.awiki-cli/cache/` | 无 | | 日志目录 | `~/.awiki-cli/logs/` | 无 | +| MLS 状态目录（P6 group E2EE / `anp-mls`） | `~/.awiki-cli/mls/` | 无 | > 说明：`~/.awiki-cli/` 是跨平台固定的工作区目录（Windows 对应 `%USERPROFILE%\.awiki-cli\`），也是默认唯一入口。 > `AWIKI_CLI_WORKSPACE_HOME_DIR` 只负责切换整个工作区根目录；`config / data / runtime / cache` 不再允许分别配置。 @@ -128,11 +129,46 @@ awiki-cli 默认采用单根目录工作区模型，默认路径如下： > - `data/awiki-cli.db` > - `cache/` > - `runtime/` +> - `mls/`（由 `anp-mls` exec provider 使用；MLS 私有状态不写入主业务 SQLite） > - `logs/` > - workspace upgrade 元数据 > - upgrade lock / journal > - 备份快照 + +### 3.3 `anp-mls` binary discovery and release staging + +Group E2EE commands keep the Go CLI pure-Go/no-CGO by invoking the Rust `anp-mls` binary as a one-shot process. Discovery order is: + +1. `AWIKI_ANP_MLS_BINARY` absolute path override. +2. Runtime/test injected provider path. +3. `PATH` lookup for `anp-mls`. + +Plain direct/group messaging does not require this binary. `awiki-cli doctor` reports an informational `anp_mls` check when the binary is missing; group E2EE commands return an actionable remediation error. + +`awiki-cli doctor` also probes compatibility with the stable machine-readable contract: + +```bash +anp-mls system version --json-in - +``` + +The response must include `api_version`, `binary_name`, `binary_version` or `build_version`, and `supported_commands`. The supported API for this CLI build is `anp-mls/v1`; a mismatch is reported as a warning with remediation instead of breaking plain messaging. + +For local release preparation, stage the Rust helper from the sibling ANP repository: + +```bash +scripts/release/build-anp-mls.sh --dry-run +scripts/release/build-anp-mls.sh +``` + +By default the script builds `../anp/anp/rust` with Cargo and copies `anp-mls` to `dist/anp-mls/-/anp-mls`. Release jobs can bundle that file beside the `awiki-cli` archive, or users can place it on `PATH`. If a deployment keeps the helper outside the archive, set: + +```bash +export AWIKI_ANP_MLS_BINARY=/absolute/path/to/anp-mls +``` + +The MLS private state root remains `~/.awiki-cli/mls/` (or the current `AWIKI_CLI_WORKSPACE_HOME_DIR` equivalent). Runtime OpenMLS state is agent/device-scoped under that root (`mls/agents///state.db`) so two local identities do not share private KeyPackage storage. Incoming decrypt first tries the default device and then scans the local agent-scoped device directories, allowing one-shot CLI commands to restore state for KeyPackages published with a named device. `group e2ee pending` / `repair` use the hidden/test-only P6 `group.e2ee.notice` pull path to list and replay durable welcome and commit-delivery notices; repair passes `welcome_b64u + ratchet_tree_b64u` or opaque `commit_b64u` public commit artifacts back to `anp-mls` and marks only successfully processed notices delivered. E2EE group removal uses local pending commit prepare plus hidden `group.e2ee.remove`, finalizing local MLS state only after service acceptance and aborting pending commits on deterministic service rejection. E2EE `group leave` now creates a hidden/test-only `group.e2ee.leave_request` control-plane record instead of preparing a local-terminal self-leave; an owner/admin processes it with `group e2ee process-leave-request --group --member [--leave-request-id ]`, which reuses the epoch-advancing remove-member orchestration. `doctor` reports the root directory permissions plus state file status, including warnings when cached group-E2EE groups exist but MLS state is missing. + ### 3.2 config.yaml 配置文件位于 `~/.awiki-cli/config.yaml`。推荐先执行 `awiki-cli init` 自动创建最小配置；如需手动创建，可参考仓库根目录的 `config.template.yaml`，或直接使用下面的模板： @@ -505,10 +541,11 @@ CGO_ENABLED=0 go build ./cmd/awiki-cli/ ### Q: 编译报错找不到 ANP SDK -确认当前模块依赖已成功下载，并且 `go.mod` 中使用的是远端版本 `github.com/agent-network-protocol/anp/golang v0.8.5`： +确认当前模块依赖已成功下载，并且 `go.mod` 中保留了 `github.com/agent-network-protocol/anp/golang v0.8.7` 远端依赖： ```bash -go get github.com/agent-network-protocol/anp/golang@v0.8.5 +go get github.com/agent-network-protocol/anp/golang@v0.8.7 +grep 'github.com/agent-network-protocol/anp/golang v0.8.7' go.mod ``` ### Q: `go mod tidy` 报错 diff --git a/docs/pr-notes/group-e2ee-p6-conformance-before-discovery.md b/docs/pr-notes/group-e2ee-p6-conformance-before-discovery.md new file mode 100644 index 0000000..84f42f5 --- /dev/null +++ b/docs/pr-notes/group-e2ee-p6-conformance-before-discovery.md @@ -0,0 +1,39 @@ +# Group E2EE Step B P6 Conformance Notes — awiki-cli + +## Scope + +- Update CLI P6 wire builders to match the current method matrix: + - publish/get/notice: `transport-protected` and service/agent target. + - create: `group-e2ee` with service target plus `body.group_did`. + - add/send: `group-e2ee` with group target. +- Pass `ratchet_tree_b64u` through add/welcome processing. +- Pass stable message/operation IDs and P6 AAD metadata into `anp-mls` encrypt/decrypt. +- Refresh the service group state before encrypted send and avoid treating MLS epoch as P4 `group_state_version`. +- Replace pending/repair skeletons with hidden/test-only `group.e2ee.notice` pull, welcome replay, and mark-delivered. + +## Public discovery stance + +- CLI commands remain diagnostics/maintenance for hidden Group E2EE v1 focused validation. +- Do not update public help/docs to imply broad Group E2EE support; discovery remains controlled by message-service and must stay hidden by default. + +## Config / packaging impact + +- No new config key. +- `AWIKI_ANP_MLS_BINARY` and release-staged `anp-mls` remain the install path. +- Go CLI remains pure Go / no CGO; plaintext and MLS private state stay out of argv and service storage. + +## Fresh validation evidence + +- `go test ./internal/message ./internal/cli ./internal/cmdmeta ./internal/doctor -count=1` → passed (`internal/message` 3.329s, `internal/cli` 55.483s, `internal/cmdmeta` 3.412s, `internal/doctor` 6.761s). +- `go vet ./internal/message ./internal/doctor` → passed. +- Focused CLI system loop via `awiki-system-test` with `--with-message-v2 --use-local-anp` → passed: 2 passed in 18.65s. +- After focused system tests, the local environment was stopped and published ANP Python/Rust dependencies were restored. + +## Rollback + +- Revert the Step B wire/service additions. Pending/repair would return to diagnostic-only behavior and public discovery must remain disabled. + +## Caveats + +- Owner/admin remove is routed through hidden PR-A `group.e2ee.remove` orchestration with local pending commit finalize/abort semantics. E2EE `group leave` is now routed to hidden/test-only `group.e2ee.leave_request`; owner/admin processing uses `group e2ee process-leave-request` and the existing epoch-advancing `group.e2ee.remove` orchestration instead of submitting a same-epoch local-terminal leave artifact. Still no External Commit, attachment group E2EE, cloud snapshot, or product-wide public beta claim. +- No k1 DID compatibility is included. diff --git a/docs/pr-notes/group-e2ee-v1-pr-closeout.md b/docs/pr-notes/group-e2ee-v1-pr-closeout.md new file mode 100644 index 0000000..267c804 --- /dev/null +++ b/docs/pr-notes/group-e2ee-v1-pr-closeout.md @@ -0,0 +1,35 @@ +# Group E2EE v1 PR Closeout Notes — awiki-cli + +## Scope + +- Extend `awiki-cli doctor` to inspect both legacy/root MLS state and real agent/device-scoped `anp-mls` state directories. +- Clean user-facing E2EE diagnostic wording so status/pending/repair no longer describe the real MLS path as a contract-test scaffold. +- Keep the Go CLI pure Go / no CGO and keep `anp-mls` as an exec provider using stdin/stdout. + +## Commits / branch context + +- Current branch is ahead of origin with recent Group E2EE work through `b0856f4 Make group E2EE CLI restore named-device MLS state`. +- This closeout is Step A only; it does not implement P6 conformance Step B. + +## Config / migration impact + +- No CLI business SQLite schema migration. +- No new config key. +- `AWIKI_ANP_MLS_BINARY` remains the explicit binary override. +- Doctor now reports scoped MLS state under `/mls/agents///state.db` and `state.lock` when present. + +## Validation + +- Fresh evidence collected in this Ralph pass: + - `go test -count=1 ./internal/doctor ./internal/message ./internal/cli` → passed after the deslop pass (`internal/doctor` 3.643s, `internal/message` 1.256s, `internal/cli` 54.811s). + - `go vet ./internal/doctor ./internal/message` → passed. + +## Rollback + +- Revert doctor scoped-state scan and CLI wording changes. This does not affect plain messaging. + +## Caveats + +- `group e2ee pending` / `repair` now use the hidden/test-only P6 notice pull/replay path; public discovery still requires the separate gate in Step B notes. +- Group E2EE remains hidden/test-only; no public discovery. +- No k1 DID compatibility work is included. diff --git a/go.mod b/go.mod index d766b89..8eb39e7 100644 --- a/go.mod +++ b/go.mod @@ -4,7 +4,7 @@ go 1.22 require ( github.com/Microsoft/go-winio v0.6.2 - github.com/agent-network-protocol/anp/golang v0.8.5 + github.com/agent-network-protocol/anp/golang v0.8.7 github.com/coder/websocket v1.8.12 github.com/itchyny/gojq v0.12.17 github.com/kardianos/service v1.2.3 @@ -19,6 +19,7 @@ require ( github.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.1 // indirect github.com/dustin/go-humanize v1.0.1 // indirect github.com/google/uuid v1.6.0 // indirect + github.com/gowebpki/jcs v1.0.1 // indirect github.com/inconshreveable/mousetrap v1.1.0 // indirect github.com/itchyny/timefmt-go v0.1.6 // indirect github.com/mattn/go-isatty v0.0.20 // indirect diff --git a/go.sum b/go.sum index bfe7df5..426b8c1 100644 --- a/go.sum +++ b/go.sum @@ -1,7 +1,7 @@ github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERoyfY= github.com/Microsoft/go-winio v0.6.2/go.mod h1:yd8OoFMLzJbo9gZq8j5qaps8bJ9aShtEA8Ipt1oGCvU= -github.com/agent-network-protocol/anp/golang v0.8.5 h1:umG/WfOs/0mGVE3JKWROUxbeFpBOwFdBJ0JWAE8zKu0= -github.com/agent-network-protocol/anp/golang v0.8.5/go.mod h1:IarvwCndJNX0fZUuT8dWnkG67Ah2zMXuunzQr6lbE30= +github.com/agent-network-protocol/anp/golang v0.8.7 h1:QG3IysTnvuZxUXLnafZ14jYFZFLqBSxpdPiQNHdBy4Q= +github.com/agent-network-protocol/anp/golang v0.8.7/go.mod h1:r3tSNMTTU4WW1fW68C1NJyn2Q0w59ajOQ4OERrFGTEU= github.com/btcsuite/btcd/btcec/v2 v2.3.6 h1:IzlsEr9olcSRKB/n7c4351F3xHKxS2lma+1UFGCYd4E= github.com/btcsuite/btcd/btcec/v2 v2.3.6/go.mod h1:m22FrOAiuxl/tht9wIqAoGHcbnCCaPWyauO8y2LGGtQ= github.com/btcsuite/btcd/chaincfg/chainhash v1.0.1 h1:q0rUy8C/TYNBQS1+CGKw68tLOFYSNEs0TFnxxnS9+4U= @@ -9,6 +9,7 @@ github.com/btcsuite/btcd/chaincfg/chainhash v1.0.1/go.mod h1:7SFka0XMvUgj3hfZtyd github.com/coder/websocket v1.8.12 h1:5bUXkEPPIbewrnkU8LTCLVaxi4N4J8ahufH2vlo4NAo= github.com/coder/websocket v1.8.12/go.mod h1:LNVeNrXQZfe5qhS9ALED3uA+l5pPqvwXg3CKoDBB2gs= github.com/cpuguy83/go-md2man/v2 v2.0.6/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g= +github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/decred/dcrd/crypto/blake256 v1.1.0 h1:zPMNGQCm0g4QTY27fOCorQW7EryeQ/U0x++OzVrdms8= @@ -21,6 +22,8 @@ github.com/google/pprof v0.0.0-20240409012703-83162a5b38cd h1:gbpYu9NMq8jhDVbvlG github.com/google/pprof v0.0.0-20240409012703-83162a5b38cd/go.mod h1:kf6iHlnVGwgKolg33glAes7Yg/8iWP8ukqeldJSO7jw= github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0= github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= +github.com/gowebpki/jcs v1.0.1 h1:Qjzg8EOkrOTuWP7DqQ1FbYtcpEbeTzUoTN9bptp8FOU= +github.com/gowebpki/jcs v1.0.1/go.mod h1:CID1cNZ+sHp1CCpAR8mPf6QRtagFBgPJE0FCUQ6+BrI= github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8= github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw= github.com/itchyny/gojq v0.12.17 h1:8av8eGduDb5+rvEdaOO+zQUjA04MS0m3Ps8HiD+fceg= @@ -33,6 +36,8 @@ github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWE github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y= github.com/ncruces/go-strftime v0.1.9 h1:bY0MQC28UADQmHmaF5dgpLmImcShSi2kHU9XLdhx/f4= github.com/ncruces/go-strftime v0.1.9/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls= +github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= +github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE= github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo= github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM= @@ -40,6 +45,10 @@ github.com/spf13/cobra v1.9.1 h1:CXSaggrXdbHK9CF+8ywj8Amf7PBRmPCOJugH954Nnlo= github.com/spf13/cobra v1.9.1/go.mod h1:nDyEzZ8ogv936Cinf6g1RU9MRY64Ir93oCnqb9wxYW0= github.com/spf13/pflag v1.0.6 h1:jFzHGLGAlb3ruxLB8MhbI6A8+AQX/2eW4qeyNZXNp2o= github.com/spf13/pflag v1.0.6/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg= +github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= +github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= +github.com/stretchr/testify v1.8.0 h1:pSgiaMZlXftHpm5L7V1+rVB+AZJydKsMxsQBIJw4PKk= +github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU= golang.org/x/crypto v0.31.0 h1:ihbySMvVjLAeSH1IbfcRTkD/iNscyz8rGzjF/E5hV6U= golang.org/x/crypto v0.31.0/go.mod h1:kDsLvtWBEx7MV9tJOj9bnXsPbxwJQ6csT/x4KIN4Ssk= golang.org/x/mod v0.16.0 h1:QX4fJ0Rr5cPQCF7O9lh9Se4pmwfwskqZfq5moyldzic= @@ -52,6 +61,7 @@ golang.org/x/tools v0.19.0 h1:tfGCXNR1OsFG+sVdLAitlpjAvD/I6dHDKnYrpEZUHkw= golang.org/x/tools v0.19.0/go.mod h1:qoJWxmGSIBmAeriMx19ogtrEPrGtDbPK634QFIcLAhc= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= modernc.org/cc/v4 v4.21.4 h1:3Be/Rdo1fpr8GrQ7IVw9OHtplU4gWbb+wNgeoBMmGLQ= diff --git a/internal/anpsdk/registry.go b/internal/anpsdk/registry.go index 40ece1b..bd8c39f 100644 --- a/internal/anpsdk/registry.go +++ b/internal/anpsdk/registry.go @@ -9,7 +9,7 @@ import ( const ( ModulePath = "github.com/agent-network-protocol/anp/golang" - ModuleVersion = "v0.8.5" + ModuleVersion = "v0.8.7" ) type ( @@ -27,6 +27,7 @@ type ( HttpSignatureOptions = anpauth.HttpSignatureOptions MessageServiceE2EEClient = directe2ee.MessageServiceDirectE2eeClient PrekeyBundle = directe2ee.PrekeyBundle + OneTimePrekey = directe2ee.OneTimePrekey DirectSessionState = directe2ee.DirectSessionState IMProof = anpproof.IMProof IMGenerationOptions = anpproof.IMGenerationOptions @@ -36,6 +37,7 @@ type ( RFC9421OriginProof = anpproof.RFC9421OriginProof RFC9421OriginProofGenerationOptions = anpproof.RFC9421OriginProofGenerationOptions RFC9421OriginProofVerificationOptions = anpproof.RFC9421OriginProofVerificationOptions + DidWbaBindingVerificationOptions = anpproof.DidWbaBindingVerificationOptions ) var ( @@ -60,6 +62,7 @@ var ( NewDidWbaVerifier = anpauth.NewDidWbaVerifier NewFileSessionStore = directe2ee.NewFileSessionStore NewFileSignedPrekeyStore = directe2ee.NewFileSignedPrekeyStore + NewFileOneTimePrekeyStore = directe2ee.NewFileOneTimePrekeyStore NewFilePendingOutboundStore = directe2ee.NewFilePendingOutboundStore NewMessageServiceDirectE2eeClient = directe2ee.NewMessageServiceDirectE2eeClient BuildIMContentDigest = anpproof.BuildIMContentDigest @@ -70,6 +73,8 @@ var ( VerifyIMProofWithDocument = anpproof.VerifyIMProofWithDocument GenerateGroupReceiptProof = anpproof.GenerateGroupReceiptProof VerifyGroupReceiptProof = anpproof.VerifyGroupReceiptProof + GenerateDidWbaBinding = anpproof.GenerateDidWbaBinding + VerifyDidWbaBinding = anpproof.VerifyDidWbaBinding BuildSignedRequestObject = anpproof.BuildSignedRequestObject CanonicalizeSignedRequestObject = anpproof.CanonicalizeSignedRequestObject BuildLogicalTargetURI = anpproof.BuildLogicalTargetURI diff --git a/internal/cli/group.go b/internal/cli/group.go index 9be1875..f6cab12 100644 --- a/internal/cli/group.go +++ b/internal/cli/group.go @@ -13,6 +13,8 @@ func (a *App) runGroupCreate(cmd *cobra.Command, args []string) error { description, _ := cmd.Flags().GetString("description") discoverability, _ := cmd.Flags().GetString("discoverability") admissionMode, _ := cmd.Flags().GetString("admission-mode") + messageSecurityProfile, _ := cmd.Flags().GetString("message-security-profile") + e2ee, _ := cmd.Flags().GetBool("e2ee") slug, _ := cmd.Flags().GetString("slug") goal, _ := cmd.Flags().GetString("goal") rules, _ := cmd.Flags().GetString("rules") @@ -27,20 +29,22 @@ func (a *App) runGroupCreate(cmd *cobra.Command, args []string) error { return a.messageExit(err, "Run `awiki-cli doctor` to inspect configuration and identity state.") } request := message.GroupCreateRequest{ - IdentityName: a.globals.Identity, - Name: name, - Description: description, - Discoverability: discoverability, - AdmissionMode: admissionMode, - Slug: slug, - Goal: goal, - Rules: rules, - MessagePrompt: messagePrompt, - DocURL: docURL, - AttachmentsAllowed: attachmentsAllowed, - MaxMembers: maxMembers, - MemberMaxMessages: memberMaxMessages, - MemberMaxTotalChars: memberMaxTotalChars, + IdentityName: a.globals.Identity, + Name: name, + Description: description, + Discoverability: discoverability, + AdmissionMode: admissionMode, + MessageSecurityProfile: messageSecurityProfile, + E2EE: e2ee, + Slug: slug, + Goal: goal, + Rules: rules, + MessagePrompt: messagePrompt, + DocURL: docURL, + AttachmentsAllowed: attachmentsAllowed, + MaxMembers: maxMembers, + MemberMaxMessages: memberMaxMessages, + MemberMaxTotalChars: memberMaxTotalChars, } if a.globals.DryRun { return a.renderSuccess(cmd.CommandPath(), format, a.globals.JQ, map[string]any{"plan": map[string]any{"action": "group.create", "identity": a.globals.Identity, "runtime_mode": service.Config().RuntimeMode, "request": request}}, "Dry run: group create planned", nil, a.identityMeta()) @@ -100,11 +104,12 @@ func (a *App) runGroupMemberMutation(cmd *cobra.Command, publicAction string, me member, _ := cmd.Flags().GetString("member") role, _ := cmd.Flags().GetString("role") reason, _ := cmd.Flags().GetString("reason") + e2ee, _ := cmd.Flags().GetBool("e2ee") service, format, err := a.messageService() if err != nil { return a.messageExit(err, "Run `awiki-cli doctor` to inspect configuration and identity state.") } - request := message.GroupMemberRequest{IdentityName: a.globals.Identity, Group: group, Member: member, Role: role, ReasonText: reason} + request := message.GroupMemberRequest{IdentityName: a.globals.Identity, Group: group, Member: member, Role: role, ReasonText: reason, E2EE: e2ee} if a.globals.DryRun { plan := map[string]any{"action": "group." + publicAction, "identity": a.globals.Identity, "runtime_mode": service.Config().RuntimeMode, "request": request} if completed := message.CompleteBareHandle(member, service.Config().DIDDomain); completed != strings.TrimSpace(member) { @@ -119,7 +124,7 @@ func (a *App) runGroupMemberMutation(cmd *cobra.Command, publicAction string, me result, err = service.RemoveGroupMember(cmd.Context(), request) } if err != nil { - return a.messageExit(err, "Make sure the group and member exist and the active identity has the required role.") + return a.messageExit(err, "Make sure the group and member exist and the active identity has the owner role required for membership changes.") } if result == nil { return commandResultMissing(cmd.CommandPath()) @@ -135,11 +140,13 @@ func (a *App) runGroupMemberMutation(cmd *cobra.Command, publicAction string, me func (a *App) runGroupLeave(cmd *cobra.Command, args []string) error { group, _ := cmd.Flags().GetString("group") + reason, _ := cmd.Flags().GetString("reason") + e2ee, _ := cmd.Flags().GetBool("e2ee") service, format, err := a.messageService() if err != nil { return a.messageExit(err, "Run `awiki-cli doctor` to inspect configuration and identity state.") } - request := message.GroupLeaveRequest{IdentityName: a.globals.Identity, Group: group} + request := message.GroupLeaveRequest{IdentityName: a.globals.Identity, Group: group, ReasonText: reason, E2EE: e2ee} if a.globals.DryRun { return a.renderSuccess(cmd.CommandPath(), format, a.globals.JQ, map[string]any{"plan": map[string]any{"action": "group.leave", "identity": a.globals.Identity, "runtime_mode": service.Config().RuntimeMode, "request": request}}, "Dry run: group leave planned", nil, a.identityMeta()) } diff --git a/internal/cli/group_e2ee.go b/internal/cli/group_e2ee.go new file mode 100644 index 0000000..02ce940 --- /dev/null +++ b/internal/cli/group_e2ee.go @@ -0,0 +1,208 @@ +package cli + +import ( + "github.com/agentconnect/awiki-cli/internal/identity" + "github.com/agentconnect/awiki-cli/internal/message" + "github.com/spf13/cobra" +) + +func (a *App) runGroupE2EEStatus(cmd *cobra.Command, args []string) error { + group, _ := cmd.Flags().GetString("group") + service, format, err := a.messageService() + if err != nil { + return a.messageExit(err, "Run `awiki-cli doctor` to inspect configuration and identity state.") + } + provider := message.NewDefaultMLSExecProvider(service.Config()) + plan := map[string]any{ + "action": "group.e2ee.status", + "identity": a.globals.Identity, + "runtime_mode": service.Config().RuntimeMode, + "profile": message.GroupE2EEProfile, + "security_profile": message.GroupE2EESecurityProfile, + "provider": "exec", + "binary": provider.BinaryPath, + "mls_data_dir": provider.DataDir, + "group": group, + "discovery_advertised": false, + } + if a.globals.DryRun { + return a.renderSuccess(cmd.CommandPath(), format, a.globals.JQ, map[string]any{"plan": plan}, "Dry run: group e2ee status planned", nil, a.identityMeta()) + } + result, statusErr := service.InspectGroupE2EEStatus(cmd.Context(), a.globals.Identity, group, 50) + if statusErr != nil { + return a.messageExit(statusErr, "Install anp-mls, set AWIKI_ANP_MLS_BINARY, and ensure message-service group E2EE APIs are enabled for focused validation.") + } + result.Data["plan"] = plan + return a.renderMessageResult(cmd, format, result) +} + +func (a *App) runGroupE2EEPublishKeyPackage(cmd *cobra.Command, args []string) error { + device, _ := cmd.Flags().GetString("device") + group, _ := cmd.Flags().GetString("group") + recovery, _ := cmd.Flags().GetBool("recovery") + contractTest, _ := cmd.Flags().GetBool("contract-test") + service, format, err := a.messageService() + if err != nil { + return a.messageExit(err, "Run `awiki-cli doctor` to inspect configuration and identity state.") + } + provider := message.NewDefaultMLSExecProvider(service.Config()) + plan := map[string]any{ + "action": "group.e2ee.publish_key_package", + "identity": a.globals.Identity, + "runtime_mode": service.Config().RuntimeMode, + "provider": "exec", + "binary": provider.BinaryPath, + "mls_data_dir": provider.DataDir, + "device": device, + "group": group, + "recovery": recovery, + "contract_test_only": contractTest, + } + if a.globals.DryRun { + return a.renderSuccess(cmd.CommandPath(), format, a.globals.JQ, map[string]any{"plan": plan}, "Dry run: group e2ee key package publish planned", nil, a.identityMeta()) + } + result, publishErr := service.PublishGroupE2EEKeyPackage(cmd.Context(), a.globals.Identity, device, group, recovery, contractTest) + if publishErr != nil { + return a.messageExit(publishErr, "Install anp-mls, set AWIKI_ANP_MLS_BINARY, pass --group when --recovery is used, and ensure message-service group E2EE APIs are enabled.") + } + result.Data["plan"] = plan + return a.renderMessageResult(cmd, format, result) +} + +func (a *App) runGroupE2EEPending(cmd *cobra.Command, args []string) error { + group, _ := cmd.Flags().GetString("group") + service, format, err := a.messageService() + if err != nil { + return a.messageExit(err, "Run `awiki-cli doctor` to inspect configuration and identity state.") + } + provider := message.NewDefaultMLSExecProvider(service.Config()) + plan := map[string]any{ + "action": "group.e2ee.pending", + "identity": a.globals.Identity, + "runtime_mode": service.Config().RuntimeMode, + "provider": "exec", + "mls_data_dir": provider.DataDir, + "group": group, + } + if a.globals.DryRun { + return a.renderSuccess(cmd.CommandPath(), format, a.globals.JQ, map[string]any{"plan": plan}, "Dry run: group e2ee pending planned", nil, a.identityMeta()) + } + result, pendingErr := service.PullGroupE2EENotices(cmd.Context(), a.globals.Identity, group, 50) + if pendingErr != nil { + return a.messageExit(pendingErr, "Ensure message-service group E2EE test flag is enabled for focused validation; discovery remains hidden by default.") + } + result.Data["plan"] = plan + return a.renderMessageResult(cmd, format, result) +} + +func (a *App) runGroupE2EERepair(cmd *cobra.Command, args []string) error { + group, _ := cmd.Flags().GetString("group") + service, format, err := a.messageService() + if err != nil { + return a.messageExit(err, "Run `awiki-cli doctor` to inspect configuration and identity state.") + } + provider := message.NewDefaultMLSExecProvider(service.Config()) + plan := map[string]any{ + "action": "group.e2ee.repair", + "identity": a.globals.Identity, + "runtime_mode": service.Config().RuntimeMode, + "provider": "exec", + "mls_data_dir": provider.DataDir, + "group": group, + "scope": "compare local MLS status to service head, safely finalize accepted pending commits, replay welcome/commit notices, and fail closed on unrecoverable gaps", + } + if a.globals.DryRun { + return a.renderSuccess(cmd.CommandPath(), format, a.globals.JQ, map[string]any{"plan": plan}, "Dry run: group e2ee repair planned", nil, a.identityMeta()) + } + result, repairErr := service.RepairGroupE2EENotices(cmd.Context(), a.globals.Identity, group, 50) + if repairErr != nil { + return a.messageExit(repairErr, "Install anp-mls, set AWIKI_ANP_MLS_BINARY, and ensure message-service group E2EE APIs are enabled for focused validation.") + } + result.Data["plan"] = plan + return a.renderMessageResult(cmd, format, result) +} + +func (a *App) runGroupE2EEProcessLeaveRequest(cmd *cobra.Command, args []string) error { + group, _ := cmd.Flags().GetString("group") + member, _ := cmd.Flags().GetString("member") + leaveRequestID, _ := cmd.Flags().GetString("leave-request-id") + reason, _ := cmd.Flags().GetString("reason") + service, format, err := a.messageService() + if err != nil { + return a.messageExit(err, "Run `awiki-cli doctor` to inspect configuration and identity state.") + } + provider := message.NewDefaultMLSExecProvider(service.Config()) + request := message.GroupE2EEProcessLeaveRequest{ + IdentityName: a.globals.Identity, + Group: group, + Member: member, + LeaveRequestID: leaveRequestID, + ReasonText: reason, + } + plan := map[string]any{ + "action": "group.e2ee.process_leave_request", + "identity": a.globals.Identity, + "runtime_mode": service.Config().RuntimeMode, + "provider": "exec", + "mls_data_dir": provider.DataDir, + "group": group, + "member": member, + "leave_request_id": leaveRequestID, + "request": request, + } + if a.globals.DryRun { + return a.renderSuccess(cmd.CommandPath(), format, a.globals.JQ, map[string]any{"plan": plan}, "Dry run: group e2ee leave request process planned", nil, a.identityMeta()) + } + result, processErr := service.ProcessGroupE2EELeaveRequest(cmd.Context(), request) + if processErr != nil { + return a.messageExit(processErr, "Ensure the leave request exists, the active identity can remove members, and anp-mls/message-service group E2EE APIs are enabled.") + } + result.Data["plan"] = plan + return a.renderMessageResult(cmd, format, result) +} + +func (a *App) runGroupE2EERecoverMember(cmd *cobra.Command, args []string) error { + group, _ := cmd.Flags().GetString("group") + member, _ := cmd.Flags().GetString("member") + device, _ := cmd.Flags().GetString("device") + service, format, err := a.messageService() + if err != nil { + return a.messageExit(err, "Run `awiki-cli doctor` to inspect configuration and identity state.") + } + provider := message.NewDefaultMLSExecProvider(service.Config()) + request := message.GroupE2EERecoverMemberRequest{ + IdentityName: a.globals.Identity, + Group: group, + Member: member, + DeviceID: device, + } + plan := map[string]any{ + "action": "group.e2ee.recover_member", + "identity": a.globals.Identity, + "runtime_mode": service.Config().RuntimeMode, + "provider": "exec", + "mls_data_dir": provider.DataDir, + "group": group, + "member": member, + "device": device, + "p4_membership_mutate": false, + "orchestration": []string{"lease recovery KeyPackage", "anp-mls recover-member-prepare", "hidden group.e2ee.recover_member", "finalize on accept", "abort on deterministic rejection"}, + } + if a.globals.DryRun { + return a.renderSuccess(cmd.CommandPath(), format, a.globals.JQ, map[string]any{"plan": plan}, "Dry run: group e2ee recover-member planned", nil, a.identityMeta()) + } + result, recoverErr := service.RecoverGroupE2EEMember(cmd.Context(), request) + if recoverErr != nil { + return a.messageExit(recoverErr, "Ensure the target remains an active P4 member, has published a --recovery --group KeyPackage, and anp-mls/message-service PR-B3 APIs are enabled.") + } + result.Data["plan"] = plan + return a.renderMessageResult(cmd, format, result) +} + +func activeIdentityDID(service *message.Service, name string) (string, error) { + record, err := identity.NewManager(service.Config().Paths).Load(name) + if err != nil { + return "", err + } + return record.DID, nil +} diff --git a/internal/cli/group_test.go b/internal/cli/group_test.go index c431194..cf312da 100644 --- a/internal/cli/group_test.go +++ b/internal/cli/group_test.go @@ -38,7 +38,7 @@ func TestGroupDryRunPlansRenderStableContracts(t *testing.T) { { name: "group remove uses kick action and member", spec: "group.remove", - setFlags: map[string]string{"group": "did:wba:example.com:groups:demo:e1_group", "member": "bob"}, + setFlags: map[string]string{"group": "did:wba:example.com:groups:demo:e1_group", "member": "bob", "e2ee": "true"}, wantSummary: "Dry run: group membership change planned", wantAction: "group.kick", verifyPlan: func(t *testing.T, plan map[string]any) { @@ -49,11 +49,31 @@ func TestGroupDryRunPlansRenderStableContracts(t *testing.T) { if request["Member"] != "bob" { t.Fatalf("request.Member = %#v, want %q", request["Member"], "bob") } + if request["E2EE"] != true { + t.Fatalf("request.E2EE = %#v, want true", request["E2EE"]) + } if plan["member_handle"] != "bob.awiki.ai" { t.Fatalf("plan.member_handle = %#v, want bob.awiki.ai", plan["member_handle"]) } }, }, + + { + name: "group leave e2ee plans hidden leave request", + spec: "group.leave", + setFlags: map[string]string{"group": "did:wba:example.com:groups:demo:e1_group", "e2ee": "true", "reason": "done"}, + wantSummary: "Dry run: group leave planned", + wantAction: "group.leave", + verifyPlan: func(t *testing.T, plan map[string]any) { + request, ok := plan["request"].(map[string]any) + if !ok { + t.Fatalf("plan.request type = %T, want map[string]any", plan["request"]) + } + if request["E2EE"] != true || request["ReasonText"] != "done" { + t.Fatalf("request = %#v, want E2EE leave request plan", request) + } + }, + }, { name: "group messages includes cursor and limit", spec: "group.messages", @@ -73,6 +93,86 @@ func TestGroupDryRunPlansRenderStableContracts(t *testing.T) { } }, }, + { + name: "group e2ee status exposes exec provider data dir without advertising discovery", + spec: "group.e2ee.status", + setFlags: map[string]string{"group": "did:wba:example.com:groups:demo:e1_group"}, + wantSummary: "Dry run: group e2ee status planned", + wantAction: "group.e2ee.status", + verifyPlan: func(t *testing.T, plan map[string]any) { + if plan["provider"] != "exec" { + t.Fatalf("plan.provider = %#v, want exec", plan["provider"]) + } + if plan["discovery_advertised"] != false { + t.Fatalf("plan.discovery_advertised = %#v, want false", plan["discovery_advertised"]) + } + if _, ok := plan["artifact_mode"]; ok { + t.Fatalf("plan.artifact_mode should not be exposed by real MLS status diagnostics: %#v", plan) + } + if plan["mls_data_dir"] == "" { + t.Fatal("plan.mls_data_dir should be populated") + } + }, + }, + { + name: "group e2ee pending plans P6 notice pull", + spec: "group.e2ee.pending", + setFlags: map[string]string{"group": "did:wba:example.com:groups:demo:e1_group"}, + wantSummary: "Dry run: group e2ee pending planned", + wantAction: "group.e2ee.pending", + verifyPlan: func(t *testing.T, plan map[string]any) { + if plan["provider"] != "exec" { + t.Fatalf("plan.provider = %#v, want exec", plan["provider"]) + } + if plan["group"] != "did:wba:example.com:groups:demo:e1_group" { + t.Fatalf("plan.group = %#v, want group DID", plan["group"]) + } + }, + }, + { + name: "group e2ee repair plans P6 notice replay", + spec: "group.e2ee.repair", + setFlags: map[string]string{"group": "did:wba:example.com:groups:demo:e1_group"}, + wantSummary: "Dry run: group e2ee repair planned", + wantAction: "group.e2ee.repair", + verifyPlan: func(t *testing.T, plan map[string]any) { + if plan["provider"] != "exec" { + t.Fatalf("plan.provider = %#v, want exec", plan["provider"]) + } + if plan["scope"] == "" { + t.Fatalf("plan.scope should describe durable notice replay: %#v", plan) + } + }, + }, + + { + name: "group e2ee process leave request plans owner remove", + spec: "group.e2ee.process-leave-request", + setFlags: map[string]string{"group": "did:wba:example.com:groups:demo:e1_group", "member": "bob", "leave-request-id": "lr-bob-1"}, + wantSummary: "Dry run: group e2ee leave request process planned", + wantAction: "group.e2ee.process_leave_request", + verifyPlan: func(t *testing.T, plan map[string]any) { + if plan["member"] != "bob" || plan["leave_request_id"] != "lr-bob-1" { + t.Fatalf("plan = %#v, want member and leave request id", plan) + } + }, + }, + { + name: "group create e2ee alias maps to group-e2ee request", + spec: "group.create", + setFlags: map[string]string{"name": "Secret Group", "e2ee": "true"}, + wantSummary: "Dry run: group create planned", + wantAction: "group.create", + verifyPlan: func(t *testing.T, plan map[string]any) { + request, ok := plan["request"].(map[string]any) + if !ok { + t.Fatalf("plan.request type = %T, want map[string]any", plan["request"]) + } + if request["E2EE"] != true { + t.Fatalf("request.E2EE = %#v, want true", request["E2EE"]) + } + }, + }, } for _, tc := range cases { @@ -97,6 +197,16 @@ func TestGroupDryRunPlansRenderStableContracts(t *testing.T) { return app.runGroupKick(cmd, nil) case "group.messages": return app.runGroupMessages(cmd, nil) + case "group.leave": + return app.runGroupLeave(cmd, nil) + case "group.e2ee.status": + return app.runGroupE2EEStatus(cmd, nil) + case "group.e2ee.pending": + return app.runGroupE2EEPending(cmd, nil) + case "group.e2ee.repair": + return app.runGroupE2EERepair(cmd, nil) + case "group.e2ee.process-leave-request": + return app.runGroupE2EEProcessLeaveRequest(cmd, nil) default: t.Fatalf("unsupported spec %q", tc.spec) return nil diff --git a/internal/cli/msg.go b/internal/cli/msg.go index 502b67c..ed3185a 100644 --- a/internal/cli/msg.go +++ b/internal/cli/msg.go @@ -73,7 +73,9 @@ func (a *App) messageExit(err error, hint string) error { case errors.Is(err, identity.ErrUserRegistrationRequired): return output.NewExitError("identity_required", 3, err.Error(), "Complete user setup with `awiki-cli id register --handle ...` or recover an existing handle before using msg commands.") case errors.Is(err, message.ErrSecureNotSupported): - return output.NewExitError("unsupported_mode", 1, err.Error(), "Direct secure messaging is planned for Phase 5.") + return output.NewExitError("unsupported_mode", 1, err.Error(), "Secure messaging is currently supported only for direct text messaging.") + case errors.Is(err, message.ErrGroupE2EESelfLeaveUnsupported): + return output.NewExitError("unsupported_mode", 1, err.Error(), "For PR-A group E2EE, ask a group owner/admin to remove the member; self-leave requires a future epoch-advancing leave-request flow.") case errors.Is(err, message.ErrTransportUnavailable): return output.NewExitError("transport_unavailable", 1, err.Error(), "Start the websocket listener/daemon or switch runtime.mode back to http.") default: @@ -357,6 +359,149 @@ func (a *App) runMsgMarkRead(cmd *cobra.Command, args []string) error { return a.renderMessageResult(cmd, format, result) } +func (a *App) runMsgSecureStatus(cmd *cobra.Command, args []string) error { + with, _ := cmd.Flags().GetString("with") + service, format, err := a.messageService() + if err != nil { + return a.messageExit(err, "Run `awiki-cli doctor` to inspect configuration and identity state.") + } + request := message.SecureStatusRequest{ + IdentityName: a.globals.Identity, + With: with, + } + if a.globals.DryRun { + data := map[string]any{"plan": map[string]any{ + "action": "msg.secure.status", + "identity": a.globals.Identity, + "with": with, + }} + return a.renderSuccess(cmd.CommandPath(), format, a.globals.JQ, data, "Dry run: secure status planned", nil, a.identityMeta()) + } + result, err := service.SecureStatus(cmd.Context(), request) + if err != nil { + return a.messageExit(err, "Make sure the active identity exists and the peer filter is valid.") + } + return a.renderMessageResult(cmd, format, result) +} + +func (a *App) runMsgSecureInit(cmd *cobra.Command, args []string) error { + with, _ := cmd.Flags().GetString("with") + service, format, err := a.messageService() + if err != nil { + return a.messageExit(err, "Run `awiki-cli doctor` to inspect configuration and identity state.") + } + request := message.SecurePeerRequest{ + IdentityName: a.globals.Identity, + With: with, + } + if a.globals.DryRun { + data := map[string]any{"plan": map[string]any{ + "action": "msg.secure.init", + "identity": a.globals.Identity, + "with": with, + }} + return a.renderSuccess(cmd.CommandPath(), format, a.globals.JQ, data, "Dry run: secure init planned", nil, a.identityMeta()) + } + result, err := service.SecureInit(cmd.Context(), request) + if err != nil { + return a.messageExit(err, "Make sure the target exists and the active identity has secure E2EE key material.") + } + return a.renderMessageResult(cmd, format, result) +} + +func (a *App) runMsgSecureRepair(cmd *cobra.Command, args []string) error { + with, _ := cmd.Flags().GetString("with") + service, format, err := a.messageService() + if err != nil { + return a.messageExit(err, "Run `awiki-cli doctor` to inspect configuration and identity state.") + } + request := message.SecurePeerRequest{ + IdentityName: a.globals.Identity, + With: with, + } + if a.globals.DryRun { + data := map[string]any{"plan": map[string]any{ + "action": "msg.secure.repair", + "identity": a.globals.Identity, + "with": with, + }} + return a.renderSuccess(cmd.CommandPath(), format, a.globals.JQ, data, "Dry run: secure repair planned", nil, a.identityMeta()) + } + result, err := service.SecureRepair(cmd.Context(), request) + if err != nil { + return a.messageExit(err, "Make sure the target exists and the active identity can rebuild secure state.") + } + return a.renderMessageResult(cmd, format, result) +} + +func (a *App) runMsgSecureFailed(cmd *cobra.Command, args []string) error { + service, format, err := a.messageService() + if err != nil { + return a.messageExit(err, "Run `awiki-cli doctor` to inspect configuration and identity state.") + } + request := message.SecureStatusRequest{IdentityName: a.globals.Identity} + if a.globals.DryRun { + data := map[string]any{"plan": map[string]any{ + "action": "msg.secure.failed", + "identity": a.globals.Identity, + }} + return a.renderSuccess(cmd.CommandPath(), format, a.globals.JQ, data, "Dry run: secure failed listing planned", nil, a.identityMeta()) + } + result, err := service.SecureFailed(cmd.Context(), request) + if err != nil { + return a.messageExit(err, "Make sure the active identity exists and local storage is readable.") + } + return a.renderMessageResult(cmd, format, result) +} + +func (a *App) runMsgSecureRetry(cmd *cobra.Command, args []string) error { + if len(args) != 1 { + return output.NewExitError("invalid_argument", 2, "msg secure retry requires one outbox id.", "Usage: awiki-cli msg secure retry ") + } + service, format, err := a.messageService() + if err != nil { + return a.messageExit(err, "Run `awiki-cli doctor` to inspect configuration and identity state.") + } + request := message.SecureOutboxActionRequest{IdentityName: a.globals.Identity, OutboxID: args[0]} + if a.globals.DryRun { + data := map[string]any{"plan": map[string]any{ + "action": "msg.secure.retry", + "identity": a.globals.Identity, + "outbox_id": args[0], + }} + return a.renderSuccess(cmd.CommandPath(), format, a.globals.JQ, data, "Dry run: secure retry planned", nil, a.identityMeta()) + } + result, err := service.SecureRetry(cmd.Context(), request) + if err != nil { + return a.messageExit(err, "Make sure the outbox id exists and the active identity can reach the target service.") + } + return a.renderMessageResult(cmd, format, result) +} + +func (a *App) runMsgSecureDrop(cmd *cobra.Command, args []string) error { + if len(args) != 1 { + return output.NewExitError("invalid_argument", 2, "msg secure drop requires one outbox id.", "Usage: awiki-cli msg secure drop ") + } + service, format, err := a.messageService() + if err != nil { + return a.messageExit(err, "Run `awiki-cli doctor` to inspect configuration and identity state.") + } + request := message.SecureOutboxActionRequest{IdentityName: a.globals.Identity, OutboxID: args[0]} + if a.globals.DryRun { + data := map[string]any{"plan": map[string]any{ + "action": "msg.secure.drop", + "identity": a.globals.Identity, + "outbox_id": args[0], + }} + return a.renderSuccess(cmd.CommandPath(), format, a.globals.JQ, data, "Dry run: secure drop planned", nil, a.identityMeta()) + } + result, err := service.SecureDrop(cmd.Context(), request) + if err != nil { + return a.messageExit(err, "Make sure the outbox id exists for the active identity.") + } + return a.renderMessageResult(cmd, format, result) +} + func defaultString(value string, fallback string) string { if strings.TrimSpace(value) == "" { return fallback diff --git a/internal/cli/msg_test.go b/internal/cli/msg_test.go index 468644c..1b9a7f7 100644 --- a/internal/cli/msg_test.go +++ b/internal/cli/msg_test.go @@ -81,6 +81,72 @@ func TestMsgDryRunPlansRenderStableContracts(t *testing.T) { } }, }, + { + name: "secure status carries peer filter", + spec: "msg.secure.status", + setFlags: map[string]string{"with": "bob"}, + wantSummary: "Dry run: secure status planned", + wantAction: "msg.secure.status", + verifyPlan: func(t *testing.T, plan map[string]any) { + if plan["with"] != "bob" { + t.Fatalf("plan.with = %#v, want bob", plan["with"]) + } + }, + }, + { + name: "secure init carries peer filter", + spec: "msg.secure.init", + setFlags: map[string]string{"with": "bob"}, + wantSummary: "Dry run: secure init planned", + wantAction: "msg.secure.init", + verifyPlan: func(t *testing.T, plan map[string]any) { + if plan["with"] != "bob" { + t.Fatalf("plan.with = %#v, want bob", plan["with"]) + } + }, + }, + { + name: "secure repair carries peer filter", + spec: "msg.secure.repair", + setFlags: map[string]string{"with": "bob"}, + wantSummary: "Dry run: secure repair planned", + wantAction: "msg.secure.repair", + verifyPlan: func(t *testing.T, plan map[string]any) { + if plan["with"] != "bob" { + t.Fatalf("plan.with = %#v, want bob", plan["with"]) + } + }, + }, + { + name: "secure failed lists current identity", + spec: "msg.secure.failed", + wantSummary: "Dry run: secure failed listing planned", + wantAction: "msg.secure.failed", + }, + { + name: "secure retry keeps outbox id", + spec: "msg.secure.retry", + args: []string{"outbox-1"}, + wantSummary: "Dry run: secure retry planned", + wantAction: "msg.secure.retry", + verifyPlan: func(t *testing.T, plan map[string]any) { + if plan["outbox_id"] != "outbox-1" { + t.Fatalf("plan.outbox_id = %#v, want outbox-1", plan["outbox_id"]) + } + }, + }, + { + name: "secure drop keeps outbox id", + spec: "msg.secure.drop", + args: []string{"outbox-1"}, + wantSummary: "Dry run: secure drop planned", + wantAction: "msg.secure.drop", + verifyPlan: func(t *testing.T, plan map[string]any) { + if plan["outbox_id"] != "outbox-1" { + t.Fatalf("plan.outbox_id = %#v, want outbox-1", plan["outbox_id"]) + } + }, + }, } for _, tc := range cases { @@ -105,6 +171,18 @@ func TestMsgDryRunPlansRenderStableContracts(t *testing.T) { return app.runMsgHistory(cmd, nil) case "msg.mark-read": return app.runMsgMarkRead(cmd, tc.args) + case "msg.secure.status": + return app.runMsgSecureStatus(cmd, tc.args) + case "msg.secure.init": + return app.runMsgSecureInit(cmd, tc.args) + case "msg.secure.repair": + return app.runMsgSecureRepair(cmd, tc.args) + case "msg.secure.failed": + return app.runMsgSecureFailed(cmd, tc.args) + case "msg.secure.retry": + return app.runMsgSecureRetry(cmd, tc.args) + case "msg.secure.drop": + return app.runMsgSecureDrop(cmd, tc.args) default: t.Fatalf("unsupported spec %q", tc.spec) return nil @@ -160,3 +238,20 @@ func TestRunMsgSendRejectsInvalidFlagCombinationsBeforeService(t *testing.T) { t.Fatalf("runMsgSend() error = %v, want attachment file validation", err) } } + +func TestRunMsgSecureRetryAndDropRequireOutboxID(t *testing.T) { + t.Parallel() + + catalog := cmdmeta.NewCatalog() + app := &App{catalog: catalog, globals: GlobalOptions{Format: string(output.FormatJSON)}} + + retryCmd := app.commandFromSpec(catalog.MustLookup("msg.secure.retry")) + if err := app.runMsgSecureRetry(retryCmd, nil); err == nil || !strings.Contains(err.Error(), "requires one outbox id") { + t.Fatalf("runMsgSecureRetry() error = %v, want outbox id error", err) + } + + dropCmd := app.commandFromSpec(catalog.MustLookup("msg.secure.drop")) + if err := app.runMsgSecureDrop(dropCmd, nil); err == nil || !strings.Contains(err.Error(), "requires one outbox id") { + t.Fatalf("runMsgSecureDrop() error = %v, want outbox id error", err) + } +} diff --git a/internal/cli/root.go b/internal/cli/root.go index 9b19dbf..f30307f 100644 --- a/internal/cli/root.go +++ b/internal/cli/root.go @@ -193,6 +193,18 @@ func (a *App) handlerFor(spec cmdmeta.CommandSpec) func(*cobra.Command, []string return a.runMsgHistory case "msg.mark-read": return a.runMsgMarkRead + case "msg.secure.status": + return a.runMsgSecureStatus + case "msg.secure.init": + return a.runMsgSecureInit + case "msg.secure.repair": + return a.runMsgSecureRepair + case "msg.secure.failed": + return a.runMsgSecureFailed + case "msg.secure.retry": + return a.runMsgSecureRetry + case "msg.secure.drop": + return a.runMsgSecureDrop case "mail.inbox": return a.runMailInbox case "mail.read": @@ -229,6 +241,18 @@ func (a *App) handlerFor(spec cmdmeta.CommandSpec) func(*cobra.Command, []string return a.runGroupMembers case "group.messages": return a.runGroupMessages + case "group.e2ee.status": + return a.runGroupE2EEStatus + case "group.e2ee.publish-key-package": + return a.runGroupE2EEPublishKeyPackage + case "group.e2ee.pending": + return a.runGroupE2EEPending + case "group.e2ee.repair": + return a.runGroupE2EERepair + case "group.e2ee.recover-member": + return a.runGroupE2EERecoverMember + case "group.e2ee.process-leave-request": + return a.runGroupE2EEProcessLeaveRequest case "page.create": return a.runPageCreate case "page.list": diff --git a/internal/cmdmeta/catalog.go b/internal/cmdmeta/catalog.go index af3678c..cf09046 100644 --- a/internal/cmdmeta/catalog.go +++ b/internal/cmdmeta/catalog.go @@ -149,23 +149,30 @@ func defaultSpecs() []CommandSpec { {Name: "mail.send", Use: "send", Short: "Send a mail message", Phase: "phase5", Implemented: true, Handler: "mail.send", SideEffect: true, Outputs: []string{"json", "pretty"}, Flags: []FlagSpec{{Name: "to", Type: "string", Usage: "Recipient addresses (comma-separated)", Required: true}, {Name: "cc", Type: "string", Usage: "CC addresses (comma-separated)"}, {Name: "subject", Type: "string", Usage: "Mail subject", Required: true}, {Name: "body", Type: "string", Usage: "Plain text body", Required: true}, {Name: "html", Type: "string", Usage: "HTML body"}}}, {Name: "mail.attachment", Use: "attachment", Short: "Mail attachment commands", Phase: "phase5", Implemented: true}, {Name: "mail.attachment.download", Use: "download", Short: "Download a mail attachment", Phase: "phase5", Implemented: true, Handler: "mail.attachment.download", SideEffect: true, Outputs: []string{"json", "pretty"}, Flags: []FlagSpec{{Name: "message-id", Type: "string", Usage: "Message id", Required: true}, {Name: "attachment-index", Type: "int", Usage: "Attachment index (0-based)", Default: "0"}, {Name: "output", Type: "string", Usage: "Output file path"}}}, - {Name: "msg.secure", Use: "secure", Short: "Secure direct messaging commands", Phase: "phase5", Implemented: false}, - {Name: "msg.secure.status", Use: "status", Short: "Inspect secure messaging status", Phase: "phase5", Implemented: false, Handler: "stub", Outputs: []string{"json", "pretty", "table"}, Flags: []FlagSpec{{Name: "with", Type: "string", Usage: "Target peer DID or handle"}}}, - {Name: "msg.secure.init", Use: "init", Short: "Initialize a secure session", Phase: "phase5", Implemented: false, Handler: "stub", SideEffect: true, Outputs: []string{"json", "pretty"}, Flags: []FlagSpec{{Name: "with", Type: "string", Usage: "Target peer DID or handle", Required: true}}}, - {Name: "msg.secure.repair", Use: "repair", Short: "Repair a secure session", Phase: "phase5", Implemented: false, Handler: "stub", SideEffect: true, Outputs: []string{"json", "pretty"}, Flags: []FlagSpec{{Name: "with", Type: "string", Usage: "Target peer DID or handle", Required: true}}}, - {Name: "msg.secure.failed", Use: "failed", Short: "List failed secure outbox items", Phase: "phase5", Implemented: false, Handler: "stub", Outputs: []string{"json", "pretty", "table"}}, - {Name: "msg.secure.retry", Use: "retry ", Short: "Retry one failed secure outbox item", Phase: "phase5", Implemented: false, Handler: "stub", SideEffect: true, Outputs: []string{"json", "pretty"}}, - {Name: "msg.secure.drop", Use: "drop ", Short: "Drop one failed secure outbox item", Phase: "phase5", Implemented: false, Handler: "stub", SideEffect: true, Outputs: []string{"json", "pretty"}}, + {Name: "msg.secure", Use: "secure", Short: "Secure direct messaging commands", Phase: "phase5", Implemented: true}, + {Name: "msg.secure.status", Use: "status", Short: "Inspect secure messaging status", Phase: "phase5", Implemented: true, Handler: "msg.secure.status", Outputs: []string{"json", "pretty", "table"}, Flags: []FlagSpec{{Name: "with", Type: "string", Usage: "Target peer DID or handle"}}}, + {Name: "msg.secure.init", Use: "init", Short: "Initialize a secure session", Phase: "phase5", Implemented: true, Handler: "msg.secure.init", SideEffect: true, Outputs: []string{"json", "pretty"}, Flags: []FlagSpec{{Name: "with", Type: "string", Usage: "Target peer DID or handle", Required: true}}}, + {Name: "msg.secure.repair", Use: "repair", Short: "Repair a secure session", Phase: "phase5", Implemented: true, Handler: "msg.secure.repair", SideEffect: true, Outputs: []string{"json", "pretty"}, Flags: []FlagSpec{{Name: "with", Type: "string", Usage: "Target peer DID or handle", Required: true}}}, + {Name: "msg.secure.failed", Use: "failed", Short: "List failed secure outbox items", Phase: "phase5", Implemented: true, Handler: "msg.secure.failed", Outputs: []string{"json", "pretty", "table"}}, + {Name: "msg.secure.retry", Use: "retry ", Short: "Retry one failed secure outbox item", Phase: "phase5", Implemented: true, Handler: "msg.secure.retry", SideEffect: true, Outputs: []string{"json", "pretty"}}, + {Name: "msg.secure.drop", Use: "drop ", Short: "Drop one failed secure outbox item", Phase: "phase5", Implemented: true, Handler: "msg.secure.drop", SideEffect: true, Outputs: []string{"json", "pretty"}}, {Name: "group", Use: "group", Short: "Group lifecycle commands", Phase: "phase1", Implemented: true}, - {Name: "group.create", Use: "create", Short: "Create a new group", Phase: "phase5", Implemented: true, Handler: "group.create", SideEffect: true, Outputs: []string{"json", "pretty"}, Flags: []FlagSpec{{Name: "name", Type: "string", Usage: "Group display name", Required: true}, {Name: "description", Type: "string", Usage: "Group description"}, {Name: "discoverability", Type: "string", Usage: "Discoverability mode", Default: "private"}, {Name: "admission-mode", Type: "string", Usage: "Admission mode", Default: "open-join"}, {Name: "slug", Type: "string", Usage: "Group slug"}, {Name: "goal", Type: "string", Usage: "Group goal"}, {Name: "rules", Type: "string", Usage: "Group rules"}, {Name: "message-prompt", Type: "string", Usage: "Default group prompt"}, {Name: "doc-url", Type: "string", Usage: "Group document URL"}, {Name: "attachments-allowed", Type: "bool", Usage: "Allow attachments"}, {Name: "max-members", Type: "string", Usage: "Maximum group members"}, {Name: "member-max-messages", Type: "int", Usage: "Per-member message limit"}, {Name: "member-max-total-chars", Type: "int", Usage: "Per-member total char limit"}}}, + {Name: "group.create", Use: "create", Short: "Create a new group", Phase: "phase5", Implemented: true, Handler: "group.create", SideEffect: true, Outputs: []string{"json", "pretty"}, Flags: []FlagSpec{{Name: "name", Type: "string", Usage: "Group display name", Required: true}, {Name: "description", Type: "string", Usage: "Group description"}, {Name: "discoverability", Type: "string", Usage: "Discoverability mode", Default: "private"}, {Name: "admission-mode", Type: "string", Usage: "Admission mode", Default: "open-join"}, {Name: "message-security-profile", Type: "string", Usage: "Message security profile", Default: "transport-protected", Choices: []string{"transport-protected", "group-e2ee"}}, {Name: "e2ee", Type: "bool", Usage: "Alias for --message-security-profile group-e2ee"}, {Name: "slug", Type: "string", Usage: "Group slug"}, {Name: "goal", Type: "string", Usage: "Group goal"}, {Name: "rules", Type: "string", Usage: "Group rules"}, {Name: "message-prompt", Type: "string", Usage: "Default group prompt"}, {Name: "doc-url", Type: "string", Usage: "Group document URL"}, {Name: "attachments-allowed", Type: "bool", Usage: "Allow attachments"}, {Name: "max-members", Type: "string", Usage: "Maximum group members"}, {Name: "member-max-messages", Type: "int", Usage: "Per-member message limit"}, {Name: "member-max-total-chars", Type: "int", Usage: "Per-member total char limit"}}}, {Name: "group.get", Use: "get", Short: "Show group details", Aliases: []string{"show"}, Phase: "phase5", Implemented: true, Handler: "group.get", Outputs: []string{"json", "pretty", "table"}, Flags: []FlagSpec{{Name: "group", Type: "string", Usage: "Group DID", Required: true}}}, {Name: "group.join", Use: "join", Short: "Join an open group", Phase: "phase5", Implemented: true, Handler: "group.join", SideEffect: true, Outputs: []string{"json", "pretty"}, Flags: []FlagSpec{{Name: "group", Type: "string", Usage: "Group DID", Required: true}, {Name: "reason", Type: "string", Usage: "Join reason"}}}, - {Name: "group.add", Use: "add", Short: "Add a member to a group", Phase: "phase5", Implemented: true, Handler: "group.add", SideEffect: true, Outputs: []string{"json", "pretty"}, Flags: []FlagSpec{{Name: "group", Type: "string", Usage: "Group DID", Required: true}, {Name: "member", Type: "string", Usage: "Member DID or handle", Required: true}, {Name: "role", Type: "string", Usage: "Member role", Default: "member"}}}, - {Name: "group.remove", Use: "remove", Short: "Remove a member from a group", Aliases: []string{"kick"}, Phase: "phase5", Implemented: true, Handler: "group.remove", SideEffect: true, Outputs: []string{"json", "pretty"}, Flags: []FlagSpec{{Name: "group", Type: "string", Usage: "Group DID", Required: true}, {Name: "member", Type: "string", Usage: "Member DID or handle", Required: true}, {Name: "reason", Type: "string", Usage: "Removal reason"}}}, - {Name: "group.leave", Use: "leave", Short: "Leave a group", Phase: "phase5", Implemented: true, Handler: "group.leave", SideEffect: true, Outputs: []string{"json", "pretty"}, Flags: []FlagSpec{{Name: "group", Type: "string", Usage: "Group DID", Required: true}}}, + {Name: "group.add", Use: "add", Short: "Add a member to a group", Phase: "phase5", Implemented: true, Handler: "group.add", SideEffect: true, Outputs: []string{"json", "pretty"}, Flags: []FlagSpec{{Name: "group", Type: "string", Usage: "Group DID", Required: true}, {Name: "member", Type: "string", Usage: "Member DID or handle", Required: true}, {Name: "role", Type: "string", Usage: "Member role", Default: "member"}, {Name: "e2ee", Type: "bool", Usage: "Force group E2EE add-member orchestration when cache is unavailable"}}}, + {Name: "group.remove", Use: "remove", Short: "Remove a member from a group", Aliases: []string{"kick"}, Phase: "phase5", Implemented: true, Handler: "group.remove", SideEffect: true, Outputs: []string{"json", "pretty"}, Flags: []FlagSpec{{Name: "group", Type: "string", Usage: "Group DID", Required: true}, {Name: "member", Type: "string", Usage: "Member DID or handle", Required: true}, {Name: "reason", Type: "string", Usage: "Removal reason"}, {Name: "e2ee", Type: "bool", Usage: "Force group E2EE remove-member orchestration when cache is unavailable"}}}, + {Name: "group.leave", Use: "leave", Short: "Leave a group", Phase: "phase5", Implemented: true, Handler: "group.leave", SideEffect: true, Outputs: []string{"json", "pretty"}, Flags: []FlagSpec{{Name: "group", Type: "string", Usage: "Group DID", Required: true}, {Name: "reason", Type: "string", Usage: "Leave reason"}, {Name: "e2ee", Type: "bool", Usage: "Force group E2EE leave-request orchestration when cache is unavailable"}}}, {Name: "group.update", Use: "update", Short: "Update group profile or policy", Phase: "phase5", Implemented: true, Handler: "group.update", SideEffect: true, Outputs: []string{"json", "pretty"}, Flags: []FlagSpec{{Name: "group", Type: "string", Usage: "Group DID", Required: true}, {Name: "name", Type: "string", Usage: "New group display name"}, {Name: "description", Type: "string", Usage: "New group description"}, {Name: "discoverability", Type: "string", Usage: "Discoverability mode"}, {Name: "admission-mode", Type: "string", Usage: "Admission mode"}, {Name: "slug", Type: "string", Usage: "New group slug"}, {Name: "goal", Type: "string", Usage: "New group goal"}, {Name: "rules", Type: "string", Usage: "New group rules"}, {Name: "message-prompt", Type: "string", Usage: "New group prompt"}, {Name: "doc-url", Type: "string", Usage: "New group document URL"}, {Name: "attachments-allowed", Type: "bool", Usage: "Allow attachments"}, {Name: "max-members", Type: "string", Usage: "Maximum group members"}, {Name: "member-max-messages", Type: "int", Usage: "Per-member message limit"}, {Name: "member-max-total-chars", Type: "int", Usage: "Per-member total char limit"}}}, {Name: "group.members", Use: "members", Short: "List active group members", Phase: "phase5", Implemented: true, Handler: "group.members", Outputs: []string{"json", "pretty", "table"}, Flags: []FlagSpec{{Name: "group", Type: "string", Usage: "Group DID", Required: true}, {Name: "limit", Type: "int", Usage: "Maximum number of rows", Default: "100"}}}, {Name: "group.messages", Use: "messages", Short: "List group messages", Phase: "phase5", Implemented: true, Handler: "group.messages", Outputs: []string{"json", "pretty", "table"}, Flags: []FlagSpec{{Name: "group", Type: "string", Usage: "Group DID", Required: true}, {Name: "limit", Type: "int", Usage: "Maximum number of rows", Default: "50"}, {Name: "cursor", Type: "string", Usage: "Pagination cursor"}}}, + {Name: "group.e2ee", Use: "e2ee", Short: "Inspect test-only group E2EE state", Phase: "phase6", Implemented: true}, + {Name: "group.e2ee.status", Use: "status", Short: "Inspect local group E2EE MLS provider status", Phase: "phase6", Implemented: true, Handler: "group.e2ee.status", Outputs: []string{"json", "pretty", "table"}, Flags: []FlagSpec{{Name: "group", Type: "string", Usage: "Group DID"}}}, + {Name: "group.e2ee.publish-key-package", Use: "publish-key-package", Short: "Plan a test-only group E2EE KeyPackage publish", Phase: "phase6", Implemented: true, Handler: "group.e2ee.publish-key-package", SideEffect: true, Outputs: []string{"json", "pretty"}, Flags: []FlagSpec{{Name: "device", Type: "string", Usage: "Local MLS device id", Default: "default"}, {Name: "recovery", Type: "bool", Usage: "Publish a same-device recovery KeyPackage"}, {Name: "group", Type: "string", Usage: "Target group DID for recovery KeyPackages"}, {Name: "contract-test", Type: "bool", Usage: "Use non-cryptographic contract-test artifacts"}}}, + {Name: "group.e2ee.pending", Use: "pending", Short: "Pull pending group E2EE P6 notices", Phase: "phase6", Implemented: true, Handler: "group.e2ee.pending", Outputs: []string{"json", "pretty", "table"}, Flags: []FlagSpec{{Name: "group", Type: "string", Usage: "Optional group DID filter"}}}, + {Name: "group.e2ee.repair", Use: "repair", Short: "Replay pending group E2EE P6 notices", Phase: "phase6", Implemented: true, Handler: "group.e2ee.repair", SideEffect: true, Outputs: []string{"json", "pretty"}, Flags: []FlagSpec{{Name: "group", Type: "string", Usage: "Optional group DID filter"}}}, + {Name: "group.e2ee.recover-member", Use: "recover-member", Short: "Recover an active same-device group E2EE member", Phase: "phase6", Implemented: true, Handler: "group.e2ee.recover-member", SideEffect: true, Outputs: []string{"json", "pretty"}, Flags: []FlagSpec{{Name: "group", Type: "string", Usage: "Group DID", Required: true}, {Name: "member", Type: "string", Usage: "Active member DID or handle to recover", Required: true}, {Name: "device", Type: "string", Usage: "Target MLS device id", Default: "default"}}}, + {Name: "group.e2ee.process-leave-request", Use: "process-leave-request", Short: "Process a pending group E2EE leave request", Phase: "phase6", Implemented: true, Handler: "group.e2ee.process-leave-request", SideEffect: true, Outputs: []string{"json", "pretty"}, Flags: []FlagSpec{{Name: "group", Type: "string", Usage: "Group DID", Required: true}, {Name: "member", Type: "string", Usage: "Leaving member DID or handle", Required: true}, {Name: "leave-request-id", Type: "string", Usage: "Leave request id to consume"}, {Name: "reason", Type: "string", Usage: "Owner/admin processing reason"}}}, {Name: "group.code", Use: "code", Short: "Inspect or manage group join codes", Phase: "phase5", Implemented: false}, {Name: "group.code.get", Use: "get", Short: "Show group join code status", Phase: "phase5", Implemented: false, Handler: "stub", Outputs: []string{"json", "pretty", "table"}, Flags: []FlagSpec{{Name: "group", Type: "string", Usage: "Group DID", Required: true}}}, {Name: "group.code.refresh", Use: "refresh", Short: "Rotate the current group join code", Phase: "phase5", Implemented: false, Handler: "stub", SideEffect: true, Outputs: []string{"json", "pretty"}, Flags: []FlagSpec{{Name: "group", Type: "string", Usage: "Group DID", Required: true}}}, diff --git a/internal/doctor/doctor.go b/internal/doctor/doctor.go index c04810d..58cbe81 100644 --- a/internal/doctor/doctor.go +++ b/internal/doctor/doctor.go @@ -2,14 +2,18 @@ package doctor import ( "context" + "database/sql" "errors" + "fmt" "os" "path/filepath" "strings" + "time" "github.com/agentconnect/awiki-cli/internal/buildinfo" "github.com/agentconnect/awiki-cli/internal/config" "github.com/agentconnect/awiki-cli/internal/identity" + "github.com/agentconnect/awiki-cli/internal/message" runtimecfg "github.com/agentconnect/awiki-cli/internal/runtime" listenerrt "github.com/agentconnect/awiki-cli/internal/runtime/listener" "github.com/agentconnect/awiki-cli/internal/store" @@ -45,6 +49,7 @@ func Run(resolved *config.Resolved) Report { runtimeCheck(resolved), identityStoreCheck(resolved), sqliteCheck(resolved), + anpMLSCheck(resolved), upgradeStateCheck(resolved), legacyCheck(resolved), } @@ -70,6 +75,345 @@ func Run(resolved *config.Resolved) Report { return Report{Checks: checks, Summary: summary, Counts: counts} } +func anpMLSCheck(resolved *config.Resolved) Check { + provider := message.NewDefaultMLSExecProvider(resolved) + binary, resolveErr := provider.ResolveBinaryPath() + state := inspectMLSState(resolved, provider.DataDir) + details := map[string]any{ + "binary": binary, + "data_dir": provider.DataDir, + "env_override": message.ANPMLSBinaryEnv, + "plain_unaffected": true, + "resolve_error": errorString(resolveErr), + "remediation": anpMLSRemediation(resolveErr, nil, nil, state), + "data_dir_status": state.DataDirStatus, + "data_dir_exists": state.DataDirExists, + "data_dir_error": state.DataDirError, + "state_db": state.StateDBPath, + "state_db_status": state.StateDBStatus, + "state_db_error": state.StateDBError, + "state_lock": state.StateLockPath, + "state_lock_status": state.StateLockStatus, + "state_lock_error": state.StateLockError, + "scoped_states": state.ScopedStates, + "scoped_state_count": state.ScopedStateCount, + "scoped_state_db_count": state.ScopedStateDBCount, + "scoped_state_lock_count": state.ScopedStateLockCount, + "scoped_state_warning_count": state.ScopedStateWarningCount, + "e2ee_group_count": state.E2EEGroupCount, + } + status := "ok" + summary := "anp-mls binary and compatibility probe are ready for group E2EE operations" + if resolveErr != nil { + status = "info" + summary = "anp-mls binary not found; plain messaging is unaffected, but group E2EE commands will fail" + details["remediation"] = anpMLSRemediation(resolveErr, nil, nil, state) + return Check{Name: "anp_mls", Status: status, Summary: summary, Details: details} + } + + ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second) + defer cancel() + versionInfo, probeErr := provider.ProbeVersion(ctx) + details["version"] = versionInfo + details["probe_error"] = errorString(probeErr) + compatErr := anpMLSCompatibilityError(versionInfo) + details["compatibility_error"] = errorString(compatErr) + details["remediation"] = anpMLSRemediation(resolveErr, probeErr, compatErr, state) + if probeErr != nil { + status = "warn" + summary = "anp-mls binary is present but the version compatibility probe failed" + } else if compatErr != nil { + status = "warn" + summary = "anp-mls binary version is not compatible with this awiki-cli build" + } else if state.HasWarning() { + status = "warn" + summary = "anp-mls binary is compatible but MLS state needs attention" + } + return Check{Name: "anp_mls", Status: status, Summary: summary, Details: details} +} + +type mlsStateInspection struct { + DataDirExists bool + DataDirStatus string + DataDirError string + StateDBPath string + StateDBStatus string + StateDBError string + StateLockPath string + StateLockStatus string + StateLockError string + ScopedStates []mlsScopedStateInspection + ScopedStateCount int + ScopedStateDBCount int + ScopedStateLockCount int + ScopedStateWarningCount int + E2EEGroupCount int +} + +type mlsScopedStateInspection struct { + AgentKey string `json:"agent_key"` + DeviceID string `json:"device_id"` + Dir string `json:"dir"` + StateDBPath string `json:"state_db"` + StateDBStatus string `json:"state_db_status"` + StateDBError string `json:"state_db_error,omitempty"` + StateLockPath string `json:"state_lock"` + StateLockStatus string `json:"state_lock_status"` + StateLockError string `json:"state_lock_error,omitempty"` +} + +func (s mlsStateInspection) HasWarning() bool { + return strings.HasPrefix(s.DataDirStatus, "warn") || strings.HasPrefix(s.StateDBStatus, "warn") || strings.HasPrefix(s.StateLockStatus, "warn") || s.ScopedStateWarningCount > 0 +} + +func inspectMLSState(resolved *config.Resolved, dataDir string) mlsStateInspection { + state := mlsStateInspection{ + DataDirStatus: "missing", + StateDBPath: filepath.Join(dataDir, "state.db"), + StateDBStatus: "missing", + StateLockPath: filepath.Join(dataDir, "state.lock"), + StateLockStatus: "missing", + } + if strings.TrimSpace(dataDir) == "" { + state.DataDirStatus = "not_configured" + state.StateDBPath = "" + state.StateLockPath = "" + return state + } + state.E2EEGroupCount = cachedGroupE2EECount(resolved) + if info, err := os.Stat(dataDir); err != nil { + if os.IsNotExist(err) { + if state.E2EEGroupCount > 0 { + state.DataDirStatus = "warn_missing_with_cached_groups" + } else { + state.DataDirStatus = "missing" + } + } else { + state.DataDirStatus = "warn_stat_failed" + state.DataDirError = err.Error() + } + return state + } else if !info.IsDir() { + state.DataDirStatus = "warn_not_directory" + return state + } + state.DataDirExists = true + state.DataDirStatus = "ok" + if err := canReadDir(dataDir); err != nil { + state.DataDirStatus = "warn_not_readable" + state.DataDirError = err.Error() + } else if err := canWriteDir(dataDir); err != nil { + state.DataDirStatus = "warn_not_writable" + state.DataDirError = err.Error() + } + + state.ScopedStates = inspectScopedMLSStates(dataDir) + state.ScopedStateCount = len(state.ScopedStates) + for _, scoped := range state.ScopedStates { + if scoped.StateDBStatus == "ok" { + state.ScopedStateDBCount++ + } + if scoped.StateLockStatus != "missing" { + state.ScopedStateLockCount++ + } + if strings.HasPrefix(scoped.StateDBStatus, "warn") || strings.HasPrefix(scoped.StateLockStatus, "warn") { + state.ScopedStateWarningCount++ + } + } + + state.StateDBStatus, state.StateDBError = inspectMLSStateDB(state.StateDBPath) + if state.StateDBStatus == "missing" && state.E2EEGroupCount > 0 && state.ScopedStateDBCount == 0 { + state.StateDBStatus = "warn_missing_with_cached_groups" + } + + state.StateLockStatus, state.StateLockError = inspectMLSLock(state.StateLockPath) + return state +} + +func inspectScopedMLSStates(dataDir string) []mlsScopedStateInspection { + agentsDir := filepath.Join(dataDir, "agents") + agentEntries, err := os.ReadDir(agentsDir) + if err != nil { + return nil + } + states := []mlsScopedStateInspection{} + for _, agentEntry := range agentEntries { + if !agentEntry.IsDir() { + continue + } + agentKey := agentEntry.Name() + agentDir := filepath.Join(agentsDir, agentKey) + deviceEntries, err := os.ReadDir(agentDir) + if err != nil { + states = append(states, mlsScopedStateInspection{ + AgentKey: agentKey, + Dir: agentDir, + StateDBStatus: "warn_read_devices_failed", + StateDBError: err.Error(), + StateLockStatus: "missing", + }) + continue + } + for _, deviceEntry := range deviceEntries { + if !deviceEntry.IsDir() { + continue + } + deviceID := deviceEntry.Name() + dir := filepath.Join(agentDir, deviceID) + dbPath := filepath.Join(dir, "state.db") + lockPath := filepath.Join(dir, "state.lock") + dbStatus, dbErr := inspectMLSStateDB(dbPath) + lockStatus, lockErr := inspectMLSLock(lockPath) + states = append(states, mlsScopedStateInspection{ + AgentKey: agentKey, + DeviceID: deviceID, + Dir: dir, + StateDBPath: dbPath, + StateDBStatus: dbStatus, + StateDBError: dbErr, + StateLockPath: lockPath, + StateLockStatus: lockStatus, + StateLockError: lockErr, + }) + } + } + return states +} + +func inspectMLSStateDB(path string) (string, string) { + if info, err := os.Stat(path); err != nil { + if os.IsNotExist(err) { + return "missing", "" + } + return "warn_stat_failed", err.Error() + } else if info.IsDir() { + return "warn_not_file", "" + } else if err := canReadFile(path); err != nil { + return "warn_not_readable", err.Error() + } + return "ok", "" +} + +func inspectMLSLock(path string) (string, string) { + if info, err := os.Stat(path); err != nil { + if os.IsNotExist(err) { + return "missing", "" + } + return "warn_stat_failed", err.Error() + } else if info.IsDir() { + return "warn_not_file", "" + } else if err := canReadFile(path); err != nil { + return "warn_not_readable", err.Error() + } else if time.Since(info.ModTime()) > 15*time.Minute { + return "warn_stale_candidate", "" + } + return "present_active_or_recent", "" +} + +func canReadDir(path string) error { + entries, err := os.ReadDir(path) + if err != nil { + return err + } + _ = entries + return nil +} + +func canWriteDir(path string) error { + probe, err := os.CreateTemp(path, ".awiki-cli-doctor-*") + if err != nil { + return err + } + name := probe.Name() + if err := probe.Close(); err != nil { + _ = os.Remove(name) + return err + } + return os.Remove(name) +} + +func canReadFile(path string) error { + file, err := os.Open(path) + if err != nil { + return err + } + return file.Close() +} + +func cachedGroupE2EECount(resolved *config.Resolved) int { + if resolved == nil || strings.TrimSpace(resolved.Paths.DatabaseFile) == "" || !pathExists(resolved.Paths.DatabaseFile) { + return 0 + } + db, err := store.OpenReadOnly(resolved.Paths.DatabaseFile) + if err != nil { + return 0 + } + defer db.Close() + var tableCount int + if err := db.QueryRow(`SELECT COUNT(*) FROM sqlite_master WHERE type = 'table' AND name = 'groups'`).Scan(&tableCount); err != nil || tableCount == 0 { + return 0 + } + var count sql.NullInt64 + if err := db.QueryRow(`SELECT COUNT(*) FROM groups WHERE metadata LIKE ?`, "%group-e2ee%").Scan(&count); err != nil { + return 0 + } + if !count.Valid { + return 0 + } + return int(count.Int64) +} + +func anpMLSCompatibilityError(info *message.MLSVersionInfo) error { + if info == nil { + return fmt.Errorf("missing version info") + } + if info.APIVersion != "anp-mls/v1" { + return fmt.Errorf("api_version %q is not supported; want anp-mls/v1", info.APIVersion) + } + if info.BinaryName != "anp-mls" { + return fmt.Errorf("binary_name %q is not supported; want anp-mls", info.BinaryName) + } + if !containsString(info.SupportedCommands, "system version") { + return fmt.Errorf("supported_commands does not include system version") + } + return nil +} + +func containsString(values []string, want string) bool { + for _, value := range values { + if strings.EqualFold(strings.TrimSpace(value), want) { + return true + } + } + return false +} + +func anpMLSRemediation(resolveErr error, probeErr error, compatErr error, state mlsStateInspection) string { + switch { + case resolveErr != nil: + return "Build anp-mls from ../anp/anp/rust, put it next to release artifacts or on PATH, or set AWIKI_ANP_MLS_BINARY to the absolute binary path. Plain messaging does not require anp-mls." + case probeErr != nil: + return "Install a current anp-mls build that supports `anp-mls system version --json-in -`; rebuild from ../anp/anp/rust if this probe fails." + case compatErr != nil: + return "Replace anp-mls with a build that reports api_version anp-mls/v1, binary_name anp-mls, and supported command `system version`." + case state.DataDirStatus == "warn_not_writable" || state.DataDirStatus == "warn_not_readable": + return "Fix permissions on the MLS data directory or move the workspace with AWIKI_CLI_WORKSPACE_HOME_DIR." + case state.StateDBStatus == "warn_missing_with_cached_groups": + return "The business database has cached group-e2ee groups but no root or agent/device-scoped MLS state.db was found; restore the MLS data directory from backup before sending encrypted group messages." + case strings.HasPrefix(state.StateLockStatus, "warn") || state.ScopedStateWarningCount > 0: + return "If no anp-mls process is running, inspect root and agent/device-scoped state.lock files, then remove stale locks only after backing up the MLS data directory." + default: + return "No action required." + } +} + +func errorString(err error) string { + if err == nil { + return "" + } + return err.Error() +} + func buildCheck(resolved *config.Resolved) Check { info := buildinfo.Current() status := "ok" diff --git a/internal/doctor/doctor_test.go b/internal/doctor/doctor_test.go index 4b3875f..01f6358 100644 --- a/internal/doctor/doctor_test.go +++ b/internal/doctor/doctor_test.go @@ -2,14 +2,18 @@ package doctor import ( "context" + "crypto/sha256" + "encoding/base64" "os" "path/filepath" + "strings" "testing" "time" "github.com/agentconnect/awiki-cli/internal/buildinfo" appconfig "github.com/agentconnect/awiki-cli/internal/config" "github.com/agentconnect/awiki-cli/internal/identity" + "github.com/agentconnect/awiki-cli/internal/message" runtimecfg "github.com/agentconnect/awiki-cli/internal/runtime" "github.com/agentconnect/awiki-cli/internal/store" "github.com/agentconnect/awiki-cli/internal/upgrade" @@ -42,7 +46,7 @@ func TestRunReturnsStableReportContracts(t *testing.T) { OK: 4, Warn: 2, Error: 0, - Info: 3, + Info: 4, }, wantStatuses: map[string]string{ "build": "ok", @@ -52,6 +56,7 @@ func TestRunReturnsStableReportContracts(t *testing.T) { "runtime": "ok", "identity_store": "warn", "sqlite": "info", + "anp_mls": "info", "workspace_upgrade": "ok", "legacy_paths": "info", }, @@ -120,7 +125,7 @@ INSERT INTO contact_handle_bindings ( OK: 8, Warn: 0, Error: 0, - Info: 1, + Info: 2, }, wantStatuses: map[string]string{ "build": "ok", @@ -130,6 +135,7 @@ INSERT INTO contact_handle_bindings ( "runtime": "ok", "identity_store": "ok", "sqlite": "ok", + "anp_mls": "info", "workspace_upgrade": "ok", "legacy_paths": "info", }, @@ -176,7 +182,7 @@ INSERT INTO contact_handle_bindings ( OK: 2, Warn: 2, Error: 2, - Info: 3, + Info: 4, }, wantStatuses: map[string]string{ "build": "ok", @@ -186,6 +192,7 @@ INSERT INTO contact_handle_bindings ( "runtime": "ok", "identity_store": "warn", "sqlite": "info", + "anp_mls": "info", "workspace_upgrade": "warn", "legacy_paths": "info", }, @@ -247,6 +254,8 @@ func resolveDoctorConfigForWorkspace(t *testing.T, workspace string, keepEnvHits t.Setenv("HOME", home) t.Setenv("USERPROFILE", home) t.Setenv("AWIKI_CLI_WORKSPACE_HOME_DIR", workspace) + t.Setenv("AWIKI_ANP_MLS_BINARY", "") + t.Setenv("PATH", t.TempDir()) resolved, err := appconfig.Resolve(appconfig.Overrides{}) if err != nil { t.Fatalf("Resolve() error = %v", err) @@ -270,3 +279,220 @@ func checkByName(t *testing.T, report Report, name string) Check { t.Fatalf("check %q not found", name) return Check{} } + +func TestANPMLSDoctorCompatibilityAndStateDiagnostics(t *testing.T) { + resolved := resolveDoctorConfig(t, false) + binDir := t.TempDir() + writeFakeANPMLS(t, binDir, `{"ok":true,"api_version":"anp-mls/v1","request_id":"doctor-system-version","result":{"api_version":"anp-mls/v1","binary_name":"anp-mls","binary_version":"test","supported_commands":["system version","key-package generate","group create"]}}`) + t.Setenv("PATH", binDir) + mlsDir := filepath.Join(resolved.Paths.WorkspaceHomeDir, "mls") + if err := os.MkdirAll(mlsDir, 0o700); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(mlsDir, "state.db"), []byte("sqlite placeholder"), 0o600); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(mlsDir, "state.lock"), []byte("lock"), 0o600); err != nil { + t.Fatal(err) + } + + report := Run(resolved) + check := checkByName(t, report, "anp_mls") + if check.Status != "ok" { + t.Fatalf("anp_mls status = %q, want ok; details=%#v", check.Status, check.Details) + } + if check.Details["data_dir_status"] != "ok" || check.Details["state_db_status"] != "ok" { + t.Fatalf("unexpected MLS state details: %#v", check.Details) + } + version, ok := check.Details["version"].(*message.MLSVersionInfo) + if !ok || version.BinaryName != "anp-mls" { + t.Fatalf("version detail = %#v", check.Details["version"]) + } + if check.Details["remediation"] != "No action required." { + t.Fatalf("remediation = %#v", check.Details["remediation"]) + } +} + +func TestANPMLSDoctorVersionMismatchIsActionableWarning(t *testing.T) { + resolved := resolveDoctorConfig(t, false) + binDir := t.TempDir() + writeFakeANPMLS(t, binDir, `{"ok":true,"api_version":"anp-mls/v0","request_id":"doctor-system-version","result":{"api_version":"anp-mls/v0","binary_name":"anp-mls","binary_version":"old","supported_commands":["system version"]}}`) + t.Setenv("PATH", binDir) + + report := Run(resolved) + check := checkByName(t, report, "anp_mls") + if check.Status != "warn" { + t.Fatalf("anp_mls status = %q, want warn; details=%#v", check.Status, check.Details) + } + if got := check.Details["compatibility_error"]; !strings.Contains(got.(string), "api_version") { + t.Fatalf("compatibility_error = %#v", got) + } + if got := check.Details["remediation"].(string); !strings.Contains(got, "api_version anp-mls/v1") { + t.Fatalf("remediation = %q", got) + } +} + +func TestANPMLSDoctorWarnsWhenCachedE2EEGroupsHaveNoMLSState(t *testing.T) { + resolved := resolveDoctorConfig(t, false) + binDir := t.TempDir() + writeFakeANPMLS(t, binDir, `{"ok":true,"api_version":"anp-mls/v1","request_id":"doctor-system-version","result":{"api_version":"anp-mls/v1","binary_name":"anp-mls","binary_version":"test","supported_commands":["system version"]}}`) + t.Setenv("PATH", binDir) + db, err := store.Open(resolved.Paths) + if err != nil { + t.Fatal(err) + } + defer db.Close() + if err := store.EnsureSchema(context.Background(), db); err != nil { + t.Fatal(err) + } + if err := store.UpsertGroup(context.Background(), db, store.GroupRecord{ + OwnerDID: "did:wba:alice.example", + GroupID: "group-1", + Name: "Secret group", + Metadata: `{"message_security_profile":"group-e2ee"}`, + CredentialName: "alice", + }); err != nil { + t.Fatal(err) + } + + report := Run(resolved) + check := checkByName(t, report, "anp_mls") + if check.Status != "warn" { + t.Fatalf("anp_mls status = %q, want warn; details=%#v", check.Status, check.Details) + } + if check.Details["e2ee_group_count"] != 1 { + t.Fatalf("e2ee_group_count = %#v, want 1", check.Details["e2ee_group_count"]) + } + if check.Details["data_dir_status"] != "warn_missing_with_cached_groups" { + t.Fatalf("data_dir_status = %#v", check.Details["data_dir_status"]) + } +} + +func TestANPMLSDoctorAcceptsAgentDeviceScopedState(t *testing.T) { + resolved := resolveDoctorConfig(t, false) + binDir := t.TempDir() + writeFakeANPMLS(t, binDir, `{"ok":true,"api_version":"anp-mls/v1","request_id":"doctor-system-version","result":{"api_version":"anp-mls/v1","binary_name":"anp-mls","binary_version":"test","supported_commands":["system version"]}}`) + t.Setenv("PATH", binDir) + db, err := store.Open(resolved.Paths) + if err != nil { + t.Fatal(err) + } + defer db.Close() + if err := store.EnsureSchema(context.Background(), db); err != nil { + t.Fatal(err) + } + if err := store.UpsertGroup(context.Background(), db, store.GroupRecord{ + OwnerDID: "did:wba:alice.example", + GroupID: "group-1", + Name: "Secret group", + Metadata: `{"message_security_profile":"group-e2ee"}`, + CredentialName: "alice", + }); err != nil { + t.Fatal(err) + } + scopedDir := filepath.Join( + resolved.Paths.WorkspaceHomeDir, + "mls", + "agents", + testMLSAgentKey("did:wba:alice.example"), + "laptop", + ) + if err := os.MkdirAll(scopedDir, 0o700); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(scopedDir, "state.db"), []byte("sqlite placeholder"), 0o600); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(scopedDir, "state.lock"), []byte("lock"), 0o600); err != nil { + t.Fatal(err) + } + + report := Run(resolved) + check := checkByName(t, report, "anp_mls") + if check.Status != "ok" { + t.Fatalf("anp_mls status = %q, want ok; details=%#v", check.Status, check.Details) + } + if check.Details["state_db_status"] != "missing" { + t.Fatalf("root state_db_status = %#v, want missing when scoped state exists", check.Details["state_db_status"]) + } + if check.Details["scoped_state_db_count"] != 1 { + t.Fatalf("scoped_state_db_count = %#v, want 1", check.Details["scoped_state_db_count"]) + } + if check.Details["scoped_state_lock_count"] != 1 { + t.Fatalf("scoped_state_lock_count = %#v, want 1", check.Details["scoped_state_lock_count"]) + } + scoped, ok := check.Details["scoped_states"].([]mlsScopedStateInspection) + if !ok || len(scoped) != 1 { + t.Fatalf("scoped_states = %#v, want one scoped state", check.Details["scoped_states"]) + } + if scoped[0].AgentKey != testMLSAgentKey("did:wba:alice.example") || scoped[0].DeviceID != "laptop" { + t.Fatalf("scoped state identity = %#v", scoped[0]) + } + if check.Details["remediation"] != "No action required." { + t.Fatalf("remediation = %#v", check.Details["remediation"]) + } +} + +func TestANPMLSDoctorWarnsOnStaleScopedLock(t *testing.T) { + resolved := resolveDoctorConfig(t, false) + binDir := t.TempDir() + writeFakeANPMLS(t, binDir, `{"ok":true,"api_version":"anp-mls/v1","request_id":"doctor-system-version","result":{"api_version":"anp-mls/v1","binary_name":"anp-mls","binary_version":"test","supported_commands":["system version"]}}`) + t.Setenv("PATH", binDir) + scopedDir := filepath.Join( + resolved.Paths.WorkspaceHomeDir, + "mls", + "agents", + testMLSAgentKey("did:wba:alice.example"), + "default", + ) + if err := os.MkdirAll(scopedDir, 0o700); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(scopedDir, "state.db"), []byte("sqlite placeholder"), 0o600); err != nil { + t.Fatal(err) + } + lockPath := filepath.Join(scopedDir, "state.lock") + if err := os.WriteFile(lockPath, []byte("lock"), 0o600); err != nil { + t.Fatal(err) + } + old := time.Now().Add(-30 * time.Minute) + if err := os.Chtimes(lockPath, old, old); err != nil { + t.Fatal(err) + } + + report := Run(resolved) + check := checkByName(t, report, "anp_mls") + if check.Status != "warn" { + t.Fatalf("anp_mls status = %q, want warn; details=%#v", check.Status, check.Details) + } + if check.Details["scoped_state_warning_count"] != 1 { + t.Fatalf("scoped_state_warning_count = %#v, want 1", check.Details["scoped_state_warning_count"]) + } + scoped := check.Details["scoped_states"].([]mlsScopedStateInspection) + if scoped[0].StateLockStatus != "warn_stale_candidate" { + t.Fatalf("scoped state lock status = %#v, want stale warning", scoped[0]) + } + if got := check.Details["remediation"].(string); !strings.Contains(got, "agent/device-scoped state.lock") { + t.Fatalf("remediation = %q", got) + } +} + +func writeFakeANPMLS(t *testing.T, dir string, response string) string { + t.Helper() + if os.PathSeparator == ';' { + t.Skip("shell-script fake anp-mls is only used on Unix-like test hosts") + } + path := filepath.Join(dir, "anp-mls") + script := "#!/bin/sh\n" + + "if [ \"$1 $2 $3 $4\" != \"system version --json-in -\" ]; then echo unexpected args >&2; exit 2; fi\n" + + "printf '%s' '" + strings.ReplaceAll(response, "'", "'\\''") + "'\n" + if err := os.WriteFile(path, []byte(script), 0o755); err != nil { + t.Fatal(err) + } + return path +} + +func testMLSAgentKey(agentDID string) string { + sum := sha256.Sum256([]byte(agentDID)) + return base64.RawURLEncoding.EncodeToString(sum[:])[:24] +} diff --git a/internal/message/group_e2ee_provider.go b/internal/message/group_e2ee_provider.go new file mode 100644 index 0000000..e0c108b --- /dev/null +++ b/internal/message/group_e2ee_provider.go @@ -0,0 +1,454 @@ +package message + +import ( + "bytes" + "context" + "crypto/sha256" + "encoding/base64" + "encoding/json" + "fmt" + "os" + "os/exec" + "path/filepath" + "runtime" + "strings" + "time" + + appconfig "github.com/agentconnect/awiki-cli/internal/config" +) + +const ( + GroupE2EEProfile = "anp.group.e2ee.v1" + GroupE2EESecurityProfile = "group-e2ee" + GroupE2EETransportProfile = "transport-protected" + GroupE2EEContractArtifactMode = "contract-test" + DefaultANPMLSBinary = "anp-mls" + ANPMLSBinaryEnv = "AWIKI_ANP_MLS_BINARY" +) + +type MLSRequest struct { + APIVersion string `json:"api_version"` + RequestID string `json:"request_id"` + AgentDID string `json:"agent_did,omitempty"` + DeviceID string `json:"device_id,omitempty"` + ContractTestEnabled bool `json:"contract_test_enabled,omitempty"` + Params map[string]any `json:"params"` +} + +type MLSResponse struct { + OK bool `json:"ok"` + APIVersion string `json:"api_version"` + RequestID string `json:"request_id"` + Result map[string]any `json:"result,omitempty"` + Error *MLSError `json:"error,omitempty"` +} + +type MLSError struct { + Code string `json:"code"` + Message string `json:"message"` +} + +// MLSVersionInfo is the machine-readable compatibility contract returned by +// `anp-mls system version --json-in -`. +type MLSVersionInfo struct { + APIVersion string `json:"api_version"` + BinaryName string `json:"binary_name"` + BinaryVersion string `json:"binary_version,omitempty"` + BuildVersion string `json:"build_version,omitempty"` + SupportedCommands []string `json:"supported_commands"` +} + +type MLSCommandRunner interface { + Run(ctx context.Context, binary string, args []string, stdin []byte) (stdout []byte, stderr []byte, err error) +} + +type OSMLSCommandRunner struct{} + +func (OSMLSCommandRunner) Run(ctx context.Context, binary string, args []string, stdin []byte) ([]byte, []byte, error) { + cmd := exec.CommandContext(ctx, binary, args...) + cmd.Stdin = bytes.NewReader(stdin) + var stdout bytes.Buffer + var stderr bytes.Buffer + cmd.Stdout = &stdout + cmd.Stderr = &stderr + err := cmd.Run() + return stdout.Bytes(), stderr.Bytes(), err +} + +type MLSExecProvider struct { + BinaryPath string + DataDir string + Timeout time.Duration + Runner MLSCommandRunner +} + +func NewDefaultMLSExecProvider(resolved *appconfig.Resolved) MLSExecProvider { + return MLSExecProvider{ + DataDir: DefaultMLSDataDir(resolved), + Timeout: 15 * time.Second, + } +} + +func DefaultMLSDataDir(resolved *appconfig.Resolved) string { + if resolved == nil || resolved.Paths.WorkspaceHomeDir == "" { + return filepath.Join(".awiki-cli", "mls") + } + return filepath.Join(resolved.Paths.WorkspaceHomeDir, "mls") +} + +func (p MLSExecProvider) ResolveBinaryPath() (string, error) { + candidates := make([]string, 0, 3) + if envPath := strings.TrimSpace(os.Getenv(ANPMLSBinaryEnv)); envPath != "" { + candidates = append(candidates, envPath) + } + if injected := strings.TrimSpace(p.BinaryPath); injected != "" { + candidates = append(candidates, injected) + } + candidates = append(candidates, DefaultANPMLSBinary) + + seen := map[string]struct{}{} + for _, candidate := range candidates { + if _, ok := seen[candidate]; ok { + continue + } + seen[candidate] = struct{}{} + if filepath.IsAbs(candidate) || strings.ContainsRune(candidate, filepath.Separator) { + if info, err := os.Stat(candidate); err == nil && isExecutableFile(info) { + return candidate, nil + } + continue + } + if resolved, err := exec.LookPath(candidate); err == nil { + return resolved, nil + } + } + return "", fmt.Errorf( + "unable to locate anp-mls binary (checked %s, injected path, then PATH). Set %s to an absolute anp-mls path, build/install anp-mls, or run `awiki-cli doctor` for diagnostics", + ANPMLSBinaryEnv, + ANPMLSBinaryEnv, + ) +} + +func isExecutableFile(info os.FileInfo) bool { + if info == nil || info.IsDir() { + return false + } + if runtime.GOOS == "windows" { + return true + } + return info.Mode()&0o111 != 0 +} + +func (p MLSExecProvider) ProbeVersion(ctx context.Context) (*MLSVersionInfo, error) { + timeout := p.Timeout + if timeout <= 0 { + timeout = 15 * time.Second + } + runner := p.Runner + if runner == nil { + runner = OSMLSCommandRunner{} + } + binary := strings.TrimSpace(p.BinaryPath) + if binary == "" || p.Runner == nil { + resolvedBinary, err := p.ResolveBinaryPath() + if err != nil { + return nil, err + } + binary = resolvedBinary + } + req := MLSRequest{ + APIVersion: "anp-mls/v1", + RequestID: "doctor-system-version", + Params: map[string]any{}, + } + body, err := json.Marshal(req) + if err != nil { + return nil, err + } + ctx, cancel := context.WithTimeout(ctx, timeout) + defer cancel() + stdout, stderr, err := runner.Run(ctx, binary, []string{"system", "version", "--json-in", "-"}, body) + if err != nil && len(stdout) == 0 { + return nil, fmt.Errorf("anp-mls version probe failed: %w: %s", err, string(stderr)) + } + var resp MLSResponse + if decodeErr := json.Unmarshal(stdout, &resp); decodeErr != nil { + return nil, fmt.Errorf("decode anp-mls version response: %w: stderr=%s", decodeErr, string(stderr)) + } + if !resp.OK { + if resp.Error != nil { + return nil, fmt.Errorf("anp-mls version probe error %s: %s", resp.Error.Code, resp.Error.Message) + } + return nil, fmt.Errorf("anp-mls version probe returned ok=false") + } + info, err := versionInfoFromResponse(resp) + if err != nil { + return nil, err + } + return info, nil +} + +func versionInfoFromResponse(resp MLSResponse) (*MLSVersionInfo, error) { + resultBytes, err := json.Marshal(resp.Result) + if err != nil { + return nil, err + } + var info MLSVersionInfo + if err := json.Unmarshal(resultBytes, &info); err != nil { + return nil, fmt.Errorf("decode anp-mls version result: %w", err) + } + if info.APIVersion == "" { + info.APIVersion = resp.APIVersion + } + if strings.TrimSpace(info.APIVersion) == "" { + return nil, fmt.Errorf("anp-mls version response missing api_version") + } + if strings.TrimSpace(info.BinaryName) == "" { + return nil, fmt.Errorf("anp-mls version response missing binary_name") + } + if strings.TrimSpace(info.BinaryVersion) == "" && strings.TrimSpace(info.BuildVersion) == "" { + return nil, fmt.Errorf("anp-mls version response missing binary_version/build_version") + } + if len(info.SupportedCommands) == 0 { + return nil, fmt.Errorf("anp-mls version response missing supported_commands") + } + return &info, nil +} + +func (p MLSExecProvider) Call(ctx context.Context, domain string, action string, req MLSRequest) (*MLSResponse, error) { + timeout := p.Timeout + if timeout <= 0 { + timeout = 15 * time.Second + } + runner := p.Runner + if runner == nil { + runner = OSMLSCommandRunner{} + } + binary := strings.TrimSpace(p.BinaryPath) + if binary == "" || p.Runner == nil { + resolvedBinary, err := p.ResolveBinaryPath() + if err != nil { + return nil, err + } + binary = resolvedBinary + } + body, err := json.Marshal(req) + if err != nil { + return nil, err + } + dataDir := p.effectiveDataDir(req) + if dataDir != "" { + if err := os.MkdirAll(dataDir, 0o700); err != nil { + return nil, fmt.Errorf("prepare anp-mls data dir %s: %w", dataDir, err) + } + } + ctx, cancel := context.WithTimeout(ctx, timeout) + defer cancel() + args := []string{domain, action, "--json-in", "-"} + if dataDir != "" { + args = append(args, "--data-dir", dataDir) + } + stdout, stderr, err := runner.Run(ctx, binary, args, body) + if err != nil && len(stdout) == 0 { + return nil, fmt.Errorf("anp-mls exec failed: %w: %s", err, string(stderr)) + } + var resp MLSResponse + if decodeErr := json.Unmarshal(stdout, &resp); decodeErr != nil { + return nil, fmt.Errorf("decode anp-mls response: %w: stderr=%s", decodeErr, string(stderr)) + } + if !resp.OK { + if resp.Error != nil { + return &resp, fmt.Errorf("anp-mls error %s: %s", resp.Error.Code, resp.Error.Message) + } + return &resp, fmt.Errorf("anp-mls returned ok=false") + } + return &resp, nil +} + +func (p MLSExecProvider) effectiveDataDir(req MLSRequest) string { + baseDir := strings.TrimSpace(p.DataDir) + if baseDir == "" { + return "" + } + agentDID := strings.TrimSpace(req.AgentDID) + if agentDID == "" { + for _, key := range []string{"agent_did", "owner_did", "actor_did", "sender_did", "recipient_did"} { + if value := stringFromAny(req.Params[key]); value != "" { + agentDID = value + break + } + } + } + if agentDID == "" { + return baseDir + } + deviceID := strings.TrimSpace(req.DeviceID) + if deviceID == "" { + deviceID = stringFromAny(req.Params["device_id"]) + } + if deviceID == "" { + deviceID = "default" + } + return filepath.Join(baseDir, "agents", mlsAgentKey(agentDID), safeMLSPathComponent(deviceID)) +} + +func (p MLSExecProvider) candidateDeviceIDs(agentDID string) []string { + agentDID = strings.TrimSpace(agentDID) + if agentDID == "" { + return []string{"default"} + } + candidates := []string{"default"} + baseDir := strings.TrimSpace(p.DataDir) + if baseDir == "" { + return candidates + } + entries, err := os.ReadDir(filepath.Join(baseDir, "agents", mlsAgentKey(agentDID))) + if err != nil { + return candidates + } + seen := map[string]struct{}{"default": {}} + for _, entry := range entries { + if !entry.IsDir() { + continue + } + deviceID := strings.TrimSpace(entry.Name()) + if deviceID == "" { + continue + } + if _, ok := seen[deviceID]; ok { + continue + } + seen[deviceID] = struct{}{} + candidates = append(candidates, deviceID) + } + return candidates +} + +func mlsAgentKey(agentDID string) string { + sum := sha256.Sum256([]byte(agentDID)) + return base64.RawURLEncoding.EncodeToString(sum[:])[:24] +} + +func safeMLSPathComponent(value string) string { + value = strings.TrimSpace(value) + if value == "" { + return "default" + } + var builder strings.Builder + for _, r := range value { + switch { + case r >= 'a' && r <= 'z', r >= 'A' && r <= 'Z', r >= '0' && r <= '9', r == '-', r == '_', r == '.': + builder.WriteRune(r) + default: + builder.WriteByte('_') + } + } + if builder.Len() == 0 { + return "default" + } + return builder.String() +} + +func (p MLSExecProvider) GenerateKeyPackage(ctx context.Context, req MLSRequest) (map[string]any, error) { + resp, err := p.Call(ctx, "key-package", "generate", req) + if err != nil { + return nil, err + } + return resp.Result, nil +} + +func (p MLSExecProvider) CreateGroup(ctx context.Context, req MLSRequest) (map[string]any, error) { + resp, err := p.Call(ctx, "group", "create", req) + if err != nil { + return nil, err + } + return resp.Result, nil +} + +func (p MLSExecProvider) AddMember(ctx context.Context, req MLSRequest) (map[string]any, error) { + resp, err := p.Call(ctx, "group", "add-member", req) + if err != nil { + return nil, err + } + return resp.Result, nil +} + +func (p MLSExecProvider) RemoveMember(ctx context.Context, req MLSRequest) (map[string]any, error) { + resp, err := p.Call(ctx, "group", "remove-member", req) + if err != nil { + return nil, err + } + return resp.Result, nil +} + +func (p MLSExecProvider) RecoverMemberPrepare(ctx context.Context, req MLSRequest) (map[string]any, error) { + resp, err := p.Call(ctx, "group", "recover-member-prepare", req) + if err != nil { + return nil, err + } + return resp.Result, nil +} + +func (p MLSExecProvider) LeaveGroup(ctx context.Context, req MLSRequest) (map[string]any, error) { + resp, err := p.Call(ctx, "group", "leave", req) + if err != nil { + return nil, err + } + return resp.Result, nil +} + +func (p MLSExecProvider) CommitFinalize(ctx context.Context, req MLSRequest) (map[string]any, error) { + resp, err := p.Call(ctx, "group", "commit-finalize", req) + if err != nil { + return nil, err + } + return resp.Result, nil +} + +func (p MLSExecProvider) CommitAbort(ctx context.Context, req MLSRequest) (map[string]any, error) { + resp, err := p.Call(ctx, "group", "commit-abort", req) + if err != nil { + return nil, err + } + return resp.Result, nil +} + +func (p MLSExecProvider) ProcessWelcome(ctx context.Context, req MLSRequest) (map[string]any, error) { + resp, err := p.Call(ctx, "welcome", "process", req) + if err != nil { + return nil, err + } + return resp.Result, nil +} + +func (p MLSExecProvider) ProcessCommit(ctx context.Context, req MLSRequest) (map[string]any, error) { + resp, err := p.Call(ctx, "commit", "process", req) + if err != nil { + return nil, err + } + return resp.Result, nil +} + +func (p MLSExecProvider) Encrypt(ctx context.Context, req MLSRequest) (map[string]any, error) { + resp, err := p.Call(ctx, "message", "encrypt", req) + if err != nil { + return nil, err + } + return resp.Result, nil +} + +func (p MLSExecProvider) Decrypt(ctx context.Context, req MLSRequest) (map[string]any, error) { + resp, err := p.Call(ctx, "message", "decrypt", req) + if err != nil { + return nil, err + } + return resp.Result, nil +} + +func (p MLSExecProvider) Status(ctx context.Context, req MLSRequest) (map[string]any, error) { + resp, err := p.Call(ctx, "group", "status", req) + if err != nil { + return nil, err + } + return resp.Result, nil +} diff --git a/internal/message/group_e2ee_provider_test.go b/internal/message/group_e2ee_provider_test.go new file mode 100644 index 0000000..4d12c4e --- /dev/null +++ b/internal/message/group_e2ee_provider_test.go @@ -0,0 +1,235 @@ +package message + +import ( + "context" + "encoding/json" + "os" + "path/filepath" + "strings" + "testing" +) + +type recordingMLSRunner struct { + args []string + stdin []byte +} + +func TestMLSExecProviderBinaryDiscoveryOrder(t *testing.T) { + t.Setenv(ANPMLSBinaryEnv, "") + pathDir := t.TempDir() + pathBinary := filepath.Join(pathDir, "anp-mls") + if err := os.WriteFile(pathBinary, []byte("#!/bin/sh\n"), 0o755); err != nil { + t.Fatal(err) + } + t.Setenv("PATH", pathDir) + + injected := filepath.Join(t.TempDir(), "runtime-anp-mls") + if err := os.WriteFile(injected, []byte("#!/bin/sh\n"), 0o755); err != nil { + t.Fatal(err) + } + + provider := MLSExecProvider{BinaryPath: injected} + got, err := provider.ResolveBinaryPath() + if err != nil { + t.Fatal(err) + } + if got != injected { + t.Fatalf("ResolveBinaryPath injected = %q, want %q", got, injected) + } + + envBinary := filepath.Join(t.TempDir(), "env-anp-mls") + if err := os.WriteFile(envBinary, []byte("#!/bin/sh\n"), 0o755); err != nil { + t.Fatal(err) + } + t.Setenv(ANPMLSBinaryEnv, envBinary) + got, err = provider.ResolveBinaryPath() + if err != nil { + t.Fatal(err) + } + if got != envBinary { + t.Fatalf("ResolveBinaryPath env = %q, want %q", got, envBinary) + } + + t.Setenv(ANPMLSBinaryEnv, "") + provider.BinaryPath = "" + got, err = provider.ResolveBinaryPath() + if err != nil { + t.Fatal(err) + } + if got != pathBinary { + t.Fatalf("ResolveBinaryPath PATH = %q, want %q", got, pathBinary) + } +} + +func (r *recordingMLSRunner) Run(_ context.Context, _ string, args []string, stdin []byte) ([]byte, []byte, error) { + r.args = append([]string(nil), args...) + r.stdin = append([]byte(nil), stdin...) + if strings.Join(args, " ") == "system version --json-in -" { + return []byte(`{"ok":true,"api_version":"anp-mls/v1","request_id":"doctor-system-version","result":{"api_version":"anp-mls/v1","binary_name":"anp-mls","binary_version":"test","supported_commands":["system version","message encrypt"]}}`), nil, nil + } + return []byte(`{"ok":true,"api_version":"anp-mls/v1","request_id":"req-1","result":{"non_cryptographic":true}}`), nil, nil +} + +func TestMLSExecProviderPassesPlaintextOnStdinNotArgv(t *testing.T) { + runner := &recordingMLSRunner{} + provider := MLSExecProvider{BinaryPath: "anp-mls", DataDir: t.TempDir(), Runner: runner} + _, err := provider.Call(context.Background(), "message", "encrypt", MLSRequest{ + APIVersion: "anp-mls/v1", + RequestID: "req-1", + AgentDID: "did:wba:example.com:users:alice:e1", + DeviceID: "device-1", + ContractTestEnabled: true, + Params: map[string]any{ + "agent_did": "did:wba:example.com:users:alice:e1", + "device_id": "device-1", + "application_plaintext": map[string]any{"text": "super secret"}, + }, + }) + if err != nil { + t.Fatal(err) + } + if strings.Contains(strings.Join(runner.args, " "), "super secret") { + t.Fatalf("plaintext leaked into argv: %#v", runner.args) + } + if !strings.Contains(string(runner.stdin), "super secret") { + t.Fatalf("plaintext request was not sent via stdin: %s", string(runner.stdin)) + } + var req MLSRequest + if err := json.Unmarshal(runner.stdin, &req); err != nil { + t.Fatal(err) + } + if !req.ContractTestEnabled { + t.Fatal("contract test flag not preserved") + } + if req.AgentDID == "" || req.Params["agent_did"] != req.AgentDID { + t.Fatalf("agent_did not preserved in envelope and params: %#v", req) + } + if req.DeviceID == "" || req.Params["device_id"] != req.DeviceID { + t.Fatalf("device_id not preserved in envelope and params: %#v", req) + } + if got := runner.args[len(runner.args)-2]; got != "--data-dir" { + t.Fatalf("args missing --data-dir before final value: %#v", runner.args) + } + dataDir := runner.args[len(runner.args)-1] + if !strings.Contains(dataDir, filepath.Join("agents")) || !strings.HasSuffix(dataDir, filepath.Join("agents", filepath.Base(filepath.Dir(dataDir)), "device-1")) { + t.Fatalf("data dir = %q, want agent-scoped device directory", dataDir) + } + if strings.Contains(dataDir, "did:wba") { + t.Fatalf("data dir leaked raw DID: %q", dataDir) + } +} + +func TestMLSExecProviderMembershipLifecycleUsesStableCommands(t *testing.T) { + cases := []struct { + name string + call func(context.Context, MLSExecProvider, MLSRequest) (map[string]any, error) + want string + }{ + {name: "remove", call: func(ctx context.Context, p MLSExecProvider, req MLSRequest) (map[string]any, error) { + return p.RemoveMember(ctx, req) + }, want: "group remove-member --json-in -"}, + {name: "leave", call: func(ctx context.Context, p MLSExecProvider, req MLSRequest) (map[string]any, error) { + return p.LeaveGroup(ctx, req) + }, want: "group leave --json-in -"}, + {name: "finalize", call: func(ctx context.Context, p MLSExecProvider, req MLSRequest) (map[string]any, error) { + return p.CommitFinalize(ctx, req) + }, want: "group commit-finalize --json-in -"}, + {name: "abort", call: func(ctx context.Context, p MLSExecProvider, req MLSRequest) (map[string]any, error) { + return p.CommitAbort(ctx, req) + }, want: "group commit-abort --json-in -"}, + {name: "commit process", call: func(ctx context.Context, p MLSExecProvider, req MLSRequest) (map[string]any, error) { + return p.ProcessCommit(ctx, req) + }, want: "commit process --json-in -"}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + runner := &recordingMLSRunner{} + provider := MLSExecProvider{BinaryPath: "anp-mls", Runner: runner} + _, err := tc.call(context.Background(), provider, MLSRequest{ + APIVersion: "anp-mls/v1", + RequestID: "req-" + tc.name, + AgentDID: "did:wba:example.com:users:alice:e1_alice", + Params: map[string]any{ + "agent_did": "did:wba:example.com:users:alice:e1_alice", + "group_did": "did:wba:example.com:groups:demo:e1_group", + "operation_id": "op-1", + }, + }) + if err != nil { + t.Fatalf("provider call error = %v", err) + } + if got := strings.Join(runner.args, " "); got != tc.want { + t.Fatalf("args = %q, want %q", got, tc.want) + } + }) + } +} + +func TestMLSExecProviderCandidateDeviceIDsScansAgentScopedState(t *testing.T) { + root := t.TempDir() + agentDID := "did:wba:example.com:users:bob:e1" + agentDir := filepath.Join(root, "agents", mlsAgentKey(agentDID)) + if err := os.MkdirAll(filepath.Join(agentDir, "bob-main"), 0o700); err != nil { + t.Fatal(err) + } + if err := os.MkdirAll(filepath.Join(agentDir, "bob.backup"), 0o700); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(agentDir, "not-a-device"), []byte("state"), 0o600); err != nil { + t.Fatal(err) + } + + got := MLSExecProvider{DataDir: root}.candidateDeviceIDs(agentDID) + want := map[string]bool{"default": true, "bob-main": true, "bob.backup": true} + if len(got) != len(want) { + t.Fatalf("candidateDeviceIDs() = %#v, want keys %#v", got, want) + } + for _, deviceID := range got { + if !want[deviceID] { + t.Fatalf("candidateDeviceIDs() returned unexpected device %q in %#v", deviceID, got) + } + } + if got[0] != "default" { + t.Fatalf("candidateDeviceIDs()[0] = %q, want default first", got[0]) + } +} + +func TestMLSExecProviderRejectsNonExecutablePath(t *testing.T) { + if os.PathSeparator == ';' { + t.Skip("Windows executable bit semantics differ") + } + t.Setenv(ANPMLSBinaryEnv, "") + nonExecutable := filepath.Join(t.TempDir(), "anp-mls") + if err := os.WriteFile(nonExecutable, []byte("#!/bin/sh\n"), 0o600); err != nil { + t.Fatal(err) + } + provider := MLSExecProvider{BinaryPath: nonExecutable} + if got, err := provider.ResolveBinaryPath(); err == nil { + t.Fatalf("ResolveBinaryPath() = %q, nil error; want non-executable path rejected", got) + } +} + +func TestMLSExecProviderProbeVersionUsesStableSystemContract(t *testing.T) { + runner := &recordingMLSRunner{} + provider := MLSExecProvider{BinaryPath: "anp-mls", DataDir: t.TempDir(), Runner: runner} + info, err := provider.ProbeVersion(context.Background()) + if err != nil { + t.Fatal(err) + } + if got := strings.Join(runner.args, " "); got != "system version --json-in -" { + t.Fatalf("ProbeVersion args = %q, want stable system version probe", got) + } + if strings.Contains(strings.Join(runner.args, " "), provider.DataDir) { + t.Fatalf("ProbeVersion should not require or expose data dir args: %#v", runner.args) + } + if info.APIVersion != "anp-mls/v1" || info.BinaryName != "anp-mls" || info.BinaryVersion == "" { + t.Fatalf("ProbeVersion info = %#v", info) + } + var req MLSRequest + if err := json.Unmarshal(runner.stdin, &req); err != nil { + t.Fatal(err) + } + if req.RequestID != "doctor-system-version" || req.Params == nil { + t.Fatalf("ProbeVersion stdin request = %#v", req) + } +} diff --git a/internal/message/group_e2ee_service.go b/internal/message/group_e2ee_service.go new file mode 100644 index 0000000..74cc2ec --- /dev/null +++ b/internal/message/group_e2ee_service.go @@ -0,0 +1,1741 @@ +package message + +import ( + "context" + "encoding/json" + "errors" + "fmt" + "net/http" + "strconv" + "strings" + + "github.com/agentconnect/awiki-cli/internal/anpsdk" + "github.com/agentconnect/awiki-cli/internal/identity" + "github.com/agentconnect/awiki-cli/internal/store" +) + +func (s *Service) publishGroupE2EEKeyPackage(ctx context.Context, record *identity.StoredIdentity, deviceID string, groupDID string, recovery bool, contractTest bool) (map[string]any, map[string]any, error) { + provider := s.groupMLSProvider() + if strings.TrimSpace(deviceID) == "" { + deviceID = "default" + } + groupDID = strings.TrimSpace(groupDID) + if recovery && groupDID == "" { + return nil, nil, fmt.Errorf("group DID is required when publishing a recovery KeyPackage") + } + params := map[string]any{ + "agent_did": record.DID, + "device_id": deviceID, + "owner_did": record.DID, + } + if recovery { + params["purpose"] = "recovery" + params["group_did"] = groupDID + } + packageResult, err := provider.GenerateKeyPackage(ctx, MLSRequest{ + APIVersion: "anp-mls/v1", + RequestID: "group-e2ee-key-package-" + generateOperationID(), + AgentDID: record.DID, + DeviceID: deviceID, + ContractTestEnabled: contractTest, + Params: params, + }) + if err != nil { + return nil, nil, err + } + packageResult = tagGroupKeyPackagePurpose(packageResult, groupDID, deviceID, recovery) + packageResult, err = signGroupKeyPackageDIDWBABinding(record, packageResult) + if err != nil { + return nil, nil, err + } + transport, _, err := s.httpTransport(record) + if err != nil { + return packageResult, nil, err + } + published, err := transport.PublishGroupE2EEKeyPackage(ctx, packageResult) + if err != nil { + return packageResult, nil, err + } + return packageResult, published, nil +} + +func tagGroupKeyPackagePurpose(packageResult map[string]any, groupDID string, deviceID string, recovery bool) map[string]any { + if !recovery { + return packageResult + } + tagged := cloneStringAnyMap(packageResult) + groupKeyPackage, _ := packageResult["group_key_package"].(map[string]any) + if len(groupKeyPackage) == 0 { + return tagged + } + taggedPackage := cloneStringAnyMap(groupKeyPackage) + taggedPackage["purpose"] = "recovery" + taggedPackage["group_did"] = groupDID + taggedPackage["device_id"] = defaultString(strings.TrimSpace(deviceID), "default") + tagged["group_key_package"] = taggedPackage + return tagged +} + +func signGroupKeyPackageDIDWBABinding(record *identity.StoredIdentity, packageResult map[string]any) (map[string]any, error) { + if record == nil { + return nil, fmt.Errorf("identity record is required") + } + if len(packageResult) == 0 { + return nil, fmt.Errorf("anp-mls key-package response is empty") + } + groupKeyPackage, ok := packageResult["group_key_package"].(map[string]any) + if !ok || len(groupKeyPackage) == 0 { + return nil, fmt.Errorf("anp-mls key-package response missing group_key_package") + } + binding, ok := groupKeyPackage["did_wba_binding"].(map[string]any) + if !ok || len(binding) == 0 { + return nil, fmt.Errorf("group_key_package.did_wba_binding is required") + } + ownerDID := stringFromAny(groupKeyPackage["owner_did"]) + if ownerDID == "" { + ownerDID = record.DID + } + if ownerDID != record.DID { + return nil, fmt.Errorf("group_key_package.owner_did must match active identity") + } + agentDID := stringFromAny(binding["agent_did"]) + if agentDID == "" { + agentDID = record.DID + } + if agentDID != record.DID { + return nil, fmt.Errorf("did_wba_binding.agent_did must match active identity") + } + verificationMethod := verificationMethodID(record.DIDDocument) + if verificationMethod == "" { + return nil, fmt.Errorf("active DID document does not expose a signing verification method") + } + leafSignatureKey := stringFromAny(binding["leaf_signature_key_b64u"]) + if leafSignatureKey == "" { + return nil, fmt.Errorf("did_wba_binding.leaf_signature_key_b64u is required") + } + issuedAt := stringFromAny(binding["issued_at"]) + if issuedAt == "" { + return nil, fmt.Errorf("did_wba_binding.issued_at is required") + } + expiresAt := stringFromAny(binding["expires_at"]) + if expiresAt == "" { + return nil, fmt.Errorf("did_wba_binding.expires_at is required") + } + privateKey, err := loadPrivateKeyMaterial(record.Key1PrivatePEM) + if err != nil { + return nil, fmt.Errorf("load active identity signing key: %w", err) + } + signedBinding, err := anpsdk.GenerateDidWbaBinding( + record.DID, + verificationMethod, + leafSignatureKey, + privateKey, + issuedAt, + expiresAt, + issuedAt, + ) + if err != nil { + return nil, fmt.Errorf("sign did_wba_binding: %w", err) + } + signedPackageResult := cloneStringAnyMap(packageResult) + signedGroupKeyPackage := cloneStringAnyMap(groupKeyPackage) + signedGroupKeyPackage["owner_did"] = record.DID + signedGroupKeyPackage["did_wba_binding"] = signedBinding + signedPackageResult["group_key_package"] = signedGroupKeyPackage + return signedPackageResult, nil +} + +func cloneStringAnyMap(source map[string]any) map[string]any { + clone := make(map[string]any, len(source)) + for key, value := range source { + clone[key] = value + } + return clone +} + +func (s *Service) PublishGroupE2EEKeyPackage(ctx context.Context, identityName string, deviceID string, groupDID string, recovery bool, contractTest bool) (*CommandResult, error) { + record, err := s.requireActiveIdentity(identityName) + if err != nil { + return nil, err + } + packageResult, published, err := s.publishGroupE2EEKeyPackage(ctx, record, deviceID, groupDID, recovery, contractTest) + if err != nil { + return nil, err + } + return &CommandResult{ + Data: map[string]any{ + "mls": packageResult, + "published": published, + "recovery": recovery, + "group": strings.TrimSpace(groupDID), + "device_id": defaultString(strings.TrimSpace(deviceID), "default"), + "argv_safe": true, + "p4_mutates": false, + }, + Summary: "Published group E2EE KeyPackage", + }, nil +} + +func (s *Service) createGroupE2EE(ctx context.Context, record *identity.StoredIdentity, groupDID string) (map[string]any, []string) { + provider := s.groupMLSProvider() + mlsHead, err := provider.CreateGroup(ctx, MLSRequest{ + APIVersion: "anp-mls/v1", + RequestID: "group-e2ee-create-" + generateOperationID(), + AgentDID: record.DID, + DeviceID: "default", + Params: map[string]any{ + "agent_did": record.DID, + "device_id": "default", + "group_did": groupDID, + }, + }) + if err != nil { + return nil, []string{fmt.Sprintf("Group E2EE MLS create failed: %v", err)} + } + transport, _, err := s.httpTransport(record) + if err != nil { + return map[string]any{"mls": mlsHead}, []string{fmt.Sprintf("Group E2EE service transport unavailable: %v", err)} + } + delivery, err := transport.CreateGroupE2EE(ctx, groupDID, mlsHead) + if err != nil { + return map[string]any{"mls": mlsHead}, []string{fmt.Sprintf("Group E2EE create delivery failed: %v", err)} + } + warnings := s.persistGroupE2EESummary(ctx, record, groupDID, mlsHead, delivery) + return map[string]any{"mls": mlsHead, "delivery": delivery}, warnings +} + +func (s *Service) addGroupMemberE2EE(ctx context.Context, record *identity.StoredIdentity, groupDID string, memberDID string) (map[string]any, []string) { + transport, _, err := s.httpTransport(record) + if err != nil { + return nil, []string{fmt.Sprintf("Group E2EE service transport unavailable: %v", err)} + } + leasedPackage, err := transport.GetGroupE2EEKeyPackage(ctx, memberDID) + if err != nil { + return nil, []string{fmt.Sprintf("Group E2EE member KeyPackage lookup failed: %v", err)} + } + provider := s.groupMLSProvider() + mlsHead, err := provider.AddMember(ctx, MLSRequest{ + APIVersion: "anp-mls/v1", + RequestID: "group-e2ee-add-" + generateOperationID(), + AgentDID: record.DID, + DeviceID: "default", + Params: map[string]any{ + "agent_did": record.DID, + "device_id": "default", + "group_did": groupDID, + "member_did": memberDID, + "group_key_package": leasedPackage["group_key_package"], + "key_package_id": leasedPackage["key_package_id"], + "target_key_package": leasedPackage, + }, + }) + if err != nil { + return map[string]any{"leased_key_package": redactedKeyPackageSummary(leasedPackage)}, []string{fmt.Sprintf("Group E2EE MLS add-member failed: %v", err)} + } + if keyPackageID := stringFromAny(leasedPackage["key_package_id"]); keyPackageID != "" { + mlsHead["key_package_id"] = keyPackageID + } + if groupKeyPackage, ok := leasedPackage["group_key_package"]; ok { + mlsHead["group_key_package"] = groupKeyPackage + } + delivery, err := transport.AddGroupE2EE(ctx, groupDID, memberDID, mlsHead) + if err != nil { + return map[string]any{"mls": mlsHead, "leased_key_package": redactedKeyPackageSummary(leasedPackage)}, []string{fmt.Sprintf("Group E2EE add delivery failed: %v", err)} + } + warnings := s.persistGroupE2EESummary(ctx, record, groupDID, mlsHead, delivery) + localWelcome, localWelcomeWarnings := s.processLocalGroupWelcome(ctx, memberDID, groupDID, delivery, leasedPackage) + warnings = append(warnings, localWelcomeWarnings...) + result := map[string]any{"mls": mlsHead, "delivery": delivery, "leased_key_package": redactedKeyPackageSummary(leasedPackage)} + if localWelcome != nil { + result["local_welcome"] = localWelcome + } + return result, warnings +} + +func (s *Service) removeGroupMemberE2EE(ctx context.Context, record *identity.StoredIdentity, request GroupMemberRequest) (map[string]any, []string, error) { + operationID := "op-" + generateOperationID() + provider := s.groupMLSProvider() + prepared, err := provider.RemoveMember(ctx, MLSRequest{ + APIVersion: "anp-mls/v1", + RequestID: "group-e2ee-remove-" + generateOperationID(), + AgentDID: record.DID, + DeviceID: "default", + Params: map[string]any{ + "agent_did": record.DID, + "actor_did": record.DID, + "device_id": "default", + "group_did": request.Group, + "member_did": request.Member, + "subject_did": request.Member, + "operation_id": operationID, + "group_state_ref": s.localGroupStateRef(ctx, record, request.Group), + }, + }) + if err != nil { + return nil, nil, err + } + return s.submitPreparedGroupE2EECommit(ctx, record, request.Group, request.Member, request.ReasonText, prepared, func(transport *HTTPTransport) (map[string]any, error) { + return transport.RemoveGroupE2EE(ctx, request.Group, request.Member, prepared, request.ReasonText, request.LeaveRequestID) + }) +} + +func (s *Service) leaveGroupE2EE(ctx context.Context, record *identity.StoredIdentity, request GroupLeaveRequest) (map[string]any, []string, error) { + transport, warnings, err := s.httpTransport(record) + if err != nil { + return nil, warnings, err + } + delivery, err := transport.CreateGroupE2EELeaveRequest(ctx, request.Group, request.ReasonText) + if err != nil { + return nil, warnings, err + } + requestID := firstNonEmptyString(stringFromAny(delivery["leave_request_id"]), stringFromAny(delivery["request_id"])) + data := map[string]any{ + "delivery": delivery, + "group_did": request.Group, + "subject_did": record.DID, + "subject_status": "leave_requested", + "leave_request_id": requestID, + } + warnings = append(warnings, "Group E2EE leave request created; an owner/admin must process it with `group e2ee process-leave-request` to advance the MLS epoch.") + return data, warnings, nil +} + +func (s *Service) ProcessGroupE2EELeaveRequest(ctx context.Context, request GroupE2EEProcessLeaveRequest) (*CommandResult, error) { + if strings.TrimSpace(request.Group) == "" { + return nil, ErrGroupRequired + } + if strings.TrimSpace(request.Member) == "" { + return nil, ErrMemberRequired + } + record, err := s.requireActiveIdentity(request.IdentityName) + if err != nil { + return nil, err + } + memberDID, memberHandle, err := s.resolveTarget(ctx, request.Member) + if err != nil { + return nil, err + } + mutation := GroupMemberRequest{ + IdentityName: request.IdentityName, + Group: request.Group, + Member: memberDID, + ReasonText: firstNonEmptyString(strings.TrimSpace(request.ReasonText), "leave request processed by owner/admin"), + E2EE: true, + LeaveRequestID: strings.TrimSpace(request.LeaveRequestID), + } + e2eeResult, e2eeWarnings, err := s.removeGroupMemberE2EE(ctx, record, mutation) + if err != nil { + return nil, err + } + warnings := append([]string(nil), e2eeWarnings...) + warnings = append(warnings, s.syncGroupState(ctx, record, request.Group, true)...) + snapshot, _ := s.readCachedGroupSnapshot(ctx, record, request.Group) + members, _ := s.readCachedGroupMembers(ctx, record, request.Group, 100) + return &CommandResult{ + Data: map[string]any{ + "group": snapshot, + "members": members, + "delivery": e2eeResult["delivery"], + "member": map[string]any{"did": memberDID, "handle": memberHandle}, + "leave_request_id": mutation.LeaveRequestID, + "e2ee": e2eeResult, + }, + Summary: "Processed group E2EE leave request with epoch-advancing remove", + Warnings: compactWarnings(warnings), + }, nil +} + +func (s *Service) RecoverGroupE2EEMember(ctx context.Context, request GroupE2EERecoverMemberRequest) (*CommandResult, error) { + if strings.TrimSpace(request.Group) == "" { + return nil, ErrGroupRequired + } + if strings.TrimSpace(request.Member) == "" { + return nil, ErrMemberRequired + } + record, err := s.requireActiveIdentity(request.IdentityName) + if err != nil { + return nil, err + } + memberDID, memberHandle, err := s.resolveTarget(ctx, request.Member) + if err != nil { + return nil, err + } + deviceID := defaultString(strings.TrimSpace(request.DeviceID), "default") + transport, warnings, err := s.httpTransport(record) + if err != nil { + return nil, err + } + serviceHead, headErr := transport.GetGroupE2EEHead(ctx, request.Group) + if headErr != nil { + warnings = append(warnings, fmt.Sprintf("Group E2EE service head unavailable before recovery: %v", headErr)) + } else if !boolFromAny(serviceHead["actor_recovery_eligible"]) { + return nil, fmt.Errorf("group E2EE recovery requires the actor to be an active P4 group member/admin; status=%s", stringFromAny(serviceHead["actor_membership_status"])) + } + leasedPackage, err := transport.GetGroupE2EERecoveryKeyPackage(ctx, request.Group, memberDID, deviceID) + if err != nil { + return nil, err + } + provider := s.groupMLSProvider() + operationID := "op-" + generateOperationID() + prepared, err := provider.RecoverMemberPrepare(ctx, MLSRequest{ + APIVersion: "anp-mls/v1", + RequestID: "group-e2ee-recover-member-prepare-" + generateOperationID(), + AgentDID: record.DID, + DeviceID: "default", + Params: map[string]any{ + "agent_did": record.DID, + "actor_did": record.DID, + "device_id": "default", + "group_did": request.Group, + "target": map[string]any{"agent_did": memberDID, "device_id": deviceID}, + "target_did": memberDID, + "target_device_id": deviceID, + "recovery_key_package_id": leasedPackage["key_package_id"], + "group_key_package": leasedPackage["group_key_package"], + "target_key_package": leasedPackage, + "operation_id": operationID, + "group_state_ref": s.localGroupStateRef(ctx, record, request.Group), + "p4_membership_mutate": false, + "recovery_operation_purpose": "same-device-crypto-recovery", + }, + }) + if err != nil { + return nil, err + } + delivery, submitErr := transport.RecoverGroupE2EEMember(ctx, request.Group, memberDID, deviceID, prepared, leasedPackage) + if submitErr != nil { + if shouldAbortGroupE2EEPendingCommit(submitErr) { + abortResult, abortErr := s.abortPreparedGroupE2EECommit(ctx, record, request.Group, prepared) + if abortErr != nil { + warnings = append(warnings, fmt.Sprintf("Group E2EE recovery pending commit abort failed after service rejection: %v", abortErr)) + } else { + warnings = append(warnings, "Group E2EE recovery pending commit aborted after deterministic service rejection.") + return nil, fmt.Errorf("%w; local group E2EE recovery pending commit aborted: %v", submitErr, abortResult) + } + } + return nil, submitErr + } + finalized, finalizeErr := s.finalizePreparedGroupE2EECommit(ctx, record, request.Group, prepared) + if finalizeErr != nil { + warnings = append(warnings, fmt.Sprintf("Group E2EE recovery accepted by service but local finalize failed: %v", finalizeErr)) + } + summarySource := prepared + if finalized != nil { + summarySource = finalized + } + warnings = append(warnings, s.persistGroupE2EESummary(ctx, record, request.Group, summarySource, delivery)...) + localWelcome, localWelcomeWarnings := s.processLocalGroupWelcome(ctx, memberDID, request.Group, delivery, leasedPackage) + warnings = append(warnings, localWelcomeWarnings...) + data := map[string]any{ + "group": request.Group, + "member": map[string]any{"did": memberDID, "handle": memberHandle}, + "target": map[string]any{"agent_did": memberDID, "device_id": deviceID}, + "recovery_key_package": redactedKeyPackageSummary(leasedPackage), + "mls_prepare": prepared, + "mls_finalize": finalized, + "delivery": delivery, + "p4_membership_mutate": false, + "argv_sensitive_fields": "stdin-json-only", + } + if localWelcome != nil { + data["local_welcome"] = localWelcome + } + return &CommandResult{ + Data: data, + Summary: "Recovered active group E2EE member without P4 membership mutation", + Warnings: compactWarnings(warnings), + }, nil +} + +func (s *Service) submitPreparedGroupE2EECommit( + ctx context.Context, + record *identity.StoredIdentity, + groupDID string, + subjectDID string, + reasonText string, + prepared map[string]any, + submit func(*HTTPTransport) (map[string]any, error), +) (map[string]any, []string, error) { + transport, warnings, err := s.httpTransport(record) + if err != nil { + return map[string]any{"mls_prepare": prepared}, warnings, err + } + delivery, err := submit(transport) + if err != nil { + if shouldAbortGroupE2EEPendingCommit(err) { + if abortResult, abortErr := s.abortPreparedGroupE2EECommit(ctx, record, groupDID, prepared); abortErr != nil { + warnings = append(warnings, fmt.Sprintf("Group E2EE pending commit abort failed after service rejection: %v", abortErr)) + } else { + warnings = append(warnings, "Group E2EE pending commit aborted after deterministic service rejection.") + return map[string]any{"mls_prepare": prepared, "mls_abort": abortResult, "subject_did": subjectDID, "reason_text": reasonText}, warnings, fmt.Errorf("%w; local group E2EE pending commit aborted", err) + } + } else { + warnings = append(warnings, "Group E2EE pending commit left intact after retryable or unknown service failure; retry with the same operation_id or inspect group e2ee status before finalize/abort.") + } + return map[string]any{"mls_prepare": prepared, "subject_did": subjectDID, "reason_text": reasonText}, warnings, fmt.Errorf("%w; local group E2EE pending commit retained for retry", err) + } + finalized, finalizeErr := s.finalizePreparedGroupE2EECommit(ctx, record, groupDID, prepared) + if finalizeErr != nil { + warnings = append(warnings, fmt.Sprintf("Group E2EE service accepted commit but local finalize failed: %v", finalizeErr)) + } + summarySource := prepared + if finalized != nil { + summarySource = finalized + } + warnings = append(warnings, s.persistGroupE2EESummary(ctx, record, groupDID, summarySource, delivery)...) + return map[string]any{ + "mls_prepare": prepared, + "mls_finalize": finalized, + "delivery": delivery, + "subject_did": subjectDID, + "reason_text": reasonText, + }, warnings, nil +} + +func (s *Service) finalizePreparedGroupE2EECommit(ctx context.Context, record *identity.StoredIdentity, groupDID string, prepared map[string]any) (map[string]any, error) { + provider := s.groupMLSProvider() + return provider.CommitFinalize(ctx, MLSRequest{ + APIVersion: "anp-mls/v1", + RequestID: "group-e2ee-commit-finalize-" + generateOperationID(), + AgentDID: record.DID, + DeviceID: "default", + Params: pendingCommitParams(record, groupDID, prepared), + }) +} + +func (s *Service) abortPreparedGroupE2EECommit(ctx context.Context, record *identity.StoredIdentity, groupDID string, prepared map[string]any) (map[string]any, error) { + provider := s.groupMLSProvider() + return provider.CommitAbort(ctx, MLSRequest{ + APIVersion: "anp-mls/v1", + RequestID: "group-e2ee-commit-abort-" + generateOperationID(), + AgentDID: record.DID, + DeviceID: "default", + Params: pendingCommitParams(record, groupDID, prepared), + }) +} + +func unsupportedGroupE2EESelfLeaveReason(prepared map[string]any) string { + if len(prepared) == 0 { + return "" + } + if stringFromAny(prepared["artifact_type"]) == "local-terminal-leave" { + return "anp-mls returned a local-terminal leave artifact instead of an MLS epoch-advancing commit" + } + fromEpoch, hasFromEpoch := int64FromAny(prepared["from_epoch"]) + toEpoch, hasToEpoch := int64FromAny(prepared["to_epoch"]) + if !hasToEpoch { + toEpoch, hasToEpoch = int64FromAny(prepared["epoch"]) + } + if hasFromEpoch && hasToEpoch && toEpoch <= fromEpoch { + return fmt.Sprintf("anp-mls returned non-advancing leave epochs from_epoch=%d to_epoch=%d", fromEpoch, toEpoch) + } + return "" +} + +func int64FromAny(value any) (int64, bool) { + switch typed := value.(type) { + case int: + return int64(typed), true + case int8: + return int64(typed), true + case int16: + return int64(typed), true + case int32: + return int64(typed), true + case int64: + return typed, true + case uint: + return int64(typed), true + case uint8: + return int64(typed), true + case uint16: + return int64(typed), true + case uint32: + return int64(typed), true + case uint64: + if typed > uint64(^uint64(0)>>1) { + return 0, false + } + return int64(typed), true + case float64: + if typed != float64(int64(typed)) { + return 0, false + } + return int64(typed), true + case string: + parsed, err := strconv.ParseInt(strings.TrimSpace(typed), 10, 64) + if err != nil { + return 0, false + } + return parsed, true + default: + return 0, false + } +} + +func pendingCommitParams(record *identity.StoredIdentity, groupDID string, prepared map[string]any) map[string]any { + params := map[string]any{ + "agent_did": record.DID, + "actor_did": record.DID, + "device_id": "default", + "group_did": groupDID, + "commit_b64u": prepared["commit_b64u"], + } + for _, key := range []string{"pending_commit_id", "operation_id", "subject_did", "subject_status", "from_epoch", "to_epoch"} { + if value, ok := prepared[key]; ok { + params[key] = value + } + } + return params +} + +func shouldAbortGroupE2EEPendingCommit(err error) bool { + var serviceErr *ServiceError + if !errors.As(err, &serviceErr) { + return false + } + if serviceErr.StatusCode >= http.StatusInternalServerError { + return false + } + if serviceErr.StatusCode >= http.StatusBadRequest { + return true + } + return serviceErr.RPCCode >= 2000 +} + +func (s *Service) sendGroupE2EE(ctx context.Context, record *identity.StoredIdentity, request SendRequest) (*CommandResult, error) { + provider := s.groupMLSProvider() + warnings := s.syncGroupState(ctx, record, request.Group, false) + deviceID := "default" + if status, candidateDeviceID, statusErr := groupE2EEStatusForRecovery(ctx, provider, record.DID, request.Group, ""); statusErr != nil { + warnings = append(warnings, fmt.Sprintf("Group E2EE send could not inspect device-scoped MLS status before encrypt: %v", statusErr)) + } else if strings.EqualFold(stringFromAny(status["status"]), "active") { + deviceID = candidateDeviceID + } + groupStateRef := s.localGroupStateRef(ctx, record, request.Group) + operationID := "op-" + generateOperationID() + messageID := "msg-" + generateOperationID() + contentType := "application/anp-group-cipher+json" + encryptResult, err := provider.Encrypt(ctx, MLSRequest{ + APIVersion: "anp-mls/v1", + RequestID: "group-e2ee-encrypt-" + generateOperationID(), + AgentDID: record.DID, + DeviceID: deviceID, + Params: map[string]any{ + "agent_did": record.DID, + "device_id": deviceID, + "group_did": request.Group, + "group_state_ref": groupStateRef, + "sender_did": record.DID, + "content_type": contentType, + "security_profile": GroupE2EESecurityProfile, + "message_id": messageID, + "operation_id": operationID, + "message_type": request.MessageType, + "application_plaintext": map[string]any{ + "application_content_type": contentTypeForMessageType(request.MessageType), + "text": request.Text, + }, + }, + }) + if err != nil { + return nil, err + } + cipher, _ := encryptResult["group_cipher_object"].(map[string]any) + if len(cipher) == 0 { + return nil, fmt.Errorf("anp-mls encrypt response missing group_cipher_object") + } + transport, transportWarnings, err := s.httpTransport(record) + warnings = append(warnings, transportWarnings...) + if err != nil { + return nil, err + } + delivery, err := transport.SendGroupE2EE(ctx, request.Group, cipher, operationID, messageID) + if err != nil && isGroupE2EEEpochMismatch(err) { + if finalized, finalizeErr := s.finalizePendingGroupE2EECommitFromStatus(ctx, record, request.Group); finalizeErr == nil && finalized { + warnings = append(warnings, "Group E2EE local pending commit finalized after service epoch mismatch.") + } + repairResult, repairErr := s.RepairGroupE2EENotices(ctx, record.IdentityName, request.Group, 50) + if repairErr != nil { + warnings = append(warnings, fmt.Sprintf("Group E2EE send saw stale epoch and notice repair failed: %v", repairErr)) + return nil, err + } + warnings = append(warnings, "Group E2EE local epoch was stale; repaired pending notices and retried send.") + if repairResult != nil { + warnings = append(warnings, repairResult.Warnings...) + } + operationID = "op-" + generateOperationID() + messageID = "msg-" + generateOperationID() + if status, candidateDeviceID, statusErr := groupE2EEStatusForRecovery(ctx, provider, record.DID, request.Group, deviceID); statusErr != nil { + warnings = append(warnings, fmt.Sprintf("Group E2EE send could not inspect device-scoped MLS status after repair: %v", statusErr)) + } else if strings.EqualFold(stringFromAny(status["status"]), "active") { + deviceID = candidateDeviceID + } + groupStateRef = s.localGroupStateRef(ctx, record, request.Group) + encryptResult, err = provider.Encrypt(ctx, MLSRequest{ + APIVersion: "anp-mls/v1", + RequestID: "group-e2ee-encrypt-retry-" + generateOperationID(), + AgentDID: record.DID, + DeviceID: deviceID, + Params: map[string]any{ + "agent_did": record.DID, + "device_id": deviceID, + "group_did": request.Group, + "group_state_ref": groupStateRef, + "sender_did": record.DID, + "content_type": contentType, + "security_profile": GroupE2EESecurityProfile, + "message_id": messageID, + "operation_id": operationID, + "message_type": request.MessageType, + "application_plaintext": map[string]any{ + "application_content_type": contentTypeForMessageType(request.MessageType), + "text": request.Text, + }, + }, + }) + if err != nil { + return nil, err + } + cipher, _ = encryptResult["group_cipher_object"].(map[string]any) + if len(cipher) == 0 { + return nil, fmt.Errorf("anp-mls retry encrypt response missing group_cipher_object") + } + delivery, err = transport.SendGroupE2EE(ctx, request.Group, cipher, operationID, messageID) + } + if err != nil { + return nil, err + } + return s.persistGroupE2EESendResult(ctx, record, request, delivery, encryptResult, warnings) +} + +func (s *Service) finalizePendingGroupE2EECommitFromStatus(ctx context.Context, record *identity.StoredIdentity, groupDID string) (bool, error) { + provider := s.groupMLSProvider() + status, err := provider.Status(ctx, MLSRequest{ + APIVersion: "anp-mls/v1", + RequestID: "group-e2ee-pending-status-" + generateOperationID(), + AgentDID: record.DID, + DeviceID: "default", + Params: map[string]any{ + "agent_did": record.DID, + "device_id": "default", + "group_did": groupDID, + }, + }) + if err != nil { + return false, err + } + for _, pending := range messagesFromResult(status["pending_commits"]) { + pendingID := stringFromAny(pending["pending_commit_id"]) + if pendingID == "" { + continue + } + _, finalizeErr := s.finalizePreparedGroupE2EECommit(ctx, record, groupDID, map[string]any{"pending_commit_id": pendingID}) + if finalizeErr != nil { + return false, finalizeErr + } + return true, nil + } + return false, nil +} + +func isGroupE2EEEpochMismatch(err error) bool { + if err == nil { + return false + } + text := strings.ToLower(err.Error()) + return strings.Contains(text, "group.e2ee.send") && strings.Contains(text, "epoch mismatch") +} + +func (s *Service) maybeDecryptGroupMessages(ctx context.Context, record *identity.StoredIdentity, groupDID string, raw map[string]any) ([]string, map[string]any) { + messages := messagesFromResult(raw["messages"]) + if len(messages) == 0 { + return nil, raw + } + provider := s.groupMLSProvider() + deviceIDs := provider.candidateDeviceIDs(record.DID) + warnings := make([]string, 0) + for _, item := range messages { + cipher := groupCipherObjectFromMessage(item) + if len(cipher) == 0 { + continue + } + aad := groupE2EEAADParamsFromMessage(groupDID, item, cipher) + plain, err := decryptGroupCipherWithDevices(ctx, provider, record.DID, groupDID, cipher, aad, deviceIDs) + if err != nil { + warnings = append(warnings, fmt.Sprintf("Group E2EE decrypt failed for message %s: %v", stringFromAny(item["id"]), err)) + continue + } + if appPlaintext, ok := plain["application_plaintext"].(map[string]any); ok { + item["content"] = defaultString(stringFromAny(appPlaintext["text"]), metadataString(appPlaintext)) + item["content_type"] = defaultString(stringFromAny(appPlaintext["application_content_type"]), "text/plain") + item["decrypted"] = true + } + } + raw["messages"] = messages + return compactWarnings(warnings), raw +} + +func decryptGroupCipherWithDevices(ctx context.Context, provider MLSExecProvider, agentDID string, groupDID string, cipher map[string]any, aad map[string]any, deviceIDs []string) (map[string]any, error) { + if len(deviceIDs) == 0 { + deviceIDs = []string{"default"} + } + var lastErr error + for _, deviceID := range deviceIDs { + deviceID = defaultString(strings.TrimSpace(deviceID), "default") + params := map[string]any{ + "agent_did": agentDID, + "recipient_did": agentDID, + "device_id": deviceID, + "group_did": groupDID, + "group_cipher_object": cipher, + "private_message_b64u": cipher["private_message_b64u"], + } + for key, value := range aad { + if value != nil { + params[key] = value + } + } + plain, err := provider.Decrypt(ctx, MLSRequest{ + APIVersion: "anp-mls/v1", + RequestID: "group-e2ee-decrypt-" + generateOperationID(), + AgentDID: agentDID, + DeviceID: deviceID, + Params: params, + }) + if err == nil { + return plain, nil + } + lastErr = err + } + if lastErr != nil { + return nil, lastErr + } + return nil, fmt.Errorf("no candidate MLS device state found") +} + +func (s *Service) persistGroupE2EESummary(ctx context.Context, record *identity.StoredIdentity, groupDID string, mls map[string]any, delivery map[string]any) []string { + db, err := store.Open(s.resolved.Paths) + if err != nil { + return []string{fmt.Sprintf("Failed to open local store for group E2EE summary: %v", err)} + } + defer db.Close() + if err := store.EnsureSchema(ctx, db); err != nil { + return []string{fmt.Sprintf("Failed to ensure local schema for group E2EE summary: %v", err)} + } + metadata := map[string]any{ + "message_security_profile": GroupE2EESecurityProfile, + "group_e2ee": map[string]any{ + "crypto_group_id_b64u": stringFromAny(firstNonNil(mls["crypto_group_id_b64u"], delivery["crypto_group_id_b64u"])), + "epoch": stringFromAny(firstNonNil(mls["epoch"], delivery["epoch"])), + "epoch_authenticator": stringFromAny(firstNonNil(mls["epoch_authenticator"], mls["epoch_authenticator_b64u"], delivery["epoch_authenticator"])), + "suite": stringFromAny(firstNonNil(mls["suite"], delivery["suite"])), + "updated_at": stringFromAny(delivery["updated_at"]), + "operation_id": stringFromAny(delivery["operation_id"]), + }, + } + if err := store.UpsertGroup(ctx, db, store.GroupRecord{ + OwnerDID: record.DID, + GroupID: groupStorageKey(groupDID), + GroupDID: groupDID, + MembershipStatus: "active", + Metadata: metadataString(metadata), + CredentialName: record.IdentityName, + }); err != nil { + return []string{fmt.Sprintf("Failed to persist group E2EE summary: %v", err)} + } + return nil +} + +func (s *Service) persistGroupE2EESendResult(ctx context.Context, record *identity.StoredIdentity, request SendRequest, result *groupSendResult, encryptResult map[string]any, warnings []string) (*CommandResult, error) { + commandResult, err := s.persistGroupSendResult(ctx, record, request, result, warnings, "http") + if err != nil { + return nil, err + } + if message, ok := commandResult.Data["message"].(map[string]any); ok { + message["secure"] = true + message["security_profile"] = GroupE2EESecurityProfile + } + commandResult.Data["e2ee"] = map[string]any{ + "encrypted": true, + "group_state_ref": groupStateRefFromCipher(encryptResult), + "cipher_object_sent": true, + } + commandResult.Summary = fmt.Sprintf("Sent a group %s message with group E2EE", request.MessageType) + return commandResult, nil +} + +func (s *Service) processLocalGroupWelcome(ctx context.Context, memberDID string, groupDID string, delivery map[string]any, leasedPackage map[string]any) (map[string]any, []string) { + notice := e2eeNoticeObject(delivery) + welcomeB64U := stringFromAny(notice["welcome_b64u"]) + if welcomeB64U == "" { + return nil, nil + } + ratchetTreeB64U := stringFromAny(notice["ratchet_tree_b64u"]) + if ratchetTreeB64U == "" { + return nil, []string{"Group E2EE local welcome processing skipped: notice missing ratchet_tree_b64u"} + } + memberRecord, err := s.localIdentityByDID(memberDID) + if err != nil { + return nil, nil + } + deviceID := groupE2EEWelcomeDeviceID(leasedPackage) + provider := s.groupMLSProvider() + welcomeResult, err := provider.ProcessWelcome(ctx, MLSRequest{ + APIVersion: "anp-mls/v1", + RequestID: "group-e2ee-welcome-" + generateOperationID(), + AgentDID: memberRecord.DID, + DeviceID: deviceID, + Params: map[string]any{ + "agent_did": memberRecord.DID, + "device_id": deviceID, + "group_did": groupDID, + "welcome_b64u": welcomeB64U, + "ratchet_tree_b64u": ratchetTreeB64U, + "group_state_ref": map[string]any{"group_did": groupDID}, + }, + }) + if err != nil { + return nil, []string{fmt.Sprintf("Group E2EE local welcome processing failed for member %s: %v", memberDID, err)} + } + warnings := s.persistGroupE2EESummary(ctx, memberRecord, groupDID, welcomeResult, delivery) + return map[string]any{ + "processed": true, + "group_did": groupDID, + "member_did": memberRecord.DID, + "device_id": deviceID, + "epoch": welcomeResult["epoch"], + }, warnings +} + +func (s *Service) InspectGroupE2EEStatus(ctx context.Context, identityName string, groupDID string, limit int) (*CommandResult, error) { + record, err := s.requireActiveIdentity(identityName) + if err != nil { + return nil, err + } + if limit <= 0 { + limit = 50 + } + provider := s.groupMLSProvider() + localStatus, localDeviceID, localErr := groupE2EEStatusForRecovery(ctx, provider, record.DID, groupDID, "") + warnings := []string(nil) + if localErr != nil { + warnings = append(warnings, fmt.Sprintf("Group E2EE local MLS status unavailable: %v", localErr)) + } + + transport, transportWarnings, transportErr := s.httpTransport(record) + warnings = append(warnings, transportWarnings...) + var serviceHead map[string]any + var pending map[string]any + if transportErr != nil { + warnings = append(warnings, fmt.Sprintf("Group E2EE service status unavailable: %v", transportErr)) + } else { + var headErr error + serviceHead, headErr = transport.GetGroupE2EEHead(ctx, groupDID) + if headErr != nil { + warnings = append(warnings, fmt.Sprintf("Group E2EE service head unavailable: %v", headErr)) + } + var pendingErr error + pending, pendingErr = transport.PullGroupE2EENotices(ctx, groupDID, limit, false) + if pendingErr != nil { + warnings = append(warnings, fmt.Sprintf("Group E2EE pending notice status unavailable: %v", pendingErr)) + } + } + + pendingNotices := noticesFromResult(nil) + pendingNoticeCount := 0 + if pending != nil { + pendingNotices = noticesFromResult(pending["notices"]) + if count, ok := int64FromAny(pending["pending_count"]); ok { + pendingNoticeCount = int(count) + } else { + pendingNoticeCount = len(pendingNotices) + } + } + diagnosis := groupE2EERecoveryDiagnosis(localStatus, serviceHead, pendingNoticeCount, localErr) + recoveryArtifact := groupE2EERecoveryArtifact(record, groupDID, localDeviceID, localStatus, serviceHead, diagnosis) + return &CommandResult{ + Data: map[string]any{ + "group": groupDID, + "available": localErr == nil, + "mls": localStatus, + "local": localStatus, + "local_device_id": localDeviceID, + "service_head": serviceHead, + "pending_notices": pendingNotices, + "pending_notice_count": pendingNoticeCount, + "diagnosis": diagnosis, + "recovery_artifact": recoveryArtifact, + }, + Summary: "Group E2EE recovery status inspected", + Warnings: compactWarnings(warnings), + }, nil +} + +func (s *Service) PullGroupE2EENotices(ctx context.Context, identityName string, groupDID string, limit int) (*CommandResult, error) { + record, err := s.requireActiveIdentity(identityName) + if err != nil { + return nil, err + } + transport, warnings, err := s.httpTransport(record) + if err != nil { + return nil, err + } + result, err := transport.PullGroupE2EENotices(ctx, groupDID, limit, false) + if err != nil { + return nil, err + } + return &CommandResult{ + Data: map[string]any{ + "notices": noticesFromResult(result["notices"]), + "pending_count": result["pending_count"], + "group": groupDID, + }, + Summary: "Pulled group E2EE pending notices", + Warnings: warnings, + }, nil +} + +func (s *Service) RepairGroupE2EENotices(ctx context.Context, identityName string, groupDID string, limit int) (*CommandResult, error) { + record, err := s.requireActiveIdentity(identityName) + if err != nil { + return nil, err + } + transport, warnings, err := s.httpTransport(record) + if err != nil { + return nil, err + } + serviceHead, headErr := transport.GetGroupE2EEHead(ctx, groupDID) + if headErr != nil { + warnings = append(warnings, fmt.Sprintf("Group E2EE service head unavailable during repair: %v", headErr)) + } + finalizedPending := []map[string]any(nil) + if len(serviceHead) > 0 { + var finalizeWarnings []string + finalizedPending, finalizeWarnings = s.finalizeAcceptedPendingGroupE2EECommits(ctx, record, groupDID, serviceHead) + warnings = append(warnings, finalizeWarnings...) + } + pending, err := transport.PullGroupE2EENotices(ctx, groupDID, limit, false) + if err != nil { + return nil, err + } + processed := make([]map[string]any, 0) + noticeIDs := make([]string, 0) + provider := s.groupMLSProvider() + for _, notice := range noticesFromResult(pending["notices"]) { + noticeType := stringFromAny(notice["notice_type"]) + if noticeType != "welcome-delivery" && noticeType != "commit-delivery" { + continue + } + targetGroupDID := defaultString(stringFromAny(notice["group_did"]), groupDID) + if targetGroupDID == "" { + warnings = append(warnings, fmt.Sprintf("Group E2EE repair skipped %s notice without group_did", noticeType)) + continue + } + recipient := firstNonEmptyString(notice["recipient_did"], notice["member_did"]) + if noticeType == "welcome-delivery" && recipient == "" { + recipient = stringFromAny(notice["subject_did"]) + } + if recipient != "" && recipient != record.DID { + warnings = append(warnings, fmt.Sprintf("Group E2EE repair skipped notice for different recipient %s", recipient)) + continue + } + if noticeType == "commit-delivery" { + commit, noticeWarnings := s.processGroupCommitNotice(ctx, record, targetGroupDID, notice) + if commit != nil { + processed = append(processed, commit) + if noticeID := stringFromAny(notice["notice_id"]); noticeID != "" { + noticeIDs = append(noticeIDs, noticeID) + } + continue + } + warnings = append(warnings, noticeWarnings...) + continue + } + welcome, noticeWarnings := s.processGroupWelcomeNotice(ctx, record, targetGroupDID, notice) + if welcome != nil { + processed = append(processed, welcome) + if noticeID := stringFromAny(notice["notice_id"]); noticeID != "" { + noticeIDs = append(noticeIDs, noticeID) + } + continue + } + if s.groupWelcomeAlreadyAvailable(ctx, provider, record, targetGroupDID, notice) { + processed = append(processed, map[string]any{ + "processed": true, + "already_restored": true, + "notice_id": notice["notice_id"], + "group_did": targetGroupDID, + "member_did": record.DID, + "device_id": defaultString(stringFromAny(notice["device_id"]), "default"), + }) + if noticeID := stringFromAny(notice["notice_id"]); noticeID != "" { + noticeIDs = append(noticeIDs, noticeID) + } + continue + } + warnings = append(warnings, noticeWarnings...) + } + delivered := map[string]any(nil) + if len(noticeIDs) > 0 { + delivered, err = transport.MarkGroupE2EENoticesDelivered(ctx, groupDID, noticeIDs) + if err != nil { + warnings = append(warnings, fmt.Sprintf("Group E2EE repair processed notices but failed to mark delivered: %v", err)) + } + } + localStatus, localDeviceID, localErr := groupE2EEStatusForRecovery(ctx, provider, record.DID, groupDID, "") + if localErr != nil { + warnings = append(warnings, fmt.Sprintf("Group E2EE local MLS status unavailable after repair: %v", localErr)) + } + remainingPending := len(noticesFromResult(pending["notices"])) - len(noticeIDs) + if remainingPending < 0 { + remainingPending = 0 + } + diagnosis := groupE2EERecoveryDiagnosis(localStatus, serviceHead, remainingPending, localErr) + if action := stringFromAny(diagnosis["next_action"]); action == "needs_snapshot_or_readd" { + warnings = append(warnings, "Group E2EE repair could not prove epoch continuity; fail closed and ask an owner/admin to run group e2ee recover-member after this member publishes a --recovery --group KeyPackage.") + } + recoveryArtifact := groupE2EERecoveryArtifact(record, groupDID, localDeviceID, localStatus, serviceHead, diagnosis) + return &CommandResult{ + Data: map[string]any{ + "processed": processed, + "processed_count": len(processed), + "finalized_pending_commits": finalizedPending, + "finalized_pending_count": len(finalizedPending), + "pending_count": pending["pending_count"], + "delivered_result": delivered, + "group": groupDID, + "local": localStatus, + "local_device_id": localDeviceID, + "service_head": serviceHead, + "diagnosis": diagnosis, + "recovery_artifact": recoveryArtifact, + }, + Summary: "Replayed group E2EE pending notices", + Warnings: compactWarnings(warnings), + }, nil +} + +func groupE2EERecoveryArtifact(record *identity.StoredIdentity, groupDID string, deviceID string, localStatus map[string]any, serviceHead map[string]any, diagnosis map[string]any) map[string]any { + if stringFromAny(diagnosis["next_action"]) != "needs_snapshot_or_readd" { + return nil + } + artifact := map[string]any{ + "recovery_type": "same-device-owner-assisted", + "group_did": groupDID, + "member_did": record.DID, + "device_id": defaultString(strings.TrimSpace(deviceID), "default"), + "diagnosis_state": diagnosis["state"], + "fail_closed": true, + "p4_membership_mutate": false, + "publish_command": fmt.Sprintf("group e2ee publish-key-package --recovery --group %s --device %s", groupDID, defaultString(strings.TrimSpace(deviceID), "default")), + "owner_command": fmt.Sprintf("group e2ee recover-member --group %s --member %s --device %s", groupDID, record.DID, defaultString(strings.TrimSpace(deviceID), "default")), + } + if localEpoch, ok := groupE2EELocalEpochFromStatus(localStatus); ok { + artifact["local_epoch"] = strconv.FormatInt(localEpoch, 10) + } + if serviceHead != nil { + artifact["service_epoch"] = serviceHead["epoch"] + artifact["member_status"] = serviceHead["actor_membership_status"] + } + return artifact +} + +func (s *Service) processGroupCommitNotice(ctx context.Context, record *identity.StoredIdentity, groupDID string, notice map[string]any) (map[string]any, []string) { + commitB64U := stringFromAny(notice["commit_b64u"]) + if commitB64U == "" { + return nil, []string{"Group E2EE repair skipped commit notice missing commit_b64u"} + } + groupStateRef, _ := notice["group_state_ref"].(map[string]any) + if len(groupStateRef) == 0 { + groupStateRef = map[string]any{ + "group_did": groupDID, + } + if cryptoGroupID := stringFromAny(notice["crypto_group_id_b64u"]); cryptoGroupID != "" { + groupStateRef["crypto_group_id_b64u"] = cryptoGroupID + } + if fromEpoch := stringFromAny(notice["from_epoch"]); fromEpoch != "" { + groupStateRef["epoch"] = fromEpoch + } + } + provider := s.groupMLSProvider() + deviceIDs := groupE2EERecoveryDeviceIDs(provider, record.DID, stringFromAny(notice["device_id"])) + warnings := []string(nil) + for _, deviceID := range deviceIDs { + params := map[string]any{ + "agent_did": record.DID, + "device_id": deviceID, + "group_did": groupDID, + "group_state_ref": groupStateRef, + "commit_b64u": commitB64U, + "ratchet_tree_b64u": notice["ratchet_tree_b64u"], + "group_info_b64u": notice["group_info_b64u"], + "operation_id": notice["operation_id"], + "notice_id": notice["notice_id"], + "actor_did": notice["actor_did"], + "subject_did": notice["subject_did"], + "subject_status": notice["subject_status"], + "from_epoch": notice["from_epoch"], + "to_epoch": notice["to_epoch"], + "crypto_group_id_b64u": notice["crypto_group_id_b64u"], + "epoch_authenticator": firstNonNil(notice["epoch_authenticator"], notice["epoch_authenticator_b64u"]), + } + commitResult, err := provider.ProcessCommit(ctx, MLSRequest{ + APIVersion: "anp-mls/v1", + RequestID: "group-e2ee-commit-repair-" + generateOperationID(), + AgentDID: record.DID, + DeviceID: deviceID, + Params: params, + }) + if err != nil { + if alreadyApplied, statusWarnings := s.groupCommitNoticeAlreadyApplied(ctx, provider, record, groupDID, deviceID, notice); alreadyApplied { + return map[string]any{ + "processed": true, + "already_applied": true, + "notice_type": "commit-delivery", + "notice_id": notice["notice_id"], + "group_did": groupDID, + "member_did": record.DID, + "device_id": deviceID, + "epoch": notice["to_epoch"], + "subject_did": notice["subject_did"], + "subject_status": notice["subject_status"], + }, statusWarnings + } + warnings = append(warnings, fmt.Sprintf("Group E2EE repair commit processing failed on device %s: %v", deviceID, err)) + continue + } + resultWarnings := []string(nil) + if persistWarnings := s.persistGroupE2EESummary(ctx, record, groupDID, commitResult, notice); len(persistWarnings) > 0 { + resultWarnings = append(resultWarnings, persistWarnings...) + } + return map[string]any{ + "processed": true, + "notice_type": "commit-delivery", + "notice_id": notice["notice_id"], + "group_did": groupDID, + "member_did": record.DID, + "device_id": deviceID, + "epoch": commitResult["epoch"], + "subject_did": notice["subject_did"], + "subject_status": notice["subject_status"], + }, resultWarnings + } + return nil, compactWarnings(warnings) +} + +func (s *Service) finalizeAcceptedPendingGroupE2EECommits(ctx context.Context, record *identity.StoredIdentity, groupDID string, serviceHead map[string]any) ([]map[string]any, []string) { + provider := s.groupMLSProvider() + status, err := provider.Status(ctx, MLSRequest{ + APIVersion: "anp-mls/v1", + RequestID: "group-e2ee-pending-repair-status-" + generateOperationID(), + AgentDID: record.DID, + DeviceID: "default", + Params: map[string]any{ + "agent_did": record.DID, + "device_id": "default", + "group_did": groupDID, + }, + }) + if err != nil { + return nil, []string{fmt.Sprintf("Group E2EE pending commit status unavailable during repair: %v", err)} + } + finalized := make([]map[string]any, 0) + warnings := []string(nil) + for _, pending := range messagesFromResult(status["pending_commits"]) { + pendingID := stringFromAny(pending["pending_commit_id"]) + if pendingID == "" { + continue + } + if !groupE2EEPendingCommitAcceptedByService(pending, serviceHead) { + warnings = append(warnings, fmt.Sprintf("Group E2EE pending commit %s retained: service head has not accepted its target epoch.", pendingID)) + continue + } + result, finalizeErr := s.finalizePreparedGroupE2EECommit(ctx, record, groupDID, map[string]any{"pending_commit_id": pendingID}) + if finalizeErr != nil { + warnings = append(warnings, fmt.Sprintf("Group E2EE pending commit %s matched service head but local finalize failed: %v", pendingID, finalizeErr)) + continue + } + finalized = append(finalized, result) + } + return finalized, warnings +} + +func groupE2EEPendingCommitAcceptedByService(pending map[string]any, serviceHead map[string]any) bool { + if len(pending) == 0 || len(serviceHead) == 0 { + return false + } + if groupDID := stringFromAny(pending["group_did"]); groupDID != "" { + if serviceGroupDID := stringFromAny(serviceHead["group_did"]); serviceGroupDID != "" && serviceGroupDID != groupDID { + return false + } + } + if cryptoGroupID := stringFromAny(pending["crypto_group_id_b64u"]); cryptoGroupID != "" { + if serviceCryptoGroupID := stringFromAny(serviceHead["crypto_group_id_b64u"]); serviceCryptoGroupID != "" && serviceCryptoGroupID != cryptoGroupID { + return false + } + } + toEpoch, hasToEpoch := int64FromAny(pending["to_epoch"]) + serviceEpoch, hasServiceEpoch := int64FromAny(serviceHead["epoch"]) + return hasToEpoch && hasServiceEpoch && serviceEpoch >= toEpoch +} + +func (s *Service) groupCommitNoticeAlreadyApplied(ctx context.Context, provider MLSExecProvider, record *identity.StoredIdentity, groupDID string, deviceID string, notice map[string]any) (bool, []string) { + toEpoch, hasToEpoch := int64FromAny(notice["to_epoch"]) + if !hasToEpoch { + return false, nil + } + status, err := provider.Status(ctx, MLSRequest{ + APIVersion: "anp-mls/v1", + RequestID: "group-e2ee-commit-duplicate-status-" + generateOperationID(), + AgentDID: record.DID, + DeviceID: deviceID, + Params: map[string]any{ + "agent_did": record.DID, + "device_id": deviceID, + "group_did": groupDID, + }, + }) + if err != nil { + return false, []string{fmt.Sprintf("Group E2EE repair could not inspect local status after commit failure: %v", err)} + } + localEpoch, hasLocalEpoch := groupE2EELocalEpochFromStatus(status) + if !hasLocalEpoch || localEpoch < toEpoch { + return false, nil + } + if noticeCryptoGroupID := stringFromAny(notice["crypto_group_id_b64u"]); noticeCryptoGroupID != "" { + if localCryptoGroupID := stringFromAny(status["crypto_group_id_b64u"]); localCryptoGroupID != "" && localCryptoGroupID != noticeCryptoGroupID { + return false, nil + } + } + return true, []string{"Group E2EE repair treated duplicate/already-applied commit notice as delivered."} +} + +func groupE2EERecoveryDiagnosis(localStatus map[string]any, serviceHead map[string]any, pendingNoticeCount int, localErr error) map[string]any { + diagnosis := map[string]any{ + "state": "unknown", + "next_action": "inspect", + "fail_closed": true, + "pending_notice_count": pendingNoticeCount, + } + if localErr != nil { + diagnosis["local_error"] = localErr.Error() + } + if localText := stringFromAny(localStatus["status"]); localText != "" { + diagnosis["local_status"] = localText + } + if serviceHead != nil { + diagnosis["actor_membership_status"] = serviceHead["actor_membership_status"] + diagnosis["actor_recovery_eligible"] = serviceHead["actor_recovery_eligible"] + diagnosis["service_epoch"] = serviceHead["epoch"] + } + if localEpoch, ok := groupE2EELocalEpochFromStatus(localStatus); ok { + diagnosis["local_epoch"] = strconv.FormatInt(localEpoch, 10) + } + pendingCommitCount := len(messagesFromResult(localStatus["pending_commits"])) + diagnosis["pending_commit_count"] = pendingCommitCount + + actorStatus := strings.ToLower(strings.TrimSpace(stringFromAny(serviceHead["actor_membership_status"]))) + if actorStatus == "removed" || actorStatus == "left" || actorStatus == "non_member" || actorStatus == "inactive" { + diagnosis["state"] = "inactive" + diagnosis["next_action"] = "fail_closed" + diagnosis["fail_closed"] = true + return diagnosis + } + if pendingCommitCount > 0 { + diagnosis["state"] = "pending_commit" + diagnosis["next_action"] = "run_group_e2ee_repair" + diagnosis["fail_closed"] = false + return diagnosis + } + if pendingNoticeCount > 0 { + diagnosis["state"] = "pending_notices" + diagnosis["next_action"] = "run_group_e2ee_repair" + diagnosis["fail_closed"] = false + return diagnosis + } + localEpoch, hasLocalEpoch := groupE2EELocalEpochFromStatus(localStatus) + serviceEpoch, hasServiceEpoch := int64FromAny(serviceHead["epoch"]) + localState := strings.ToLower(strings.TrimSpace(stringFromAny(localStatus["status"]))) + if localErr != nil || localState == "" || localState == "empty" || !hasLocalEpoch { + diagnosis["state"] = "missing_state" + diagnosis["next_action"] = "needs_snapshot_or_readd" + diagnosis["fail_closed"] = true + return diagnosis + } + if hasLocalEpoch && hasServiceEpoch { + switch { + case localEpoch == serviceEpoch: + diagnosis["state"] = "in_sync" + diagnosis["next_action"] = "none" + diagnosis["fail_closed"] = false + case localEpoch < serviceEpoch: + diagnosis["state"] = "epoch_lag" + diagnosis["epoch_gap"] = serviceEpoch - localEpoch + diagnosis["next_action"] = "needs_snapshot_or_readd" + diagnosis["fail_closed"] = true + default: + diagnosis["state"] = "local_ahead" + diagnosis["epoch_gap"] = localEpoch - serviceEpoch + diagnosis["next_action"] = "stop_and_inspect" + diagnosis["fail_closed"] = true + } + return diagnosis + } + diagnosis["state"] = "local_only" + diagnosis["next_action"] = "inspect_service_head" + diagnosis["fail_closed"] = true + return diagnosis +} + +func groupE2EELocalEpochFromStatus(status map[string]any) (int64, bool) { + for _, value := range []any{status["epoch"], status["local_epoch"]} { + if epoch, ok := int64FromAny(value); ok { + return epoch, true + } + } + for _, binding := range messagesFromResult(status["bindings"]) { + if epoch, ok := int64FromAny(binding["epoch"]); ok { + return epoch, true + } + } + return 0, false +} + +func groupE2EEStatusForRecovery(ctx context.Context, provider MLSExecProvider, agentDID string, groupDID string, preferredDeviceID string) (map[string]any, string, error) { + deviceIDs := groupE2EERecoveryDeviceIDs(provider, agentDID, preferredDeviceID) + var best map[string]any + bestDeviceID := "default" + bestRank := -1 + var bestEpoch int64 + var lastErr error + for _, deviceID := range deviceIDs { + status, err := provider.Status(ctx, MLSRequest{ + APIVersion: "anp-mls/v1", + RequestID: "group-e2ee-status-" + generateOperationID(), + AgentDID: agentDID, + DeviceID: deviceID, + Params: map[string]any{ + "agent_did": agentDID, + "device_id": deviceID, + "group_did": groupDID, + }, + }) + if err != nil { + lastErr = err + continue + } + rank := groupE2EEStatusRank(status) + epoch, hasEpoch := groupE2EELocalEpochFromStatus(status) + if !hasEpoch { + epoch = -1 + } + if best == nil || rank > bestRank || (rank == bestRank && epoch > bestEpoch) { + best = status + bestDeviceID = deviceID + bestRank = rank + bestEpoch = epoch + } + } + if best != nil { + best["device_id"] = bestDeviceID + return best, bestDeviceID, nil + } + if lastErr != nil { + return nil, "", lastErr + } + return map[string]any{"status": "empty", "device_id": bestDeviceID}, bestDeviceID, nil +} + +func groupE2EERecoveryDeviceIDs(provider MLSExecProvider, agentDID string, preferredDeviceID string) []string { + ordered := make([]string, 0) + seen := make(map[string]struct{}) + add := func(deviceID string) { + deviceID = defaultString(strings.TrimSpace(deviceID), "default") + if _, ok := seen[deviceID]; ok { + return + } + seen[deviceID] = struct{}{} + ordered = append(ordered, deviceID) + } + if preferredDeviceID != "" { + add(preferredDeviceID) + } + for _, deviceID := range provider.candidateDeviceIDs(agentDID) { + add(deviceID) + } + if len(ordered) == 0 { + add("default") + } + return ordered +} + +func groupE2EEStatusRank(status map[string]any) int { + state := strings.ToLower(strings.TrimSpace(stringFromAny(status["status"]))) + switch state { + case "active": + return 3 + case "left", "removed", "inactive": + return 2 + case "empty", "": + if _, ok := groupE2EELocalEpochFromStatus(status); ok { + return 1 + } + return 0 + default: + if _, ok := groupE2EELocalEpochFromStatus(status); ok { + return 1 + } + return 0 + } +} + +func (s *Service) groupWelcomeAlreadyAvailable(ctx context.Context, provider MLSExecProvider, record *identity.StoredIdentity, groupDID string, notice map[string]any) bool { + deviceID := defaultString(stringFromAny(notice["device_id"]), "default") + resp, err := provider.Call(ctx, "group", "status", MLSRequest{ + APIVersion: "anp-mls/v1", + RequestID: "group-e2ee-welcome-status-" + generateOperationID(), + AgentDID: record.DID, + DeviceID: deviceID, + Params: map[string]any{ + "agent_did": record.DID, + "device_id": deviceID, + "group_did": groupDID, + }, + }) + if err != nil || resp == nil { + return false + } + return stringFromAny(resp.Result["status"]) == "active" +} + +func (s *Service) processGroupWelcomeNotice(ctx context.Context, record *identity.StoredIdentity, groupDID string, notice map[string]any) (map[string]any, []string) { + welcomeB64U := stringFromAny(notice["welcome_b64u"]) + if welcomeB64U == "" { + return nil, []string{"Group E2EE repair skipped welcome notice missing welcome_b64u"} + } + ratchetTreeB64U := stringFromAny(notice["ratchet_tree_b64u"]) + if ratchetTreeB64U == "" { + return nil, []string{"Group E2EE repair skipped welcome notice missing ratchet_tree_b64u"} + } + deviceID := defaultString(stringFromAny(notice["device_id"]), "default") + provider := s.groupMLSProvider() + welcomeResult, err := provider.ProcessWelcome(ctx, MLSRequest{ + APIVersion: "anp-mls/v1", + RequestID: "group-e2ee-welcome-repair-" + generateOperationID(), + AgentDID: record.DID, + DeviceID: deviceID, + Params: map[string]any{ + "agent_did": record.DID, + "device_id": deviceID, + "group_did": groupDID, + "welcome_b64u": welcomeB64U, + "ratchet_tree_b64u": ratchetTreeB64U, + "group_state_ref": firstNonNil(notice["group_state_ref"], map[string]any{"group_did": groupDID}), + }, + }) + if err != nil { + return nil, []string{fmt.Sprintf("Group E2EE repair welcome processing failed: %v", err)} + } + warnings := s.persistGroupE2EESummary(ctx, record, groupDID, welcomeResult, notice) + return map[string]any{ + "processed": true, + "notice_id": notice["notice_id"], + "group_did": groupDID, + "member_did": record.DID, + "device_id": deviceID, + "epoch": welcomeResult["epoch"], + }, warnings +} + +func (s *Service) localIdentityByDID(did string) (*identity.StoredIdentity, error) { + if s == nil || s.manager == nil { + return nil, identity.ErrIdentityNotFound + } + summaries, err := s.manager.List() + if err != nil { + return nil, err + } + for _, summary := range summaries { + if summary.DID == did { + return s.manager.Load(summary.IdentityName) + } + } + return nil, identity.ErrIdentityNotFound +} + +func e2eeNoticeObject(delivery map[string]any) map[string]any { + if notice, ok := delivery["e2ee_notice"].(map[string]any); ok { + return notice + } + return nil +} + +func groupE2EEWelcomeDeviceID(leasedPackage map[string]any) string { + if groupKeyPackage, ok := leasedPackage["group_key_package"].(map[string]any); ok { + if deviceID := stringFromAny(groupKeyPackage["device_id"]); deviceID != "" { + return deviceID + } + } + if deviceID := stringFromAny(leasedPackage["device_id"]); deviceID != "" { + return deviceID + } + return "default" +} + +func (s *Service) localGroupStateRef(ctx context.Context, record *identity.StoredIdentity, groupDID string) map[string]any { + snapshot, err := s.readCachedGroupSnapshot(ctx, record, groupDID) + if err != nil { + return map[string]any{"group_did": groupDID} + } + return groupStateRefFromSnapshot(groupDID, snapshot) +} + +func groupStateRefFromSnapshot(groupDID string, snapshot map[string]any) map[string]any { + ref := map[string]any{"group_did": groupDID} + metadata := decodeMetadataMap(snapshot["metadata"]) + if version := firstNonEmptyString(snapshot["group_state_version"], metadata["group_state_version"]); version != "" { + ref["group_state_version"] = version + } + if e2ee, ok := metadata["group_e2ee"].(map[string]any); ok { + if version := stringFromAny(e2ee["group_state_version"]); version != "" { + ref["group_state_version"] = version + } + if cryptoGroupID := stringFromAny(e2ee["crypto_group_id_b64u"]); cryptoGroupID != "" { + ref["crypto_group_id_b64u"] = cryptoGroupID + } + } + return ref +} + +func (s *Service) groupHasLocalE2EEState(ctx context.Context, record *identity.StoredIdentity, groupDID string) bool { + if strings.TrimSpace(groupDID) == "" || record == nil { + return false + } + provider := s.groupMLSProvider() + resp, err := provider.Status(ctx, MLSRequest{ + APIVersion: "anp-mls/v1", + RequestID: "group-e2ee-send-detect-" + generateOperationID(), + AgentDID: record.DID, + Params: map[string]any{ + "agent_did": record.DID, + "group_did": groupDID, + }, + }) + if err != nil || len(resp) == 0 { + return false + } + status := strings.ToLower(strings.TrimSpace(stringFromAny(resp["status"]))) + if status == "active" || status == "pending_commit" { + return true + } + if stringFromAny(resp["crypto_group_id_b64u"]) != "" { + return true + } + return false +} + +func groupRequestUsesE2EE(request GroupCreateRequest) bool { + return request.E2EE || strings.TrimSpace(request.MessageSecurityProfile) == GroupE2EESecurityProfile +} + +func groupMemberMutationUsesE2EE(request GroupMemberRequest, preMutationSnapshot map[string]any, postMutationSnapshot map[string]any) bool { + return request.E2EE || groupSnapshotUsesE2EE(preMutationSnapshot) || groupSnapshotUsesE2EE(postMutationSnapshot) +} + +func groupSnapshotUsesE2EE(snapshot map[string]any) bool { + if len(snapshot) == 0 { + return false + } + if stringFromAny(snapshot["message_security_profile"]) == GroupE2EESecurityProfile { + return true + } + if policy, ok := snapshot["group_policy"].(map[string]any); ok { + if stringFromAny(policy["message_security_profile"]) == GroupE2EESecurityProfile { + return true + } + } + metadata := decodeMetadataMap(snapshot["metadata"]) + if stringFromAny(metadata["message_security_profile"]) == GroupE2EESecurityProfile { + return true + } + return false +} + +func decodeMetadataMap(value any) map[string]any { + text := stringFromAny(value) + if text == "" { + if typed, ok := value.(map[string]any); ok { + return typed + } + return nil + } + var result map[string]any + if err := json.Unmarshal([]byte(text), &result); err != nil { + return nil + } + return result +} + +func groupCipherObjectFromMessage(item map[string]any) map[string]any { + for _, key := range []string{"group_cipher_object", "content"} { + if cipher, ok := item[key].(map[string]any); ok { + if nested, ok := cipher["group_cipher_object"].(map[string]any); ok { + return nested + } + if _, ok := cipher["private_message_b64u"]; ok { + return cipher + } + } + } + body, _ := item["body"].(map[string]any) + if cipher, ok := body["group_cipher_object"].(map[string]any); ok { + return cipher + } + return nil +} + +func groupE2EEAADParamsFromMessage(groupDID string, item map[string]any, cipher map[string]any) map[string]any { + receipt, _ := item["receipt"].(map[string]any) + groupStateRef, _ := cipher["group_state_ref"].(map[string]any) + if len(groupStateRef) == 0 { + groupStateRef = map[string]any{"group_did": groupDID} + } + params := map[string]any{ + "group_state_ref": groupStateRef, + "sender_did": stringFromAny(item["sender_did"]), + "content_type": "application/anp-group-cipher+json", + "security_profile": GroupE2EESecurityProfile, + "message_id": firstNonEmptyString(item["message_id"], item["id"]), + "operation_id": firstNonEmptyString(item["operation_id"], receipt["operation_id"]), + } + return params +} + +func groupStateRefFromCipher(encryptResult map[string]any) map[string]any { + cipher, _ := encryptResult["group_cipher_object"].(map[string]any) + ref, _ := cipher["group_state_ref"].(map[string]any) + return ref +} + +func firstNonEmptyString(values ...any) string { + for _, value := range values { + if text := stringFromAny(value); text != "" { + return text + } + } + return "" +} + +func noticesFromResult(value any) []map[string]any { + return messagesFromResult(value) +} + +func redactedKeyPackageSummary(raw map[string]any) map[string]any { + return map[string]any{ + "target_did": raw["target_did"], + "key_package_id": raw["key_package_id"], + "leased": true, + "private_material": false, + } +} diff --git a/internal/message/group_service.go b/internal/message/group_service.go index f1b3197..c5a625c 100644 --- a/internal/message/group_service.go +++ b/internal/message/group_service.go @@ -34,15 +34,24 @@ func (s *Service) CreateGroup(ctx context.Context, request GroupCreateRequest) ( } groupDID := stringFromAny(result["group_did"]) warnings = append(warnings, s.syncGroupState(ctx, record, groupDID, true)...) + var e2eeResult map[string]any + if groupRequestUsesE2EE(request) { + e2eeCandidate, e2eeWarnings := s.createGroupE2EE(ctx, record, groupDID) + e2eeResult, warnings = appendE2EEResult(warnings, e2eeCandidate, e2eeWarnings) + } snapshot, _ := s.readCachedGroupSnapshot(ctx, record, groupDID) members, _ := s.readCachedGroupMembers(ctx, record, groupDID, 100) + data := map[string]any{ + "group": snapshot, + "members": members, + "delivery": result, + "source": groupControlSource(result), + } + if e2eeResult != nil { + data["e2ee"] = e2eeResult + } return &CommandResult{ - Data: map[string]any{ - "group": snapshot, - "members": members, - "delivery": result, - "source": groupControlSource(result), - }, + Data: data, Summary: fmt.Sprintf("Created group %s", groupDID), Warnings: compactWarnings(warnings), }, nil @@ -118,6 +127,31 @@ func (s *Service) mutateGroupMember(ctx context.Context, request GroupMemberRequ return nil, err } request.Member = memberDID + var preMutationSnapshot map[string]any + if action == "add" || action == "remove" { + preMutationSnapshot, _ = s.readCachedGroupSnapshot(ctx, record, request.Group) + } + if action == "remove" && groupMemberMutationUsesE2EE(request, preMutationSnapshot, nil) { + e2eeResult, e2eeWarnings, err := s.removeGroupMemberE2EE(ctx, record, request) + if err != nil { + return nil, err + } + warnings := append([]string(nil), e2eeWarnings...) + warnings = append(warnings, s.syncGroupState(ctx, record, request.Group, true)...) + snapshot, _ := s.readCachedGroupSnapshot(ctx, record, request.Group) + members, _ := s.readCachedGroupMembers(ctx, record, request.Group, 100) + return &CommandResult{ + Data: map[string]any{ + "group": snapshot, + "members": members, + "delivery": e2eeResult["delivery"], + "member": map[string]any{"did": memberDID, "handle": memberHandle}, + "e2ee": e2eeResult, + }, + Summary: "Removed member from group with group E2EE", + Warnings: compactWarnings(warnings), + }, nil + } transport, warnings, err := s.groupControlTransport(record) if err != nil { return nil, err @@ -134,7 +168,16 @@ func (s *Service) mutateGroupMember(ctx context.Context, request GroupMemberRequ warnings = append(warnings, s.syncGroupState(ctx, record, request.Group, true)...) snapshot, _ := s.readCachedGroupSnapshot(ctx, record, request.Group) members, _ := s.readCachedGroupMembers(ctx, record, request.Group, 100) - return &CommandResult{Data: map[string]any{"group": snapshot, "members": members, "delivery": result, "member": map[string]any{"did": memberDID, "handle": memberHandle}}, Summary: fmt.Sprintf("Updated group membership via %s", action), Warnings: compactWarnings(warnings)}, nil + var e2eeResult map[string]any + if action == "add" && groupMemberMutationUsesE2EE(request, preMutationSnapshot, snapshot) { + e2eeCandidate, e2eeWarnings := s.addGroupMemberE2EE(ctx, record, request.Group, memberDID) + e2eeResult, warnings = appendE2EEResult(warnings, e2eeCandidate, e2eeWarnings) + } + data := map[string]any{"group": snapshot, "members": members, "delivery": result, "member": map[string]any{"did": memberDID, "handle": memberHandle}} + if e2eeResult != nil { + data["e2ee"] = e2eeResult + } + return &CommandResult{Data: data, Summary: fmt.Sprintf("Updated group membership via %s", action), Warnings: compactWarnings(warnings)}, nil } func (s *Service) LeaveGroup(ctx context.Context, request GroupLeaveRequest) (*CommandResult, error) { @@ -149,6 +192,22 @@ func (s *Service) LeaveGroup(ctx context.Context, request GroupLeaveRequest) (*C if snapshotErr == nil && isActiveGroupOwner(cachedSnapshot) { return nil, ErrGroupOwnerCannotLeave } + if request.E2EE || groupSnapshotUsesE2EE(cachedSnapshot) { + e2eeResult, e2eeWarnings, err := s.leaveGroupE2EE(ctx, record, request) + if err != nil { + return nil, err + } + warnings := append([]string(nil), e2eeWarnings...) + return &CommandResult{ + Data: map[string]any{ + "delivery": e2eeResult["delivery"], + "group": request.Group, + "e2ee": e2eeResult, + }, + Summary: fmt.Sprintf("Requested group E2EE leave for %s", request.Group), + Warnings: compactWarnings(warnings), + }, nil + } transport, warnings, err := s.groupControlTransport(record) if err != nil { return nil, err @@ -256,6 +315,11 @@ func (s *Service) GroupMessages(ctx context.Context, request GroupMessagesReques warnings = append(warnings, websocketHTTPFallbackWarning(wsErr)) warnings = append(warnings, httpWarnings...) } + decryptWarnings, decryptedResult := s.maybeDecryptGroupMessages(ctx, record, request.Group, result) + warnings = append(warnings, decryptWarnings...) + if decryptedResult != nil { + result = decryptedResult + } warnings = append(warnings, s.persistGroupMessages(ctx, record, request.Group, result)...) messages, _ := s.readCachedGroupMessages(ctx, record, request.Group, request.Limit, request.Cursor) if len(messages) == 0 { @@ -273,12 +337,26 @@ func (s *Service) sendGroup(ctx context.Context, request SendRequest) (*CommandR return nil, ErrTextRequired } if request.SecureMode == "on" { - return nil, ErrSecureNotSupported + // For groups, --secure on selects the explicit group E2EE path when the + // cached group summary indicates group-e2ee. + record, err := s.requireActiveIdentity(request.IdentityName) + if err != nil { + return nil, err + } + snapshot, _ := s.readCachedGroupSnapshot(ctx, record, request.Group) + if !groupSnapshotUsesE2EE(snapshot) { + return nil, ErrSecureNotSupported + } + return s.sendGroupE2EE(ctx, record, request) } record, err := s.requireActiveIdentity(request.IdentityName) if err != nil { return nil, err } + snapshot, _ := s.readCachedGroupSnapshot(ctx, record, request.Group) + if groupSnapshotUsesE2EE(snapshot) || s.groupHasLocalE2EEState(ctx, record, request.Group) { + return s.sendGroupE2EE(ctx, record, request) + } sourceMode := s.runtimeConfig().Mode transport, warnings, err := s.transportFor(record) if err != nil { @@ -303,6 +381,14 @@ func (s *Service) sendGroup(ctx context.Context, request SendRequest) (*CommandR return s.persistGroupSendResult(ctx, record, request, result, warnings, sourceMode) } +func appendE2EEResult(warnings []string, result map[string]any, e2eeWarnings []string) (map[string]any, []string) { + warnings = append(warnings, e2eeWarnings...) + if result == nil { + return nil, warnings + } + return result, warnings +} + func (s *Service) syncGroupState(ctx context.Context, record *identity.StoredIdentity, groupDID string, includeMembers bool) []string { if strings.TrimSpace(groupDID) == "" { return nil diff --git a/internal/message/group_service_test.go b/internal/message/group_service_test.go index 0037f4f..6e6e55a 100644 --- a/internal/message/group_service_test.go +++ b/internal/message/group_service_test.go @@ -2,7 +2,14 @@ package message import ( "context" + "encoding/json" "errors" + "fmt" + "net/http" + "net/http/httptest" + "os" + "path/filepath" + "strings" "testing" appconfig "github.com/agentconnect/awiki-cli/internal/config" @@ -49,6 +56,79 @@ func TestLeaveGroupRejectsActiveOwnerFromCachedSnapshot(t *testing.T) { } } +func TestLeaveGroupE2EECreatesLeaveRequestWithoutLocalMLSLeave(t *testing.T) { + t.Parallel() + + var captured rpcRequestEnvelope + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + captured = decodeRPCRequest(t, r) + _ = json.NewEncoder(w).Encode(map[string]any{ + "jsonrpc": "2.0", + "id": captured.ID, + "result": map[string]any{ + "accepted": true, + "leave_request_id": "lr-bob-1", + "delivery_state": "pending_owner_action", + }, + }) + })) + defer server.Close() + + service, _, record := newMessageServiceForTest(t, server.URL) + provider := MLSExecProvider{BinaryPath: "anp-mls", Runner: &groupLeaveSafetyMLSRunner{}} + service.mlsProvider = &provider + + result, warnings, err := service.leaveGroupE2EE(context.Background(), record, GroupLeaveRequest{ + Group: "did:wba:awiki.ai:groups:test-e2ee-leave", + ReasonText: "done", + }) + if err != nil { + t.Fatalf("leaveGroupE2EE() error = %v", err) + } + if captured.Method != "group.e2ee.leave_request" { + t.Fatalf("captured.Method = %q, want leave_request", captured.Method) + } + body := mustMapValue(t, captured.Params["body"], "params.body") + if got := stringFromAny(body["subject_status"]); got != "leave_requested" { + t.Fatalf("subject_status = %q, want leave_requested", got) + } + if got := stringFromAny(body["reason_text"]); got != "done" { + t.Fatalf("reason_text = %q, want done", got) + } + if got := stringFromAny(result["leave_request_id"]); got != "lr-bob-1" { + t.Fatalf("leave_request_id = %q, want lr-bob-1", got) + } + if got := strings.Join(warnings, "\n"); !strings.Contains(got, "owner/admin") { + t.Fatalf("warnings = %#v, want owner/admin processing guidance", warnings) + } +} + +type groupLeaveSafetyMLSRunner struct { + calls []string +} + +func (r *groupLeaveSafetyMLSRunner) Run(_ context.Context, _ string, args []string, _ []byte) ([]byte, []byte, error) { + r.calls = append(r.calls, strings.Join(args, " ")) + return []byte(`{"ok":true,"api_version":"anp-mls/v1","request_id":"req-other","result":{}}`), nil, nil +} + +func TestUnsupportedGroupE2EESelfLeaveReasonDetectsNonAdvancingEpoch(t *testing.T) { + t.Parallel() + + if reason := unsupportedGroupE2EESelfLeaveReason(map[string]any{"from_epoch": "7", "to_epoch": "7"}); !strings.Contains(reason, "non-advancing") { + t.Fatalf("same epoch reason = %q, want non-advancing", reason) + } + if reason := unsupportedGroupE2EESelfLeaveReason(map[string]any{"from_epoch": "7", "epoch": "6"}); !strings.Contains(reason, "non-advancing") { + t.Fatalf("regressed epoch reason = %q, want non-advancing", reason) + } + if reason := unsupportedGroupE2EESelfLeaveReason(map[string]any{"artifact_type": "local-terminal-leave", "from_epoch": "7", "to_epoch": "8"}); !strings.Contains(reason, "local-terminal") { + t.Fatalf("local terminal reason = %q, want local-terminal", reason) + } + if reason := unsupportedGroupE2EESelfLeaveReason(map[string]any{"from_epoch": "7", "to_epoch": "8"}); reason != "" { + t.Fatalf("advancing epoch reason = %q, want empty", reason) + } +} + func TestMarkCachedGroupLeftClearsMembersAndResetsRole(t *testing.T) { t.Parallel() @@ -129,6 +209,342 @@ func TestShouldUseCachedGroupFallbackRejectsInactiveViewerErrors(t *testing.T) { } } +func TestGroupMemberMutationUsesPreMutationE2EESnapshot(t *testing.T) { + t.Parallel() + + preMutation := map[string]any{ + "metadata": `{"message_security_profile":"group-e2ee","group_e2ee":{"epoch":"1"}}`, + } + postMutation := map[string]any{ + "metadata": `{"group_state_version":"2"}`, + } + if !groupMemberMutationUsesE2EE(GroupMemberRequest{}, preMutation, postMutation) { + t.Fatal("groupMemberMutationUsesE2EE() = false, want true from pre-mutation E2EE summary") + } + if !groupMemberMutationUsesE2EE(GroupMemberRequest{E2EE: true}, nil, nil) { + t.Fatal("groupMemberMutationUsesE2EE() = false, want true from explicit E2EE request") + } + if groupMemberMutationUsesE2EE(GroupMemberRequest{}, nil, postMutation) { + t.Fatal("groupMemberMutationUsesE2EE() = true, want false for non-E2EE snapshots") + } +} + +func TestShouldAbortGroupE2EEPendingCommitOnlyForDeterministicServiceRejection(t *testing.T) { + t.Parallel() + + if !shouldAbortGroupE2EEPendingCommit(&ServiceError{StatusCode: http.StatusForbidden, Message: "forbidden"}) { + t.Fatal("403 service rejection should abort pending commit") + } + if !shouldAbortGroupE2EEPendingCommit(&ServiceError{RPCCode: 2501, Message: "inactive member"}) { + t.Fatal("2xxx RPC rejection should abort pending commit") + } + if shouldAbortGroupE2EEPendingCommit(&ServiceError{StatusCode: http.StatusServiceUnavailable, Message: "retry later"}) { + t.Fatal("5xx service failure should leave pending commit intact") + } + if shouldAbortGroupE2EEPendingCommit(&ServiceError{RPCCode: 1503, Message: "temporarily unavailable"}) { + t.Fatal("1xxx retryable RPC failure should leave pending commit intact") + } + if shouldAbortGroupE2EEPendingCommit(fmt.Errorf("connection reset")) { + t.Fatal("transport/network errors should leave pending commit intact") + } +} + +func TestInspectGroupE2EEStatusComparesLocalEpochToServiceHead(t *testing.T) { + t.Parallel() + + groupDID := "did:wba:awiki.ai:groups:repair:e1_group" + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + envelope := decodeRPCRequest(t, r) + var result map[string]any + switch envelope.Method { + case "group.e2ee.head": + result = map[string]any{ + "group_did": groupDID, + "crypto_group_id_b64u": "crypto-1", + "epoch": "2", + "actor_membership_status": "active", + "actor_recovery_eligible": true, + "epoch_authenticator_b64u": "auth-2", + "latest_notice_cursor": "notice-commit-2", + } + case "group.e2ee.notice": + result = map[string]any{ + "pending_count": 1, + "notices": []map[string]any{{ + "notice_id": "notice-commit-2", + "notice_type": "commit-delivery", + "group_did": groupDID, + "crypto_group_id_b64u": "crypto-1", + "from_epoch": "1", + "to_epoch": "2", + }}, + } + default: + t.Fatalf("unexpected RPC method %q", envelope.Method) + } + _ = json.NewEncoder(w).Encode(map[string]any{"jsonrpc": "2.0", "id": envelope.ID, "result": result}) + })) + defer server.Close() + + service, _, record := newMessageServiceForTest(t, server.URL) + service.mlsProvider = &MLSExecProvider{ + BinaryPath: "anp-mls", + Runner: &groupE2EEStatusMLSRunner{status: map[string]any{ + "status": "active", + "epoch": "1", + "crypto_group_id_b64u": "crypto-1", + "pending_commits": []map[string]any{}, + }}, + } + result, err := service.InspectGroupE2EEStatus(context.Background(), record.IdentityName, groupDID, 50) + if err != nil { + t.Fatalf("InspectGroupE2EEStatus() error = %v", err) + } + diagnosis := mustMapValue(t, result.Data["diagnosis"], "diagnosis") + if got := stringFromAny(diagnosis["state"]); got != "pending_notices" { + t.Fatalf("diagnosis.state = %q, want pending_notices: %#v", got, diagnosis) + } + if got := stringFromAny(diagnosis["next_action"]); got != "run_group_e2ee_repair" { + t.Fatalf("diagnosis.next_action = %q, want repair: %#v", got, diagnosis) + } + if got := intValueFromAny(result.Data["pending_notice_count"], 0); got != 1 { + t.Fatalf("pending_notice_count = %d, want 1", got) + } +} + +func TestProcessGroupCommitNoticeTreatsDuplicateAsAlreadyApplied(t *testing.T) { + t.Parallel() + + groupDID := "did:wba:awiki.ai:groups:already-applied:e1_group" + service := &Service{ + mlsProvider: &MLSExecProvider{ + BinaryPath: "anp-mls", + Runner: &groupE2EEStatusMLSRunner{ + status: map[string]any{ + "status": "active", + "epoch": "2", + "crypto_group_id_b64u": "crypto-1", + "pending_commits": []map[string]any{}, + }, + failCommitProcess: true, + }, + }, + } + record := &identity.StoredIdentity{IdentityName: "bob", DID: "did:wba:awiki.ai:users:bob:e1_bob"} + processed, warnings := service.processGroupCommitNotice(context.Background(), record, groupDID, map[string]any{ + "notice_id": "notice-2", + "notice_type": "commit-delivery", + "group_did": groupDID, + "crypto_group_id_b64u": "crypto-1", + "from_epoch": "1", + "to_epoch": "2", + "commit_b64u": "opaque-commit", + }) + if processed == nil { + t.Fatalf("processed = nil, warnings = %#v", warnings) + } + if got := boolFromAny(processed["already_applied"]); !got { + t.Fatalf("already_applied = %#v, want true: %#v", processed["already_applied"], processed) + } + if !warningContains(warnings, "already-applied") { + t.Fatalf("warnings = %#v, want already-applied guidance", warnings) + } +} + +func TestGroupE2EEStatusForRecoveryScansNonDefaultDevice(t *testing.T) { + t.Parallel() + + groupDID := "did:wba:awiki.ai:groups:scan-device:e1_group" + agentDID := "did:wba:awiki.ai:users:bob:e1_bob" + dataDir := t.TempDir() + if err := os.MkdirAll(filepath.Join(dataDir, "agents", mlsAgentKey(agentDID), "bob-main"), 0o755); err != nil { + t.Fatalf("MkdirAll() error = %v", err) + } + provider := MLSExecProvider{ + BinaryPath: "anp-mls", + DataDir: dataDir, + Runner: &groupE2EEStatusMLSRunner{statusByDevice: map[string]map[string]any{ + "default": {"status": "empty"}, + "bob-main": {"status": "active", "epoch": "2", "crypto_group_id_b64u": "crypto-1"}, + }}, + } + + status, deviceID, err := groupE2EEStatusForRecovery(context.Background(), provider, agentDID, groupDID, "") + if err != nil { + t.Fatalf("groupE2EEStatusForRecovery() error = %v", err) + } + if deviceID != "bob-main" { + t.Fatalf("deviceID = %q, want bob-main; status=%#v", deviceID, status) + } + if got := stringFromAny(status["epoch"]); got != "2" { + t.Fatalf("epoch = %q, want 2: %#v", got, status) + } +} + +func TestProcessGroupCommitNoticeScansNonDefaultDeviceWhenNoticeOmitsDevice(t *testing.T) { + t.Parallel() + + groupDID := "did:wba:awiki.ai:groups:commit-scan:e1_group" + record := &identity.StoredIdentity{IdentityName: "bob", DID: "did:wba:awiki.ai:users:bob:e1_bob"} + dataDir := t.TempDir() + if err := os.MkdirAll(filepath.Join(dataDir, "agents", mlsAgentKey(record.DID), "bob-main"), 0o755); err != nil { + t.Fatalf("MkdirAll() error = %v", err) + } + service := &Service{ + resolved: testResolvedConfig(t), + mlsProvider: &MLSExecProvider{ + BinaryPath: "anp-mls", + DataDir: dataDir, + Runner: &groupE2EEStatusMLSRunner{ + statusByDevice: map[string]map[string]any{ + "default": {"status": "empty"}, + "bob-main": {"status": "active", "epoch": "1", "crypto_group_id_b64u": "crypto-1"}, + }, + failCommitProcessDevices: map[string]bool{"default": true}, + commitResultByDevice: map[string]map[string]any{ + "bob-main": {"status": "active", "epoch": "2", "crypto_group_id_b64u": "crypto-1"}, + }, + }, + }, + } + + processed, warnings := service.processGroupCommitNotice(context.Background(), record, groupDID, map[string]any{ + "notice_id": "notice-2", + "notice_type": "commit-delivery", + "group_did": groupDID, + "crypto_group_id_b64u": "crypto-1", + "from_epoch": "1", + "to_epoch": "2", + "commit_b64u": "opaque-commit", + }) + if processed == nil { + t.Fatalf("processed = nil, warnings = %#v", warnings) + } + if got := stringFromAny(processed["device_id"]); got != "bob-main" { + t.Fatalf("device_id = %q, want bob-main: %#v", got, processed) + } +} + +type groupE2EEStatusMLSRunner struct { + status map[string]any + statusByDevice map[string]map[string]any + commitResultByDevice map[string]map[string]any + failCommitProcess bool + failCommitProcessDevices map[string]bool +} + +func (r *groupE2EEStatusMLSRunner) Run(_ context.Context, _ string, args []string, stdin []byte) ([]byte, []byte, error) { + var req MLSRequest + _ = json.Unmarshal(stdin, &req) + command := strings.Join(args, " ") + deviceID := defaultString(req.DeviceID, stringFromAny(req.Params["device_id"])) + if strings.Contains(command, "commit process") { + if r.failCommitProcess || r.failCommitProcessDevices[deviceID] { + return []byte(fmt.Sprintf(`{"ok":false,"api_version":"anp-mls/v1","request_id":%q,"error":{"code":"group_epoch_mismatch","message":"commit from_epoch does not match local epoch"}}`, req.RequestID)), nil, nil + } + result := r.commitResultByDevice[deviceID] + if result == nil { + result = map[string]any{"status": "active", "epoch": "2"} + } + return []byte(mustJSONForTest(map[string]any{ + "ok": true, + "api_version": "anp-mls/v1", + "request_id": req.RequestID, + "result": result, + })), nil, nil + } + result := r.statusByDevice[deviceID] + if result == nil { + result = r.status + } + if result == nil { + result = map[string]any{"status": "empty"} + } + return []byte(mustJSONForTest(map[string]any{ + "ok": true, + "api_version": "anp-mls/v1", + "request_id": req.RequestID, + "result": result, + })), nil, nil +} + +func mustJSONForTest(value any) string { + raw, err := json.Marshal(value) + if err != nil { + panic(err) + } + return string(raw) +} + +func TestGroupStateRefFromSnapshotUsesServerStateVersionNotMLSEpoch(t *testing.T) { + t.Parallel() + + ref := groupStateRefFromSnapshot("did:wba:example.com:groups:demo:e1", map[string]any{ + "metadata": `{"group_state_version":"2","group_e2ee":{"epoch":"1","crypto_group_id_b64u":"Y3J5cHRv"}}`, + }) + if got := stringFromAny(ref["group_state_version"]); got != "2" { + t.Fatalf("group_state_version = %q, want server version 2", got) + } + if got := stringFromAny(ref["crypto_group_id_b64u"]); got != "Y3J5cHRv" { + t.Fatalf("crypto_group_id_b64u = %q, want cached crypto id", got) + } + + ref = groupStateRefFromSnapshot("did:wba:example.com:groups:demo:e1", map[string]any{ + "metadata": `{"group_e2ee":{"epoch":"7","crypto_group_id_b64u":"Y3J5cHRv"}}`, + }) + if _, ok := ref["group_state_version"]; ok { + t.Fatalf("group_state_version = %#v, want absent when only MLS epoch is cached", ref["group_state_version"]) + } +} + +func TestLocalIdentityByDIDFindsStoredMemberForWelcomeProcessing(t *testing.T) { + t.Parallel() + + resolved := testResolvedConfig(t) + manager := identity.NewManager(resolved.Paths) + createTestIdentity(t, manager, identity.SaveInput{ + IdentityName: "alice", + UserID: "user-alice", + DisplayName: "Alice", + Handle: "alice", + }) + createTestIdentity(t, manager, identity.SaveInput{ + IdentityName: "bob", + UserID: "user-bob", + DisplayName: "Bob", + Handle: "bob", + }) + bob, err := manager.Load("bob") + if err != nil { + t.Fatalf("Load(bob) error = %v", err) + } + service := &Service{resolved: resolved, manager: manager} + + found, err := service.localIdentityByDID(bob.DID) + if err != nil { + t.Fatalf("localIdentityByDID() error = %v", err) + } + if found.IdentityName != "bob" { + t.Fatalf("localIdentityByDID() identity = %q, want bob", found.IdentityName) + } +} + +func TestGroupE2EEWelcomeDeviceIDUsesPublicKeyPackageDevice(t *testing.T) { + t.Parallel() + + leasedPackage := map[string]any{ + "group_key_package": map[string]any{ + "device_id": "bob-main", + }, + } + if got := groupE2EEWelcomeDeviceID(leasedPackage); got != "bob-main" { + t.Fatalf("groupE2EEWelcomeDeviceID() = %q, want bob-main", got) + } + if got := groupE2EEWelcomeDeviceID(nil); got != "default" { + t.Fatalf("groupE2EEWelcomeDeviceID(nil) = %q, want default", got) + } +} + func writeCachedGroupState(t *testing.T, resolved *appconfig.Resolved, record *identity.StoredIdentity, group store.GroupRecord, members []store.GroupMemberRecord) { t.Helper() @@ -162,3 +578,120 @@ func writeCachedGroupState(t *testing.T, resolved *appconfig.Resolved, record *i t.Fatalf("ReplaceGroupMembers() error = %v", err) } } + +func TestRecoverGroupE2EEMemberUsesRecoverMemberWithoutGroupAdd(t *testing.T) { + t.Parallel() + + groupDID := "did:wba:awiki.ai:groups:recover:e1_group" + bobDID := "did:wba:awiki.ai:user:bob:e1_bob" + methods := []string(nil) + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + envelope := decodeRPCRequest(t, r) + methods = append(methods, envelope.Method) + var result map[string]any + switch envelope.Method { + case "anp.get_capabilities": + result = map[string]any{"service_did": "did:wba:awiki.ai:services:message:e1_service"} + case "group.e2ee.head": + result = map[string]any{ + "group_did": groupDID, + "epoch": "5", + "actor_membership_status": "active", + "actor_recovery_eligible": true, + } + case "group.e2ee.get_key_package": + body := mustMapValue(t, envelope.Params["body"], "get_key_package.body") + if got := stringFromAny(body["purpose"]); got != "recovery" { + t.Fatalf("get_key_package purpose = %q, want recovery", got) + } + result = map[string]any{ + "key_package_id": "kp-recovery-1", + "group_key_package": map[string]any{ + "owner_did": bobDID, + "purpose": "recovery", + "group_did": groupDID, + "device_id": "bob-main", + }, + } + case "group.e2ee.recover_member": + body := mustMapValue(t, envelope.Params["body"], "recover_member.body") + if _, ok := body["member_did"]; ok { + t.Fatalf("recover_member body must not include P4 member_did: %#v", body) + } + target := mustMapValue(t, body["target"], "recover_member.target") + if got := stringFromAny(target["agent_did"]); got != bobDID { + t.Fatalf("target.agent_did = %q, want bob", got) + } + result = map[string]any{"accepted": true, "epoch": "6", "operation_id": stringFromAny(mustMapValue(t, envelope.Params["meta"], "recover_member.meta")["operation_id"])} + default: + t.Fatalf("unexpected RPC method %q", envelope.Method) + } + _ = json.NewEncoder(w).Encode(map[string]any{"jsonrpc": "2.0", "id": envelope.ID, "result": result}) + })) + defer server.Close() + + service, _, record := newMessageServiceForTest(t, server.URL) + runner := &groupE2EERecoverMLSRunner{} + service.mlsProvider = &MLSExecProvider{BinaryPath: "anp-mls", Runner: runner} + + result, err := service.RecoverGroupE2EEMember(context.Background(), GroupE2EERecoverMemberRequest{ + IdentityName: record.IdentityName, + Group: groupDID, + Member: bobDID, + DeviceID: "bob-main", + }) + if err != nil { + t.Fatalf("RecoverGroupE2EEMember() error = %v", err) + } + if got := boolFromAny(result.Data["p4_membership_mutate"]); got { + t.Fatalf("p4_membership_mutate = true, want false") + } + for _, method := range methods { + if method == "group.add" { + t.Fatalf("recovery must not call group.add; methods=%#v", methods) + } + } + if !runner.sawRecoverPrepare || !runner.sawFinalize { + t.Fatalf("runner prepare/finalize = %v/%v, calls=%#v", runner.sawRecoverPrepare, runner.sawFinalize, runner.calls) + } +} + +type groupE2EERecoverMLSRunner struct { + calls []string + sawRecoverPrepare bool + sawFinalize bool +} + +func (r *groupE2EERecoverMLSRunner) Run(_ context.Context, _ string, args []string, stdin []byte) ([]byte, []byte, error) { + var req MLSRequest + _ = json.Unmarshal(stdin, &req) + command := strings.Join(args, " ") + r.calls = append(r.calls, command) + result := map[string]any{"status": "active", "epoch": "6", "crypto_group_id_b64u": "crypto-1"} + if strings.Contains(command, "recover-member-prepare") { + r.sawRecoverPrepare = true + if _, ok := req.Params["p4_membership_mutate"]; !ok { + return []byte(fmt.Sprintf(`{"ok":false,"api_version":"anp-mls/v1","request_id":%q,"error":{"code":"missing_guard","message":"missing no-P4 guard"}}`, req.RequestID)), nil, nil + } + result = map[string]any{ + "operation_id": stringFromAny(req.Params["operation_id"]), + "pending_commit_id": "pc-recover-1", + "crypto_group_id_b64u": "crypto-1", + "from_epoch": "5", + "to_epoch": "6", + "epoch": "6", + "epoch_authenticator_b64u": "auth-6", + "commit_b64u": "opaque-commit", + "welcome_b64u": "opaque-welcome", + } + } + if strings.Contains(command, "commit-finalize") { + r.sawFinalize = true + } + return []byte(mustJSONForTest(map[string]any{ + "ok": true, + "api_version": "anp-mls/v1", + "request_id": req.RequestID, + "result": result, + })), nil, nil +} diff --git a/internal/message/group_wire.go b/internal/message/group_wire.go index a4eed03..5f0d03e 100644 --- a/internal/message/group_wire.go +++ b/internal/message/group_wire.go @@ -24,6 +24,10 @@ func BuildGroupCreateRPCParams(record *identity.StoredIdentity, manager *identit if len(policy) == 0 { policy = buildGroupPolicyPatch("open-join", boolPtr(true), "500", nil, nil) } + if securityProfile := normalizedGroupSecurityProfile(request); securityProfile != "" { + policy["message_security_profile"] = securityProfile + policy["bootstrap_security_profile"] = securityProfile + } meta := map[string]any{ "anp_version": "1.0", "profile": "anp.group.base.v1", @@ -172,6 +176,286 @@ func BuildGroupSendRPCParams(record *identity.StoredIdentity, manager *identity. }, nil } +func BuildGroupE2EECreateRPCParams(record *identity.StoredIdentity, manager *identity.Manager, serviceDID string, groupDID string, mlsHead map[string]any) (map[string]any, error) { + return buildGroupE2EERPCParams(record, manager, "service", serviceDID, "group.e2ee.create", e2eeHeadBody(groupDID, "", mlsHead), "", "", "", GroupE2EESecurityProfile) +} + +func BuildGroupE2EEAddRPCParams(record *identity.StoredIdentity, manager *identity.Manager, groupDID string, memberDID string, mlsHead map[string]any) (map[string]any, error) { + body := e2eeHeadBody(groupDID, memberDID, mlsHead) + if value, ok := mlsHead["welcome_b64u"]; ok { + body["welcome_b64u"] = value + } + if value, ok := mlsHead["commit_b64u"]; ok { + body["commit_b64u"] = value + } + if value, ok := mlsHead["ratchet_tree_b64u"]; ok { + body["ratchet_tree_b64u"] = value + } + if value, ok := mlsHead["key_package_id"]; ok { + body["key_package_id"] = value + body["subject_key_package_id"] = value + } + if value, ok := mlsHead["group_key_package"]; ok { + body["group_key_package"] = value + } + return buildGroupE2EERPCParams(record, manager, "group", groupDID, "group.e2ee.add", body, "", "", "", GroupE2EESecurityProfile) +} + +func BuildGroupE2EERemoveRPCParams(record *identity.StoredIdentity, manager *identity.Manager, groupDID string, memberDID string, preparedCommit map[string]any, reasonText string, leaveRequestID string) (map[string]any, error) { + body := e2eeMembershipCommitBody(groupDID, memberDID, "removed", preparedCommit) + if reason := strings.TrimSpace(reasonText); reason != "" { + body["reason_text"] = reason + } + if requestID := strings.TrimSpace(leaveRequestID); requestID != "" { + body["leave_request_id"] = requestID + } + return buildGroupE2EERPCParams(record, manager, "group", groupDID, "group.e2ee.remove", body, "", stringFromAny(preparedCommit["operation_id"]), "", GroupE2EESecurityProfile) +} + +func BuildGroupE2EELeaveRequestRPCParams(record *identity.StoredIdentity, manager *identity.Manager, groupDID string, reasonText string) (map[string]any, error) { + body := map[string]any{ + "group_did": strings.TrimSpace(groupDID), + "subject_did": record.DID, + "member_did": record.DID, + "subject_status": "leave_requested", + "group_state_ref": map[string]any{"group_did": strings.TrimSpace(groupDID)}, + } + if reason := strings.TrimSpace(reasonText); reason != "" { + body["reason_text"] = reason + } + return buildGroupE2EERPCParams(record, manager, "group", groupDID, "group.e2ee.leave_request", body, "", "", "", GroupE2EETransportProfile) +} + +func BuildGroupE2EELeaveRPCParams(record *identity.StoredIdentity, manager *identity.Manager, groupDID string, preparedCommit map[string]any) (map[string]any, error) { + body := e2eeMembershipCommitBody(groupDID, record.DID, "left", preparedCommit) + return buildGroupE2EERPCParams(record, manager, "group", groupDID, "group.e2ee.leave", body, "", stringFromAny(preparedCommit["operation_id"]), "", GroupE2EESecurityProfile) +} + +func BuildGroupE2EESendRPCParams(record *identity.StoredIdentity, manager *identity.Manager, groupDID string, cipher map[string]any, operationID string, messageID string) (map[string]any, error) { + return buildGroupE2EERPCParams(record, manager, "group", groupDID, "group.e2ee.send", map[string]any{"group_cipher_object": sanitizeGroupCipherObjectForService(cipher)}, "application/anp-group-cipher+json", operationID, messageID, GroupE2EESecurityProfile) +} + +func sanitizeGroupCipherObjectForService(cipher map[string]any) map[string]any { + sanitized := make(map[string]any) + for _, key := range []string{ + "crypto_group_id_b64u", + "epoch", + "private_message_b64u", + "group_state_ref", + "epoch_authenticator", + "non_cryptographic", + "artifact_mode", + } { + if value, ok := cipher[key]; ok { + sanitized[key] = value + } + } + return sanitized +} + +func BuildGroupE2EEPublishKeyPackageRPCParams(record *identity.StoredIdentity, manager *identity.Manager, serviceDID string, packageResult map[string]any) (map[string]any, error) { + serviceDID = strings.TrimSpace(serviceDID) + if serviceDID == "" { + return nil, fmt.Errorf("message service did is required") + } + auth, err := newAuthContext(record, manager) + if err != nil { + return nil, err + } + groupKeyPackage, _ := packageResult["group_key_package"].(map[string]any) + if len(groupKeyPackage) == 0 { + return nil, fmt.Errorf("group_key_package is required") + } + groupKeyPackage = sanitizeGroupKeyPackageForService(groupKeyPackage) + meta := map[string]any{ + "anp_version": "1.0", + "profile": GroupE2EEProfile, + "security_profile": GroupE2EETransportProfile, + "sender_did": record.DID, + "target": map[string]any{"kind": "service", "did": serviceDID}, + "operation_id": "op-" + generateOperationID(), + "created_at": nowRFC3339(), + "content_type": "application/json", + } + body := map[string]any{"group_key_package": groupKeyPackage} + payload := signedPayload{Method: "group.e2ee.publish_key_package", Meta: meta, Body: body} + originProof, err := buildOriginProof(auth, payload) + if err != nil { + return nil, err + } + return map[string]any{ + "meta": meta, + "auth": map[string]any{"scheme": OriginProofScheme, "origin_proof": originProof}, + "body": body, + }, nil +} + +func BuildGroupE2EEGetKeyPackageRPCParams(record *identity.StoredIdentity, manager *identity.Manager, serviceDID string, targetDID string) (map[string]any, error) { + return buildGroupE2EEGetKeyPackageRPCParams(record, manager, serviceDID, map[string]any{"target_did": strings.TrimSpace(targetDID)}) +} + +func BuildGroupE2EEGetRecoveryKeyPackageRPCParams(record *identity.StoredIdentity, manager *identity.Manager, serviceDID string, groupDID string, targetDID string, deviceID string) (map[string]any, error) { + body := map[string]any{ + "target_did": targetDID, + "purpose": "recovery", + "group_did": strings.TrimSpace(groupDID), + "device_id": defaultString(strings.TrimSpace(deviceID), "default"), + } + return buildGroupE2EEGetKeyPackageRPCParams(record, manager, serviceDID, body) +} + +func buildGroupE2EEGetKeyPackageRPCParams(record *identity.StoredIdentity, manager *identity.Manager, serviceDID string, body map[string]any) (map[string]any, error) { + serviceDID = strings.TrimSpace(serviceDID) + targetDID := strings.TrimSpace(stringFromAny(body["target_did"])) + if serviceDID == "" { + return nil, fmt.Errorf("message service did is required") + } + if targetDID == "" { + return nil, ErrMemberRequired + } + auth, err := newAuthContext(record, manager) + if err != nil { + return nil, err + } + meta := map[string]any{ + "anp_version": "1.0", + "profile": GroupE2EEProfile, + "security_profile": GroupE2EETransportProfile, + "sender_did": record.DID, + "target": map[string]any{"kind": "service", "did": serviceDID}, + "operation_id": "op-" + generateOperationID(), + "created_at": nowRFC3339(), + "content_type": "application/json", + } + payload := signedPayload{Method: "group.e2ee.get_key_package", Meta: meta, Body: body} + originProof, err := buildOriginProof(auth, payload) + if err != nil { + return nil, err + } + return map[string]any{ + "meta": meta, + "auth": map[string]any{"scheme": OriginProofScheme, "origin_proof": originProof}, + "body": body, + }, nil +} + +func BuildGroupE2EERecoverMemberRPCParams(record *identity.StoredIdentity, manager *identity.Manager, groupDID string, memberDID string, deviceID string, prepared map[string]any, leasedPackage map[string]any) (map[string]any, error) { + body := e2eeRecoveryCommitBody(groupDID, memberDID, deviceID, prepared, leasedPackage) + return buildGroupE2EERPCParams(record, manager, "group", groupDID, "group.e2ee.recover_member", body, "", stringFromAny(prepared["operation_id"]), "", GroupE2EESecurityProfile) +} + +func BuildGroupE2EENoticeRPCParams(record *identity.StoredIdentity, manager *identity.Manager, groupDID string, limit int, markDelivered bool, noticeIDs []string) (map[string]any, error) { + auth, err := newAuthContext(record, manager) + if err != nil { + return nil, err + } + if limit <= 0 { + limit = 50 + } + if limit > 100 { + limit = 100 + } + meta := map[string]any{ + "anp_version": "1.0", + "profile": GroupE2EEProfile, + "security_profile": GroupE2EETransportProfile, + "sender_did": record.DID, + "target": map[string]any{"kind": "agent", "did": record.DID}, + "operation_id": "op-" + generateOperationID(), + "created_at": nowRFC3339(), + "content_type": "application/json", + } + body := map[string]any{"limit": limit} + if strings.TrimSpace(groupDID) != "" { + body["group_did"] = strings.TrimSpace(groupDID) + } + if markDelivered { + body["mark_delivered"] = true + } + if len(noticeIDs) > 0 { + ids := make([]string, 0, len(noticeIDs)) + for _, noticeID := range noticeIDs { + if trimmed := strings.TrimSpace(noticeID); trimmed != "" { + ids = append(ids, trimmed) + } + } + if len(ids) > 0 { + body["notice_ids"] = ids + } + } + payload := signedPayload{Method: "group.e2ee.notice", Meta: meta, Body: body} + originProof, err := buildOriginProof(auth, payload) + if err != nil { + return nil, err + } + return map[string]any{ + "meta": meta, + "auth": map[string]any{"scheme": OriginProofScheme, "origin_proof": originProof}, + "body": body, + }, nil +} + +func BuildGroupE2EEHeadRPCParams(record *identity.StoredIdentity, manager *identity.Manager, groupDID string) (map[string]any, error) { + groupDID = strings.TrimSpace(groupDID) + if groupDID == "" { + return nil, ErrGroupRequired + } + auth, err := newAuthContext(record, manager) + if err != nil { + return nil, err + } + meta := map[string]any{ + "anp_version": "1.0", + "profile": GroupE2EEProfile, + "security_profile": GroupE2EETransportProfile, + "sender_did": record.DID, + "target": map[string]any{"kind": "group", "did": groupDID}, + "operation_id": "op-" + generateOperationID(), + "created_at": nowRFC3339(), + "content_type": "application/json", + } + body := map[string]any{ + "group_did": groupDID, + "group_state_ref": map[string]any{"group_did": groupDID}, + } + payload := signedPayload{Method: "group.e2ee.head", Meta: meta, Body: body} + originProof, err := buildOriginProof(auth, payload) + if err != nil { + return nil, err + } + return map[string]any{ + "meta": meta, + "auth": map[string]any{"scheme": OriginProofScheme, "origin_proof": originProof}, + "body": body, + }, nil +} + +func sanitizeGroupKeyPackageForService(input map[string]any) map[string]any { + allowed := map[string]struct{}{ + "owner_did": {}, + "device_id": {}, + "key_package_id": {}, + "suite": {}, + "mls_key_package_b64u": {}, + "did_wba_binding": {}, + "expires_at": {}, + "purpose": {}, + "group_did": {}, + "non_cryptographic": {}, + "artifact_mode": {}, + } + output := make(map[string]any, len(input)) + for key, value := range input { + if _, ok := allowed[key]; ok { + if (key == "group_did" || key == "purpose") && strings.TrimSpace(stringFromAny(value)) == "" { + continue + } + output[key] = value + } + } + return output +} + func BuildGroupGetRPCParams(record *identity.StoredIdentity, request GroupGetRequest) (map[string]any, error) { groupDID := strings.TrimSpace(request.Group) if groupDID == "" { @@ -348,6 +632,207 @@ func buildGroupPolicyPatch(admissionMode string, attachmentsAllowed *bool, maxMe return patch } +func normalizedGroupSecurityProfile(request GroupCreateRequest) string { + if request.E2EE { + return GroupE2EESecurityProfile + } + switch strings.TrimSpace(request.MessageSecurityProfile) { + case "", "transport-protected": + return "" + case GroupE2EESecurityProfile: + return GroupE2EESecurityProfile + default: + return strings.TrimSpace(request.MessageSecurityProfile) + } +} + +func buildGroupE2EERPCParams(record *identity.StoredIdentity, manager *identity.Manager, targetKind string, targetDID string, method string, body map[string]any, contentType string, operationID string, messageID string, securityProfile string) (map[string]any, error) { + targetKind = strings.TrimSpace(targetKind) + targetDID = strings.TrimSpace(targetDID) + if targetKind == "" { + targetKind = "group" + } + if targetDID == "" { + return nil, ErrGroupRequired + } + auth, err := newAuthContext(record, manager) + if err != nil { + return nil, err + } + if contentType == "" { + contentType = "application/json" + } + operationID = strings.TrimSpace(operationID) + if operationID == "" { + operationID = "op-" + generateOperationID() + } + securityProfile = strings.TrimSpace(securityProfile) + if securityProfile == "" { + securityProfile = GroupE2EESecurityProfile + } + meta := map[string]any{ + "anp_version": "1.0", + "profile": GroupE2EEProfile, + "security_profile": securityProfile, + "sender_did": record.DID, + "target": map[string]any{"kind": targetKind, "did": targetDID}, + "operation_id": operationID, + "created_at": nowRFC3339(), + "content_type": contentType, + } + if method == "group.e2ee.send" { + messageID = strings.TrimSpace(messageID) + if messageID == "" { + messageID = "msg-" + generateOperationID() + } + meta["message_id"] = messageID + } + payload := signedPayload{Method: method, Meta: meta, Body: body} + originProof, err := buildOriginProof(auth, payload) + if err != nil { + return nil, err + } + return map[string]any{ + "meta": meta, + "auth": map[string]any{"scheme": OriginProofScheme, "origin_proof": originProof}, + "body": body, + }, nil +} + +func e2eeHeadBody(groupDID string, memberDID string, mlsHead map[string]any) map[string]any { + body := map[string]any{ + "group_did": groupDID, + "group_state_ref": map[string]any{ + "group_did": groupDID, + }, + } + for _, key := range []string{"crypto_group_id_b64u", "epoch", "epoch_authenticator", "epoch_authenticator_b64u", "suite", "last_handshake_digest"} { + if value, ok := mlsHead[key]; ok { + body[key] = value + } + } + if value, ok := body["epoch_authenticator_b64u"]; ok { + body["epoch_authenticator"] = value + } + if memberDID != "" { + body["member_did"] = memberDID + body["subject_did"] = memberDID + } + return body +} + +func e2eeMembershipCommitBody(groupDID string, subjectDID string, defaultSubjectStatus string, preparedCommit map[string]any) map[string]any { + body := e2eeHeadBody(groupDID, subjectDID, preparedCommit) + for _, key := range []string{ + "pending_commit_id", + "operation_id", + "commit_b64u", + "ratchet_tree_b64u", + "group_info_b64u", + "from_epoch", + "to_epoch", + "actor_did", + "subject_status", + } { + if value, ok := preparedCommit[key]; ok { + body[key] = value + } + } + if _, ok := body["epoch"]; !ok { + if value, ok := preparedCommit["to_epoch"]; ok { + body["epoch"] = value + } + } + if _, ok := body["epoch_authenticator"]; !ok { + if value, ok := preparedCommit["epoch_authenticator_b64u"]; ok { + body["epoch_authenticator"] = value + } + } + if _, ok := body["subject_status"]; !ok && defaultSubjectStatus != "" { + body["subject_status"] = defaultSubjectStatus + } + groupStateRef, _ := body["group_state_ref"].(map[string]any) + if len(groupStateRef) == 0 { + groupStateRef = map[string]any{"group_did": groupDID} + body["group_state_ref"] = groupStateRef + } + if cryptoGroupID := stringFromAny(body["crypto_group_id_b64u"]); cryptoGroupID != "" { + groupStateRef["crypto_group_id_b64u"] = cryptoGroupID + } + if fromEpoch := stringFromAny(body["from_epoch"]); fromEpoch != "" { + groupStateRef["epoch"] = fromEpoch + } + return body +} + +func e2eeRecoveryCommitBody(groupDID string, memberDID string, deviceID string, prepared map[string]any, leasedPackage map[string]any) map[string]any { + body := map[string]any{ + "group_did": groupDID, + "group_state_ref": map[string]any{ + "group_did": groupDID, + }, + "target": map[string]any{ + "agent_did": memberDID, + "device_id": defaultString(strings.TrimSpace(deviceID), "default"), + }, + } + for _, key := range []string{ + "crypto_group_id_b64u", + "epoch", + "epoch_authenticator", + "epoch_authenticator_b64u", + "suite", + "last_handshake_digest", + "pending_commit_id", + "operation_id", + "commit_b64u", + "welcome_b64u", + "ratchet_tree_b64u", + "group_info_b64u", + "from_epoch", + "to_epoch", + "old_generation_id", + "new_generation_id", + } { + if value, ok := prepared[key]; ok { + body[key] = value + } + } + if _, ok := body["epoch"]; !ok { + if value, ok := prepared["to_epoch"]; ok { + body["epoch"] = value + } + } + if _, ok := body["epoch_authenticator"]; !ok { + if value, ok := prepared["epoch_authenticator_b64u"]; ok { + body["epoch_authenticator"] = value + } + } + keyPackageID := firstNonEmptyString( + stringFromAny(prepared["recovery_key_package_id"]), + stringFromAny(prepared["key_package_id"]), + stringFromAny(leasedPackage["key_package_id"]), + ) + if keyPackageID != "" { + body["recovery_key_package_id"] = keyPackageID + } + if groupKeyPackage, ok := leasedPackage["group_key_package"].(map[string]any); ok && len(groupKeyPackage) > 0 { + body["group_key_package"] = sanitizeGroupKeyPackageForService(groupKeyPackage) + } + groupStateRef, _ := body["group_state_ref"].(map[string]any) + if len(groupStateRef) == 0 { + groupStateRef = map[string]any{"group_did": groupDID} + body["group_state_ref"] = groupStateRef + } + if cryptoGroupID := stringFromAny(body["crypto_group_id_b64u"]); cryptoGroupID != "" { + groupStateRef["crypto_group_id_b64u"] = cryptoGroupID + } + if fromEpoch := stringFromAny(body["from_epoch"]); fromEpoch != "" { + groupStateRef["epoch"] = fromEpoch + } + return body +} + func boolPtr(value bool) *bool { return &value } diff --git a/internal/message/group_wire_test.go b/internal/message/group_wire_test.go index a176862..3689495 100644 --- a/internal/message/group_wire_test.go +++ b/internal/message/group_wire_test.go @@ -3,6 +3,7 @@ package message import ( "testing" + "github.com/agentconnect/awiki-cli/internal/anpsdk" "github.com/agentconnect/awiki-cli/internal/identity" ) @@ -138,6 +139,473 @@ func TestBuildGroupCreateRPCParamsAppliesDefaultPolicyContract(t *testing.T) { } } +func TestBuildGroupCreateRPCParamsAppliesGroupE2EEProfile(t *testing.T) { + t.Parallel() + + record := testStoredIdentity(t) + params, err := BuildGroupCreateRPCParams( + record, + nil, + "did:wba:awiki.ai:services:message:e1_service", + GroupCreateRequest{Name: "Encrypted Group", E2EE: true}, + ) + if err != nil { + t.Fatalf("BuildGroupCreateRPCParams() error = %v", err) + } + body := mustMapValue(t, params["body"], "params.body") + policy := mustMapValue(t, body["group_policy"], "body.group_policy") + if got := stringFromAny(policy["message_security_profile"]); got != GroupE2EESecurityProfile { + t.Fatalf("message_security_profile = %q, want %q", got, GroupE2EESecurityProfile) + } + if got := stringFromAny(policy["bootstrap_security_profile"]); got != GroupE2EESecurityProfile { + t.Fatalf("bootstrap_security_profile = %q, want %q", got, GroupE2EESecurityProfile) + } +} + +func TestBuildGroupE2EESendRPCParamsSendsOnlyOpaqueCipherObject(t *testing.T) { + t.Parallel() + + record := testStoredIdentity(t) + params, err := BuildGroupE2EESendRPCParams(record, nil, "did:wba:awiki.ai:groups:demo:e1_group", map[string]any{ + "crypto_group_id_b64u": "Y3J5cHRv", + "openmls_group_id_b64u": "provider-local", + "epoch": "1", + "private_message_b64u": "Y2lwaGVy", + "epoch_authenticator": "YXV0aA", + "group_state_ref": map[string]any{"group_did": "did:wba:awiki.ai:groups:demo:e1_group"}, + "application_plaintext": map[string]any{"text": "secret"}, + }, "op-e2ee-send", "msg-e2ee-send") + if err != nil { + t.Fatalf("BuildGroupE2EESendRPCParams() error = %v", err) + } + meta := mustMapValue(t, params["meta"], "params.meta") + if got := stringFromAny(meta["profile"]); got != GroupE2EEProfile { + t.Fatalf("meta.profile = %q, want %q", got, GroupE2EEProfile) + } + if got := stringFromAny(meta["security_profile"]); got != GroupE2EESecurityProfile { + t.Fatalf("meta.security_profile = %q, want %q", got, GroupE2EESecurityProfile) + } + if got := stringFromAny(meta["content_type"]); got != "application/anp-group-cipher+json" { + t.Fatalf("meta.content_type = %q, want group cipher", got) + } + if got := stringFromAny(meta["operation_id"]); got != "op-e2ee-send" { + t.Fatalf("meta.operation_id = %q, want op-e2ee-send", got) + } + if got := stringFromAny(meta["message_id"]); got != "msg-e2ee-send" { + t.Fatalf("meta.message_id = %q, want msg-e2ee-send", got) + } + body := mustMapValue(t, params["body"], "params.body") + if _, ok := body["application_plaintext"]; ok { + t.Fatalf("plaintext leaked into E2EE send body: %#v", body) + } + groupCipher := mustMapValue(t, body["group_cipher_object"], "body.group_cipher_object") + if _, ok := body["group_cipher_object"]; !ok { + t.Fatalf("group_cipher_object missing: %#v", body) + } + if _, ok := groupCipher["openmls_group_id_b64u"]; ok { + t.Fatalf("provider-local OpenMLS group id leaked into service body: %#v", groupCipher) + } + if _, ok := groupCipher["application_plaintext"]; ok { + t.Fatalf("plaintext leaked into service cipher object: %#v", groupCipher) + } + if got := stringFromAny(groupCipher["crypto_group_id_b64u"]); got != "Y3J5cHRv" { + t.Fatalf("crypto_group_id_b64u = %q, want Y3J5cHRv", got) + } +} + +func TestBuildGroupE2EEAddRPCParamsIncludesConsumedKeyPackageID(t *testing.T) { + t.Parallel() + + record := testStoredIdentity(t) + params, err := BuildGroupE2EEAddRPCParams(record, nil, "did:wba:awiki.ai:groups:demo:e1_group", "did:wba:awiki.ai:user:bob:e1_bob", map[string]any{ + "crypto_group_id_b64u": "Y3J5cHRv", + "epoch": "2", + "epoch_authenticator": "YXV0aDI", + "welcome_b64u": "d2VsY29tZQ", + "commit_b64u": "Y29tbWl0", + "ratchet_tree_b64u": "cmF0Y2hldA", + "key_package_id": "kp-bob-1", + "group_key_package": map[string]any{"owner_did": "did:wba:awiki.ai:user:bob:e1_bob", "key_package_id": "kp-bob-1", "device_id": "phone"}, + }) + if err != nil { + t.Fatalf("BuildGroupE2EEAddRPCParams() error = %v", err) + } + body := mustMapValue(t, params["body"], "params.body") + if got := stringFromAny(body["subject_did"]); got != "did:wba:awiki.ai:user:bob:e1_bob" { + t.Fatalf("subject_did = %q, want bob", got) + } + if got := stringFromAny(body["member_did"]); got != "did:wba:awiki.ai:user:bob:e1_bob" { + t.Fatalf("member_did = %q, want bob", got) + } + if got := stringFromAny(body["key_package_id"]); got != "kp-bob-1" { + t.Fatalf("key_package_id = %q, want leased id", got) + } + groupKeyPackage := mustMapValue(t, body["group_key_package"], "body.group_key_package") + if got := stringFromAny(groupKeyPackage["device_id"]); got != "phone" { + t.Fatalf("group_key_package.device_id = %q, want phone", got) + } + if got := stringFromAny(body["subject_key_package_id"]); got != "kp-bob-1" { + t.Fatalf("subject_key_package_id = %q, want leased id", got) + } + if got := stringFromAny(body["ratchet_tree_b64u"]); got != "cmF0Y2hldA" { + t.Fatalf("ratchet_tree_b64u = %q, want ratchet tree", got) + } +} + +func TestBuildGroupE2EERemoveRPCParamsUsesHiddenCompositeMutation(t *testing.T) { + t.Parallel() + + record := testStoredIdentity(t) + params, err := BuildGroupE2EERemoveRPCParams(record, nil, "did:wba:awiki.ai:groups:demo:e1_group", "did:wba:awiki.ai:user:bob:e1_bob", map[string]any{ + "pending_commit_id": "pc-remove-1", + "operation_id": "op-remove-1", + "crypto_group_id_b64u": "Y3J5cHRv", + "from_epoch": "4", + "to_epoch": "5", + "commit_b64u": "Y29tbWl0", + "ratchet_tree_b64u": "cmF0Y2hldA", + "group_info_b64u": "Z3JvdXBpbmZv", + "epoch_authenticator_b64u": "YXV0aDU", + "application_plaintext": "must-not-leak", + "provider_private_material": "must-not-leak", + "group_state_ref": map[string]any{"group_did": "did:wba:awiki.ai:groups:demo:e1_group", "epoch": "4"}, + }, "cleanup", "leave-req-1") + if err != nil { + t.Fatalf("BuildGroupE2EERemoveRPCParams() error = %v", err) + } + meta := mustMapValue(t, params["meta"], "params.meta") + if got := stringFromAny(meta["profile"]); got != GroupE2EEProfile { + t.Fatalf("meta.profile = %q, want %q", got, GroupE2EEProfile) + } + if got := stringFromAny(meta["security_profile"]); got != GroupE2EESecurityProfile { + t.Fatalf("meta.security_profile = %q, want %q", got, GroupE2EESecurityProfile) + } + if got := stringFromAny(meta["operation_id"]); got != "op-remove-1" { + t.Fatalf("meta.operation_id = %q, want prepared operation id", got) + } + target := mustMapValue(t, meta["target"], "meta.target") + if got := stringFromAny(target["kind"]); got != "group" { + t.Fatalf("target.kind = %q, want group", got) + } + body := mustMapValue(t, params["body"], "params.body") + if got := stringFromAny(body["subject_did"]); got != "did:wba:awiki.ai:user:bob:e1_bob" { + t.Fatalf("subject_did = %q, want bob", got) + } + if got := stringFromAny(body["subject_status"]); got != "removed" { + t.Fatalf("subject_status = %q, want removed", got) + } + if got := stringFromAny(body["commit_b64u"]); got != "Y29tbWl0" { + t.Fatalf("commit_b64u = %q, want opaque commit", got) + } + if got := stringFromAny(body["leave_request_id"]); got != "leave-req-1" { + t.Fatalf("leave_request_id = %q, want leave-req-1", got) + } + if _, ok := body["application_plaintext"]; ok { + t.Fatalf("plaintext leaked into remove body: %#v", body) + } + if _, ok := body["provider_private_material"]; ok { + t.Fatalf("provider private material leaked into remove body: %#v", body) + } + ref := mustMapValue(t, body["group_state_ref"], "body.group_state_ref") + if got := stringFromAny(ref["epoch"]); got != "4" { + t.Fatalf("group_state_ref.epoch = %q, want from_epoch/pre-remove epoch", got) + } +} + +func TestBuildGroupE2EELeaveRequestRPCParamsUsesHiddenTransportProtectedControlPlane(t *testing.T) { + t.Parallel() + + record := testStoredIdentity(t) + params, err := BuildGroupE2EELeaveRequestRPCParams(record, nil, "did:wba:awiki.ai:groups:demo:e1_group", "done for now") + if err != nil { + t.Fatalf("BuildGroupE2EELeaveRequestRPCParams() error = %v", err) + } + meta := mustMapValue(t, params["meta"], "params.meta") + if got := stringFromAny(meta["profile"]); got != GroupE2EEProfile { + t.Fatalf("meta.profile = %q, want %q", got, GroupE2EEProfile) + } + if got := stringFromAny(meta["security_profile"]); got != GroupE2EETransportProfile { + t.Fatalf("meta.security_profile = %q, want transport protected", got) + } + if _, ok := meta["message_id"]; ok { + t.Fatalf("leave_request meta must not look like a public message: %#v", meta) + } + body := mustMapValue(t, params["body"], "params.body") + if got := stringFromAny(body["subject_did"]); got != record.DID { + t.Fatalf("subject_did = %q, want actor DID", got) + } + if got := stringFromAny(body["subject_status"]); got != "leave_requested" { + t.Fatalf("subject_status = %q, want leave_requested", got) + } + if got := stringFromAny(body["reason_text"]); got != "done for now" { + t.Fatalf("reason_text = %q, want reason", got) + } +} + +func TestBuildGroupE2EELeaveRPCParamsUsesActorAsSubject(t *testing.T) { + t.Parallel() + + record := testStoredIdentity(t) + params, err := BuildGroupE2EELeaveRPCParams(record, nil, "did:wba:awiki.ai:groups:demo:e1_group", map[string]any{ + "operation_id": "op-leave-1", + "pending_commit_id": "pc-leave-1", + "crypto_group_id_b64u": "Y3J5cHRv", + "from_epoch": "5", + "to_epoch": "6", + "commit_b64u": "Y29tbWl0LWxlYXZl", + }) + if err != nil { + t.Fatalf("BuildGroupE2EELeaveRPCParams() error = %v", err) + } + body := mustMapValue(t, params["body"], "params.body") + if got := stringFromAny(body["subject_did"]); got != record.DID { + t.Fatalf("subject_did = %q, want actor DID", got) + } + if got := stringFromAny(body["member_did"]); got != record.DID { + t.Fatalf("member_did = %q, want actor DID", got) + } + if got := stringFromAny(body["subject_status"]); got != "left" { + t.Fatalf("subject_status = %q, want left", got) + } + if got := stringFromAny(body["epoch"]); got != "6" { + t.Fatalf("epoch = %q, want to_epoch", got) + } +} + +func TestBuildGroupE2EEPublishKeyPackageRPCParamsStripsProviderOnlyFields(t *testing.T) { + t.Parallel() + + record := testStoredIdentity(t) + params, err := BuildGroupE2EEPublishKeyPackageRPCParams(record, nil, "did:wba:awiki.ai:services:message:e1_service", map[string]any{ + "group_key_package": map[string]any{ + "owner_did": record.DID, + "key_package_id": "kp-bob-main", + "suite": "MLS_128_DHKEMX25519_AES128GCM_SHA256_Ed25519", + "mls_key_package_b64u": "a3A", + "did_wba_binding": map[string]any{"agent_did": record.DID}, + "device_id": "bob-main", + "purpose": "recovery", + "group_did": "did:wba:awiki.ai:groups:demo:e1_group", + "private_key_package_b64u": "must-not-leak", + }, + }) + if err != nil { + t.Fatalf("BuildGroupE2EEPublishKeyPackageRPCParams() error = %v", err) + } + body := mustMapValue(t, params["body"], "params.body") + groupKeyPackage := mustMapValue(t, body["group_key_package"], "body.group_key_package") + if got := stringFromAny(groupKeyPackage["device_id"]); got != "bob-main" { + t.Fatalf("device_id = %q, want public device binding", got) + } + if _, ok := groupKeyPackage["private_key_package_b64u"]; ok { + t.Fatalf("private provider field leaked into service KeyPackage payload: %#v", groupKeyPackage) + } + if got := stringFromAny(groupKeyPackage["key_package_id"]); got != "kp-bob-main" { + t.Fatalf("key_package_id = %q, want kp-bob-main", got) + } + if got := stringFromAny(groupKeyPackage["purpose"]); got != "recovery" { + t.Fatalf("purpose = %q, want recovery", got) + } + if got := stringFromAny(groupKeyPackage["group_did"]); got != "did:wba:awiki.ai:groups:demo:e1_group" { + t.Fatalf("group_did = %q, want recovery group DID", got) + } + if _, ok := params["auth"]; !ok { + t.Fatalf("auth missing from publish params: %#v", params) + } + meta := mustMapValue(t, params["meta"], "params.meta") + if got := stringFromAny(meta["security_profile"]); got != "transport-protected" { + t.Fatalf("publish security_profile = %q, want transport-protected", got) + } +} + +func TestSanitizeGroupKeyPackageForServiceOmitsEmptyOptionalRecoveryFields(t *testing.T) { + t.Parallel() + + got := sanitizeGroupKeyPackageForService(map[string]any{ + "owner_did": "did:wba:awiki.ai:user:bob:e1_bob", + "device_id": "bob-main", + "key_package_id": "kp-bob-main", + "suite": "MLS_128_DHKEMX25519_AES128GCM_SHA256_Ed25519", + "mls_key_package_b64u": "a3A", + "did_wba_binding": map[string]any{"agent_did": "did:wba:awiki.ai:user:bob:e1_bob"}, + "purpose": "", + "group_did": "", + }) + if _, ok := got["purpose"]; ok { + t.Fatalf("empty purpose must not be sent to service: %#v", got) + } + if _, ok := got["group_did"]; ok { + t.Fatalf("empty group_did must not be sent to service: %#v", got) + } +} + +func TestSignGroupKeyPackageDIDWBABindingAddsStrictObjectProof(t *testing.T) { + t.Parallel() + + record := testStoredIdentity(t) + providerResult := map[string]any{ + "group_key_package": map[string]any{ + "owner_did": record.DID, + "device_id": "alice-main", + "key_package_id": "kp-alice-main", + "suite": "MLS_128_DHKEMX25519_AES128GCM_SHA256_Ed25519", + "mls_key_package_b64u": "a3A", + "did_wba_binding": map[string]any{ + "agent_did": record.DID, + "verification_method": record.DID + "#provider-placeholder", + "leaf_signature_key_b64u": "MDEyMzQ1Njc4OWFiY2RlZjAxMjM0NTY3ODlhYmNkZWY", + "issued_at": "2026-01-01T00:00:00Z", + "expires_at": "2099-01-01T00:00:00Z", + }, + }, + } + + signedResult, err := signGroupKeyPackageDIDWBABinding(record, providerResult) + if err != nil { + t.Fatalf("signGroupKeyPackageDIDWBABinding() error = %v", err) + } + groupKeyPackage := mustMapValue(t, signedResult["group_key_package"], "signed.group_key_package") + binding := mustMapValue(t, groupKeyPackage["did_wba_binding"], "signed.did_wba_binding") + if got := stringFromAny(binding["verification_method"]); got != verificationMethodID(record.DIDDocument) { + t.Fatalf("verification_method = %q, want active identity method", got) + } + proof := mustMapValue(t, binding["proof"], "did_wba_binding.proof") + if got := stringFromAny(proof["verificationMethod"]); got != verificationMethodID(record.DIDDocument) { + t.Fatalf("proof.verificationMethod = %q, want active identity method", got) + } + if got := stringFromAny(proof["proofValue"]); got == "" || got[0] != 'z' { + t.Fatalf("proofValue = %q, want multibase z proof", got) + } + if err := anpsdk.VerifyDidWbaBinding(binding, record.DIDDocument, anpsdk.DidWbaBindingVerificationOptions{ + Now: "2026-01-02T00:00:00Z", + ExpectedLeafSignatureKey: "MDEyMzQ1Njc4OWFiY2RlZjAxMjM0NTY3ODlhYmNkZWY", + ExpectedCredentialIdentity: record.DID, + }); err != nil { + t.Fatalf("signed did_wba_binding did not verify: %v", err) + } + originalGroupKeyPackage := mustMapValue(t, providerResult["group_key_package"], "provider.group_key_package") + originalBinding := mustMapValue(t, originalGroupKeyPackage["did_wba_binding"], "provider.did_wba_binding") + if _, ok := originalBinding["proof"]; ok { + t.Fatalf("signing should not mutate provider result: %#v", originalBinding) + } +} + +func TestBuildGroupE2EECreateRPCParamsUsesServiceTarget(t *testing.T) { + t.Parallel() + + record := testStoredIdentity(t) + params, err := BuildGroupE2EECreateRPCParams(record, nil, "did:wba:awiki.ai:services:message:e1_service", "did:wba:awiki.ai:groups:demo:e1_group", map[string]any{ + "crypto_group_id_b64u": "Y3J5cHRv", + "epoch": "0", + "epoch_authenticator": "YXV0aA", + }) + if err != nil { + t.Fatalf("BuildGroupE2EECreateRPCParams() error = %v", err) + } + meta := mustMapValue(t, params["meta"], "params.meta") + target := mustMapValue(t, meta["target"], "meta.target") + if got := stringFromAny(target["kind"]); got != "service" { + t.Fatalf("create target.kind = %q, want service", got) + } + if got := stringFromAny(target["did"]); got != "did:wba:awiki.ai:services:message:e1_service" { + t.Fatalf("create target.did = %q, want service DID", got) + } + body := mustMapValue(t, params["body"], "params.body") + if got := stringFromAny(body["group_did"]); got != "did:wba:awiki.ai:groups:demo:e1_group" { + t.Fatalf("body.group_did = %q, want group DID", got) + } + ref := mustMapValue(t, body["group_state_ref"], "body.group_state_ref") + if got := stringFromAny(ref["group_did"]); got != "did:wba:awiki.ai:groups:demo:e1_group" { + t.Fatalf("group_state_ref.group_did = %q, want group DID", got) + } +} + +func TestBuildGroupE2EENoticeRPCParamsUsesTransportProtectedAgentTarget(t *testing.T) { + t.Parallel() + + record := testStoredIdentity(t) + params, err := BuildGroupE2EENoticeRPCParams(record, nil, "did:wba:awiki.ai:groups:demo:e1_group", 500, true, []string{"notice-1"}) + if err != nil { + t.Fatalf("BuildGroupE2EENoticeRPCParams() error = %v", err) + } + meta := mustMapValue(t, params["meta"], "params.meta") + if got := stringFromAny(meta["security_profile"]); got != GroupE2EETransportProfile { + t.Fatalf("notice security_profile = %q, want transport-protected", got) + } + target := mustMapValue(t, meta["target"], "meta.target") + if got := stringFromAny(target["kind"]); got != "agent" { + t.Fatalf("notice target.kind = %q, want agent", got) + } + if got := stringFromAny(target["did"]); got != record.DID { + t.Fatalf("notice target.did = %q, want identity DID", got) + } + body := mustMapValue(t, params["body"], "params.body") + if got := intValueFromAny(body["limit"], 0); got != 100 { + t.Fatalf("notice limit = %d, want capped 100", got) + } + if got := stringFromAny(body["group_did"]); got != "did:wba:awiki.ai:groups:demo:e1_group" { + t.Fatalf("notice group_did = %q, want group DID", got) + } + ids, ok := body["notice_ids"].([]string) + if !ok || len(ids) != 1 || ids[0] != "notice-1" { + t.Fatalf("notice_ids = %#v, want [notice-1]", body["notice_ids"]) + } + if got := boolFromAny(body["mark_delivered"]); !got { + t.Fatalf("mark_delivered = %v, want true", got) + } +} + +func TestBuildGroupE2EEHeadRPCParamsUsesTransportProtectedGroupTarget(t *testing.T) { + t.Parallel() + + record := testStoredIdentity(t) + groupDID := "did:wba:awiki.ai:groups:demo:e1_group" + params, err := BuildGroupE2EEHeadRPCParams(record, nil, groupDID) + if err != nil { + t.Fatalf("BuildGroupE2EEHeadRPCParams() error = %v", err) + } + meta := mustMapValue(t, params["meta"], "params.meta") + if got := stringFromAny(meta["profile"]); got != GroupE2EEProfile { + t.Fatalf("head profile = %q, want group E2EE profile", got) + } + if got := stringFromAny(meta["security_profile"]); got != GroupE2EETransportProfile { + t.Fatalf("head security_profile = %q, want transport-protected", got) + } + target := mustMapValue(t, meta["target"], "meta.target") + if got := stringFromAny(target["kind"]); got != "group" { + t.Fatalf("head target.kind = %q, want group", got) + } + if got := stringFromAny(target["did"]); got != groupDID { + t.Fatalf("head target.did = %q, want group DID", got) + } + body := mustMapValue(t, params["body"], "params.body") + if got := stringFromAny(body["group_did"]); got != groupDID { + t.Fatalf("body.group_did = %q, want group DID", got) + } + ref := mustMapValue(t, body["group_state_ref"], "body.group_state_ref") + if got := stringFromAny(ref["group_did"]); got != groupDID { + t.Fatalf("group_state_ref.group_did = %q, want group DID", got) + } +} + +func TestBuildGroupE2EEGetKeyPackageUsesTransportProtectedServiceTarget(t *testing.T) { + t.Parallel() + + record := testStoredIdentity(t) + params, err := BuildGroupE2EEGetKeyPackageRPCParams(record, nil, "did:wba:awiki.ai:services:message:e1_service", "did:wba:awiki.ai:users:bob:e1_bob") + if err != nil { + t.Fatalf("BuildGroupE2EEGetKeyPackageRPCParams() error = %v", err) + } + meta := mustMapValue(t, params["meta"], "params.meta") + if got := stringFromAny(meta["security_profile"]); got != "transport-protected" { + t.Fatalf("get security_profile = %q, want transport-protected", got) + } + target := mustMapValue(t, meta["target"], "meta.target") + if got := stringFromAny(target["kind"]); got != "service" { + t.Fatalf("get target.kind = %q, want service", got) + } +} + func TestBuildGroupMembersRPCParamsDefaultsLimitToHundred(t *testing.T) { t.Parallel() @@ -184,3 +652,85 @@ func TestBuildGroupMessagesRPCParamsDefaultsLimitToFifty(t *testing.T) { t.Fatalf("body.skip should be absent when skip is zero: %#v", body) } } + +func testStoredIdentity(t *testing.T) *identity.StoredIdentity { + t.Helper() + generated, err := identity.GenerateIdentity(identity.GenerateOptions{ + Hostname: "awiki.ai", + PathPrefix: []string{"user"}, + ProofDomain: "awiki.ai", + }) + if err != nil { + t.Fatalf("GenerateIdentity() error = %v", err) + } + return &identity.StoredIdentity{ + IdentityName: "alice", + DID: generated.DID, + DIDDocument: generated.DIDDocument, + Key1PrivatePEM: generated.Key1PrivatePEM, + } +} + +func TestBuildGroupE2EERecoverMemberRPCParamsAvoidsP4MembershipFields(t *testing.T) { + t.Parallel() + + record := testStoredIdentity(t) + params, err := BuildGroupE2EERecoverMemberRPCParams(record, nil, "did:wba:awiki.ai:groups:demo:e1_group", "did:wba:awiki.ai:user:bob:e1_bob", "bob-main", map[string]any{ + "operation_id": "op-recover-1", + "pending_commit_id": "pc-recover-1", + "crypto_group_id_b64u": "Y3J5cHRv", + "from_epoch": "5", + "to_epoch": "6", + "commit_b64u": "Y29tbWl0", + "welcome_b64u": "d2VsY29tZQ", + "ratchet_tree_b64u": "cmF0Y2hldA", + "epoch_authenticator_b64u": "YXV0aDY", + "application_plaintext": "must-not-leak", + "member_did": "must-not-be-forwarded", + }, map[string]any{ + "key_package_id": "kp-recovery-1", + "group_key_package": map[string]any{ + "owner_did": "did:wba:awiki.ai:user:bob:e1_bob", + "purpose": "recovery", + "device_id": "bob-main", + "private_key_package_b64u": "must-not-leak", + }, + }) + if err != nil { + t.Fatalf("BuildGroupE2EERecoverMemberRPCParams() error = %v", err) + } + meta := mustMapValue(t, params["meta"], "params.meta") + if got := stringFromAny(meta["operation_id"]); got != "op-recover-1" { + t.Fatalf("operation_id = %q, want prepared operation", got) + } + body := mustMapValue(t, params["body"], "params.body") + if _, ok := body["member_did"]; ok { + t.Fatalf("recover_member must not carry P4 member_did: %#v", body) + } + if _, ok := body["role"]; ok { + t.Fatalf("recover_member must not carry P4 role: %#v", body) + } + target := mustMapValue(t, body["target"], "body.target") + if got := stringFromAny(target["agent_did"]); got != "did:wba:awiki.ai:user:bob:e1_bob" { + t.Fatalf("target.agent_did = %q, want bob", got) + } + if got := stringFromAny(target["device_id"]); got != "bob-main" { + t.Fatalf("target.device_id = %q, want bob-main", got) + } + if got := stringFromAny(body["recovery_key_package_id"]); got != "kp-recovery-1" { + t.Fatalf("recovery_key_package_id = %q, want kp", got) + } + if _, ok := body["application_plaintext"]; ok { + t.Fatalf("plaintext leaked into recovery body: %#v", body) + } + recoveryPackage := mustMapValue(t, body["group_key_package"], "body.group_key_package") + if _, ok := recoveryPackage["private_key_package_b64u"]; ok { + t.Fatalf("private KeyPackage material leaked: %#v", recoveryPackage) + } + if got := stringFromAny(recoveryPackage["purpose"]); got != "recovery" { + t.Fatalf("recovery group_key_package.purpose = %q, want recovery", got) + } + if got := stringFromAny(recoveryPackage["device_id"]); got != "bob-main" { + t.Fatalf("recovery group_key_package.device_id = %q, want bob-main", got) + } +} diff --git a/internal/message/http_client.go b/internal/message/http_client.go index 49efcd1..6ee19c7 100644 --- a/internal/message/http_client.go +++ b/internal/message/http_client.go @@ -151,6 +151,28 @@ func (t *HTTPTransport) SendGroup(ctx context.Context, request SendRequest) (*gr return &result, nil } +func (t *HTTPTransport) SendGroupE2EE(ctx context.Context, groupDID string, cipher map[string]any, operationID string, messageID string) (*groupSendResult, error) { + params, err := BuildGroupE2EESendRPCParams(t.auth.record, nil, groupDID, cipher, operationID, messageID) + if err != nil { + return nil, err + } + meta, _ := params["meta"].(map[string]any) + var result groupSendResult + if err := t.rpcCall(ctx, "group.e2ee.send", params, &result); err != nil { + return nil, err + } + if result.GroupDID == "" { + result.GroupDID = groupDID + } + if result.MessageID == "" { + result.MessageID = stringFromAny(meta["message_id"]) + } + if result.OperationID == "" { + result.OperationID = stringFromAny(meta["operation_id"]) + } + return &result, nil +} + func (t *HTTPTransport) GetInbox(ctx context.Context, request InboxRequest) (map[string]any, error) { params := map[string]any{ "meta": map[string]any{ @@ -225,6 +247,118 @@ func (t *HTTPTransport) CreateGroup(ctx context.Context, request GroupCreateRequ return t.rpcMapCall(ctx, "group.create", params) } +func (t *HTTPTransport) PublishGroupE2EEKeyPackage(ctx context.Context, packageResult map[string]any) (map[string]any, error) { + serviceDID, err := t.GetMessageServiceDID(ctx) + if err != nil { + return nil, err + } + params, err := BuildGroupE2EEPublishKeyPackageRPCParams(t.auth.record, nil, serviceDID, packageResult) + if err != nil { + return nil, err + } + return t.rpcMapCall(ctx, "group.e2ee.publish_key_package", params) +} + +func (t *HTTPTransport) GetGroupE2EEKeyPackage(ctx context.Context, targetDID string) (map[string]any, error) { + serviceDID, err := t.GetMessageServiceDID(ctx) + if err != nil { + return nil, err + } + params, err := BuildGroupE2EEGetKeyPackageRPCParams(t.auth.record, nil, serviceDID, targetDID) + if err != nil { + return nil, err + } + return t.rpcMapCall(ctx, "group.e2ee.get_key_package", params) +} + +func (t *HTTPTransport) GetGroupE2EERecoveryKeyPackage(ctx context.Context, groupDID string, targetDID string, deviceID string) (map[string]any, error) { + serviceDID, err := t.GetMessageServiceDID(ctx) + if err != nil { + return nil, err + } + params, err := BuildGroupE2EEGetRecoveryKeyPackageRPCParams(t.auth.record, nil, serviceDID, groupDID, targetDID, deviceID) + if err != nil { + return nil, err + } + return t.rpcMapCall(ctx, "group.e2ee.get_key_package", params) +} + +func (t *HTTPTransport) CreateGroupE2EE(ctx context.Context, groupDID string, mlsHead map[string]any) (map[string]any, error) { + serviceDID, err := t.GetMessageServiceDID(ctx) + if err != nil { + return nil, err + } + params, err := BuildGroupE2EECreateRPCParams(t.auth.record, nil, serviceDID, groupDID, mlsHead) + if err != nil { + return nil, err + } + return t.rpcMapCall(ctx, "group.e2ee.create", params) +} + +func (t *HTTPTransport) AddGroupE2EE(ctx context.Context, groupDID string, memberDID string, mlsHead map[string]any) (map[string]any, error) { + params, err := BuildGroupE2EEAddRPCParams(t.auth.record, nil, groupDID, memberDID, mlsHead) + if err != nil { + return nil, err + } + return t.rpcMapCall(ctx, "group.e2ee.add", params) +} + +func (t *HTTPTransport) RecoverGroupE2EEMember(ctx context.Context, groupDID string, memberDID string, deviceID string, prepared map[string]any, leasedPackage map[string]any) (map[string]any, error) { + params, err := BuildGroupE2EERecoverMemberRPCParams(t.auth.record, nil, groupDID, memberDID, deviceID, prepared, leasedPackage) + if err != nil { + return nil, err + } + return t.rpcMapCall(ctx, "group.e2ee.recover_member", params) +} + +func (t *HTTPTransport) RemoveGroupE2EE(ctx context.Context, groupDID string, memberDID string, preparedCommit map[string]any, reasonText string, leaveRequestID string) (map[string]any, error) { + params, err := BuildGroupE2EERemoveRPCParams(t.auth.record, nil, groupDID, memberDID, preparedCommit, reasonText, leaveRequestID) + if err != nil { + return nil, err + } + return t.rpcMapCall(ctx, "group.e2ee.remove", params) +} + +func (t *HTTPTransport) CreateGroupE2EELeaveRequest(ctx context.Context, groupDID string, reasonText string) (map[string]any, error) { + params, err := BuildGroupE2EELeaveRequestRPCParams(t.auth.record, nil, groupDID, reasonText) + if err != nil { + return nil, err + } + return t.rpcMapCall(ctx, "group.e2ee.leave_request", params) +} + +func (t *HTTPTransport) LeaveGroupE2EE(ctx context.Context, groupDID string, preparedCommit map[string]any) (map[string]any, error) { + params, err := BuildGroupE2EELeaveRPCParams(t.auth.record, nil, groupDID, preparedCommit) + if err != nil { + return nil, err + } + return t.rpcMapCall(ctx, "group.e2ee.leave", params) +} + +func (t *HTTPTransport) PullGroupE2EENotices(ctx context.Context, groupDID string, limit int, markDelivered bool) (map[string]any, error) { + params, err := BuildGroupE2EENoticeRPCParams(t.auth.record, nil, groupDID, limit, markDelivered, nil) + if err != nil { + return nil, err + } + return t.rpcMapCall(ctx, "group.e2ee.notice", params) +} + +func (t *HTTPTransport) MarkGroupE2EENoticesDelivered(ctx context.Context, groupDID string, noticeIDs []string) (map[string]any, error) { + params, err := BuildGroupE2EENoticeRPCParams(t.auth.record, nil, groupDID, len(noticeIDs), true, noticeIDs) + if err != nil { + return nil, err + } + return t.rpcMapCall(ctx, "group.e2ee.notice", params) +} + +func (t *HTTPTransport) GetGroupE2EEHead(ctx context.Context, groupDID string) (map[string]any, error) { + params, err := BuildGroupE2EEHeadRPCParams(t.auth.record, nil, groupDID) + if err != nil { + return nil, err + } + return t.rpcMapCall(ctx, "group.e2ee.head", params) +} + func (t *HTTPTransport) GetGroupInfo(ctx context.Context, request GroupInfoRequest) (map[string]any, error) { params, err := BuildGroupGetInfoRPCParams(t.auth.record, request) if err != nil { diff --git a/internal/message/http_client_test.go b/internal/message/http_client_test.go index 4587777..836ee53 100644 --- a/internal/message/http_client_test.go +++ b/internal/message/http_client_test.go @@ -301,6 +301,60 @@ func TestHTTPTransportGroupMethodsUseExpectedRPCMethods(t *testing.T) { } }, }, + { + name: "e2ee remove member", + wantMethod: "group.e2ee.remove", + call: func(transport *HTTPTransport) error { + _, err := transport.RemoveGroupE2EE(context.Background(), "did:group", "did:member", map[string]any{ + "operation_id": "op-remove", + "pending_commit_id": "pc-remove", + "crypto_group_id_b64u": "Y3J5cHRv", + "from_epoch": "1", + "to_epoch": "2", + "commit_b64u": "Y29tbWl0", + }, "cleanup", "leave-req-1") + return err + }, + verifyBody: func(t *testing.T, body map[string]any) { + if body["member_did"] != "did:member" || body["commit_b64u"] != "Y29tbWl0" || body["reason_text"] != "cleanup" || body["leave_request_id"] != "leave-req-1" { + t.Fatalf("body = %#v, want member/commit/reason/leave request", body) + } + }, + }, + + { + name: "e2ee leave request", + wantMethod: "group.e2ee.leave_request", + call: func(transport *HTTPTransport) error { + _, err := transport.CreateGroupE2EELeaveRequest(context.Background(), "did:group", "done") + return err + }, + verifyBody: func(t *testing.T, body map[string]any) { + if body["group_did"] != "did:group" || body["subject_status"] != "leave_requested" || body["reason_text"] != "done" { + t.Fatalf("body = %#v, want leave request control body", body) + } + }, + }, + { + name: "e2ee leave group", + wantMethod: "group.e2ee.leave", + call: func(transport *HTTPTransport) error { + _, err := transport.LeaveGroupE2EE(context.Background(), "did:group", map[string]any{ + "operation_id": "op-leave", + "pending_commit_id": "pc-leave", + "crypto_group_id_b64u": "Y3J5cHRv", + "from_epoch": "2", + "to_epoch": "3", + "commit_b64u": "Y29tbWl0LWxlYXZl", + }) + return err + }, + verifyBody: func(t *testing.T, body map[string]any) { + if body["group_did"] != "did:group" || body["subject_status"] != "left" || body["commit_b64u"] != "Y29tbWl0LWxlYXZl" { + t.Fatalf("body = %#v, want group/left/commit", body) + } + }, + }, { name: "list members", wantMethod: "group.list_members", diff --git a/internal/message/secure.go b/internal/message/secure.go new file mode 100644 index 0000000..7816279 --- /dev/null +++ b/internal/message/secure.go @@ -0,0 +1,195 @@ +package message + +import ( + "context" + "fmt" + "path/filepath" + + "github.com/agentconnect/awiki-cli/internal/anpsdk" + appconfig "github.com/agentconnect/awiki-cli/internal/config" + "github.com/agentconnect/awiki-cli/internal/identity" +) + +func (s *Service) sendSecureDirect(ctx context.Context, request SendRequest) (*CommandResult, error) { + record, err := s.requireActiveIdentity(request.IdentityName) + if err != nil { + return nil, err + } + if record.E2EEAgreementPrivatePEM == "" || record.Key1PrivatePEM == "" { + return nil, fmt.Errorf("secure direct messaging requires DID signing and X25519 E2EE private keys") + } + targetDID, targetHandle, err := s.resolveTarget(ctx, request.Target) + if err != nil { + return nil, err + } + transport, warnings, err := s.httpTransport(record) + if err != nil { + return nil, err + } + warnings = append(warnings, s.maybePublishSecurePrekeys(ctx, record)...) + client, err := s.secureE2EEClient(ctx, record, transport) + if err != nil { + return nil, err + } + messageID := "msg-" + generateOperationID() + resultMap, err := client.SendText(ctx, targetDID, request.Text, messageID, messageID) + if err != nil { + if isPendingConfirmationError(err) { + outboxID, queueErr := queueSecureOutboxRecord(ctx, s.resolved, s.manager, record, targetDID, defaultString(request.MessageType, "text"), request.Text) + if queueErr != nil { + return nil, queueErr + } + return &CommandResult{ + Data: map[string]any{ + "action": "queue_secure_message", + "target": map[string]any{ + "did": targetDID, + "handle": targetHandle, + "kind": "direct", + }, + "message": map[string]any{ + "type": defaultString(request.MessageType, "text"), + "secure": true, + "queued": true, + }, + "delivery": map[string]any{ + "delivery_state": "queued", + "outbox_id": outboxID, + "target_did": targetDID, + }, + }, + Summary: "Queued secure direct message pending peer confirmation", + Warnings: warnings, + }, nil + } + return nil, err + } + result := directSendResult{ + Accepted: boolFromAny(resultMap["accepted"]), + MessageID: stringFromAny(resultMap["message_id"]), + OperationID: stringFromAny(resultMap["operation_id"]), + TargetDID: stringFromAny(resultMap["target_did"]), + } + if result.MessageID == "" { + result.MessageID = messageID + } + if result.OperationID == "" { + result.OperationID = messageID + } + if result.TargetDID == "" { + result.TargetDID = targetDID + } + request.Target = targetDID + request.SecureMode = "on" + return s.persistSendResult(ctx, record, targetDID, targetHandle, request, &result, warnings) +} + +func (s *Service) maybePublishSecurePrekeys(ctx context.Context, record *identity.StoredIdentity) []string { + return PublishSecurePrekeys(ctx, s.resolved, s.manager, record) +} + +func (s *Service) secureE2EEClient(ctx context.Context, record *identity.StoredIdentity, transport *HTTPTransport) (*anpsdk.MessageServiceE2EEClient, error) { + return NewSecureE2EEClientForRecord(ctx, s.manager, record, func(method string, params map[string]any) (map[string]any, error) { + return transport.rpcMapCall(ctx, method, params) + }) +} + +// PublishSecurePrekeys ensures one identity has a published secure prekey bundle. +func PublishSecurePrekeys(ctx context.Context, resolved *appconfig.Resolved, manager *identity.Manager, record *identity.StoredIdentity) []string { + if record == nil || record.E2EEAgreementPrivatePEM == "" || record.Key1PrivatePEM == "" { + return nil + } + auth, err := newAuthContext(record, manager) + if err != nil { + return compactWarnings([]string{fmt.Sprintf("Failed to initialize secure prekey auth: %v", err)}) + } + primeAuthSession(auth, resolved) + transport := NewHTTPTransport(resolved, auth, nil) + client, err := NewSecureE2EEClientForRecord(ctx, manager, record, func(method string, params map[string]any) (map[string]any, error) { + return transport.rpcMapCall(ctx, method, params) + }) + if err != nil { + return compactWarnings([]string{fmt.Sprintf("Failed to initialize secure prekey publisher: %v", err)}) + } + if _, err := client.PublishPrekeyBundle(); err != nil { + return compactWarnings([]string{fmt.Sprintf("Failed to publish secure prekeys: %v", err)}) + } + return nil +} + +// NewSecureE2EEClientForRecord creates one direct-e2ee client for the given identity. +func NewSecureE2EEClientForRecord(ctx context.Context, manager *identity.Manager, record *identity.StoredIdentity, rpc func(string, map[string]any) (map[string]any, error)) (*anpsdk.MessageServiceE2EEClient, error) { + _ = ctx + if manager == nil { + return nil, fmt.Errorf("identity manager is required") + } + if record == nil { + return nil, fmt.Errorf("identity record is required") + } + paths, err := manager.PathsForIdentity(record.IdentityName) + if err != nil { + return nil, err + } + signingPrivate, err := anpsdk.PrivateKeyFromPEM(record.Key1PrivatePEM) + if err != nil { + return nil, fmt.Errorf("parse DID signing private key: %w", err) + } + agreementPrivate, err := anpsdk.PrivateKeyFromPEM(record.E2EEAgreementPrivatePEM) + if err != nil { + return nil, fmt.Errorf("parse E2EE agreement private key: %w", err) + } + sessionStore, err := anpsdk.NewFileSessionStore(filepath.Join(paths.IdentityDir, "p5-e2ee-sessions")) + if err != nil { + return nil, err + } + signedPrekeyStore, err := anpsdk.NewFileSignedPrekeyStore(filepath.Join(paths.IdentityDir, "p5-signed-prekeys")) + if err != nil { + return nil, err + } + oneTimePrekeyStore, err := anpsdk.NewFileOneTimePrekeyStore(filepath.Join(paths.IdentityDir, "p5-one-time-prekeys")) + if err != nil { + return nil, err + } + resolver := func(ctx context.Context, did string) (map[string]any, error) { + if did == record.DID && record.DIDDocument != nil { + return record.DIDDocument, nil + } + if localDocument, ok := loadLocalDIDDocument(manager, did); ok { + return localDocument, nil + } + return anpsdk.ResolveDidDocument(ctx, did, true) + } + return anpsdk.NewMessageServiceDirectE2eeClient( + record.DID, + signingPrivate, + record.DID+"#key-1", + agreementPrivate, + record.DID+"#key-3", + rpc, + resolver, + sessionStore, + signedPrekeyStore, + oneTimePrekeyStore, + ) +} + +func loadLocalDIDDocument(manager *identity.Manager, did string) (map[string]any, bool) { + if manager == nil || did == "" { + return nil, false + } + summaries, err := manager.List() + if err != nil { + return nil, false + } + for _, summary := range summaries { + if summary.DID != did { + continue + } + record, err := manager.Load(summary.IdentityName) + if err != nil || record == nil || record.DIDDocument == nil { + return nil, false + } + return record.DIDDocument, true + } + return nil, false +} diff --git a/internal/message/secure_commands.go b/internal/message/secure_commands.go new file mode 100644 index 0000000..f8f0b71 --- /dev/null +++ b/internal/message/secure_commands.go @@ -0,0 +1,392 @@ +package message + +import ( + "context" + "encoding/json" + "fmt" + "os" + "path/filepath" + "sort" + "strings" + + "github.com/agentconnect/awiki-cli/internal/anpsdk" + "github.com/agentconnect/awiki-cli/internal/identity" + "github.com/agentconnect/awiki-cli/internal/store" +) + +func (s *Service) SecureStatus(ctx context.Context, request SecureStatusRequest) (*CommandResult, error) { + record, err := s.requireActiveIdentity(request.IdentityName) + if err != nil { + return nil, err + } + peerDID := "" + peerHandle := "" + if strings.TrimSpace(request.With) != "" { + peerDID, peerHandle, err = s.resolveTarget(ctx, request.With) + if err != nil { + return nil, err + } + } + sessions, err := s.listSecureSessions(record, peerDID) + if err != nil { + return nil, err + } + db, err := store.Open(s.resolved.Paths) + if err != nil { + return nil, err + } + defer db.Close() + if err := store.EnsureSchema(ctx, db); err != nil { + return nil, err + } + outboxRows, err := store.ListE2EEOutbox(ctx, db, record.DID, record.IdentityName, "") + if err != nil { + return nil, err + } + if peerDID != "" { + filtered := make([]map[string]any, 0, len(outboxRows)) + for _, row := range outboxRows { + if stringFromAny(row["peer_did"]) == peerDID { + filtered = append(filtered, row) + } + } + outboxRows = filtered + } + outboxSummary := map[string]int{} + for _, row := range outboxRows { + status := defaultString(stringFromAny(row["local_status"]), "unknown") + outboxSummary[status]++ + } + return &CommandResult{ + Data: map[string]any{ + "with": peerHandleOrDid(peerHandle, peerDID), + "sessions": sessions, + "outbox": map[string]any{ + "total": len(outboxRows), + "by_status": outboxSummary, + "records": redactSecureOutboxRowsForStatus(outboxRows), + }, + }, + Summary: fmt.Sprintf("Loaded %d secure session(s) and %d secure outbox record(s)", len(sessions), len(outboxRows)), + }, nil +} + +func (s *Service) SecureFailed(ctx context.Context, request SecureStatusRequest) (*CommandResult, error) { + record, err := s.requireActiveIdentity(request.IdentityName) + if err != nil { + return nil, err + } + db, err := store.Open(s.resolved.Paths) + if err != nil { + return nil, err + } + defer db.Close() + if err := store.EnsureSchema(ctx, db); err != nil { + return nil, err + } + rows, err := store.ListE2EEOutbox(ctx, db, record.DID, record.IdentityName, "failed") + if err != nil { + return nil, err + } + return &CommandResult{ + Data: map[string]any{ + "failed": rows, + "total": len(rows), + }, + Summary: fmt.Sprintf("Loaded %d failed secure outbox record(s)", len(rows)), + }, nil +} + +func (s *Service) SecureDrop(ctx context.Context, request SecureOutboxActionRequest) (*CommandResult, error) { + record, err := s.requireActiveIdentity(request.IdentityName) + if err != nil { + return nil, err + } + db, err := store.Open(s.resolved.Paths) + if err != nil { + return nil, err + } + defer db.Close() + if err := store.EnsureSchema(ctx, db); err != nil { + return nil, err + } + if _, err := store.GetE2EEOutbox(ctx, db, request.OutboxID, record.DID, record.IdentityName); err != nil { + return nil, err + } + if err := store.UpdateE2EEOutboxStatus(ctx, db, request.OutboxID, record.DID, record.IdentityName, "dropped"); err != nil { + return nil, err + } + return &CommandResult{ + Data: map[string]any{ + "outbox_id": request.OutboxID, + "status": "dropped", + }, + Summary: fmt.Sprintf("Dropped secure outbox record %s", request.OutboxID), + }, nil +} + +func (s *Service) SecureRetry(ctx context.Context, request SecureOutboxActionRequest) (*CommandResult, error) { + record, err := s.requireActiveIdentity(request.IdentityName) + if err != nil { + return nil, err + } + db, err := store.Open(s.resolved.Paths) + if err != nil { + return nil, err + } + defer db.Close() + if err := store.EnsureSchema(ctx, db); err != nil { + return nil, err + } + row, err := store.GetE2EEOutbox(ctx, db, request.OutboxID, record.DID, record.IdentityName) + if err != nil { + return nil, err + } + if err := store.UpdateE2EEOutboxStatus(ctx, db, request.OutboxID, record.DID, record.IdentityName, "queued"); err != nil { + return nil, err + } + warnings := FlushQueuedSecureOutbox(ctx, s.resolved, s.manager, record, stringFromAny(row["peer_did"]), func(method string, params map[string]any) (map[string]any, error) { + transport, _, err := s.httpTransport(record) + if err != nil { + return nil, err + } + return transport.rpcMapCall(ctx, method, params) + }) + row, _ = store.GetE2EEOutbox(ctx, db, request.OutboxID, record.DID, record.IdentityName) + return &CommandResult{ + Data: map[string]any{ + "outbox_id": request.OutboxID, + "record": row, + }, + Summary: fmt.Sprintf("Retried secure outbox record %s", request.OutboxID), + Warnings: warnings, + }, nil +} + +func (s *Service) SecureInit(ctx context.Context, request SecurePeerRequest) (*CommandResult, error) { + record, peerDID, peerHandle, warnings, err := s.prepareSecurePeerAction(ctx, request) + if err != nil { + return nil, err + } + session, ok, err := s.loadSecureSessionState(record, peerDID) + if err != nil { + return nil, err + } + if ok { + return &CommandResult{ + Data: map[string]any{ + "target": map[string]any{"did": peerDID, "handle": peerHandle, "kind": "direct"}, + "session": session, + "reused": true, + }, + Summary: fmt.Sprintf("Secure session already exists for %s", peerHandleOrDid(peerHandle, peerDID)), + Warnings: compactWarnings(warnings), + }, nil + } + transport, httpWarnings, err := s.httpTransport(record) + if err != nil { + return nil, err + } + warnings = append(warnings, httpWarnings...) + client, err := s.secureE2EEClient(ctx, record, transport) + if err != nil { + return nil, err + } + messageID := "secure-init-" + generateOperationID() + resultMap, err := client.SendJSON(ctx, peerDID, BuildSecureInitPayload(), messageID, messageID) + if err != nil { + return nil, err + } + session, _, _ = s.loadSecureSessionState(record, peerDID) + return &CommandResult{ + Data: map[string]any{ + "target": map[string]any{"did": peerDID, "handle": peerHandle, "kind": "direct"}, + "session": session, + "delivery": map[string]any{ + "message_id": defaultString(stringFromAny(resultMap["message_id"]), messageID), + "operation_id": defaultString(stringFromAny(resultMap["operation_id"]), messageID), + "target_did": defaultString(stringFromAny(resultMap["target_did"]), peerDID), + }, + "initialized": true, + }, + Summary: fmt.Sprintf("Initialized secure session with %s", peerHandleOrDid(peerHandle, peerDID)), + Warnings: compactWarnings(warnings), + }, nil +} + +func (s *Service) SecureRepair(ctx context.Context, request SecurePeerRequest) (*CommandResult, error) { + record, peerDID, peerHandle, warnings, err := s.prepareSecurePeerAction(ctx, request) + if err != nil { + return nil, err + } + resetCount, err := s.resetSecurePeerState(ctx, record, peerDID) + if err != nil { + return nil, err + } + initResult, err := s.SecureInit(ctx, request) + if err != nil { + return nil, err + } + initResult.Data["repair"] = map[string]any{ + "peer_did": peerDID, + "peer_handle": peerHandle, + "reset_records": resetCount, + } + initResult.Summary = fmt.Sprintf("Repaired secure session with %s", peerHandleOrDid(peerHandle, peerDID)) + initResult.Warnings = compactWarnings(append(warnings, initResult.Warnings...)) + return initResult, nil +} + +func (s *Service) listSecureSessions(record *identity.StoredIdentity, peerDID string) ([]map[string]any, error) { + paths, err := s.manager.PathsForIdentity(record.IdentityName) + if err != nil { + return nil, err + } + root := filepath.Join(paths.IdentityDir, "p5-e2ee-sessions") + entries, err := filepath.Glob(filepath.Join(root, "*.json")) + if err != nil { + return nil, err + } + result := make([]map[string]any, 0, len(entries)) + for _, path := range entries { + raw, err := os.ReadFile(path) + if err != nil { + return nil, err + } + var session map[string]any + if err := json.Unmarshal(raw, &session); err != nil { + return nil, err + } + if peerDID != "" && stringFromAny(session["peer_did"]) != peerDID { + continue + } + result = append(result, redactSecureSessionForStatus(session)) + } + sort.Slice(result, func(i, j int) bool { + return stringFromAny(result[i]["peer_did"]) < stringFromAny(result[j]["peer_did"]) + }) + return result, nil +} + +func redactSecureSessionForStatus(session map[string]any) map[string]any { + summary := map[string]any{ + "session_id": stringFromAny(session["session_id"]), + "suite": stringFromAny(session["suite"]), + "peer_did": stringFromAny(session["peer_did"]), + "status": stringFromAny(session["status"]), + "is_initiator": boolFromAny(session["is_initiator"]), + "send_n": intValueFromAny(session["send_n"], 0), + "recv_n": intValueFromAny(session["recv_n"], 0), + "previous_send_chain_length": intValueFromAny(session["previous_send_chain_length"], 0), + "skipped_key_count": countStatusArrayItems(session["skipped_message_keys"]), + } + return summary +} + +func countStatusArrayItems(value any) int { + switch typed := value.(type) { + case []any: + return len(typed) + case []map[string]any: + return len(typed) + default: + return 0 + } +} + +func redactSecureOutboxRowsForStatus(rows []map[string]any) []map[string]any { + redacted := make([]map[string]any, 0, len(rows)) + for _, row := range rows { + redacted = append(redacted, map[string]any{ + "outbox_id": stringFromAny(row["outbox_id"]), + "peer_did": stringFromAny(row["peer_did"]), + "session_id": stringFromAny(row["session_id"]), + "original_type": stringFromAny(row["original_type"]), + "local_status": stringFromAny(row["local_status"]), + "attempt_count": intValueFromAny(row["attempt_count"], 0), + "sent_msg_id": stringFromAny(row["sent_msg_id"]), + "sent_server_seq": row["sent_server_seq"], + "last_error_code": stringFromAny(row["last_error_code"]), + "retry_hint": stringFromAny(row["retry_hint"]), + "failed_msg_id": stringFromAny(row["failed_msg_id"]), + "failed_server_seq": row["failed_server_seq"], + "last_attempt_at": stringFromAny(row["last_attempt_at"]), + "created_at": stringFromAny(row["created_at"]), + "updated_at": stringFromAny(row["updated_at"]), + }) + } + return redacted +} + +func (s *Service) prepareSecurePeerAction(ctx context.Context, request SecurePeerRequest) (*identity.StoredIdentity, string, string, []string, error) { + record, err := s.requireActiveIdentity(request.IdentityName) + if err != nil { + return nil, "", "", nil, err + } + if record.E2EEAgreementPrivatePEM == "" || record.Key1PrivatePEM == "" { + return nil, "", "", nil, fmt.Errorf("secure direct messaging requires DID signing and X25519 E2EE private keys") + } + if strings.TrimSpace(request.With) == "" { + return nil, "", "", nil, ErrTargetRequired + } + peerDID, peerHandle, err := s.resolveTarget(ctx, request.With) + if err != nil { + return nil, "", "", nil, err + } + return record, peerDID, peerHandle, s.maybePublishSecurePrekeys(ctx, record), nil +} + +func (s *Service) loadSecureSessionState(record *identity.StoredIdentity, peerDID string) (map[string]any, bool, error) { + sessions, err := s.listSecureSessions(record, peerDID) + if err != nil { + return nil, false, err + } + if len(sessions) == 0 { + return nil, false, nil + } + return sessions[0], true, nil +} + +func (s *Service) resetSecurePeerState(ctx context.Context, record *identity.StoredIdentity, peerDID string) (int, error) { + paths, err := s.manager.PathsForIdentity(record.IdentityName) + if err != nil { + return 0, err + } + sessionStore, err := anpsdk.NewFileSessionStore(filepath.Join(paths.IdentityDir, "p5-e2ee-sessions")) + if err != nil { + return 0, err + } + session, ok, err := sessionStore.FindByPeerDID(peerDID) + if err != nil { + return 0, err + } + resetCount := 0 + if ok { + if err := sessionStore.DeleteSession(session.SessionID); err != nil { + return 0, err + } + resetCount++ + } + db, err := store.Open(s.resolved.Paths) + if err != nil { + return resetCount, err + } + defer db.Close() + if err := store.EnsureSchema(ctx, db); err != nil { + return resetCount, err + } + rows, err := store.ListE2EEOutbox(ctx, db, record.DID, record.IdentityName, "failed") + if err != nil { + return resetCount, err + } + for _, row := range rows { + if stringFromAny(row["peer_did"]) != peerDID { + continue + } + if err := store.UpdateE2EEOutboxStatus(ctx, db, stringFromAny(row["outbox_id"]), record.DID, record.IdentityName, "queued"); err != nil { + return resetCount, err + } + resetCount++ + } + return resetCount, nil +} diff --git a/internal/message/secure_control.go b/internal/message/secure_control.go new file mode 100644 index 0000000..e28c9a4 --- /dev/null +++ b/internal/message/secure_control.go @@ -0,0 +1,210 @@ +package message + +import ( + "context" + "encoding/json" + "fmt" + "path/filepath" + "sort" + "strings" + + "github.com/agentconnect/awiki-cli/internal/anpsdk" + appconfig "github.com/agentconnect/awiki-cli/internal/config" + "github.com/agentconnect/awiki-cli/internal/identity" + "github.com/agentconnect/awiki-cli/internal/store" +) + +const secureAckSystemType = "awiki.direct.secure_ack.v1" +const secureInitSystemType = "awiki.direct.secure_init.v1" + +func BuildSecureAckPayload(sessionID string, ackedMessageID string) map[string]any { + return map[string]any{ + "system_type": secureAckSystemType, + "session_id": strings.TrimSpace(sessionID), + "acked_message_id": strings.TrimSpace(ackedMessageID), + } +} + +func BuildSecureInitPayload() map[string]any { + return map[string]any{ + "system_type": secureInitSystemType, + "reason": "manual_init", + } +} + +func IsSecureAckPlaintext(plaintext map[string]any) bool { + if stringFromAny(plaintext["application_content_type"]) != "application/json" { + return false + } + payload, err := mapFromAny(plaintext["payload"]) + if err != nil { + return false + } + return stringFromAny(payload["system_type"]) == secureAckSystemType +} + +func IsSecureInitPlaintext(plaintext map[string]any) bool { + if stringFromAny(plaintext["application_content_type"]) != "application/json" { + return false + } + payload, err := mapFromAny(plaintext["payload"]) + if err != nil { + return false + } + return stringFromAny(payload["system_type"]) == secureInitSystemType +} + +func secureAckSessionID(plaintext map[string]any) string { + payload, err := mapFromAny(plaintext["payload"]) + if err != nil { + return "" + } + return stringFromAny(payload["session_id"]) +} + +func isPendingConfirmationError(err error) bool { + if err == nil { + return false + } + message := strings.ToLower(err.Error()) + return strings.Contains(message, "pending confirmation") || strings.Contains(message, "pending-confirmation") +} + +func queueSecureOutboxRecord(ctx context.Context, resolved *appconfig.Resolved, manager *identity.Manager, record *identity.StoredIdentity, peerDID string, originalType string, plaintext string) (string, error) { + if record == nil { + return "", fmt.Errorf("identity record is required") + } + db, err := store.Open(resolved.Paths) + if err != nil { + return "", err + } + defer db.Close() + if err := store.EnsureSchema(ctx, db); err != nil { + return "", err + } + return store.QueueE2EEOutbox(ctx, db, store.E2EEOutboxRecord{ + OwnerDID: record.DID, + PeerDID: peerDID, + SessionID: currentSecureSessionID(manager, record, peerDID), + OriginalType: defaultString(originalType, "text"), + Plaintext: plaintext, + LocalStatus: "queued", + CredentialName: record.IdentityName, + Metadata: metadataString(map[string]any{"reason": "pending_confirmation"}), + }) +} + +func currentSecureSessionID(manager *identity.Manager, record *identity.StoredIdentity, peerDID string) string { + if manager == nil || record == nil { + return "" + } + paths, err := manager.PathsForIdentity(record.IdentityName) + if err != nil { + return "" + } + sessionStore, err := anpsdk.NewFileSessionStore(filepath.Join(paths.IdentityDir, "p5-e2ee-sessions")) + if err != nil { + return "" + } + session, ok, err := sessionStore.FindByPeerDID(peerDID) + if err != nil || !ok { + return "" + } + return strings.TrimSpace(session.SessionID) +} + +func FlushQueuedSecureOutbox(ctx context.Context, resolved *appconfig.Resolved, manager *identity.Manager, record *identity.StoredIdentity, peerDID string, rpc func(string, map[string]any) (map[string]any, error)) []string { + if record == nil { + return nil + } + db, err := store.Open(resolved.Paths) + if err != nil { + return compactWarnings([]string{fmt.Sprintf("Failed to open secure outbox store: %v", err)}) + } + defer db.Close() + if err := store.EnsureSchema(ctx, db); err != nil { + return compactWarnings([]string{fmt.Sprintf("Failed to ensure secure outbox schema: %v", err)}) + } + rows, err := store.ListE2EEOutbox(ctx, db, record.DID, record.IdentityName, "queued") + if err != nil { + return compactWarnings([]string{fmt.Sprintf("Failed to list secure outbox: %v", err)}) + } + if len(rows) == 0 { + return nil + } + client, err := NewSecureE2EEClientForRecord(ctx, manager, record, rpc) + if err != nil { + return compactWarnings([]string{fmt.Sprintf("Failed to initialize secure outbox sender: %v", err)}) + } + sort.SliceStable(rows, func(i, j int) bool { + return stringFromAny(rows[i]["created_at"]) < stringFromAny(rows[j]["created_at"]) + }) + warnings := make([]string, 0) + for _, row := range rows { + if strings.TrimSpace(peerDID) != "" && stringFromAny(row["peer_did"]) != strings.TrimSpace(peerDID) { + continue + } + outboxID := stringFromAny(row["outbox_id"]) + targetDID := stringFromAny(row["peer_did"]) + originalType := defaultString(stringFromAny(row["original_type"]), "text") + plaintext := stringFromAny(row["plaintext"]) + if outboxID == "" || targetDID == "" { + continue + } + var resultMap map[string]any + switch originalType { + case "text", "": + resultMap, err = client.SendText(ctx, targetDID, plaintext, outboxID, outboxID) + case "json": + var payload map[string]any + if err := json.Unmarshal([]byte(plaintext), &payload); err != nil { + _ = store.SetE2EEOutboxFailureByID(ctx, db, outboxID, record.DID, record.IdentityName, "invalid_payload", "drop", metadataString(map[string]any{"detail": err.Error()})) + warnings = append(warnings, fmt.Sprintf("Failed to parse queued secure JSON payload %s: %v", outboxID, err)) + continue + } + resultMap, err = client.SendJSON(ctx, targetDID, payload, outboxID, outboxID) + default: + _ = store.SetE2EEOutboxFailureByID(ctx, db, outboxID, record.DID, record.IdentityName, "unsupported_original_type", "drop", metadataString(map[string]any{"original_type": originalType})) + warnings = append(warnings, fmt.Sprintf("Queued secure outbox %s uses unsupported original_type=%s", outboxID, originalType)) + continue + } + if err != nil { + _ = store.SetE2EEOutboxFailureByID(ctx, db, outboxID, record.DID, record.IdentityName, "send_failed", "retry", metadataString(map[string]any{"detail": err.Error()})) + warnings = append(warnings, fmt.Sprintf("Failed to flush queued secure outbox %s: %v", outboxID, err)) + continue + } + sessionID := currentSecureSessionID(manager, record, targetDID) + sentMsgID := stringFromAny(resultMap["message_id"]) + if sentMsgID == "" { + sentMsgID = outboxID + } + metadata := metadataString(map[string]any{ + "target_did": targetDID, + "operation_id": stringFromAny(resultMap["operation_id"]), + "delivery_state": stringFromAny(resultMap["delivery_state"]), + "flushed_from": "queued", + }) + if err := store.MarkE2EEOutboxSent(ctx, db, outboxID, record.DID, sessionID, sentMsgID, nil, metadata); err != nil { + warnings = append(warnings, fmt.Sprintf("Failed to mark secure outbox %s sent: %v", outboxID, err)) + continue + } + if err := store.StoreMessage(ctx, db, store.MessageRecord{ + MsgID: sentMsgID, + OwnerDID: record.DID, + ThreadID: store.MakeThreadID(record.DID, targetDID, ""), + Direction: 1, + SenderDID: record.DID, + ReceiverDID: targetDID, + ContentType: contentTypeForMessageType(originalType), + Content: plaintext, + SentAt: stringFromAny(resultMap["accepted_at"]), + IsRead: true, + IsE2EE: true, + Metadata: metadata, + CredentialName: record.IdentityName, + }); err != nil { + warnings = append(warnings, fmt.Sprintf("Failed to persist flushed secure outbox %s: %v", outboxID, err)) + } + } + return compactWarnings(warnings) +} diff --git a/internal/message/secure_incoming.go b/internal/message/secure_incoming.go new file mode 100644 index 0000000..8456893 --- /dev/null +++ b/internal/message/secure_incoming.go @@ -0,0 +1,186 @@ +package message + +import ( + "context" + "fmt" + "sort" + + "github.com/agentconnect/awiki-cli/internal/anpsdk" + "github.com/agentconnect/awiki-cli/internal/identity" +) + +func isDirectE2EEWireContentType(contentType string) bool { + switch contentType { + case "application/anp-direct-init+json", "application/anp-direct-cipher+json": + return true + default: + return false + } +} + +func (s *Service) maybeDecryptDirectE2EEMessages(ctx context.Context, record *identity.StoredIdentity, messages []map[string]any) []string { + if len(messages) == 0 || !containsDirectE2EEMessages(messages) { + return nil + } + transport, _, err := s.httpTransport(record) + if err != nil { + return compactWarnings([]string{fmt.Sprintf("Failed to initialize secure direct decryptor: %v", err)}) + } + client, err := s.secureE2EEClient(ctx, record, transport) + if err != nil { + return compactWarnings([]string{fmt.Sprintf("Failed to initialize secure direct decryptor: %v", err)}) + } + ordered := append([]map[string]any(nil), messages...) + sort.SliceStable(ordered, func(i, j int) bool { + leftSeq := int64Value(ordered[i]["server_seq"]) + rightSeq := int64Value(ordered[j]["server_seq"]) + if leftSeq == rightSeq { + return stringFromAny(ordered[i]["id"]) < stringFromAny(ordered[j]["id"]) + } + if leftSeq == 0 { + return false + } + if rightSeq == 0 { + return true + } + return leftSeq < rightSeq + }) + warnings := make([]string, 0) + for _, message := range ordered { + contentType := stringFromAny(message["content_type"]) + if !isDirectE2EEWireContentType(contentType) { + continue + } + notification, err := directE2EENotificationFromMessageView(message) + if err != nil { + warnings = append(warnings, fmt.Sprintf("Skipped secure direct message %s: %v", stringFromAny(message["id"]), err)) + continue + } + result, err := client.ProcessIncoming(ctx, notification) + if err != nil { + warnings = append(warnings, fmt.Sprintf("Failed to decrypt secure direct message %s: %v", stringFromAny(message["id"]), err)) + continue + } + warnings = append(warnings, s.maybeAckPollingDirectInit(ctx, record, transport, client, message, result)...) + applyDirectE2EEProcessingResult(message, result) + } + return compactWarnings(warnings) +} + +func (s *Service) maybeAckPollingDirectInit(ctx context.Context, record *identity.StoredIdentity, transport *HTTPTransport, client *anpsdk.MessageServiceE2EEClient, message map[string]any, result map[string]any) []string { + if record == nil || transport == nil || client == nil { + return nil + } + if stringFromAny(message["content_type"]) != "application/anp-direct-init+json" { + return nil + } + if stringFromAny(result["state"]) != "decrypted" { + return nil + } + sessionID := directInitSessionIDFromMessage(message) + messageID := stringFromAny(message["id"]) + peerDID := stringFromAny(message["sender_did"]) + if sessionID == "" || messageID == "" || peerDID == "" || peerDID == record.DID { + return nil + } + ackID := "ack-" + sessionID + if _, err := client.SendJSON(ctx, peerDID, BuildSecureAckPayload(sessionID, messageID), ackID, ackID); err != nil { + return []string{fmt.Sprintf("Failed to send secure direct ACK for %s: %v", messageID, err)} + } + return FlushQueuedSecureOutbox(ctx, s.resolved, s.manager, record, peerDID, func(method string, params map[string]any) (map[string]any, error) { + return transport.rpcMapCall(ctx, method, params) + }) +} + +func directInitSessionIDFromMessage(message map[string]any) string { + body, err := mapFromAny(message["content"]) + if err != nil { + return "" + } + return stringFromAny(body["session_id"]) +} + +func containsDirectE2EEMessages(messages []map[string]any) bool { + for _, message := range messages { + if isDirectE2EEWireContentType(stringFromAny(message["content_type"])) { + return true + } + } + return false +} + +func directE2EENotificationFromMessageView(message map[string]any) (map[string]any, error) { + body, err := mapFromAny(message["content"]) + if err != nil { + return nil, fmt.Errorf("content is not a direct-e2ee object") + } + senderDID := stringFromAny(message["sender_did"]) + receiverDID := stringFromAny(message["receiver_did"]) + messageID := stringFromAny(message["id"]) + if senderDID == "" || receiverDID == "" || messageID == "" { + return nil, fmt.Errorf("missing sender_did/receiver_did/id") + } + result := map[string]any{ + "meta": map[string]any{ + "sender_did": senderDID, + "target": map[string]any{"kind": "agent", "did": receiverDID}, + "message_id": messageID, + "profile": "anp.direct.e2ee.v1", + "security_profile": "direct-e2ee", + "content_type": stringFromAny(message["content_type"]), + }, + "body": body, + } + if serverSeq := int64Value(message["server_seq"]); serverSeq != 0 { + result["server_seq"] = serverSeq + } + return result, nil +} + +func applyDirectE2EEProcessingResult(message map[string]any, result map[string]any) { + message["secure"] = true + state := stringFromAny(result["state"]) + if state == "" { + return + } + message["decryption_state"] = state + if state != "decrypted" { + return + } + plaintext, err := mapFromAny(result["plaintext"]) + if err != nil { + return + } + if IsSecureAckPlaintext(plaintext) || IsSecureInitPlaintext(plaintext) { + message["secure_control"] = true + message["type"] = "secure_control" + message["content"] = "" + return + } + contentType := stringFromAny(plaintext["application_content_type"]) + if contentType != "" { + message["content_type"] = contentType + } + switch { + case stringFromAny(plaintext["text"]) != "": + message["content"] = stringFromAny(plaintext["text"]) + message["type"] = "text" + case plaintext["payload"] != nil: + message["content"] = plaintext["payload"] + if contentType == attachmentManifestContentType { + message["type"] = "attachment_manifest" + } else { + message["type"] = "json" + } + case stringFromAny(plaintext["payload_b64u"]) != "": + message["content"] = stringFromAny(plaintext["payload_b64u"]) + message["type"] = "binary" + } +} + +func int64Value(value any) int64 { + if parsed := int64PtrFromAny(value); parsed != nil { + return *parsed + } + return 0 +} diff --git a/internal/message/secure_test.go b/internal/message/secure_test.go new file mode 100644 index 0000000..4b3b0d6 --- /dev/null +++ b/internal/message/secure_test.go @@ -0,0 +1,1404 @@ +package message + +import ( + "context" + "crypto/ecdh" + "crypto/rand" + "encoding/json" + "fmt" + "net/http" + "net/http/httptest" + "os" + "path/filepath" + "strings" + "testing" + + anp "github.com/agent-network-protocol/anp/golang" + directe2ee "github.com/agent-network-protocol/anp/golang/direct_e2ee" + "github.com/agentconnect/awiki-cli/internal/anpsdk" + appconfig "github.com/agentconnect/awiki-cli/internal/config" + "github.com/agentconnect/awiki-cli/internal/identity" + "github.com/agentconnect/awiki-cli/internal/store" +) + +func TestServiceSendSecureDirectUsesP5KeyServiceTargetAndPersistsPendingSession(t *testing.T) { + t.Parallel() + + resolved := testResolvedConfig(t) + manager := identity.NewManager(resolved.Paths) + createTestIdentity(t, manager, identity.SaveInput{ + IdentityName: "alice", + UserID: "user-123", + DisplayName: "Alice", + Handle: "alice", + }) + resolved.ActiveIdentity = "alice" + + record, err := manager.Load("alice") + if err != nil { + t.Fatalf("manager.Load() error = %v", err) + } + serviceDID := testMessageServiceDID(t, record.DIDDocument) + prekeyBundle, oneTimePrekey := buildTestSecurePrekeyMaterial(t, resolved, record) + + var sawGetPrekey bool + var sawPublishPrekey bool + var sawDirectSend bool + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if r.URL.Path != MessageRPCEndpoint { + http.NotFound(w, r) + return + } + envelope := decodeRPCRequest(t, r) + switch envelope.Method { + case "direct.e2ee.publish_prekey_bundle": + sawPublishPrekey = true + meta := mustMapValue(t, envelope.Params["meta"], "params.meta") + target := mustMapValue(t, meta["target"], "meta.target") + if got := stringFromAny(target["kind"]); got != "service" { + t.Fatalf("publish meta.target.kind = %q, want service", got) + } + if got := stringFromAny(target["did"]); got != serviceDID { + t.Fatalf("publish meta.target.did = %q, want %q", got, serviceDID) + } + body := mustMapValue(t, envelope.Params["body"], "params.body") + if _, ok := body["prekey_bundle"]; !ok { + t.Fatalf("publish body.prekey_bundle missing: %#v", body) + } + if _, ok := body["one_time_prekeys"]; !ok { + t.Fatalf("publish body.one_time_prekeys missing: %#v", body) + } + _ = json.NewEncoder(w).Encode(map[string]any{ + "jsonrpc": "2.0", + "id": envelope.ID, + "result": map[string]any{ + "published": true, + "owner_did": record.DID, + "bundle_id": mustMapValue(t, body["prekey_bundle"], "body.prekey_bundle")["bundle_id"], + "published_opk_count": len(body["one_time_prekeys"].([]any)), + "published_at": "2026-04-23T09:25:00Z", + }, + }) + case "direct.e2ee.get_prekey_bundle": + sawGetPrekey = true + if _, ok := envelope.Params["auth"]; ok { + t.Fatalf("direct.e2ee.get_prekey_bundle params.auth should be omitted: %#v", envelope.Params) + } + meta := mustMapValue(t, envelope.Params["meta"], "params.meta") + target := mustMapValue(t, meta["target"], "meta.target") + if got := stringFromAny(meta["profile"]); got != "anp.direct.e2ee.v1" { + t.Fatalf("meta.profile = %q, want anp.direct.e2ee.v1", got) + } + if got := stringFromAny(meta["security_profile"]); got != "transport-protected" { + t.Fatalf("meta.security_profile = %q, want transport-protected", got) + } + if got := stringFromAny(target["kind"]); got != "service" { + t.Fatalf("meta.target.kind = %q, want service", got) + } + if got := stringFromAny(target["did"]); got != serviceDID { + t.Fatalf("meta.target.did = %q, want %q", got, serviceDID) + } + body := mustMapValue(t, envelope.Params["body"], "params.body") + if got := stringFromAny(body["target_did"]); got != record.DID { + t.Fatalf("body.target_did = %q, want %q", got, record.DID) + } + _ = json.NewEncoder(w).Encode(map[string]any{ + "jsonrpc": "2.0", + "id": envelope.ID, + "result": map[string]any{ + "target_did": record.DID, + "prekey_bundle": prekeyBundle, + "one_time_prekey": map[string]any{"key_id": oneTimePrekey.KeyID, "public_key_b64u": oneTimePrekey.PublicKeyB64U}, + }, + }) + case "direct.send": + sawDirectSend = true + if _, ok := envelope.Params["auth"]; ok { + t.Fatalf("direct.send params.auth should be omitted for P5 direct-e2ee: %#v", envelope.Params) + } + meta := mustMapValue(t, envelope.Params["meta"], "params.meta") + target := mustMapValue(t, meta["target"], "meta.target") + if got := stringFromAny(meta["content_type"]); got != "application/anp-direct-init+json" { + t.Fatalf("meta.content_type = %q, want application/anp-direct-init+json", got) + } + if got := stringFromAny(meta["operation_id"]); got != stringFromAny(meta["message_id"]) { + t.Fatalf("meta.operation_id = %q, want same as message_id %q", got, meta["message_id"]) + } + if got := stringFromAny(target["kind"]); got != "agent" { + t.Fatalf("meta.target.kind = %q, want agent", got) + } + if got := stringFromAny(target["did"]); got != record.DID { + t.Fatalf("meta.target.did = %q, want %q", got, record.DID) + } + body := mustMapValue(t, envelope.Params["body"], "params.body") + if _, ok := body["recipient_static_key_agreement_id"]; ok { + t.Fatalf("body should not contain recipient_static_key_agreement_id: %#v", body) + } + if got := stringFromAny(body["recipient_one_time_prekey_id"]); got != oneTimePrekey.KeyID { + t.Fatalf("body.recipient_one_time_prekey_id = %q, want %q", got, oneTimePrekey.KeyID) + } + _ = json.NewEncoder(w).Encode(map[string]any{ + "jsonrpc": "2.0", + "id": envelope.ID, + "result": map[string]any{ + "accepted": true, + "message_id": meta["message_id"], + "operation_id": meta["operation_id"], + "target_did": record.DID, + "accepted_at": "2026-04-23T09:30:00Z", + }, + }) + default: + t.Fatalf("unexpected RPC method: %s", envelope.Method) + } + })) + defer server.Close() + + resolved.ServiceBaseURL = server.URL + service, err := NewService(resolved) + if err != nil { + t.Fatalf("NewService() error = %v", err) + } + + result, err := service.Send(context.Background(), SendRequest{ + IdentityName: "alice", + Target: record.DID, + Text: "hello secure world", + SecureMode: "on", + }) + if err != nil { + t.Fatalf("Send() error = %v", err) + } + if !sawPublishPrekey || !sawGetPrekey || !sawDirectSend { + t.Fatalf("expected secure send to publish/get/send, sawPublishPrekey=%v sawGetPrekey=%v sawDirectSend=%v", sawPublishPrekey, sawGetPrekey, sawDirectSend) + } + + message := mustMapValue(t, result.Data["message"], "result.data.message") + if got := stringFromAny(message["type"]); got != "text" { + t.Fatalf("result.data.message.type = %q, want text", got) + } + if got, ok := message["secure"].(bool); !ok || !got { + t.Fatalf("result.data.message.secure = %#v, want true", message["secure"]) + } + db, err := store.Open(resolved.Paths) + if err != nil { + t.Fatalf("store.Open() error = %v", err) + } + defer db.Close() + var isE2EE int + if err := db.QueryRowContext(context.Background(), "SELECT is_e2ee FROM messages WHERE msg_id = ?", stringFromAny(message["id"])).Scan(&isE2EE); err != nil { + t.Fatalf("query stored secure message is_e2ee error = %v", err) + } + if isE2EE != 1 { + t.Fatalf("stored secure message is_e2ee = %d, want 1", isE2EE) + } + + paths, err := manager.PathsForIdentity("alice") + if err != nil { + t.Fatalf("PathsForIdentity() error = %v", err) + } + sessionEntries, err := os.ReadDir(filepath.Join(paths.IdentityDir, "p5-e2ee-sessions")) + if err != nil { + t.Fatalf("ReadDir(session store) error = %v", err) + } + if len(sessionEntries) != 1 { + t.Fatalf("len(sessionEntries) = %d, want 1", len(sessionEntries)) + } + sessionBytes, err := os.ReadFile(filepath.Join(paths.IdentityDir, "p5-e2ee-sessions", sessionEntries[0].Name())) + if err != nil { + t.Fatalf("ReadFile(session) error = %v", err) + } + var session map[string]any + if err := json.Unmarshal(sessionBytes, &session); err != nil { + t.Fatalf("Unmarshal(session) error = %v", err) + } + if got := stringFromAny(session["status"]); got != "pending-confirmation" { + t.Fatalf("session.status = %q, want pending-confirmation", got) + } +} + +func TestServiceSendSecureDirectQueuesFollowUpWhilePendingConfirmation(t *testing.T) { + t.Parallel() + + resolved := testResolvedConfig(t) + manager := identity.NewManager(resolved.Paths) + createTestIdentity(t, manager, identity.SaveInput{ + IdentityName: "alice", + UserID: "user-123", + DisplayName: "Alice", + Handle: "alice", + }) + resolved.ActiveIdentity = "alice" + + record, err := manager.Load("alice") + if err != nil { + t.Fatalf("manager.Load() error = %v", err) + } + prekeyBundle, oneTimePrekey := buildTestSecurePrekeyMaterial(t, resolved, record) + + var directSendCalls int + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + envelope := decodeRPCRequest(t, r) + switch envelope.Method { + case "direct.e2ee.publish_prekey_bundle": + body := mustMapValue(t, envelope.Params["body"], "params.body") + _ = json.NewEncoder(w).Encode(map[string]any{ + "jsonrpc": "2.0", + "id": envelope.ID, + "result": map[string]any{ + "published": true, + "owner_did": record.DID, + "bundle_id": mustMapValue(t, body["prekey_bundle"], "body.prekey_bundle")["bundle_id"], + "published_opk_count": len(body["one_time_prekeys"].([]any)), + "published_at": "2026-04-23T09:25:00Z", + }, + }) + case "direct.e2ee.get_prekey_bundle": + _ = json.NewEncoder(w).Encode(map[string]any{ + "jsonrpc": "2.0", + "id": envelope.ID, + "result": map[string]any{ + "target_did": record.DID, + "prekey_bundle": prekeyBundle, + "one_time_prekey": map[string]any{"key_id": oneTimePrekey.KeyID, "public_key_b64u": oneTimePrekey.PublicKeyB64U}, + }, + }) + case "direct.send": + directSendCalls++ + meta := mustMapValue(t, envelope.Params["meta"], "params.meta") + _ = json.NewEncoder(w).Encode(map[string]any{ + "jsonrpc": "2.0", + "id": envelope.ID, + "result": map[string]any{ + "accepted": true, + "message_id": meta["message_id"], + "operation_id": meta["operation_id"], + "target_did": mustMapValue(t, meta["target"], "meta.target")["did"], + "accepted_at": "2026-04-23T09:30:00Z", + }, + }) + default: + t.Fatalf("unexpected RPC method: %s", envelope.Method) + } + })) + defer server.Close() + + resolved.ServiceBaseURL = server.URL + service, err := NewService(resolved) + if err != nil { + t.Fatalf("NewService() error = %v", err) + } + + first, err := service.Send(context.Background(), SendRequest{ + IdentityName: "alice", + Target: record.DID, + Text: "first secure", + SecureMode: "on", + }) + if err != nil { + t.Fatalf("first Send() error = %v", err) + } + if first.Data["action"] != "send_message" { + t.Fatalf("first action = %#v, want send_message", first.Data["action"]) + } + second, err := service.Send(context.Background(), SendRequest{ + IdentityName: "alice", + Target: record.DID, + Text: "queued secure", + SecureMode: "on", + }) + if err != nil { + t.Fatalf("second Send() error = %v", err) + } + if second.Data["action"] != "queue_secure_message" { + t.Fatalf("second action = %#v, want queue_secure_message", second.Data["action"]) + } + delivery := mustMapValue(t, second.Data["delivery"], "second.data.delivery") + if got := stringFromAny(delivery["delivery_state"]); got != "queued" { + t.Fatalf("delivery_state = %q, want queued", got) + } + if directSendCalls != 1 { + t.Fatalf("directSendCalls = %d, want 1", directSendCalls) + } + db, err := store.Open(resolved.Paths) + if err != nil { + t.Fatalf("store.Open() error = %v", err) + } + defer db.Close() + rows, err := store.ListE2EEOutbox(context.Background(), db, record.DID, record.IdentityName, "queued") + if err != nil { + t.Fatalf("ListE2EEOutbox() error = %v", err) + } + if len(rows) != 1 || rows[0]["plaintext"] != "queued secure" { + t.Fatalf("queued outbox rows = %#v, want one queued secure row", rows) + } +} + +func TestPollingInboxDecryptsDirectInitAndSendsSecureAck(t *testing.T) { + t.Parallel() + + resolved := testResolvedConfig(t) + manager := identity.NewManager(resolved.Paths) + createTestIdentity(t, manager, identity.SaveInput{ + IdentityName: "alice", + UserID: "user-alice-123", + DisplayName: "Alice", + Handle: "alice", + JWTToken: "token-alice", + }) + createTestIdentity(t, manager, identity.SaveInput{ + IdentityName: "bob", + UserID: "user-bob-123", + DisplayName: "Bob", + Handle: "bob", + JWTToken: "token-bob", + }) + aliceRecord, err := manager.Load("alice") + if err != nil { + t.Fatalf("manager.Load(alice) error = %v", err) + } + bobRecord, err := manager.Load("bob") + if err != nil { + t.Fatalf("manager.Load(bob) error = %v", err) + } + prekeyBundle, oneTimePrekey := buildTestSecurePrekeyMaterial(t, resolved, bobRecord) + + var capturedInit map[string]any + aliceClient, err := NewSecureE2EEClientForRecord(context.Background(), manager, aliceRecord, func(method string, params map[string]any) (map[string]any, error) { + switch method { + case "direct.e2ee.get_prekey_bundle": + return map[string]any{ + "target_did": bobRecord.DID, + "prekey_bundle": prekeyBundle, + "one_time_prekey": map[string]any{"key_id": oneTimePrekey.KeyID, "public_key_b64u": oneTimePrekey.PublicKeyB64U}, + }, nil + case "direct.send": + capturedInit = params + meta := mustMapValue(t, params["meta"], "init.meta") + return map[string]any{ + "accepted": true, + "message_id": meta["message_id"], + "operation_id": meta["operation_id"], + "target_did": bobRecord.DID, + "accepted_at": "2026-04-24T08:00:00Z", + }, nil + default: + return nil, fmt.Errorf("unexpected alice RPC method: %s", method) + } + }) + if err != nil { + t.Fatalf("NewSecureE2EEClientForRecord(alice) error = %v", err) + } + if _, err := aliceClient.SendText(context.Background(), bobRecord.DID, "hello via polling", "msg-init-poll-001", "msg-init-poll-001"); err != nil { + t.Fatalf("alice SendText() error = %v", err) + } + if capturedInit == nil { + t.Fatal("alice direct init was not captured") + } + + var ackDirectSendCalls int + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + envelope := decodeRPCRequest(t, r) + if envelope.Method != "direct.send" { + t.Fatalf("unexpected bob RPC method: %s", envelope.Method) + } + ackDirectSendCalls++ + meta := mustMapValue(t, envelope.Params["meta"], "ack.meta") + if got := stringFromAny(meta["content_type"]); got != "application/anp-direct-cipher+json" { + t.Fatalf("ack meta.content_type = %q, want direct cipher", got) + } + if got := stringFromAny(meta["operation_id"]); got == "" || got != stringFromAny(meta["message_id"]) { + t.Fatalf("ack operation/message mismatch: %#v", meta) + } + if got := stringFromAny(meta["sender_did"]); got != bobRecord.DID { + t.Fatalf("ack sender_did = %q, want %q", got, bobRecord.DID) + } + target := mustMapValue(t, meta["target"], "ack.meta.target") + if got := stringFromAny(target["did"]); got != aliceRecord.DID { + t.Fatalf("ack target.did = %q, want %q", got, aliceRecord.DID) + } + _ = json.NewEncoder(w).Encode(map[string]any{ + "jsonrpc": "2.0", + "id": envelope.ID, + "result": map[string]any{ + "accepted": true, + "message_id": meta["message_id"], + "operation_id": meta["operation_id"], + "target_did": aliceRecord.DID, + "accepted_at": "2026-04-24T08:00:01Z", + }, + }) + })) + defer server.Close() + resolved.ServiceBaseURL = server.URL + resolved.ActiveIdentity = "bob" + service, err := NewService(resolved) + if err != nil { + t.Fatalf("NewService() error = %v", err) + } + initMeta := mustMapValue(t, capturedInit["meta"], "captured init meta") + initBody := mustMapValue(t, capturedInit["body"], "captured init body") + messages, _, warnings := service.persistInboxMessages(context.Background(), bobRecord, map[string]any{ + "messages": []any{ + map[string]any{ + "id": stringFromAny(initMeta["message_id"]), + "sender_did": aliceRecord.DID, + "receiver_did": bobRecord.DID, + "content_type": stringFromAny(initMeta["content_type"]), + "content": initBody, + "server_seq": float64(1), + "sent_at": "2026-04-24T08:00:00Z", + }, + }, + "total": 1, + }, "alice") + if len(warnings) != 0 { + t.Fatalf("persistInboxMessages warnings = %#v, want none", warnings) + } + if len(messages) != 1 { + t.Fatalf("len(messages) = %d, want 1", len(messages)) + } + if got := stringFromAny(messages[0]["content"]); got != "hello via polling" { + t.Fatalf("decrypted content = %q, want hello via polling", got) + } + if ackDirectSendCalls != 1 { + t.Fatalf("ackDirectSendCalls = %d, want 1", ackDirectSendCalls) + } +} + +func TestServiceSecureInitCreatesPendingSession(t *testing.T) { + t.Parallel() + + resolved := testResolvedConfig(t) + manager := identity.NewManager(resolved.Paths) + createTestIdentity(t, manager, identity.SaveInput{ + IdentityName: "alice", + UserID: "user-123", + DisplayName: "Alice", + Handle: "alice", + }) + resolved.ActiveIdentity = "alice" + + record, err := manager.Load("alice") + if err != nil { + t.Fatalf("manager.Load() error = %v", err) + } + serviceDID := testMessageServiceDID(t, record.DIDDocument) + prekeyBundle, oneTimePrekey := buildTestSecurePrekeyMaterial(t, resolved, record) + + var directSendCalls int + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + envelope := decodeRPCRequest(t, r) + switch envelope.Method { + case "direct.e2ee.publish_prekey_bundle": + body := mustMapValue(t, envelope.Params["body"], "params.body") + _ = json.NewEncoder(w).Encode(map[string]any{ + "jsonrpc": "2.0", + "id": envelope.ID, + "result": map[string]any{ + "published": true, + "owner_did": record.DID, + "bundle_id": mustMapValue(t, body["prekey_bundle"], "body.prekey_bundle")["bundle_id"], + "published_opk_count": len(body["one_time_prekeys"].([]any)), + "published_at": "2026-04-23T09:25:00Z", + }, + }) + case "direct.e2ee.get_prekey_bundle": + meta := mustMapValue(t, envelope.Params["meta"], "params.meta") + target := mustMapValue(t, meta["target"], "meta.target") + if stringFromAny(target["did"]) != serviceDID { + t.Fatalf("get target.did = %q, want %q", target["did"], serviceDID) + } + _ = json.NewEncoder(w).Encode(map[string]any{ + "jsonrpc": "2.0", + "id": envelope.ID, + "result": map[string]any{ + "target_did": record.DID, + "prekey_bundle": prekeyBundle, + "one_time_prekey": map[string]any{"key_id": oneTimePrekey.KeyID, "public_key_b64u": oneTimePrekey.PublicKeyB64U}, + }, + }) + case "direct.send": + directSendCalls++ + meta := mustMapValue(t, envelope.Params["meta"], "params.meta") + _ = json.NewEncoder(w).Encode(map[string]any{ + "jsonrpc": "2.0", + "id": envelope.ID, + "result": map[string]any{ + "accepted": true, + "message_id": meta["message_id"], + "operation_id": meta["operation_id"], + "target_did": mustMapValue(t, meta["target"], "meta.target")["did"], + "accepted_at": "2026-04-23T09:30:00Z", + }, + }) + default: + t.Fatalf("unexpected RPC method: %s", envelope.Method) + } + })) + defer server.Close() + + resolved.ServiceBaseURL = server.URL + service, err := NewService(resolved) + if err != nil { + t.Fatalf("NewService() error = %v", err) + } + result, err := service.SecureInit(context.Background(), SecurePeerRequest{ + IdentityName: "alice", + With: record.DID, + }) + if err != nil { + t.Fatalf("SecureInit() error = %v", err) + } + if result.Data["initialized"] != true { + t.Fatalf("initialized = %#v, want true", result.Data["initialized"]) + } + if directSendCalls != 1 { + t.Fatalf("directSendCalls = %d, want 1", directSendCalls) + } + paths, err := manager.PathsForIdentity("alice") + if err != nil { + t.Fatalf("PathsForIdentity() error = %v", err) + } + sessionEntries, err := os.ReadDir(filepath.Join(paths.IdentityDir, "p5-e2ee-sessions")) + if err != nil { + t.Fatalf("ReadDir(session store) error = %v", err) + } + if len(sessionEntries) != 1 { + t.Fatalf("len(sessionEntries) = %d, want 1", len(sessionEntries)) + } +} + +func TestFlushQueuedSecureOutboxSendsCipherAfterConfirmation(t *testing.T) { + t.Parallel() + + resolved := testResolvedConfig(t) + manager := identity.NewManager(resolved.Paths) + aliceGenerated, err := identity.GenerateIdentity(identity.GenerateOptions{ + Hostname: "awiki.ai", + PathPrefix: []string{"user"}, + ProofDomain: "awiki.ai", + }) + if err != nil { + t.Fatalf("GenerateIdentity(alice) error = %v", err) + } + if _, err := manager.Save(identity.SaveInput{ + IdentityName: "alice", + DID: aliceGenerated.DID, + UniqueID: aliceGenerated.UniqueID, + UserID: "user-alice-123", + DisplayName: "Alice", + Handle: "alice", + JWTToken: "token-alice", + DIDDocument: aliceGenerated.DIDDocument, + Key1PrivatePEM: aliceGenerated.Key1PrivatePEM, + Key1PublicPEM: aliceGenerated.Key1PublicPEM, + E2EESigningPrivatePEM: aliceGenerated.E2EESigningPrivatePEM, + E2EEAgreementPrivatePEM: aliceGenerated.E2EEAgreementPrivatePEM, + }); err != nil { + t.Fatalf("manager.Save(alice) error = %v", err) + } + aliceRecord, err := manager.Load("alice") + if err != nil { + t.Fatalf("manager.Load(alice) error = %v", err) + } + + bobGenerated, err := identity.GenerateIdentity(identity.GenerateOptions{ + Hostname: "awiki.ai", + PathPrefix: []string{"user"}, + ProofDomain: "awiki.ai", + }) + if err != nil { + t.Fatalf("GenerateIdentity(bob) error = %v", err) + } + aliceStatic, err := secureECDHPrivateKey(aliceGenerated.E2EEAgreementPrivatePEM) + if err != nil { + t.Fatalf("secureECDHPrivateKey(alice) error = %v", err) + } + bobStatic, err := secureECDHPrivateKey(bobGenerated.E2EEAgreementPrivatePEM) + if err != nil { + t.Fatalf("secureECDHPrivateKey(bob) error = %v", err) + } + bobSPK, err := ecdh.X25519().GenerateKey(rand.Reader) + if err != nil { + t.Fatalf("GenerateKey(bobSPK) error = %v", err) + } + bobSigning, err := anp.PrivateKeyFromPEM(bobGenerated.Key1PrivatePEM) + if err != nil { + t.Fatalf("PrivateKeyFromPEM(bob signing) error = %v", err) + } + bundle, err := directe2ee.BuildPrekeyBundle( + "bundle-bob-001", + bobGenerated.DID, + bobGenerated.DID+"#key-3", + directe2ee.SignedPrekeyFromPrivateKey("spk-bob-001", bobSPK, "2026-04-07T00:00:00Z"), + bobSigning, + bobGenerated.DID+"#key-1", + "2026-03-31T09:58:58Z", + ) + if err != nil { + t.Fatalf("BuildPrekeyBundle() error = %v", err) + } + recipientStaticPublic, err := directe2ee.ExtractX25519PublicKey(bobGenerated.DIDDocument, bundle.StaticKeyAgreementID) + if err != nil { + t.Fatalf("ExtractX25519PublicKey(bob) error = %v", err) + } + recipientSignedPrekey := fixed32(t, bobSPK.PublicKey().Bytes()) + sessionBuilder := directe2ee.DirectE2eeSession{} + initMetadata := directe2ee.DirectEnvelopeMetadata{ + SenderDID: aliceGenerated.DID, + RecipientDID: bobGenerated.DID, + MessageID: "msg-init-001", + Profile: "anp.direct.e2ee.v1", + SecurityProfile: "direct-e2ee", + } + aliceSession, _, initBody, err := sessionBuilder.InitiateSession( + initMetadata, + "msg-init-001", + aliceGenerated.DID+"#key-3", + aliceStatic, + bundle, + recipientStaticPublic, + recipientSignedPrekey, + directe2ee.NewTextPlaintext("text/plain", "init"), + ) + if err != nil { + t.Fatalf("InitiateSession() error = %v", err) + } + bobSession, _, err := sessionBuilder.AcceptIncomingInit( + initMetadata, + bobGenerated.DID+"#key-3", + bobStatic, + bobSPK, + fixed32(t, aliceStatic.PublicKey().Bytes()), + initBody, + ) + if err != nil { + t.Fatalf("AcceptIncomingInit() error = %v", err) + } + replyMetadata := directe2ee.DirectEnvelopeMetadata{ + SenderDID: bobGenerated.DID, + RecipientDID: aliceGenerated.DID, + MessageID: "msg-reply-001", + Profile: "anp.direct.e2ee.v1", + SecurityProfile: "direct-e2ee", + } + _, replyBody, err := sessionBuilder.EncryptFollowUp(&bobSession, replyMetadata, "msg-reply-001", directe2ee.NewJSONPlaintext("application/json", BuildSecureAckPayload(aliceSession.SessionID, initMetadata.MessageID))) + if err != nil { + t.Fatalf("EncryptFollowUp(reply) error = %v", err) + } + if _, err := sessionBuilder.DecryptFollowUp(&aliceSession, replyMetadata, replyBody); err != nil { + t.Fatalf("DecryptFollowUp(reply) error = %v", err) + } + paths, err := manager.PathsForIdentity("alice") + if err != nil { + t.Fatalf("PathsForIdentity(alice) error = %v", err) + } + sessionStore, err := anpsdk.NewFileSessionStore(filepath.Join(paths.IdentityDir, "p5-e2ee-sessions")) + if err != nil { + t.Fatalf("NewFileSessionStore() error = %v", err) + } + if err := sessionStore.SaveSession(aliceSession); err != nil { + t.Fatalf("SaveSession() error = %v", err) + } + + db, err := store.Open(resolved.Paths) + if err != nil { + t.Fatalf("store.Open() error = %v", err) + } + defer db.Close() + if err := store.EnsureSchema(context.Background(), db); err != nil { + t.Fatalf("EnsureSchema() error = %v", err) + } + outboxID, err := store.QueueE2EEOutbox(context.Background(), db, store.E2EEOutboxRecord{ + OwnerDID: aliceRecord.DID, + PeerDID: bobGenerated.DID, + SessionID: aliceSession.SessionID, + OriginalType: "text", + Plaintext: "queued follow-up", + LocalStatus: "queued", + CredentialName: aliceRecord.IdentityName, + }) + if err != nil { + t.Fatalf("QueueE2EEOutbox() error = %v", err) + } + + var sendCalls int + warnings := FlushQueuedSecureOutbox(context.Background(), resolved, manager, aliceRecord, bobGenerated.DID, func(method string, params map[string]any) (map[string]any, error) { + if method != "direct.send" { + t.Fatalf("unexpected method: %s", method) + } + sendCalls++ + meta := mustMapValue(t, params["meta"], "params.meta") + if got := stringFromAny(meta["content_type"]); got != "application/anp-direct-cipher+json" { + t.Fatalf("meta.content_type = %q, want application/anp-direct-cipher+json", got) + } + return map[string]any{ + "accepted": true, + "message_id": meta["message_id"], + "operation_id": meta["operation_id"], + "target_did": mustMapValue(t, meta["target"], "meta.target")["did"], + "accepted_at": "2026-04-23T10:00:00Z", + }, nil + }) + if len(warnings) != 0 { + t.Fatalf("FlushQueuedSecureOutbox() warnings = %#v, want none", warnings) + } + if sendCalls != 1 { + t.Fatalf("sendCalls = %d, want 1", sendCalls) + } + outboxRow, err := store.GetE2EEOutbox(context.Background(), db, outboxID, aliceRecord.DID, aliceRecord.IdentityName) + if err != nil { + t.Fatalf("GetE2EEOutbox() error = %v", err) + } + if got := outboxRow["local_status"]; got != "sent" { + t.Fatalf("outbox local_status = %#v, want sent", got) + } + messageRow, err := store.GetMessageByID(context.Background(), db, outboxID, aliceRecord.DID, aliceRecord.IdentityName) + if err != nil { + t.Fatalf("GetMessageByID() error = %v", err) + } + if got := messageRow["content"]; got != "queued follow-up" { + t.Fatalf("stored content = %#v, want queued follow-up", got) + } +} + +func TestServiceSecureStatusReturnsSessionAndOutboxSummary(t *testing.T) { + t.Parallel() + + resolved := testResolvedConfig(t) + manager := identity.NewManager(resolved.Paths) + createTestIdentity(t, manager, identity.SaveInput{ + IdentityName: "alice", + UserID: "user-123", + DisplayName: "Alice", + Handle: "alice", + }) + resolved.ActiveIdentity = "alice" + + record, err := manager.Load("alice") + if err != nil { + t.Fatalf("manager.Load() error = %v", err) + } + paths, err := manager.PathsForIdentity("alice") + if err != nil { + t.Fatalf("PathsForIdentity() error = %v", err) + } + sessionStore, err := anpsdk.NewFileSessionStore(filepath.Join(paths.IdentityDir, "p5-e2ee-sessions")) + if err != nil { + t.Fatalf("NewFileSessionStore() error = %v", err) + } + if err := sessionStore.SaveSession(anpsdk.DirectSessionState{ + SessionID: "session-001", + Suite: directe2ee.MTIDirectE2EESuite, + PeerDID: "did:wba:awiki.ai:user:bob:e1_bob", + LocalKeyAgreementID: record.DID + "#key-3", + PeerKeyAgreementID: "did:wba:awiki.ai:user:bob:e1_bob#key-3", + RootKeyB64U: "root", + SendChainKeyB64U: "send", + RecvChainKeyB64U: "recv", + RatchetPrivateKeyB64U: "priv", + RatchetPublicKeyB64U: "pub", + SendN: 1, + RecvN: 1, + SkippedMessageKeys: []directe2ee.SkippedMessageKey{ + { + DHPubB64U: "skipped-dh", + N: 7, + MessageKeyB64U: "skipped-message-key", + NonceB64U: "skipped-nonce", + }, + }, + IsInitiator: true, + Status: "established", + }); err != nil { + t.Fatalf("SaveSession() error = %v", err) + } + db, err := store.Open(resolved.Paths) + if err != nil { + t.Fatalf("store.Open() error = %v", err) + } + defer db.Close() + if err := store.EnsureSchema(context.Background(), db); err != nil { + t.Fatalf("EnsureSchema() error = %v", err) + } + if _, err := store.QueueE2EEOutbox(context.Background(), db, store.E2EEOutboxRecord{ + OwnerDID: record.DID, + PeerDID: "did:wba:awiki.ai:user:bob:e1_bob", + SessionID: "session-001", + OriginalType: "text", + Plaintext: "secret queued body", + LocalStatus: "failed", + CredentialName: record.IdentityName, + }); err != nil { + t.Fatalf("QueueE2EEOutbox() error = %v", err) + } + + service, err := NewService(resolved) + if err != nil { + t.Fatalf("NewService() error = %v", err) + } + result, err := service.SecureStatus(context.Background(), SecureStatusRequest{ + IdentityName: "alice", + With: "did:wba:awiki.ai:user:bob:e1_bob", + }) + if err != nil { + t.Fatalf("SecureStatus() error = %v", err) + } + sessions, ok := result.Data["sessions"].([]map[string]any) + if !ok || len(sessions) != 1 { + t.Fatalf("sessions = %#v, want 1 session", result.Data["sessions"]) + } + if got := sessions[0]["skipped_key_count"]; got != 1 { + t.Fatalf("session.skipped_key_count = %#v, want 1", got) + } + outbox := mustMapValue(t, result.Data["outbox"], "data.outbox") + if got := outbox["total"]; got != 1 { + t.Fatalf("outbox.total = %#v, want 1", got) + } + byStatus, ok := outbox["by_status"].(map[string]int) + if !ok { + t.Fatalf("outbox.by_status = %#v, want map[string]int", outbox["by_status"]) + } + if byStatus["failed"] != 1 { + t.Fatalf("by_status = %#v, want failed=1", byStatus) + } + records, ok := outbox["records"].([]map[string]any) + if !ok || len(records) != 1 { + t.Fatalf("outbox.records = %#v, want one redacted record", outbox["records"]) + } + if _, ok := records[0]["plaintext"]; ok { + t.Fatalf("status outbox record leaked plaintext: %#v", records[0]) + } + encoded, err := json.Marshal(result.Data) + if err != nil { + t.Fatalf("Marshal(status data) error = %v", err) + } + forbidden := []string{ + "root_key_b64u", + "send_chain_key_b64u", + "recv_chain_key_b64u", + "ratchet_private_key_b64u", + "message_key_b64u", + "nonce_b64u", + "secret queued body", + } + for _, value := range forbidden { + if strings.Contains(string(encoded), value) { + t.Fatalf("SecureStatus leaked %q in %s", value, encoded) + } + } +} + +func TestServiceSecureFailedAndDropOperateOnOutbox(t *testing.T) { + t.Parallel() + + resolved := testResolvedConfig(t) + manager := identity.NewManager(resolved.Paths) + createTestIdentity(t, manager, identity.SaveInput{ + IdentityName: "alice", + UserID: "user-123", + DisplayName: "Alice", + Handle: "alice", + }) + resolved.ActiveIdentity = "alice" + + record, err := manager.Load("alice") + if err != nil { + t.Fatalf("manager.Load() error = %v", err) + } + db, err := store.Open(resolved.Paths) + if err != nil { + t.Fatalf("store.Open() error = %v", err) + } + defer db.Close() + if err := store.EnsureSchema(context.Background(), db); err != nil { + t.Fatalf("EnsureSchema() error = %v", err) + } + outboxID, err := store.QueueE2EEOutbox(context.Background(), db, store.E2EEOutboxRecord{ + OwnerDID: record.DID, + PeerDID: "did:wba:awiki.ai:user:bob:e1_bob", + SessionID: "session-001", + OriginalType: "text", + Plaintext: "failed", + LocalStatus: "failed", + CredentialName: record.IdentityName, + }) + if err != nil { + t.Fatalf("QueueE2EEOutbox() error = %v", err) + } + service, err := NewService(resolved) + if err != nil { + t.Fatalf("NewService() error = %v", err) + } + failedResult, err := service.SecureFailed(context.Background(), SecureStatusRequest{IdentityName: "alice"}) + if err != nil { + t.Fatalf("SecureFailed() error = %v", err) + } + if failedResult.Data["total"] != 1 { + t.Fatalf("failed total = %#v, want 1", failedResult.Data["total"]) + } + dropResult, err := service.SecureDrop(context.Background(), SecureOutboxActionRequest{ + IdentityName: "alice", + OutboxID: outboxID, + }) + if err != nil { + t.Fatalf("SecureDrop() error = %v", err) + } + if dropResult.Data["status"] != "dropped" { + t.Fatalf("drop status = %#v, want dropped", dropResult.Data["status"]) + } + row, err := store.GetE2EEOutbox(context.Background(), db, outboxID, record.DID, record.IdentityName) + if err != nil { + t.Fatalf("GetE2EEOutbox() error = %v", err) + } + if got := row["local_status"]; got != "dropped" { + t.Fatalf("local_status = %#v, want dropped", got) + } +} + +func TestServiceSecureRetryMarksQueuedRecordSent(t *testing.T) { + t.Parallel() + + resolved := testResolvedConfig(t) + manager := identity.NewManager(resolved.Paths) + createTestIdentity(t, manager, identity.SaveInput{ + IdentityName: "alice", + UserID: "user-123", + DisplayName: "Alice", + Handle: "alice", + }) + resolved.ActiveIdentity = "alice" + + record, err := manager.Load("alice") + if err != nil { + t.Fatalf("manager.Load() error = %v", err) + } + paths, err := manager.PathsForIdentity("alice") + if err != nil { + t.Fatalf("PathsForIdentity() error = %v", err) + } + sessionStore, err := anpsdk.NewFileSessionStore(filepath.Join(paths.IdentityDir, "p5-e2ee-sessions")) + if err != nil { + t.Fatalf("NewFileSessionStore() error = %v", err) + } + bobGenerated, err := identity.GenerateIdentity(identity.GenerateOptions{ + Hostname: "awiki.ai", + PathPrefix: []string{"user"}, + ProofDomain: "awiki.ai", + }) + if err != nil { + t.Fatalf("GenerateIdentity(bob) error = %v", err) + } + aliceStatic, err := secureECDHPrivateKey(record.E2EEAgreementPrivatePEM) + if err != nil { + t.Fatalf("secureECDHPrivateKey(alice) error = %v", err) + } + bobStatic, err := secureECDHPrivateKey(bobGenerated.E2EEAgreementPrivatePEM) + if err != nil { + t.Fatalf("secureECDHPrivateKey(bob) error = %v", err) + } + bobSPK, err := ecdh.X25519().GenerateKey(rand.Reader) + if err != nil { + t.Fatalf("GenerateKey(bobSPK) error = %v", err) + } + bobSigning, err := anp.PrivateKeyFromPEM(bobGenerated.Key1PrivatePEM) + if err != nil { + t.Fatalf("PrivateKeyFromPEM(bob signing) error = %v", err) + } + bundle, err := directe2ee.BuildPrekeyBundle( + "bundle-bob-001", + bobGenerated.DID, + bobGenerated.DID+"#key-3", + directe2ee.SignedPrekeyFromPrivateKey("spk-bob-001", bobSPK, "2026-04-07T00:00:00Z"), + bobSigning, + bobGenerated.DID+"#key-1", + "2026-03-31T09:58:58Z", + ) + if err != nil { + t.Fatalf("BuildPrekeyBundle() error = %v", err) + } + recipientStaticPublic, err := directe2ee.ExtractX25519PublicKey(bobGenerated.DIDDocument, bundle.StaticKeyAgreementID) + if err != nil { + t.Fatalf("ExtractX25519PublicKey(bob) error = %v", err) + } + recipientSignedPrekey := fixed32(t, bobSPK.PublicKey().Bytes()) + sessionBuilder := directe2ee.DirectE2eeSession{} + initMetadata := directe2ee.DirectEnvelopeMetadata{ + SenderDID: record.DID, + RecipientDID: bobGenerated.DID, + MessageID: "msg-init-retry-001", + Profile: "anp.direct.e2ee.v1", + SecurityProfile: "direct-e2ee", + } + aliceSession, _, initBody, err := sessionBuilder.InitiateSession( + initMetadata, + "msg-init-retry-001", + record.DID+"#key-3", + aliceStatic, + bundle, + recipientStaticPublic, + recipientSignedPrekey, + directe2ee.NewTextPlaintext("text/plain", "init"), + ) + if err != nil { + t.Fatalf("InitiateSession() error = %v", err) + } + bobSession, _, err := sessionBuilder.AcceptIncomingInit( + initMetadata, + bobGenerated.DID+"#key-3", + bobStatic, + bobSPK, + fixed32(t, aliceStatic.PublicKey().Bytes()), + initBody, + ) + if err != nil { + t.Fatalf("AcceptIncomingInit() error = %v", err) + } + replyMetadata := directe2ee.DirectEnvelopeMetadata{ + SenderDID: bobGenerated.DID, + RecipientDID: record.DID, + MessageID: "msg-reply-retry-001", + Profile: "anp.direct.e2ee.v1", + SecurityProfile: "direct-e2ee", + } + _, replyBody, err := sessionBuilder.EncryptFollowUp(&bobSession, replyMetadata, "msg-reply-retry-001", directe2ee.NewJSONPlaintext("application/json", BuildSecureAckPayload(aliceSession.SessionID, initMetadata.MessageID))) + if err != nil { + t.Fatalf("EncryptFollowUp(reply) error = %v", err) + } + if _, err := sessionBuilder.DecryptFollowUp(&aliceSession, replyMetadata, replyBody); err != nil { + t.Fatalf("DecryptFollowUp(reply) error = %v", err) + } + if err := sessionStore.SaveSession(aliceSession); err != nil { + t.Fatalf("SaveSession() error = %v", err) + } + db, err := store.Open(resolved.Paths) + if err != nil { + t.Fatalf("store.Open() error = %v", err) + } + defer db.Close() + if err := store.EnsureSchema(context.Background(), db); err != nil { + t.Fatalf("EnsureSchema() error = %v", err) + } + outboxID, err := store.QueueE2EEOutbox(context.Background(), db, store.E2EEOutboxRecord{ + OwnerDID: record.DID, + PeerDID: bobGenerated.DID, + SessionID: aliceSession.SessionID, + OriginalType: "text", + Plaintext: "retry me", + LocalStatus: "failed", + CredentialName: record.IdentityName, + }) + if err != nil { + t.Fatalf("QueueE2EEOutbox() error = %v", err) + } + + var sendCalls int + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + envelope := decodeRPCRequest(t, r) + switch envelope.Method { + case "direct.e2ee.publish_prekey_bundle": + _ = json.NewEncoder(w).Encode(map[string]any{ + "jsonrpc": "2.0", + "id": envelope.ID, + "result": map[string]any{"published": true, "owner_did": record.DID, "bundle_id": "bundle-001", "published_at": "2026-04-23T09:25:00Z"}, + }) + case "direct.send": + sendCalls++ + meta := mustMapValue(t, envelope.Params["meta"], "params.meta") + _ = json.NewEncoder(w).Encode(map[string]any{ + "jsonrpc": "2.0", + "id": envelope.ID, + "result": map[string]any{ + "accepted": true, + "message_id": meta["message_id"], + "operation_id": meta["operation_id"], + "target_did": mustMapValue(t, meta["target"], "meta.target")["did"], + "accepted_at": "2026-04-23T09:30:00Z", + }, + }) + default: + t.Fatalf("unexpected RPC method: %s", envelope.Method) + } + })) + defer server.Close() + resolved.ServiceBaseURL = server.URL + service, err := NewService(resolved) + if err != nil { + t.Fatalf("NewService() error = %v", err) + } + result, err := service.SecureRetry(context.Background(), SecureOutboxActionRequest{ + IdentityName: "alice", + OutboxID: outboxID, + }) + if err != nil { + t.Fatalf("SecureRetry() error = %v", err) + } + if sendCalls != 1 { + t.Fatalf("sendCalls = %d, want 1", sendCalls) + } + recordMap := mustMapValue(t, result.Data["record"], "data.record") + if got := recordMap["local_status"]; got != "sent" { + t.Fatalf("record.local_status = %#v, want sent", got) + } +} + +func TestServiceSecureRepairResetsFailedOutboxAndStartsNewInit(t *testing.T) { + t.Parallel() + + resolved := testResolvedConfig(t) + manager := identity.NewManager(resolved.Paths) + createTestIdentity(t, manager, identity.SaveInput{ + IdentityName: "alice", + UserID: "user-123", + DisplayName: "Alice", + Handle: "alice", + }) + resolved.ActiveIdentity = "alice" + + record, err := manager.Load("alice") + if err != nil { + t.Fatalf("manager.Load() error = %v", err) + } + serviceDID := testMessageServiceDID(t, record.DIDDocument) + prekeyBundle, oneTimePrekey := buildTestSecurePrekeyMaterial(t, resolved, record) + paths, err := manager.PathsForIdentity("alice") + if err != nil { + t.Fatalf("PathsForIdentity() error = %v", err) + } + sessionStore, err := anpsdk.NewFileSessionStore(filepath.Join(paths.IdentityDir, "p5-e2ee-sessions")) + if err != nil { + t.Fatalf("NewFileSessionStore() error = %v", err) + } + if err := sessionStore.SaveSession(anpsdk.DirectSessionState{ + SessionID: "old-session", + Suite: directe2ee.MTIDirectE2EESuite, + PeerDID: record.DID, + LocalKeyAgreementID: record.DID + "#key-3", + PeerKeyAgreementID: record.DID + "#key-3", + RootKeyB64U: "root", + SendChainKeyB64U: "send", + RecvChainKeyB64U: "recv", + RatchetPrivateKeyB64U: "priv", + RatchetPublicKeyB64U: "pub", + SendN: 1, + RecvN: 1, + IsInitiator: true, + Status: "established", + }); err != nil { + t.Fatalf("SaveSession() error = %v", err) + } + db, err := store.Open(resolved.Paths) + if err != nil { + t.Fatalf("store.Open() error = %v", err) + } + defer db.Close() + if err := store.EnsureSchema(context.Background(), db); err != nil { + t.Fatalf("EnsureSchema() error = %v", err) + } + outboxID, err := store.QueueE2EEOutbox(context.Background(), db, store.E2EEOutboxRecord{ + OwnerDID: record.DID, + PeerDID: record.DID, + SessionID: "old-session", + OriginalType: "text", + Plaintext: "failed message", + LocalStatus: "failed", + CredentialName: record.IdentityName, + }) + if err != nil { + t.Fatalf("QueueE2EEOutbox() error = %v", err) + } + + var directSendCalls int + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + envelope := decodeRPCRequest(t, r) + switch envelope.Method { + case "direct.e2ee.publish_prekey_bundle": + body := mustMapValue(t, envelope.Params["body"], "params.body") + _ = json.NewEncoder(w).Encode(map[string]any{ + "jsonrpc": "2.0", + "id": envelope.ID, + "result": map[string]any{ + "published": true, + "owner_did": record.DID, + "bundle_id": mustMapValue(t, body["prekey_bundle"], "body.prekey_bundle")["bundle_id"], + "published_opk_count": len(body["one_time_prekeys"].([]any)), + "published_at": "2026-04-23T09:25:00Z", + }, + }) + case "direct.e2ee.get_prekey_bundle": + meta := mustMapValue(t, envelope.Params["meta"], "params.meta") + target := mustMapValue(t, meta["target"], "meta.target") + if stringFromAny(target["did"]) != serviceDID { + t.Fatalf("get target.did = %q, want %q", target["did"], serviceDID) + } + _ = json.NewEncoder(w).Encode(map[string]any{ + "jsonrpc": "2.0", + "id": envelope.ID, + "result": map[string]any{ + "target_did": record.DID, + "prekey_bundle": prekeyBundle, + "one_time_prekey": map[string]any{"key_id": oneTimePrekey.KeyID, "public_key_b64u": oneTimePrekey.PublicKeyB64U}, + }, + }) + case "direct.send": + directSendCalls++ + meta := mustMapValue(t, envelope.Params["meta"], "params.meta") + _ = json.NewEncoder(w).Encode(map[string]any{ + "jsonrpc": "2.0", + "id": envelope.ID, + "result": map[string]any{ + "accepted": true, + "message_id": meta["message_id"], + "operation_id": meta["operation_id"], + "target_did": mustMapValue(t, meta["target"], "meta.target")["did"], + "accepted_at": "2026-04-23T09:30:00Z", + }, + }) + default: + t.Fatalf("unexpected RPC method: %s", envelope.Method) + } + })) + defer server.Close() + resolved.ServiceBaseURL = server.URL + service, err := NewService(resolved) + if err != nil { + t.Fatalf("NewService() error = %v", err) + } + result, err := service.SecureRepair(context.Background(), SecurePeerRequest{ + IdentityName: "alice", + With: record.DID, + }) + if err != nil { + t.Fatalf("SecureRepair() error = %v", err) + } + repair := mustMapValue(t, result.Data["repair"], "data.repair") + if repair["reset_records"] != 2 { + t.Fatalf("repair.reset_records = %#v, want 2", repair["reset_records"]) + } + if directSendCalls != 1 { + t.Fatalf("directSendCalls = %d, want 1", directSendCalls) + } + row, err := store.GetE2EEOutbox(context.Background(), db, outboxID, record.DID, record.IdentityName) + if err != nil { + t.Fatalf("GetE2EEOutbox() error = %v", err) + } + if got := row["local_status"]; got != "queued" { + t.Fatalf("local_status = %#v, want queued", got) + } +} + +func buildTestSecurePrekeyMaterial( + t *testing.T, + resolved *appconfig.Resolved, + record *identity.StoredIdentity, +) (map[string]any, anpsdk.OneTimePrekey) { + t.Helper() + + paths, err := identity.NewManager(resolved.Paths).PathsForIdentity(record.IdentityName) + if err != nil { + t.Fatalf("PathsForIdentity() error = %v", err) + } + signingPrivate, err := anpsdk.PrivateKeyFromPEM(record.Key1PrivatePEM) + if err != nil { + t.Fatalf("PrivateKeyFromPEM(signing) error = %v", err) + } + agreementPrivate, err := anpsdk.PrivateKeyFromPEM(record.E2EEAgreementPrivatePEM) + if err != nil { + t.Fatalf("PrivateKeyFromPEM(agreement) error = %v", err) + } + sessionStore, err := anpsdk.NewFileSessionStore(filepath.Join(paths.IdentityDir, "p5-e2ee-sessions")) + if err != nil { + t.Fatalf("NewFileSessionStore() error = %v", err) + } + signedPrekeyStore, err := anpsdk.NewFileSignedPrekeyStore(filepath.Join(paths.IdentityDir, "p5-signed-prekeys")) + if err != nil { + t.Fatalf("NewFileSignedPrekeyStore() error = %v", err) + } + oneTimePrekeyStore, err := anpsdk.NewFileOneTimePrekeyStore(filepath.Join(paths.IdentityDir, "p5-one-time-prekeys")) + if err != nil { + t.Fatalf("NewFileOneTimePrekeyStore() error = %v", err) + } + resolver := func(_ context.Context, did string) (map[string]any, error) { + if did != record.DID { + return nil, fmt.Errorf("unexpected did: %s", did) + } + return record.DIDDocument, nil + } + client, err := anpsdk.NewMessageServiceDirectE2eeClient( + record.DID, + signingPrivate, + record.DID+"#key-1", + agreementPrivate, + record.DID+"#key-3", + func(string, map[string]any) (map[string]any, error) { return map[string]any{}, nil }, + resolver, + sessionStore, + signedPrekeyStore, + oneTimePrekeyStore, + ) + if err != nil { + t.Fatalf("NewMessageServiceDirectE2eeClient() error = %v", err) + } + bundle, err := client.EnsureFreshPrekeyBundle() + if err != nil { + t.Fatalf("EnsureFreshPrekeyBundle() error = %v", err) + } + oneTimePrekeys, err := oneTimePrekeyStore.ListOneTimePrekeys() + if err != nil { + t.Fatalf("ListOneTimePrekeys() error = %v", err) + } + if len(oneTimePrekeys) == 0 { + t.Fatal("expected EnsureFreshPrekeyBundle to materialize local OPKs") + } + raw, err := json.Marshal(bundle) + if err != nil { + t.Fatalf("Marshal(prekey bundle) error = %v", err) + } + var result map[string]any + if err := json.Unmarshal(raw, &result); err != nil { + t.Fatalf("Unmarshal(prekey bundle) error = %v", err) + } + return result, oneTimePrekeys[0] +} + +func testMessageServiceDID(t *testing.T, didDocument map[string]any) string { + t.Helper() + services, ok := didDocument["service"].([]any) + if !ok { + t.Fatalf("didDocument.service = %#v, want []any", didDocument["service"]) + } + for _, entry := range services { + service, ok := entry.(map[string]any) + if !ok { + continue + } + if stringFromAny(service["type"]) != "ANPMessageService" { + continue + } + serviceDID := stringFromAny(service["serviceDid"]) + if serviceDID != "" { + return serviceDID + } + } + t.Fatalf("ANPMessageService.serviceDid not found in %#v", didDocument["service"]) + return "" +} + +func secureECDHPrivateKey(pemValue string) (*ecdh.PrivateKey, error) { + privateKey, err := anp.PrivateKeyFromPEM(pemValue) + if err != nil { + return nil, err + } + return ecdh.X25519().NewPrivateKey(privateKey.Bytes) +} + +func fixed32(t *testing.T, value []byte) [32]byte { + t.Helper() + if len(value) != 32 { + t.Fatalf("len(value) = %d, want 32", len(value)) + } + var result [32]byte + copy(result[:], value) + return result +} diff --git a/internal/message/service.go b/internal/message/service.go index 7ec37f4..6e09631 100644 --- a/internal/message/service.go +++ b/internal/message/service.go @@ -19,9 +19,10 @@ import ( ) type Service struct { - resolved *appconfig.Resolved - manager *identity.Manager - remote *identity.RemoteClient + resolved *appconfig.Resolved + manager *identity.Manager + remote *identity.RemoteClient + mlsProvider *MLSExecProvider } func NewService(resolved *appconfig.Resolved) (*Service, error) { @@ -36,6 +37,13 @@ func NewService(resolved *appconfig.Resolved) (*Service, error) { }, nil } +func (s *Service) groupMLSProvider() MLSExecProvider { + if s != nil && s.mlsProvider != nil { + return *s.mlsProvider + } + return NewDefaultMLSExecProvider(s.resolved) +} + func (s *Service) Config() *appconfig.Resolved { return s.resolved } @@ -61,7 +69,7 @@ func (s *Service) Send(ctx context.Context, request SendRequest) (*CommandResult return nil, ErrTextRequired } if request.SecureMode == "on" { - return nil, ErrSecureNotSupported + return s.sendSecureDirect(ctx, request) } record, err := s.requireActiveIdentity(request.IdentityName) if err != nil { @@ -105,8 +113,13 @@ func (s *Service) Inbox(ctx context.Context, request InboxRequest) (*CommandResu if err != nil { return nil, err } + publishWarnings := s.maybePublishSecurePrekeys(ctx, record) if request.Scope == "all" { - return s.allInbox(ctx, record, request) + result, err := s.allInbox(ctx, record, request) + if result != nil { + result.Warnings = append(result.Warnings, publishWarnings...) + } + return result, err } originalWith := strings.TrimSpace(request.With) targetIsHandle := originalWith != "" && !strings.HasPrefix(originalWith, "did:") @@ -140,7 +153,7 @@ func (s *Service) Inbox(ctx context.Context, request InboxRequest) (*CommandResu } mode := s.runtimeConfig() - warnings := make([]string, 0) + warnings := append([]string(nil), publishWarnings...) var raw map[string]any switch mode.Mode { case runtime.ModeWebSocket: @@ -262,6 +275,7 @@ func (s *Service) History(ctx context.Context, request HistoryRequest) (*Command if err != nil { return nil, err } + publishWarnings := s.maybePublishSecurePrekeys(ctx, record) originalWith := strings.TrimSpace(request.With) targetIsHandle := originalWith != "" && !strings.HasPrefix(originalWith, "did:") peerDID, peerHandle, err := s.resolveTarget(ctx, request.With) @@ -291,7 +305,7 @@ func (s *Service) History(ctx context.Context, request HistoryRequest) (*Command request.With = peerDID mode := s.runtimeConfig() - warnings := make([]string, 0) + warnings := append([]string(nil), publishWarnings...) var raw map[string]any switch mode.Mode { case runtime.ModeWebSocket: @@ -650,22 +664,7 @@ func (s *Service) httpTransport(record *identity.StoredIdentity) (*HTTPTransport if err != nil { return nil, nil, err } - if auth != nil && auth.session != nil { - rememberAuthScopes(auth.session, s.resolved) - } - if auth != nil && auth.session != nil && strings.TrimSpace(record.JWTToken) != "" { - token := strings.TrimSpace(record.JWTToken) - auth.session.SetBearer(s.resolved.ServiceBaseURL, token) - auth.session.SetBearer( - appconfig.JoinBaseURL(s.resolved.ServiceBaseURL, "/user-service/did-auth/rpc"), - token, - ) - auth.session.SetBearer( - appconfig.JoinBaseURL(s.resolved.ServiceBaseURL, MessageRPCEndpoint), - token, - ) - auth.session.SetBearer(s.resolved.ANPServiceEndpoint, token) - } + primeAuthSession(auth, s.resolved) client := http.DefaultClient if s.remote != nil && s.remote.Client() != nil { client = s.remote.Client() @@ -673,6 +672,27 @@ func (s *Service) httpTransport(record *identity.StoredIdentity) (*HTTPTransport return NewHTTPTransport(s.resolved, auth, client), nil, nil } +func primeAuthSession(auth *authContext, resolved *appconfig.Resolved) { + if auth == nil || auth.session == nil || resolved == nil { + return + } + rememberAuthScopes(auth.session, resolved) + if strings.TrimSpace(auth.record.JWTToken) == "" { + return + } + token := strings.TrimSpace(auth.record.JWTToken) + auth.session.SetBearer(resolved.ServiceBaseURL, token) + auth.session.SetBearer( + appconfig.JoinBaseURL(resolved.ServiceBaseURL, "/user-service/did-auth/rpc"), + token, + ) + auth.session.SetBearer( + appconfig.JoinBaseURL(resolved.ServiceBaseURL, MessageRPCEndpoint), + token, + ) + auth.session.SetBearer(resolved.ANPServiceEndpoint, token) +} + func rememberAuthScopes(session *authsdk.Session, resolved *appconfig.Resolved) { if session == nil || resolved == nil { return @@ -762,6 +782,10 @@ func (s *Service) resolveTarget(ctx context.Context, target string) (string, str func (s *Service) persistSendResult(ctx context.Context, record *identity.StoredIdentity, targetDID string, targetHandle string, request SendRequest, result *directSendResult, warnings []string) (*CommandResult, error) { finish := traceutil.LocalDBPhase(ctx, "persist_direct_send") defer finish() + messageType := strings.TrimSpace(request.MessageType) + if messageType == "" { + messageType = "text" + } db, err := store.Open(s.resolved.Paths) if err != nil { return nil, err @@ -782,6 +806,7 @@ func (s *Service) persistSendResult(ctx context.Context, record *identity.Stored ServerSeq: nil, SentAt: result.AcceptedAt, IsRead: true, + IsE2EE: strings.TrimSpace(request.SecureMode) == "on", Metadata: metadataString(map[string]any{"delivery_state": result.DeliveryState, "operation_id": result.OperationID, "target_handle": targetHandle}), CredentialName: record.IdentityName, }); err != nil { @@ -797,13 +822,13 @@ func (s *Service) persistSendResult(ctx context.Context, record *identity.Stored }, "message": map[string]any{ "id": result.MessageID, - "type": request.MessageType, - "secure": false, + "type": messageType, + "secure": strings.TrimSpace(request.SecureMode) == "on", "sent_at": result.AcceptedAt, }, "delivery": result, }, - Summary: fmt.Sprintf("Sent a direct %s message", request.MessageType), + Summary: fmt.Sprintf("Sent a direct %s message", messageType), Warnings: warnings, }, nil } @@ -818,8 +843,12 @@ func (s *Service) persistInboxMessages(ctx context.Context, record *identity.Sto defer db.Close() _ = store.EnsureSchema(ctx, db) messages := messagesFromResult(raw["messages"]) + warnings := s.maybeDecryptDirectE2EEMessages(ctx, record, messages) storable := make([]store.MessageRecord, 0, len(messages)) for _, message := range messages { + if boolFromAny(message["secure_control"]) { + continue + } msgID := stringFromAny(message["id"]) if msgID == "" { continue @@ -830,6 +859,11 @@ func (s *Service) persistInboxMessages(ctx context.Context, record *identity.Sto if peerDID == record.DID { peerDID = receiverDID } + contentValue := message["content"] + content := stringFromAny(contentValue) + if content == "" { + content = metadataString(contentValue) + } storable = append(storable, store.MessageRecord{ MsgID: msgID, OwnerDID: record.DID, @@ -838,9 +872,10 @@ func (s *Service) persistInboxMessages(ctx context.Context, record *identity.Sto SenderDID: senderDID, ReceiverDID: receiverDID, ContentType: stringFromAny(message["content_type"]), - Content: stringFromAny(message["content"]), + Content: content, ServerSeq: int64PtrFromAny(message["server_seq"]), SentAt: stringFromAny(message["sent_at"]), + IsE2EE: boolFromAny(message["secure"]), IsRead: boolFromAny(message["is_read"]), SenderName: stringFromAny(message["sender_name"]), Metadata: metadataString(message), @@ -850,7 +885,7 @@ func (s *Service) persistInboxMessages(ctx context.Context, record *identity.Sto _ = store.StoreMessagesBatch(ctx, db, storable) contactFinish := traceutil.PhaseContext(ctx, "contact_sync") defer contactFinish() - warnings := s.syncDirectPeerHandles(ctx, db, record.DID, messages, knownHandle, "msg.inbox") + warnings = append(warnings, s.syncDirectPeerHandles(ctx, db, record.DID, messages, knownHandle, "msg.inbox")...) return messages, intValueFromAny(raw["total"], len(messages)), warnings } @@ -864,8 +899,12 @@ func (s *Service) persistHistoryMessages(ctx context.Context, record *identity.S defer db.Close() _ = store.EnsureSchema(ctx, db) messages := messagesFromResult(raw["messages"]) + warnings := s.maybeDecryptDirectE2EEMessages(ctx, record, messages) storable := make([]store.MessageRecord, 0, len(messages)) for _, message := range messages { + if boolFromAny(message["secure_control"]) { + continue + } msgID := stringFromAny(message["id"]) if msgID == "" { continue @@ -876,6 +915,11 @@ func (s *Service) persistHistoryMessages(ctx context.Context, record *identity.S if senderDID == record.DID { direction = 1 } + contentValue := message["content"] + content := stringFromAny(contentValue) + if content == "" { + content = metadataString(contentValue) + } storable = append(storable, store.MessageRecord{ MsgID: msgID, OwnerDID: record.DID, @@ -884,9 +928,10 @@ func (s *Service) persistHistoryMessages(ctx context.Context, record *identity.S SenderDID: senderDID, ReceiverDID: receiverDID, ContentType: stringFromAny(message["content_type"]), - Content: stringFromAny(message["content"]), + Content: content, ServerSeq: int64PtrFromAny(message["server_seq"]), SentAt: stringFromAny(message["sent_at"]), + IsE2EE: boolFromAny(message["secure"]), IsRead: boolFromAny(message["is_read"]), SenderName: stringFromAny(message["sender_name"]), Metadata: metadataString(message), @@ -896,7 +941,7 @@ func (s *Service) persistHistoryMessages(ctx context.Context, record *identity.S _ = store.StoreMessagesBatch(ctx, db, storable) contactFinish := traceutil.PhaseContext(ctx, "contact_sync") defer contactFinish() - warnings := s.syncDirectPeerHandles(ctx, db, record.DID, messages, knownHandle, "msg.history") + warnings = append(warnings, s.syncDirectPeerHandles(ctx, db, record.DID, messages, knownHandle, "msg.history")...) return messages, intValueFromAny(raw["total"], len(messages)), warnings } diff --git a/internal/message/types.go b/internal/message/types.go index 63c5d0d..fc5bfdd 100644 --- a/internal/message/types.go +++ b/internal/message/types.go @@ -24,13 +24,14 @@ var ( ErrDownloadTargetConflict = errors.New( "attachment download accepts either --with or --group, but not both", ) - ErrAttachmentNotFound = errors.New("attachment not found in message content") - ErrAttachmentIDRequired = errors.New("attachment_id is required for messages with multiple attachments") - ErrAttachmentMessageInvalid = errors.New("message is not an attachment manifest") - ErrAttachmentSenderRequired = errors.New("attachment message sender_did is required") - ErrTransportUnavailable = errors.New("message transport is unavailable") - ErrSecureNotSupported = errors.New("direct secure messaging is not implemented yet") - ErrMessageNotFound = errors.New("message not found") + ErrAttachmentNotFound = errors.New("attachment not found in message content") + ErrAttachmentIDRequired = errors.New("attachment_id is required for messages with multiple attachments") + ErrAttachmentMessageInvalid = errors.New("message is not an attachment manifest") + ErrAttachmentSenderRequired = errors.New("attachment message sender_did is required") + ErrTransportUnavailable = errors.New("message transport is unavailable") + ErrSecureNotSupported = errors.New("secure messaging is not supported for this command yet") + ErrGroupE2EESelfLeaveUnsupported = errors.New("group E2EE self-leave is not cryptographically supported yet") + ErrMessageNotFound = errors.New("message not found") ) type CommandResult struct { @@ -86,6 +87,21 @@ type AttachmentDownloadRequest struct { OutputPath string } +type SecureStatusRequest struct { + IdentityName string + With string +} + +type SecurePeerRequest struct { + IdentityName string + With string +} + +type SecureOutboxActionRequest struct { + IdentityName string + OutboxID string +} + type directSendResult struct { Accepted bool `json:"accepted"` MessageID string `json:"message_id"` @@ -97,20 +113,22 @@ type directSendResult struct { } type GroupCreateRequest struct { - IdentityName string - Name string - Description string - Discoverability string - AdmissionMode string - Slug string - Goal string - Rules string - MessagePrompt string - DocURL string - AttachmentsAllowed *bool - MaxMembers string - MemberMaxMessages *int64 - MemberMaxTotalChars *int64 + IdentityName string + Name string + Description string + Discoverability string + AdmissionMode string + MessageSecurityProfile string + E2EE bool + Slug string + Goal string + Rules string + MessagePrompt string + DocURL string + AttachmentsAllowed *bool + MaxMembers string + MemberMaxMessages *int64 + MemberMaxTotalChars *int64 } type GroupGetRequest struct { @@ -132,16 +150,35 @@ type GroupJoinRequest struct { } type GroupMemberRequest struct { + IdentityName string + Group string + Member string + Role string + ReasonText string + E2EE bool + LeaveRequestID string +} + +type GroupLeaveRequest struct { IdentityName string Group string - Member string - Role string ReasonText string + E2EE bool } -type GroupLeaveRequest struct { +type GroupE2EEProcessLeaveRequest struct { + IdentityName string + Group string + Member string + LeaveRequestID string + ReasonText string +} + +type GroupE2EERecoverMemberRequest struct { IdentityName string Group string + Member string + DeviceID string } type GroupUpdateRequest struct { diff --git a/internal/message/ws_proxy_client_test.go b/internal/message/ws_proxy_client_test.go index ee814cb..2b0e0bb 100644 --- a/internal/message/ws_proxy_client_test.go +++ b/internal/message/ws_proxy_client_test.go @@ -81,7 +81,7 @@ func TestWSProxyTransportCallsLocalBridgeAndDecodesResponses(t *testing.T) { defer listener.Close() requests := make(chan runtime.BridgeRequest, 1) - go serveBridgeOnce(t, listener, requests, runtime.BridgeResponse{OK: true, Result: map[string]any{ + go serveBridgeProbeAndRequest(t, listener, requests, runtime.BridgeResponse{OK: true, Result: map[string]any{ "message_id": "msg-1", "operation_id": "op-1", }}) @@ -119,7 +119,7 @@ func TestWSProxyTransportWrapsBridgeFailures(t *testing.T) { } } -func serveBridgeOnce(t *testing.T, listener net.Listener, requests chan<- runtime.BridgeRequest, response runtime.BridgeResponse) { +func serveBridgeProbeAndRequest(t *testing.T, listener net.Listener, requests chan<- runtime.BridgeRequest, response runtime.BridgeResponse) { t.Helper() for { conn, err := listener.Accept() diff --git a/internal/runtime/listener/server.go b/internal/runtime/listener/server.go index c313781..5c83273 100644 --- a/internal/runtime/listener/server.go +++ b/internal/runtime/listener/server.go @@ -7,13 +7,17 @@ import ( "encoding/json" "errors" "fmt" + "log" "net" "os" + "path/filepath" "strconv" "strings" "sync" "time" + directe2ee "github.com/agent-network-protocol/anp/golang/direct_e2ee" + "github.com/agentconnect/awiki-cli/internal/anpsdk" "github.com/agentconnect/awiki-cli/internal/authsdk" appconfig "github.com/agentconnect/awiki-cli/internal/config" "github.com/agentconnect/awiki-cli/internal/identity" @@ -29,24 +33,27 @@ type Supervisor struct { statusMu sync.Mutex status Status - sessionsMu sync.Mutex - sessions map[string]*session - listener net.Listener - db *sql.DB - hostNotify HostNotifySink + sessionsMu sync.Mutex + sessions map[string]*session + localNotificationsMu sync.Mutex + localNotifications map[string][]map[string]any + listener net.Listener + db *sql.DB + hostNotify HostNotifySink } type session struct { - identityName string - record *identity.StoredIdentity - client *WSClient - lastError string - connected bool - ctx context.Context - cancelFunc context.CancelFunc - initResult chan error - initOnce sync.Once - mu sync.RWMutex + identityName string + record *identity.StoredIdentity + client *WSClient + secureRPCCall func(context.Context, string, map[string]any) (map[string]any, error) + lastError string + connected bool + ctx context.Context + cancelFunc context.CancelFunc + initResult chan error + initOnce sync.Once + mu sync.RWMutex } const ( @@ -102,9 +109,10 @@ func NewSupervisor(resolved *appconfig.Resolved) (*Supervisor, error) { StartedAt: time.Now().UTC().Format(time.RFC3339), HostNotify: hostNotifyStatus, }, - sessions: map[string]*session{}, - db: db, - hostNotify: hostNotifySink, + sessions: map[string]*session{}, + localNotifications: map[string][]map[string]any{}, + db: db, + hostNotify: hostNotifySink, }, nil } @@ -476,6 +484,10 @@ func (s *Supervisor) consumeNotifications(ctx context.Context, session *session, } func (s *Supervisor) handleNotification(ctx context.Context, session *session, notification map[string]any) { + notification = s.normalizeDirectSecureNotification(ctx, session, notification) + if notification == nil { + return + } receivedAt := time.Now().UTC() event, shouldNotify := NormalizeHostNotification(notification, receivedAt) if record, ok := messageRecordFromDirectIncoming(notification, session.record.IdentityName); ok { @@ -513,6 +525,420 @@ func (s *Supervisor) handleNotification(ctx context.Context, session *session, n s.dispatchHostNotification(ctx, event, shouldNotify) } +func (s *Supervisor) normalizeDirectSecureNotification(ctx context.Context, session *session, notification map[string]any) map[string]any { + if !isDirectSecureIncomingNotification(notification) { + return notification + } + record := session.currentRecord() + if record == nil { + return notification + } + rpcCall := session.secureRPC() + if rpcCall == nil { + return notification + } + client, err := message.NewSecureE2EEClientForRecord(ctx, s.manager, record, func(method string, params map[string]any) (map[string]any, error) { + return rpcCall(ctx, method, params) + }) + if err != nil { + return notification + } + params, _ := notification["params"].(map[string]any) + result, err := client.ProcessIncoming(ctx, params) + if err != nil { + return notification + } + if stringValue(result["state"]) != "decrypted" { + return notification + } + plaintext, ok := result["plaintext"].(map[string]any) + if !ok { + return notification + } + meta, _ := params["meta"].(map[string]any) + originalBody := params["body"] + originalContentType := stringValue(meta["content_type"]) + meta["content_type"] = stringValue(plaintext["application_content_type"]) + params["body"] = plaintextBodyToNotificationBody(plaintext) + params["secure_state"] = "decrypted" + params["secure_wire_content_type"] = originalContentType + params["secure_wire_body"] = originalBody + if message.IsSecureAckPlaintext(plaintext) { + peerDID := stringValue(meta["sender_did"]) + _ = message.FlushQueuedSecureOutbox(ctx, s.resolved, s.manager, record, peerDID, func(method string, params map[string]any) (map[string]any, error) { + return rpcCall(ctx, method, params) + }) + notification["method"] = "direct.secure.ack" + return notification + } + if message.IsSecureInitPlaintext(plaintext) { + notification["method"] = "direct.secure.init" + } + if originalContentType == "application/anp-direct-init+json" { + sessionID := stringValue(mapValue(originalBody)["session_id"]) + messageID := stringValue(meta["message_id"]) + if sessionID != "" && messageID != "" { + ackID := "ack-" + sessionID + if !s.deliverLocalSecureAckInProcess(ctx, record, stringValue(meta["sender_did"]), sessionID, messageID, ackID) { + ackResult, ackErr := client.SendJSON(ctx, stringValue(meta["sender_did"]), message.BuildSecureAckPayload(sessionID, messageID), ackID, ackID) + if ackErr == nil { + s.deliverLocalSecureAck(ctx, record.DID, stringValue(meta["sender_did"]), ackID, ackResult) + } else { + } + } + s.flushPeerQueuedSecureOutbox(ctx, stringValue(meta["sender_did"]), record.DID) + } + } + return notification +} + +func isDirectSecureIncomingNotification(notification map[string]any) bool { + method, _ := notification["method"].(string) + if method != "direct.incoming" { + return false + } + params, _ := notification["params"].(map[string]any) + meta, _ := params["meta"].(map[string]any) + return isSecureDirectWireContentType(stringValue(meta["content_type"])) +} + +func isSecureDirectWireContentType(contentType string) bool { + switch contentType { + case "application/anp-direct-init+json", "application/anp-direct-cipher+json": + return true + default: + return false + } +} + +func secureNotificationFromMessageView(messageView map[string]any) (map[string]any, error) { + body, ok := messageView["content"].(map[string]any) + if !ok { + return nil, fmt.Errorf("content is not a direct-e2ee object") + } + senderDID := stringValue(messageView["sender_did"]) + receiverDID := stringValue(messageView["receiver_did"]) + messageID := stringValue(messageView["id"]) + if senderDID == "" || receiverDID == "" || messageID == "" { + return nil, fmt.Errorf("missing sender_did/receiver_did/id") + } + params := map[string]any{ + "meta": map[string]any{ + "sender_did": senderDID, + "target": map[string]any{"kind": "agent", "did": receiverDID}, + "message_id": messageID, + "profile": "anp.direct.e2ee.v1", + "security_profile": "direct-e2ee", + "content_type": stringValue(messageView["content_type"]), + }, + "body": body, + } + if serverSeq := messageView["server_seq"]; serverSeq != nil { + params["server_seq"] = serverSeq + } + return map[string]any{"method": "direct.incoming", "params": params}, nil +} + +func (s *Supervisor) flushPeerQueuedSecureOutbox(ctx context.Context, ownerDID string, peerDID string) { + s.sessionsMu.Lock() + sessions := make([]*session, 0, len(s.sessions)) + for _, item := range s.sessions { + sessions = append(sessions, item) + } + s.sessionsMu.Unlock() + for _, item := range sessions { + record := item.currentRecord() + if record == nil || record.DID != ownerDID { + continue + } + rpcCall := item.secureRPC() + if rpcCall == nil { + return + } + warnings := message.FlushQueuedSecureOutbox(ctx, s.resolved, s.manager, record, peerDID, func(method string, params map[string]any) (map[string]any, error) { + return rpcCall(ctx, method, params) + }) + log.Printf("listener queued secure outbox flush owner_did=%s peer_did=%s warnings=%v", ownerDID, peerDID, warnings) + return + } +} + +func (s *Supervisor) deliverLocalSecureAck(ctx context.Context, senderDID string, recipientDID string, fallbackMessageID string, ackResult map[string]any) { + targetSession := s.activeSessionByDID(recipientDID) + if targetSession == nil { + return + } + body, _ := ackResult["body"].(map[string]any) + if len(body) == 0 { + return + } + messageID := fallbackString(stringValue(ackResult["message_id"]), fallbackMessageID) + notification := map[string]any{ + "method": "direct.incoming", + "params": map[string]any{ + "meta": map[string]any{ + "sender_did": senderDID, + "target": map[string]any{"kind": "agent", "did": recipientDID}, + "message_id": messageID, + "profile": "anp.direct.e2ee.v1", + "security_profile": "direct-e2ee", + "content_type": "application/anp-direct-cipher+json", + }, + "body": body, + }, + } + s.handleNotification(ctx, targetSession, notification) +} + +func (s *Supervisor) deliverLocalSecureAckInProcess(ctx context.Context, senderRecord *identity.StoredIdentity, recipientDID string, sessionID string, repliedMessageID string, ackMessageID string) bool { + if senderRecord == nil { + log.Printf("listener local secure ack skipped: sender record missing") + return false + } + recipientRecord := s.recordByDID(recipientDID) + if recipientRecord == nil { + log.Printf("listener local secure ack skipped: recipient %s not managed locally", recipientDID) + return false + } + paths, err := s.manager.PathsForIdentity(senderRecord.IdentityName) + if err != nil { + log.Printf("listener local secure ack skipped: sender paths error: %v", err) + return false + } + // Rebuild the sender-side session via file store so we can emit one local encrypted ack + // even when the service/websocket ack path is unavailable during reconnect recovery. + fileStore, err := anpsdk.NewFileSessionStore(filepath.Join(paths.IdentityDir, "p5-e2ee-sessions")) + if err != nil { + log.Printf("listener local secure ack skipped: session store error: %v", err) + return false + } + senderSession, ok, err := fileStore.FindByPeerDID(recipientDID) + if err != nil || !ok { + log.Printf("listener local secure ack skipped: sender session lookup peer=%s ok=%v err=%v", recipientDID, ok, err) + return false + } + candidateSession := senderSession + builder := directe2ee.DirectE2eeSession{} + _, ackBody, err := builder.EncryptFollowUp( + &candidateSession, + directe2ee.DirectEnvelopeMetadata{ + SenderDID: senderRecord.DID, + RecipientDID: recipientDID, + MessageID: ackMessageID, + Profile: "anp.direct.e2ee.v1", + SecurityProfile: "direct-e2ee", + }, + ackMessageID, + directe2ee.NewJSONPlaintext("application/json", message.BuildSecureAckPayload(sessionID, repliedMessageID)), + ) + if err != nil { + log.Printf("listener local secure ack skipped: encrypt follow-up error: %v", err) + return false + } + notification := map[string]any{ + "meta": map[string]any{ + "sender_did": senderRecord.DID, + "target": map[string]any{"kind": "agent", "did": recipientDID}, + "message_id": ackMessageID, + "profile": "anp.direct.e2ee.v1", + "security_profile": "direct-e2ee", + "content_type": "application/anp-direct-cipher+json", + }, + "body": structToMap(ackBody), + } + recipientClient, err := message.NewSecureE2EEClientForRecord(ctx, s.manager, recipientRecord, func(string, map[string]any) (map[string]any, error) { + return nil, fmt.Errorf("local secure ack delivery does not use outbound rpc") + }) + if err != nil { + log.Printf("listener local secure ack skipped: recipient client init error: %v", err) + return false + } + result, err := recipientClient.ProcessIncoming(ctx, notification) + if err != nil || stringValue(result["state"]) != "decrypted" { + recipientPaths, pathErr := s.manager.PathsForIdentity(recipientRecord.IdentityName) + if pathErr != nil { + log.Printf("listener local secure ack skipped: recipient process incoming err=%v state=%s recipientPathsErr=%v", err, stringValue(result["state"]), pathErr) + return false + } + recipientStore, storeErr := anpsdk.NewFileSessionStore(filepath.Join(recipientPaths.IdentityDir, "p5-e2ee-sessions")) + if storeErr != nil { + log.Printf("listener local secure ack skipped: recipient process incoming err=%v state=%s recipientStoreErr=%v", err, stringValue(result["state"]), storeErr) + return false + } + recipientSession, loadErr := recipientStore.LoadSession(sessionID) + if loadErr != nil { + log.Printf("listener local secure ack skipped: recipient process incoming err=%v state=%s recipientLoadErr=%v", err, stringValue(result["state"]), loadErr) + return false + } + var ackCipher directe2ee.DirectCipherBody + rawBody, marshalErr := json.Marshal(notification["body"]) + if marshalErr != nil { + log.Printf("listener local secure ack skipped: recipient process incoming err=%v state=%s marshalAckErr=%v", err, stringValue(result["state"]), marshalErr) + return false + } + if unmarshalErr := json.Unmarshal(rawBody, &ackCipher); unmarshalErr != nil { + log.Printf("listener local secure ack skipped: recipient process incoming err=%v state=%s unmarshalAckErr=%v", err, stringValue(result["state"]), unmarshalErr) + return false + } + _, decryptErr := builder.DecryptFollowUp( + &recipientSession, + directe2ee.DirectEnvelopeMetadata{ + SenderDID: senderRecord.DID, + RecipientDID: recipientDID, + MessageID: ackMessageID, + Profile: "anp.direct.e2ee.v1", + SecurityProfile: "direct-e2ee", + }, + ackCipher, + ) + if decryptErr != nil { + log.Printf("listener local secure ack skipped: recipient process incoming err=%v state=%s decryptFallbackErr=%v", err, stringValue(result["state"]), decryptErr) + return false + } + if saveErr := recipientStore.SaveSession(recipientSession); saveErr != nil { + log.Printf("listener local secure ack skipped: recipient process incoming err=%v state=%s recipientSaveErr=%v", err, stringValue(result["state"]), saveErr) + return false + } + } + if err := fileStore.SaveSession(candidateSession); err != nil { + log.Printf("listener local secure ack skipped: save sender session error: %v", err) + return false + } + if targetSession := s.activeSessionByDID(recipientDID); targetSession != nil { + if rpcCall := targetSession.secureRPC(); rpcCall != nil { + warnings := message.FlushQueuedSecureOutbox(ctx, s.resolved, s.manager, recipientRecord, senderRecord.DID, func(method string, params map[string]any) (map[string]any, error) { + return rpcCall(ctx, method, params) + }) + log.Printf("listener local secure ack delivered recipient=%s sender=%s flush_warnings=%v", recipientRecord.DID, senderRecord.DID, warnings) + } + log.Printf("listener local secure ack delivered recipient=%s sender=%s", recipientRecord.DID, senderRecord.DID) + return true + } + if s.hasRuntimeSessionForDID(recipientDID) { + s.queueLocalNotification(recipientDID, map[string]any{ + "method": "direct.incoming", + "params": notification, + }) + log.Printf("listener local secure ack queued for recipient=%s sender=%s until session activates", recipientRecord.DID, senderRecord.DID) + return true + } + log.Printf("listener local secure ack fallback to network: recipient session not managed recipient=%s sender=%s", recipientRecord.DID, senderRecord.DID) + return false +} + +func (s *Supervisor) activeSessionByDID(did string) *session { + if strings.TrimSpace(did) == "" { + return nil + } + s.sessionsMu.Lock() + defer s.sessionsMu.Unlock() + for _, item := range s.sessions { + record := item.currentRecord() + if record != nil && record.DID == did { + return item + } + } + return nil +} + +func (s *Supervisor) recordByDID(did string) *identity.StoredIdentity { + if strings.TrimSpace(did) == "" || s.manager == nil { + return nil + } + identities, err := s.manager.List() + if err != nil { + return nil + } + for _, summary := range identities { + if summary.DID != did { + continue + } + record, err := s.manager.Load(summary.IdentityName) + if err == nil && record != nil { + return record + } + } + return nil +} + +func (s *Supervisor) hasRuntimeSessionForDID(did string) bool { + if strings.TrimSpace(did) == "" { + return false + } + s.sessionsMu.Lock() + defer s.sessionsMu.Unlock() + for _, item := range s.sessions { + if record := item.currentRecord(); record != nil && record.DID == did { + return true + } + if s.manager == nil { + continue + } + record, err := s.manager.Load(item.identityName) + if err == nil && record != nil && record.DID == did { + return true + } + } + return false +} + +func (s *Supervisor) queueLocalNotification(recipientDID string, notification map[string]any) { + if strings.TrimSpace(recipientDID) == "" || notification == nil { + return + } + s.localNotificationsMu.Lock() + defer s.localNotificationsMu.Unlock() + s.localNotifications[recipientDID] = append(s.localNotifications[recipientDID], notification) +} + +func (s *Supervisor) flushQueuedLocalNotifications(targetSession *session) { + if targetSession == nil { + return + } + record := targetSession.currentRecord() + if record == nil || strings.TrimSpace(record.DID) == "" { + return + } + s.localNotificationsMu.Lock() + queued := append([]map[string]any(nil), s.localNotifications[record.DID]...) + delete(s.localNotifications, record.DID) + s.localNotificationsMu.Unlock() + for _, notification := range queued { + s.handleNotification(targetSession.ctx, targetSession, notification) + } +} + +func structToMap(value any) map[string]any { + raw, err := json.Marshal(value) + if err != nil { + return map[string]any{} + } + var result map[string]any + if err := json.Unmarshal(raw, &result); err != nil { + return map[string]any{} + } + return result +} + +func plaintextBodyToNotificationBody(plaintext map[string]any) map[string]any { + body := map[string]any{} + for _, key := range []string{"conversation_id", "reply_to_message_id", "annotations"} { + if value, ok := plaintext[key]; ok && value != nil { + body[key] = value + } + } + if text := stringValue(plaintext["text"]); text != "" { + body["text"] = text + } + if payload, ok := plaintext["payload"]; ok && payload != nil { + body["payload"] = payload + } + if payloadB64U := stringValue(plaintext["payload_b64u"]); payloadB64U != "" { + body["payload_b64u"] = payloadB64U + } + return body +} + func (s *Supervisor) dispatchHostNotification(ctx context.Context, event *HostNotificationEvent, shouldNotify bool) { if !shouldNotify || event == nil || s.hostNotify == nil { return @@ -549,6 +975,21 @@ func messageRecordFromDirectIncoming(notification map[string]any, identityName s if sentAt == "" { sentAt = time.Now().UTC().Format(time.RFC3339) } + contentValue := body["text"] + if text := stringValue(body["text"]); text == "" { + switch { + case body["payload"] != nil: + contentValue = body["payload"] + case stringValue(body["payload_b64u"]) != "": + contentValue = body["payload_b64u"] + default: + contentValue = body + } + } + content := stringValue(contentValue) + if content == "" { + content = metadataValue(contentValue) + } return store.MessageRecord{ MsgID: stringValue(meta["message_id"]), OwnerDID: targetDID, @@ -557,7 +998,8 @@ func messageRecordFromDirectIncoming(notification map[string]any, identityName s SenderDID: senderDID, ReceiverDID: targetDID, ContentType: contentType, - Content: stringValue(body["text"]), + Content: content, + IsE2EE: stringValue(meta["security_profile"]) == "direct-e2ee" || stringValue(params["secure_state"]) == "decrypted", SentAt: sentAt, IsRead: false, Metadata: metadataValue(params), @@ -1042,8 +1484,13 @@ func (s *Supervisor) runSessionLoop(session *session) { session.markConnected(record, client) session.signalInitial(nil) s.refreshStatus() + s.flushQueuedLocalNotifications(session) + publishCtx, publishCancel := context.WithCancel(session.ctx) + go s.retryPublishSecurePrekeys(publishCtx, record) + go s.pollUnreadSecureDirectInbox(publishCtx, session, client) err = s.consumeNotifications(session.ctx, session, client) + publishCancel() _ = client.Close() session.markDisconnected(err) s.refreshStatus() @@ -1057,6 +1504,161 @@ func (s *Supervisor) runSessionLoop(session *session) { } } +func (s *Supervisor) retryPublishSecurePrekeys(ctx context.Context, record *identity.StoredIdentity) { + for { + warnings := message.PublishSecurePrekeys(ctx, s.resolved, s.manager, record) + if len(warnings) == 0 { + return + } + log.Printf("listener secure prekey publish retry identity=%s warnings=%s", record.IdentityName, strings.Join(warnings, "; ")) + if !sleepWithContext(ctx, time.Second) { + return + } + } +} + +func (s *Supervisor) syncUnreadSecureDirectInbox(ctx context.Context, session *session, client *WSClient) { + record := session.currentRecord() + if record == nil { + return + } + syncCtx, cancel := context.WithTimeout(ctx, 15*time.Second) + defer cancel() + params := message.BuildInboxRPCParams(record, message.InboxRequest{Scope: "direct", UnreadOnly: true, Limit: 100}) + result, err := client.SendRPC(syncCtx, "inbox.get", params) + if err != nil { + return + } + items, _ := result["messages"].([]any) + for _, item := range items { + messageView, ok := item.(map[string]any) + if !ok || !isSecureDirectWireContentType(stringValue(messageView["content_type"])) { + continue + } + ownerDID := stringValue(messageView["receiver_did"]) + if ownerDID == "" { + ownerDID = session.record.DID + } + if _, err := store.GetMessageByID(syncCtx, s.db, stringValue(messageView["id"]), ownerDID, session.identityName); err == nil { + continue + } else if !errors.Is(err, sql.ErrNoRows) { + continue + } + notification, err := secureNotificationFromMessageView(messageView) + if err != nil { + continue + } + log.Printf("listener secure backlog replay identity=%s message_id=%s content_type=%s", session.identityName, stringValue(messageView["id"]), stringValue(messageView["content_type"])) + s.handleNotification(syncCtx, session, notification) + } +} + +func (s *Supervisor) pollUnreadSecureDirectInbox(ctx context.Context, session *session, client *WSClient) { + s.syncUnreadSecureDirectInbox(ctx, session, client) + s.syncPendingConfirmationSecureHistory(ctx, session, client) + ticker := time.NewTicker(2 * time.Second) + defer ticker.Stop() + for { + select { + case <-ctx.Done(): + return + case <-ticker.C: + s.syncUnreadSecureDirectInbox(ctx, session, client) + s.syncPendingConfirmationSecureHistory(ctx, session, client) + } + } +} + +func (s *Supervisor) syncPendingConfirmationSecureHistory(ctx context.Context, session *session, client *WSClient) { + record := session.currentRecord() + if record == nil { + return + } + peerDIDs := s.pendingConfirmationPeerDIDs(record.IdentityName) + if len(peerDIDs) == 0 { + return + } + for _, peerDID := range peerDIDs { + syncCtx, cancel := context.WithTimeout(ctx, 15*time.Second) + params, buildErr := message.BuildHistoryRPCParams(record, message.HistoryRequest{With: peerDID, Limit: 50}) + if buildErr != nil { + cancel() + continue + } + result, err := client.SendRPC(syncCtx, "direct.get_history", params) + cancel() + if err != nil { + continue + } + items, _ := result["messages"].([]any) + for _, item := range items { + messageView, ok := item.(map[string]any) + if !ok || !isSecureDirectWireContentType(stringValue(messageView["content_type"])) { + continue + } + if stringValue(messageView["sender_did"]) == record.DID { + continue + } + ownerDID := stringValue(messageView["receiver_did"]) + if ownerDID == "" { + ownerDID = record.DID + } + if _, err := store.GetMessageByID(ctx, s.db, stringValue(messageView["id"]), ownerDID, session.identityName); err == nil { + continue + } else if !errors.Is(err, sql.ErrNoRows) { + continue + } + notification, err := secureNotificationFromMessageView(messageView) + if err != nil { + continue + } + s.handleNotification(ctx, session, notification) + } + } +} + +func (s *Supervisor) pendingConfirmationPeerDIDs(identityName string) []string { + if s.manager == nil || strings.TrimSpace(identityName) == "" { + return nil + } + paths, err := s.manager.PathsForIdentity(identityName) + if err != nil { + return nil + } + entries, err := filepath.Glob(filepath.Join(paths.IdentityDir, "p5-e2ee-sessions", "*.json")) + if err != nil { + return nil + } + peers := make([]string, 0, len(entries)) + seen := map[string]struct{}{} + for _, path := range entries { + var payload struct { + PeerDID string `json:"peer_did"` + Status string `json:"status"` + } + if err := readJSONFile(path, &payload); err != nil { + continue + } + if payload.Status != "pending-confirmation" || strings.TrimSpace(payload.PeerDID) == "" { + continue + } + if _, ok := seen[payload.PeerDID]; ok { + continue + } + seen[payload.PeerDID] = struct{}{} + peers = append(peers, payload.PeerDID) + } + return peers +} + +func readJSONFile(path string, out any) error { + raw, err := os.ReadFile(path) + if err != nil { + return err + } + return json.Unmarshal(raw, out) +} + func (s *Supervisor) connectSession(identityName string) (*identity.StoredIdentity, *WSClient, error) { record, err := s.manager.Load(identityName) if err != nil { @@ -1109,6 +1711,18 @@ func (s *session) currentClient() *WSClient { return s.client } +func (s *session) secureRPC() func(context.Context, string, map[string]any) (map[string]any, error) { + s.mu.RLock() + defer s.mu.RUnlock() + if s.secureRPCCall != nil { + return s.secureRPCCall + } + if s.client == nil { + return nil + } + return s.client.SendRPC +} + func (s *session) currentRecord() *identity.StoredIdentity { s.mu.RLock() defer s.mu.RUnlock() diff --git a/internal/runtime/listener/server_test.go b/internal/runtime/listener/server_test.go index 7ef4f12..30a84e2 100644 --- a/internal/runtime/listener/server_test.go +++ b/internal/runtime/listener/server_test.go @@ -2,17 +2,25 @@ package listener import ( "context" + "crypto/ecdh" "encoding/json" + "fmt" "net/http" "net/http/httptest" + "os" "path/filepath" "strings" + "sync" "sync/atomic" "testing" "time" + anp "github.com/agent-network-protocol/anp/golang" + directe2ee "github.com/agent-network-protocol/anp/golang/direct_e2ee" + "github.com/agentconnect/awiki-cli/internal/anpsdk" appconfig "github.com/agentconnect/awiki-cli/internal/config" "github.com/agentconnect/awiki-cli/internal/identity" + "github.com/agentconnect/awiki-cli/internal/message" "github.com/agentconnect/awiki-cli/internal/runtime" "github.com/agentconnect/awiki-cli/internal/store" "github.com/coder/websocket" @@ -166,6 +174,428 @@ func TestMessageRecordFromGroupIncomingUsesProtocolFieldsOnly(t *testing.T) { } } +func TestHandleNotificationDecryptsSecureDirectIncomingAndStoresPlaintext(t *testing.T) { + t.Parallel() + + root := t.TempDir() + paths := appconfig.Paths{ + WorkspaceHomeDir: filepath.Join(root, ".awiki-cli"), + IdentityDir: filepath.Join(root, "identities"), + LegacyCredentialsDir: filepath.Join(root, "legacy"), + DataDir: filepath.Join(root, "data"), + StateDir: filepath.Join(root, "state"), + DatabaseFile: filepath.Join(root, "data", "awiki-cli.db"), + } + manager := identity.NewManager(paths) + aliceGenerated, err := identity.GenerateIdentity(identity.GenerateOptions{ + Hostname: "awiki.ai", + PathPrefix: []string{"user"}, + ProofDomain: "awiki.ai", + }) + if err != nil { + t.Fatalf("GenerateIdentity(alice) error = %v", err) + } + if _, err := manager.Save(identity.SaveInput{ + IdentityName: "alice", + DID: aliceGenerated.DID, + UniqueID: aliceGenerated.UniqueID, + UserID: "user-alice-123", + DisplayName: "Alice", + Handle: "alice", + JWTToken: "token-123", + DIDDocument: aliceGenerated.DIDDocument, + Key1PrivatePEM: aliceGenerated.Key1PrivatePEM, + Key1PublicPEM: aliceGenerated.Key1PublicPEM, + E2EESigningPrivatePEM: aliceGenerated.E2EESigningPrivatePEM, + E2EEAgreementPrivatePEM: aliceGenerated.E2EEAgreementPrivatePEM, + }); err != nil { + t.Fatalf("manager.Save(alice) error = %v", err) + } + resolved := &appconfig.Resolved{ + Paths: paths, + ServiceBaseURL: "https://awiki.test", + DIDDomain: "awiki.ai", + RuntimeMode: "websocket", + ActiveIdentity: "alice", + } + supervisor, err := NewSupervisor(resolved) + if err != nil { + t.Fatalf("NewSupervisor() error = %v", err) + } + defer supervisor.Close() + + record, err := manager.Load("alice") + if err != nil { + t.Fatalf("manager.Load(alice) error = %v", err) + } + aliceClient, err := message.NewSecureE2EEClientForRecord(context.Background(), manager, record, func(string, map[string]any) (map[string]any, error) { + return map[string]any{}, nil + }) + if err != nil { + t.Fatalf("NewSecureE2EEClientForRecord(alice) error = %v", err) + } + aliceBundle, err := aliceClient.EnsureFreshPrekeyBundle() + if err != nil { + t.Fatalf("EnsureFreshPrekeyBundle(alice) error = %v", err) + } + alicePaths, err := manager.PathsForIdentity("alice") + if err != nil { + t.Fatalf("PathsForIdentity(alice) error = %v", err) + } + aliceOPKStore, err := anpsdk.NewFileOneTimePrekeyStore(filepath.Join(alicePaths.IdentityDir, "p5-one-time-prekeys")) + if err != nil { + t.Fatalf("NewFileOneTimePrekeyStore(alice) error = %v", err) + } + aliceOPKs, err := aliceOPKStore.ListOneTimePrekeys() + if err != nil { + t.Fatalf("ListOneTimePrekeys(alice) error = %v", err) + } + if len(aliceOPKs) == 0 { + t.Fatal("expected at least one Alice OPK") + } + + bobGenerated, err := identity.GenerateIdentity(identity.GenerateOptions{ + Hostname: "awiki.ai", + PathPrefix: []string{"user"}, + ProofDomain: "awiki.ai", + }) + if err != nil { + t.Fatalf("GenerateIdentity(bob) error = %v", err) + } + if _, err := manager.Save(identity.SaveInput{ + IdentityName: "bob", + DID: bobGenerated.DID, + UniqueID: bobGenerated.UniqueID, + UserID: "user-bob-123", + DisplayName: "Bob", + Handle: "bob", + JWTToken: "token-bob", + DIDDocument: bobGenerated.DIDDocument, + Key1PrivatePEM: bobGenerated.Key1PrivatePEM, + Key1PublicPEM: bobGenerated.Key1PublicPEM, + E2EESigningPrivatePEM: bobGenerated.E2EESigningPrivatePEM, + E2EEAgreementPrivatePEM: bobGenerated.E2EEAgreementPrivatePEM, + }); err != nil { + t.Fatalf("manager.Save(bob) error = %v", err) + } + bobPrivate, err := privateKeyFromPEM(t, bobGenerated.E2EEAgreementPrivatePEM) + if err != nil { + t.Fatalf("privateKeyFromPEM(bob) error = %v", err) + } + recipientStaticPublic, err := directe2ee.ExtractX25519PublicKey(record.DIDDocument, aliceBundle.StaticKeyAgreementID) + if err != nil { + t.Fatalf("ExtractX25519PublicKey(alice) error = %v", err) + } + recipientSignedPrekey := decodeFixed32Value(t, aliceBundle.SignedPrekey.PublicKeyB64U) + recipientOneTimePrekey := decodeFixed32Value(t, aliceOPKs[0].PublicKeyB64U) + metadata := directe2ee.DirectEnvelopeMetadata{ + SenderDID: bobGenerated.DID, + RecipientDID: record.DID, + MessageID: "msg-secure-init-001", + Profile: "anp.direct.e2ee.v1", + SecurityProfile: "direct-e2ee", + } + sessionBuilder := directe2ee.DirectE2eeSession{} + _, _, initBody, err := sessionBuilder.InitiateSessionWithOPK( + metadata, + "msg-secure-init-001", + bobGenerated.DID+"#key-3", + bobPrivate, + aliceBundle, + recipientStaticPublic, + recipientSignedPrekey, + &recipientOneTimePrekey, + aliceOPKs[0].KeyID, + directe2ee.NewTextPlaintext("text/plain", "hello secure listener"), + ) + if err != nil { + t.Fatalf("InitiateSessionWithOPK() error = %v", err) + } + notification := map[string]any{ + "jsonrpc": "2.0", + "method": "direct.incoming", + "params": map[string]any{ + "meta": map[string]any{ + "sender_did": bobGenerated.DID, + "message_id": metadata.MessageID, + "created_at": "2026-04-07T00:00:00Z", + "profile": metadata.Profile, + "security_profile": metadata.SecurityProfile, + "content_type": "application/anp-direct-init+json", + "target": map[string]any{ + "kind": "agent", + "did": record.DID, + }, + }, + "body": mustJSONMap(t, initBody), + }, + } + + var ackSent atomic.Int32 + supervisor.handleNotification(context.Background(), &session{ + identityName: "alice", + record: record, + secureRPCCall: func(_ context.Context, method string, params map[string]any) (map[string]any, error) { + if method != "direct.send" { + return nil, fmt.Errorf("unexpected rpc method: %s", method) + } + ackSent.Add(1) + meta := params["meta"].(map[string]any) + return map[string]any{ + "accepted": true, + "message_id": meta["message_id"], + "operation_id": meta["operation_id"], + "target_did": meta["target"].(map[string]any)["did"], + "accepted_at": "2026-04-07T00:00:01Z", + }, nil + }, + }, notification) + + row, err := store.GetMessageByID(context.Background(), supervisor.db, metadata.MessageID, record.DID, record.IdentityName) + if err != nil { + t.Fatalf("GetMessageByID() error = %v", err) + } + if got := row["content"]; got != "hello secure listener" { + t.Fatalf("stored content = %#v, want plaintext", got) + } + if got := row["content_type"]; got != "text/plain" { + t.Fatalf("stored content_type = %#v, want text/plain", got) + } + if got := row["is_e2ee"]; got != int64(1) { + t.Fatalf("stored is_e2ee = %#v, want 1", got) + } + sessionEntries, err := os.ReadDir(filepath.Join(alicePaths.IdentityDir, "p5-e2ee-sessions")) + if err != nil { + t.Fatalf("ReadDir(session store) error = %v", err) + } + if len(sessionEntries) != 1 { + t.Fatalf("len(sessionEntries) = %d, want 1", len(sessionEntries)) + } + sessionBytes, err := os.ReadFile(filepath.Join(alicePaths.IdentityDir, "p5-e2ee-sessions", sessionEntries[0].Name())) + if err != nil { + t.Fatalf("ReadFile(session) error = %v", err) + } + var savedSession map[string]any + if err := json.Unmarshal(sessionBytes, &savedSession); err != nil { + t.Fatalf("json.Unmarshal(session) error = %v", err) + } + if got := savedSession["status"]; got != "established" { + t.Fatalf("saved session status = %#v, want established", got) + } + if _, _, err := aliceOPKStore.LoadOneTimePrekey(aliceOPKs[0].KeyID); err == nil { + t.Fatalf("one-time prekey %s should be consumed by secure listener decrypt", aliceOPKs[0].KeyID) + } + if ackSent.Load() != 1 { + t.Fatalf("ackSent = %d, want 1 auto secure ack", ackSent.Load()) + } +} + +func TestDeliverLocalSecureAckInProcessPromotesPendingInitiatorSession(t *testing.T) { + t.Parallel() + + root := t.TempDir() + paths := appconfig.Paths{ + WorkspaceHomeDir: filepath.Join(root, ".awiki-cli"), + IdentityDir: filepath.Join(root, "identities"), + LegacyCredentialsDir: filepath.Join(root, "legacy"), + DataDir: filepath.Join(root, "data"), + StateDir: filepath.Join(root, "state"), + DatabaseFile: filepath.Join(root, "data", "awiki-cli.db"), + } + manager := identity.NewManager(paths) + resolved := &appconfig.Resolved{ + Paths: paths, + ServiceBaseURL: "https://awiki.test", + DIDDomain: "awiki.ai", + RuntimeMode: "websocket", + ActiveIdentity: "alice", + } + supervisor, err := NewSupervisor(resolved) + if err != nil { + t.Fatalf("NewSupervisor() error = %v", err) + } + defer supervisor.Close() + + aliceGenerated, err := identity.GenerateIdentity(identity.GenerateOptions{Hostname: "awiki.ai", PathPrefix: []string{"user"}, ProofDomain: "awiki.ai"}) + if err != nil { + t.Fatalf("GenerateIdentity(alice) error = %v", err) + } + if _, err := manager.Save(identity.SaveInput{ + IdentityName: "alice", + DID: aliceGenerated.DID, + UniqueID: aliceGenerated.UniqueID, + UserID: "user-alice", + DisplayName: "Alice", + Handle: "alice", + JWTToken: "token-alice", + DIDDocument: aliceGenerated.DIDDocument, + Key1PrivatePEM: aliceGenerated.Key1PrivatePEM, + Key1PublicPEM: aliceGenerated.Key1PublicPEM, + E2EESigningPrivatePEM: aliceGenerated.E2EESigningPrivatePEM, + E2EEAgreementPrivatePEM: aliceGenerated.E2EEAgreementPrivatePEM, + }); err != nil { + t.Fatalf("manager.Save(alice) error = %v", err) + } + bobGenerated, err := identity.GenerateIdentity(identity.GenerateOptions{Hostname: "awiki.ai", PathPrefix: []string{"user"}, ProofDomain: "awiki.ai"}) + if err != nil { + t.Fatalf("GenerateIdentity(bob) error = %v", err) + } + if _, err := manager.Save(identity.SaveInput{ + IdentityName: "bob", + DID: bobGenerated.DID, + UniqueID: bobGenerated.UniqueID, + UserID: "user-bob", + DisplayName: "Bob", + Handle: "bob", + JWTToken: "token-bob", + DIDDocument: bobGenerated.DIDDocument, + Key1PrivatePEM: bobGenerated.Key1PrivatePEM, + Key1PublicPEM: bobGenerated.Key1PublicPEM, + E2EESigningPrivatePEM: bobGenerated.E2EESigningPrivatePEM, + E2EEAgreementPrivatePEM: bobGenerated.E2EEAgreementPrivatePEM, + }); err != nil { + t.Fatalf("manager.Save(bob) error = %v", err) + } + + aliceRecord, err := manager.Load("alice") + if err != nil { + t.Fatalf("manager.Load(alice) error = %v", err) + } + bobRecord, err := manager.Load("bob") + if err != nil { + t.Fatalf("manager.Load(bob) error = %v", err) + } + + bobClient, err := message.NewSecureE2EEClientForRecord(context.Background(), manager, bobRecord, func(string, map[string]any) (map[string]any, error) { + return map[string]any{}, nil + }) + if err != nil { + t.Fatalf("NewSecureE2EEClientForRecord(bob) error = %v", err) + } + bobBundle, err := bobClient.EnsureFreshPrekeyBundle() + if err != nil { + t.Fatalf("EnsureFreshPrekeyBundle(bob) error = %v", err) + } + bobPaths, err := manager.PathsForIdentity("bob") + if err != nil { + t.Fatalf("PathsForIdentity(bob) error = %v", err) + } + bobOPKStore, err := anpsdk.NewFileOneTimePrekeyStore(filepath.Join(bobPaths.IdentityDir, "p5-one-time-prekeys")) + if err != nil { + t.Fatalf("NewFileOneTimePrekeyStore(bob) error = %v", err) + } + bobOPKs, err := bobOPKStore.ListOneTimePrekeys() + if err != nil { + t.Fatalf("ListOneTimePrekeys(bob) error = %v", err) + } + if len(bobOPKs) == 0 { + t.Fatal("expected at least one Bob OPK") + } + + alicePrivate, err := privateKeyFromPEM(t, aliceGenerated.E2EEAgreementPrivatePEM) + if err != nil { + t.Fatalf("privateKeyFromPEM(alice) error = %v", err) + } + recipientStaticPublic, err := directe2ee.ExtractX25519PublicKey(bobRecord.DIDDocument, bobBundle.StaticKeyAgreementID) + if err != nil { + t.Fatalf("ExtractX25519PublicKey(bob) error = %v", err) + } + recipientSignedPrekey := decodeFixed32Value(t, bobBundle.SignedPrekey.PublicKeyB64U) + recipientOneTimePrekey := decodeFixed32Value(t, bobOPKs[0].PublicKeyB64U) + initMetadata := directe2ee.DirectEnvelopeMetadata{ + SenderDID: aliceRecord.DID, + RecipientDID: bobRecord.DID, + MessageID: "msg-pending-init-001", + Profile: "anp.direct.e2ee.v1", + SecurityProfile: "direct-e2ee", + } + builder := directe2ee.DirectE2eeSession{} + aliceSession, _, initBody, err := builder.InitiateSessionWithOPK( + initMetadata, + "msg-pending-init-001", + aliceRecord.DID+"#key-3", + alicePrivate, + bobBundle, + recipientStaticPublic, + recipientSignedPrekey, + &recipientOneTimePrekey, + bobOPKs[0].KeyID, + directe2ee.NewTextPlaintext("text/plain", "hello pending"), + ) + if err != nil { + t.Fatalf("InitiateSessionWithOPK() error = %v", err) + } + alicePaths, err := manager.PathsForIdentity("alice") + if err != nil { + t.Fatalf("PathsForIdentity(alice) error = %v", err) + } + aliceSessionStore, err := anpsdk.NewFileSessionStore(filepath.Join(alicePaths.IdentityDir, "p5-e2ee-sessions")) + if err != nil { + t.Fatalf("NewFileSessionStore(alice) error = %v", err) + } + if err := aliceSessionStore.SaveSession(aliceSession); err != nil { + t.Fatalf("SaveSession(alice) error = %v", err) + } + + bobStatic, err := privateKeyFromPEM(t, bobGenerated.E2EEAgreementPrivatePEM) + if err != nil { + t.Fatalf("privateKeyFromPEM(bob) error = %v", err) + } + bobSPKStore, err := anpsdk.NewFileSignedPrekeyStore(filepath.Join(bobPaths.IdentityDir, "p5-signed-prekeys")) + if err != nil { + t.Fatalf("NewFileSignedPrekeyStore(bob) error = %v", err) + } + bobSPKMaterial, _, err := bobSPKStore.LoadSignedPrekey(initBody.RecipientSignedPrekeyID) + if err != nil { + t.Fatalf("LoadSignedPrekey(bob) error = %v", err) + } + bobSPKPrivate, err := ecdh.X25519().NewPrivateKey(bobSPKMaterial.Bytes) + if err != nil { + t.Fatalf("NewPrivateKey(bob spk) error = %v", err) + } + bobOPKMaterial, _, err := bobOPKStore.LoadOneTimePrekey(initBody.RecipientOneTimePrekeyID) + if err != nil { + t.Fatalf("LoadOneTimePrekey(bob) error = %v", err) + } + bobOPKPrivate, err := ecdh.X25519().NewPrivateKey(bobOPKMaterial.Bytes) + if err != nil { + t.Fatalf("NewPrivateKey(bob opk) error = %v", err) + } + senderStaticPublic, err := directe2ee.ExtractX25519PublicKey(aliceRecord.DIDDocument, initBody.SenderStaticKeyAgreementID) + if err != nil { + t.Fatalf("ExtractX25519PublicKey(alice) error = %v", err) + } + bobSession, _, err := builder.AcceptIncomingInitWithOPK(initMetadata, bobRecord.DID+"#key-3", bobStatic, bobSPKPrivate, bobOPKPrivate, senderStaticPublic, initBody) + if err != nil { + t.Fatalf("AcceptIncomingInitWithOPK() error = %v", err) + } + bobSessionStore, err := anpsdk.NewFileSessionStore(filepath.Join(bobPaths.IdentityDir, "p5-e2ee-sessions")) + if err != nil { + t.Fatalf("NewFileSessionStore(bob) error = %v", err) + } + if err := bobSessionStore.SaveSession(bobSession); err != nil { + t.Fatalf("SaveSession(bob) error = %v", err) + } + + supervisor.sessions["alice"] = &session{identityName: "alice", record: aliceRecord} + supervisor.sessions["bob"] = &session{identityName: "bob", record: bobRecord} + + ackID := "ack-" + aliceSession.SessionID + if ok := supervisor.deliverLocalSecureAckInProcess(context.Background(), bobRecord, aliceRecord.DID, aliceSession.SessionID, initMetadata.MessageID, ackID); !ok { + t.Fatal("deliverLocalSecureAckInProcess() = false, want true") + } + updatedAliceSession, ok, err := aliceSessionStore.FindByPeerDID(bobRecord.DID) + if err != nil { + t.Fatalf("FindByPeerDID(alice) error = %v", err) + } + if !ok { + t.Fatal("expected Alice session to remain present") + } + if updatedAliceSession.Status != directe2ee.SessionStatusEstablished { + t.Fatalf("Alice session status = %q, want established", updatedAliceSession.Status) + } +} + func TestRecordsFromGroupStateChangedBuildsMemberAndSystemMessage(t *testing.T) { t.Parallel() @@ -222,6 +652,38 @@ func TestSessionLoopReconnectsAndStoresNotifications(t *testing.T) { return } defer conn.Close(websocket.StatusNormalClosure, "done") + var writeMu sync.Mutex + go func() { + for { + _, raw, err := conn.Read(r.Context()) + if err != nil { + return + } + var request map[string]any + if err := json.Unmarshal(raw, &request); err != nil { + continue + } + if request["method"] != "inbox.get" { + continue + } + response, err := json.Marshal(map[string]any{ + "jsonrpc": "2.0", + "id": request["id"], + "result": map[string]any{ + "messages": []any{}, + }, + }) + if err != nil { + return + } + writeMu.Lock() + err = conn.Write(r.Context(), websocket.MessageText, response) + writeMu.Unlock() + if err != nil { + return + } + } + }() index := connectionCount.Add(1) payload := map[string]any{ @@ -250,7 +712,10 @@ func TestSessionLoopReconnectsAndStoresNotifications(t *testing.T) { } writeCtx, cancel := context.WithTimeout(r.Context(), 5*time.Second) defer cancel() - if err := conn.Write(writeCtx, websocket.MessageText, raw); err != nil { + writeMu.Lock() + err = conn.Write(writeCtx, websocket.MessageText, raw) + writeMu.Unlock() + if err != nil { t.Errorf("conn.Write() error = %v", err) return } @@ -558,3 +1023,39 @@ func createTestIdentity(t *testing.T, manager *identity.Manager, input identity. t.Fatalf("Save() error = %v", err) } } + +func privateKeyFromPEM(t *testing.T, pemValue string) (*ecdh.PrivateKey, error) { + t.Helper() + privateKey, err := anp.PrivateKeyFromPEM(pemValue) + if err != nil { + return nil, err + } + return ecdh.X25519().NewPrivateKey(privateKey.Bytes) +} + +func decodeFixed32Value(t *testing.T, value string) [32]byte { + t.Helper() + decoded, err := anp.DecodeBase64URL(value) + if err != nil { + t.Fatalf("DecodeBase64URL(%q) error = %v", value, err) + } + if len(decoded) != 32 { + t.Fatalf("DecodeBase64URL(%q) len = %d, want 32", value, len(decoded)) + } + var result [32]byte + copy(result[:], decoded) + return result +} + +func mustJSONMap(t *testing.T, value any) map[string]any { + t.Helper() + raw, err := json.Marshal(value) + if err != nil { + t.Fatalf("json.Marshal() error = %v", err) + } + var result map[string]any + if err := json.Unmarshal(raw, &result); err != nil { + t.Fatalf("json.Unmarshal() error = %v", err) + } + return result +} diff --git a/internal/store/import.go b/internal/store/import.go index 9a677ad..13e44ef 100644 --- a/internal/store/import.go +++ b/internal/store/import.go @@ -14,9 +14,7 @@ import ( func ScanLegacyDatabase(ctx context.Context, paths appconfig.Paths) (*LegacyScan, error) { legacyPath := strings.TrimSpace(paths.LegacyDataDir) - if strings.HasSuffix(strings.ToLower(legacyPath), ".db") { - legacyPath = legacyPath - } else { + if !strings.HasSuffix(strings.ToLower(legacyPath), ".db") { legacyPath = filepath.Join(paths.LegacyDataDir, "database", "awiki.db") } scan := &LegacyScan{ diff --git a/scripts/release/build-anp-mls.sh b/scripts/release/build-anp-mls.sh new file mode 100755 index 0000000..e7931ec --- /dev/null +++ b/scripts/release/build-anp-mls.sh @@ -0,0 +1,93 @@ +#!/usr/bin/env bash +set -euo pipefail + +usage() { + cat <<'USAGE' +Build and stage the Rust anp-mls binary for awiki-cli release artifacts. + +Usage: + scripts/release/build-anp-mls.sh [--dry-run] [--output DIR] + +Environment: + ANP_MLS_SOURCE_DIR Path to the anp/anp/rust crate (default: ../anp/anp/rust) + AWIKI_ANP_MLS_RELEASE_DIR Output directory (default: dist/anp-mls) + CARGO Cargo binary (default: cargo) + +The staged binary can be bundled next to awiki-cli artifacts or installed on PATH. +Users can always override discovery with AWIKI_ANP_MLS_BINARY=/absolute/path/to/anp-mls. +USAGE +} + +repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" +source_dir="${ANP_MLS_SOURCE_DIR:-$(cd "${repo_root}/.." && pwd)/anp/anp/rust}" +output_dir="${AWIKI_ANP_MLS_RELEASE_DIR:-${repo_root}/dist/anp-mls}" +dry_run=0 + +while [[ $# -gt 0 ]]; do + case "$1" in + --dry-run) + dry_run=1 + shift + ;; + --output) + if [[ $# -lt 2 ]]; then + echo "--output requires a directory" >&2 + exit 2 + fi + output_dir="$2" + shift 2 + ;; + -h|--help) + usage + exit 0 + ;; + *) + echo "unknown argument: $1" >&2 + usage >&2 + exit 2 + ;; + esac +done + +cargo_bin="${CARGO:-cargo}" +manifest_path="${source_dir}/Cargo.toml" +if [[ ! -f "${manifest_path}" ]]; then + echo "anp-mls Cargo.toml not found: ${manifest_path}" >&2 + echo "Set ANP_MLS_SOURCE_DIR to the sibling anp/anp/rust crate." >&2 + exit 1 +fi + +case "$(uname -s)" in + Darwin) os_name="darwin" ;; + Linux) os_name="linux" ;; + *) os_name="$(uname -s | tr '[:upper:]' '[:lower:]')" ;; +esac +case "$(uname -m)" in + x86_64|amd64) arch_name="amd64" ;; + arm64|aarch64) arch_name="arm64" ;; + *) arch_name="$(uname -m)" ;; +esac + +bin_name="anp-mls" +if [[ "${os_name}" == "windows" ]]; then + bin_name="anp-mls.exe" +fi +src_bin="${source_dir}/target/release/${bin_name}" +stage_dir="${output_dir}/${os_name}-${arch_name}" +staged_bin="${stage_dir}/${bin_name}" + +if [[ ${dry_run} -eq 1 ]]; then + cat < ${staged_bin} +EOF_DRY + exit 0 +fi + +"${cargo_bin}" build --manifest-path "${manifest_path}" --bin anp-mls --release +mkdir -p "${stage_dir}" +cp "${src_bin}" "${staged_bin}" +chmod 0755 "${staged_bin}" + +echo "anp-mls staged at ${staged_bin}" +echo "Set AWIKI_ANP_MLS_BINARY=${staged_bin} or place it on PATH for awiki-cli group E2EE." diff --git a/skills/references/04-groups.md b/skills/references/04-groups.md index 503b503..7f9cd3d 100644 --- a/skills/references/04-groups.md +++ b/skills/references/04-groups.md @@ -40,6 +40,7 @@ This file is a **reference**, not an entry skill. Load it only when the task cle - Need to inspect metadata or policy -> `group get` - Need to join an open group -> `group join` - Need to add or remove one member -> `group add` / `group remove` +- Need an E2EE member to leave safely -> `group leave --e2ee` creates a hidden leave request, then an owner/admin processes it with `group e2ee process-leave-request` - Need to change the name, description, or policy -> `group update` - Need to send text to the group -> use `03-messaging.md` @@ -50,7 +51,8 @@ This file is a **reference**, not an entry skill. Load it only when the task cle - `awiki-cli group join --group [--reason "..."]` - `awiki-cli group add --group --member [--role ...]` - `awiki-cli group remove --group --member [--reason "..."]` -- `awiki-cli group leave --group ` +- `awiki-cli group leave --group [--reason "..."] [--e2ee]` +- `awiki-cli group e2ee process-leave-request --group --member [--leave-request-id ]` - `awiki-cli group update --group [--name ...] [--description ...] [...]` - `awiki-cli group members --group [--limit ]` - `awiki-cli group messages --group [--limit ] [--cursor ]`