Skip to content

merge: integrate upstream v0.3.0 with security hardening and doc cleanup - #6

Open
1263-ux wants to merge 32 commits into
open-agent-power:mainfrom
1263-ux:codex/upstream-v0.2.3-integration
Open

merge: integrate upstream v0.3.0 with security hardening and doc cleanup#6
1263-ux wants to merge 32 commits into
open-agent-power:mainfrom
1263-ux:codex/upstream-v0.2.3-integration

Conversation

@1263-ux

@1263-ux 1263-ux commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

概述

open-agent-power/main(v0.3.0,50 个提交)合并到本分支,叠加 28 个本分支提交。总共 32 commits66 files, +2,033 / −2,394

前 22 个提交是合并前的本分支工作(Book POC 闭环验收、过期资产清理、原子写入加固);后 4 个是合并后的收尾(构建修复、安全加固、文档死链修复)。

变更分类

🟢 吸收上游 v0.3.0(合并自动合入)

  • oks eval recall / oks eval compare — 离线召回质量评测
  • oks trace start/append/judge/feedback/propose/finish/validate — 执行轨迹(provenance,不参与召回)
  • CONSTITUTION P0(Python ≥3.12)、P3/P5/A2/A4 修正
  • 飞书 Worker 模块化:scripts/feishu_worker/ 子包(12 模块)
  • 打包修复:.codex/.agents 入 wheel,维护者 skill 排包

🔴 安全加固(3 个独立修复)

这些是本分支发现并修复的,非上游代码的原有问题:

漏洞 严重程度 修复
Draft status: rejected 仍可晋升为 Wiki store.py:promote_draft() — 晋升前检查 meta.get("status") == "rejected",匹配 CONSTITUTION A3「人工门禁」不变量
wiki create --area "../../outside" 路径穿越 store.py:write_wiki_page() — 拒绝不符合 [a-z][a-z0-9-]* 的 area,覆盖 wiki createdrafts promotedrafts reject 三个入口
extractors/web.py 绕过 SSRF 防护直接 requests.get(allow_redirects=True) 新增 _safe_fetch() — 调用 network.pyassert_public_network_target 检查初始 URL 及每一跳重定向目标,拦截 127.0.0.110.0.0.0/8172.16.0.0/12192.168.0.0/16169.254.0.0/16

🟡 构建修复

cli/setup.pysetup() 前加 _sync_from_checkout() 调用。此前 vendoring 只在 build_py 钩子中执行,但 setuptools 在配置阶段就验证 oks_connector 包目录存在,导致 pipx install ./cli --force 从干净工作树直接失败。

🟡 资产清理

  • 删除 settings/handlers.json — 注册已移入代码(cli._CAPABILITIES 字典),无运行时代码引用
  • 删除 _meta/frontmatter-schema.md — 被 raw-evidence-schema.md 替代,无运行时代码引用
  • 删除 capabilities/ 目录 — 能力定义在 schemas/capability_check.py 使用硬编码模块映射
  • 删除死代码(raw_ingest.pymultimodal_feedback.pykeyframe_probe.py
  • 删除未使用依赖(python-frontmattergitpython

🟡 README 更新

Core CLI 区块增加 v0.3.0 新增命令(oks evaloks traceoks capabilityrecall --goal --format --explain),区分普通用户安装与开发者源码安装。

🟡 文档死链修复

修复 docs/ 下 11 处死链接(core-learning-loop-poc.mdmemories.mdraw-materials.mdphase-history-summary.md 等),重写 AGENTS.md 与当前 CLAUDE.md 对齐。

合并注意事项

树冲突

三个文件已被删除,上游修改了它们。需要按本分支决定处理:

文件 本分支 建议
settings/handlers.json 已删除 保持删除 — 无运行时代码引用,注册在 cli._CAPABILITIES
_meta/frontmatter-schema.md 已删除 保持删除 — 被 raw-evidence-schema.md 替代
capabilities/README.md 已删除 保持删除 — 整个目录的能力定义已移至 schemas/ 和代码中

test_optional_extensions.py 中原来的 test_handler_install_hints_point_at_real_channels 测试需更新——它读取了已删除的 handlers.json。本分支已用 test_capability_registry_entries_are_valid 替换,验证对象改为 cli._CAPABILITIES 字典。

内容冲突

本分支的 _lark_cli.py 与上游 100% 一致,零冲突。feishu_base_worker.py 的改动已通过上游模块化重构自然消解。

不建议 revert 的提交

d74ee65(merge commit)之后的 3 个提交(5cc358b3d2f2f238ece4f)各自独立,每个都修复了真实问题。建议整体合入,不要挑拣。

验证

345 tests passed (89 cli + 256 scripts)
pipx install ./cli --force → oks 0.3.0 ✅
wheel + sdist build ✅
Source→Raw→Agent Draft→Promote→Wiki→Search(1.01)→Recall(double-path) ✅
rejected Draft promote blocked ✅
area="../" path traversal blocked ✅
SSRF: _safe_fetch blocks 127.0.0.1, 10.0.0.1, 172.16.0.1, 192.168.1.1, 169.254.169.254 ✅
all docs/ cross-references resolve ✅
git status --short clean ✅

提交列表

38ece4f docs: fix 11 dead cross-references and rewrite stale AGENTS.md
3d2f2f2 fix: security hardening — rejected draft gate, area path traversal, web SSRF
5cc358b fix: build vendoring timing, Rich date rendering, test API adaptation
d74ee65 merge: integrate upstream v0.3.0 (50 commits)
fca0799 docs: clarify install methods and document capability scope
ceaabf9 docs: remove stale book-poc-report reference from architecture evidence table
c0982c7 fix: resolve import isolation and formula capability consistency
c6bc093 feat: add accept skill for evidence-first isolated acceptance testing
034cd80 fix: verify capability env-python can import required module
e6faf36 docs: translate key reports to Chinese, update Kimi K3 case, expand anti-bot research
f3384eb docs: remove stale acceptance logs, handoff notes, and superseded designs
547926b feat: deduplicate lark-cli path resolver into shared _lark_cli.py
14a0dbb fix: harden atomic writes to full A5 compliance (mkstemp+fsync+os.replace+dir fsync)
050e1ba docs: fix schemas location and add runtime-notice README
3015768 docs: remove stale capabilities/ and handlers.json, fix all references
9436967 docs: remove wrong-project _meta files, fix references
f1e4652 chore: remove unused dependencies (python-frontmatter, gitpython)
7875f65 chore: remove dead tools (raw_ingest, multimodal_feedback, keyframe_probe)
f697b72 docs: record openclaw validation gap
118f632 docs: align architecture and agent prompt links
5279744 docs: record remote acceptance cleanup
ac5c695 docs: add platform research and kimi case
7a3b8fb docs: record clean server deployment acceptance
1e7cfaf fix: write ingest raw bundles under active kb
87315a3 docs: align word goal landing status
125baf2 docs: complete book poc core loop
feb84f9 test: freeze book poc ab evaluation
e19bdbf docs: refine book candidate review evidence
ee01a2f docs: audit book poc and execution gates
f6784a7 fix: render dated drafts in review list
49a1112 docs: record book poc awaiting review
0f0c855 fix: decode utf8 plain text intake

1263-ux added 30 commits July 28, 2026 23:06
…robe)

Delete three unreferenced tools and their tests:

- scripts/raw_ingest.py — superseded second ingest pipeline; oks ingest
  calls raw_bundle_adapter.run_ingest directly since monorepo merge.
  Unique capabilities (doctor probing, MinerU leaf discovery,
  formula-candidate generation, proxy bypass) are unused by any
  runtime path and can be ported to raw_bundle_adapter if needed.

- scripts/multimodal_feedback.py — zero callers; dev-loop tracker
  never wired into skills, CLI, or adapter.

- scripts/experiments/keyframe_probe.py — broken import since
  _adaptive_scene_detector moved to extractors/watch.py; capability
  already integrated into run_watch (watch.py:872).

Also removes multimodal_feedback from cli/pyproject.toml py-modules
to prevent build failure from missing module.

Tests: 81 passed, 1 pre-existing failure (knowledge_studio import
in test env, unrelated).
Neither package is imported anywhere:
- python-frontmatter → store.py parses YAML frontmatter via PyYAML
- gitpython → cli.py runs git init via subprocess.run(["git","init"])

Also removes stale cli/build/lib/ (gitignored build artifact,
regenerated by python -m build).
Delete _meta/learning-schema.json and _meta/frontmatter-schema.md.
Both described an external autpilot-web project schema (v0.7, dated
2026-06-04) that never matched this repo's actual wiki/profiles
frontmatter emitted by store.write_wiki_page().

The live frontmatter contract is documented in:
- docs/frontmatter-schema.md (docs-site page)
- .claude/rules/wiki-writing.md (checked-in enforcement contract)
- .claude/hooks/validate-wiki-write.sh (hardcoded validation)

_meta/ retains raw-evidence-schema.md (accurate, referenced by
recall_episodic).

Update references:
- CONSTITUTION.md: replace stale _meta entries with single
  raw-evidence-schema entry; drop false "CI-enforced" claim
- CLAUDE.md: align _meta description
- profiles/INDEX.md: repoint to docs/frontmatter-schema.md
Delete capabilities/ (7 JSON manifests + README) — nothing reads them.
Routing is hardcoded in route.py, capability_check.py, and the
_CAPABILITIES dict. The README's claim "Studio routing reads these
manifests" was false.

Delete settings/handlers.json — inert, superseded by route.py.
L1 handler names (oks-video etc.) don't match live capability names;
agent-reach/yt-dlp URL routing is obsolete post-monorepo-merge.

Fix all doc references (no hardcoded registries refactored):
- docs/architecture.md: remove capabilities/ from tree and text
- AGENTS.md: same
- CONSTITUTION.md: remove handlers.json references
- _meta/raw-evidence-schema.md: remove handlers.json reference
- docs/index.md: fix stale "independent oks-connector repo" claim
- docs/raw-multimodal-standard.md: remove capabilities/ reference
- .claude/skills/media-ingest/SKILL.md: same
- .agents/skills/media-ingest/SKILL.md: same
- CLAUDE.md: fix oks-connector/capabilities reference
Fix remaining stale "independent oks-connector repository" references
in _meta/raw-evidence-schema.md and docs/raw-multimodal-standard.md.
Connector was merged into this monorepo at commit 5f60ef2; schemas/
now lives here as the canonical reference.

Add schemas/README.md documenting that these JSON Schema files are
normative references, NOT auto-loaded or enforced at runtime (validator
uses hardcoded checks; no jsonschema dependency).
…lace+dir fsync)

_shared.py write_json/write_jsonl: replace bare write_text/open('w') with
mkstemp+flush+fsync+os.replace+dir fsync pattern, fixing ~20 extractor
call sites at once.

digest.py update_raw_index: delegate to _shared.write_json for aggregate
raw/index.json safety on crash.

cli.py _wire_userpromptsubmit: use store._atomic_write for
.claude/settings.json (config, explicitly named in A5).

config.py save_config: add missing flush+fsync before os.replace for
~/.oks/config.json.

validator.py _atomic_write_text: add missing dir fsync after os.replace.

feishu_base_worker.py: add _fsync_dir helper and dir fsync to
atomic_write_json/atomic_write_text (40+ call sites); replace bare
evidence_path.write_text with atomic_write_text.

All config, wiki, draft, and worker state writes now follow the
store.py:_atomic_write reference: mkstemp + flush + fsync + os.replace
+ dir fsync.
Extract lark-cli executable resolution from two inline copies
(feishu_base_worker.resolve_lark_cli and feishu_setup._resolve_lark_cli)
into a single scripts/_lark_cli.py module.

Resolution order: LARK_CLI_EXE env var → platform-specific fallbacks
(lark-cli.cmd/lark-cli.exe on Windows, lark-cli on POSIX) → npm
global install directory.
…igns

Delete 12 files that are historical/archived and no longer reflect
the current project state:

- Old acceptance logs (07-24 CLI test manual, 07-29 execution plan,
  word-goal landing status)
- Completed book POC reports (report, A/B protocol)
- Superseded designs (capability-architecture, core-learning-loop-poc,
  future-considerations)
- Handoff notes (PR explanation, transition doc)
- Stale SVG assets (architecture-overview, pipeline)
…nti-bot research

- Translate clean-server-deployment-report, feishu-e2e-status to Chinese
- Rewrite Kimi K3 deep analysis: evidence-first with locator traces
- Expand platform-antibot report: competitor analysis, dependency
  simplification, Feishu CI assessment, closed-loop positioning
- Update README, oks-core-architecture, agent-one-prompt-installation
capability_check.is_capability_available() previously checked only
that the configured interpreter path exists. Add python_can_import()
subprocess probe so env-var overrides (OKS_DOCUMENT_PYTHON etc.)
must actually load the extractor module before reporting "available".

Also:
- Add formula capability to _MODULES/_ENV_VARS maps
- Bump PLUGIN_VERSION to 0.2.4
- Add capability_verify_failed i18n string
- Add corresponding tests in test_optional_extensions and
  test_raw_bundle_adapter
New .claude/skills/accept/ skill for clean-install, document, PDF,
formula, video watch, and Feishu end-to-end validation. Preserves
honest reports and removes only per-run environments.

Includes settings.local.json (local Claude Code config) and
test_accept_skill.py.
- promote_candidate_document(): derive cli path from module location
  instead of monkeypatchable ROOT, fixing test import failure
- accept.py: replace non-existent --formula-secondary with capability
  install + import verification (formula is a PDF sub-capability)
- raw_bundle_adapter.py: register formula in run_check() and all
  extractor lookup dicts for consistent check/install surface
…ce table

The file docs/acceptance/book-poc-report.md was deleted in f3384eb.
Replace with the remaining clean-server-deployment-report.md.
- Distinguish end-user install (pipx install open-knowledge-studio) from
  developer source install (pipx install ./cli --force) in both docs
- Clarify document capability covers Office/HTML/CSV; .md/.txt work
  out of the box with the built-in connector
Upstream changes absorbed:
- oks eval recall: offline recall quality evaluation
- oks trace: append-only execution provenance (not recallable memory)
- CONSTITUTION P0 (Python >= 3.12), P3/P5/A2/A4 fixes
- Feishu worker modularization (feishu_worker/ subpackage, 12 modules)
- Packaging fixes (.codex/.agents in wheel, maintainer skills excluded)

Conflict resolutions:
- Tree conflicts (3): keep deletions of handlers.json,
  _meta/frontmatter-schema.md, capabilities/ — zero runtime
  code references in upstream
- Content conflicts (13): use upstream for feishu_worker refactor,
  cli.py (eval/trace), pyproject.toml, _shared.py, digest.py;
  keep our capability_check.py (import verification) and
  raw-multimodal-standard.md (schemas/ references)
- Tests: merge both sides' capability tests; fix Rich date
  rendering in drafts_list; update test for code-based registry

Verification:
- 345 tests pass (89 cli + 256 scripts)
- 3 high-risk checks: rejected draft isolation, path traversal,
  SSRF protection — all intact
- cli/setup.py: call _sync_from_checkout() before setup() so
  oks_connector/ exists during setuptools config validation
- cli/cli.py: handle date objects from YAML frontmatter in
  drafts_list (convert to str before Rich table rendering)
- cli/tests: adapt tests for upstream _CAPABILITIES structure
  (purpose/deps keys), remove _connector_error references,
  update capability install test expectations
1263-ux added 2 commits July 31, 2026 23:18
…eb SSRF

- store.py promote_draft: refuse promotion when status=rejected,
  matching the invariant that explicitly rejected candidates never
  enter Wiki (CONSTITUTION A3 human-gated Dreaming)
- store.py write_wiki_page: validate area against [a-z][a-z0-9-]*
  to prevent path traversal via wiki create --area, drafts promote,
  and drafts reject entry points
- extractors/web.py: add _safe_fetch() with assert_public_network_target
  on the initial URL AND every redirect hop, closing the gap where
  web_raw_probe bypassed network.py SSRF guards
- README: add eval/trace/recall params to Core CLI, separate normal
  vs dev install in Quick Start
Dead links fixed:
- docs/index.md: replace 2 POC links (neither file exists in this branch)
  with references to architecture/oks-core-architecture.md and
  engineering-rounds-2-3.md
- docs/raw-multimodal-standard.md: drop core-learning-loop-poc.md
  and phase-history-summary.md links (neither file exists)
- 7 link fixes: memories.md→wiki.md (3 sites),
  raw-materials.md→raw-multimodal-standard.md (4 sites)
- docs/architecture.md: _meta/ description narrowed from
  'frontmatter/learning' to 'raw evidence / trace / eval'

AGENTS.md fully rewritten: sync with CLAUDE.md — Claude Code
target, 6+1-factor recall, pipx install + oks init, current CLI
(eval/trace/capability/--goal), correct skill descriptions,
project safety rules
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant