merge: integrate upstream v0.3.0 with security hardening and doc cleanup - #6
Open
1263-ux wants to merge 32 commits into
Open
merge: integrate upstream v0.3.0 with security hardening and doc cleanup#61263-ux wants to merge 32 commits into
1263-ux wants to merge 32 commits into
Conversation
…robe) Delete three unreferenced tools and their tests: - scripts/raw_ingest.py — superseded second ingest pipeline; oks ingest calls raw_bundle_adapter.run_ingest directly since monorepo merge. Unique capabilities (doctor probing, MinerU leaf discovery, formula-candidate generation, proxy bypass) are unused by any runtime path and can be ported to raw_bundle_adapter if needed. - scripts/multimodal_feedback.py — zero callers; dev-loop tracker never wired into skills, CLI, or adapter. - scripts/experiments/keyframe_probe.py — broken import since _adaptive_scene_detector moved to extractors/watch.py; capability already integrated into run_watch (watch.py:872). Also removes multimodal_feedback from cli/pyproject.toml py-modules to prevent build failure from missing module. Tests: 81 passed, 1 pre-existing failure (knowledge_studio import in test env, unrelated).
Neither package is imported anywhere: - python-frontmatter → store.py parses YAML frontmatter via PyYAML - gitpython → cli.py runs git init via subprocess.run(["git","init"]) Also removes stale cli/build/lib/ (gitignored build artifact, regenerated by python -m build).
Delete _meta/learning-schema.json and _meta/frontmatter-schema.md. Both described an external autpilot-web project schema (v0.7, dated 2026-06-04) that never matched this repo's actual wiki/profiles frontmatter emitted by store.write_wiki_page(). The live frontmatter contract is documented in: - docs/frontmatter-schema.md (docs-site page) - .claude/rules/wiki-writing.md (checked-in enforcement contract) - .claude/hooks/validate-wiki-write.sh (hardcoded validation) _meta/ retains raw-evidence-schema.md (accurate, referenced by recall_episodic). Update references: - CONSTITUTION.md: replace stale _meta entries with single raw-evidence-schema entry; drop false "CI-enforced" claim - CLAUDE.md: align _meta description - profiles/INDEX.md: repoint to docs/frontmatter-schema.md
Delete capabilities/ (7 JSON manifests + README) — nothing reads them. Routing is hardcoded in route.py, capability_check.py, and the _CAPABILITIES dict. The README's claim "Studio routing reads these manifests" was false. Delete settings/handlers.json — inert, superseded by route.py. L1 handler names (oks-video etc.) don't match live capability names; agent-reach/yt-dlp URL routing is obsolete post-monorepo-merge. Fix all doc references (no hardcoded registries refactored): - docs/architecture.md: remove capabilities/ from tree and text - AGENTS.md: same - CONSTITUTION.md: remove handlers.json references - _meta/raw-evidence-schema.md: remove handlers.json reference - docs/index.md: fix stale "independent oks-connector repo" claim - docs/raw-multimodal-standard.md: remove capabilities/ reference - .claude/skills/media-ingest/SKILL.md: same - .agents/skills/media-ingest/SKILL.md: same - CLAUDE.md: fix oks-connector/capabilities reference
Fix remaining stale "independent oks-connector repository" references in _meta/raw-evidence-schema.md and docs/raw-multimodal-standard.md. Connector was merged into this monorepo at commit 5f60ef2; schemas/ now lives here as the canonical reference. Add schemas/README.md documenting that these JSON Schema files are normative references, NOT auto-loaded or enforced at runtime (validator uses hardcoded checks; no jsonschema dependency).
…lace+dir fsync)
_shared.py write_json/write_jsonl: replace bare write_text/open('w') with
mkstemp+flush+fsync+os.replace+dir fsync pattern, fixing ~20 extractor
call sites at once.
digest.py update_raw_index: delegate to _shared.write_json for aggregate
raw/index.json safety on crash.
cli.py _wire_userpromptsubmit: use store._atomic_write for
.claude/settings.json (config, explicitly named in A5).
config.py save_config: add missing flush+fsync before os.replace for
~/.oks/config.json.
validator.py _atomic_write_text: add missing dir fsync after os.replace.
feishu_base_worker.py: add _fsync_dir helper and dir fsync to
atomic_write_json/atomic_write_text (40+ call sites); replace bare
evidence_path.write_text with atomic_write_text.
All config, wiki, draft, and worker state writes now follow the
store.py:_atomic_write reference: mkstemp + flush + fsync + os.replace
+ dir fsync.
Extract lark-cli executable resolution from two inline copies (feishu_base_worker.resolve_lark_cli and feishu_setup._resolve_lark_cli) into a single scripts/_lark_cli.py module. Resolution order: LARK_CLI_EXE env var → platform-specific fallbacks (lark-cli.cmd/lark-cli.exe on Windows, lark-cli on POSIX) → npm global install directory.
…igns Delete 12 files that are historical/archived and no longer reflect the current project state: - Old acceptance logs (07-24 CLI test manual, 07-29 execution plan, word-goal landing status) - Completed book POC reports (report, A/B protocol) - Superseded designs (capability-architecture, core-learning-loop-poc, future-considerations) - Handoff notes (PR explanation, transition doc) - Stale SVG assets (architecture-overview, pipeline)
…nti-bot research - Translate clean-server-deployment-report, feishu-e2e-status to Chinese - Rewrite Kimi K3 deep analysis: evidence-first with locator traces - Expand platform-antibot report: competitor analysis, dependency simplification, Feishu CI assessment, closed-loop positioning - Update README, oks-core-architecture, agent-one-prompt-installation
capability_check.is_capability_available() previously checked only that the configured interpreter path exists. Add python_can_import() subprocess probe so env-var overrides (OKS_DOCUMENT_PYTHON etc.) must actually load the extractor module before reporting "available". Also: - Add formula capability to _MODULES/_ENV_VARS maps - Bump PLUGIN_VERSION to 0.2.4 - Add capability_verify_failed i18n string - Add corresponding tests in test_optional_extensions and test_raw_bundle_adapter
New .claude/skills/accept/ skill for clean-install, document, PDF, formula, video watch, and Feishu end-to-end validation. Preserves honest reports and removes only per-run environments. Includes settings.local.json (local Claude Code config) and test_accept_skill.py.
- promote_candidate_document(): derive cli path from module location instead of monkeypatchable ROOT, fixing test import failure - accept.py: replace non-existent --formula-secondary with capability install + import verification (formula is a PDF sub-capability) - raw_bundle_adapter.py: register formula in run_check() and all extractor lookup dicts for consistent check/install surface
…ce table The file docs/acceptance/book-poc-report.md was deleted in f3384eb. Replace with the remaining clean-server-deployment-report.md.
- Distinguish end-user install (pipx install open-knowledge-studio) from developer source install (pipx install ./cli --force) in both docs - Clarify document capability covers Office/HTML/CSV; .md/.txt work out of the box with the built-in connector
Upstream changes absorbed: - oks eval recall: offline recall quality evaluation - oks trace: append-only execution provenance (not recallable memory) - CONSTITUTION P0 (Python >= 3.12), P3/P5/A2/A4 fixes - Feishu worker modularization (feishu_worker/ subpackage, 12 modules) - Packaging fixes (.codex/.agents in wheel, maintainer skills excluded) Conflict resolutions: - Tree conflicts (3): keep deletions of handlers.json, _meta/frontmatter-schema.md, capabilities/ — zero runtime code references in upstream - Content conflicts (13): use upstream for feishu_worker refactor, cli.py (eval/trace), pyproject.toml, _shared.py, digest.py; keep our capability_check.py (import verification) and raw-multimodal-standard.md (schemas/ references) - Tests: merge both sides' capability tests; fix Rich date rendering in drafts_list; update test for code-based registry Verification: - 345 tests pass (89 cli + 256 scripts) - 3 high-risk checks: rejected draft isolation, path traversal, SSRF protection — all intact
- cli/setup.py: call _sync_from_checkout() before setup() so oks_connector/ exists during setuptools config validation - cli/cli.py: handle date objects from YAML frontmatter in drafts_list (convert to str before Rich table rendering) - cli/tests: adapt tests for upstream _CAPABILITIES structure (purpose/deps keys), remove _connector_error references, update capability install test expectations
…eb SSRF - store.py promote_draft: refuse promotion when status=rejected, matching the invariant that explicitly rejected candidates never enter Wiki (CONSTITUTION A3 human-gated Dreaming) - store.py write_wiki_page: validate area against [a-z][a-z0-9-]* to prevent path traversal via wiki create --area, drafts promote, and drafts reject entry points - extractors/web.py: add _safe_fetch() with assert_public_network_target on the initial URL AND every redirect hop, closing the gap where web_raw_probe bypassed network.py SSRF guards - README: add eval/trace/recall params to Core CLI, separate normal vs dev install in Quick Start
Dead links fixed: - docs/index.md: replace 2 POC links (neither file exists in this branch) with references to architecture/oks-core-architecture.md and engineering-rounds-2-3.md - docs/raw-multimodal-standard.md: drop core-learning-loop-poc.md and phase-history-summary.md links (neither file exists) - 7 link fixes: memories.md→wiki.md (3 sites), raw-materials.md→raw-multimodal-standard.md (4 sites) - docs/architecture.md: _meta/ description narrowed from 'frontmatter/learning' to 'raw evidence / trace / eval' AGENTS.md fully rewritten: sync with CLAUDE.md — Claude Code target, 6+1-factor recall, pipx install + oks init, current CLI (eval/trace/capability/--goal), correct skill descriptions, project safety rules
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
概述
从
open-agent-power/main(v0.3.0,50 个提交)合并到本分支,叠加 28 个本分支提交。总共 32 commits,66 files, +2,033 / −2,394。前 22 个提交是合并前的本分支工作(Book POC 闭环验收、过期资产清理、原子写入加固);后 4 个是合并后的收尾(构建修复、安全加固、文档死链修复)。
变更分类
🟢 吸收上游 v0.3.0(合并自动合入)
oks eval recall/oks eval compare— 离线召回质量评测oks trace start/append/judge/feedback/propose/finish/validate— 执行轨迹(provenance,不参与召回)scripts/feishu_worker/子包(12 模块).codex/.agents入 wheel,维护者 skill 排包🔴 安全加固(3 个独立修复)
这些是本分支发现并修复的,非上游代码的原有问题:
status: rejected仍可晋升为 Wikistore.py:promote_draft()— 晋升前检查meta.get("status") == "rejected",匹配 CONSTITUTION A3「人工门禁」不变量wiki create --area "../../outside"路径穿越store.py:write_wiki_page()— 拒绝不符合[a-z][a-z0-9-]*的 area,覆盖wiki create、drafts promote、drafts reject三个入口extractors/web.py绕过 SSRF 防护直接requests.get(allow_redirects=True)_safe_fetch()— 调用network.py的assert_public_network_target检查初始 URL 及每一跳重定向目标,拦截127.0.0.1、10.0.0.0/8、172.16.0.0/12、192.168.0.0/16、169.254.0.0/16🟡 构建修复
cli/setup.py—setup()前加_sync_from_checkout()调用。此前 vendoring 只在build_py钩子中执行,但 setuptools 在配置阶段就验证oks_connector包目录存在,导致pipx install ./cli --force从干净工作树直接失败。🟡 资产清理
settings/handlers.json— 注册已移入代码(cli._CAPABILITIES字典),无运行时代码引用_meta/frontmatter-schema.md— 被raw-evidence-schema.md替代,无运行时代码引用capabilities/目录 — 能力定义在schemas/,capability_check.py使用硬编码模块映射raw_ingest.py、multimodal_feedback.py、keyframe_probe.py)python-frontmatter、gitpython)🟡 README 更新
Core CLI 区块增加 v0.3.0 新增命令(
oks eval、oks trace、oks capability、recall --goal --format --explain),区分普通用户安装与开发者源码安装。🟡 文档死链修复
修复
docs/下 11 处死链接(core-learning-loop-poc.md、memories.md、raw-materials.md、phase-history-summary.md等),重写AGENTS.md与当前CLAUDE.md对齐。合并注意事项
树冲突
三个文件已被删除,上游修改了它们。需要按本分支决定处理:
settings/handlers.jsoncli._CAPABILITIES中_meta/frontmatter-schema.mdraw-evidence-schema.md替代capabilities/README.mdschemas/和代码中test_optional_extensions.py中原来的test_handler_install_hints_point_at_real_channels测试需更新——它读取了已删除的handlers.json。本分支已用test_capability_registry_entries_are_valid替换,验证对象改为cli._CAPABILITIES字典。内容冲突
本分支的
_lark_cli.py与上游 100% 一致,零冲突。feishu_base_worker.py的改动已通过上游模块化重构自然消解。不建议 revert 的提交
d74ee65(merge commit)之后的 3 个提交(5cc358b、3d2f2f2、38ece4f)各自独立,每个都修复了真实问题。建议整体合入,不要挑拣。验证
提交列表