Skip to content

ci(docs): enforce governed knowledge records - #369

Open
testikun wants to merge 1 commit into
openpi-dev:mainfrom
testikun:codex/issue-198-knowledge-contract
Open

ci(docs): enforce governed knowledge records#369
testikun wants to merge 1 commit into
openpi-dev:mainfrom
testikun:codex/issue-198-knowledge-contract

Conversation

@testikun

@testikun testikun commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Problem

Issue #198 established the forward-only knowledge and evidence contract, but new Research and Benchmark records still have no repository check for required metadata, category-index reachability, or local link integrity. A complete-looking Benchmark can therefore omit frozen identities, accounting, failure classification, limitations, or a retrievable evidence reference without failing CI.

Value

  • Makes future governed Research and Benchmark records reviewable from a clean checkout.
  • Keeps legacy documents untouched until an explicit migration.
  • Fails visibly on structural evidence gaps without claiming that a well-formed record is independently validated.

Approach

  • Add a dependency-free check-knowledge-contract.mjs for governed records under docs/research and docs/benchmarks.
  • Treat frontmatter as the explicit opt-in boundary; records without it remain legacy.
  • Require shared lifecycle/provenance metadata, Research fact/inference/recommendation/unknown sections, and Benchmark source/model/task/verifier/accounting/failure/limitations/evidence/rerun fields.
  • Verify every governed record is linked from its category index and that repository-relative Markdown links resolve inside the checkout.
  • Add Research and Benchmark templates, wire the check into bun run check, and append the Decision-0001-backed promise to the discipline ledger.
  • Keep this complementary to ci: validate decision document metadata #309, whose open PR validates Decision-only metadata.

Validation

  • bun run check — passed.
  • bun run test — passed: 1246 Node tests passed, 1 platform test skipped; 30/30 Vitest tests passed.
  • Focused knowledge-contract tests — 5/5 passed, covering valid records, legacy exclusion, missing Benchmark evidence fields, unindexed records, and broken links.
  • git diff --check — passed.

Impact

  • User-visible behavior: none.
  • Model-visible context/tools: none.
  • Runtime/lifecycle: none; CI/documentation validation only.
  • Persisted config/data: none.
  • Compatibility/risk: low. Existing records without frontmatter are intentionally ignored, and no ignored/local evidence is scanned or published.

Refs #198.

@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Sep 4, 2026

@tt-a1i tt-a1i left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

方向对,覆盖面不够

docs/README.md 治理的是 research、design、architecture、Decision、benchmarks。检查器只扫了 docs/researchdocs/benchmarks#386/#382 这种 validated 的 architecture/design 记录现在不会被拦。

请扩到 design / architecture / Decision,或明确写「其余类别暂不强制」并改 README。#309 是 Decision-only 且 Windows 路径 fail-open,请吸收进来,不要并排合。

frontmatter 选择加入、「结构 ≠ 验证」是对的。

@tt-a1i tt-a1i left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exact-head review: 81320b2

[P1] The new CI contract is opt-out, so it does not enforce the forward-only policy for new records. scripts/check-knowledge-contract.mjs:136-142 classifies every frontmatter-less research/benchmark Markdown file as legacy and skips it. A contributor can therefore add a brand-new record without frontmatter and bypass required metadata, index reachability, links, frozen identities, accounting, failures, limitations, and evidence checks while bun run check remains green.

Decision 0001 says documents from that Decision forward are governed; legacy is a fixed historical set, not a property a new file may claim by omitting metadata. Please encode an immutable legacy allowlist or baseline and fail closed for every new unlisted record. Add a regression proving that a newly added frontmatter-less record fails, while the enumerated pre-Decision records remain accepted.

Verification boundary: exact-head checker/policy review. Existing tests intentionally assert the bypass as legacy exclusion, so their green result confirms rather than mitigates the issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants