diff --git a/docs/evidence/CATALOG_DOC_ANTI_PATTERNS.md b/docs/evidence/CATALOG_DOC_ANTI_PATTERNS.md new file mode 100644 index 00000000..2fd12f51 --- /dev/null +++ b/docs/evidence/CATALOG_DOC_ANTI_PATTERNS.md @@ -0,0 +1,12 @@ +# Documentation Anti-Patterns Catalog + +Common anti-patterns to avoid. + +| Anti-Pattern | Problem | Solution | +|-------------|---------|----------| +| Vanishing source | A doc describes a feature that no longer exists | Keep docs in sync with code | +| Copy-paste drift | Same info in multiple places, gets out of sync | Single source of truth, cross-reference | +| Zombie doc | Deprecated doc still linked from navigation | Archive and update links | +| Claim without caveat | States a result without its limitations | Always pair claims with caveats | +| Doc as dumpster | One doc accumulates everything | Split into focused documents | +| No owner | No one is responsible for keeping it current | Assign an owner | diff --git a/docs/evidence/CHECKLIST_DOC_QUALITY.md b/docs/evidence/CHECKLIST_DOC_QUALITY.md new file mode 100644 index 00000000..176e24be --- /dev/null +++ b/docs/evidence/CHECKLIST_DOC_QUALITY.md @@ -0,0 +1,19 @@ +# Documentation Quality Checklist + +Use this to assess documentation quality. + +## Structure +- [ ] Document has a clear purpose stated at the top +- [ ] Headings follow a logical hierarchy +- [ ] Related documents are linked + +## Content +- [ ] No overclaiming language (run `check_claims.py`) +- [ ] All links resolve (run `check_doc_links.py`) +- [ ] Examples work from a clean checkout +- [ ] Limitations are stated where relevant + +## Maintenance +- [ ] Document is no more than 6 months old +- [ ] Document is indexed in PROJECT_INDEX.md +- [ ] Document has an identified owner diff --git a/docs/evidence/CHECKLIST_DOC_REVIEW_BY_TYPE.md b/docs/evidence/CHECKLIST_DOC_REVIEW_BY_TYPE.md new file mode 100644 index 00000000..dbc68c89 --- /dev/null +++ b/docs/evidence/CHECKLIST_DOC_REVIEW_BY_TYPE.md @@ -0,0 +1,19 @@ +# Doc Review Checklist by Change Type + +## New Document +- [ ] Has a clear purpose statement +- [ ] Is linked from PROJECT_INDEX.md +- [ ] Passes `check_claims.py` +- [ ] Passes `check_doc_links.py` +- [ ] Has the correct metadata header + +## Document Update +- [ ] Changes are accurate +- [ ] Links still resolve +- [ ] No overclaiming language introduced +- [ ] Metadata updated (Last updated date) + +## Document Deprecation +- [ ] Deprecation notice added at top +- [ ] Cross-references updated +- [ ] Archive path documented in archive index diff --git a/docs/evidence/GUIDE_RESOLVING_DOC_CONFLICTS.md b/docs/evidence/GUIDE_RESOLVING_DOC_CONFLICTS.md new file mode 100644 index 00000000..8c0b88d0 --- /dev/null +++ b/docs/evidence/GUIDE_RESOLVING_DOC_CONFLICTS.md @@ -0,0 +1,18 @@ +# Guide: Resolving Doc Conflicts + +When two documents give contradictory guidance. + +## Steps +1. Identify the authoritative source for the topic. +2. If neither is authoritative, use the more recently updated document. +3. Update the less authoritative doc to match. +4. Add cross-references in both docs noting the resolution. +5. If the conflict affects policy, log it in the decision log. + +## Sources of Truth +| Topic | Source | +|-------|--------| +| Metrics | METRICS_CURRENT.md | +| Evidence | EVIDENCE_CERTIFICATE.md | +| Safety | SAFETY.md | +| Agent rules | AGENTS.md | diff --git a/docs/evidence/POLICY_DOCS_FRESHNESS.md b/docs/evidence/POLICY_DOCS_FRESHNESS.md new file mode 100644 index 00000000..e831eb32 --- /dev/null +++ b/docs/evidence/POLICY_DOCS_FRESHNESS.md @@ -0,0 +1,17 @@ +# Docs Freshness Policy + +Documents must be reviewed and updated on a regular cadence. + +| Document Type | Maximum Age | Review Trigger | +|---------------|:-----------:|----------------| +| Policy docs | 6 months | Calendar reminder | +| Getting-started guides | 3 months | After CLI changes | +| Evidence docs | 6 months | After benchmark changes | +| Concept cards | 12 months | After policy changes | +| Runbooks | 3 months | After workflow changes | +| Audit checklists | 6 months | Before release | + +## Process +1. Reviewer checks accuracy, completeness, links. +2. Outdated docs are updated or flagged for retirement. +3. Review results recorded in decision log. diff --git a/docs/evidence/RUNBOOK_BENCHMARK_REGRESSION_TRIAGE.md b/docs/evidence/RUNBOOK_BENCHMARK_REGRESSION_TRIAGE.md new file mode 100644 index 00000000..700c30d9 --- /dev/null +++ b/docs/evidence/RUNBOOK_BENCHMARK_REGRESSION_TRIAGE.md @@ -0,0 +1,11 @@ +# Maintainer Runbook: Benchmark Regression Triage + +When a benchmark regression is reported: + +1. Confirm the regression by re-running the benchmark. +2. Check recent commits for changes that could affect results. +3. Use `git bisect` to find the exact commit if needed. +4. Determine if the regression is acceptable (documented tradeoff) or a bug. +5. If a bug, open an issue and assign priority. +6. If a tradeoff, update METRICS_CURRENT.md and document the reason. +7. Update BENCHMARKING.md if the target needs adjustment. diff --git a/docs/evidence/RUNBOOK_DEPENDENCY_CHANGE_REVIEW.md b/docs/evidence/RUNBOOK_DEPENDENCY_CHANGE_REVIEW.md new file mode 100644 index 00000000..f4af7819 --- /dev/null +++ b/docs/evidence/RUNBOOK_DEPENDENCY_CHANGE_REVIEW.md @@ -0,0 +1,10 @@ +# Maintainer Runbook: Dependency Change Review + +When a dependency changes: + +1. Verify the new dependency is necessary (not scope creep). +2. Check the license is compatible with Apache 2.0. +3. Check for known vulnerabilities. +4. Verify the change works on all supported platforms. +5. Update pyproject.toml and lock file if applicable. +6. Update any docs that reference the old dependency. diff --git a/docs/evidence/RUNBOOK_EXTERNAL_REVIEW_DELAYS.md b/docs/evidence/RUNBOOK_EXTERNAL_REVIEW_DELAYS.md new file mode 100644 index 00000000..74c7b5da --- /dev/null +++ b/docs/evidence/RUNBOOK_EXTERNAL_REVIEW_DELAYS.md @@ -0,0 +1,9 @@ +# Maintainer Runbook: External Review Delays + +When an external review is delayed: + +1. Check the review request for completeness (all required fields present). +2. Follow up with the reviewer — allow 1 week between follow-ups. +3. If the reviewer is unavailable, identify an alternative reviewer. +4. If no alternative is available, escalate to the project lead. +5. Document the delay and resolution in the decision log. diff --git a/docs/evidence/RUNBOOK_SCHEMA_CHANGE_REVIEW.md b/docs/evidence/RUNBOOK_SCHEMA_CHANGE_REVIEW.md new file mode 100644 index 00000000..75ffcb6c --- /dev/null +++ b/docs/evidence/RUNBOOK_SCHEMA_CHANGE_REVIEW.md @@ -0,0 +1,11 @@ +# Maintainer Runbook: Schema Change Review + +When a schema changes: + +1. Verify the new schema is valid JSON Schema. +2. Run validation tests against existing data. +3. Check if the change is backward-compatible (new optional fields are OK). +4. If breaking, update all consumers of the schema. +5. Update SCHEMA_REGISTRY.md. +6. Update any docs that reference the old schema. +7. Verify the reproducibility report still works. diff --git a/docs/evidence/STANDARD_DOC_METADATA.md b/docs/evidence/STANDARD_DOC_METADATA.md new file mode 100644 index 00000000..18f79fa5 --- /dev/null +++ b/docs/evidence/STANDARD_DOC_METADATA.md @@ -0,0 +1,19 @@ +# Documentation Metadata Format + +Each document should include metadata at the top. + +## Format +```markdown +> **Purpose:** One-sentence description of the document's purpose. +> **Status:** Draft | Internal | Review-only | Public-safe | Archived +> **Last updated:** YYYY-MM-DD +> **Owner:** Role or GitHub username +``` + +## Example +```markdown +> **Purpose:** Defines how documentation is governed in the project. +> **Status:** Internal +> **Last updated:** 2026-07-08 +> **Owner:** Maintainer +``` diff --git a/tests/test_runbooks_batch.py b/tests/test_runbooks_batch.py new file mode 100644 index 00000000..6b39f366 --- /dev/null +++ b/tests/test_runbooks_batch.py @@ -0,0 +1,20 @@ +"""Tests for batch of 10 docs.""" +from pathlib import Path + +FILES = [ + "RUNBOOK_BENCHMARK_REGRESSION_TRIAGE.md", + "RUNBOOK_SCHEMA_CHANGE_REVIEW.md", + "RUNBOOK_DEPENDENCY_CHANGE_REVIEW.md", + "RUNBOOK_EXTERNAL_REVIEW_DELAYS.md", + "CHECKLIST_DOC_QUALITY.md", + "POLICY_DOCS_FRESHNESS.md", + "GUIDE_RESOLVING_DOC_CONFLICTS.md", + "STANDARD_DOC_METADATA.md", + "CHECKLIST_DOC_REVIEW_BY_TYPE.md", + "CATALOG_DOC_ANTI_PATTERNS.md", +] + + +def test_all_exist(): + for f in FILES: + assert Path("docs/evidence", f).exists(), f"Missing: {f}"