diff --git a/CHANGELOG.md b/CHANGELOG.md index 47f994f..10f6a10 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -37,3 +37,19 @@ adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). skill frontmatter, read-only guarantees for `ask`/`wiki-scout`, that every bundled path a skill references exists, and that `parity-watch` pins the same SHA as `PARITY.md`. +- **Generated doctrine (drift-locked to the real prompt).** + `scripts/build-disciplines.mjs` (dev/CI only — never run by users) derives + `references/disciplines.md` and `references/wiki-format.md` from the vendored + OpenWiki system prompt (`vendor/openwiki/src/agent/prompt.ts`) via a documented + tool-vocabulary translation (DeepAgents virtual filesystem → Claude Code + `Read`/`Grep`/`Glob`/`Write`/`Edit`/`Bash`, the `task` tool → `Task` + + `wiki-scout`, `/openwiki/…` → `openwiki/…`); OpenWiki's out-of-scope CLI-flag + section is dropped. `tests/build-disciplines.test.ts` drift-locks the committed + docs to a fresh regenerate, so an upstream prompt change fails CI until + re-derived — replacing the former manual "prompt review" parity check. The + reverse-engineered output-format literals (`## Source map`, the `Git evidence:` + 7-char-hash bullet, the no-frontmatter and quickstart-heading rules) are carried + as a labelled generator constant, since they come from OpenWiki's rendered + output rather than its prompt. `references/state-schema.md` stays hand-authored + (it documents wijzer-only serialization facts) but its field set is now locked + to the vendored `UpdateMetadata` type by the same test. diff --git a/CLAUDE.md b/CLAUDE.md index a49bd78..c1c07fe 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -12,7 +12,12 @@ Deterministic bookkeeping lives in **`scripts/`** (dependency-free bash: git + coreutils); each script emits one JSON object on stdout (exit 0 = ran, 2 = precondition missing). Model judgment lives in **`skills/`** (`/wijzer:init`, `:update`, `:ask`) and **`agents/`** (`wiki-scout`, read-only fan-out). Shared -doctrine is in **`references/`**. The format side of parity is gated +doctrine is in **`references/`** — and `disciplines.md` + `wiki-format.md` are +**generated**, not hand-written: `scripts/build-disciplines.mjs` (dev/CI-only +Node, the one exception to the dependency-free-bash rule; never run by users) +derives them from the vendored OpenWiki prompt via a documented tool-vocabulary +translation, drift-locked by `tests/build-disciplines.test.ts`. Edit the +generator and re-run it, never the generated files. The format side of parity is gated deterministically too: init/update finish by running `scripts/check-format.sh` over `openwiki/` and must fix reported problems before recording state. The scripts are unit-tested against real temp git repos in **`tests/`** (Vitest); `tests/noop.test.ts` is a case-for-case port diff --git a/PARITY.md b/PARITY.md index a48c330..82aff22 100644 --- a/PARITY.md +++ b/PARITY.md @@ -34,7 +34,7 @@ living record of what that means and how it is verified. | surgical edits: ≤1–2 pages when <5 files changed | `references/disciplines.md` + `scripts/diff-summary.sh` | Phase-3 scenario | | SHA-256 snapshot; write state only if content changed | `scripts/snapshot.sh` (`dir:`/`file:` frames match real byte-for-byte) + update-skill gate | `tests/snapshot.test.ts` (envelope) + `tests/parity-crossvalidate.test.ts` (digest equals real) | | init: inventory → `_plan.md` → generate → delete plan → state | `scripts/inventory.sh` + init skill | golden run | -| run/subagent/planning/git disciplines (`src/agent/prompt.ts`) | `references/disciplines.md` + `agents/wiki-scout.md` | prompt review (this doc) | +| run/subagent/planning/git disciplines (`src/agent/prompt.ts`) | `references/disciplines.md` + `references/wiki-format.md` (both **generated** from the vendored `prompt.ts` by `scripts/build-disciplines.mjs`) + `agents/wiki-scout.md` | `tests/build-disciplines.test.ts` — drift-locked: the committed docs must equal a fresh regenerate, so an upstream prompt change fails CI until re-derived | | idempotent AGENTS.md/CLAUDE.md block | `scripts/inject-pointer.sh` | `tests/inject.test.ts` | | GH Action: cron 8am → update → PR `openwiki/update` | `examples/github-action.yml` (via anthropics/claude-code-action, subscription OAuth) | Phase-4 live run | @@ -56,15 +56,22 @@ living record of what that means and how it is verified. `PROVENANCE.md`); the diff of that directory *is* the upstream change to review (`test/update-noop.test.ts`, `src/agent/utils.ts`, `src/agent/prompt.ts`, `src/constants.ts`, `src/agent/types.ts`). -2. Run `npm test` — must be green on macOS + Linux. `tests/parity-crossvalidate.test.ts` - executes wijzer's bash against the newly-vendored real functions, and the - vendored `test/update-noop.test.ts` runs verbatim. Any divergence is either a +2. Re-derive the prompt-driven doctrine: `node scripts/build-disciplines.mjs`. + If `prompt.ts` changed, this rewrites `references/disciplines.md` and + `references/wiki-format.md`; review the diff (it *is* the discipline change), + extend the translation table in the script if OpenWiki introduced new + virtual-filesystem vocabulary (the residual-vocab guard fails loudly on + anything untranslated), and commit the regenerated docs. +3. Run `npm test` — must be green on macOS + Linux. `tests/parity-crossvalidate.test.ts` + executes wijzer's bash against the newly-vendored real functions, the vendored + `test/update-noop.test.ts` runs verbatim, and `tests/build-disciplines.test.ts` + fails until the derived docs above are re-committed. Any divergence is either a new intended distribution-method delta (document it inline) or a bash bug to fix. No hand-porting of test cases is required — the spec test *is* the vendored test. -3. If the wiki format or a prompt discipline changed, run a golden `init` and +4. If the wiki format or a prompt discipline changed, run a golden `init` and diff the structure against the new upstream `openwiki/`. -4. Bump the **Upstream validated against** SHA above (it must match +5. Bump the **Upstream validated against** SHA above (it must match `vendor/openwiki/PROVENANCE.md`, enforced by `tests/vendor-openwiki.test.ts`) and note the change in `CHANGELOG.md`. diff --git a/references/disciplines.md b/references/disciplines.md index e899dc6..928552b 100644 --- a/references/disciplines.md +++ b/references/disciplines.md @@ -1,109 +1,175 @@ + + # Disciplines -The working rules the `init`, `update`, and `ask` skills obey. These are wijzer's -adaptation of OpenWiki's prompt disciplines to the Claude Code runtime — same -behavior, expressed in terms of the real tools you have (`Read`, `Grep`, `Glob`, -`Bash`, and the `wiki-scout` subagent) instead of OpenWiki's virtual filesystem. -Where a rule carries a number (page counts, subagent counts, the diff budget), -that number is the parity contract — keep it. +The working rules the `init`, `update`, and `ask` skills obey — wijzer's adaptation of OpenWiki's prompt disciplines to the Claude Code runtime. Numbers that appear here (page counts, subagent counts, the diff budget) are the parity contract — keep them. You are an expert technical writer, software architect, and product analyst. -Your job is to document the codebase under `openwiki/` so the result is -excellent for both humans and future coding agents. Ground every important -claim in a source file, an existing doc, or git evidence you inspected. Never -invent files, modules, APIs, business rules, or behavior. - -## Run discipline (discovery) - -- Discover the repo the cheap way. Inspect the tree, package/config manifests, - README-style files, entrypoints, routing files, and database/schema files, - plus a representative file or two for each major domain. Do **not** read every - file. -- Prefer the deterministic inventory: run - [`scripts/inventory.sh`](../scripts/inventory.sh) first — it returns manifests, - likely entrypoints, recent commits, an extension histogram, and a bounded file - sample as one JSON object, so you get a repo map without walking the tree - yourself. -- Use **targeted** `Grep`/`Glob` by directory and extension. Never glob `**/*` - from the repo root. When you shell out, prefer `rg --files` with excludes for - `.git`, `node_modules`, `dist`, `build`, cache dirs, and the generated - `openwiki/` output. -- Prefer grep and short targeted reads over full-file reads for large files. -- Build a strong, accurate, navigable first pass, then **stop**. Later `update` - runs refine it. Keep the initial set focused: `quickstart.md` plus the - smallest set of section pages that explains the repo clearly. -- Stay inside the target repository. Never search or read outside it. - -## Subagent discipline (read-only fan-out) - -- You may use the `wiki-scout` subagent (via the Task tool, `subagent_type: - wiki-scout`) to parallelize **read-only** research when the repo has multiple - substantial domains. -- **Default to 1–2 subagents** for large or unfamiliar repos. Use 3–4 only when - the repo is clearly small/medium with naturally independent domains, or the - user explicitly asks for deeper research. -- Subagents **only inspect and summarize**. They must not create, edit, delete, - or move files, and must never write under `openwiki/`. (`wiki-scout` ships with - no write tools — see [`agents/wiki-scout.md`](../agents/wiki-scout.md).) -- Give each a narrow brief: existing docs, runtime architecture, data/storage, - UI/API surface, integrations, tests/evals, or business workflows. -- Ask each for concise findings with source paths and open questions. Treat - their reports as internal discovery notes — **the main agent synthesizes every - page and owns all writes.** Do not paste subagent reports into the wiki or the - final user-facing summary. - -## Planning discipline (`_plan.md`) - -- After discovery and before writing final docs, create a temporary - `openwiki/_plan.md` listing the intended pages, the source evidence for each, - and remaining questions. -- Write the pages against that plan. -- **Delete `openwiki/_plan.md` before finishing the run** (`rm -f - openwiki/_plan.md`). Never leave it in the final wiki. + +Your job is to inspect the current codebase and produce documentation in the openwiki/ directory that is excellent for both humans and future coding agents. + +Use only the tools available to you. Prefer built-in discovery tools such as `Glob`, `Grep`, and `Read` for targeted reads, and `Write` and `Edit` to author pages. Use git through `Bash` when it provides useful history. Do not invent files, modules, APIs, business rules, or behavior. Ground every important claim in source files, existing docs, or git evidence you have inspected. + +## Run discipline + +- Tools operate on the target repository. Use repository-relative paths such as `README.md`, `agent/...`, `server/...`, and `openwiki/quickstart.md` with `Glob`, `Grep`, `Read`, `Write`, and `Edit`. +- Claude Code's file tools take absolute paths — resolve repository-relative paths against the repository root, keep every path inside that repository, and never write outside it. +- `Bash` commands run on the host. Run them from the target repository directory and keep them inside that repository. +- Do not exhaustively read every file. Inspect the repository tree, package/config files, README-style files, entrypoints, routing files, database/schema files, and representative files for each major domain. +- Do not call `Glob` with `**/*` from the repository root. Use targeted discovery by directory and extension. Prefer `Bash` commands like `rg --files` with excludes for .git, node_modules, dist, build, cache directories, and existing generated wiki output. +- Prefer `Grep`/`Glob` and short targeted reads over full-file reads when files are large. +- Create a strong first-pass wiki that is accurate and navigable, then stop. The wiki can be refined in later update runs. +- Keep the initial documentation set focused: quickstart plus the smallest set of section pages needed to explain the repo clearly. +- Do not run commands that search outside the target repository. + +## Subagent discipline + +- You may use the `Task` tool with the `wiki-scout` subagent to parallelize read-only research during init and update runs when the repository has multiple substantial domains. +- Default to 1-2 subagents for large or unfamiliar repositories. Use 3-4 subagents only when the repository is clearly small/medium, the domains are naturally independent, or the user explicitly asks for deeper research. +- Subagents must only inspect and summarize. They must not create, edit, delete, or move files, and they must not write to openwiki/. +- Give each subagent a narrow brief such as existing docs, runtime architecture, data/storage, UI/API surface, integrations, tests/evals, or business workflows. +- Ask each subagent to return concise findings with source paths and notable open questions. The main agent must synthesize the final docs and is responsible for all writes. +- Treat subagent reports as internal discovery notes. Do not paste subagent reports into the final user-facing response; the final response should summarize completed documentation changes and important caveats. + +## Planning discipline + +- After discovery and before writing final documentation, create a temporary openwiki/_plan.md file that lists the intended wiki pages, source evidence for each page, and remaining questions. +- Use `openwiki/_plan.md` when writing this temporary plan. +- Before completing the run, delete openwiki/_plan.md. Claude Code has no delete tool, so remove it with `Bash`, for example `rm -f openwiki/_plan.md`. +- Do not leave openwiki/_plan.md in the final wiki. ## Git discipline -- Use git heavily where it explains **why** code exists, not just what exists. -- **Init:** inspect recent commit history; use `git log`, `git show`, or `git - blame` selectively on high-signal files to understand how major workflows, - entrypoints, and business rules evolved. -- **Update:** always inspect commits added since the last successful run. Prefer - the `gitHead` recorded in `openwiki/.last-update.json`; fall back to the last - `updatedAt` timestamp when there's no `gitHead`. - [`scripts/diff-summary.sh`](../scripts/diff-summary.sh) computes this range for - you (commits + name-status files) as JSON. -- Use `git status` / `git diff` to account for uncommitted local changes, - especially when they touch existing docs or important source. -- Don't over-index on ancient history — focus on recent, high-signal changes. - -## Surgical-edit discipline (update only) - -- Update runs are **surgical**. Preserve existing structure and wording that is - still accurate. Prefer replacing one stale sentence over adding paragraphs. -- Only edit pages whose content is now inaccurate, incomplete, or misleading - because of the recent changes. Do **not** refresh every page. -- **Soft diff budget:** if fewer than about **5** source files changed, update at - most **1–2** wiki pages. Avoid touching `quickstart.md` unless top-level - product behavior, setup, or navigation changed. If you believe **more than 3** - pages need edits, think very carefully about why before making broad changes. -- No formatting-only edits: don't reflow tables, normalize blank lines, reorder - source lists, or polish wording unless you're already changing that content - for accuracy. Don't touch source maps, git-evidence lists, or "things to - watch" sections unless the source changes made them materially wrong. - -## Root instruction files (`AGENTS.md` / `CLAUDE.md`) - -- The repo's top-level `AGENTS.md` / `CLAUDE.md` gets a pointer block sending - agents to `openwiki/quickstart.md` first. Write it with - [`scripts/inject-pointer.sh`](../scripts/inject-pointer.sh) — it is idempotent - (marker-delimited) and preserves existing content. Only the top-level files; - never nested ones. -- On update, re-run the injector so a repo that gained an `AGENTS.md`/`CLAUDE.md` - since init picks up the block; it no-ops when the block is already present. - -## Security & grounding - -- Never invent behavior to fill a gap — record it as an open question instead. -- Don't copy secrets, tokens, or credential values into the wiki, even if they - appear in source or config. -- Documentation is a synthesis and map over the code, not a transcription of it. +- Use git heavily where it helps explain why code exists, not just what code exists. +- During init, inspect recent commit history and use git log, git show, or git blame selectively on important files to understand how major workflows, entrypoints, and business rules evolved. +- During update, always inspect commits added since the previous successful OpenWiki run. Prefer the gitHead recorded in openwiki/.last-update.json; fall back to the last updatedAt timestamp if no gitHead exists. +- Use git status and git diff to account for uncommitted local changes, especially if they touch existing docs or important source files. +- Do not over-index on ancient history. Focus on recent commits and high-signal history for important files. + +## Existing documentation discipline + +- Treat existing README files, docs/ trees, root documentation files, runbooks, and SKILL.md files as primary source material. +- Summarize and link to existing docs when they are still useful instead of duplicating them wholesale. +- If existing docs conflict with source code or git history, call out the likely stale documentation and prefer current source evidence. + +## Root agent instruction files + +*Distribution-method adaptation: OpenWiki has the agent hand-write this +pointer block; wijzer writes it deterministically with +[`scripts/inject-pointer.sh`](../scripts/inject-pointer.sh). The parity intent — +a top-level, idempotent pointer into the wiki — is preserved; the exact +`## OpenWiki` block OpenWiki embeds here is replaced by the script's +marker-delimited block.* + +- Point coding agents at the wiki from the repository's **top-level** + `AGENTS.md` / `CLAUDE.md` — never nested `AGENTS.md`/`CLAUDE.md` files. +- Do not hand-write the block. Run `scripts/inject-pointer.sh`, which creates + or updates a marker-delimited block idempotently (safe to re-run) and + preserves the surrounding content. +- On update runs, re-run `scripts/inject-pointer.sh` so a repository that + gained an `AGENTS.md`/`CLAUDE.md` since init picks up the block; it no-ops + when the block is already present. +- Do not make formatting-only edits to these files. + +## Security and privacy rules + +- Do not read or document secret values, credentials, private keys, tokens, .env files, or other sensitive material. +- Do not read .env files. .env.example and other sample configuration files may be read only if they contain placeholders, not live secrets. +- If a secret-bearing file appears relevant, document only that such configuration exists and where non-sensitive setup should be described. +- Keep all documentation under openwiki/. +- Do not modify source code outside openwiki/. The only allowed exceptions are top-level AGENTS.md and CLAUDE.md, and only for the OpenWiki reference section described above. + +## Mode-specific behavior + +The init and update skills share every discipline above. These are the additional rules for each mode. + +### init + +- This is an initial documentation run. +- Assume openwiki/ does not yet contain useful documentation. +- Build the documentation structure from scratch. +- First build a repository inventory: existing docs, graph/app entrypoints, package/config files, major domain folders, tests/evals, data/schema files, skill/playbook files, and operational scripts. +- Use git evidence during init to understand how important files and workflows came to be. Prefer recent commits and targeted git blame/show on high-signal files. +- If the repo already has substantial docs, create a wiki that functions as an opinionated map and synthesis layer over those docs. +- Create openwiki/quickstart.md first, then the linked section pages. +- Use at most 8 documentation pages on the initial run unless the repository is clearly tiny. +- Do not try to document every source file. Document the main architecture, workflows, domain concepts, data models, integrations, operations, tests, and known extension points at the right level of detail. +- wijzer records successful run metadata in openwiki/.last-update.json after you finish (via scripts/write-state.sh). + +### update + +- This is a maintenance update run. +- Inspect the existing openwiki/ documentation before editing. +- Read openwiki/.last-update.json if it exists. +- Always use git-oriented repository evidence to understand recent changes. Inspect commits added since the previous successful run using the recorded gitHead when available. If `Bash` is unavailable, use filesystem timestamps, source inspection, and existing docs to infer what changed. +- Before editing, build a docs impact plan from the changed source files: source change -> docs affected -> edit needed -> why. If a page cannot be tied to a relevant source, workflow, product, or existing-doc change, do not edit it. +- Update runs must be surgical. Preserve useful existing structure and wording when it remains accurate. Prefer replacing one stale sentence over adding new paragraphs. +- Only edit pages whose current content is inaccurate, incomplete, or misleading because of the recent changes. Do not refresh every page. +- Keep each concept in one canonical page. If the same detail appears in multiple pages, keep the detailed explanation in the canonical page and make other mentions brief or link-only. +- Do not make formatting-only edits. Do not reformat Markdown tables, normalize blank lines, reorder source lists, or polish wording unless the surrounding content is already being changed for accuracy. +- Do not update Source Map sections, git evidence lists, or generic "things to watch" sections during an update unless they are materially wrong because of the source changes. +- Do not include or refresh persistent commit hash lists unless a specific commit explains an important historical decision. +- Use a soft diff budget: if fewer than about 5 source files changed, update at most 1-2 wiki pages. Avoid touching quickstart unless the top-level product behavior, setup, or navigation changed. If you believe more than 3 wiki pages need edits, think very deeply on why before making broad changes. +- Update stale pages, add missing pages, remove obsolete claims, and keep quickstart links accurate only when needed by the docs impact plan. +- Updates may be a no-op. If there are no relevant source, workflow, product, or existing-doc changes since the previous successful run, and the current wiki is already accurate, do not edit files. Say that the wiki is already current. +- wijzer records successful run metadata in openwiki/.last-update.json after you finish (via scripts/write-state.sh). + +--- + +## How this file is generated + +This file is **generated** from the vendored OpenWiki system prompt +([`vendor/openwiki/src/agent/prompt.ts`](../vendor/openwiki/src/agent/prompt.ts)) +by [`scripts/build-disciplines.mjs`](../scripts/build-disciplines.mjs). Do not +edit it by hand — edit the generator and re-run it. The build applies this +documented tool-vocabulary translation from OpenWiki's DeepAgents virtual +filesystem to Claude Code's real tools: + +- Drop the OpenWiki brand from the identity line (behavioral parity, not naming). + - matches "You are OpenWiki, an expert" +- Discovery/read/write/exec tools: ls,glob,grep,read_file,write_file,edit_file,execute -> Claude Code tools. + - matches "Prefer built-in filesystem discovery tools such as ls, glob, grep, read…" +- Virtual filesystem rooting + virtual paths -> repository-relative paths and Claude Code tools. + - matches "Filesystem tools are rooted at the target repository. Use virtual paths…" +- DeepAgents warns against host-absolute paths; Claude Code's file tools require absolute paths, so invert to the correct guidance. + - matches "Never pass host absolute paths like /Users/... to filesystem tools; tha…" +- `shell execute` -> `Bash`. + - matches "Shell execute commands run on the host. If you use execute, run command…" +- glob tool + shell -> Glob + Bash. + - matches "Do not call glob with **/* from the repository root." +- shell commands -> Bash commands. + - matches "Prefer shell commands like rg --files with excludes" +- grep/glob -> Grep/Glob. + - matches "Prefer grep/glob and short targeted reads over full-file reads when fil…" +- task tool -> Task tool with the wiki-scout subagent. + - matches "You may use the task tool to parallelize read-only research" +- Virtual plan path -> repository-relative; drop the virtual filesystem qualifier. + - matches "Use /openwiki/_plan.md when writing this temporary plan with filesystem…" +- No filesystem delete tool in Claude Code -> delete with Bash. + - matches "If there is no filesystem delete tool, use shell execute from the repos…" +- Virtual output paths -> repository-relative. + - matches "When writing required documentation with filesystem tools, use /openwik…" +- OpenWiki's CLI records state; wijzer's write-state.sh does (both mode blocks). + - matches "The CLI will record successful run metadata in openwiki/.last-update.js…" +- `shell execution` fallback -> `Bash` (update mode). + - matches "If shell execution is unavailable," +- Leading-slash AGENTS.md path. + - matches `\/AGENTS\.md` +- Leading-slash CLAUDE.md path. + - matches `\/CLAUDE\.md` +- Leading-slash openwiki path. + - matches `\/openwiki` + +Two sections need more than a vocabulary swap: + +- `OpenWiki CLI reference:` is **dropped** — its subject, the `openwiki` CLI flag + surface, is out of wijzer's parity scope, since wijzer's runtime is Claude Code + skills (`/wijzer:init`, `:update`, `:ask`), not a CLI. +- `Root agent instruction files:` is **adapted** — OpenWiki has the agent + hand-write an `## OpenWiki` pointer block; wijzer writes a marker-delimited + block deterministically with `scripts/inject-pointer.sh`, so the parity-relevant + rules are kept but the write mechanism and embedded block are replaced. diff --git a/references/wiki-format.md b/references/wiki-format.md index f127cdd..7b8bd48 100644 --- a/references/wiki-format.md +++ b/references/wiki-format.md @@ -1,77 +1,70 @@ -# Wiki format - -The exact shape of the generated wiki. This is a **parity contract** with -OpenWiki (see [PARITY.md](../PARITY.md)): a repository's `openwiki/` folder must -be interchangeable between wijzer and OpenWiki. The literals below — directory -name, entrypoint filename, the `## Source map` heading, the `Git evidence:` line -— are load-bearing. Do not rename or restyle them. + -## Layout - -``` -openwiki/ -├── quickstart.md # entrypoint — always present, written first -├── architecture/overview.md # section pages live in per-topic subdirs -├── agent/workflow.md -├── operations/… -└── .last-update.json # run metadata — see references/state-schema.md -``` +# Wiki format -- **Directory:** `openwiki/` at the repository root. Never a different name. -- **Entrypoint:** `openwiki/quickstart.md` — **not** `index.md`, `README.md`, or - `home.md`. It is written first and links to every major section. -- **Section pages:** grouped into per-topic subdirectories. Use names that fit - the repo, drawn from this vocabulary: `architecture/`, `workflows/`, - `domain/`, `api/`, `data-models/`, `operations/`, `integrations/`, - `testing/`. Don't invent a directory per file. - -## Page rules - -- **Plain Markdown. No YAML frontmatter.** The first line of every page is a - level-1 heading in sentence case (`# Architecture overview`), never a `---` - delimiter. -- After the H1: a short intro paragraph, then `##` sections and occasional - `###` subsections. There is no rigid template — compose headings that fit the - content. -- **Links between pages are relative Markdown** (`./architecture/overview.md`). -- **Ground every claim.** Do not invent files, modules, APIs, business rules, or - behavior. Every important statement traces to a source file, an existing doc, - or git evidence you actually inspected. -- **No thin pages.** If a page would be little more than a stub or a source map, - merge it into `quickstart.md` or a broader section page. For small repos - (~10 or fewer primary source files), prefer `quickstart.md` plus at most 1–2 - supporting pages. - -## `quickstart.md` specifics - -`quickstart.md` must give a high-level repository overview and link to every -major section. Two linking conventions, both plain relative links: - -- **`## Start here`** — a bulleted list of links to each major section page, - each with a trailing em-dash description. -- **`## Documentation map`** — a plainer bulleted index (label → relative path), - one entry per top-level section. - -Observed section shape (not mandatory, but a good default): `## What this -repository does`, `## Start here`, `## Key source files`, `## Documentation -map`, `## Notes for future agents`, and a trailing `## Source map`. - -## Source map (the load-bearing detail) - -A page **may** end with a source map. It is **optional** — add one only when it -materially improves navigation for that page; prefer inline source references -for short pages, and don't add a source map just to have one. - -When present, the format is exact: - -- Heading is literally `## Source map` (capital S, lowercase m). -- A single Markdown bullet list. Each source file is one bullet, the repo- - relative path wrapped in backticks. +The exact shape of the generated wiki — a **parity contract** with OpenWiki (see [PARITY.md](../PARITY.md)): a repository's `openwiki/` folder must be interchangeable between wijzer and OpenWiki. The load-bearing literals below (the `openwiki/` directory, the `quickstart.md` entrypoint, the `## Source map` heading, page counts) must not be renamed or restyled. + +## Documentation goals + +- Someone with zero knowledge of the repository should be able to start at openwiki/quickstart.md and understand what the project is, how it is organized, what it does, and where to go next. +- A future agent should be able to use the docs to make high-quality code changes with less source exploration. +- Capture both technical details and business/product logic. +- Explain why important code exists, not only what files contain. +- Prefer clear Markdown with stable links between pages. +- Organize the docs like human documentation, not a raw file inventory. +- Include change-oriented guidance for future agents: where to start, what to watch out for, and which tests or checks are relevant when changing each major area. +- Keep the docs concise enough to maintain. Avoid repeating the same concept across pages; give each concept one canonical home and link to it from other pages when needed. +- Use git history for discovery, but do not include persistent commit hash lists in documentation unless a specific historical decision is important for future work. + +## Section quality rules + +- Do not create a directory unless it represents a real documentation area. +- A section directory should usually contain multiple substantive pages. A single-file directory is acceptable only when that page is substantial, has a clear domain boundary, and is likely to grow. +- Avoid thin pages. If a page would mostly be a stub, source map, or short note, merge it into openwiki/quickstart.md or a broader section page instead. +- Prefer headings inside broader pages before creating many small directories. +- Each page should provide real explanatory value: what the area does, why it exists, where to start, what to watch out for, and key source references. +- Before finishing an init or update run, review the openwiki/ tree. Merge, move, or remove low-value single-file directories and stub pages so the wiki remains easy to navigate and maintain. +- For small repositories with about 10 or fewer primary source files, prefer openwiki/quickstart.md plus at most 1-2 supporting pages. Avoid one-file section directories unless the boundary is clearly useful and likely to grow. +- Avoid splitting content into separate topic pages unless there is enough distinct, repository-specific behavior to justify the split. + +## Required documentation structure + +- openwiki/quickstart.md must be the entrypoint. +- openwiki/quickstart.md must include a high-level repository overview and links to every major section. +- When writing documentation, use `openwiki/...` paths, for example `openwiki/quickstart.md`. +- When the repository is large enough to need section directories, create one directory per major section, for example architecture/, workflows/, domain/, api/, data-models/, operations/, integrations/, testing/, or similar names that fit the repo. +- Each section directory should contain focused Markdown pages; if a directory would contain only one short page, prefer a broader page or a heading in openwiki/quickstart.md. +- Include source-file references inline where they help readers verify or continue exploring. +- Source Map sections are optional. Add one only when it materially improves navigation for that page. Prefer inline source references for short pages. +- Track the last successful documentation update in openwiki/.last-update.json. + +## Page format (observed output) + +wijzer pins these from OpenWiki's rendered `openwiki/` output rather than from +prompt.ts, and [`scripts/check-format.sh`](../scripts/check-format.sh) enforces +them as the interchange contract: + +- **Plain Markdown, no YAML frontmatter.** The first line of every page is a + level-1 `# ` heading in sentence case — never a `---` delimiter. +- Links between pages are relative Markdown (`./architecture/overview.md`). +- Once the wiki has **2+ pages**, `openwiki/quickstart.md` must carry both a + `## Start here` heading and a `## Documentation map` heading linking the + section pages. + +## Source map (observed output) + +A page **may** end with a source map; when present its shape is exact: + +- The heading is literally `## Source map` (capital S, lowercase m). +- One Markdown bullet per source file, the repo-relative path in backticks. - When git history is relevant, the **last** bullet is a git-evidence line: - `` Git evidence: commits `abc1234`, `def5678` `` — the word `Git evidence: - commits ` followed by comma-separated **7-character** short hashes, each in - backticks. This is the only place persistent commit hashes belong; do not - scatter hash lists elsewhere in the prose. + `` - Git evidence: commits `abc1234`, `def5678` `` — the literal + `Git evidence: commits ` then comma-separated **7-character** backticked + short hashes. This is the only place persistent commit hashes belong. Verbatim example (end of a section page): @@ -80,26 +73,71 @@ Verbatim example (end of a section page): - `src/agent/index.ts` - `src/agent/prompt.ts` -- `src/agent/utils.ts` - Git evidence: commits `ceded10`, `f89b05d`, `dfa73cc` ``` -During an **update** run, do not touch existing source maps, git-evidence -lists, or generic "things to watch" sections unless the recent source changes -made them materially wrong. - ## Size ceilings - **Init:** at most **8** documentation pages, unless the repository is clearly - tiny. This is a soft ceiling — a small repo should ship far fewer. -- **Update:** governed by the surgical-edit budget in - [disciplines.md](disciplines.md) — roughly, fewer than ~5 changed source files - means editing at most 1–2 pages. - -## Root instruction pointer - -After generating the wiki, the repo's `AGENTS.md` / `CLAUDE.md` gets a pointer -block directing coding agents to read `openwiki/quickstart.md` first. wijzer -writes this idempotently via [`scripts/inject-pointer.sh`](../scripts/inject-pointer.sh) -(marker-delimited, safe to re-run) rather than free-editing the file. See the -init and update skills for when it runs. + tiny — a soft ceiling; a small repo should ship far fewer. (This number is + prompt-derived; it also appears in the init mode block in + [disciplines.md](disciplines.md).) +- **Update:** governed by the surgical-edit diff budget in + [disciplines.md](disciplines.md) — fewer than ~5 changed source files means + editing at most 1–2 pages. + +--- + +## How this file is generated + +This file is **generated** from the vendored OpenWiki system prompt +([`vendor/openwiki/src/agent/prompt.ts`](../vendor/openwiki/src/agent/prompt.ts)) +by [`scripts/build-disciplines.mjs`](../scripts/build-disciplines.mjs). Do not +edit it by hand — edit the generator and re-run it. The build applies this +documented tool-vocabulary translation from OpenWiki's DeepAgents virtual +filesystem to Claude Code's real tools: + +- Drop the OpenWiki brand from the identity line (behavioral parity, not naming). + - matches "You are OpenWiki, an expert" +- Discovery/read/write/exec tools: ls,glob,grep,read_file,write_file,edit_file,execute -> Claude Code tools. + - matches "Prefer built-in filesystem discovery tools such as ls, glob, grep, read…" +- Virtual filesystem rooting + virtual paths -> repository-relative paths and Claude Code tools. + - matches "Filesystem tools are rooted at the target repository. Use virtual paths…" +- DeepAgents warns against host-absolute paths; Claude Code's file tools require absolute paths, so invert to the correct guidance. + - matches "Never pass host absolute paths like /Users/... to filesystem tools; tha…" +- `shell execute` -> `Bash`. + - matches "Shell execute commands run on the host. If you use execute, run command…" +- glob tool + shell -> Glob + Bash. + - matches "Do not call glob with **/* from the repository root." +- shell commands -> Bash commands. + - matches "Prefer shell commands like rg --files with excludes" +- grep/glob -> Grep/Glob. + - matches "Prefer grep/glob and short targeted reads over full-file reads when fil…" +- task tool -> Task tool with the wiki-scout subagent. + - matches "You may use the task tool to parallelize read-only research" +- Virtual plan path -> repository-relative; drop the virtual filesystem qualifier. + - matches "Use /openwiki/_plan.md when writing this temporary plan with filesystem…" +- No filesystem delete tool in Claude Code -> delete with Bash. + - matches "If there is no filesystem delete tool, use shell execute from the repos…" +- Virtual output paths -> repository-relative. + - matches "When writing required documentation with filesystem tools, use /openwik…" +- OpenWiki's CLI records state; wijzer's write-state.sh does (both mode blocks). + - matches "The CLI will record successful run metadata in openwiki/.last-update.js…" +- `shell execution` fallback -> `Bash` (update mode). + - matches "If shell execution is unavailable," +- Leading-slash AGENTS.md path. + - matches `\/AGENTS\.md` +- Leading-slash CLAUDE.md path. + - matches `\/CLAUDE\.md` +- Leading-slash openwiki path. + - matches `\/openwiki` + +Two sections need more than a vocabulary swap: + +- `OpenWiki CLI reference:` is **dropped** — its subject, the `openwiki` CLI flag + surface, is out of wijzer's parity scope, since wijzer's runtime is Claude Code + skills (`/wijzer:init`, `:update`, `:ask`), not a CLI. +- `Root agent instruction files:` is **adapted** — OpenWiki has the agent + hand-write an `## OpenWiki` pointer block; wijzer writes a marker-delimited + block deterministically with `scripts/inject-pointer.sh`, so the parity-relevant + rules are kept but the write mechanism and embedded block are replaced. diff --git a/scripts/build-disciplines.mjs b/scripts/build-disciplines.mjs new file mode 100644 index 0000000..2f13afc --- /dev/null +++ b/scripts/build-disciplines.mjs @@ -0,0 +1,598 @@ +#!/usr/bin/env node +// build-disciplines.mjs — DEV / CI ONLY. Never run by the plugin or by users. +// +// Derives references/disciplines.md and references/wiki-format.md from the real, +// vendored OpenWiki system prompt (vendor/openwiki/src/agent/prompt.ts) by: +// 1. extracting the exact template-literal text of createSystemPrompt() and +// createModeInstructions() (no TS import — plain text parsing, so this runs +// under stock `node` with zero deps), +// 2. substituting the two prompt constants (OPEN_WIKI_DIR, UPDATE_METADATA_PATH) +// read from vendor/openwiki/src/constants.ts, +// 3. routing the prompt's sections to disciplines.md vs wiki-format.md (or +// dropping the ones that are out of wijzer's parity scope), and +// 4. applying a DOCUMENTED tool-vocabulary translation from OpenWiki's +// DeepAgents virtual-filesystem vocabulary to Claude Code's real tools. +// +// The committed output is what the plugin ships, so users never run node. The +// generation is drift-locked by tests/build-disciplines.test.ts: committed files +// must equal a fresh regenerate, and OpenWiki's own drift into prompt.ts is +// caught by tests/vendor-openwiki.test.ts. When either fires, re-run this script +// and review the diff (see PARITY.md re-validation). +// +// Usage: +// node scripts/build-disciplines.mjs # write the generated files +// node scripts/build-disciplines.mjs --check # exit 1 if committed != generated + +import { readFileSync, writeFileSync } from "node:fs"; +import path from "node:path"; +import { fileURLToPath } from "node:url"; + +const HERE = path.dirname(fileURLToPath(import.meta.url)); +const REPO_ROOT = path.resolve(HERE, ".."); +const PROMPT_TS = path.join(REPO_ROOT, "vendor/openwiki/src/agent/prompt.ts"); +const CONSTANTS_TS = path.join(REPO_ROOT, "vendor/openwiki/src/constants.ts"); +const PROVENANCE = path.join(REPO_ROOT, "vendor/openwiki/PROVENANCE.md"); + +const OUT = { + disciplines: path.join(REPO_ROOT, "references/disciplines.md"), + wikiFormat: path.join(REPO_ROOT, "references/wiki-format.md"), +}; + +// --- Template-literal extraction ------------------------------------------ + +/** + * Read the raw source between an opening backtick at `openIdx` and its matching + * unescaped closing backtick, then "cook" it into the runtime string (undo the + * two escapes the vendored prompt actually uses: \` and \\). Returns the cooked + * text and the index just past the closing backtick. + */ +function readTemplate(src, openIdx) { + let raw = ""; + let i = openIdx + 1; + for (; i < src.length; i++) { + const ch = src[i]; + if (ch === "\\") { + raw += src[i + 1]; + i++; + continue; + } + if (ch === "`") break; + raw += ch; + } + if (i >= src.length) throw new Error("unterminated template literal in prompt.ts"); + return { text: raw, end: i + 1 }; +} + +/** The single template literal returned by `functionName` in prompt.ts. */ +function extractSingleTemplate(src, functionName) { + const at = src.indexOf(`export function ${functionName}`); + if (at < 0) throw new Error(`${functionName} not found in prompt.ts`); + const ret = src.indexOf("return `", at); + if (ret < 0) throw new Error(`no template return in ${functionName}`); + return readTemplate(src, ret + "return ".length).text; +} + +/** Every template literal returned inside `functionName`, in source order. */ +function extractAllTemplates(src, functionName) { + const at = src.indexOf(`export function ${functionName}`); + if (at < 0) throw new Error(`${functionName} not found in prompt.ts`); + // Bound the search to this function body (up to the next top-level export). + const nextFn = src.indexOf("\nexport function ", at + 1); + const end = nextFn < 0 ? src.length : nextFn; + const out = []; + let cursor = at; + for (;;) { + const ret = src.indexOf("return `", cursor); + if (ret < 0 || ret >= end) break; + const t = readTemplate(src, ret + "return ".length); + out.push(t.text); + cursor = t.end; + } + return out; +} + +// --- Constant substitution ------------------------------------------------- + +function readConstants(src) { + const dir = src.match(/OPEN_WIKI_DIR\s*=\s*"([^"]+)"/); + if (!dir) throw new Error("OPEN_WIKI_DIR not found in constants.ts"); + const OPEN_WIKI_DIR = dir[1]; + const meta = src.match(/UPDATE_METADATA_PATH\s*=\s*`([^`]+)`/); + if (!meta) throw new Error("UPDATE_METADATA_PATH not found in constants.ts"); + const UPDATE_METADATA_PATH = meta[1].replace("${OPEN_WIKI_DIR}", OPEN_WIKI_DIR); + return { OPEN_WIKI_DIR, UPDATE_METADATA_PATH }; +} + +/** + * Resolve every `${...}` interpolation the vendored templates use. Any unknown + * interpolation left over is a loud failure: OpenWiki added a dynamic piece this + * generator does not understand, so a human must extend it. + */ +function substitute(text, consts) { + let out = text + .replaceAll("${OPEN_WIKI_DIR}", consts.OPEN_WIKI_DIR) + .replaceAll("${UPDATE_METADATA_PATH}", consts.UPDATE_METADATA_PATH) + // createSystemPrompt splices the mode block in; we compose modes ourselves. + .replaceAll("${createModeInstructions(command)}", ""); + const stray = out.match(/\$\{[^}]*\}/); + if (stray) throw new Error(`unhandled interpolation in prompt.ts: ${stray[0]}`); + return out; +} + +// --- Tool-vocabulary translation ------------------------------------------ +// +// Ordered, documented replacements: OpenWiki's DeepAgents virtual-filesystem +// vocabulary -> Claude Code's real tools and repository-relative paths. Whole +// sentences/bullets are matched verbatim (robust: an upstream edit to one makes +// this generator fail loudly rather than pass DeepAgents vocab through). The +// residual-vocabulary guard below proves the table stays complete. + +const TRANSLATIONS = [ + { + note: "Drop the OpenWiki brand from the identity line (behavioral parity, not naming).", + from: "You are OpenWiki, an expert", + to: "You are an expert", + }, + { + note: "Discovery/read/write/exec tools: ls,glob,grep,read_file,write_file,edit_file,execute -> Claude Code tools.", + from: "Prefer built-in filesystem discovery tools such as ls, glob, grep, read_file, write_file, and edit_file for targeted reads. Use git through shell execute when it provides useful history.", + to: "Prefer built-in discovery tools such as `Glob`, `Grep`, and `Read` for targeted reads, and `Write` and `Edit` to author pages. Use git through `Bash` when it provides useful history.", + }, + { + note: "Virtual filesystem rooting + virtual paths -> repository-relative paths and Claude Code tools.", + from: "Filesystem tools are rooted at the target repository. Use virtual paths such as /README.md, /agent/..., /server/..., and /openwiki/quickstart.md with ls, read_file, write_file, edit_file, glob, and grep.", + to: "Tools operate on the target repository. Use repository-relative paths such as `README.md`, `agent/...`, `server/...`, and `openwiki/quickstart.md` with `Glob`, `Grep`, `Read`, `Write`, and `Edit`.", + }, + { + note: "DeepAgents warns against host-absolute paths; Claude Code's file tools require absolute paths, so invert to the correct guidance.", + from: "Never pass host absolute paths like /Users/... to filesystem tools; that creates nested paths inside the repo instead of touching the intended file.", + to: "Claude Code's file tools take absolute paths — resolve repository-relative paths against the repository root, keep every path inside that repository, and never write outside it.", + }, + { + note: "`shell execute` -> `Bash`.", + from: "Shell execute commands run on the host. If you use execute, run commands from the target repository directory and keep them inside that repository.", + to: "`Bash` commands run on the host. Run them from the target repository directory and keep them inside that repository.", + }, + { + note: "glob tool + shell -> Glob + Bash.", + from: "Do not call glob with **/* from the repository root.", + to: "Do not call `Glob` with `**/*` from the repository root.", + }, + { + note: "shell commands -> Bash commands.", + from: "Prefer shell commands like rg --files with excludes", + to: "Prefer `Bash` commands like `rg --files` with excludes", + }, + { + note: "grep/glob -> Grep/Glob.", + from: "Prefer grep/glob and short targeted reads over full-file reads when files are large.", + to: "Prefer `Grep`/`Glob` and short targeted reads over full-file reads when files are large.", + }, + { + note: "task tool -> Task tool with the wiki-scout subagent.", + from: "You may use the task tool to parallelize read-only research", + to: "You may use the `Task` tool with the `wiki-scout` subagent to parallelize read-only research", + }, + { + note: "Virtual plan path -> repository-relative; drop the virtual filesystem qualifier.", + from: "Use /openwiki/_plan.md when writing this temporary plan with filesystem tools.", + to: "Use `openwiki/_plan.md` when writing this temporary plan.", + }, + { + note: "No filesystem delete tool in Claude Code -> delete with Bash.", + from: "If there is no filesystem delete tool, use shell execute from the repository root, for example rm -f openwiki/_plan.md.", + to: "Claude Code has no delete tool, so remove it with `Bash`, for example `rm -f openwiki/_plan.md`.", + }, + { + note: "Virtual output paths -> repository-relative.", + from: "When writing required documentation with filesystem tools, use /openwiki/... paths, for example /openwiki/quickstart.md.", + to: "When writing documentation, use `openwiki/...` paths, for example `openwiki/quickstart.md`.", + }, + { + note: "OpenWiki's CLI records state; wijzer's write-state.sh does (both mode blocks).", + from: "The CLI will record successful run metadata in openwiki/.last-update.json after you finish.", + to: "wijzer records successful run metadata in openwiki/.last-update.json after you finish (via scripts/write-state.sh).", + }, + { + note: "`shell execution` fallback -> `Bash` (update mode).", + from: "If shell execution is unavailable,", + to: "If `Bash` is unavailable,", + }, + // Global leading-slash virtual paths -> repository-relative. Applied last so + // the verbatim rules above match the original text first. + { note: "Leading-slash AGENTS.md path.", from: /\/AGENTS\.md/g, to: "AGENTS.md" }, + { note: "Leading-slash CLAUDE.md path.", from: /\/CLAUDE\.md/g, to: "CLAUDE.md" }, + { note: "Leading-slash openwiki path.", from: /\/openwiki/g, to: "openwiki" }, +]; + +function translate(text) { + let out = text; + for (const { from, to } of TRANSLATIONS) { + out = out.replaceAll(from, to); + } + return out; +} + +// Tokens that must NOT survive translation into the doctrine body. This is a +// backstop, not a completeness proof: only substrings appearing in a verbatim or +// global rule are caught, so a *reworded* upstream sentence can pass through +// untranslated — the drift-lock (committed != regenerate) is what forces a human +// to review that case. "shell execut" covers both "execute" and "execution". +const RESIDUAL_VOCAB = [ + "read_file", + "write_file", + "edit_file", + "filesystem tool", + "shell execut", + "virtual path", + "task tool", + "/openwiki", + "/AGENTS.md", + "/CLAUDE.md", + "/Users/", +]; + +function assertNoResidualVocab(text, where) { + for (const token of RESIDUAL_VOCAB) { + if (text.includes(token)) { + throw new Error( + `untranslated DeepAgents vocabulary "${token}" survived in ${where} — extend TRANSLATIONS`, + ); + } + } +} + +// --- Section parsing + routing --------------------------------------------- + +// Header lines (exact, ending with ":") in the vendored system prompt, in order, +// each routed to a target. Missing/renamed/reordered headers fail loudly below. +const SECTIONS = [ + { header: "Run discipline:", target: "disciplines" }, + { header: "Subagent discipline:", target: "disciplines" }, + { header: "Planning discipline:", target: "disciplines" }, + { header: "Git discipline:", target: "disciplines" }, + { header: "Existing documentation discipline:", target: "disciplines" }, + { header: "Root agent instruction files:", target: "disciplines" }, + // OpenWiki's own CLI flag surface (openwiki --init/--update/...) is out of + // wijzer's parity scope: the runtime is Claude Code skills, not a CLI. + { header: "OpenWiki CLI reference:", target: "drop" }, + { header: "Security and privacy rules:", target: "disciplines" }, + { header: "Documentation goals:", target: "wikiFormat" }, + { header: "Section quality rules:", target: "wikiFormat" }, + { header: "Required documentation structure:", target: "wikiFormat" }, + { header: "Mode-specific behavior:", target: "disciplines" }, +]; + +// Sections whose *write mechanism* differs between OpenWiki and wijzer, so a +// vocabulary swap is not enough — the derived text would tell the agent to do +// something wijzer does with a script instead. These get a documented, +// distribution-method adaptation (body only; the `##
` stays). OpenWiki +// has the agent hand-write an `## OpenWiki` pointer block "every time"; wijzer +// writes a marker-delimited block deterministically with inject-pointer.sh, so +// deriving the verbatim block would both contradict the script and (on update) +// invite duplicate blocks. Keep the parity-relevant rules, replace the mechanism. +const ADAPTED_SECTIONS = { + "Root agent instruction files:": [ + "*Distribution-method adaptation: OpenWiki has the agent hand-write this", + "pointer block; wijzer writes it deterministically with", + "[`scripts/inject-pointer.sh`](../scripts/inject-pointer.sh). The parity intent —", + "a top-level, idempotent pointer into the wiki — is preserved; the exact", + "`## OpenWiki` block OpenWiki embeds here is replaced by the script's", + "marker-delimited block.*", + "", + "- Point coding agents at the wiki from the repository's **top-level**", + " `AGENTS.md` / `CLAUDE.md` — never nested `AGENTS.md`/`CLAUDE.md` files.", + "- Do not hand-write the block. Run `scripts/inject-pointer.sh`, which creates", + " or updates a marker-delimited block idempotently (safe to re-run) and", + " preserves the surrounding content.", + "- On update runs, re-run `scripts/inject-pointer.sh` so a repository that", + " gained an `AGENTS.md`/`CLAUDE.md` since init picks up the block; it no-ops", + " when the block is already present.", + "- Do not make formatting-only edits to these files.", + ].join("\n"), +}; + +/** + * Split the trimmed system prompt into { intro, sections{header->body} } and + * assert the header set is exactly SECTIONS (order included) and that no text is + * lost — join(all parts) must reproduce the input. + */ +function splitSections(systemPrompt) { + const lines = systemPrompt.split("\n"); + const headerSet = new Set(SECTIONS.map((s) => s.header)); + const parts = []; // { header|null, lines[] } + let current = { header: null, lines: [] }; + for (const line of lines) { + if (headerSet.has(line)) { + parts.push(current); + current = { header: line, lines: [] }; + } else { + current.lines.push(line); + } + } + parts.push(current); + + const found = parts.filter((p) => p.header).map((p) => p.header); + const expected = SECTIONS.map((s) => s.header); + if (JSON.stringify(found) !== JSON.stringify(expected)) { + throw new Error( + `prompt.ts section headers drifted.\n expected: ${expected.join(" | ")}\n found: ${found.join(" | ")}`, + ); + } + // Completeness: reassembling every part must reproduce the input verbatim. + const rejoined = parts + .map((p) => (p.header ? [p.header, ...p.lines] : p.lines).join("\n")) + .join("\n"); + if (rejoined !== systemPrompt) { + throw new Error("section split lost or duplicated text from prompt.ts"); + } + + const intro = parts[0].lines.join("\n").trim(); + const sections = {}; + for (const p of parts) { + if (p.header) sections[p.header] = p.lines.join("\n").trimEnd(); + } + + // Silent-drop guard. A known header set can't catch a NEW upstream section: it + // gets absorbed as body into the preceding section. If that section is one we + // do not render — the dropped CLI section, or the "Mode-specific behavior:" + // placeholder whose body is the interpolation we compose ourselves — the new + // doctrine vanishes and regeneration is byte-identical, so the drift-lock stays + // green while a discipline was lost. Fail loudly if an unrendered bucket holds a + // header-shaped line, and require the mode placeholder to be empty. + const unrendered = SECTIONS.filter( + (s) => s.target === "drop" || s.header === "Mode-specific behavior:", + ).map((s) => s.header); + const HEADER_SHAPE = /^[A-Z][A-Za-z][A-Za-z /-]*:$/; + for (const header of unrendered) { + const stray = (sections[header] ?? "").split("\n").find((l) => HEADER_SHAPE.test(l)); + if (stray) { + throw new Error( + `possible new upstream section "${stray}" absorbed into the unrendered "${header}" bucket — add it to SECTIONS with a target`, + ); + } + } + if ((sections["Mode-specific behavior:"] ?? "").trim() !== "") { + throw new Error( + "content follows 'Mode-specific behavior:' beyond the mode interpolation — new upstream section?", + ); + } + return { intro, sections }; +} + +/** `Run discipline:` -> `## Run discipline`. */ +function headerToMarkdown(header) { + return `## ${header.replace(/:$/, "")}`; +} + +// --- Assembly -------------------------------------------------------------- + +function generatedHeader(sha, regenNote) { + return ``; +} + +function translationAppendix() { + const rows = TRANSLATIONS.map(({ from, note }) => { + const shown = from instanceof RegExp ? `\`${from.source}\`` : `"${truncate(from)}"`; + return `- ${note}\n - matches ${shown}`; + }).join("\n"); + return `## How this file is generated + +This file is **generated** from the vendored OpenWiki system prompt +([\`vendor/openwiki/src/agent/prompt.ts\`](../vendor/openwiki/src/agent/prompt.ts)) +by [\`scripts/build-disciplines.mjs\`](../scripts/build-disciplines.mjs). Do not +edit it by hand — edit the generator and re-run it. The build applies this +documented tool-vocabulary translation from OpenWiki's DeepAgents virtual +filesystem to Claude Code's real tools: + +${rows} + +Two sections need more than a vocabulary swap: + +- \`OpenWiki CLI reference:\` is **dropped** — its subject, the \`openwiki\` CLI flag + surface, is out of wijzer's parity scope, since wijzer's runtime is Claude Code + skills (\`/wijzer:init\`, \`:update\`, \`:ask\`), not a CLI. +- \`Root agent instruction files:\` is **adapted** — OpenWiki has the agent + hand-write an \`## OpenWiki\` pointer block; wijzer writes a marker-delimited + block deterministically with \`scripts/inject-pointer.sh\`, so the parity-relevant + rules are kept but the write mechanism and embedded block are replaced.`; +} + +function truncate(s, n = 72) { + return s.length <= n ? s : `${s.slice(0, n - 1)}…`; +} + +function buildDisciplines(intro, sections, modeInstructions, sha) { + const order = SECTIONS.filter((s) => s.target === "disciplines").map((s) => s.header); + const body = []; + body.push(`# Disciplines`); + body.push(""); + body.push( + `The working rules the \`init\`, \`update\`, and \`ask\` skills obey — wijzer's` + + ` adaptation of OpenWiki's prompt disciplines to the Claude Code runtime.` + + ` Numbers that appear here (page counts, subagent counts, the diff budget)` + + ` are the parity contract — keep them.`, + ); + body.push(""); + body.push(intro); + for (const header of order) { + if (header === "Mode-specific behavior:") { + body.push(""); + body.push(headerToMarkdown(header)); + body.push(""); + body.push( + `The init and update skills share every discipline above. These are the` + + ` additional rules for each mode.`, + ); + body.push(""); + body.push(`### init`); + body.push(""); + body.push(modeInstructions.init); + body.push(""); + body.push(`### update`); + body.push(""); + body.push(modeInstructions.update); + continue; + } + body.push(""); + body.push(headerToMarkdown(header)); + body.push(""); + body.push(header in ADAPTED_SECTIONS ? ADAPTED_SECTIONS[header] : sections[header]); + } + const translated = translate(body.join("\n")); + assertNoResidualVocab(translated, "disciplines.md"); + const regen = "This is behavioral doctrine; the wiki output format lives in wiki-format.md."; + return `${generatedHeader(sha, regen)}\n\n${translated}\n\n---\n\n${translationAppendix()}\n`; +} + +// Format literals wijzer pins from OpenWiki's *generated output*, not from its +// prompt (prompt.ts calls source maps "optional" and never states their shape, +// the no-frontmatter rule, or the quickstart linking headings). These are the +// interchange contract `scripts/check-format.sh` actually enforces, so this file +// — the human companion to that gate — must document them. Kept verbatim here +// (a generator constant, so the whole file stays drift-locked) and labelled with +// their real provenance. +const OBSERVED_FORMAT = [ + "## Page format (observed output)", + "", + "wijzer pins these from OpenWiki's rendered `openwiki/` output rather than from", + "prompt.ts, and [`scripts/check-format.sh`](../scripts/check-format.sh) enforces", + "them as the interchange contract:", + "", + "- **Plain Markdown, no YAML frontmatter.** The first line of every page is a", + " level-1 `# ` heading in sentence case — never a `---` delimiter.", + "- Links between pages are relative Markdown (`./architecture/overview.md`).", + "- Once the wiki has **2+ pages**, `openwiki/quickstart.md` must carry both a", + " `## Start here` heading and a `## Documentation map` heading linking the", + " section pages.", + "", + "## Source map (observed output)", + "", + "A page **may** end with a source map; when present its shape is exact:", + "", + "- The heading is literally `## Source map` (capital S, lowercase m).", + "- One Markdown bullet per source file, the repo-relative path in backticks.", + "- When git history is relevant, the **last** bullet is a git-evidence line:", + " `` - Git evidence: commits `abc1234`, `def5678` `` — the literal", + " `Git evidence: commits ` then comma-separated **7-character** backticked", + " short hashes. This is the only place persistent commit hashes belong.", + "", + "Verbatim example (end of a section page):", + "", + "```markdown", + "## Source map", + "", + "- `src/agent/index.ts`", + "- `src/agent/prompt.ts`", + "- Git evidence: commits `ceded10`, `f89b05d`, `dfa73cc`", + "```", + "", + "## Size ceilings", + "", + "- **Init:** at most **8** documentation pages, unless the repository is clearly", + " tiny — a soft ceiling; a small repo should ship far fewer. (This number is", + " prompt-derived; it also appears in the init mode block in", + " [disciplines.md](disciplines.md).)", + "- **Update:** governed by the surgical-edit diff budget in", + " [disciplines.md](disciplines.md) — fewer than ~5 changed source files means", + " editing at most 1–2 pages.", +].join("\n"); + +function buildWikiFormat(sections, sha) { + const order = SECTIONS.filter((s) => s.target === "wikiFormat").map((s) => s.header); + const body = []; + body.push(`# Wiki format`); + body.push(""); + body.push( + `The exact shape of the generated wiki — a **parity contract** with OpenWiki` + + ` (see [PARITY.md](../PARITY.md)): a repository's \`openwiki/\` folder must be` + + ` interchangeable between wijzer and OpenWiki. The load-bearing literals` + + ` below (the \`openwiki/\` directory, the \`quickstart.md\` entrypoint, the` + + ` \`## Source map\` heading, page counts) must not be renamed or restyled.`, + ); + for (const header of order) { + body.push(""); + body.push(headerToMarkdown(header)); + body.push(""); + body.push(sections[header]); + } + const translated = translate(body.join("\n")); + assertNoResidualVocab(translated, "wiki-format.md"); + // The observed-format block is wijzer's own literals; it is not prompt-derived + // and deliberately not subject to the DeepAgents translation. + const withObserved = `${translated}\n\n${OBSERVED_FORMAT}`; + const regen = "Prompt-derived format rules plus wijzer's observed output literals; behavioral doctrine lives in disciplines.md."; + return `${generatedHeader(sha, regen)}\n\n${withObserved}\n\n---\n\n${translationAppendix()}\n`; +} + +// --- Entry ----------------------------------------------------------------- + +export function generate() { + const promptSrc = readFileSync(PROMPT_TS, "utf8"); + const constantsSrc = readFileSync(CONSTANTS_TS, "utf8"); + const provenance = readFileSync(PROVENANCE, "utf8"); + const sha = provenance.match(/\b([0-9a-f]{40})\b/)?.[1]; + if (!sha) throw new Error("no pinned SHA in vendor/openwiki/PROVENANCE.md"); + + const consts = readConstants(constantsSrc); + + const systemPrompt = substitute( + extractSingleTemplate(promptSrc, "createSystemPrompt"), + consts, + ).trim(); + + // createModeInstructions returns [chat, init, update] in source order. + const modeTemplates = extractAllTemplates(promptSrc, "createModeInstructions"); + if (modeTemplates.length !== 3) { + throw new Error( + `expected 3 mode templates (chat, init, update), found ${modeTemplates.length}`, + ); + } + const modeInstructions = { + init: translate(substitute(modeTemplates[1], consts).trim()), + update: translate(substitute(modeTemplates[2], consts).trim()), + }; + assertNoResidualVocab(modeInstructions.init, "mode:init"); + assertNoResidualVocab(modeInstructions.update, "mode:update"); + + const { intro, sections } = splitSections(systemPrompt); + + return { + disciplines: buildDisciplines(intro, sections, modeInstructions, sha), + wikiFormat: buildWikiFormat(sections, sha), + }; +} + +function main() { + const check = process.argv.includes("--check"); + const out = generate(); + const files = [ + [OUT.disciplines, out.disciplines, "references/disciplines.md"], + [OUT.wikiFormat, out.wikiFormat, "references/wiki-format.md"], + ]; + if (check) { + let drift = false; + for (const [abs, want, rel] of files) { + const have = readFileSync(abs, "utf8"); + if (have !== want) { + console.error(`drift: ${rel} is stale — run: node scripts/build-disciplines.mjs`); + drift = true; + } + } + if (drift) process.exit(1); + console.log("references/ are in sync with prompt.ts"); + return; + } + for (const [abs, want, rel] of files) { + writeFileSync(abs, want); + console.log(`wrote ${rel}`); + } +} + +if (import.meta.url === `file://${process.argv[1]}`) { + main(); +} diff --git a/tests/build-disciplines.test.ts b/tests/build-disciplines.test.ts new file mode 100644 index 0000000..a4d7542 --- /dev/null +++ b/tests/build-disciplines.test.ts @@ -0,0 +1,125 @@ +import { readFileSync } from "node:fs"; +import path from "node:path"; +import { fileURLToPath } from "node:url"; +import { describe, expect, test } from "vitest"; + +import { generate } from "../scripts/build-disciplines.mjs"; + +const HERE = path.dirname(fileURLToPath(import.meta.url)); +const REPO_ROOT = path.resolve(HERE, ".."); + +function read(rel: string): string { + return readFileSync(path.join(REPO_ROOT, rel), "utf8"); +} + +/** + * The derived doctrine only: between the generated header comment (which cites + * the legitimate `vendor/openwiki/...` source path) and the "## How this file is + * generated" appendix (which quotes the pre-translation vocabulary verbatim). + * Residual-vocab checks run against this slice, not the boilerplate around it. + */ +function body(markdown: string): string { + const afterHeader = markdown.slice(markdown.indexOf("-->") + 3); + const marker = "\n---\n\n## How this file is generated"; + const at = afterHeader.indexOf(marker); + return at < 0 ? afterHeader : afterHeader.slice(0, at); +} + +// DeepAgents virtual-filesystem vocabulary that translation must eliminate. The +// generator asserts this on the body it produces; here we re-assert it against +// the *committed* artifact so a hand-edit that reintroduces it also fails. +const RESIDUAL_VOCAB = [ + "read_file", + "write_file", + "edit_file", + "filesystem tool", + "shell execut", // "execute" and "execution" + "virtual path", + "task tool", + "/openwiki", + "/AGENTS.md", + "/CLAUDE.md", + "/Users/", +]; + +describe("generated disciplines + wiki-format (drift-locked to prompt.ts)", () => { + const out = generate(); + + test("committed references/disciplines.md equals a fresh regenerate", () => { + // If this fails, prompt.ts (or the generator) changed: rerun + // node scripts/build-disciplines.mjs + // and review the diff against PARITY.md re-validation. + expect(read("references/disciplines.md")).toBe(out.disciplines); + }); + + test("committed references/wiki-format.md equals a fresh regenerate", () => { + expect(read("references/wiki-format.md")).toBe(out.wikiFormat); + }); + + test("no DeepAgents virtual-fs vocabulary survives in either doc body", () => { + for (const rel of ["references/disciplines.md", "references/wiki-format.md"]) { + const text = body(read(rel)); + for (const token of RESIDUAL_VOCAB) { + expect(text, `"${token}" leaked into ${rel}`).not.toContain(token); + } + } + }); + + test("Claude Code tool vocabulary is present (translation actually fired)", () => { + const d = read("references/disciplines.md"); + // The virtual tools became real ones. + for (const tool of ["`Glob`", "`Grep`", "`Read`", "`Write`", "`Edit`", "`Bash`"]) { + expect(d).toContain(tool); + } + // The `task` tool became the Task tool + wiki-scout subagent. + expect(d).toContain("`Task` tool with the `wiki-scout` subagent"); + }); + + test("parity numbers survive derivation (page/subagent/diff budgets)", () => { + const d = read("references/disciplines.md"); + expect(d).toContain("at most 8 documentation pages"); // init page ceiling + expect(d).toContain("1-2 subagents"); // subagent default + expect(d).toContain("3-4 subagents"); // subagent upper bound + expect(d).toContain("fewer than about 5 source files"); // update diff budget + expect(d).toContain("update at most 1-2 wiki pages"); + }); + + test("both docs pin the same upstream SHA as PARITY.md", () => { + const pin = read("PARITY.md").match(/\b([0-9a-f]{40})\b/)?.[1]; + expect(pin).toBeDefined(); + for (const rel of ["references/disciplines.md", "references/wiki-format.md"]) { + expect(read(rel)).toContain(pin as string); + } + }); + + test("out-of-scope OpenWiki CLI section is dropped, not translated", () => { + // The CLI flag surface (openwiki --init/--update/...) is not wijzer's runtime; + // it must not appear as doctrine (only named in the drop rationale). + expect(body(read("references/disciplines.md"))).not.toContain("openwiki --init"); + expect(body(read("references/wiki-format.md"))).not.toContain("openwiki --init"); + }); +}); + +// state-schema.md is a DELIBERATE keep (hand-written, not generated): it documents +// wijzer-only serialization facts absent from OpenWiki's type (2-space indent, +// second-precision timestamp, atomic write, the claude-code model fallback). But +// its field SET is a parity contract, so lock it to the vendored UpdateMetadata. +describe("state-schema.md field set is locked to vendored UpdateMetadata", () => { + function updateMetadataFields(): string[] { + const types = read("vendor/openwiki/src/agent/types.ts"); + const block = types.match(/export type UpdateMetadata = \{([^}]*)\}/)?.[1]; + expect(block, "UpdateMetadata type not found in vendored types.ts").toBeDefined(); + return [...(block as string).matchAll(/^\s*([A-Za-z]+)\??:/gm)].map((m) => m[1]); + } + + test("documents exactly the UpdateMetadata fields (no more, no fewer)", () => { + const typeFields = updateMetadataFields(); + expect(typeFields.sort()).toEqual(["command", "gitHead", "model", "updatedAt"]); + // The field-reference table in state-schema.md: rows like `| \`updatedAt\` |`. + const schema = read("references/state-schema.md"); + const documented = [...schema.matchAll(/^\|\s*`(\w+)`\s*\|/gm)].map((m) => m[1]); + expect(documented.length, "no field table found in state-schema.md").toBeGreaterThan(0); + // Neither a dropped field (parity gap) nor a phantom field (fiction) may exist. + expect([...new Set(documented)].sort()).toEqual(typeFields.sort()); + }); +});