Add documentation set with drift-detecting coverage checks - #2
Conversation
The repository had no recorded engineering principles and no planning artifacts. This adds the Spec Kit scaffold plus a constitution derived from practices the codebase already follows (single SQLite artifact behind PRAGMA user_version, one deterministic LLM boundary, CLI/MCP parity over a single query layer, content-hash incrementality, an offline-deterministic test gate). Constitution is v1.1.0: the initial ratification plus one amendment extending "documentation as contract" from README.md alone to both README.md (orientation) and docs/ (authoritative depth), and requiring surface coverage to be mechanically verifiable in both directions. The amendment is a prerequisite for the docs/ work in the following commit, which would otherwise contradict the governance rule it is measured against. Also includes the spec, plan, research, data model, contracts, validation guide, and task list for the documentation feature under specs/001-project-documentation/.
The tool exposed 16 commands, 19 configuration keys, 7 MCP tools and 3 export formats; README.md documented 12 commands, 17 keys, and named 6 of the 7 MCP tools incorrectly (it advertised get_entity, list_entities, get_neighbors, find_path, list_relations, get_schema; the server registers those names with a _tool suffix). Anyone scripting against the documented MCP surface got unknown-tool errors, and the four commands needed to recover from a partial build (extract, resolve, induce, reset) were undiscoverable. Adds 16 pages under docs/ covering install, quickstart, concepts, full reference, MCP integration, maintenance, troubleshooting, three end-to-end walkthroughs, and contributor/release material. README.md becomes the orientation surface and PyPI landing page, linking into docs/. The load-bearing part is tests/test_docs.py: 22 offline checks that derive the real surface from the Click command tree, DEFAULT_CONFIG, an AST scan of mcp_server.py, and the --format choice list, then assert bidirectional set equality against the reference pages. Documenting something that does not exist fails the suite, which is the direction that would have caught the wrong tool names. Also verifies per-page version stamps against kgmd.__version__, internal link resolution, two-link reachability from the index, quoted error strings still present in kgmd/**/*.py, walkthrough structure, and the absence of credential-shaped strings. No new dependencies; no network or credential required, so the gate stays hermetic. Documented, not fixed (each verified against source, out of scope for a documentation change): - kgmd reset and reset --hard are no-ops in 0.1.0: VACUUM runs inside the transaction opened by the preceding DELETEs and raises. Deleting .kgmd/graph.db is the working reset. - kgmd build --config binds a value that is never read. - Deleted or renamed notes are never removed from the graph. - reset leaves vec_ rows behind while integer ids get reused. - The three inert configuration keys are marked as having no effect.
Two P1 findings, both confirmed against the tree: .specify/integrations/omp.manifest.json pins ten .omp/commands/speckit.*.md files by sha256, but those files were left untracked, so the committed Spec Kit scaffold was unusable on a fresh checkout. Commits them so the manifest and the tree agree. docs/reference/cli.md described kgmd build --config as "Config file path", implying it works. It does not: the value is bound and discarded, and build always loads the corpus .kgmd/config.yaml. The row now reads "Accepted but ignored" with a warning, matching what the configuration reference already said. The coverage test asserts that every option is documented, not that its description is true, so prose accuracy stays reviewer-owned by design.
Greptile SummaryThe PR replaces the project’s orientation documentation with a comprehensive installation, CLI, configuration, MCP, maintenance, and walkthrough set, backed by offline drift-detection tests.
Confidence Score: 3/5This PR should not merge until repository-provided Spec Kit hooks can no longer execute automatically without an explicit trust boundary. The new default OMP workflow reaches command definitions that load hook command identifiers from the checkout and require their automatic invocation, allowing untrusted repository configuration to direct execution with the agent session's authority. Files Needing Attention: .omp/commands/speckit.specify.md and the sibling .omp/commands/speckit.*.md command definitions
|
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Greptile flagged a P1 security issue on the vendored Spec Kit command definitions this PR added: their hook sections instructed the agent to invoke any mandatory hook automatically, taking the command id from .specify/extensions.yml. That file is repository-controlled, so a checkout the user did not author could name any command there and have it run with the agent session's authority. No allowlist, sandbox, or confirmation stood in the way. Patches all 20 hook sites across the 10 command definitions to fail closed: the hook block is still emitted, but the command is never invoked without explicit user approval that quotes the extension name and the exact command id. Only a user-level allowlist stored outside the checkout may pre-approve a hook; an allowlist inside the repository is itself untrusted input and is explicitly non-authoritative. The .omp/commands/*.md files are vendored and pinned by checksum, so .specify/integrations/omp.manifest.json is regenerated against the patched content and records the divergence. docs/contributing/development.md documents that re-running `specify init` restores the upstream text and drops the patch, which must then be re-applied. This repository defines no hooks and expects none: there is no .specify/extensions.yml.
The 3.10 CI leg failed with `ImportError: cannot import name 'NotRequired' from 'typing'` while importing tests/test_extract.py, test_resolve.py and test_induce.py. Those modules import kgmd.extract/resolve/induce, which reach litellm. Cause is dependency drift, not a code change. `litellm>=1.50` had no upper bound, so CI resolved 1.83+, and litellm/types/compression.py does `from typing import NotRequired` — added in Python 3.11 — while the package still declares requires-python >=3.10. Verified by scanning published wheels: 1.78.0 through 1.82.0 are clean, 1.83.14 through the current 1.98.0 all carry the 3.11-only import. main was green in May under an older resolution. Capped at <1.83, which resolves to 1.82.6. Verified on a Python 3.10.21 virtualenv: litellm imports and the full suite passes 72/72. Also re-verified on 3.12 after the downgrade, so nothing in kgmd depends on a litellm API newer than 1.82. Lift the cap once litellm either fixes the import or drops its 3.10 claim.
CI: all four legs green (
|
| Leg | Result |
|---|---|
| test (3.10) | success |
| test (3.11) | success |
| test (3.12) | success |
| test (3.13) | success |
1 participant
Summary
README.mddocumented 12 of 16 commands, 17 of 19 configuration keys, and named 6 of 7 MCP tools incorrectly. It advertisedget_entity,list_entities,get_neighbors,find_path,list_relations,get_schema; the server registers those with a_toolsuffix. Anyone scripting against the documented MCP surface got unknown-tool errors.extract,resolve,induce) or when you want to start over (reset) were undiscoverable.docs/set (16 pages) covering install, use, and maintain, plus three end-to-end walkthroughs.tests/test_docs.py: 22 offline checks that make documentation drift fail the build. Prose alone would have rotted the same way.Approach: the tool's surface is introspectable at runtime (Click command tree,
DEFAULT_CONFIG, the FastMCP registry, the--formatchoice list), so coverage is a computable property rather than a review checklist. The checks assert bidirectional set equality — documenting something that does not exist fails too, which is the direction that would have caught the wrong tool names.Changes
Documentation (16 pages, ~3.4k lines)
docs/README.mdindex;install.md(incl. the loadable-SQLite-extension prerequisite with symptom and pyenv remedy);quickstart.md(two corpus paths, incl. a checkout-free heredoc forpip installreaders);concepts.md(pipeline + the six vocabulary terms).docs/reference/:cli.md(all 16 commands, 49 parameters, global--debug, structured output for the 8--jsoncommands),configuration.md(all 19 keys, precedence with a worked merge, per-platform paths),export.md(3 formats).docs/guides/:mcp.md(7 real tool names, client config, the load-bearingcwd),maintenance.md(incrementality, spend, reset, locking, backup),troubleshooting.md(12 symptom/cause/fix entries).docs/examples/: three walkthroughs againsttests/fixtures/.docs/contributing/: development, architecture, release (incl. the manual pre-release checklist).README.mdbecomes the orientation surface and PyPI landing page.Verification (
tests/test_docs.py, 22 checks, 0.11s)kgmd.__version__; internal link resolution; two-link reachability from the index; quoted troubleshooting errors still present inkgmd/**/*.py; walkthrough structure; no credential-shaped strings.Governance
specs/001-project-documentation/(spec, plan, research, data model, contracts, validation guide, tasks) and the Spec Kit scaffold.Testing
tests/test_docs.pyruff check .clean,pytest -v72 passed (50 pre-existing + 22 new)Negative validation (10/10). A coverage test that never fails is decoration, so each check was proven to fail: adding an undocumented command, deleting a documented section, adding a phantom MCP heading, adding a config key, adding an option, bumping the version with stale stamps, rewording an error message, renaming a linked page, pasting a credential-shaped string, and burying a page three levels deep. Every scenario produced exactly the predicted failure and reverted clean.
Hermetic. The suite passes with
OPENROUTER_API_KEY/OPENAI_API_KEY/ANTHROPIC_API_KEYunset — no provider calls, no model downloads, no new dependencies.Not automated (by design). Examples that build a real graph need a credential and make nondeterministic paid calls, so they are a checklist in
docs/contributing/release.md, not a CI step. Onboarding/contributor timings need a human; no baseline exists yet and the release page says so.Additional Notes
No breaking changes. No runtime code touched — nothing under
kgmd/is modified. No migration steps. No dependencies added.Product defects found while documenting — documented, not fixed (each verified against source; fixing them is out of scope for a documentation change):
kgmd resetandkgmd reset --hardare no-ops in 0.1.0.VACUUMruns inside the transaction opened by the precedingDELETEs:Error: cannot VACUUM from within a transaction, exit 1, nothing reset. Reproduced directly. Docs give deleting.kgmd/graph.dbas the working reset. Recommend a separate fix PR — this is user-facing.kgmd build --configbinds a path that is never read (caught by Greptile pre-review; row now reads "Accepted but ignored").resetleavesvec_chunks/vec_entity_mentionsrows behind while integer ids get reused, so stale vectors can bind to fresh rows.extraction.max_entities_per_chunk,extraction.max_relations_per_chunk,induction.include_attribute_summary) are marked as having no effect rather than implied to work.Follow-up recommended: rename the MCP tools to drop the
_toolsuffix. This PR documents reality; renaming is a breaking change to a public surface and needs its own spec and a compatibility decision.Unverified claim, flagged in-page: the Windows global-config path
%LOCALAPPDATA%\kgmd\kgmd\config.yamlis derived from theplatformdirssource, not executed on Windows.configuration.mdmarks it unconfirmed and the release checklist carries a confirmation item.Reviewer note: the coverage tests assert that every surface item is documented, not that its description is true. Prose accuracy is reviewer-owned — Greptile's
--configcatch is exactly that gap in action.