From e294cc473e04e6b534408205c3a267930b7dc283 Mon Sep 17 00:00:00 2001 From: John Carpenter Date: Fri, 28 Aug 2026 16:13:09 -0600 Subject: [PATCH] Prepare 0.2.0 Bumps `__version__` in kgmd/__init__.py, the single source hatchling reads. Verified by building the wheel: kgmd-0.2.0-py3-none-any.whl. The stamp on line 2 of all 16 pages under docs/ moves to `> Applies to kgmd 0.2.x`. tests/test_docs.py compares that stamp against kgmd.__version__, so crossing a minor boundary turns the suite red until every page is walked -- which is the point. Eleven prose references to 0.1.0 or 0.1.x are updated too. Four of them describe the broken `reset` command and the absent migration path; those statements are still true, so they now name 0.2.0 rather than implying a version where the behaviour was fixed. The upgrade section gains the note this release actually needs: the schema is unchanged and `PRAGMA user_version` is still 1, so a 0.1.x database opens as-is with no migration, but the first build after upgrading reconciles the graph against the corpus and removes documents whose files are gone. On a corpus with a long edit history that can be a large one-off removal, so it points at `kgmd build --dry-run` first. Adds CHANGELOG.md, linked from the README documentation table. The 0.2.0 entry records the known issues as well as the additions, including the reset defects and the extract --force mention-vector corruption tracked in #4, so the release does not read as though the graph now cleans itself up completely. Also corrects the development clone URL in README.md, which pointed at github.com/2lines/kgmd -- a repository that does not exist. README is the PyPI landing page, so that URL ships to every visitor. --- CHANGELOG.md | 88 +++++++++++++++++++++++++++++++ README.md | 3 +- docs/README.md | 2 +- docs/concepts.md | 2 +- docs/contributing/architecture.md | 2 +- docs/contributing/development.md | 4 +- docs/contributing/release.md | 6 +-- docs/examples/graph-export.md | 4 +- docs/examples/mcp-assistant.md | 4 +- docs/examples/personal-notes.md | 4 +- docs/guides/maintenance.md | 18 +++++-- docs/guides/mcp.md | 2 +- docs/guides/troubleshooting.md | 4 +- docs/install.md | 2 +- docs/quickstart.md | 2 +- docs/reference/cli.md | 2 +- docs/reference/configuration.md | 2 +- docs/reference/export.md | 2 +- kgmd/__init__.py | 2 +- 19 files changed, 126 insertions(+), 29 deletions(-) create mode 100644 CHANGELOG.md diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..e654c47 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,88 @@ +# Changelog + +All notable changes to kgmd are recorded here. Versions follow +[semantic versioning](https://semver.org/spec/v2.0.0.html), and the version number lives in +`kgmd/__init__.py`. + +## 0.2.0 — 2026-08-28 + +### Added + +- **`.kgmdignore`** — a gitignore-style file at the corpus root that excludes paths from indexing. + Supports `#` comments, blank lines, `*` (never crossing `/`), `?`, `**`, trailing-`/` directory + rules, leading-`/` anchoring, and `!` negation resolved last-match-wins. Because every indexed file + is chunked and each chunk is one model call, this is the cheapest control over build spend. See + [the configuration reference](docs/reference/configuration.md). +- **`kgmd build --dry-run`** — reports the files a build would index, the counts excluded by + `.kgmdignore` and by the dot-path rule, and how many indexed documents would be removed. Takes no + build lock, makes no provider call, and creates no database. `--json` gives the machine-readable + form and requires `--dry-run`. +- **`kgmd init` writes a starter `.kgmdignore`** at the corpus root, every line commented so a fresh + corpus indexes exactly what it did before. An existing file is never overwritten. + +### Changed + +- **Ingest now reconciles the graph against the corpus.** A document whose recorded path is no longer + in the resolved file set is removed, along with its chunks, its mentions, relations whose evidence + came from it, its rows in both vector tables, and any entity it leaves with no mention and no + relation. Deleted, renamed, and newly-excluded files are one state and are handled identically. + This applies to `kgmd build` and `kgmd extract`, which share the ingest path. + + **Upgrade note:** the first build after upgrading from 0.1.x may perform a large one-off removal on + a corpus with a long edit history. Run `kgmd build --dry-run` first to see the count. The schema is + unchanged — `PRAGMA user_version` is still `1` — so no migration is required. +- Ingest summaries in `kgmd build` and `kgmd extract` print a `Removed:` line when documents left the + graph, and stay silent when none did. + +### Fixed + +- **A deleted or renamed note is no longer stranded in the graph.** Previously ingest only inserted + and updated rows for files it found, so a note deleted from disk kept answering queries + indefinitely. This was documented as a limitation rather than fixed. +- **Stale vectors no longer survive a removal.** `chunks.id` and `entity_mentions.id` are reused by + SQLite after deletes, and `embed_new_chunks` skips any chunk that already has a vector row, so a + leftover vector would silently bind to an unrelated future chunk and search would answer from + deleted text. Both `sqlite-vec` tables are now cleared for removed ids. +- **Relations no longer survive with missing evidence.** `relations.evidence_chunk_id` is + `ON DELETE SET NULL`, so relations bound to a removed chunk previously persisted with no + provenance. They are now deleted. +- Two pre-existing `E501` violations in `kgmd/resolve.py` and `tests/test_resolve.py` that were + failing `ruff check` on `main`, plus pending `ruff format` drift in `kgmd/llm.py` and + `tests/test_docs.py`. +- `README.md` pointed the development clone at a repository URL that does not exist. + +### Safety + +- An ignore ruleset that resolves to an empty file set while the graph still holds documents aborts + the build **before any write**, rather than emptying the graph. A stray `*` is a mistake, not an + instruction. + +### Notes + +- No new runtime dependency. The pattern matcher is stdlib-only: `fnmatch`'s `*` crosses `/`, and + `PurePath.match` is right-anchored with single-segment `**` on Python 3.10–3.12, so neither can + express these semantics. `pathspec` was also rejected on merit — it reproduces git's refusal to + re-include a file inside an excluded directory, which kgmd deliberately allows. +- No new configuration key. `corpus.exclude` was rejected rather than shipping two mechanisms for one + job. +- **One deliberate divergence from `.gitignore`:** `archive/` followed by + `!archive/2024-decisions.md` re-includes that file. git refuses this. + +### Known issues + +- `kgmd reset` and `kgmd reset --hard` remain broken: both run `VACUUM` inside the transaction their + deletes opened, so the command exits 1 and changes nothing. Neither clears the vector tables, and + `kgmd extract --force` leaves stale mention vectors that can corrupt entity resolution. Tracked in + [#4](https://github.com/johncarpenter/kgmd/issues/4); deleting `.kgmd/graph.db` and rebuilding + remains the only complete reset. +- Entities stranded by an earlier `kgmd extract --force` are still not swept. The new sweep is scoped + to entities that a document removal itself orphaned. +- The three inert configuration keys (`extraction.max_entities_per_chunk`, + `extraction.max_relations_per_chunk`, `induction.include_attribute_summary`) still have no read + site. + +## 0.1.0 — 2026-05-09 + +Initial release. Extraction, entity resolution, and schema induction over a directory of markdown +files, stored in a single SQLite file with `sqlite-vec` vector indexes, queryable from the CLI and +from an MCP server. Includes the `docs/` set and the documentation coverage gate. diff --git a/README.md b/README.md index b3c8c9d..6a360a1 100644 --- a/README.md +++ b/README.md @@ -82,13 +82,14 @@ the exact registered tool names, and client configuration are in | Troubleshooting | [docs/guides/troubleshooting.md](docs/guides/troubleshooting.md) | | Worked examples | [docs/examples/personal-notes.md](docs/examples/personal-notes.md) | | Contributing | [docs/contributing/development.md](docs/contributing/development.md) | +| Release history | [CHANGELOG.md](CHANGELOG.md) | Index: [docs/README.md](docs/README.md). ## Development ```bash -git clone https://github.com/2lines/kgmd.git +git clone https://github.com/johncarpenter/kgmd.git cd kgmd make install # pip install -e ".[dev]" make test # pytest diff --git a/docs/README.md b/docs/README.md index 802f988..a3565fd 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,5 +1,5 @@ # kgmd Documentation -> Applies to kgmd 0.1.x +> Applies to kgmd 0.2.x Everything needed to install, use, operate, and contribute to kgmd. Start with [Install](./install.md) and the [Quickstart](./quickstart.md); come back here to look things up. diff --git a/docs/concepts.md b/docs/concepts.md index ecb4400..a6abd60 100644 --- a/docs/concepts.md +++ b/docs/concepts.md @@ -1,5 +1,5 @@ # Concepts -> Applies to kgmd 0.1.x +> Applies to kgmd 0.2.x For anyone about to run, tune, or debug a build. This page defines the six words the rest of the documentation uses without explanation — document, chunk, entity, mention, relation, induced schema — diff --git a/docs/contributing/architecture.md b/docs/contributing/architecture.md index dffb932..b8a040b 100644 --- a/docs/contributing/architecture.md +++ b/docs/contributing/architecture.md @@ -1,5 +1,5 @@ # Architecture -> Applies to kgmd 0.1.x +> Applies to kgmd 0.2.x For contributors who need to know where a change belongs before making it. This page maps every module in `kgmd/`, states the one-way dependency rule the package follows, and identifies the single diff --git a/docs/contributing/development.md b/docs/contributing/development.md index eb3e12d..97ca9b3 100644 --- a/docs/contributing/development.md +++ b/docs/contributing/development.md @@ -1,5 +1,5 @@ # Development -> Applies to kgmd 0.1.x +> Applies to kgmd 0.2.x For contributors changing kgmd's code or its documentation. Working through this page from a fresh clone gets you an installed development environment, a clean local check run that mirrors the @@ -125,7 +125,7 @@ The consequence is symmetrical: So a capability change and its documentation ship in the same commit. There is no follow-up window in which the docs are allowed to be wrong. -The version stamp on line 2 of every page (`> Applies to kgmd 0.1.x`) is checked against +The version stamp on line 2 of every page (`> Applies to kgmd 0.2.x`) is checked against `kgmd/__init__.py`. Bumping `__version__` across a minor boundary turns every page red until the stamps are updated — see [the release process](./release.md). diff --git a/docs/contributing/release.md b/docs/contributing/release.md index a526427..71abb4b 100644 --- a/docs/contributing/release.md +++ b/docs/contributing/release.md @@ -1,5 +1,5 @@ # Release -> Applies to kgmd 0.1.x +> Applies to kgmd 0.2.x For maintainers cutting a kgmd release. This page states where the version number lives, the exact sequence that publishes to PyPI, what is prohibited, and the manual verification that the automated @@ -10,7 +10,7 @@ suite structurally cannot perform. `__version__` in `kgmd/__init__.py` is the single source of truth: ```text -kgmd/__init__.py __version__ = "0.1.0" +kgmd/__init__.py __version__ = "0.2.0" ``` `pyproject.toml` declares `dynamic = ["version"]` and points hatchling at that file: @@ -37,7 +37,7 @@ gate immediately. A bump across a minor boundary therefore turns the suite red until every stamp matches. That is deliberate: it forces a maintainer to walk the whole documentation set at each release rather than shipping pages that silently describe an older version. A patch bump within the same minor - (`0.1.0` to `0.1.1`) leaves the stamps valid, because the stamp names the minor series. + (`0.2.0` to `0.2.1`) leaves the stamps valid, because the stamp names the minor series. 3. **Run the full local check sequence** from [the development page](./development.md): diff --git a/docs/examples/graph-export.md b/docs/examples/graph-export.md index e78d291..f1e162d 100644 --- a/docs/examples/graph-export.md +++ b/docs/examples/graph-export.md @@ -1,5 +1,5 @@ # Walkthrough: Load the Graph Into Another Tool -> Applies to kgmd 0.1.x +> Applies to kgmd 0.2.x For anyone who wants the graph outside kgmd — laid out visually in Gephi or yEd, queried with Cypher in Neo4j, or consumed as linked data. By the end you will have exported the same graph in two @@ -14,7 +14,7 @@ nothing else in the database is exported. ## Prerequisites -- kgmd 0.1.0 installed, and a corpus already built with `kgmd build` so that entities and relations +- kgmd 0.2.0 installed, and a corpus already built with `kgmd build` so that entities and relations exist. `kgmd stats` shows non-zero counts for both. Exporting an empty graph is not an error — it produces a well-formed but nodeless document (and, for `cypher`, an empty file). - The external tool you intend to load into: Gephi or yEd for GraphML, Neo4j (`cypher-shell` or the diff --git a/docs/examples/mcp-assistant.md b/docs/examples/mcp-assistant.md index ad99648..2eca484 100644 --- a/docs/examples/mcp-assistant.md +++ b/docs/examples/mcp-assistant.md @@ -1,5 +1,5 @@ # Walkthrough: Ask Questions Through an Assistant -> Applies to kgmd 0.1.x +> Applies to kgmd 0.2.x For anyone who already has a built kgmd corpus and wants to ask about it in natural language instead of composing CLI invocations. By the end an MCP-capable assistant will be wired to your graph, you @@ -14,7 +14,7 @@ last build produced and never writes to the graph. ## Prerequisites -- kgmd 0.1.0 installed and on `PATH`, and a corpus you have already built with `kgmd build`. The +- kgmd 0.2.0 installed and on `PATH`, and a corpus you have already built with `kgmd build`. The walkthrough in [personal-notes.md](./personal-notes.md) produces one; so does [quickstart.md](../quickstart.md). - An MCP-capable client that can launch a stdio server with a working directory — Claude Desktop, diff --git a/docs/examples/personal-notes.md b/docs/examples/personal-notes.md index eab27bb..a4071ca 100644 --- a/docs/examples/personal-notes.md +++ b/docs/examples/personal-notes.md @@ -1,5 +1,5 @@ # Walkthrough: A Searchable Graph Over Your Own Notes -> Applies to kgmd 0.1.x +> Applies to kgmd 0.2.x For anyone with a directory of markdown notes — a Zettelkasten, meeting notes, a research journal — who wants to query it instead of grepping it. By the end you will have a graph built from your own @@ -20,7 +20,7 @@ then answer questions about it from the command line. Three kinds of question ar ## Prerequisites -- kgmd 0.1.0 installed and on `PATH`. See [install.md](../install.md). +- kgmd 0.2.0 installed and on `PATH`. See [install.md](../install.md). - A provider credential for the LLM stages, exported in the environment before you build. kgmd never reads, prompts for, stores, or logs credentials — `litellm` picks the variable up implicitly from the model id you configured: diff --git a/docs/guides/maintenance.md b/docs/guides/maintenance.md index 08c359a..be84748 100644 --- a/docs/guides/maintenance.md +++ b/docs/guides/maintenance.md @@ -1,5 +1,5 @@ # Maintaining a kgmd corpus -> Applies to kgmd 0.1.x +> Applies to kgmd 0.2.x For anyone who already has a built graph and now has to keep it current. This page explains exactly what a re-run repeats and what it skips, how to force full reprocessing, which stages spend provider @@ -150,7 +150,7 @@ run log to project cost. Full key reference: ## Starting over -There are three levels, and in 0.1.0 only the third one works — see the warning below. +There are three levels, and in 0.2.0 only the third one works — see the warning below. | Action | Removes | Preserves | |---|---|---| @@ -161,7 +161,7 @@ There are three levels, and in 0.1.0 only the third one works — see the warnin `kgmd reset` prompts for confirmation; `--yes` skips the prompt. It requires the database to exist and takes the build lock while it works. -> **Both `reset` forms fail in 0.1.0.** The command issues its `DELETE` statements and then runs +> **Both `reset` forms fail in 0.2.0.** The command issues its `DELETE` statements and then runs > `conn.execute("VACUUM")` on the same connection, which SQLite refuses inside the open transaction > the deletes started. The command exits 1 with `Error: cannot VACUUM from within a transaction` and, > because the transaction is never committed, changes nothing. Until this is fixed, delete @@ -274,14 +274,22 @@ cost of re-spending the extraction budget. ## Upgrading kgmd -There is no migration path in 0.1.0, and this is a limitation rather than a guarantee of stability. +There is no migration path in 0.2.0, and this is a limitation rather than a guarantee of stability. `init_db` reads `PRAGMA user_version`; when it is `0` the full schema is created and the version set to `1`, and when it is anything else the function returns the open connection untouched. No code inspects the version further and no upgrade steps exist. Consequences for a version bump: -- Within 0.1.x, an existing `.kgmd/graph.db` opens as-is. +- Within 0.2.x, an existing `.kgmd/graph.db` opens as-is. +- **Upgrading from 0.1.x to 0.2.0 changes what a build does, not what the database looks like.** The + schema is untouched — `PRAGMA user_version` is still `1` — so an existing graph opens as-is with no + migration. But the first build after upgrading reconciles the graph against the corpus: every + document whose file has since been deleted, renamed, or newly excluded by `.kgmdignore` is removed, + together with its chunks, its mentions, relations whose evidence came from it, its vectors, and any + entity it leaves with no mention and no relation. On a corpus with a long edit history that can be + a large one-off removal. Run `kgmd build --dry-run` first: it reports how many indexed documents + would be removed, without touching anything. - If a future release changes the schema, an old database will be opened without being upgraded, and the fix will be to delete `.kgmd/graph.db` and rebuild — the same recovery as an embedding-model change. diff --git a/docs/guides/mcp.md b/docs/guides/mcp.md index 5cd798c..afa0445 100644 --- a/docs/guides/mcp.md +++ b/docs/guides/mcp.md @@ -1,5 +1,5 @@ # MCP Server -> Applies to kgmd 0.1.x +> Applies to kgmd 0.2.x For anyone who wants an MCP-capable assistant to read a built kgmd graph. By the end of this page you will have a client configured against a corpus, you will know the exact name and signature of all diff --git a/docs/guides/troubleshooting.md b/docs/guides/troubleshooting.md index fec14f0..5679d90 100644 --- a/docs/guides/troubleshooting.md +++ b/docs/guides/troubleshooting.md @@ -1,5 +1,5 @@ # Troubleshooting -> Applies to kgmd 0.1.x +> Applies to kgmd 0.2.x For anyone whose build, query, or MCP server just failed. Each entry below quotes the literal text kgmd emits, explains why it is emitted, and gives the fix. Errors are rendered as a single @@ -244,7 +244,7 @@ softer: an unknown name gives an empty result rather than an error. **Symptom**: `conn.execute("VACUUM")` -**Cause**: In 0.1.0 both `kgmd reset` and `kgmd reset --hard` issue their `DELETE` statements and then +**Cause**: In 0.2.0 both `kgmd reset` and `kgmd reset --hard` issue their `DELETE` statements and then run `VACUUM` on the same connection. SQLite refuses to vacuum inside the transaction those deletes opened, so the command exits 1 with `Error: cannot VACUUM from within a transaction`. Because the transaction is never committed, nothing is deleted — the reset is a no-op, not a partial wipe. diff --git a/docs/install.md b/docs/install.md index f043837..c4bdfd4 100644 --- a/docs/install.md +++ b/docs/install.md @@ -1,5 +1,5 @@ # Install -> Applies to kgmd 0.1.x +> Applies to kgmd 0.2.x For anyone putting kgmd on a machine for the first time. By the end you will have the `kgmd` command available, a verified interpreter that can load SQLite extensions, and — if you intend to diff --git a/docs/quickstart.md b/docs/quickstart.md index 56c8289..24736e3 100644 --- a/docs/quickstart.md +++ b/docs/quickstart.md @@ -1,5 +1,5 @@ # Quickstart -> Applies to kgmd 0.1.x +> Applies to kgmd 0.2.x For a reader who has kgmd installed and wants a working graph, not a tour. Follow this page in order and in about ten minutes you will have a queryable knowledge graph over a directory of markdown diff --git a/docs/reference/cli.md b/docs/reference/cli.md index 711d43e..d6ab13c 100644 --- a/docs/reference/cli.md +++ b/docs/reference/cli.md @@ -1,5 +1,5 @@ # CLI Reference -> Applies to kgmd 0.1.x +> Applies to kgmd 0.2.x This page is the complete reference for the `kgmd` command-line interface: every command, every parameter, every default, and what each command actually touches on disk. Read it when you need the diff --git a/docs/reference/configuration.md b/docs/reference/configuration.md index 4b92881..a80c1f5 100644 --- a/docs/reference/configuration.md +++ b/docs/reference/configuration.md @@ -1,5 +1,5 @@ # Configuration Reference -> Applies to kgmd 0.1.x +> Applies to kgmd 0.2.x For operators tuning a corpus: this page lists every configuration key kgmd reads, where the files live, how the two files are merged, and which pipeline stage consumes each setting. After reading it diff --git a/docs/reference/export.md b/docs/reference/export.md index aa1afbe..c670405 100644 --- a/docs/reference/export.md +++ b/docs/reference/export.md @@ -1,5 +1,5 @@ # Export Reference -> Applies to kgmd 0.1.x +> Applies to kgmd 0.2.x For anyone moving a built graph into another tool: this page describes the three formats `kgmd export` can emit, exactly what each one contains, and which downstream tool consumes it. After diff --git a/kgmd/__init__.py b/kgmd/__init__.py index 1119729..bd69d57 100644 --- a/kgmd/__init__.py +++ b/kgmd/__init__.py @@ -1,3 +1,3 @@ """kgmd — Knowledge graph from markdown files.""" -__version__ = "0.1.0" +__version__ = "0.2.0"