Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
88 changes: 88 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
# Changelog

All notable changes to kgmd are recorded here. Versions follow
[semantic versioning](https://semver.org/spec/v2.0.0.html), and the version number lives in
`kgmd/__init__.py`.

## 0.2.0 — 2026-08-28

### Added

- **`.kgmdignore`** — a gitignore-style file at the corpus root that excludes paths from indexing.
Supports `#` comments, blank lines, `*` (never crossing `/`), `?`, `**`, trailing-`/` directory
rules, leading-`/` anchoring, and `!` negation resolved last-match-wins. Because every indexed file
is chunked and each chunk is one model call, this is the cheapest control over build spend. See
[the configuration reference](docs/reference/configuration.md).
- **`kgmd build --dry-run`** — reports the files a build would index, the counts excluded by
`.kgmdignore` and by the dot-path rule, and how many indexed documents would be removed. Takes no
build lock, makes no provider call, and creates no database. `--json` gives the machine-readable
form and requires `--dry-run`.
- **`kgmd init` writes a starter `.kgmdignore`** at the corpus root, every line commented so a fresh
corpus indexes exactly what it did before. An existing file is never overwritten.

### Changed

- **Ingest now reconciles the graph against the corpus.** A document whose recorded path is no longer
in the resolved file set is removed, along with its chunks, its mentions, relations whose evidence
came from it, its rows in both vector tables, and any entity it leaves with no mention and no
relation. Deleted, renamed, and newly-excluded files are one state and are handled identically.
This applies to `kgmd build` and `kgmd extract`, which share the ingest path.

**Upgrade note:** the first build after upgrading from 0.1.x may perform a large one-off removal on
a corpus with a long edit history. Run `kgmd build --dry-run` first to see the count. The schema is
unchanged — `PRAGMA user_version` is still `1` — so no migration is required.
- Ingest summaries in `kgmd build` and `kgmd extract` print a `Removed:` line when documents left the
graph, and stay silent when none did.

### Fixed

- **A deleted or renamed note is no longer stranded in the graph.** Previously ingest only inserted
and updated rows for files it found, so a note deleted from disk kept answering queries
indefinitely. This was documented as a limitation rather than fixed.
- **Stale vectors no longer survive a removal.** `chunks.id` and `entity_mentions.id` are reused by
SQLite after deletes, and `embed_new_chunks` skips any chunk that already has a vector row, so a
leftover vector would silently bind to an unrelated future chunk and search would answer from
deleted text. Both `sqlite-vec` tables are now cleared for removed ids.
- **Relations no longer survive with missing evidence.** `relations.evidence_chunk_id` is
`ON DELETE SET NULL`, so relations bound to a removed chunk previously persisted with no
provenance. They are now deleted.
- Two pre-existing `E501` violations in `kgmd/resolve.py` and `tests/test_resolve.py` that were
failing `ruff check` on `main`, plus pending `ruff format` drift in `kgmd/llm.py` and
`tests/test_docs.py`.
- `README.md` pointed the development clone at a repository URL that does not exist.

### Safety

- An ignore ruleset that resolves to an empty file set while the graph still holds documents aborts
the build **before any write**, rather than emptying the graph. A stray `*` is a mistake, not an
instruction.

### Notes

- No new runtime dependency. The pattern matcher is stdlib-only: `fnmatch`'s `*` crosses `/`, and
`PurePath.match` is right-anchored with single-segment `**` on Python 3.10–3.12, so neither can
express these semantics. `pathspec` was also rejected on merit — it reproduces git's refusal to
re-include a file inside an excluded directory, which kgmd deliberately allows.
- No new configuration key. `corpus.exclude` was rejected rather than shipping two mechanisms for one
job.
- **One deliberate divergence from `.gitignore`:** `archive/` followed by
`!archive/2024-decisions.md` re-includes that file. git refuses this.

### Known issues

- `kgmd reset` and `kgmd reset --hard` remain broken: both run `VACUUM` inside the transaction their
deletes opened, so the command exits 1 and changes nothing. Neither clears the vector tables, and
`kgmd extract --force` leaves stale mention vectors that can corrupt entity resolution. Tracked in
[#4](https://github.com/johncarpenter/kgmd/issues/4); deleting `.kgmd/graph.db` and rebuilding
remains the only complete reset.
- Entities stranded by an earlier `kgmd extract --force` are still not swept. The new sweep is scoped
to entities that a document removal itself orphaned.
- The three inert configuration keys (`extraction.max_entities_per_chunk`,
`extraction.max_relations_per_chunk`, `induction.include_attribute_summary`) still have no read
site.

## 0.1.0 — 2026-05-09

Initial release. Extraction, entity resolution, and schema induction over a directory of markdown
files, stored in a single SQLite file with `sqlite-vec` vector indexes, queryable from the CLI and
from an MCP server. Includes the `docs/` set and the documentation coverage gate.
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,13 +82,14 @@ the exact registered tool names, and client configuration are in
| Troubleshooting | [docs/guides/troubleshooting.md](docs/guides/troubleshooting.md) |
| Worked examples | [docs/examples/personal-notes.md](docs/examples/personal-notes.md) |
| Contributing | [docs/contributing/development.md](docs/contributing/development.md) |
| Release history | [CHANGELOG.md](CHANGELOG.md) |

Index: [docs/README.md](docs/README.md).

## Development

```bash
git clone https://github.com/2lines/kgmd.git
git clone https://github.com/johncarpenter/kgmd.git
cd kgmd
make install # pip install -e ".[dev]"
make test # pytest
Expand Down
2 changes: 1 addition & 1 deletion docs/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# kgmd Documentation
> Applies to kgmd 0.1.x
> Applies to kgmd 0.2.x

Everything needed to install, use, operate, and contribute to kgmd. Start with
[Install](./install.md) and the [Quickstart](./quickstart.md); come back here to look things up.
Expand Down
2 changes: 1 addition & 1 deletion docs/concepts.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Concepts
> Applies to kgmd 0.1.x
> Applies to kgmd 0.2.x

For anyone about to run, tune, or debug a build. This page defines the six words the rest of the
documentation uses without explanation — document, chunk, entity, mention, relation, induced schema —
Expand Down
2 changes: 1 addition & 1 deletion docs/contributing/architecture.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Architecture
> Applies to kgmd 0.1.x
> Applies to kgmd 0.2.x

For contributors who need to know where a change belongs before making it. This page maps every
module in `kgmd/`, states the one-way dependency rule the package follows, and identifies the single
Expand Down
4 changes: 2 additions & 2 deletions docs/contributing/development.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Development
> Applies to kgmd 0.1.x
> Applies to kgmd 0.2.x

For contributors changing kgmd's code or its documentation. Working through this page from a fresh
clone gets you an installed development environment, a clean local check run that mirrors the
Expand Down Expand Up @@ -125,7 +125,7 @@ The consequence is symmetrical:
So a capability change and its documentation ship in the same commit. There is no follow-up window
in which the docs are allowed to be wrong.

The version stamp on line 2 of every page (`> Applies to kgmd 0.1.x`) is checked against
The version stamp on line 2 of every page (`> Applies to kgmd 0.2.x`) is checked against
`kgmd/__init__.py`. Bumping `__version__` across a minor boundary turns every page red until the
stamps are updated — see [the release process](./release.md).

Expand Down
6 changes: 3 additions & 3 deletions docs/contributing/release.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Release
> Applies to kgmd 0.1.x
> Applies to kgmd 0.2.x

For maintainers cutting a kgmd release. This page states where the version number lives, the exact
sequence that publishes to PyPI, what is prohibited, and the manual verification that the automated
Expand All @@ -10,7 +10,7 @@ suite structurally cannot perform.
`__version__` in `kgmd/__init__.py` is the single source of truth:

```text
kgmd/__init__.py __version__ = "0.1.0"
kgmd/__init__.py __version__ = "0.2.0"
```

`pyproject.toml` declares `dynamic = ["version"]` and points hatchling at that file:
Expand All @@ -37,7 +37,7 @@ gate immediately.
A bump across a minor boundary therefore turns the suite red until every stamp matches. That is
deliberate: it forces a maintainer to walk the whole documentation set at each release rather than
shipping pages that silently describe an older version. A patch bump within the same minor
(`0.1.0` to `0.1.1`) leaves the stamps valid, because the stamp names the minor series.
(`0.2.0` to `0.2.1`) leaves the stamps valid, because the stamp names the minor series.

3. **Run the full local check sequence** from [the development page](./development.md):

Expand Down
4 changes: 2 additions & 2 deletions docs/examples/graph-export.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Walkthrough: Load the Graph Into Another Tool
> Applies to kgmd 0.1.x
> Applies to kgmd 0.2.x

For anyone who wants the graph outside kgmd — laid out visually in Gephi or yEd, queried with Cypher
in Neo4j, or consumed as linked data. By the end you will have exported the same graph in two
Expand All @@ -14,7 +14,7 @@ nothing else in the database is exported.

## Prerequisites

- kgmd 0.1.0 installed, and a corpus already built with `kgmd build` so that entities and relations
- kgmd 0.2.0 installed, and a corpus already built with `kgmd build` so that entities and relations
exist. `kgmd stats` shows non-zero counts for both. Exporting an empty graph is not an error — it
produces a well-formed but nodeless document (and, for `cypher`, an empty file).
- The external tool you intend to load into: Gephi or yEd for GraphML, Neo4j (`cypher-shell` or the
Expand Down
4 changes: 2 additions & 2 deletions docs/examples/mcp-assistant.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Walkthrough: Ask Questions Through an Assistant
> Applies to kgmd 0.1.x
> Applies to kgmd 0.2.x

For anyone who already has a built kgmd corpus and wants to ask about it in natural language instead
of composing CLI invocations. By the end an MCP-capable assistant will be wired to your graph, you
Expand All @@ -14,7 +14,7 @@ last build produced and never writes to the graph.

## Prerequisites

- kgmd 0.1.0 installed and on `PATH`, and a corpus you have already built with `kgmd build`. The
- kgmd 0.2.0 installed and on `PATH`, and a corpus you have already built with `kgmd build`. The
walkthrough in [personal-notes.md](./personal-notes.md) produces one; so does
[quickstart.md](../quickstart.md).
- An MCP-capable client that can launch a stdio server with a working directory — Claude Desktop,
Expand Down
4 changes: 2 additions & 2 deletions docs/examples/personal-notes.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Walkthrough: A Searchable Graph Over Your Own Notes
> Applies to kgmd 0.1.x
> Applies to kgmd 0.2.x

For anyone with a directory of markdown notes — a Zettelkasten, meeting notes, a research journal —
who wants to query it instead of grepping it. By the end you will have a graph built from your own
Expand All @@ -20,7 +20,7 @@ then answer questions about it from the command line. Three kinds of question ar

## Prerequisites

- kgmd 0.1.0 installed and on `PATH`. See [install.md](../install.md).
- kgmd 0.2.0 installed and on `PATH`. See [install.md](../install.md).
- A provider credential for the LLM stages, exported in the environment before you build. kgmd never
reads, prompts for, stores, or logs credentials — `litellm` picks the variable up implicitly from
the model id you configured:
Expand Down
18 changes: 13 additions & 5 deletions docs/guides/maintenance.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Maintaining a kgmd corpus
> Applies to kgmd 0.1.x
> Applies to kgmd 0.2.x

For anyone who already has a built graph and now has to keep it current. This page explains exactly
what a re-run repeats and what it skips, how to force full reprocessing, which stages spend provider
Expand Down Expand Up @@ -150,7 +150,7 @@ run log to project cost. Full key reference:

## Starting over

There are three levels, and in 0.1.0 only the third one works — see the warning below.
There are three levels, and in 0.2.0 only the third one works — see the warning below.

| Action | Removes | Preserves |
|---|---|---|
Expand All @@ -161,7 +161,7 @@ There are three levels, and in 0.1.0 only the third one works — see the warnin
`kgmd reset` prompts for confirmation; `--yes` skips the prompt. It requires the database to exist
and takes the build lock while it works.

> **Both `reset` forms fail in 0.1.0.** The command issues its `DELETE` statements and then runs
> **Both `reset` forms fail in 0.2.0.** The command issues its `DELETE` statements and then runs
> `conn.execute("VACUUM")` on the same connection, which SQLite refuses inside the open transaction
> the deletes started. The command exits 1 with `Error: cannot VACUUM from within a transaction` and,
> because the transaction is never committed, changes nothing. Until this is fixed, delete
Expand Down Expand Up @@ -274,14 +274,22 @@ cost of re-spending the extraction budget.

## Upgrading kgmd

There is no migration path in 0.1.0, and this is a limitation rather than a guarantee of stability.
There is no migration path in 0.2.0, and this is a limitation rather than a guarantee of stability.
`init_db` reads `PRAGMA user_version`; when it is `0` the full schema is created and the version set
to `1`, and when it is anything else the function returns the open connection untouched. No code
inspects the version further and no upgrade steps exist.

Consequences for a version bump:

- Within 0.1.x, an existing `.kgmd/graph.db` opens as-is.
- Within 0.2.x, an existing `.kgmd/graph.db` opens as-is.
- **Upgrading from 0.1.x to 0.2.0 changes what a build does, not what the database looks like.** The
schema is untouched — `PRAGMA user_version` is still `1` — so an existing graph opens as-is with no
migration. But the first build after upgrading reconciles the graph against the corpus: every
document whose file has since been deleted, renamed, or newly excluded by `.kgmdignore` is removed,
together with its chunks, its mentions, relations whose evidence came from it, its vectors, and any
entity it leaves with no mention and no relation. On a corpus with a long edit history that can be
a large one-off removal. Run `kgmd build --dry-run` first: it reports how many indexed documents
would be removed, without touching anything.
- If a future release changes the schema, an old database will be opened without being upgraded, and
the fix will be to delete `.kgmd/graph.db` and rebuild — the same recovery as an embedding-model
change.
Expand Down
2 changes: 1 addition & 1 deletion docs/guides/mcp.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# MCP Server
> Applies to kgmd 0.1.x
> Applies to kgmd 0.2.x

For anyone who wants an MCP-capable assistant to read a built kgmd graph. By the end of this page you
will have a client configured against a corpus, you will know the exact name and signature of all
Expand Down
4 changes: 2 additions & 2 deletions docs/guides/troubleshooting.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Troubleshooting
> Applies to kgmd 0.1.x
> Applies to kgmd 0.2.x

For anyone whose build, query, or MCP server just failed. Each entry below quotes the literal text
kgmd emits, explains why it is emitted, and gives the fix. Errors are rendered as a single
Expand Down Expand Up @@ -244,7 +244,7 @@ softer: an unknown name gives an empty result rather than an error.

**Symptom**: `conn.execute("VACUUM")`

**Cause**: In 0.1.0 both `kgmd reset` and `kgmd reset --hard` issue their `DELETE` statements and then
**Cause**: In 0.2.0 both `kgmd reset` and `kgmd reset --hard` issue their `DELETE` statements and then
run `VACUUM` on the same connection. SQLite refuses to vacuum inside the transaction those deletes
opened, so the command exits 1 with `Error: cannot VACUUM from within a transaction`. Because the
transaction is never committed, nothing is deleted — the reset is a no-op, not a partial wipe.
Expand Down
2 changes: 1 addition & 1 deletion docs/install.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Install
> Applies to kgmd 0.1.x
> Applies to kgmd 0.2.x

For anyone putting kgmd on a machine for the first time. By the end you will have the `kgmd`
command available, a verified interpreter that can load SQLite extensions, and — if you intend to
Expand Down
2 changes: 1 addition & 1 deletion docs/quickstart.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Quickstart
> Applies to kgmd 0.1.x
> Applies to kgmd 0.2.x

For a reader who has kgmd installed and wants a working graph, not a tour. Follow this page in order
and in about ten minutes you will have a queryable knowledge graph over a directory of markdown
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/cli.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# CLI Reference
> Applies to kgmd 0.1.x
> Applies to kgmd 0.2.x

This page is the complete reference for the `kgmd` command-line interface: every command, every
parameter, every default, and what each command actually touches on disk. Read it when you need the
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/configuration.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Configuration Reference
> Applies to kgmd 0.1.x
> Applies to kgmd 0.2.x

For operators tuning a corpus: this page lists every configuration key kgmd reads, where the files
live, how the two files are merged, and which pipeline stage consumes each setting. After reading it
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/export.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Export Reference
> Applies to kgmd 0.1.x
> Applies to kgmd 0.2.x

For anyone moving a built graph into another tool: this page describes the three formats
`kgmd export` can emit, exactly what each one contains, and which downstream tool consumes it. After
Expand Down
2 changes: 1 addition & 1 deletion kgmd/__init__.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
"""kgmd — Knowledge graph from markdown files."""

__version__ = "0.1.0"
__version__ = "0.2.0"