Add .kgmdignore exclusions and remove files that leave the corpus

README.md

@@ -57,6 +57,10 @@ Full walkthrough: [docs/quickstart.md](docs/quickstart.md).
 All state lives in `.kgmd/graph.db`, a single SQLite file. Re-running `kgmd build` is incremental —
 unchanged files are skipped. See [docs/concepts.md](docs/concepts.md).
 
+A `.kgmdignore` file at the corpus root keeps material out of the graph, and therefore out of the
+paid extraction path; `kgmd build --dry-run` shows what would be indexed before you spend anything.
+Syntax and precedence: [docs/reference/configuration.md](docs/reference/configuration.md).
+
 ## MCP server
 
 `kgmd mcp` launches an MCP server over stdio exposing seven read-only tools over the graph. Setup,

docs/contributing/architecture.md

@@ -12,7 +12,7 @@ places that own reads, state, prompts, and data contracts.
 | `kgmd/cli.py` | Click entry point (`main`) and every subcommand. Resolves the corpus and database path, opens connections, calls stage and query functions, and renders results as `rich` tables or as JSON. Contains no graph logic. |
 | `kgmd/mcp_server.py` | MCP stdio server built on `FastMCP`. Registers the tool functions, resolves its connection and configuration from the client's working directory, and delegates each tool to `kgmd/query.py`. |
 | `kgmd/query.py` | The read layer: `search_chunks`, `list_entities`, `get_entity`, `list_relations`, `get_neighbors`, `find_path`, `get_current_schema`. Takes a connection, returns plain `dict`/`list[dict]`. Builds a `networkx.DiGraph` in memory for traversal and pathfinding. |
-| `kgmd/ingest.py` | Markdown discovery, sha256 content hashing, and chunking. `find_markdown_files` applies `corpus.include` and skips dotted paths; `chunk_markdown` splits by paragraph, heading, or fixed window; `ingest_documents` writes documents and chunks. |
+| `kgmd/ingest.py` | Markdown discovery, sha256 content hashing, and chunking. `scan_corpus_files` is the single source of the resolved file set, composing `find_markdown_files` for `corpus.include` scoping, the `.kgmdignore` pass, and the dot-path rule, which is applied last and dominates both; `chunk_markdown` splits by paragraph, heading, or fixed window; `ingest_documents` calls `prune_missing_documents` to drop the rows of files that have left the corpus, then writes documents and chunks; `dry_run_report` is the read-only payload behind `kgmd build --dry-run`. |
 | `kgmd/extract.py` | Extraction stage. Builds the extraction prompt with type and predicate vocabulary drawn from the existing graph, calls the LLM per chunk, and upserts entities, mentions, and relations under an `extraction_runs` row. |
 | `kgmd/resolve.py` | Entity resolution. Clusters mention embeddings by cosine similarity, optionally asks the LLM to verify each cluster, then merges duplicate entities onto a survivor and records a `resolution_runs` row. |
 | `kgmd/induce.py` | Schema induction. Summarises entity and relation statistics from the graph, asks the LLM for a typed schema, and stores the result as a new `schema_versions` row. |
@@ -21,6 +21,7 @@ places that own reads, state, prompts, and data contracts.
 | `kgmd/embed.py` | Embedding backends behind an `Embedder` protocol: `FastembedEmbedder` (local, the default) and `LitellmEmbedder` (provider API). `get_embedder` selects one from `embedding.backend`. `embed_new_chunks` and `embed_new_mentions` fill the vector tables incrementally. |
 | `kgmd/db.py` | Connection ownership. `get_connection` opens SQLite, sets `row_factory`, loads the `sqlite-vec` extension, and applies `journal_mode = WAL`, `foreign_keys = ON`, `synchronous = NORMAL`. `init_db` runs the DDL once and sets `PRAGMA user_version = 1`. Also `check_embedding_model` and the `build_lock` context manager. |
 | `kgmd/config.py` | `DEFAULT_CONFIG`, the platform-specific global config location via `platformdirs`, and `load_config`, which deep-merges built-in defaults, the global file, and the corpus `.kgmd/config.yaml`. |
+| `kgmd/ignore.py` | `.kgmdignore` parsing. `load_ignore_rules` reads the corpus-root file into ordered rules with every pattern compiled to an anchored regex at parse time, `is_ignored` answers one corpus-relative path under last-match-wins ordering, and `DEFAULT_IGNORE_TEMPLATE` is the all-comments starter file `kgmd init` writes. Stdlib only; imports nothing from `kgmd` and issues no SQL. |
 | `kgmd/schema.py` | `SCHEMA_SQL` (all tables, indexes, and the `current_schema` view), `vec_tables_sql(dim)` for the `vec0` virtual tables, `KV_DEFAULTS`, and the pydantic models used to validate LLM output. |
 | `kgmd/prompts/` | Bundled prompt text assets: `extract.txt`, `resolve.txt`, `induce.txt`. Data, not code. |
 
@@ -49,12 +50,16 @@ The rules that follow from this:
   job.
 - **Nothing imports `mcp_server.py`** except `cli.py`, from inside the `mcp` command body.
 - **Stage modules never import each other.** `extract.py` and `resolve.py` depend on `llm.py` and
-  `schema.py`; `induce.py` depends on `llm.py`; `ingest.py`, `export.py`, and `query.py` import
-  nothing from `kgmd` at all — they receive an open connection as their first argument.
+  `schema.py`; `induce.py` depends on `llm.py`; `ingest.py` imports `ignore.py` and nothing else;
+  `export.py` and `query.py` import nothing from `kgmd` at all — they receive an open connection as
+  their first argument.
 - **`db.py` is the only module that imports `schema.py`** for DDL purposes, and `schema.py` imports
   nothing internal.
 - **`config.py` is a leaf.** It is read by the surfaces and passed down as a plain `dict`; no stage
   module loads configuration for itself.
+- **`ignore.py` is a leaf below `ingest.py`.** It is stdlib-only and imports nothing from `kgmd`, so
+  the one-way direction extends as `ingest` -> `ignore` -> nothing. It parses text and answers
+  questions about paths; it never sees a connection and issues no SQL.
 
 `cli.py` imports `config`, `db`, `ingest`, and `query` at module level, and imports `embed`,
 `extract`, `resolve`, `induce`, `export`, and `mcp_server` inside the command bodies that need them.
@@ -97,7 +102,10 @@ All persistent state is one SQLite file: `.kgmd/graph.db`.
   `chunks`, `extraction_runs`, `entities`, `entity_mentions`, `relations`, `resolution_runs`,
   `schema_versions`, their indexes, and the `current_schema` view. `vec_tables_sql(dim)` defines the
   `sqlite-vec` virtual tables `vec_chunks` and `vec_entity_mentions`, dimensioned at creation time.
-  No other module issues `CREATE`, `ALTER`, or `DROP`.
+  No other module issues `CREATE`, `ALTER`, or `DROP`. That constrains call sites, not just
+  definitions: `ingest.prune_missing_documents` binds the ids it is deleting in batched `IN (...)`
+  clauses rather than staging them in a `TEMP TABLE`, because a temp table would be DDL outside
+  `kgmd/schema.py`.
 - **`kgmd/db.py` owns connections and their invariants.** Extension loading, pragmas, `PRAGMA
   user_version` as the migration marker, `KV_DEFAULTS` seeding, the embedding-model guard, and the
   `fcntl` exclusive lock at `.kgmd/build.lock` all live there. Stage modules receive a connection;

docs/examples/mcp-assistant.md

@@ -166,5 +166,7 @@ in a session pays the model load; subsequent calls in the same server process do
 - **One corpus per server entry.** The database path is derived from the process working directory,
   which the client fixes at launch. Serving several corpora means several registered entries.
 - **The graph is only as current as the last build.** `get_schema_tool` returns its "no schema"
-  string until an induction has run, and every tool reflects the state of the last `kgmd build`,
-  including entities extracted from notes you have since deleted.
+  string until an induction has run, and every tool reflects the state of the last `kgmd build` —
+  notes written or edited since then are invisible until you build again. That cuts both ways: a
+  note you deleted, renamed, or newly excluded with `.kgmdignore` before that build is gone from the
+  graph, so no tool will answer from it.

docs/examples/personal-notes.md

@@ -40,7 +40,8 @@ then answer questions about it from the command line. Three kinds of question ar
 
 Use your own notes directory. Any layout works: kgmd walks the tree, takes every `*.md`, and always
 skips path components beginning with a dot (so `.kgmd/`, `.git/`, and `.claude/` are never
-ingested).
+ingested). A `.kgmdignore` file at the root of the notes directory subtracts further paths from
+that walk; step 2 covers it.
 
 If you want a corpus that behaves the same on someone else's machine, use the seven fixture notes in
 a git checkout — they are small, densely cross-referential, and deliberately inconsistent about
@@ -71,8 +72,10 @@ kgmd init
 
 `kgmd init` creates `.kgmd/` next to your notes containing `config.yaml` (the full default
 configuration, written out key by key), an empty `graph.db`, and empty `logs/` and `prompts/`
-directories. It prints the corpus, database, and config paths. Run again in an initialized directory
-and it prints the existing config instead of overwriting anything.
+directories. It also writes a starter `.kgmdignore` — beside the notes, not inside `.kgmd/` — with
+every line commented out, so a fresh corpus indexes exactly what it would have without the file. It
+prints the corpus, database, and config paths. Run again in an initialized directory and it prints
+the existing config instead of overwriting anything, and leaves an existing `.kgmdignore` untouched.
 
 ### 2. Scope and chunk the corpus (optional)
 
@@ -101,6 +104,35 @@ chunking:
 
 Every key, its default, and its effect is in [configuration.md](../reference/configuration.md).
 
+`corpus.include` only decides which part of the tree is a candidate. To subtract from that
+candidate set, write a `.kgmdignore` at the corpus root — gitignore-style patterns, one per line,
+`#` for comments:
+
+```text
+archive/
+drafts/
+!archive/2024-decisions.md
+```
+
+A personal corpus is where this pays for itself: an `archive/` you never query and a `drafts/` full
+of half-finished sentences cost one LLM call per chunk on every build that first touches them.
+Patterns are read in order and the last match wins, so the negation above re-admits that one
+archived note — which git will not do for a file whose parent directory is excluded. The dot-path
+rule is applied last and dominates, so no pattern re-admits `.kgmd/` or `.git/`. Syntax, precedence,
+and the constructs that are not supported are in
+[configuration.md](../reference/configuration.md).
+
+Check the result before paying for a build:
+
+```bash
+kgmd build --dry-run
+```
+
+It prints the files that would be indexed, how many the ignore rules and the dot-path rule dropped,
+and how many already-indexed documents would be removed. It takes no build lock, makes no provider
+call, and creates no database. `--json` gives the same report machine-readably and requires
+`--dry-run`.
+
 ### 3. Build the graph
 
 ```bash
@@ -121,6 +153,15 @@ Re-run `kgmd build` after editing notes. Ingest compares a sha256 of the file co
 from the current content hash, so unchanged files cost nothing. File mtime is stored but never used
 to decide what to skip. `kgmd extract --force` re-extracts everything regardless.
 
+Removal happens in the same pass, before the insert and update loop. Ingest drops every indexed
+document whose path is no longer in the resolved file set — deleted, renamed, and newly excluded by
+`.kgmdignore` are one state, not three — together with its chunks, its mentions, the relations whose
+evidence came from those chunks, the vectors for both, and any entity the removal leaves with no
+mention and no relation. `kgmd build` and `kgmd extract` both do it and both print a `Removed:` line
+in the ingest summary when something was removed, and nothing extra when nothing was. If the
+resolved set is empty while the graph still holds documents, ingest refuses before writing anything
+rather than emptying the graph behind a pattern you did not mean.
+
 ### 4. Query the graph
 
 Run these from the corpus directory or any subdirectory — kgmd walks upward looking for `.kgmd/`.
@@ -268,11 +309,12 @@ canonical entity names — match the name `kgmd entities` shows, or find it firs
 - **Runs are not reproducible.** `llm.temperature` defaults to `0.0`, but identical input can still
   produce a different entity or relation count on a second build. Treat any count as a description
   of one run.
-- **Deleted notes are not removed from the graph.** Ingest only inserts and updates rows for files
-  it finds; there is no pass that deletes documents whose file has disappeared, so entities
-  extracted from a note you deleted survive in the graph. To drop them, clear the graph and rebuild
-  — `kgmd reset --hard` (which also clears documents and chunks) or delete `.kgmd/graph.db`
-  outright. See [maintenance.md](../guides/maintenance.md).
+- **Entities stranded by an earlier forced extraction are not swept.** Removing a note collects the
+  entities *that removal* leaves with no mention and no relation, and only those. An entity left
+  behind by an earlier `kgmd extract --force` is outside that scope: it stays in the graph and keeps
+  appearing in `kgmd entities`. Clearing the graph with `kgmd reset --hard` and rebuilding is the
+  way to drop it — see [maintenance.md](../guides/maintenance.md) for what reset does and does not
+  touch.
 - **Resolution only merges within one entity type.** Mentions clustered as `Person` never merge with
   mentions typed `Organization`, so a mistyped mention stays a separate entity. Raising
   `resolution.similarity_threshold` merges less; lowering it merges more, including things that are